Uploading data into the data foundation

The Data Upload page enables you, using CSV files, to upload data directly into the platform data foundation from the user interface. This convenient feature offers great self-serve capabilities to less tech-savvy system administrators by removing the necessity to upload their data into the platform via API or SFTP upload.

In addition to simply populating the data foundation, this feature also provides a clear, centralized view of all upload actions in the system, which allows you to keep track of data updates in the platform.

This is an note/information message.

For more information about the data upload via API and SFTP, refer to Uploading data into the data foundation via API and Uploading data into the data foundation via SFTP.

Integration principles

From an operational philosophy standpoint, the data upload via the UI is governed by the same principles as the upload via API or SFTP:

The data upload is the first and most important step of any new system setup.
The data foundation is composed of 17 standard entities and 35 enums which can be populated with data.
Only 3 of those entities are required for the system to be operational, but all entities must be present, even if they are empty.
The data foundation is fully case-sensitive. The name of fields must be respected; otherwise, an upload error will occur.
For a successful data upload, a mapping must be performed between the data of your HR system and the entities and fields of the data foundation.

This is a tip message.

Here are some basic definitions for entities and enums that will help you understand the concept behind each of them:

An entity is essentially a dimension of data. If we take the Worker entity as an example, it contains all the information related to individual workers, such as their name, address, date of birth, marital situation, gender, etc.
An enum is a list of values available for a given field. Enums are used as a list of possible values for a given field in an entity. If we switch to the Demographics entity, there is a field called Religion in this entity. This field is populated using an enum called Religion as well. This enum is where you can enter the possible values for this field (Catholic, Muslim, Jewish, Hindu, etc.). There are two types of enums in the system:
- Standard enums that you can populate with your own values.
- ISO enums that come prepopulated with values. There are four of these enums: Country, Language, Time zone, and Currency.

Entities

The following list details the entities that must contain data for the system to be operational:

Worker: the worker entity represents an individual worker within the organization. It contains information specific to the worker, outside of their current employment status or job position, such as name, contact information, gender, etc.
Organization: the organization entity encapsulates the hierarchical arrangement of units, departments, or teams within an organization. It details the structure of the organization, showcasing how different parts interconnect and relate to each other.
Employment: the employment entity defines the specific employment relationship between a worker and the organization. It contains information related to the worker's employment history, start date, type of contract, etc.

The following list details the other entities of the data foundation. Keep in mind that these entities don't need to contain data, but they must be uploaded at least once in the system:

CompensationHistory*: records the financial remuneration history of workers. It is critical for understanding compensation trends, making informed pay decisions, and managing payroll and budget planning.

This is a warning message.

This entity should not be used for any new implementations. It is still available as of now for backward compatibility purposes and it will be removed in a future version.

It is replaced by the following entities: ContractedCompensation and PaidCompensation.

CompensationElement: encompasses the various components of a worker's total compensation package. It plays a critical role in benefit administration, compensation strategy, and overall employee satisfaction.
JobArchitecture: defines the job framework within the organization, categorizing roles based on function, level, and other job-related characteristics. This entity is key in creating a structured approach to job classification and career progression.
LegalEntity: encapsulates the hierarchical arrangement of legal entities within the organization. It details the corporate structure of the organization, showcasing how different legal entities interconnect and relate to each other.
CostCenter: provides hierarchical data on cost centers and serves as a link between HR and Finance, crucial for budgeting and modeling processes. As a critical component in financial management, cost centers enable precise tracking and allocation of expenses, aiding in detailed and effective budgetary control and forecasting. Furthermore, within the application, the cost center can be utilized to define authorization levels and determine the population for specific financial or HR reports and analysis.
Absence: tracks leave and absence records of workers, offering insights into the frequency and reasons for worker time-off. This entity is essential for managing workforce ability and understanding patterns in employee absence.
Demographics: provides a snapshot of the diverse characteristics of the workforce. It offers insights into the cultural, ethnic, and social background of workers, aiding in understanding the diversity and inclusivity within the organization.
PayScale: represents the salary structure associated with various roles within the organization. It is instrumental in establishing equitable and competitive compensation practices, ensuring alignment with industry standards and internal equity.
IndividualPayRange: entity to be used if Position is not used. In this case, a pay range per position can be provided.
PerformanceRatingScales: specifies rating IDs, types, scales, and reporting order, providing a structured framework for accurately reflecting various performance measurement approaches.
PerformanceReviewRating: reflects the evaluation and performance ratings of a worker. It includes the performance rating, type of review, and the period of the rating, providing a comprehensive view of the worker's performance over time.
GoalPlan: lists the goal plans within which the goal types used for performance management are going to be distributed.
GoalAchievement*: used to upload historical goal achievement data for performance tracking purposes.

This is a warning message.

This entity should not be used for any new implementations. It is still available as of now for backward compatibility purposes and it will be removed in a future version.

It is replaced by the following entity: GoalAchievementNew.

GoalPlanTypeAssignment*: links goal types to goal plans and defines the weight assigned to each goal type.

This is a warning message.

This entity should not be used for any new implementations. It is still available as of now for backward compatibility purposes and it will be removed in a future version.

It is replaced by the following entity: GoalAchievementNew.

Goal: lists goals from any outside performance management solution.
GoalPlanGoalAssignment: links goals to goal plans.
GoalAchievementNew: tracks individual worker goal assignments and performance results within goal plans.
ContractedCompensation: defines the contractual agreement between the worker and the organization regarding the compensation. It includes data such as the agreed upon amount and the currency of the compensation.
PaidCompensation: reflects the actual compensation periods between the worker and the company through data such as the payment date and the relevant compensation period.

This is a tip message.

For a full description of the data foundation entities, including a full list of the fields included in each one, their data type, and whether or not they are mandatory, refer to Data foundation entities.

Custom entities

In addition to the standard entities listed above, the data foundation includes custom entities, which are customer-defined datasets with their own fields and structure. Custom entities extend the platform's data model with data that does not exist as a standard entity, such as internal mapping tables, organization-specific reference datasets, or worker-related enrichment records. Like standard entities, custom entities are populated with data via CSV upload or API ingestion, but only once they have been published and set to Active. For more information, refer to Working with custom entities.

Enums

The following list details the standard enums available in the data foundation:

EmploymentStatus: referenced in the Employment entity.
CompensationElementFrequency: referenced in the CompensationElement entity.
CompensationElementType: referenced in the CompensationElement entity.
PerformanceRatingType: referenced in the PerformanceReviewRating entity.
PerformanceReviewType: referenced in the PerformanceReviewRating entity.
Proficiency: referenced in the Worker entity.
ContractType: referenced in the Employment entity.
MaritalStatus: referenced in the Demographics entity.
Religion: referenced in the Demographics entity.
PoliticalAffiliation: referenced in the Demographics entity.
EducationLevel: referenced in the Demographics entity.
Ethnicity: referenced in the Demographics entity.
Nationality: referenced in the Demographics entity.
PayScaleType: referenced in the PayScale entity.
PayScaleArea: referenced in the PayScale entity.
PayScaleGroup: referenced in the PayScale entity.
PayScaleLevel: referenced in the PayScale entity.
PayScaleLocalGrade: referenced in the PayScale entity.
PayScaleGlobalGrade: referenced in the PayScale entity.
PayGrade: referenced in the PayScale entity.
JobFamily: referenced in the JobArchitecture entity.
JobFunction: referenced in the JobArchitecture entity.
JobCategory: referenced in the JobArchitecture entity.
JobLevel: referenced in the JobArchitecture entity.
AbsenceType: referenced in the Absence entity.
PerformanceRatingScale: referenced in the PerformanceRatingScales entity.
GoalPlanGroup: referenced in the PerformanceRatingScales entity.
IndividualPayRangeLocalGrade: referenced in the IndividualPayRange entity.
IndividualPayRangeGlobalGrade: referenced in the IndividualPayRange entity.
GoalType: referenced in the GoalPlanGoalAssignment entity.
OrganizationType: referenced in the Organization entity.

The following list details the ISO enums available in the data foundation:

Language: referenced in the Worker entity.
Currency: referenced in the Employment, CompensationHistory*, PayScale, CostCenter, IndividualPayRange, ContractedCompensation, and PaidCompensation entities.
Timezone: referenced in the Worker entity.
Country: referenced in the Worker, Employment, Organization, and CompensationElement entities.
Gender: referenced in the Demographics entity.
GoalUnit: referenced in the Goal entity.
AmountSource: referenced in the ContractedCompensation entity.

This is a tip message.

For a full overview of the data foundation enums, including all the values available in the ISO enums, refer to Data foundation enums.

Data types

The following list details the data types that you will encounter when working with the data foundation entities and enums:

boolean: form of data with only two possible values (usually "true" and "false").
string: sequence of characters and the most commonly used data type to store text.
enum: a data type that enables predefined constants for a variable to be a set. Two types of enums are used in the PaySuite data foundation:
- ISO enums: enumerations predetermined by international standards that cannot be modified by the customer. There are four of them: Language, Country, Currency, and Time zone.
- Standard enums: contain no values by default. To be able to use these enums, you must upload data to use them in the corresponding entity fields. For more information, refer to Data foundation enums.
date: date value in the format YYYY-MM-DD.
number: numerical value, with or without a decimal.
decimal: numerical, decimal value.
integer: whole number that stores numbers without decimal points.

This is a tip message.

The Data Upload page is available under Workbench > Data Management > Data Upload. Access to Workbench depends on your role within the platform.

This is an attention message.

Before you attempt any data upload, beqom strongly recommends that you go through the following knowledge base articles:

Upload modes

There are two upload modes:

Full data upload: Upload the complete effective-dated history for each worker included in the file. Any previously uploaded worker records missing from the file are treated as soft-deleted. Use this if you want the new upload to fully replace what is currently in the platform.
Incremental data upload: Upload only what has changed. New effective-dated records are added, and existing records are updated when field values change. Records not included remain unchanged. For ranged entities, you can now close old windows and add new ones without resending the entire history. Use this for ongoing incremental feeds.

To select your upload behavior, click the Change upload behavior button in the upper-left corner of the Data Upload page. In the Data ingestion settings pop-up, configure the following:

Upload behavior: Select Full data upload or Incremental data upload for both SFTP and UI data uploads. See above for details.
CSV separator: Set the tenant-level default CSV separator for SFTP ingestion and manual CSV uploads when no per-upload override is selected. Choose , (comma) or ; (semicolon). Click Save to save your changes.

This proves useful for tenants working with non-English versions of Excel (such as French or German), where Excel commonly saves CSV files with semicolons (;) instead of commas (,), which could otherwise cause failed uploads or incorrectly parsed files.

For backward compatibility, the platform default separator remains comma (,). Existing tenants without an explicit configuration continue to use comma-based parsing automatically.

Importantly, the CSV separator setting can be overridden per individual upload in the File upload dialog, as described in Uploading data below.

This is an note/information message.

The separator is resolved in the following order:

Separator explicitly specified in the manual upload
Tenant-level configured separator
Platform default separator (comma)

Downloading entity and enum templates

Before uploading your data, you must download a template to ensure your data is structured correctly. Templates are available for all data foundation entities and enums.

To download a template, proceed as follows:

Click the Download template button located in the upper-right corner of the page.
A pop-up window with all available templates will appear, as illustrated in the picture below:

Data Management templates

Alternatively, you can also click the Upload CSV button located in the upper-right corner of the page. In the File upload window, click Download the template.

Click the button next to the template you chose. The CSV file is downloaded locally on your device.
Fill in the template with your data to prepare it for upload.

Templates are also available for any published custom entities in your system. The same download and upload process applies. For more information, refer to Working with custom entities.

Uploading data

To upload your data, proceed as follows:

Click the Upload CSV button located in the upper-right corner of the page. A pop-up window will appear, as shown in the following figure:

File upload window

Under Choose a CSV separator, select either , (comma) or ; (semicolon). This overrides the tenant-level default for this specific upload. If no selection is made, the tenant-level configured separator will be used (comma by default). This step is optional.
Click Upload to choose a completed file from your local device or drag and drop your files.
Click Start upload to begin the uploading process.

This is an note/information message.

You can upload up to 100 files at once, with a combined maximum size of 10MB.

Refresh and email notification

Besides downloading templates and uploading files, you can perform other actions from the upper-right corner of the Data Upload page:

Click Refresh to manually refresh the upload list and see the latest status.
Click the three-dot icon and select Email notification. In the Upload notification pop-up, add the workers you want to notify by email when upload requests fail. You can type a worker's name into the pop-up's search bar to find the relevant person. To confirm, click Apply.

Viewing your uploaded files

Upload activity overview

The top of the Data Upload page displays a row of summary tiles giving you an at-a-glance status of all upload activity in your system:

Upload behavior: shows the currently active upload mode (Full data upload or Incremental data upload).
Total uploads: total number of upload records in the system.
Success: number of uploads that completed without errors.
Failed: number of uploads that encountered an unrecoverable error.
In sync: number of uploads whose data is consistent with the current state of the destination system.
Queued: number of uploads waiting to be processed.

Below the tiles, a paginated table displays all historical and ongoing uploads, as illustrated below:

Data Upload table

For each entry, you can view:

Data type name: the name of the entity or enum.
Behavior: the upload mode used: Full data upload or Incremental data upload.
Status: the current state of the upload:
- Queued: The upload is waiting to be processed.
- In progress: The upload is currently underway.
- Success: The data has been successfully uploaded and processed without any issues.
- In sync: The uploaded data is consistent with the destination system’s current state.
- Failed: The upload encountered an unrecoverable error and could not be completed.
- Stale timeout: The upload did not complete within the expected timeframe and was automatically stopped.

Source: indicates whether the upload was initiated via the UI (CSV) or programmatically via the API.
Start time and Completion time: timestamps for when the upload began and finished.

Search and filter

Use the Search bar above the table to search for uploads by data type name.

Click Filters to narrow results by any combination of the following attributes:

• Status of the upload

• Source of the upload (CSV or API)

• Start time

To use your filters, click Apply in the Select filters pop-up. To remove them all, click Clear filters.

Detailed upload view

To view upload details, find the upload in the table and click its name. An Upload log drawer will slide from the right side of the screen with the following information:

Data type name
Initiator
Start time
Completion time
Current status
Upload source
Behavior
ID

If the upload you are inspecting is not in the In progress status, the Download log file button will be available in the bottom-right corner of the drawer.

If the file was uploaded via CSV, the Download CSV file button will also be available in the bottom-right corner of the drawer.

The following picture presents an example of the Upload log drawer:

Upload log drawer