Uploading data into the data foundation
The Data Upload page enables you, using CSV files, to data directly into the platform data foundation from the user interface. This convenient feature offers great self-serve capabilities to less tech-savvy system administrators by removing the necessity to upload their data into the platform via API or SFTP upload.
In addition to simply populating the data foundation, this feature also offers a clear, centralized view of all upload actions in the system, which enables you to keep track of data updates in the platform.
This is an note/information message.
For more information about the data upload via API and SFTP, refer to Uploading data into the data foundation via API and Uploading data into the data foundation via SFTP .
Integration principles
From an operational philosophy standpoint, the data upload via the UI is governed by the same principles as the upload via API or SFTP:
The data upload is the first and most important step of any new system setup.
The data foundation is composed of 17 standard entities and 35 enums which can be populated with data.
Only 3 of those entities are required for the system to be operational, but all entities must be present, even if they are empty.
The data foundation is fully case sensitive. The name of fields must be respected otherwise, an upload error will occur.
For a successful data upload, a mapping must be performed between the data of your HR system and the entities and fields of the data foundation.
This is a tip message.
Here are some basic definitions for entities and enums that will help you understand the concept behind each of them:
An entity is essentially a dimension of data. If we take the Worker entity as an example, it contains all the information related to individual workers, such as their name, address, date of birth, marital situation, gender, etc.
-
An enum is a list of values available for a given field. Enums are used as a list of possible values for a given field in an entity. If we switch to the Demographics entity, there is in this entity a field called Religion. This field is populated using an enum called Religion as well. In this enum is where you can enter the possible values for this field (Catholic, Muslim, Jewish, Hindu, etc.). There are two types of enums in the system:
Standard enums that you can populate with your own values
ISO enums that come prepopulated with values. There are 4 of these enums: Country, Language, Time zone and Currency.
Entities
The following list details the entities that must contain data for the system to be operational:
Worker: the worker entity represents an individual worker within the organization. It contains information specific to the worker, outside of their current employment status or job position, such as name, contact information, gender, etc.
Organization: the organization entity encapsulates the hierarchical arrangement of units, departments, or teams within an organization. It details the structure of the organization, showcasing how different parts interconnect and relate to each other.
Employment: the employment entity represents the specific employment relationship between a worker and the organization. It contains information related to the worker's employment history, start date, type of contract, etc.
The following list details the other entities of the data foundation. Keep in mind that these entities don't need to contain data, but they must be uploaded at least once in the system:
CompensationHistory*: records the financial remuneration history of workers. It is critical for understanding compensation trends, making informed pay decisions, and managing payroll and budget planning.
This is a warning message.
This entity should not be used for any new implementations. It is still available as of now for backward compatibility purposes and it will be removed in a future version.
It is replaced by the following entities: ContractedCompensation and PaidCompensation.
CompensationElement: encompasses the various components of an worker's total compensation package. It plays a critical role in benefits administration, compensation strategy, and overall employee satisfaction.
JobArchitecture: defines the job framework within the organization, categorizing roles based on function, level, and other job-related characteristics. This entity is key in creating a structured approach to job classification and career progression.
LegalEntity: encapsulates the hierarchical arrangement of legal entities within organization. It details the corporate structure of the organization, showcasing how different legal entities interconnect and relate t each other.
CostCenter: provides hierarchical data on cost centers and serves as a link between HR and Finance, crucial for budgeting and modeling processes. As a critical component in financial management, cost centers enable precise tracking and allocation of expenses, aiding in detailed and effective budgetary control and forecasting. Furthermore, within the application, the cost center can be utilized to define authorization levels and determine the population for specific financial or HR reports and analysis.
Absence: tracks leave and absence records of workers, offering insights into the frequency and reasons for worker time-off. This entity is crucial for managing workforce ability and understanding patterns in employee absence.
Demographics: provides a snapshot of the diverse characteristics of the workforce. It offers insights into the cultural, ethnic, and social background of workers, aiding in understanding the diversity and inclusivity within the organization.
PayScale: represents the salary structure associated with various roles within the organization. It is instrumental in establishing equitable and competitive compensation practices, ensuring alignment with industry standards and internal equity.
IndividualPayRange: entity to be used if Position is not used. In this case, a pay range per position can be provided.
PerformanceRatingScales: specifies rating IDs, types, scales, and reporting order, providing a structured framework for accurately reflecting various performance measurements approaches.
PerformanceReviewRating: reflects the evaluation and performance ratings of a worker. It includes the performance rating, type of review, and the period of the rating, providing a comprehensive view of the worker's performance over time.
GoalPlan: lists the goal plans within which the goal types used for performance management are going to be distributed.
GoalAchievement*: used to upload historical goal achievement data for performance tracking purposes.
This is a warning message.
This entity should not be used for any new implementations. It is still available as of now for backward compatibility purposes and it will be removed in a future version.
It is replaced by the following entity: GoalAchievementNew.
GoalPlanTypeAssignment*: links goal types to goal plans and defines the weight assigned to each goal type.
This is a warning message.
This entity should not be used for any new implementations. It is still available as of now for backward compatibility purposes and it will be removed in a future version.
It is replaced by the following entity: GoalAchievementNew.
Goal: lists goals from any outside performance management solution.
GoalPlanGoalAssignment: links goals to goal plans.
GoalAchievementNew: tracks individual worker goal assignments and performance results within goal plans.
ContractedCompensation: defines the contractual agreement between the worker and the organization regarding the compensation. Includes data such as the agreed upon amount and the currency of the compensation.
PaidCompensation: reflects the actual compensation periods between the worker and the company though data such as the payment date and the relevant compensation period.
This is a tip message.
For a full description of the data foundation entities, including a full list of the fields included in each one, their data type and whether or not they are mandatory, refer to Data foundation entities.
Enums
The following list details the standard enums available in the data foundation:
EmploymentStatus: referenced in the Employment entity.
CompensationElementFrequency: referenced in the CompensationElement entity.
CompensationElementType: referenced in the CompensationElement entity.
PerformanceRatingType: referenced in the PerformanceReviewRating entity.
PerformanceReviewType: referenced in the PerformanceReviewRating entity.
Proficiency: referenced in the Worker entity.
ContractType: referenced in the Employment entity.
MaritalStatus: referenced in the Demographics entity.
Religion: referenced in the Demographics entity.
PoliticalAffiliation: referenced in the Demographics entity.
EducationLevel: referenced in the Demographics entity.
Ethnicity: referenced in the Demographics entity.
Nationality: referenced in the Demographics entity.
PayScaleType: referenced in the PayScale entity.
PayScaleArea: referenced in the PayScale entity.
PayScaleGroup: referenced in the PayScale entity.
PayScaleLevel: referenced in the PayScale entity.
PayScaleLocalGrade: referenced in the PayScale entity.
PayScaleGlobalGrade: referenced in the PayScale entity.
PayGrade: referenced in the PayScale entity.
JobFamily: referenced in the JobArchitecture entity.
JobFunction: referenced in the JobArchitecture entity.
JobCategory: referenced in the JobArchitecture entity.
JobLevel: referenced in the JobArchitecture entity.
AbsenceType: referenced in the Absence entity.
PerformanceRatingScale: referenced in the PerformanceRatingScales entity.
GoalPlanGroup: referenced in the PerformanceRatingScales entity.
IndividualPayRangeLocalGrade: referenced in the IndividualPayRange entity.
IndividualPayRangeGlobalGrade: referenced in the IndividualPayRange entity.
GoalType: referenced in the GoalPlanGoalAssignment entity.
OrganizationType: referenced in the Organization entity.
The following list details the ISO enums available in the data foundation:
Language: referenced in the Worker entity.
Currency: referenced in the Employment, CompensationHistory*, PayScale, CostCenter, IndividualPayRange, ContractedCompensation and PaidCompensation entities.
Timezone: referenced in the Worker entity.
Country: referenced in the Worker, Employment, Organization and CompensationElement entities.
Gender: referenced in the Demographics entity.
GoalUnit: referenced in the Goal entity.
AmountSource: referenced in the ContractedCompensation entity.
This is a tip message.
For a full overview of the data foundation enums, included all the values available in the ISO enums, refer to Data foundation enums.
Data types
The following list details the data types that you will encounter when working with the data foundation entities and enums:
boolean: form of data with only two possible values (usually "true" and "false").
string: sequence of characters and the most commonly used data type to store text.
-
enum: a data type that enables predefined constants for a variable to be a set. Two types of enums are used in the Pay Suite data foundation:
ISO enums: enumerations predetermined by international standards that cannot be modified by customer. There are four of them: Language, Country, Currency and Timezone.
Standard enums: contain no values by default. To be able to use these enum, you must upload data to use them in the corresponding entity fields. For more information, refer to Data foundation enums.
date: date value in the format YYYY-MM-DD
number: numerical value, with or without a decimal.
decimal: numerical, decimal value.
integer: whole number that stores numbers without decimal points.
This is a tip message.
The currency settings are available under Workbench > Data Management > Data Upload. Access to Workbench depends on your role within the platform.
This is an attention message.
Before you attempt any data upload, beqom strongly recommends that you go through the following knowledge base articles:
Upload modes
There are two upload modes:
Full data upload: Upload the complete effective-dated history for each worker included in the file. Any previously uploaded worker records missing from the file are treated as soft deleted.
Incremental data upload: Upload only what has changed. New effective-dated records are added, and existing records are updated when field values change. Records not included remain unchanged. For ranged entities, you can now close old windows and add new ones without resending the entire history.
To select your upload behavior, click Change upload behavior button. In the Data upload behavior configuration pop-up, select the upload behavior for SFTP and UI data upload and click Save.
The currently used behavior is visible in the top panel in the Upload behavior box.
Downloading entity and enum templates
Before uploading your data, you must download a template to ensure your data is structured correctly before uploading. Templates are available for all data foundation entities and enums.
To download a template, proceed as follows:
Click Download template button located in the top right corner of the page.
A pop-up window with all available templates will appear, as illustrated in the following picture:
Data management templates
Alternatively, you can also click the Upload CSV button located in the top right corner of the page. In the File upload window, click Download template.
Click Download button next to the template you chose. The CSV file is downloaded locally on your device.
Fill the template with your data to prepare it for upload.
Uploading data
To upload your data, proceed as follows:
Click Upload CSV button, located in the top right corner of the page. A pop-up window will appear, as illustrated in the following picture:
File upload window
Click Upload to choose a file from your local device or drag & drop your files.
Click Start upload to begin the uploading process.
This is an note/information message.
You can upload up to 50 files at once, with a combined maximum size of 10MB.
Viewing your uploaded files
Upload activity overview
A dedicated table displays historical and ongoing uploads. For each upload, you can view:
Name of the entity or enum
-
Upload status:
Queued: The data upload is waiting to be processed.
Success: The data has been successfully uploaded and processed without any issues.
In Sync: The uploaded data is consistent with the destination system's current state.
In Progress: The data upload is currently underway.
Failed: The data upload encountered an unrecoverable error and could not be completed.
Stale Timeout: The data upload did not complete within the expected timeframe and was automatically stopped.
Upload source (CSV or API)
Start and completion time
Powerful Search & Filter Options
You may search for any file in the table by entering the upload name or it's attributes such as:
Upload status
Upload source (CSV or API)
Start date and time
Detailed upload view
To see a detailed view of your upload, click on the name of the upload you wish to inspect. A Ingestion log entries window will appear on the right side of the screen with following information:
Data type name
Initiator
Start time
Completion time
Current status
Upload source
If the upload you are inspecting is not in the "In progress" status, the Download log file button will be available in the bottom right corner of the window.
If the file was uploaded via CSV, the Download CSV file button will be available in the bottom right corner of the window.
The Ingestion log entries window with all the available buttons and information is illustrated in the following picture:
Ingestion log entries window