Introduction to Data Foundation
Data integration into the Pay SuiteData Foundation is the first and probably the most important step of any new beqom Pay Suite project. Data integration consists in importing all of the data of your company from your HR information system (HRIS) and into Pay Suite, to be used later in all Compensation, Performance Management or Pay Transparency processes within the platform.
beqomPay Suite comes with a standard Data Foundation (i.e. a standard collection of data tables and fields) that is fully optimized for the processes and operations performed within the solution. The standard data foundation is composed of twenty (20) entities (i.e. data tables) which in turn encapsulate fields relevant to each entity. Only three entities require actual data for the system to be operational; regardless, all entity files must be present when the data upload is performed.
In addition to entities, the Data Foundation offers enums. Enums are defined set of values that can be inserted in the data foundation fields. There are two types of enums:
ISO enums: enums for which beqom offers a defined, non editable values. For example, Language or Country. There are seven (7) of these enums.
Standard enums: enums for which you can defined your own values. For example: EducationLevel or AbsenceType. There are thirty-one (31) of these enums.
For a successful integration into the Pay Suite system, you must map your HR data with the various entities and fields of the data foundation before loading the entity files. In addition, there are interactions and dependencies between the entities and the enums that you need to be aware of when uploading data into the system.
This is a miscellaneous message.
You can view a visual representation of the Data Foundation here:
(missing or bad snippet)
Entities
The following list details the entities for which data is mandatory at the time of data load:
Worker: the worker entity represents an individual worker within the organization. It contains information specific to the worker, outside of their current employment status or job position, such as name, contact information, gender, etc.
Organization: the organization entity encapsulates the hierarchical arrangement of units, departments, or teams within an organization. It details the structure of the organization, showcasing how different parts interconnect and relate to each other.
Employment: the employment entity represents the specific employment relationship between a worker and the organization. It contains information related to the worker's employment history, start date, type of contract, etc.
Other entities
The following list details the entities for which data is not mandatory at the time of data load:
CompensationHistory*: records the financial remuneration history of workers. It is critical for understanding compensation trends, making informed pay decisions, and managing payroll and budget planning.
This is a warning message.
This entity should not be used for any new implementations. It is still available as of now for backward compatibility purposes and it will be removed in a future version.
It is replaced by the following entities: ContractedCompensation and PaidCompensation.
CompensationElement: encompasses the various components of an worker's total compensation package. It plays a critical role in benefits administration, compensation strategy, and overall employee satisfaction.
JobArchitecture: defines the job framework within the organization, categorizing roles based on function, level, and other job-related characteristics. This entity is key in creating a structured approach to job classification and career progression.
LegalEntity: encapsulates the hierarchical arrangement of legal entities within organization. It details the corporate structure of the organization, showcasing how different legal entities interconnect and relate t each other.
CostCenter: provides hierarchical data on cost centers and serves as a link between HR and Finance, crucial for budgeting and modeling processes. As a critical component in financial management, cost centers enable precise tracking and allocation of expenses, aiding in detailed and effective budgetary control and forecasting. Furthermore, within the application, the cost center can be utilized to define authorization levels and determine the population for specific financial or HR reports and analysis.
Absence: tracks leave and absence records of workers, offering insights into the frequency and reasons for worker time-off. This entity is crucial for managing workforce ability and understanding patterns in employee absence.
Demographics: provides a snapshot of the diverse characteristics of the workforce. It offers insights into the cultural, ethnic, and social background of workers, aiding in understanding the diversity and inclusivity within the organization.
PayScale: represents the salary structure associated with various roles within the organization. It is instrumental in establishing equitable and competitive compensation practices, ensuring alignment with industry standards and internal equity.
IndividualPayRange: entity to be used if Position is not used. In this case, a pay range per position can be provided.
PerformanceRatingScales: specifies rating IDs, types, scales, and reporting order, providing a structured framework for accurately reflecting various performance measurements approaches.
PerformanceReviewRating: reflects the evaluation and performance ratings of a worker. It includes the performance rating, type of review, and the period of the rating, providing a comprehensive view of the worker's performance over time.
GoalPlan: lists the goal plans within which the goal types used for performance management are going to be distributed.
GoalAchievement*: used to upload historical goal achievement data for performance tracking purposes.
This is a warning message.
This entity should not be used for any new implementations. It is still available as of now for backward compatibility purposes and it will be removed in a future version.
It is replaced by the following entity: GoalAchievementNew.
GoalPlanTypeAssignment*: links goal types to goal plans and defines the weight assigned to each goal type.
This is a warning message.
This entity should not be used for any new implementations. It is still available as of now for backward compatibility purposes and it will be removed in a future version.
It is replaced by the following entity: GoalAchievementNew.
Goal: lists goals from any outside performance management solution.
GoalPlanGoalAssignment: links goals to goal plans.
GoalAchievementNew: tracks individual worker goal assignments and performance results within goal plans.
ContractedCompensation: defines the contractual agreement between the worker and the organization regarding the compensation. Includes data such as the agreed upon amount and the currency of the compensation.
PaidCompensation: reflects the actual compensation periods between the worker and the company though data such as the payment date and the relevant compensation period.
Enums
The following list details the ISO enums available in the Data Foundation:
Language: referenced in the Worker entity.
Currency: referenced in the Employment, CompensationHistory*, PayScale, CostCenter, IndividualPayRange, ContractedCompensation and PaidCompensation entities.
Timezone: referenced in the Worker entity.
Country: referenced in the Worker, Employment, Organization and CompensationElement entities.
Gender: referenced in the Demographics entity.
GoalUnit: referenced in the Goal entity.
AmountSource: referenced in the ContractedCompensation entity.
The following list details the standard enums available in the Data Foundation:
EmploymentStatus: referenced in the Employment entity.
CompensationElementFrequency: referenced in the CompensationElement entity.
CompensationElementType: referenced in the CompensationElement entity.
PerformanceRatingType: referenced in the PerformanceReviewRating entity.
PerformanceReviewType: referenced in the PerformanceReviewRating entity.
Proficiency: referenced in the Worker entity.
ContractType: referenced in the Employment entity.
MaritalStatus: referenced in the Demographics entity.
Religion: referenced in the Demographics entity.
PoliticalAffiliation: referenced in the Demographics entity.
EducationLevel: referenced in the Demographics entity.
Ethnicity: referenced in the Demographics entity.
Nationality: referenced in the Demographics entity.
PayScaleType: referenced in the PayScale entity.
PayScaleArea: referenced in the PayScale entity.
PayScaleGroup: referenced in the PayScale entity.
PayScaleLevel: referenced in the PayScale entity.
PayScaleLocalGrade: referenced in the PayScale entity.
PayScaleGlobalGrade: referenced in the PayScale entity.
PayGrade: referenced in the PayScale entity.
JobFamily: referenced in the JobArchitecture entity.
JobFunction: referenced in the JobArchitecture entity.
JobCategory: referenced in the JobArchitecture entity.
JobLevel: referenced in the JobArchitecture entity.
AbsenceType: referenced in the Absence entity.
PerformanceRatingScale: referenced in the PerformanceRatingScales entity.
GoalPlanGroup: referenced in the PerformanceRatingScales entity.
IndividualPayRangeLocalGrade: referenced in the IndividualPayRange entity.
IndividualPayRangeGlobalGrade: referenced in the IndividualPayRange entity.
GoalType: referenced in the GoalPlanGoalAssignment entity.
OrganizationType: referenced in the Organization entity.
Effective-dated entities
This is an attention message.
This option applies to CSV uploads via UI and SFTP.
In our schema, there are two simple patterns for effective dating. Knowing which one you're loading will make the process easier.
-
Point-in-Time (EffectiveDated)
This method is for data that is only valid on a single, specific date. This is also called a point-in-time record, and it has no end date.
How it works: Each new record represents a change at a specific point in time.
Common examples: Employee records, like a worker's job title on a certain day.
The key: The business key is the WorkerExternalId plus a unique identifier for that record.
How to make changes:
To add a change, you create a new record with a new date.
To update a record on an existing date, you change the value of that record.
If you try to update with the exact same value, nothing happens.
-
Date Range (EffectiveDateRanged)
This method is for data that is valid over a specific period, with a start (EffectiveRangeStartDate) and an end (EffectiveRangeEndDate) date. This is also called a date range for EffectiveDateRanged entities.
How it works: Each record covers a window of time.
Common examples: Compensation history, like a salary that was active from one date to another.
The key: The business key is the WorkerExternalId plus a unique identifier for that record.
How to make changes:
A record's date range cannot overlap with another for the same Business Key.
Every record must have both a StartDate and an EndDate.
For things that are ongoing (like a base salary with no end yet), use a far-future date like 9999-12-31 as the end date.
To update a record within its existing date range, you change the field values. If the values are the same, nothing happens.
This is a tip message.
For the same business key, date ranges may not overlap. Always close the old one before starting the new one.
Data types
The following data types are used in the fourteen entities, depending on each specific field.
This is an note/information message.
For more information about the individual fields in each entity, refer to Data foundation entities.
boolean: form of data with only two possible values (usually "true" and "false").
string: sequence of characters and the most commonly used data type to store text.
-
enum: a data type that enables predefined constants for a variable to be a set. Two types of enums are used in the Pay Suite data foundation:
ISO enums: enumerations predetermined by international standards that cannot be modified by customer. There are four of them: Language, Country, Currency and Timezone.
Standard enums: contain no values by default. To be able to use these enum, you must upload data to use them in the corresponding entity fields. For more information, refer to Data foundation enums.
date: date value in the format YYYY-MM-DD
number: numerical value, with or without a decimal.
decimal: numerical, decimal value.
integer: whole number that stores numbers without decimal points.
Once all the mapping is done, there are two ways of uploading the data:
Via API
Via SFTP upload
Via the UI
Upload modes
The platform offers two modes of data upload that you can use depending on what you are trying to achieve with the data upload: full data upload and incremental data upload.
This is an attention message.
The selected data upload mode applies to CSV file upload via UI and SFTP.
This is an note/information message.
The selection of the data upload mode via the application user interface is described in the Workbench category, under Uploading data into the data foundation.
Full data upload (complete history files):
This method enables you to upload the complete effective-dated history for every worker present your data files. Any existing records for those workers that are not in the new file will be "soft-deleted" (i.e. removed from active use).
This upload mode operates under the following principles:
What is sent: files containing all the records you want to include in the system for each worker. In other words, the entire history of the workers for that specific type of data.
What the system does: the data foundation completely rebuilds the data for the workers. Any records for those workers that already existed in the system but are not present in the new files are removed. Workers not included in the files are left unchanged.
How it works: the system uses the business key to determine if a row is a new record or an update. The business key combines the worker ID with other unique identifiers and the date(s).
Is it possible to add more people later? Yes. It is possible to upload files for different workers at different times. However, if at any point, data for a worker is re-uploaded, you must include their full history again, as omitting old records will delete the worker.
-
Best for:
When your source system can consistently export a complete history for the people you're including.
Initial data uploads into a new system.
When you want each file to be the definitive source of truth for the workers it contains.
Incremental (delta) data upload (change‑only files)
This method enables you to upload only the data has changed since the last data upload. With this method, the system adds new records or updates existing ones, but does not delete any records that aren't in the file.
What is sent: files with only the new or changed records.
-
What the system does:
If a row has a matching business key to an existing record, then the row is updated.
If there's no match, a new record is inserted.
Records not included in the file remain unchanged. No records are soft-deleted.
-
Special notes for date-ranged records:
To start a new period, you first need to "close" the old one by setting its EndDate. Then you insert the new record with its new StartDate. This is common for things like base salary changes. To update a period range, beqom recommends you use the full upload method, as this constitutes a historical correction.
-
Best for:
Daily or weekly uploads of recent changes instead of full history files.
When you want previously loaded history to stay in the system, unless you specifically change or delete it.
Quick checklists
Before any upload, ensure that:
Dates are in YYYY-MM-DD format.
All required fields are present.
For CompensationHistory, each compensation element (such as salary or bonus) is on a separate row.
If you use the Full upload method:
For each worker in your file, you've included their complete history for that entity.
You are aware that any records for included people that are missing from your file will be removed.
If you use the Incremental upload method:
You are sending only the new or changed rows.
Going further