The term ETL (extract, transform, load) refers to a process in which data are merged from one or several potentially differently structured sources into a target system. This involves reading data, transforming them into a target format, and then loading them into the target system. A typical application scenario in e-commerce is onboarding product data from different suppliers onto one platform.
Complex data structures – your challenge
Product data can be provided from different systems with different data formats and structures. During an extraction, product data are extracted from the sources and made available for transformation. Hereby, the source data schema is converted into the schema of the ETL system.
In order to supply your shop system, your platform, or the data warehouse with up-to-date data, the extraction must take place regularly.
Differently structured product data must be transformed into a uniform data schema. This requires technical conversions, such as changing the data types and formats, as well as content adaptations, such as eliminating duplicates, standardizing data, or converting measurement units for later use of the product data on an e-commerce platform.
In the last step of the data onboarding process, the data from the ETL system are transmitted to your platform. A highly efficient transmission of data to your system is crucial in order to avoid overloading your database during the load process. Logging the transmission of product data from the ETL system creates traceability and enables process monitoring.
Data Connector: The ETL system of e-commerce
The Data Connector allows you to onboard external product ranges from many suppliers onto one platform. Product data from different sources are structured, standardized, and validated according to the individual requirements of your e-commerce platform.