---
title: "Configure data imports and transformations"
description: "The Data Processing Engine (DPE) is the component that lets you create and execute data pipelines outside and within the Platform"
url: https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/getting-started-prepare-data
lang: en
lastUpdated: 2026-09-14
---
> For AI agents: the complete documentation index is available at https://docs.ovhcloud.com/en/llms.txt, the full documentation bundle is available at https://docs.ovhcloud.com/en/llms-full.txt.
# Configure data imports and transformations
## Objective
The Data Processing Engine (DPE) is the component that lets you **create and execute data pipelines outside and within the Platform**. Everything is automated, from the management and deployment to the scalability, to help you focus on the business logic of your data Project rather than its logistics.
While the Lakehouse Manager is where you plan out your data structure at the **conceptual level**, the Data Processing Engine is where you execute actions to **physically impact your data**.
:::info
This tutorial is meant for **Standard Datasets** using the _Lakehouse Manager Engine_. To know how to follow this same step with **External Datasets** click [here](https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/tutorials-external-datasets.md).
:::
## Actions
An [action](https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/landing-page-dpe-actions.md) consists of a unitary physical operation on the data. Actions can be organized in stages in order to produce automated data processing pipelines called [workflows](https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/dpe-workflows.md).
Click on the **Actions** menu of your Data Processing Engine. You should see the two _Load_ actions that were[ automatically generated in the previous step](https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/getting-started-organize-data.md#organize-your-data-in-tables):
These Load actions will physically extract the data from your sources and load it into your data warehouse, following the schema made in the Lakehouse Manager.
### Create more actions
:::info
Our Marketplace gives you access to a dozen of curated actions to kick-start your data processing Projects: _load_ actions, _aggregate_ actions, _delete_ actions, etc. If you cannot find what you need in the catalog, you can always resort to a _custom_ action that allows you to **run any piece of Python 3+ code** as part of your data pipelines.
:::
[↪ Learn more about custom actions](https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/dpe-actions-custom.md).
For this tutorial, you will create an action used to aggregate your data into the _dataset\_history_ table you created in the previous part.
Click on **New action** and select the _Aggregate action_ template from the Store.
There will be 3 simple steps to configure the aggregate action:
- **(1)** Select the source table : _stations\_rides_
- **(2)** Select the destination table: _dataset\_history_
- **(3)** Select the source table : _chicago\_calendar\_full_
Change the join condition to an _INNER join_ using the dropdown menu. This will ensure that you do not have any null fields in the records of your dataset\_history table. Also make sure to copy and paste the following sql code to let the aggregate action know how the tables are going to be linked:
```
stations_rides.date = chicago_calendar_full.date
```
:::warning
Note this is required because the concept of _joining tables using primary keys_ does not exist for **standard datasets**. If you are using an **external dataset**, then it will be automatically generated.
:::
The attributes need to be manually mapped out. Refer the screenshot below and make sure all the attributes are mapped correctly.
Make sure that the all the attribute are in the **SELECT** function.
:::warning
If you are using an **external dataset**, then the attributes will be automatically mapped out. Only the attribute _rides_ needs to be switched to **SUM**.
:::
Finally, let's define the categorical attribute _cat\_temperature_ that you created earlier. Click on the **\< map >** (short for "mapping") option in blue option dropdown as shown below and switch it to **\< sql >**.
Simply copy-paste the SQL command below:
```
CASE
WHEN chicago_calendar_full.temperature < 40 THEN 'very cold'
WHEN chicago_calendar_full.temperature < 48 THEN 'cold'
WHEN chicago_calendar_full.temperature < 55 THEN 'medium'
WHEN chicago_calendar_full.temperature < 62 THEN 'hot'
ELSE 'very hot'
END
```
:::warning
**Leaving a destination attribute unmapped in the Aggregate action configuration will trigger an error when the action is launched**. If you'd rather leave the destination field empty, make sure to simply remove it from the list of mapped attributes.
:::
[↪ Learn more about Aggregate actions](https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/dpe-actions-aggregate.md)
Click on **Create** at the top-right.
**You have now generated all the actions** required for this tutorial.
:::info
Of course, your real-life Project will probably have more than 3 actions. You can organize your actions in folders and rename them if needed. You can also use more than one repository, especially if you are working in collaboration with teammates. Repositories of actions can be versioned and also synced with external Git repositories. Check-out the [dedicated product documentation](https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/dpe-actions-manage.md#manage-actions) page to learn more about how to do that!
:::
[↪ Learn more about Actions in the Product Documentation](https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/landing-page-dpe-actions.md)
Before you proceed to the next step in creating workflows, we will create one final action - **Delete**.
Its use is very straightforward, to delete the contents of the selected table(s).
Create a new delete action and add all the tables we have created so far - the idea is simple, if you are loading a table with new data or loading the table again with old data then you will need to clean the table before doing so because its best practice.
## Workflows
A workflow defines the order in which your actions run.
Within a workflow, actions are organized in sequential stages. Within a stage, all actions will be run in parallel while stages will always run one after the other. The same action can be used multiple times in the same workflow. A workflow, just like an action, can either be launched manually, set up to run on a schedule or triggered through an API call.
:::info
Note that it is important to remember that **stages are run one after the other** in the order you planned them while **actions contained in a stage are all run all at the same time** regardless of the order. In short, the order of the actions inside a stage does not matter while the stages’ order inside a workflow does.
:::
To create your first workflow, you'll need to head to the _Workflow_ tab and click on **New Workflow**. Head to preferences or double-click on the header name to set a new name _Import Chicago Data_.
Let's start by defining three different stages by clicking on **Add a stage**. Then, add actions in each stage using the dropdown search selector following the screenshot provided as a guide for each stage.
After creating the workflow (**create**), press **Play**.
:::info
Please note that workflows might take a few minutes to run when you launch them for the first time. The total time should not exceed 10 minutes - if it does, please reach out to our support team.
:::
While the workflow is running, something you can do is **schedule it to run daily** using a trigger.
Head to the _Preferences_ tab of your workflows and scroll to the Triggers widget on the bottom left side. Click **+Add**.
Select the trigger type as _CRON_ and mode as _Simple_. Navigate to the **Daily** tab and in the list of options select _Every 1 day(s)_ as shown in the picture below:
Hit the **Confirm** button to create the new trigger event with the name of your choice and it will append it below the _Launch Endpoint_ present by default in the Trigger event table.
:::info
There is a lot more you can configure in a workflow's preferences. Namely, you can [scale horizontally and vertically](https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/dpe-jobs-resources.md) any processing job, use [workload segmentation](https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/tutorials-segmentation.md) to accelerate the data processing and even save all these configurations for repeated use thanks to [environments](https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/dpe-environments.md).
:::
[↪ Learn more about configuring the execution preferences.](https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/dpe-jobs-preferences.md)
:::warning
Make sure to click on the **Save** button on the top right of the screen whenever you make a modification to your workflows. Actions are stored in repositories which can be versioned which is not the case for workflows or environments. _Autosave_ is therefore disabled for both workflows & environments.
:::
## Jobs
To wrap up this section, here are a few words about the last tab of the Data Processing Engine component: jobs.
The Jobs tab summarizes **all executions triggered in the Data Processing Engine** and includes advanced metrics reports. Jobs are listed under three main categories: running, queued and past executions. Having a look at the last jobs executed, you can verify the status of the workflow you just launched.
:::info
Resource allocation on the Platform is done through **proprietary units called [DPU](https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/manage-pricing.md#billing-units)**, Data Platform Units. You can change the amount of DPU dedicated to an action or workflow in their [preferences](https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/dpe-jobs-preferences.md).
:::
You now have completed the Data Engineering section of the _Getting Started_ tutorial.
A good way to make sure your data properly loaded is to go back to the [Lakehouse Manager](https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/landing-page-lakehouse-manager.md) and look at the number of rows loaded into the table. Just open the **list view mode** and check the _rows_ column, if the field has a number (indicating how many rows have been loaded) then everything worked well.
Remember you are setting up a **production-grade system**, hence why you need both a Lakehouse Manager and a Data Processing Engine. Don't hesitate to reach out at the Data Platform Channel within the [Discord Server](https://discord.com/channels/850031577277792286/1163465539981672559) if you're running into any issues or you'd like help to speed up your data pipelines, optimize their performance, adding custom exotic data sources or creating custom scripts!
Now let's move on to the next component in your data system: the Analytics Manager.
[Create insightful queries with the Analytics Manager](https://docs.ovhcloud.com/en/guides/public-cloud/data-platform/getting-started-create-queries.md)
## Go further
If you need training or technical assistance to implement our solutions, contact your sales representative or click on [this link](https://www.ovhcloud.com/en-gb/professional-services/) to get a quote and ask our Professional Services experts for a custom analysis of your project.
Ask questions, give your feedback and interact directly with the team building the Data Platform on the dedicated [Discord channel](https://discord.gg/ovhcloud).
If you need support with your OVHcloud services, create a request in our [Help Centre](https://help.ovhcloud.com/csm?id=csm_get_help).
Join our [community of users](https://community.ovhcloud.com/).