What is data transformation?

Data transformation is the step where raw data gets reshaped into a form that's ready to use: cleaning bad values, standardizing formats, joining tables, and aggregating rows into metrics. It's the T in both ETL and ELT, the only difference being whether it happens before or after the data lands in its destination.

What counts as a transformation

Common transformations include deduplicating rows, casting a column to the right type, joining two tables on a shared key, filtering out invalid records, and aggregating detail rows into a summary metric like daily revenue. A single pipeline usually chains several of these together.

Why it needs review

Every transformation is a judgment call encoded as logic: which rows count as duplicates, which join key is correct, which rows to exclude as invalid. Get one of those wrong and the output still runs, it just produces a number nobody should trust. That's why a person checking the logic before it ships matters more than the transformation running successfully.

How OptimaFlo implements this

The Analytics Engineer role generates SQL transformations from a plain-English request, run against the schema already sitting in the Raw layer. You preview the generated SQL, edit it if it's not right, and approve it before it executes. Every approved transformation is version controlled in git, so there's a record of what changed and why.

Related terms

• ETL pipeline: The pattern where transformation happens before data loads into its destination.
• Medallion architecture: The Clean layer is where OptimaFlo's transformations run, against data already landed in Raw.
• Schema evolution: What happens when a source's shape changes underneath a transformation that expects a fixed schema.