Documentation
Learn OptimaFlo
Guides and tutorials to help you go from raw data to clean dashboards; whether you're connecting your first source or deploying in your own cloud.
Popular Guides
The fastest way to get productive with the platform.
Deploy BYOC on GCP
Step-by-step walkthrough: enable APIs, grant IAM roles, and deploy via the dashboard.
15 min readBuild Your First Pipeline
Connect a data source, transform with SQL, and schedule an Airflow DAG in under five minutes.
5 min readLayered Data Architecture 101
Understand Bronze, Silver, and Gold layers and how OptimaFlo automates the flow between them.
8 min readUsing AI
Describe transformations in plain English.
6 min readBrowse by Topic
Everything from first setup to production deployment.
Getting Started
Create your first pipeline in under five minutes. Covers account setup, workspace creation, and core concepts.
- Create your first pipeline
- Workspace & project setup
- Core concepts: Ingestion, Cleaning, Aggregation
- Connecting your first data source
Platform Architecture
Understand the 8-layer architecture, layered data architecture data flow, and how DuckDB, BigQuery, and Spark auto-scale under the hood.
- Layered Data Architecture overview
- Engine selection: DuckDB vs BigQuery vs Spark
- Apache Iceberg & Polaris catalog
- Data lifecycle & time-travel
Connectors & Sources
Connect to GCS, BigQuery, REST APIs, and more. Schema inference and one-click OAuth included.
- GCS file connector
- REST API connector
- BigQuery connector
- Adding new data sources
Pipeline Builder
Drag-and-drop pipeline canvas, SQL node configuration, and scheduling with Airflow.
- Visual canvas walkthrough
- Node types & configuration
- SQL generation & copilot
- Scheduling & backfills
AI Agents
Your AI data team: Manager, Ingestion Engineer, Data Engineer, Analytics Engineer, Analyst, BI Developer, and Quality Engineer. Seven roles, one workflow.
- Manager: end-to-end orchestration
- Ingestion Engineer & Data Engineer
- Analytics Engineer & Analyst
- BI Developer & Quality Engineer
GCP
BYOC Deployment
Deploy OptimaFlo in your own cloud. Terraform-managed infrastructure with full data residency. Your data never leaves.
- GCP setup guide
- Required APIs & IAM roles
- Terraform infrastructure
- Architecture & security model
BI & Dashboards
Semantic layer, chart types, KPI tiles, dashboard creation, and embedding for stakeholder access.
- Creating dashboards
- Chart types & configuration
- Semantic layer & metrics
- Sharing & embedding
Data Quality
Quality scoring across five dimensions, schema enforcement, validation rules, and self-healing SQL across your pipeline.
- Quality scores explained
- Automated profiling
- Validation rules
- Self-healing SQL & schema enforcement
Built on Open Standards
OptimaFlo is built on Apache Iceberg, Apache Airflow, and Apache Polaris. No proprietary formats, no data lock-in. Your data stays yours, and these docs show you exactly how it all fits together.
Now in early beta. Plans from $2,500/mo. Deployed in your cloud. Your data never leaves.