Talk with our engineers by joining the new Onehouse community Slack!
Open Table Formats
Make open table formats 10x easier, and faster than DIY
Apache Hudi™ and Apache Iceberg™ were never the hard part. Running them yourself on Dataproc, EMR or Fabric is: compaction, metadata, clustering, catalogs. Onehouse auto-manages all of it natively for both formats. OneFlow for ingestion, Table Optimizer for table services, Quanton for Apache Spark™, Lakegres for serving. Same tables, same buckets, from the team that created Hudi.
Ingestion: OneFlow is the fastest lakehouse ingest in the industry
OneFlow lands CDC and streams as Hudi or Iceberg tables natively, with indexing, merge-on-read and compaction run for you, and it is the fastest lakehouse ingest in the industry. Apna's CDC into BigQuery was so expensive to merge that syncs ran every few hours; on OneFlow the same feeds merge 4x faster and land in minutes. A global telecom streams satellite telemetry through it in under 15 seconds end to end, and a blockchain company rides 100x backfill spikes inside a 10-minute SLA.
Table maintenance: Table Optimizer, instead of scheduling Spark and Python jobs
Iceberg has the same chores Hudi does, with different names: manifests that bloat under near-real-time writes, Iceberg deletion vectors that pile up on hot dimensions, snapshots to expire. Table Optimizer runs compaction, clustering, cleaning and metadata maintenance on both formats, asynchronously, with 2–4x less compute. At a $700B+ pension fund it sped up Athena, Spark and Trino by up to 12x on the same tables without a query changing.
Spark: Quanton, instead of OSS Spark, Dataproc, EMR or Fabric
Dataproc, Fabric and EMR all speed Spark up the same way: a version of Apache Gluten™ or Apache DataFusion Comet™, open-source vectorizers bolted onto the engine, on rented clusters. Quanton is real Apache Spark with its own vectorized execution and low-shuffle MERGE. On TPC-DS at 10 TB it is 4.2x faster than tuned Gluten, 4.5x faster than tuned Comet and 6x faster than open-source Spark, and it clears the queries where both run out of memory. A public fintech moved 532 pipelines onto it and saves $2.6M a year. Same spark-submit, no code changes, any cloud.
Comet and Gluten are tuned here. Out of the box, both fail Q67 and Q93 out of memory. Source: quanton.dev/benchmarks.
Serving: Lakegres, instead of BigQuery, Redshift or Athena
BigQuery was built for a human running a big scan now and then. Agents send thousands of narrow queries, each of which still scans, and that pattern can destroy your budgets. Redshift scales by resizing a cluster, so concurrency has a ceiling. Lakegres serves the same Hudi and Iceberg tables through a Postgres endpoint: BigQuery-class OLAP, sub-second lookups and the vector search BigQuery offers, on open formats. On the same tables it answers in 196 ms where Athena takes 3 s, and runs 6x faster than Databricks SQL Serverless.
Operations: the people who operated lakehouses at planet scale, in your account
Everything runs in your own AWS, GCP or Azure account on a Kubernetes compute runtime, SOC 2 Type 2, your keys and your discounts. Behind it is the team that created Hudi, ships Iceberg natively, and has operated lakes past 100 PB. When something is wrong at 3am, the person on the other end has seen it before.
What 10x easier means day to day
The table-format work you would otherwise configure, schedule and monitor yourself, for Hudi and Iceberg alike, runs for you.
Format-aware monitoring and alerts
Cost, skew, file sizes, compaction backlog, snapshot count and commit lag on one screen, with alerts on the thresholds you set.
Zero-config compaction
Hudi log files and Iceberg deletion vectors compact in the background, never blocking a writer.
Indexes that build themselves
Onehouse watches your query patterns, picks the index columns, builds the indexes and keeps them current on Hudi and Iceberg tables. No DDL from you.
Adaptive clustering
Z-order and Hilbert layouts, re-clustered as query patterns change, so reads stay fast without a weekend rewrite.
Metadata kept small
Manifests rewritten, snapshots expired and the record-level index maintained on a schedule, so query planning never becomes the bottleneck.
Every engine and catalog reads the same table
Hudi, Iceberg and Delta written natively, registered in Glue, Unity, Snowflake, Apache Polaris™ and BigQuery through OneSync. Apache XTable™ when a team needs a second format on the same table data.
Bring your own cloud
Runs in your AWS, GCP or Azure account, or on any Kubernetes cluster, on-prem included. The data never leaves your buckets.
24/7 support from the people who build the formats
Round-the-clock access to the engineers who created Hudi and run Iceberg in production, for troubleshooting, tuning and architecture guidance.
Already running open tables at mega scale?
So were the teams above, on Dataproc, EMR, BigQuery and Athena. Hundreds of petabytes, hundreds of pipelines, billions of records per table. The format was fine. The DIY stack around it was leaving most of the performance on the floor.
READ THE SUCCESS STORIES
Open table formats, in depth
Case study
Overhauling Data Management at Apna: from BigQuery batches to a streaming lakehouse
Read article
Blog
From the trenches: Managing Apache Iceberg™ metadata for near-real-time workloads
Read article
Blog
How to save Apache Iceberg™ from chilly meltdown when dimensions are too hot to handle
Read article
