Cut Your Apache Hudi™/Apache Spark™ Costs: From DIY Fixes to Engine-Level Optimizations

Do you run Hudi at scale on OSS Spark or a runtime like AWS EMR? Hudi powers planet-scale data lakes—but without the right tuning, it can drive up storage and compute costs. Many teams see Spark and EMR bills balloon as their lakehouse scales. Small files, sprawling tables, and compute-intensive Spark jobs can quickly drive costs higher than expected. 

The good news: you can now take control.

In this session, we’ll show you how to control your top cost drivers, storage and compute, through two complementary approaches:

Part 1: Actionable Do-It-Yourself Hudi Optimizations

• Tried-and-tested approaches to small-file issues and file size control that slows queries and increases storage API costs.
• Carefully craft storage layouts for your workloads, that maximizes query performance.
• Intelligent index selection to scale write performance, as your data volume grows.
• Automate table maintenance such as clustering, compaction, and data cleaning for optimal performance

Part 2: Optimizing Runtimes and Engines for Hudi Workloads

• Spark pitfalls that may be affecting your Hudi data lakehouse performance and cost-efficiency
• Why your Spark runtime needs to specialize in handling lakehouse workload characteristics :  smooth autoscaling, fast merge operations and efficient cloud I/O
• Running your pipelines and Spark ETLs with the Quanton engine on top of Onehouse Compute Runtime, to lower your compute costs.

Whether you prefer hands-on tuning or are ready to explore smarter runtimes, you’ll walk away with practical strategies—and real-world examples—of cutting Hudi infrastructure costs by 30–70%.