Read the Spark UI Like a Profiler

About this session

Most Spark tuning starts in the wrong place: a slow job, a bigger cluster, and a few configs copied from a forum thread. The Spark UI already tells you where the time goes. You just have to read it the right way.

In this free working session we walk a real, slow ETL job through the Spark UI end to end: from the Jobs timeline down to individual task metrics and the SQL physical plan. Then we hand the same job to the AI agent for Spark built into the Spark UI, and show how it flags skew, shuffle, OOM risk, and executor under-utilization and turns them into concrete config and code recommendations.

You will leave with a repeatable checklist for finding the actual bottleneck before you touch a single config, and a faster way to get there with an AI agent.

What you'll learnWhat's covered

• Read the stage timeline to find the critical path, not just the slowest stage
• Spot data skew, spill, and shuffle blow-ups from task-level metrics
• Map a SQL physical plan back to the stages that execute it
• Tell executor under-utilization apart from genuinely undersized clusters
• Use the integrated AI agent for Spark to analyze the Spark UI and explain what it finds
• Turn findings into concrete config and code changes

Who it's for

Data and AI professionals: data engineers, platform engineers, and anyone who runs Apache Spark™ in production and has stared at the Spark UI wondering where to start.

Your presenter

Onehouse TeamOnehouse