Talk with our engineers by joining the new Onehouse community Slack!

Core capabilities

AI Gateway: model routing, tool access, and agent traces

Onehouse AI Gateway gives your agents one entry point to the models and tools they use. It routes inference requests to the model providers you choose, proxies tool calls to MCP servers under central access control, and records every call as open tables on your object storage. It runs in your own account on Kubernetes, like the rest of the data plane, and Onehouse adds no charge on token volume routed through it.

Connecting an agent

The gateway runs inside your network, behind the security you already have. Pointing a coding agent such as Claude Code or Codex at it takes a token and a few environment variables:

# Paste your token from the gateway token page; read -rs keeps it out of shell history.
read -rs ONEHOUSE_GATEWAY_TOKEN && export ONEHOUSE_GATEWAY_TOKEN

export ANTHROPIC_BASE_URL="<gateway url>"
export ANTHROPIC_CUSTOM_HEADERS="X-Onehouse-Authorization: Bearer $ONEHOUSE_GATEWAY_TOKEN"
export ENABLE_TOOL_SEARCH=true

claude

Model routing

Start by adding a model provider, such as OpenAI, Anthropic, or Amazon Bedrock, with your own credentials. Each provider you add has its own way of supplying API keys and its own retention policy for the data associated with it. Publish models behind stable aliases, so agents point at the gateway once and keep working when the model behind an alias changes.

For each model, choose the fallback models to use when it is unavailable, or split traffic, sending a share of requests to one model and the rest to another. You also choose which users and groups may call it. The models page lists every model across providers, and each model’s page shows its activity, routing, and permissions.

Tools and data over MCP

Agents reach your data and tools through MCP. The gateway ships with a built-in MCP server for Onehouse’s own services, including Lakegres, which exposes every table in your lake while enforcing the governance policies already set on those tables. You can register additional MCP servers for the context that lives outside the lake, such as documents, wikis, GitHub, ticketing systems, or another data platform’s MCP server.

The gateway exposes every server you configure to an agent, such as Claude Code or Codex, as one MCP server, with each server’s tools namespaced. The agent connects once, and every tool call goes through the gateway, which applies the access controls. You decide which groups get which tools, and the gateway brokers credentials, so no agent holds a provider key. Agents running outside the platform can also query the lake over standard ODBC and JDBC. For natural language questions about business metrics, the gateway uses the semantic layer, so answers come from certified definitions and stay the same every time.

Agent traces as tables

The gateway logs every call it routes, linked to the agent run it belongs to. The agent trace logging service writes these traces to your object storage as a stream, and you query them as tables. The gateway also records inference activity, latency, and estimated spend by caller and model, all in open formats on infrastructure you control.

Agents and applications can also publish traces directly over a standard OpenTelemetry endpoint or from Langfuse, from any process, whether or not their calls go through the gateway. The trace writer is built for agent volumes: a single OCU (4 cores) has sustained about 19,800 spans a second and roughly 2.9 GB/s of incoming trace data, acknowledging writes in 2.3 ms at the median. Traces land as open Iceberg tables in your storage.

Traces stay mutable, so a judge running later can attach scores and labels to a run, and every version is kept. Evaluation suites and RL environments read the trace tables with time-travel, incremental, and CDC queries. Traces live on your own object storage, so you set their retention, including keeping them indefinitely. Success and failure patterns from months ago stay available to analyze, and scored runs become the input to the next training data pipeline.

Diagram of the route and evaluate stages: an agent calls the AI Gateway, which routes to model providers and sends tool calls to Lakegres, while every call is written to the agent trace logging service, where judges add scores and the traces feed evals, RL environments, and training datasets.

Every call the gateway routes is written to the trace logging service, where it is scored later and read back as evaluation and training data.

Where it runs

The gateway is Kubernetes-native and deploys wherever the data plane does: AWS, Google Cloud, Azure and on-premises through bring your own Kubernetes, and NeoClouds, next to the models you train and serve. You keep your direct inference provider relationships and bills, and pay only for the infrastructure the gateway runs on.

Further reading