Talk with our engineers by joining the new Onehouse community Slack!

Deploying Onehouse

Bring your own Kubernetes: Azure, NeoClouds, and on-premises

Outside AWS and Google Cloud, Onehouse runs on a Kubernetes cluster you provide: bring your own Kubernetes (BYOK8s). This is how Onehouse runs on Microsoft Azure, on NeoClouds, on developer clouds, in private clouds, and in on-premises data centers. You provision and operate the Kubernetes cluster; Onehouse installs the data plane onto it and then upgrades and operates the platform remotely, exactly as it does on AWS and Google Cloud.

BYOK8s runs the full platform. The data plane depends only on Kubernetes and object storage, so Flows, Spark, SQL, Notebook and Lakegres clusters, table services, open engines, the AI capabilities, and every API run the same way. The few differences are listed in Availability by cloud.

What you provide

Requirement Detail
Kubernetes cluster Version 1.33 or a recent stable release, an overlay or per-pod CNI, and a private API endpoint. Your cluster autoscaler handles node scaling.
Node pools A worker pool for the agent and platform pods, plus pools for the workloads you plan to run, labeled as Onehouse specifies during onboarding. One pool per instance type you offer.
Object storage Azure Blob Storage / ADLS Gen2, any S3-compatible store, or HCFS storage such as HDFS, in the same region or site as the cluster. This includes stores such as NetApp, EverPure, and VAST Data.
Block storage Attached storage on the nodes for shuffle and disk spill
Network Outbound access on port 443 to a few predefined Onehouse domains, and peering or routing to your Kafka brokers and databases

The full detail is in Environment requirements.

Microsoft Azure

Onehouse is generally available on Microsoft Azure. It runs in your Azure VNet on an AKS cluster you operate, and your data stays in your ADLS storage accounts. See Bringing Onehouse Cloud to Microsoft Azure.

  • Identity. The two Onehouse identities are user-assigned managed identities, assumed through Workload Identity Federation, so no client secrets or passwords are stored in Onehouse systems.
  • Catalogs and permissions. Tables register in the OneLake catalog, so Power BI, Synapse, and other engines read them in place, and permissions sync with OneLake for Entra users, Entra groups, and OneLake roles. See Catalog interoperability.
  • Sources. Azure Event Hubs through its Kafka endpoint, and change data capture from Azure Database for PostgreSQL.
  • Sign-in and logs. Single sign-on and SCIM provisioning through Microsoft Entra, and log forwarding to your own Azure Blob Storage.

NeoClouds

NeoClouds are GPU clouds built for AI training and inference. Onehouse runs on any NeoCloud with managed Kubernetes and object storage that is S3-compatible or offers a Hadoop-compatible file system (HCFS) interface. Running Onehouse there puts ingestion, curation pipelines, agent traces, and post-training data next to the GPUs. See Run Onehouse on NeoClouds for what CoreWeave, Nebius, Crusoe, Lambda, IREN, and Nscale each provide.

Developer clouds

Onehouse runs on developer clouds like DigitalOcean, OVHcloud, and Linode (Akamai Cloud). Each provides managed Kubernetes and S3-compatible object storage: DigitalOcean Kubernetes with Spaces, OVHcloud Managed Kubernetes Service with OVHcloud Object Storage, and Linode Kubernetes Engine with Object Storage. Deployment follows the onboarding below.

On-premises and private cloud

In your own data center, Onehouse needs a Kubernetes cluster and storage that is S3-compatible or offers a Hadoop-compatible file system (HCFS) interface, such as HDFS. On an object store, file sources detect new data through the store’s bucket notifications, which most enterprise object stores emit; where a store does not, Onehouse polls the prefix instead. Database CDC does not need a Kafka cluster: LakeLog stages the change log on the object store.

Onehouse also works directly with HDFS, so you can run it against an existing Cloudera deployment. Your tables stay in HDFS, and Onehouse brings a state-of-the-art lakehouse to them: open table formats, incremental ingestion and ETL, automatic table optimization, and fast SQL and agent queries.

On a recent on-premises deployment, the data plane was running workloads two days after the Kubernetes cluster and object store were available. The preparation in environment requirements (network design, storage provisioning, firewall changes) varies by organization and usually determines how quickly Onehouse can be set up.

Onboarding

A console wizard drives onboarding and supplies the values each step needs.

  1. Start the setup wizard in the Onehouse console, under Connections > Cloud Accounts. It gives you the request ID the customer stack needs.
  2. Prepare the environment: network, subnets, NAT, storage, and administrative access, per Environment requirements.
  3. Run the customer stack. A Terraform module creates the two identities and their role assignments. It does not create the cluster or the storage account.
  4. Provision the Kubernetes cluster, using the identities from step 3 as the cluster and node identities, and naming it as the wizard specifies.
  5. Add node pools, per Node pools.
  6. Grant control plane access to the core identity, so it can install and manage the platform on the cluster. The access is scoped to the platform, and you can audit and revoke it at any time.
  7. Register the cluster. The wizard completes the link, the agent connects, and the data plane comes up.

From then on, Onehouse installs, upgrades, and operates the platform remotely. See Operations and management.

Further reading