Talk with our engineers by joining the new Onehouse community Slack!
Deploying Onehouse
Deploy Onehouse on AWS
On AWS, Onehouse runs inside your own AWS account and provisions the Kubernetes cluster for you. You provide the VPC, the S3 buckets, and the identities Onehouse uses; Onehouse creates the Amazon EKS cluster and its node pools, installs the data plane, and operates it from then on. Your data stays in your S3 buckets, and compute is billed to your AWS account at your own contract rates. To use your reserved instances, add their instance types as custom instance types in Onehouse.
Projects, the control plane, and connectivity work the same way on every deployment path; see How Onehouse deploys.
What you provide
| Requirement | Detail |
|---|---|
| VPC | Node subnets across at least two Availability Zones, with outbound access on port 443 through a NAT gateway |
| S3 buckets | A Onehouse customer bucket for platform metadata and job artifacts, and one or more lakehouse buckets in the cluster’s region for your tables |
| Storage endpoint | A VPC endpoint to S3, so data traffic does not pass through the NAT gateway |
| Source reachability | VPC peering or a private endpoint if your Kafka brokers or databases live in another network |
The full list, including address space and outbound access, is in Environment requirements.
What Onehouse provisions
- The Kubernetes cluster and node pools. Onehouse creates and manages the EKS cluster and the node pools for each workload type, spread across the Availability Zones you provide. Do not change node counts on these pools by hand; Onehouse manages capacity through pod scheduling.
- Instance types. The
oh-generalinstance type family maps to Gravitonm8ginstances. Project admins can add custom instance types. See Instance types. - Attached storage. Block storage for shuffle and disk spill is provisioned automatically and sized from the instance. See Attached storage.
- Identities. A cross-account IAM role that the Onehouse control plane assumes with an external ID, and a node role that workloads use to read and write your S3 buckets. Individual clusters can run as their own role to scope storage access per team.
AWS native integrations
| Area | What works on AWS |
|---|---|
| Sources | Amazon MSK, Amazon Kinesis, Amazon S3 with event notifications, and database CDC through any Kafka-compatible endpoint or LakeLog |
| Catalogs | AWS Glue Data Catalog and the Glue Iceberg REST Catalog, plus Hive Metastore, DataHub, Databricks Unity Catalog, and the Snowflake catalogs |
| Governance | Permission sync with AWS Glue through Lake Formation, and Lake Formation as the enforcement point for Lakegres clusters on a Glue catalog |
| Query engines | Amazon Athena and Redshift Spectrum read Onehouse tables in place through the catalog |
| Secrets | Bring-your-own-secrets: credentials stay in AWS Secrets Manager in your account, and Onehouse reads only the secrets you opt in by tag |
| Logs | Log forwarding to your own S3 bucket, using the workload identity |
| Networking | AWS PrivateLink between the data plane and the control plane |
Onboarding
- Start the setup wizard in the Onehouse console, under Connections > Cloud Accounts. It gives you the request ID the customer stack needs.
- Prepare the VPC and buckets described above.
- Run the customer stack. A Terraform module creates the IAM roles and their policies.
- Onehouse provisions the cluster. The Onehouse install wizard completes the link: Onehouse creates the EKS cluster and node pools in your account, installs the data plane, and the agent connects.
From then on, Onehouse installs, upgrades, and operates the platform remotely. See Operations and management.
Onehouse and AWS
Onehouse is an AWS Partner, and you can buy it through AWS Marketplace. The Onehouse team has worked directly with AWS service teams, including Amazon Athena and Amazon EMR, on the open-source projects behind them for many years, and can help with version upgrades and complex performance issues on those services. AWS solutions architects and Onehouse wrote How Onehouse makes it easy to leverage open source data services on AWS for the AWS Open Source Blog. It walks through ingesting from Amazon RDS, Amazon MSK, and S3 into open tables using Onehouse and querying them with Amazon Athena.
To get started on AWS, reach out to your AWS account team or contact Onehouse directly.