Lakehouse Glossary

A unified source of essential lakehouse-related knowledge.
Open source software terms: Apache Hudi / Hudi; Apache Kafka / Kafka; Apache Parquet / Parquet; Avro
Data architecture terms: Data warehouse; data lake; data lakehouse
Data integration terms: Streaming data; change data capture; ACID transactions; copy-on-write; merge-on-read

Filters

Clear All
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

A

ACID transactions

ACID transactions refers to database inserts, updates, and deletions that have four characteristics:Atomicity - each statement in a transaction (to read, write, update, or delete date) is treated as a single unit; that is, either the entire...

A

Apache Hudi

Apache Hudi was originally developed at Uber and was released as an open source project in 2017. Hudi is considered to be the first data lakehouse project and is today one of the three leading data lakehouse projects. Hudi was originally...

A

Apache Kafka

Apache Kafka was originally developed at LinkedIn and was released as an open source project in 2011. Kafka has many capabilities, but it is best known as scalable software for real-time data streaming. Kafka is fully distributed. Streaming...

A

Apache Parquet

Apache Parquet is an open source file format that stores data in column-based format, making it more useful for many analytics operations. This is in contrast to data stored in row-based formats, such as data stored in Avro, which are easier...

A

Avro

Avro is an open source row-based storage format. Avro has superior support for write operations, when compared to the column-based storage supported by Apache Parquet. Avro allows columns to be added or modified (schema evolution)...

C

Change data capture (CDC)

Database change data capture (CDC) enables changes to a database to be identified, tracked, and sent as updates in real-time. This allows downstream processes and/or systems to act on the change. A common use case for CDC is to keep a...

C

Copy on write (CoW)

Copy on write (CoW) refers to writing data incrementally; for instance, when using a column-based data storage format such as Apache Parquet, the data is updated only in the Parquet partitions that need to be changed. This avoids rewriting...

D

Data lake

A data lake is a repository for storing structured, semi-structured, and unstructured data, at any scale, without changing the structure of the data. The data lake uses object storage, which is very efficient for a wide range of data types and...

D

Data lakehouse

A data lakehouse provides storage of unstructured, semi-structured, and structured data, like the data lake. However, the data lakehouse adds core functionality and services that enable the data lakehouse to rival the data warehouse...

D

Data warehouse

A data warehouse is a repository originally designed in the 1980s for storing structured (relational) data for use in reporting and business intelligence. It has been widely used for that purpose ever since. A process called extract, transform...

H

Hudi

Apache Hudi was originally developed at Uber and was released as an open source project in 2017. Hudi is considered to be the first data lakehouse project and is today one of the three leading data lakehouse projects. Hudi was originally...

K

Kafka

Apache Kafka was originally developed at LinkedIn and was released as an open source project in 2011. Kafka has many capabilities, but it is best known as scalable software for real-time data streaming. Kafka is fully distributed. Streaming...

M

Merge on read (MoR)

Merge on read (MoR) stores upserts for file groups into a row-based delta log as they arrive, so write performance is high. Queries then check the delta log as well as the base file, which causes a small hit to query performance. A compactor...

P

Parquet

Apache Parquet is an open source file format that stores data in column-based format, making it more useful for many analytics operations. This is in contrast to data stored in row-based formats, such as data stored in Avro, which are easier...

S

Streaming data

Streaming data is data that is transferred in real-time or near real-time, without being stored en route. Streaming can be true streaming (new data is sent as it is generated or arrives) or micro-batches (data is aggregated up to a number of...

No result found.

Be the first to hear about news and product updates

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
We are hiring diverse, world-class talent — join us in building the future.

Try it Free

Meet with us to start your trial.
‍
Oops! Something went wrong while submitting the form.

Try it Free

Meet with us to start your trial.
‍
Oops! Something went wrong while submitting the form.
Resources
GlossarySolution GuidesWebinars
Contact
info@onehouse.ai
2550 Sand Hill Rd, Suite 200
Menlo Park, CA, 94025
© 2023 Onehouse. All rights reserved.