Query processing

A query is a request for information from a data store. Fast, efficient query processing is vital to many processes in business, science, and other fields.

The structured query language (SQL) format was invented, and adopted across a wide range of relational databases, to standardize and speed many aspects of query processing. In recent decades, considerable effort has been made to get SQL-like concision and performance for queries on semi-structured and unstructured data. This includes the creation of variants of SQL, such as kSQL, used to query data streamed by Apache Kafka.

Query processing is assisted by a very wide range of techniques, notably including the creation of data tables optimized for fast responses to queries. For example, columnar tables (including in Apache Parquet format) can be used to process many queries much faster than if the data were in rowstore tables. Columnar tables often include summary information to further speed specific kinds of queries.

Related terms: ingest; Apache Parquet; Parquet

On the Onehouse website: 

• Incrementally process data with Apache Hudi - Onehouse
• Hudi's Column Stats Index and Data Skipping feature help …
• Top 3 Things You Can Do to Get Fast Upsert Performance ...