Adam Thorne

C++20 · Query execution

Vectorized Columnar Analytics Engine

A typed columnar engine for scan, filter, projection, and hash aggregation.

Why

Analytical queries should process the needed columns in batches instead of paying per-row object and dispatch costs.

Tech stack

  • C++20
  • CMake
  • CTest
  • DuckDB
  • Python benchmarks

Key metrics

39.3× scan p50 speedup
5.24× group-by p50 speedup

What I built

  • Built typed nullable columns and scan, filter, project, and hash-aggregate operators.
  • Implemented an independent row engine and fixed DuckDB queries for correctness checks.
  • Created seeded data generation and benchmark reports with latency percentiles and binary hashes.

How it works

  1. Schema-defined CSV
  2. Columnar table
  3. Scan
  4. Filter and project
  5. Hash aggregate
  6. Result checksum

Evaluation

  • Use seed 42, 3 warmups, and 20 measured Release iterations with batch size 4096.
  • Check output row counts and checksums against the independent row engine and DuckDB.
Representative checked-in local benchmark
WorkloadColumnar p50Row p50Speedup
Scan · 1M rows2.04 ms80.01 ms39.30×
Filter · 1M rows76.42 ms274.10 ms3.59×
Group by · 1M rows134.03 ms702.65 ms5.24×
Filter + group by77.01 ms216.69 ms2.81×

Scan p50 latency · lower is better

Columnar 2.04 ms
Row baseline 80.01 ms

Group-by p50 latency · lower is better

Columnar 134.03 ms
Row baseline 702.65 ms

Limitations

  • Results are local AppleClang 17 measurements on arm64 macOS.
  • There is no SQL parser, optimizer, join engine, persistence, SIMD path, or multithreaded execution.
  • The public repository exists, but source publication is still pending.

Links and reproducibility

  • Build in Release mode, regenerate seed-42 data, and run 3 warmups plus 20 measured iterations.
  • Compare the fixed reference queries against DuckDB and record compiler, host, and binary hash.