Talk Python to Me: #562: DuckL... Note

Talk Python to Me: #562: DuckLake: The Lakehouse That's Just SQL and Parquet

DuckLake differentiates itself by asking a single SQL question rather than parsing numerous JSON and metadata files before accessing data. Its metadata resides in a real database, while the data itself is stored in plain Parquet files. Pedro Holanda, the lead developer, joined DuckDB in 2018. Guillermo Sanchez Dionis also contributes to DuckLake and the new Quack protocol. The Quack catalog enables DuckLake to handle 200 transactions per second under heavy contention, surpassing other open table formats. This episode is sponsored by Six Feet Up and Talk Python Courses. Guests include Pedro Holanda and Guillermo Sanchez; their links are provided. Further links detail progressive indexes, SQLite, Litestream, DuckDB, Iceberg, and DuckLake's specifications. The DuckLake-dataframe and Polars course are also highlighted. Information on CSV parsing, zero-copy Arrow, ART indexes, and async I/O within DuckDB is available. Finally, links to watch the episode on YouTube and find transcripts are included.