Every data engineer encounters tables with JSON columns, a valuable but painful part of the data landscape. These columns hold raw truth but incur significant query performance costs due to string parsing. Apache Iceberg v3 introduces the Variant type to address this, offering JSON-like flexibility with near-typed column performance. This is achieved through a technique called shredding.Before Variant, two bad options existed. Storing JSON as a string was easy to ingest but slow to read, as the entire string had to be parsed for every query, and storage was verbose. The alternative was flattening JSON into typed columns, which made queries fast but created operational pain due to schema instability and frequent migrations. Many teams resorted to maintaining both, adding complexity without new information.Variant collapses these options into one flexible, fast column. It's a data type for values whose structure varies row-to-row, supporting objects, arrays, and primitive types like dates and decimals. Crucially, Variant values are stored in a binary encoding, not text, leveraging the Apache Parquet standard. This encoding splits values into a metadata section (a dictionary of field names) and a value section, storing primitives efficiently and enabling direct jumps to specific fields.While binary encoding improves on string storage, it doesn't achieve full columnar performance because Parquet views Variant values as opaque blobs. This invisibility prevents crucial optimizations like columnar reading, compression, and pruning. Shredding solves this by making the internal structure visible to the file format.Shredding works by having the engine identify consistently appearing fields within Variant values at write time and store them as separate, typed Parquet columns. For instance, common fields like user_id and event_type are pulled out into their own columns, while less common or varying fields remain in a residual binary column. This is analogous to a mailroom clerk filing common documents (invoices, labels) separately while keeping other items in the original envelope.Mechanically, shredding creates a pair of columns for each common field: typed_value and value. The typed_value column holds the field when it matches an expected type, benefiting from all columnar optimizations. The value column acts as a fallback, storing the field in binary Variant form if it doesn't match the expected type. This ensures data integrity while dramatically speeding up queries for common fields.