README.md
February 17, 2025 ยท View on GitHub
The Variant spec is being developed in the Apache Parquet project, and has not been finalized. Spark currently implements the version of the spec at https://github.com/apache/parquet-format/blob/37b6e8b863fb510314c07649665251f6474b0c11/VariantEncoding.md and https://github.com/apache/parquet-format/blob/37b6e8b863fb510314c07649665251f6474b0c11/VariantShredding.md, but does not yet support Variant values containing UUID, Time, or nanosecond-precision Timestamp. Spark does not yet have public APIs to enable shredded writes.