array_schema.md

October 25, 2024 · View on GitHub

Array Schema File

The array schema file consists of a single generic tile, with the following data:

FieldTypeDescription
Array versionuint32_tFormat version number of the array schema
Allows dupsboolNew in version 5 Whether or not the array allows duplicate cells
Array typeuint8_tDense or sparse
Tile orderuint8_tRow or column major
Cell orderuint8_tRow or column major
Capacityuint64_tFor sparse fragments, the data tile capacity
Coords filtersFilter PipelineThe filter pipeline used as default for coordinate tiles
Offsets filtersFilter PipelineThe filter pipeline used for cell var-len offset tiles
Validity filtersFilter PipelineNew in version 7 The filter pipeline used for cell validity tiles
DomainDomainThe array domain
Num attributesuint32_tNumber of attributes in the array
Attribute 1AttributeFirst attribute
Attribute NAttributeNth attribute
Num labelsuint32_tNew in version 18 Number of dimension labels in the array
Label 1Dimension LabelNew in version 18 First dimension label
Label NDimension LabelNew in version 18 Nth dimension label
Num enumerationsuint32_tNew in version 20 Number of enumerations in the array
Enumeration name length 1uint32_tNew in version 20 The number of characters in the enumeration 1 name
Enumeration name 1uint8_t[]New in version 20 The name of enumeration 1
Enumeration filename length 1uint32_tNew in version 20 The number of characters in the enumeration 1 file
Enumeration filename 1uint8_t[]New in version 20 The name of the file in the __enumerations subdirectory that contains enumeration 1's data
Enumeration name length Nuint32_tNew in version 20 The number of characters in the enumeration N name
Enumeration name Nuint8_t[]New in version 20 The name of enumeration N
Enumeration filename length Nuint32_tNew in version 20 The number of characters in the enumeration N file
Enumeration filename Nuint8_t[]New in version 20 The name of the file in the __enumerations subdirectory that contains enumeration N's data
Current domainCurrent DomainNew in version 22 The array's current domain

Domain

The domain has internal format:

FieldTypeDescription
Domain datatypeuint8_tRemoved in version 5 Datatype of all dimensions
Num dimensionsuint32_tDimensionality/rank of the domain
Dimension 1DimensionFirst dimension
Dimension NDimensionNth dimension

Dimension

The dimension has internal format:

FieldTypeDescription
Dimension name lengthuint32_tNumber of characters in dimension name
Dimension nameuint8_t[]Dimension name character array
Dimension datatypeuint8_tNew in version 5 Datatype of the coordinate values
Cell val numuint32_tNew in version 5 Number of coordinate values per cell. For variable-length dimensions, this is std::numeric_limits<uint32_t>::max()
FiltersFilter PipelineNew in version 5 The filter pipeline used on coordinate value tiles
Domain sizeuint64_tNew in version 5 The domain size in bytes
Domainuint8_t[]Byte array of length equal to domain size above, storing the min, max values of the dimension.
Null tile extentuint8_t1 if the dimension has a null tile extent, else 0.
Tile extentuint8_t[]Byte array of length equal to the dimension datatype size, storing the space tile extent of this dimension.

Note

Prior to version 5, the size of the Domain field was always equal to twice the size of the dimension's data type (which is stored in the domain in these versions).

Attribute

The attribute has internal format:

FieldTypeDescription
Attribute name lengthuint32_tNumber of characters in attribute name
Attribute nameuint8_t[]Attribute name character array
Attribute datatypeuint8_tDatatype of the attribute values
Cell val numuint32_tNumber of attribute values per cell. For variable-length attributes, this is std::numeric_limits<uint32_t>::max()
FiltersFilter PipelineThe filter pipeline used on attribute value tiles
Fill value sizeuint64_tNew in version 6 The size in bytes of the fill value
Fill valueuint8_t[]New in version 6 The fill value
NullableboolNew in version 7 Whether or not the attribute can be null
Fill value validityuint8_tNew in version 7 The validity fill value
Orderuint8_tNew in version 17 Order of the data stored in the attribute. This may be unordered, increasing or decreasing

Dimension Label

The dimension label has internal format:

FieldTypeDescription
Dimension indexuint32_tIndex of the dimension the label is for
Label orderuint8_tOrder of the label data
Dimension label name lengthuint64_tNumber of characters in the dimension label name
Dimension label nameuint8_t []The name of the dimension label
Relative URIuint8_tIf the URI of the array the label data is stored in is relative to this array
URI lengthuint64_tThe number of characters in the URI
URIuint8_t []The URI the label data is stored in
Label attribute name lengthuint32_tThe length of the attribute name of the label data
Label attribute nameuint8_t []The name of the attribute the label data is stored in
Label datatypeuint8_tThe datatype of the label data
Label cell_val_numuint32_tThe number of values per cell of the label data. For variable-length labels, this is std::numeric_limits<uint32_t>::max()
Label domain sizeuint64_tThe size of the label domain
Label domain start sizeuint64_tThe size of the first value of the domain for variable-length datatypes. For fixed-length labels, this is 0
Label domain datauint8_t[]Byte array of length equal to domain size above, storing the min, max values of the dimension
Is externaluint8_tIf the URI is not stored as part of this array

Current Domain

If a current domain is empty, only the version number and the empty flag are serialized to storage.

The current domain format is versioned separately from arrays. The current version is 1.

FieldTypeDescription
Version numberuint32_tCurrent domain version number
Emptyuint8_tWhether the current domain has a representation (e.g. NDRectangle) set
Typeuint8_tThe type of current domain stored in this file
NDRectangleMBRA hyperrectangle defined using 1DRange items for each dimension