ReadGrib2

January 19, 2026 · View on GitHub

This repository is based on https://github.com/duckdb/extension-template, check it out if you want to build and ship your own DuckDB extension.

DuckDB GRIB2 Extension (`read_grib2`)

A DuckDB C++ extension that enables direct querying of GRIB2 meteorological data files using SQL. This extension is built on top of ECMWF ecCodes and exposes GRIB2 messages as DuckDB table functions.

Features

📦 Read GRIB2 files directly from DuckDB
🌍 One row per grid point per GRIB2 message
🧭 Extracts latitude, longitude, and value arrays
🏷 Exposes rich GRIB metadata (discipline, parameter, level, forecast time)
🔍 List available variables without decoding full grids
🚀 Streaming decode (low memory overhead)

Provided Table Functions

`read_grib2(filename)`

Reads all GRIB2 messages and emits one row per grid cell.

Schema

Column	Type	Description
`variable`	VARCHAR	GRIB short name (e.g. `t`, `u10`)
`long_name`	VARCHAR	Human-readable parameter name
`discipline`	INTEGER	GRIB discipline
`category`	INTEGER	Parameter category
`number`	INTEGER	Parameter number
`level`	INTEGER	Vertical level
`level_type`	VARCHAR	Level type (e.g. `surface`, `isobaricInhPa`)
`forecast_time`	INTEGER	Forecast lead time
`latitude`	DOUBLE	Latitude
`longitude`	DOUBLE	Longitude
`value`	DOUBLE	Data value

Example

SELECT *
FROM read_grib2('gfs_sample.grib2')
LIMIT 10;

SELECT COUNT(*)
FROM read_grib2('hrrr.t00z.wrfsfcf00.grib2')
WHERE long_name = 'Vegetation';

Example Output

D select count(*) from read_grib2('/Users/Shared/hrrr/data/hrrr.t00z.wrfsfcf00.grib2') where long_name = 'Vegetation';
[DEBUG] NextMessage: No more messages (EOF) or failed to create handle. err=0
100% ▕██████████████████████████████████████▏ (00:01:45.19 elapsed)     
┌────────────────┐
│  count_star()  │
│     int64      │
├────────────────┤
│    1905141     │
│ (1.91 million) │
└────────────────┘
D

D select count(*) from read_grib2('/Users/Shared/hrrr/data/hrrr.t00z.wrfsfcf00.grib2');
[DEBUG] NextMessage: No more messages (EOF) or failed to create handle. err=0
100% ▕██████████████████████████████████████▏ (00:01:43.96 elapsed)     
┌──────────────────┐
│   count_star()   │
│      int64       │
├──────────────────┤
│    323873970     │
│ (323.87 million) │
└──────────────────┘
D

D select * from read_grib2('/Users/Shared/hrrr/data/hrrr.t00z.wrfsfcf00.grib2') where long_name = 'Vegetation' limit 1;
100% ▕██████████████████████████████████████▏ (00:00:59.06 elapsed)     
┌──────────┬────────────┬────────────┬──────────┬────────┬───────┬────────────┬───────────────┬───────────────────┬────────────────────┬────────┐
│ variable │ long_name  │ discipline │ category │ number │ level │ level_type │ forecast_time │     latitude      │     longitude      │ value  │
│ varchar  │  varchar   │   int32    │  int32   │ int32  │ int32 │  varchar   │     int32     │      double       │       double       │ double │
├──────────┼────────────┼────────────┼──────────┼────────┼───────┼────────────┼───────────────┼───────────────────┼────────────────────┼────────┤
│ veg      │ Vegetation │     2      │    0     │   4    │   0   │ surface    │       0       │ 21.13812299999999 │ 237.28047200000003 │  0.0   │
└──────────┴────────────┴────────────┴──────────┴────────┴───────┴────────────┴───────────────┴───────────────────┴────────────────────┴────────┘
D

`read_grib2_metadata(filename)`

Lists all variables present in a GRIB2 file without emitting grid values.

Schema

Column	Type	Description
`variable`	VARCHAR	Variable name with level information

Example

SELECT *
FROM read_grib2_metadata('gfs_sample.grib2');

Example output:

t (isobaricInhPa 850)
u (heightAboveGround 10)
v (heightAboveGround 10)

How It Works

Uses ecCodes to stream-decode GRIB2 messages
Automatically sets ECCODES_DEFINITION_PATH at runtime
Decodes:
- GRIB metadata
- Grid dimensions (Ni, Nj)
- Latitude, longitude, and value arrays
Integrates with DuckDB’s table function API
Fully streaming — does not load entire files into memory

Platform Support

⚠️ Current Status

This extension is currently supported on:

✅ Linux (glibc / musl)

✅ macOS (Intel & Apple Silicon)

🚧 Windows support is not available.

Windows Builds and ecCodes (Important)

This extension depends on ecCodes, which fully supports Windows and can be built successfully as a standalone project using CMake or package managers such as vcpkg.

However, a specific incompatibility occurs when building this extension on Windows inside DuckDB’s extension build system.

The Problem

When ecCodes is added to this extension using:

add_subdirectory(dependencies/eccodes)

it brings in ecBuild, which installs global CMake logging hooks.

On Windows, these hooks attempt to write log files with timestamped filenames containing : characters, for example:

2026-01-03T11:34:37 - DuckDB - DEBUG - CMake project(RE2)

Windows filesystems do not allow : in filenames, causing CMake to fail with errors such as:

file failed to open for writing (Invalid argument)

Because ecBuild’s logging hooks are global, this failure affects all DuckDB subprojects, not just this extension. As a result, unrelated DuckDB components (for example core_functions, parquet, re2, libpg_query) fail during configuration.

Why This Cannot Be Fixed in the Extension

ecBuild’s logging behavior is global and affects every CMake project() call
DuckDB’s extension CI builds many subprojects in a single CMake invocation
This interaction causes ecBuild to attempt to log DuckDB internal projects using Windows-invalid filenames
The issue does not occur on Linux or macOS, where such filenames are permitted

Windows Build Strategy

To avoid this issue, ecCodes is not built from source on Windows inside this extension.

Instead, Windows builds require a preinstalled ecCodes library, which is linked against at build time.

Supported approaches include:

Installing ecCodes via vcpkg
Building ecCodes manually as a standalone project and installing it system-wide

In the future, this extension will automatically detect and link against an existing ecCodes installation when building on Windows.

WASM Build Status

Currently, building the read_grib2 DuckDB extension for WebAssembly (WASM) is not supported.

The build system fails because ecBuild, a required tool for configuring ecCodes, cannot run under Emscripten. ecBuild is designed as a host build tool and detects WASM as an unsupported platform, immediately aborting with a critical error.

As a result, attempts to cross-compile the extension for WASM using the normal CMake workflow will not work.

Native builds on Linux and macOS (x86_64) continue to work as expected.

Note: It may be possible to enable WASM in the future by first building ecBuild for the host system and then pointing the WASM build of ecCodes to use that pre-built host tool. This approach has not been implemented or tested in this project.

Requirements

DuckDB (extension API)
ecCodes (GRIB decoding)
CMake ≥ 3.17
C++17 compatible compiler

Runtime Dependencies

The extension expects the following directory layout next to the compiled extension:

read_grib2.duckdb_extension
└── eccodes/
    └── definitions/
        └── boot.def

The extension automatically sets:

ECCODES_DEFINITION_PATH=<extension_dir>/eccodes/definitions

If boot.def is missing, the extension will fail fast.

Building

Example build flow:

git clone --recursive https://github.com/oglego/duckdb_grib2
cd duckdb_grib2
make

Ensure ecCodes and DuckDB headers are discoverable by CMake.

Once the build completes you can load the extension by running the following:

duckdb -unsigned
LOAD "location/of/build/build/release/extension/read_grib2/read_grib2.duckdb_extension";

Tips for speedy builds

DuckDB extensions currently rely on DuckDB's build system to provide easy testing and distributing. This does however come at the downside of requiring the template to build DuckDB and its unittest binary every time you build your extension. To mitigate this, we highly recommend installing ccache and ninja. This will ensure you only need to build core DuckDB once and allows for rapid rebuilds.

To build using ninja and ccache ensure both are installed and run:

git clone --recursive https://github.com/oglego/duckdb_grib2
cd duckdb_grib2
GEN=ninja make

Error Handling

All ecCodes calls are checked and throw C++ exceptions on failure
Invalid or corrupted GRIB2 messages fail fast with descriptive errors
EOF is handled gracefully during streaming scans

Performance Notes

One row per grid point can be very large (millions of rows)
Prefer filtering in SQL:

SELECT *
FROM read_grib2('file.grib2')
WHERE variable = 't'
  AND level = 850;

Use read_grib2_metadata() to inspect contents before full scans

Example Use Cases

Weather model analysis
Climate research
Geospatial joins with DuckDB + Parquet
Feature extraction for ML pipelines
SQL-based meteorological analytics

Future Enhancements

Contributions and feature requests are welcome — this project is designed to grow alongside DuckDB’s analytics ecosystem.

License

MIT

Legal Disclaimer

This software is provided "as is", without warranty of any kind, express or implied. The author(s) and contributor(s) assume no responsibility or liability for:

The accuracy, completeness, or fitness of the data produced
Any errors, omissions, or interruptions in operation
Any loss of data, financial loss, or damages arising from use or inability to use this software
Decisions, actions, or outcomes based on data read or processed using this extension

GRIB2 data often originates from third-party sources (e.g. weather agencies or model providers). Users are solely responsible for validating data correctness, licensing, and suitability for their specific use case.

By using this software, you acknowledge that you do so entirely at your own risk.