DuckDB Snowflake Extension

A DuckDB extension for querying Snowflake from DuckDB via the Apache Arrow ADBC Snowflake driver. Data flows between the two systems as Arrow record batches, so result sets stay columnar end-to-end.

This extension works with DuckDB v1.5.5.

Quick Start

Get the Latest Extension Build (v1.5.5)

Install DuckDB 1.5.5 (or newer) and then install the Snowflake extension directly from the community repository:

INSTALL snowflake FROM community;
LOAD snowflake;

Confirm your DuckDB version meets the requirement:

PRAGMA version;

Installation

-- Install and load the extension
INSTALL snowflake FROM community;
LOAD snowflake;

Basic Usage

-- 1. Create a Snowflake profile
CREATE SECRET my_snowflake_secret (
    TYPE snowflake,
    ACCOUNT 'your_account_identifier',
    USER 'your_username',
    PASSWORD 'your_password',
    DATABASE 'your_database',      -- Optional: default database
    WAREHOUSE 'your_warehouse'     -- Optional: default warehouse
);

-- 2.1 Query Snowflake data using pass through query
SELECT * FROM snowflake_query(
    'SELECT * FROM customers WHERE state = ''CA''',
    'my_snowflake_secret'
);

-- 2.2 Query Snowflake data using local duckdb SQL syntax
ATTACH '' AS snow_db (TYPE snowflake, SECRET my_snowflake_secret, READ_ONLY);

SELECT * FROM snow_db.schema.customers WHERE state = 'CA';

Documentation

For End Users

If you want to install and use the extension, continue reading this README for:

• Installation instructions
• Configuration setup
• Function reference
• Usage examples
• Troubleshooting guide

Overview

The extension exposes Snowflake to DuckDB as a remote data source. You can either pass SQL straight through to Snowflake with snowflake_query(), or ATTACH a Snowflake database and query it with DuckDB SQL.

Capabilities

• Run SQL against Snowflake from DuckDB, either as passthrough (snowflake_query) or against an attached Snowflake database
• Arrow-native transport over the ADBC Snowflake driver
• Authentication via password, OAuth, key pair (with passphrase), external browser SSO, Okta, and MFA
• Credentials stored through DuckDB's secrets system
• Read-only storage extension for ATTACH

Installation

-- Install and load the extension
INSTALL snowflake FROM community;
LOAD snowflake;

Note: You still need to download the ADBC driver separately (see ADBC Driver Setup below).

For Developers: If you need to build the extension from source, see BUILD.md.

ADBC Driver Setup

The Snowflake extension requires the ADBC Snowflake driver to communicate with Snowflake servers. The driver is released by the ADBC Driver Foundry (release line go/vX.Y.Z); the older apache/arrow-adbc wheels are deprecated and lack native GEOMETRY/GEOGRAPHY support.

Quick Install (Recommended)

Use the automated installer script to download and install the correct ADBC driver for your platform:

Linux / macOS / WSL:

# Using curl
curl -sSL https://raw.githubusercontent.com/iqea-ai/duckdb-snowflake/main/scripts/install-adbc-driver.sh | sh

# Or using wget
wget -qO- https://raw.githubusercontent.com/iqea-ai/duckdb-snowflake/main/scripts/install-adbc-driver.sh | sh

Windows (PowerShell):

# Download and run the installer
iwr -useb https://raw.githubusercontent.com/iqea-ai/duckdb-snowflake/main/scripts/install-adbc-driver.bat -OutFile install-adbc-driver.bat
.\install-adbc-driver.bat

The installer will:

• Detect your DuckDB version and platform automatically
• Download the correct ADBC driver version
• Install it to the appropriate directory:
  • Linux/macOS: ~/.duckdb/extensions/<version>/<platform>/
  • Windows: %USERPROFILE%\.duckdb\extensions\<version>\windows_amd64\
• Verify the installation

Alternative: dbc

dbc can install the driver too:

dbc install snowflake

That is all: the extension discovers dbc-installed drivers through the standard ADBC driver manifests — snowflake.toml in the ADBC_DRIVER_PATH directories, the active conda prefix, and the per-OS manifest directories on Linux and macOS; the SOFTWARE\ADBC\Drivers\snowflake registry keys on Windows. No environment variable is needed. SNOWFLAKE_ADBC_DRIVER_PATH still works as an explicit override and takes precedence over manifest discovery.

Manual Installation (advanced)

If you prefer to install manually, download and install the appropriate driver for your platform:

Supported Platforms

Platform	DuckDB Directory	Release Asset	Status
Linux x86_64	linux_amd64	snowflake_linux_amd64_v1.11.0.tar.gz	Supported
Linux ARM64	linux_arm64	snowflake_linux_arm64_v1.11.0.tar.gz	Supported
macOS ARM64	osx_arm64	snowflake_macos_arm64_v1.11.0.tar.gz	Supported
macOS x86_64	osx_amd64	-	Build from source
Windows x86_64	windows_amd64	snowflake_windows_amd64_v1.11.0.tar.gz	Supported
Windows ARM64	-	-	Not available

Note: The ADBC Driver Foundry ships no macOS x86_64 (Intel) or Windows ARM64 builds. On those platforms, build the driver from source and set SNOWFLAKE_ADBC_DRIVER_PATH to the result.

The extension looks for the driver under the fixed name libadbc_driver_snowflake.so on every platform, so rename the macOS .dylib / Windows .dll accordingly.

Generic Installation Steps - non windows

Replace <PLATFORM> with your platform directory and <ASSET> with the release asset from the table above:

# 1. Download the release tarball
curl -L -O https://github.com/adbc-drivers/snowflake/releases/download/go/v1.11.0/<ASSET>

# 2. Extract the driver library (libadbc_driver_snowflake.so on Linux, .dylib on macOS)
tar xzf <ASSET>

# 3. Move to DuckDB extensions directory under the fixed name (DuckDB v1.5.5)
mkdir -p ~/.duckdb/extensions/v1.5.5/<PLATFORM>
mv libadbc_driver_snowflake.* ~/.duckdb/extensions/v1.5.5/<PLATFORM>/libadbc_driver_snowflake.so

# 4. Clean up
rm <ASSET>

Example: Linux x86_64 Installation

# Download
curl -L -O https://github.com/adbc-drivers/snowflake/releases/download/go/v1.11.0/snowflake_linux_amd64_v1.11.0.tar.gz

# Extract
tar xzf snowflake_linux_amd64_v1.11.0.tar.gz

# Install (DuckDB v1.5.5)
mkdir -p ~/.duckdb/extensions/v1.5.5/linux_amd64
mv libadbc_driver_snowflake.so ~/.duckdb/extensions/v1.5.5/linux_amd64/

# Clean up
rm snowflake_linux_amd64_v1.11.0.tar.gz

Installation Steps - Windows

# Download and extract (tar.exe ships with Windows 10+)
curl -L -O https://github.com/adbc-drivers/snowflake/releases/download/go/v1.11.0/snowflake_windows_amd64_v1.11.0.tar.gz
tar -xzf snowflake_windows_amd64_v1.11.0.tar.gz
del snowflake_windows_amd64_v1.11.0.tar.gz

# Place in DuckDB extensions directory under the fixed name (DuckDB v1.5.5)
mkdir C:\Users\%USERNAME%\.duckdb\extensions\v1.5.5\windows_amd64
move libadbc_driver_snowflake.dll C:\Users\%USERNAME%\.duckdb\extensions\v1.5.5\windows_amd64\libadbc_driver_snowflake.so

Verification

Test that the driver is found:

LOAD snowflake;
SELECT snowflake_version();
-- Should return: "Snowflake Extension v0.5.0"

Configuration

Authentication Methods

The DuckDB Snowflake extension supports multiple authentication methods:

• Password: Standard username/password (simple setup, development) - Tested
• OAuth 2.0: Token-based authentication (recommended for Okta/Auth0, headless environments) - Known Issues
• Key Pair: RSA key-based authentication (production, highest security, recommended) - Tested
• External Browser (SAML 2.0): SSO with any SAML provider (Okta, Auth0, AD FS, Azure AD) - Tested
• Okta: Native Okta integration (Okta IdP only) - Implemented
• MFA: Multi-factor authentication (interactive sessions only, not for programmatic use)

Note on account values:

• Use your Snowflake account identifier (e.g., myaccount or xy12345.us-east-1) for ACCOUNT in all secrets and connection strings (Password, Key Pair, OAuth, MFA, EXT_BROWSER, OKTA).
• Use the full Snowflake URL (https://<account>.snowflakecomputing.com) only in IdP configuration such as OAuth/SAML audience values.

Quick Example (Password Auth):

CREATE SECRET my_snowflake_secret (
    TYPE snowflake,
    ACCOUNT 'myaccount',
    USER 'myusername',
    PASSWORD 'mypassword',
    DATABASE 'mydatabase',
    WAREHOUSE 'mywarehouse'
);

For detailed setup instructions, configuration examples, and IdP integration guides, see Authentication Documentation

For step-by-step setup of OAuth, Key Pair, Okta, and MFA authentication, see Authentication Setup Guide

Setting Up Snowflake Credentials

Using Secrets (Recommended)

Create a named profile to securely store your Snowflake credentials:

Creating a Secret:

-- Secret with optional parameters (password authentication example)
CREATE SECRET my_snowflake_secret (
    TYPE snowflake,
    ACCOUNT 'myaccountidentifier',
    USER 'myusername',
    PASSWORD 'mypassword',
    DATABASE 'mydatabase',          -- Optional: default database
    WAREHOUSE 'mywarehouse',        -- Optional: default warehouse
    SCHEMA 'myschema'               -- Optional: default schema
);

Listing Secrets:

-- View all secrets
SELECT * FROM duckdb_secrets();

-- View only Snowflake secrets
SELECT * FROM duckdb_secrets() WHERE type = 'snowflake';

Updating a Secret:

-- Drop and recreate to update
DROP SECRET my_snowflake_secret;

CREATE SECRET ...

Deleting a Secret:

-- Remove a secret
DROP SECRET my_snowflake_secret;

Functions Reference

Scalar Functions

snowflake_version()

Returns the extension version information.

SELECT snowflake_version();
-- Returns: "Snowflake Extension v0.5.0"

Table Functions

snowflake_query(query, profile)

Executes SQL queries against Snowflake databases.

SELECT * FROM snowflake_query(
    'SELECT * FROM customers WHERE state = ''CA''',
    'my_snowflake_secret'
);

Storage Extension

ATTACH with Snowflake Storage Extension

Attaches a Snowflake database as a read-only storage extension.

-- Using secret
ATTACH '' AS snow_db (TYPE snowflake, SECRET my_snowflake_secret, READ_ONLY);

-- Using connection string
ATTACH 'account=myaccount;user=myuser;password=mypass;database=mydb;warehouse=mywh'
AS snow_db (TYPE snowflake, READ_ONLY);

Usage Examples

Basic Queries

-- Test connection
SELECT * FROM snowflake_query('SELECT 1', 'my_snowflake_secret');

-- Query with attached database
ATTACH '' AS snow (TYPE snowflake, SECRET my_snowflake_secret, READ_ONLY);
SELECT * FROM snow.public.customers LIMIT 10;

Complex Analytics

-- Run the analytics on the Snowflake side, then return the result set
SELECT * FROM snowflake_query(
    '
    WITH monthly_sales AS (
        SELECT
            DATE_TRUNC(''month'', order_date) as month,
            SUM(amount) as total_sales
        FROM orders
        GROUP BY 1
    )
    SELECT
        month,
        total_sales,
        LAG(total_sales) OVER (ORDER BY month) as prev_month_sales
    FROM monthly_sales
    ',
    'my_snowflake_secret'
);

Cross-Database Analytics

-- Combine Snowflake data with local DuckDB tables
SELECT
    sf.product_id,
    sf.sales_amount,
    local.discount_rate,
    sf.sales_amount * (1 - local.discount_rate) as discounted_amount
FROM snowflake_query(
    'SELECT product_id, SUM(amount) as sales_amount FROM sales GROUP BY product_id',
    'my_snowflake_secret'
) sf
JOIN local_discounts local ON sf.product_id = local.product_id;

Data Export Workflows

-- Export large dataset efficiently
CREATE TABLE local_fact_sales AS
SELECT * FROM snowflake_query(
    'SELECT * FROM fact_sales WHERE year >= 2020',
    'my_snowflake_secret'
);

-- Create Parquet files from Snowflake data
COPY (
    SELECT * FROM snowflake_query(
        'SELECT * FROM large_table',
        'my_snowflake_secret'
    )
) TO 'output.parquet' (FORMAT PARQUET);

Filter & Projection Pushdown

The extension can optimize queries by pushing filters and column selections to Snowflake. Pushdown is disabled by default and must be explicitly enabled.

Requires DuckDB 1.5.5 build of the Snowflake extension. Earlier versions of the extension do not include the pushdown planner improvements referenced below.

Enabling Pushdown

Add enable_pushdown true to the ATTACH statement:

-- Pushdown DISABLED (default)
ATTACH '' AS snow (TYPE snowflake, SECRET my_secret, READ_ONLY);

-- Pushdown ENABLED
ATTACH '' AS snow (TYPE snowflake, SECRET my_secret, READ_ONLY, enable_pushdown true);

How Pushdown Works (when enabled)

ATTACH '' AS snow (TYPE snowflake, SECRET my_secret, READ_ONLY, enable_pushdown true);

-- Simple filter and projection
SELECT id, name FROM snow.schema.customers WHERE age > 25;
-- Snowflake executes: SELECT "id", "name" FROM ... WHERE "age" > 25

-- Complex filters with IN and OR
SELECT * FROM snow.schema.orders
WHERE status IN ('PENDING', 'PROCESSING')
   OR (order_date >= '2024-01-01' AND order_date < '2024-02-01');
-- All filters pushed to Snowflake

-- Join queries with filter pushdown
SELECT c.id, c.name, n.country_name
FROM snow.schema.customers c
JOIN snow.schema.nations n ON c.nation_id = n.id
WHERE n.country_name = 'USA' AND c.id <= 1000;
-- Static filters pushed to both tables

Supported Pushdown (current):

• Comparison filters: =, !=, <, >, <=, >=, IS NULL, IS NOT NULL
• Logical operators: AND, OR
• IN clauses: col IN (value1, value2, ...) (converted to multiple OR conditions)

Not yet implemented: join-filter pushdown and projection pushdown. These optimizations are on the roadmap but disabled in the current build.

Important: DuckDB's Optimizer Controls Pushdown

When pushdown is enabled, DuckDB's query optimizer decides which filters to push down based on performance estimates. Not all filters in your query will necessarily be pushed to Snowflake:

-- With pushdown enabled
ATTACH '' AS snow (TYPE snowflake, SECRET my_secret, READ_ONLY, enable_pushdown true);

-- Query with multiple filters
SELECT * FROM snow.schema.customer
WHERE C_CUSTKEY > 100000 AND C_PHONE IS NOT NULL;

-- DuckDB may push down: WHERE C_CUSTKEY > 100000
-- DuckDB may apply locally: C_PHONE IS NOT NULL (cheap to evaluate after filtering)

This is optimal behavior. DuckDB keeps certain filters local when:

• The filter is very cheap to evaluate (e.g., IS NOT NULL)
• A prior filter already reduces the dataset significantly
• Local evaluation is faster than remote execution

The extension supports all standard comparison and null-check filters. DuckDB's optimizer will use them when it determines pushdown improves performance.

snowflake_query (Pushdown Disabled)

-- User-provided SQL is executed as-is, no modification
SELECT * FROM snowflake_query('SELECT * FROM customers WHERE age > 25', 'my_secret');

Use snowflake_query() when you need full control over the SQL sent to Snowflake.

Limitations

• Read-only access: All Snowflake operations are read-only
• Function calls in filters: Expressions like WHERE UPPER(name) = 'FOO' not pushed down
• LIMIT pushdown: Not supported for ATTACH - LIMIT is applied after fetching data from Snowflake

Working with LIMIT

When using ATTACH, LIMIT clauses are applied locally by DuckDB after fetching all data:

-- LIMIT applied locally (fetches all rows, then limits)
SELECT * FROM snow.schema.customer LIMIT 100;

For efficient row sampling, use snowflake_query() with Snowflake's native sampling:

-- Option 1: LIMIT pushed to Snowflake
SELECT * FROM snowflake_query(
    'SELECT * FROM customer LIMIT 100',
    'my_secret'
);

-- Option 2: Snowflake SAMPLE clause (recommended for large tables)
SELECT * FROM snowflake_query(
    'SELECT * FROM customer SAMPLE (1000 ROWS)',
    'my_secret'
);

-- Option 3: Percentage-based sampling
SELECT * FROM snowflake_query(
    'SELECT * FROM customer SAMPLE (1)',  -- 1% of rows
    'my_secret'
);

Troubleshooting

If you get "ADBC driver not supported" error:

• Verify the driver file is in the correct location
• Check file permissions (should be executable)
• Ensure you downloaded the correct architecture for your platform

If you get "Driver not found" debug messages:

• The extension will search multiple locations automatically
• Check the debug output to see which paths it's checking
• Place the driver in one of the searched locations

Debugging Tips

-- Test connection with a simple query
SELECT * FROM snowflake_query('SELECT 1', 'my_snowflake_secret');

-- Check profile configuration
SELECT * FROM duckdb_secrets() WHERE type = 'snowflake';

-- Verify warehouse status
SELECT * FROM snowflake_query(
    'SHOW WAREHOUSES',
    'my_snowflake_secret'
);

Support

For issues or questions:

• Check the GitHub repository. Raise any feature requests/issues under issues
• Review Snowflake ADBC driver documentation (https://arrow.apache.org/adbc/main/driver/snowflake.html)
• Ensure you have the latest version of the extension

For Developers

If you want to build the extension from source or contribute to development, see BUILD.md for detailed build instructions and development guidelines.