Apache Pulsar Connectors

This repository contains the IO connectors for Apache Pulsar.

Connectors are packaged as NAR files and can be deployed into any Pulsar installation by placing them in the connectors directory or mounting them into the apachepulsar/pulsar Docker image.

Available Connectors

Sources

Connector	Description
Canal	MySQL binlog via Alibaba Canal
Debezium (MySQL, PostgreSQL, MongoDB, MSSQL, Oracle)	CDC via Debezium
DynamoDB	Amazon DynamoDB Streams
File	Local filesystem
Kafka	Apache Kafka
Kinesis	Amazon Kinesis Data Streams
MongoDB	MongoDB change streams
NSQ	NSQ messaging
RabbitMQ	RabbitMQ / AMQP

Sinks

Connector	Description
Aerospike	Aerospike database
Alluxio	Alluxio distributed storage
Azure Data Explorer	Azure Data Explorer (Kusto)
Cassandra	Apache Cassandra
Elasticsearch / OpenSearch	Elasticsearch and OpenSearch
HBase	Apache HBase
HDFS3	Hadoop HDFS
HTTP	HTTP endpoint
InfluxDB	InfluxDB time-series database
JDBC (PostgreSQL, MariaDB, ClickHouse, SQLite, OpenMLDB)	JDBC databases
Kafka	Apache Kafka
Kinesis	Amazon Kinesis Data Streams
MongoDB	MongoDB
MQTT	MQTT broker
Redis	Redis
Solr	Apache Solr

Adaptor

Connector	Description
Kafka Connect Adaptor	Run Kafka Connect connectors on Pulsar

Prerequisites

• JDK 17 or later — e.g. Eclipse Temurin or Amazon Corretto

Note: This project includes a Gradle Wrapper so no separate Gradle installation is needed. Use ./gradlew on Linux/macOS and gradlew.bat on Windows.

Building

Compile and assemble all modules:

./gradlew assemble

NAR files are produced under each connector's build/libs/ directory.

Build a specific connector:

./gradlew :elastic-search:assemble

Build the distribution package containing all connector NARs:

./gradlew :distribution:pulsar-io-distribution:assemble

Check source code license headers:

./gradlew rat spotlessCheck

Auto-fix license headers:

./gradlew spotlessApply

Running Tests

# All unit tests
./gradlew test

# Specific connector
./gradlew :elastic-search:test

# Specific test class
./gradlew :elastic-search:test --tests "ElasticSearchSinkTests"

Using Connectors

With Docker

Mount connector NARs into the Pulsar container:

docker run -v /path/to/connectors:/pulsar/connectors apachepulsar/pulsar

Manual Installation

Copy NAR files to the connectors/ directory of your Pulsar installation:

cp elastic-search/build/libs/pulsar-io-elastic-search-*.nar $PULSAR_HOME/connectors/

Versioning

This repository follows its own release cadence, independent from Apache Pulsar releases. All connectors are released together as a single release. The initial release version matches the Pulsar version at the time of the split.

Each release specifies which Pulsar versions it is compatible with. The pulsar-io-core API has been stable for years, so connectors are generally compatible across Pulsar 4.x releases.

Contributing

Contributions are welcome! Please see CONTRIBUTING.md in the main Pulsar repository for guidelines.

License

Licensed under the Apache License, Version 2.0. See LICENSE for details.