Config details

September 17, 2026 ยท View on GitHub

Different tasks may require extra configs, refer to task templates and tutorial

For configuration changes between releases, see Config changelog.

[extractor]

ConfigDescriptionExampleDefault
db_typesource database typemysqlrequired
extract_typeextraction type; available values depend on db_typesnapshotrequired
urldatabase URL; credentials may be included in the URL or configured separatelymysql://127.0.0.1:3307empty
usernamedatabase connection usernamerootempty
passworddatabase connection passwordpasswordempty
ssl_modeMySQL/PostgreSQL/MSSQL/Redis/MongoDB TLS mode: disable, require, verify_ca, verify_full.verify_canot set
ssl_ca_pathCA certificate path used by MySQL/PostgreSQL/MSSQL/Redis/MongoDB TLS verification/etc/ssl/certs/ca.pemempty
ssl_client_cert_pathClient certificate PEM path (MongoDB: combined certificate/private key PEM); unsupported by MSSQL/etc/ssl/client.crtempty
ssl_client_key_pathClient private key PEM path; empty uses ssl_client_cert_path as a combined PEM; MongoDB requires the combined file/etc/ssl/client.keyempty
max_connectionsmaximum source connection pool size1010
batch_sizenumber of rows extracted per batch; if using chunk splitting, this is also the target chunk size for the source10000[pipeline].buffer_size / effective snapshot parallel_size. If set to 0, uses [pipeline].buffer_size directly
max_rpsoptional source-side rate limit in records per second; 0 disables the limit10000
max_mbpsoptional source-side rate limit in MiB per second; 0 disables the limit1000
app_nameconnection application name, currently used by MongoDBAPE_DTSAPE_DTS
parallel_typesnapshot extraction parallel strategytabletable
parallel_sizesource snapshot worker limit41; legacy fallback: [runtime].tb_parallel_size
partition_colspartition column for data splitting during MySQL/PostgreSQL snapshot migration; only one column per table is supportedjson:[{"db":"db_1","tb":"tb_1","partition_col":"id"},{"db":"db_2","tb":"tb_2","partition_col":"id"}]empty
is_direct_connectionMongoDB driver directConnection optiontruenot set (driver default)
is_clusterRedis Cluster mode for snapshot/CDC/snapshot-and-CDCtruenot set or empty (detect from the connected Redis node)

URL escaping

  • If the username/password contains special characters, the corresponding parts need to be percent-encoded, for example:
create user user1@'%' identified by 'abc%$#?@';
The url should be:
url=mysql://user1:abc%25%24%23%3F%40@127.0.0.1:3307?ssl-mode=disabled

Credentials configured through username and password are percent-encoded and merged into the URL by DTS. If ssl_mode is set, ssl_ca_path is optional unless the selected verification mode and server setup require a CA certificate.

TLS Options

ssl_mode controls the client's TLS requirements and verification of the server:

  • disable: plaintext without TLS.
  • require: encryption without verifying the server certificate or hostname.
  • verify_ca: encryption and server certificate-chain verification against a trusted CA.
  • verify_full: the checks in verify_ca, plus verification of the URL hostname/IP against the certificate SAN.

Client-certificate authentication is a separate server policy. An optional client identity can be configured with any encrypted mode; verify_full does not require one by itself.

ssl_client_cert_path and ssl_client_key_path configure a PEM client identity for MySQL, PostgreSQL and Redis, including metadata and checkpoint connections. MySQL CDC is an exception: its current binlog driver supports only disable/require and no client certificates; unsupported inputs are rejected without downgrading. An empty key path uses the same combined PEM as the certificate path. MongoDB requires a combined PEM in ssl_client_cert_path and an empty ssl_client_key_path. MSSQL does not support client certificates and rejects those inputs.

Driver limits for verify_ca:

  • Redis and PostgreSQL CDC skip hostname checks while retaining certificate-chain verification.
  • The current SQLx rustls driver maps verify_ca but does not recognize rustls's newer NotValidForNameContext error. MySQL/PostgreSQL query connections therefore still reject hostname mismatches in this mode.
  • MongoDB's rustls backend and Tiberius (MSSQL) have no independent hostname opt-out. Their verify_ca currently performs the same server checks as verify_full.

MSSQL supports all four modes. Its two verification modes require ssl_ca_path; explicit task SSL settings override URL, ADO.NET and JDBC encryption/trust settings.

Task Support

This matrix describes the five engines covered by these TLS settings on this branch. The driver-specific limits for verify_ca are described above.

Enginessl_modestructsnapshotCDCchecker
MySQLdisable, requireYesYesYesYes
MySQLverify_ca, verify_fullYesYesNo (binlog driver limitation)Yes
PostgreSQLdisable, require, verify_ca, verify_fullYesYesYesYes
MSSQLdisable, require, verify_ca, verify_fullNoYesNoNo
Redisdisable, require, verify_ca, verify_fullNoYesYes (including snapshot-and-CDC)No
MongoDBdisable, require, verify_ca, verify_fullYesYesYesYes

TLS tests and fixtures share dt-tests/tests/tls/, with sibling engine/topology/version directories (for example, mongo_shard and redis_cluster_6_2). Each directory groups fixtures by task, while the shared test harness varies SSL parameters without copying prepare data for each SSL mode. The runner uses three independent suites: tls (MySQL/PostgreSQL/MSSQL), mongo_to_mongo_tls, and redis_to_redis_tls. The tls suite covers all supported relational cells above. Redis TLS tests exercise snapshot-and-CDC, including cluster encryption; MongoDB TLS tests exercise snapshot/struct/CDC with require, including sharding, plus a verify_full snapshot. MongoDB checker and verified cluster workflows reuse the same client mapping but do not have a dedicated TLS E2E test in this change.

TLS task accounts and servers require trusted client certificates where supported. MySQL CDC uses a separate source/target container pair that requires encryption without client certificates; MSSQL does not support TLS client certificates.

Redis TLS

  • Redis URLs support redis:// and rediss://. Without ssl_mode, redis:// is plaintext and rediss:// uses TLS without server certificate verification.
  • Redis supports disable, require, verify_ca, and verify_full. Both verification modes require ssl_ca_path: verify_ca checks the CA chain, and verify_full additionally checks the URL hostname/IP against the certificate SAN.
  • An explicit ssl_mode overrides the URL scheme and fragment. A DNS URL host is sent as SNI for both verification modes.
  • These rules apply to ordinary Redis command connections and PSYNC replication streams, and are preserved for discovered Redis Cluster node URLs.
  • With Redis Cluster and either verification mode, every node must present a certificate signed by the configured CA. With verify_full, each discovered node hostname or IP must also match its certificate SAN.

MongoDB TLS

  • MongoDB uses the driver's rustls backend. Without ssl_mode, TLS options come from the MongoDB URI, including the driver's mongodb+srv:// defaults.
  • disable turns TLS off. require encrypts the connection without verifying the server certificate or hostname; no CA file is needed.
  • Both verify_ca and verify_full require ssl_ca_path and verify the certificate chain and hostname with rustls. These two modes have the same behavior for MongoDB: the URI hostname or IP must match the server certificate SAN.
  • An explicit ssl_mode replaces the URI TLS options. These settings apply to extractor, sinker, and database checkpoint connections, including replica sets and sharded clusters.

extractor.parallel_type

  • table: allocate snapshot concurrency across tables. With parallel_size=4, up to 4 tables can be extracted at the same time.
  • chunk: allocate snapshot concurrency within a single table by chunk splitting. With parallel_size=4, one table can run up to 4 chunk workers in parallel.
  • When parallel_type=chunk, [extractor].batch_size is also the target chunk size. Chunk boundaries are data-dependent, so the actual row count may differ, but the extractor tries to make each chunk close to batch_size.
  • parallel_size is the effective concurrency limit in both modes.
  • MySQL and PostgreSQL snapshot extractors support both table and chunk.
  • MongoDB snapshot extractors currently support only table; chunk is not supported.
  • Deprecated compatibility: [runtime] tb_parallel_size is kept only as a legacy fallback when [extractor] parallel_size is not set.

Redis source cluster mode

  • [extractor].url can point to any reachable node in the source cluster. DTS discovers all source master nodes through CLUSTER NODES and starts one PSYNC extractor for each master.
  • [extractor].is_cluster is optional. When omitted, DTS connects to the Redis node specified by [extractor].url and detects whether Redis Cluster mode should be used from the node's actual cluster state.
  • Set [extractor].is_cluster=true to force Redis Cluster mode. DTS discovers and syncs the whole source cluster.
  • Set [extractor].is_cluster=false to force single-node Redis mode. DTS runs PSYNC only against the node specified by [extractor].url. This can be used when the source is a Redis Cluster but only one cluster node should be synced.

Mongo source connection mode

  • [extractor].is_direct_connection maps to the MongoDB driver directConnection option.
  • Omit it to let the driver infer the topology from the URL. This is the recommended default for replica sets and sharded clusters.
  • Set it only when you intentionally want to connect directly to a specific MongoDB node. Do not set it to true when connecting through mongos for sharded-cluster CDC or snapshot tasks.

[sinker]

ConfigDescriptionExampleDefault
db_typetarget database typemysqlrequired except for sink_type=dummy
sink_typetarget operation; supported values depend on db_typewritewrite when [sinker] exists; dummy when omitted for standalone checker
urldatabase URL; credentials may be included in the URL or configured separatelymysql://127.0.0.1:3307empty
usernamedatabase connection usernamerootempty
passworddatabase connection passwordpasswordempty
ssl_modeMySQL/PostgreSQL/MSSQL/Redis/MongoDB TLS mode: disable, require, verify_ca, verify_full.verify_canot set
ssl_ca_pathCA certificate path used by MySQL/PostgreSQL/MSSQL/Redis/MongoDB TLS verification/etc/ssl/certs/ca.pemempty
ssl_client_cert_pathClient certificate PEM path (MongoDB: combined certificate/private key PEM); unsupported by MSSQL/etc/ssl/client.crtempty
ssl_client_key_pathClient private key PEM path; empty uses ssl_client_cert_path as a combined PEM; MongoDB requires the combined file/etc/ssl/client.keyempty
max_connectionsmaximum target connection pool size1010
batch_sizerecords written per batch; must be greater than 0200200
max_rpsoptional target-side rate limit in records per second; 0 disables the limit10000
max_mbpsoptional target-side rate limit in MiB per second; 0 disables the limit1000
replacereplace an existing row on insert conflict, for MySQL/PostgreSQL snapshot and CDC tasksfalsetrue
disable_foreign_key_checksdisable foreign-key checks while writing MySQL/PostgreSQLtruetrue
transaction_isolationMySQL/TiDB target transaction isolation: default, read_uncommitted, read_committed, repeatable_read, or serializableread_committeddefault
conflict_policystructure migration conflict policy: interrupt or ignoreinterruptinterrupt
app_nameconnection application name, currently used by MongoDBAPE_DTSAPE_DTS
is_direct_connectionMongoDB driver directConnection optiontruenot set (driver default)
is_clusterRedis Cluster modetruenot set or empty (detect from the connected Redis node)
mongo_require_shard_key_filterfail fast when a MongoDB update/delete/upsert filter cannot contain the complete target shard keytruetrue

Redis target cluster mode

  • [sinker].url can point to any reachable node in the target cluster. DTS discovers all target master nodes through CLUSTER NODES and routes Redis commands to the owning node by key slot.
  • In Redis target cluster mode, DTS creates sinkers according to the target master nodes, instead of limiting the sinker count by [parallelizer].parallel_size.
  • [sinker].is_cluster is optional. When omitted, DTS connects to the Redis node specified by [sinker].url and detects whether Redis Cluster mode should be used from the node's actual cluster state.
  • Set [sinker].is_cluster=true to force Redis Cluster mode when writing to the target cluster.
  • Set [sinker].is_cluster=false to force single-node Redis mode and write only to the node specified by [sinker].url.

Mongo target connection and shard-key mode

  • [sinker].is_direct_connection maps to the MongoDB driver directConnection option. Omit it to let the driver infer the topology from the URL. For sharded targets, connect through mongos and do not set it to true.
  • [sinker].mongo_require_shard_key_filter=true is the default. When the target collection is sharded, DTS checks whether update/delete/upsert filters contain the full target shard key and fails fast if required shard key fields are missing.
  • Keep mongo_require_shard_key_filter=true for normal migrations. Set it to false only when you explicitly accept MongoDB server-side routing behavior, such as a controlled best-effort migration on a compatible MongoDB version.

[checker]

Common row/structure comparison settings. The section is used in these modes:

  • Standalone snapshot/struct/check-log: set [sinker].sink_type=check. The target connection, authentication, TLS, connection limits, and database-specific options are all loaded from [sinker].
  • Inline snapshot: use extract_type=snapshot, [sinker].sink_type=write, and add a [checker] section. Checking runs synchronously after each successful sink operation.
  • Inline CDC: use extract_type=cdc, [sinker].sink_type=write, and enable [checker_cdc].is_enabled=true. Checking runs asynchronously after sink through the CDC checker queue.
ConfigDescriptionExampleDefault
batch_sizemaximum rows processed by one checker query200200
sample_percentpercentage sampled for snapshot/CDC checks25empty (check every row/change)
recheck_countnumber of retries for a temporary inconsistency40
recheck_interval_secsinterval between retries, in seconds50
recheck_queue_sizemaximum pending rows in the retry buffer1000010000
recheck_queue_memory_mbretry-buffer memory limit in MiB256256

Notes:

  • Checker tasks support only [pipeline].pipeline_type=basic.
  • sample_percent accepts 1..=100 and applies only to snapshot checks and inline CDC checks. Standalone snapshot applies sampling during extraction. Inline snapshot/CDC writes every row/change and applies deterministic key-hash sampling before target fetch.
  • Standalone snapshot check supports MySQL, PostgreSQL, and MongoDB targets. Standalone struct check supports MySQL and PostgreSQL.
  • Inline snapshot check supports MySQL, PostgreSQL, and MongoDB write targets.
  • recheck_count and recheck_interval_secs are not used by inline CDC reconciliation.
  • When either retry-buffer limit is reached, the checker does not drop the result; newly found inconsistencies skip retry and are finalized immediately.

Standalone target example

[extractor]
db_type=mysql
extract_type=snapshot
url=mysql://source-host:3306

[sinker]
db_type=mysql
sink_type=check
url=mysql://target-host:3306
username=root
password=target-password
max_connections=8

[checker]
batch_size=200
sample_percent=25
recheck_count=4
recheck_interval_secs=5
recheck_queue_size=10000
recheck_queue_memory_mb=256

[checker_output]

Check-result output configuration. If this section is omitted, results are written as local logs under runtime.log_dir/check.

ConfigDescriptionExampleDefault
output_typeresult destination: logs or s3logslogs
output_full_rowinclude complete source/target rows in difference logsfalsefalse
output_revise_sqlgenerate repair statements in sql.logtruefalse
revise_match_full_rowuse the complete row in generated repair predicatesfalsefalse
check_log_dirlocal check-log directory/tmp/checkempty (use runtime.log_dir/check)
check_log_file_sizeper-file size limit for diff.log, miss.log, and sql.log100mb100mb
check_log_max_rowsmaximum rows in diff.log/miss.log10001000
s3_bucketS3 bucket; required for output_type=s3my-bucket-
s3_access_key_idS3 access keyAKIA...empty
s3_secret_access_keyS3 secret key****empty
s3_regionS3 regionus-east-1empty
s3_endpointcustom S3 endpointhttps://...empty
s3_root_dirlocal/mounted root used by the S3 helper/tmp/checkempty
s3_root_urlroot URL used by the S3 helpers3://bucketempty
s3_key_prefixkey prefix for uploaded check logstask1/checkempty

output_type=s3 is supported for standalone snapshot check and inline CDC check. S3 output still uses the configured local rolling-log directory and limits before upload. Structure check, check-log review, and inline snapshot check support output_type=logs only.

[checker_cdc]

CDC-only asynchronous checker settings.

ConfigDescriptionExampleDefault
is_enabledenable inline CDC checktruefalse
queue_sizepending CDC checker batches200200
check_log_interval_secsperiodic CDC check-result output interval in seconds3030

Inline CDC check additionally requires:

  • [extractor].extract_type=cdc
  • [sinker].sink_type=write with a MySQL/PostgreSQL target
  • [parallelizer].parallel_type=rdb_merge
  • [resumer].resume_type=from_target or from_db

The CDC checker queue is deliberately decoupled from the migration pipeline. When full, it evicts the oldest pending batch instead of blocking writes. Checker processing/output failures are logged without failing the main CDC write path. Snapshot and structure checks do not use this queue.

[filter]

ConfigDescriptionExampleDefault
do_dbsdatabases to be synced, takes union with do_tbsdb_1,db_2*,db*&#-
ignore_dbsdatabases to be filtered, takes union with ignore_tbsdb_1,db_2*,db*&#-
do_tbstables to be synced, takes union with do_dbsdb_1.tb_1,db_2*.tb_2*,db*&#.tb*&#-
ignore_tbstables to be filtered, takes union with ignore_dbsdb_1.tb_1,db_2*.tb_2*,db*&#.tb*&#-
ignore_colstable columns to be filteredjson:[{"db":"db_1","tb":"tb_1","ignore_cols":["f_2","f_3"]},{"db":"db_2","tb":"tb_2","ignore_cols":["f_3"]}]-
do_eventsevents to be syncedinsert,update,delete*
do_ddlsddls to be synced, for mysql cdc taskscreate_database,drop_database,alter_database,create_table,drop_table,truncate_table,rename_table,alter_table,create_index,drop_index-
do_dclsDCL statements to be synced, for supported structure taskscreate_user,grant-
do_structuresstructures to be migrated in structure migration tasksmysql/pg: database,table,constraint,sequence,comment,index; mongo: collection,shardkey*
ignore_cmdscommands to be filtered, for redis cdc tasksflushall,flushdb-
where_conditionswhere conditions for the source SELECT SQL during snapshot migrationjson:[{"db":"db_1","tb":"tb_1","condition":"f_0 > 1"},{"db":"db_2","tb":"tb_2","condition":"f_0 > 1 AND f_1 < 9"}]-

Values

  • All configurations support multiple items, which are separated by ",". Example: do_dbs=db_1,db_2.
  • Set to * to match all. Example: do_dbs=*.
  • Keep empty to match nothing. Example: ignore_dbs=.
  • ignore_cols and where_conditions are in JSON format and must start with json:.
  • do_events takes one or more values from insert, update, and delete.
  • do_dcls takes one or more values from create_user, alter_user, create_role, drop_user, drop_role, grant, revoke, and set_role.
  • do_structures takes structure object types. For MySQL/PostgreSQL, common values include database, table, constraint, sequence, comment, and index. For MongoDB, supported values are collection, shardkey. MongoDB does not use a separate database structure type; databases are created implicitly by creating collections. shardkey copies source sharding definitions for sharded collections and runs only when the target is connected through mongos.

Priority

  • ignore_tbs + ignore_dbs > do_tbs + do_dbs.
  • If a table matches both ignore configs and do configs, the table will be filtered.
  • If both do_tbs and do_dbs are configured, the filter is the union of both. If both ignore_tbs and ignore_dbs are configured, the filter is the union of both.

Wildcard

WildcardDescription
*Matches multiple characters
?Matches 0 or 1 characters

Used in: do_dbs, ignore_dbs, do_tbs, and ignore_tbs.

Escapes

DatabaseBeforeAfter
mysqldb*&#`db*&#`
mysqldb*&#.tb*$#`db*&#`.`tb*$#`
pgdb*&#"db*&#"
pgdb*&#.tb*$#"db*&#"."tb*$#"

Names should be enclosed in escape characters if there are special characters.

Used in: do_dbs, ignore_dbs, do_tbs and ignore_tbs.

[router]

ConfigDescriptionExampleDefault
db_mapdatabase mappingdb_1:dst_db_1,db_2:dst_db_2-
tb_maptable mappingdb_1.tb_1:dst_db_1.dst_tb_1,db_1.tb_2:dst_db_1.dst_tb_2-
col_mapcolumn mappingjson:[{"db":"db_1","tb":"tb_1","col_map":{"f_0":"dst_f_0","f_1":"dst_f_1"}}]-
topic_maptable -> kafka topic mapping, for mysql/pg -> kafka tasks. required*.*:default_topic,test_db_2.*:topic2,test_db_2.tb_1:topic3-

Values

  • A mapping rule consists of the source and target, which are separated by ":".
  • All configurations support multiple items, which are separated by ",". Example: db_map=db_1:dst_db_1,db_2:dst_db_2.
  • col_map value is in JSON format and must start with json:.
  • If not set, data will be routed to the same databases/tables/columns with the source database.

Priority

  • tb_map > db_map.
  • col_map only works for column mapping. If a table needs database + table + column mapping, tb_map/db_map must be set.
  • topic_map: test_db_2.tb_1:topic3 > test_db_2.*:topic2 > *.*:default_topic.

Wildcard

Not supported.

Escapes

Same with [filter].

[pipeline]

ConfigDescriptionExampleDefault
buffer_sizemax cached records in memory1600016000
buffer_memory_mb[optional] memory limit for buffer, if reached, new records will be blocked even if buffer_size is not reached, 0 means not set2000
checkpoint_interval_secsinterval to flush logs/statistics/position1010
batch_sink_interval_secsmaximum interval before flushing a non-empty sink batch10
counter_time_window_secstime window for monitor counters10same with [pipeline] checkpoint_interval_secs
counter_max_sub_countmaximum number of sub-counters10001000
pipeline_typepipeline implementation; only basic is supportedbasicbasic

[parallelizer]

ConfigDescriptionExampleDefault
parallel_typeparallel typesnapshotserial
parallel_sizethreads for parallel syncing81
rebalance_strategysnapshot chunk rebalance strategy used during sink writesnonenone
rebalance_costcost metric used to measure partition sizerowsrows
rebalance_max_partitions_per_sinkermax split partitions per effective sinker22
rebalance_min_partition_rowsminimum rows kept in each split snapshot insert partition200[sinker].batch_size
rebalance_split_skew_ratioskew threshold used by the auto_split strategy1.01.0

parallel_type

TypeStrategyUsageAdvantagesDisadvantages
snapshotRecords in cache are divided into [parallel_size] partitions, and each partition will be synced in batches in a separate thread.snapshot tasks for mysql/pg/mongofast
serialSingle thread, one by one.allslow
rdb_mergeMerge row changes in cache into write-friendly insert + delete batches, then divide them into [parallel_size] partitions for parallel syncing. It is used by MySQL/PG CDC, check, review, and revise flows.mysql/pg CDC, check, review, revisefasteventual consistency
mongoMongo version of merge parallelization, also used by standalone MongoDB check and review flows.mongo CDC, check, review
redisSingle thread, batch/serial writing(determined by [sinker] batch_size)snapshot/CDC tasks for redis

snapshot chunk rebalance

When [parallelizer].parallel_type=snapshot, snapshot parallelizer uses chunk partitioner to rebalance the downstream write queue. It is mainly for snapshot write tasks and reduces sink-side long tails. It does not change source-side extractor concurrency and does not rewrite checkpoint chunk ids.

Default behavior:

[parallelizer]
parallel_type=snapshot
parallel_size=8
rebalance_strategy=none
rebalance_cost=rows

The default rebalance_strategy=none keeps logical chunk order after grouping and does not add sink-side sorting or splitting. If sink-side long tails are obvious, use rebalance_strategy=auto_split. Use table_min_rows or table_even for rows-only table-level partitioning. Use the default rebalance_cost=rows when row width is similar. If rows contain large JSON, LOB, or wide strings, use rebalance_cost=bytes. If the target has high request overhead, or you do not want to split logical chunks, use rebalance_strategy=chunk_largest_first.

For scenario-based tuning, see Snapshot Chunk Partitioner Rebalance.

[runtime]

ConfigDescriptionExampleDefault
log_levellevelinfo/warn/error/debug/traceinfo
log4rs_filelog4rs config file./log4rs.yaml./log4rs.yaml
log_diroutput dir./logs./logs
check_result_stdout_onlyoutput only check result logs to stdouttrue/falsefalse

Note that the log files contain progress information for the task, which can be used for task resuming at breakpoint. Therefore, if you have multiple tasks, please set up separate log directories for each task.

[global]

ConfigDescriptionExampleDefault
task_idUnique task identifiercdc_task_1

In some scenarios, task_id is used to distinguish task uniqueness, such as when using resumer from database. By default, it will be automatically generated based on key configuration information.

[resumer]

ConfigDescriptionExampleDefault
resume_typedummy, from_log, from_target, or from_dbfrom_targetdummy
log_dirlog directory used by from_log./logs[runtime].log_dir
config_fileoptional resume config file used by from_log./resume.configempty
urldatabase URL used by from_dbmysql://127.0.0.1:3306required for from_db
db_typedatabase type used by from_dbmysqlrequired for from_db
usernamedatabase username used by from_dbrootempty
passworddatabase password used by from_dbpasswordempty
ssl_modeMySQL/PostgreSQL/Redis/MongoDB TLS mode used by from_db: disable, require, verify_ca, verify_full.verify_canot set
ssl_ca_pathCA certificate path used by MySQL/PostgreSQL/Redis/MongoDB from_db TLS verification/etc/ssl/certs/ca.pemempty
ssl_client_cert_pathClient certificate PEM path (MongoDB: combined certificate/private key PEM); unsupported by MSSQL/etc/ssl/client.crtempty
ssl_client_key_pathClient private key PEM path; empty uses ssl_client_cert_path as a combined PEM; MongoDB requires the combined file/etc/ssl/client.keyempty
is_direct_connectionMongoDB driver directConnection option used by from_dbtruenot set
table_full_nametarget table used to store resume state for from_db or from_targetapecloud_metadata.apedts_task_positionempty
max_connectionsmaximum resumer connection pool size55

For details, please refer to the resumer documentation: resuming at breakpoint.

resume_type=from_target reuses the parsed sinker target. For a standalone checker with a dummy or omitted sinker, it reuses the checker target. The legacy keys resume_from_log, resume_log_dir, and resume_config_file are rejected; migrate them to resume_type=from_log, log_dir, and config_file.

[tracing]

ConfigDescriptionExampleDefault
task_summary_modetrace aggregation mode: task or markermarkermarker
output_formattrace output format: plain or jsonjsonplain

The runtime trace summary is dumped periodically (every pipeline.checkpoint_interval_secs) to the runtime trace log, including a final dump on task shutdown, so long-running CDC tasks get continuous diagnostics. When both metrics and tracing features are enabled, per-marker task counters and globally aggregated per-wait-point counters (runtime_trace_*) are also exposed on the Prometheus /metrics endpoint. In task summary mode, completed task details are emitted once on the next dump and then released; marker summaries and Prometheus counters remain cumulative.

[metacenter]

This optional section is used by the MySQL dbengine metadata-center mode.

ConfigDescriptionExampleDefault
typemetadata-center type: basic or dbenginedbenginebasic
urlmetadata database URL; required for MySQL dbengine modemysql://127.0.0.1:3306required
usernamemetadata database usernamerootempty
passwordmetadata database passwordpasswordempty
ssl_modeMySQL TLS mode: disable, require, verify_ca, verify_full.verify_fullnot set
ssl_ca_pathCA certificate path/etc/ssl/certs/ca.pemempty
ssl_client_cert_pathClient certificate PEM path (MongoDB: combined certificate/private key PEM); unsupported by MSSQL/etc/ssl/client.crtempty
ssl_client_key_pathClient private key PEM path; empty uses ssl_client_cert_path as a combined PEM; MongoDB requires the combined file/etc/ssl/client.keyempty
ddl_conflict_policyDDL conflict policy: interrupt or ignoreinterruptinterrupt

The metadata-center URL must differ from both the extractor URL and the effective destination URL.

[data_marker]

If this section is present, the required topology marker configuration is loaded.

ConfigDescriptionDefault
topo_nametopology namerequired
topo_nodestopology node listempty
src_nodesource noderequired
dst_nodedestination noderequired
do_nodesincluded nodesrequired
ignore_nodesexcluded nodesempty
markermarker valuerequired

[processor]

ConfigDescriptionDefault
lua_code_fileLua processor source file loaded by DTSempty

[metrics]

This section is available only when DTS is built with the metrics feature.

ConfigDescriptionExampleDefault
http_hostmetrics HTTP bind address0.0.0.00.0.0.0
http_portmetrics HTTP port90909090
workersmetrics HTTP worker count22
labelscomma-separated key=value metric labelsenv=prod,az=aempty