A map that guides what and how contributors can contribute

Table of Contents

• TiDB is an open-source distributed HTAP database compatible with the MySQL protocol
• TiKV, distributed transactional key-value database
• PD Placement driver for TiKV
• TiKV Clients
• Libraries depended by TiKV
• Ecosystem Tools: DM Data Migration Platform
• Ecosystem Tools - Binlog : A tool used to collect and merge tidb's binlog for real-time data backup and synchronization
• Ecosystem Tools - Lightning: A high-speed data import tool for TiDB
• Ecosystem Tools - BR: A command-line tool for distributed backup and restoration of the TiDB cluster data
• TiDB on K8S/Docker : Creates and manages TiDB clusters running in Kubernetes
• Deployment Tools - tidb-ansible: A tool to capture data change of TiDB cluster
• Chaos-Mesh: A Chaos Engineering Platform for Kubernetes
• Documentation
• TiDB User Group(CN)
• PingCAP University(CN)
• TiDB Developer Group

TiDB is an open-source distributed HTAP database compatible with the MySQL protocol

Contribution tutorial:

• Contribute to TiDB
• Good First Issues

Module	Description	Code Directory	Required Skills	Learning Materials	Source Reading
Parser	A Yacc generated parser that is highly compatible with MySQL syntax.	parser	Golang, Yacc	Bison manual Compilers: Principles, Techniques, and Tools	TiDB SQL Parser Implementation(CN)
DDL	This module is used to realize that when different TiDBs in a cluster transition to the new schema, all TiDBs can access and update all data online.	ddl	Golang	Online, Asynchronous Schema Change in F1 Deep analysis of Google F1's schema change algorithm(CN) TiDB DDL architecture	TiDB DDL Implementation(CN)
MySQL Protocol		server
Transaction	A transaction symbolizes a unit of work performed within TiDB. The transaction module in TiDB plays the role of coordinator of distributed transactions.	session kv tikv/client-go	Golang, ACID, 2PC, Percolator transaction model	A Critique of ANSI SQL Isolation Levels The Art of Database Transaction Processing: Transaction Management and Concurrency Control (Database Technology Series)(Chinese Edition) Two-phase commit protocol Large-scale Incremental Processing Using Distributed Transactions and Notifications	Two phase committer(CN)
Planner		planner	Golang
Executor		executor expression chunk tikv/client-go	Golang, Relational Algebra, Parallel Programming	CMU: Advanced DB System - Query Execution	Introduction to Chunk and Execution Framework(CN) TiDB Hash Join(CN) Index Lookup Join(CN) Hash Aggregation(CN)

TiKV, distributed transactional key-value database

Contribution tutorial:

• CONTRIBUTING.md
• Land your first Rust PR in TiKV
• Help Wanted Issues

Module	Description	Code Directory	Required Skills	Learning Materials	Source Reading	What I can Contribute
Util	Utilities like thread-pool, logger, encoding and decoding, etc.	Utilities Pipeline batch system	Rust	Rust book Practical networked applications in Rust Protocol buffers gRPC concepts	Service layer(CN)
Network	Network layer	Server	Rust, Protobuf, gRPC	Ditto
Raw KV API		API entrance Storage struct	Rust	Ditto
Transaction KV API		API entrance Transaction GC worker Pessimistic transaction	Rust, 2PC, Percolator transaction model	Two Phase Commit Percolator paper	Storage Layer(CN) Distributed transaction(CN) MVCC read(CN)
Multi-raft		Raftstore	Rust, Raft	Raft paper Raft implementation in etcd	Raftstore overview(CN)
Engine		Engine traits Engine rocks	Rust, RocksDB			Engine abstraction
Coprocessor		TiDB query aggr TiDB query codegen TiDB query common TiDB query executors TiDB query expr TiDB query datatype	Rust		Coprocessor Overview(CN) Coprocessor Executor(CN)	Coprocessor Issues
Backup		Backup

PD Placement driver for TiKV

Module	Description	Code Directory	Required Skills	Learning Materials	What I can Contribute
API	HTTP interfaces for interacting with PD server	API	Golang	Golang http lib Golang Mux	OpenAPI Specification
TSO	Centralized timestamp allocation	TSO	Golang, etcd	Lamport timestamps etcd clientv3	Optimize the performance
Core	Basic facilities	Core	Golang	PD best practice(CN)	Improve hotspot recognition; adaptive Scheduling; scheduling according to region histogram
Statistics	Statistics which the scheduling relies on	Statistics	Golang	Ditto	Ditto
Scheduler	Controllers of scheduling	Schedulers	Golang	Ditto	Ditto
Schedule	Components related to scheduling like selector, filter, etc.	Schedule	Golang	Ditto	Ditto

TiKV Clients

Module	Required Skills	Learning Materials	What I can Contribute
Rust Client	Rust, TiKV concepts, Transaction model	TiKV documentation Rust book Practical networked applications in Rust Percolator paper	Issues
Go Client	Golang, TiKV concepts, Transaction model	Ditto	Issues
Java Client	Java, TiKV concepts, Transaction model	Ditto	Issues
Python Client	Python, TiKV concepts, Transaction model	Ditto	Issues

Libraries depended by TiKV

Module	Description	Code Directory	Required Skills	Learning Materials	Source Reading	What I can Contribute
grpc-rs	The gRPC library for Rust built on C Core library and futures	grpc c bindings, grpc rust wrapper	C++, Rust	grpc introduction Rust bindgen Rust async book	gRPC server Initialization(CN) grpc-rs Implementation(CN)	Load balance; reflection; custom metadata; migrate to async/await
fail-rs	A fail point implementation for Rust.		Rust	FreeBSD's failpoints	Introduction for fail-rs(CN)
raft-rs	Raft distributed consensus algorithm implemented in Rust		Rust, Raft	Raft paper Raft implementation in etcd
rust-prometheus	Prometheus instrumentation library for Rust applications		Rust	Prometheus doc Prometheus Go client	rust-prometheus basic rust-prometheus advance
rust-rocksdb	Rust wrapper for RocksDB		Rust, C++, RocksDB	Rust ffi Rust bindgen RocksDB wiki		Migrate bindgen to C++
Titan	A RocksDB plugin for key-value separation		C++, LSM-tree, RocksDB	Titan storage design and implementation WiscKey: Separating Keys from Values in SSD-conscious Storage

Ecosystem Tools: DM Data Migration Platform

Module	Description	Code Directory	Required Skills	Learning Materials	What I can Contribute	Contributing Tutorials
DM-master	The component for managing and scheduling the operation of data replication task	dm-master	Golang, etcd, gRPC, Protobuf	DM overview, source code reading: DM(CN)	Issues
DM-worker	The component for executing specific data replication tasks	dm-worker	Golang, etcd, gRPC, Protobuf	Ditto	Ditto
dmctl	A command line tool used to control the DM cluster	dm-ctl	Golang, gRPC, Protobuf	Ditto	Ditto
Dump unit	The processing unit for exporting SQL data from MySQL	mydumper	Golang, MySQL	Ditto	Ditto
Import/Load unit	The processing unit for importing SQL data into TiDB	loader	Golang, MySQL	Ditto	Ditto
Relay unit	The processing unit for dumping binlog data from MySQL	relay	Golang, MySQL	Ditto	Ditto
Binlog replication/sync unit	The processing unit for replicating binlog data into TiDB	syncer	Golang, MySQL	Ditto	Ditto
Checker unit	The processing unit for checking restrictions of the replication	checker	Golang, MySQL	Ditto	Ditto
DM-portal	A web-based application for managing data migration task configuration	portal	Golang, MySQL, Frontend	Ditto	Ditto
DM-ansible	The ansible playbook for DM	dm-ansible	DM, Ansible	Ditto	Ditto
DM-tracer	A DM-specific tracer framework	dm-tracer	Golang, gRPC, Protobuf	Ditto	Ditto

Ecosystem Tools - Binlog : A tool used to collect and merge tidb's binlog for real-time data backup and synchronization

Module	Description	Code Directory	Required Skills	Learning Materials	What I can Contribute	Contributing Tutorials
Pump Client	Send Binlog to the appropriate Pump on TiDB	pump_client, binloginfo	Golang, gRPC, etcd	TiDB Binlog overview, TiDB source code reading: Binlog(CN)	Help Wanted Issues
Pump	Receive and store Binlogs, sorted by commitTS	pump	Golang, LevelDB, etcd, gRPC	Ditto	Ditto
Drainer	Pull Binlog from Pump, do merge and sort, and then sync to downstream	drainer	Golang, Mysql, Kafka, gRPC	Ditto	Ditto

Ecosystem Tools - Lightning: A high-speed data import tool for TiDB

Module	Description	Code Directory	Required Skills	Learning Materials	What I can Contribute	Contributing Tutorials
Web Interface		web
CSV and SQL Parser		mydump
Delivery backend (output as SQL or as KV pairs)		backend
Checkpoints		checkpoints
Configuration		config
Logging		log
Metrics		metric
Workers (concurrency control)		worker
Restore Driver		restore
Utilities		common
Importer		import

Ecosystem Tools - BR: A command-line tool for distributed backup and restoration of the TiDB cluster data

Module	Description	Code Directory	Required Skills	Learning Materials	What I can Contribute	Contributing Tutorials
Backup	Backup data in cluster which has 3 mode: backup full cluster, backup specified DB and backup specified Table	br/backup, tikv/backup	gRPC, golang, rust	backup-principle
Restore	Restore data to new cluster after backup, relatively can restore full cluster, restore specified DB and restore specified Table.	br/restore, tikv/sst_importer	gRPC, golang, rust	restoration-principle

TiDB on K8S/Docker : Creates and manages TiDB clusters running in Kubernetes

Module	Description	Code Directory	Required Skills	Learning Materials	What I can Contribute	Contributing Tutorials
TiDB Operator	TiDB Operator manages TiDB clusters on Kubernetes and automates tasks related to operating a TiDB cluster. It makes TiDB a truly cloud-native database.	tidb-operator	Golang, Kubernetes, TiDB	how to use tidb-operator, Kubernetes Concepts, Kubernetes client-go SDK, Kubernetes Operator Pattern, TiDB Architecture, Helm, Terraform	Help Wanted Issues	Contributing Guide
Advanced StatefulSet	This is an Advanced StatefulSet CRD implementation based on official StatefulSet in Kubernetes. In addition to the official StatefulSet, it can scale the set in an arbitrary way.	advanced-statefulset	Golang, Kubernetes	Kubernetes Concepts, Kubernetes StatefulSet, Kubernetes StatefulSet Implementations, PingCAP StatefulSet Implementations
TiDB Docker Compose	Docker compose files for TiDB. With it, you can quickly deploy a TiDB testing cluster with a single command.	tidb-docker-compose	Docker, Docker Compose	how to start tidb with tidb-docker-compose, docker get started, docker compose

Deployment Tools - tidb-ansible: A TiDB cluster deployment tool based on Ansible playbook.

Module	Description	Code Directory	Required Skills	Learning Materials	What I can Contribute	Contributing Tutorials
tidb ansible	TiDB deployment operation and maintenance tools		Ansible, Python, Jinja2, Shell	How to deploy TiDB cluster through tidb-ansible, Ansible docs(en) and docs(cn)

Chaos-Mesh: A Chaos Engineering Platform for Kubernetes

Module	Description	Code Directory	Required Skills	Learning Materials	What I can Contribute	Contributing Tutorials
chaos-mesh			go, kubenetes

Documentation

Project	Required Skills	Learning Materials	What I can Contribute	Contributing Tutorials
Documentation(EN)	Sharp eyes, Good English writing, Technical writing or communication, Translation from English to Chinese, Knowledge about TiDB	Commit Message and Pull Request Style, Code Comment Style	Fix typos or format (punctuation, space, indentation, code block, etc.); Fix or update inappropriate or outdated descriptions; Add missing content (sentence, paragraph, or a new document); Translate docs changes from English to Chinese; Submit, reply to, and resolve issues; (Advanced) Review Pull Requests	docs readme, docs contribution guide
Documentation(CN)	Sharp eyes, Good English writing, Technical writing or communication, Translation from English to Chinese, Knowledge about TiDB	Chinese documentation style guide and template, Commit Message and Pull Request Style, Code Comment Style	Ibid, except for translating document changes from Chinese to English	docs-cn readme, docs-cn contribution guide

TiDB User Group(CN)

Required Skills	Learning Materials	What I can Contribute	Contributing Tutorials
Anyone of ansible, linux, DM, Binlog, lightning, pd, tidb, tikv, grafana, prometheus. Any experience in using RDBMS like MySQL or other RDBMS.	TiDB DBA courses	Help other TiDB users by answering questions in AskTUG; Publish practice articles of using TiDB or other databases; Giving presentations at TUG activities to share experience with TiDB; Involving in the organization/management of TUG (TiDB User Group) to expand the user base of TiDB	Join TUG

PingCAP University(CN)

Required Skills	Learning Materials	What I can Contribute	Contributing Tutorials
		TiDB DBA Course Design; TiDB Contributor Course Design; Sharing notes on TiDB; Become an administrator of the Wechat (online) group of learning; Joining the group to make subtitles for the video courses or translate them; Helping answer questions on TiDB; Notes, subtitles, translation, FAQ	Join PU

TiDB Developer Group

Channel	Description
TiDB Internals Forum	This is the hub for discussing everything related to the implementation and design of TiDB, TiKV and other TiDB ecosystem projects.
TiDB Slack Channel	The official Slack channel of the TiDB community.
TiKV Slack Channel	The official Slack channel of the TiKV community.
Documentation Slack Channel	The official Slack channel of the TiDB Documentation community.