Setting up a local development environment

November 13, 2025 ยท View on GitHub

Contents

Who this guide is for

This guide is intended for:

  • Developers who would like to work on patching bugs or adding new features to the common-voice code base via pull requests
  • Developers who would like to self-host common-voice on their own infrastructure, or re-purpose it to their own needs. NOTE: unfortunately due to resourcing constraints, we are unable to assist you in self-hosting outside of a commercial support arrangement, beyond what is given in this document. Please [make contact](mailto:commonvoice@mozilla.com?Subject=Inquiry about commercial hosting arrangements for Common Voice) if you would like further information regarding a commercial arrangement.

Prerequisites

Summary to get up and running

  • Fork and clone the repo
  • Set up an Auth0 tenant or similar to provide authentication.
  • Copy the .env-local-docker.example file to .env-local-docker, and edit as required for your context
  • Run docker containers with docker compose up
  • Flush redis cache when sentences from your language are imported
  • Restart docker containers
  • Visit http://localhost:9000.

Forking and cloning the common-voice repository

  1. Fork the common-voice repository into your own GitHub account. For help with forking, refer to the GitHub documentation.

  2. Clone the common-voice repository onto your local computer. You will need to have git installed first.

git clone https://github.com/common-voice/common-voice

For help with cloning, refer to the GitHub documentation.

NOTE: The spontaneous-speech components of Common Voice are not open source and are not available in this repository.

Project organization

The project is organized into the following directories:

  • common: Features that are shared between the frontend and backend of Common Voice, such as Typescript types.
  • locales: Information about activated languages on the site, automatically generated by yarn import-locales
  • maintenance: Static code for maintenance mode on Common Voice
  • scripts: Some scripts for managing data
  • server: The server-side app logic, written in TypeScript.
  • web: The Common Voice website files, written in TypeScript. We use React to build the website.
  • bundler: Service that is creating the dataset release bundles for Common Voice, written in TypeScript.

Orchestrating the environment with docker and docker-compose

NOTE: If you are working on a branch from your local fork of common-voice and subsequently raise a PR on the common-voice project, and this PR resolves an Issue in the common-voice repo, please ensure that the Issue is mentioned in the PR or otherwise linked for traceability. Linking Issues and PRs helps us to review them more efficiently.

Common Voice uses Docker and Docker Compose for orchestrating the development environment. While it is possible to install each component separately, we neither encourage nor support this.

.env-local-docker file

docker-compose uses a configuration file - docker-compose.yaml in order to build and launch docker containers. This file passes information such as the name of the containers, the users they should be run as, and networking configuration information, to docker.

It works hand in hand with a set of environment variables that are held in the .env-local-docker file. The common-voice repo contains an example .env-local-docker file. Each variable is prefixed with CV_ so that it doesn't clash with any other docker-based applications running on your system.

  1. Copy the example file .env-local-docker.example to .env-local-docker
> cd common-voice
> cp .env-local-docker.example .env-local-docker

The environment variables held in this file are explained below:

CV_DB_ROOT_PASS="voicewebroot"
CV_MYSQLHOST="db"

These variables specify the MySQL root password and the name of the Docker container which provides the database service. In general, these should not need to be modified.

CV_STORAGE_LOCAL_DEVELOPMENT_ENDPOINT="http://storage:8080"
CV_BULK_SUBMISSION_BUCKET_NAME="common-voice-bulk-submissions"
CV_ENVIRONMENT="local"
CV_PROD="false"

CV_IMPORT_SENTENCES="true"
CV_IMPORT_LANGUAGES="en,de,tr"

These vars specify which sentences to import when the Docker containers are run. You should set CV_IMPORT_SENTENCES to "true" if you are testing a particular language, and then specify its language code in CV_IMPORT_LANGUAGES. This will instruct the application to import the sentences located in server/data/* on start up. This step is IMPORTANT to be able to contribute to specific languages. Sentences are imported alphabetically by language code, so Arabic (ar) is imported before English (en), which is imported before Kiswahili (sw). Note that importing sentences for all languages will take several hours, so be sure to specify only the language(s) you need.

CV_EMAIL_USERNAME_FROM="commonvoice@mozilla.com"
CV_EMAIL_USERNAME_TO="commonvoice@mozilla.com"

If you are self-hosting Common Voice, you should change this to your email address.

CV_REDIS_URL="redis"

This specifies the caching docker container to use. You should only need to change this if you replace Redis with another caching layer

CV_JWT_KEY=super-secure-key

This is used for JWT authentication. You don't need to change it

Running the application through docker

  1. Once you have cloned and forked the repo, copied the .env-local-docker.example file, then run the following commands:
> cd common-voice
> docker-compose up

This is going to:

  • Launch a mysql instance configured for common-voice
  • Launch a fake GCP Cloud Storage instance to store files locally and avoid going through setting up GCP Cloud Storage
  • Mount the project using a Docker volume to allow reflecting changes to the codebase directly to the container
  • Import sentences from server/data/* if the key CV_IMPORT_SENTENCES is true
  • Launch common-voice server
  • Launch bundler service

Flushing the redis cache

Once you've imported the sentences for all locales specified in CV_IMPORT_LANGUAGES, open a new terminal and flush the redis cache:

> docker exec -it redis redis-cli FLUSHALL

This will ensure that on the next restart the languages, that we just imported sentences for, will be available for contribution.

Then, restart the server by stopping the containers (Ctrl+C on the command line) and then using docker compose up again.

You should be able to visit the Common Voice web server at http://localhost:9000.

Minimal setup

The bundler service is not strictly needed to run the common voice website. Run the following commands to just run the minimal setup:

> cd common-voice
> docker-compose up web

Docker can be a very memory-intensive process. If you notice intermittent failures, or if features like auto-rebuilding crash, try increasing Docker's available memory from within Docker's Preferences > Resources settings.

Troubleshooting

Couldn't connect to Docker daemon

If you get an error like the following when running native Docker (not Docker for Desktop),

ERROR: Couldn't connect to Docker daemon at http+docker://localhost - is it running?

You may need to build a new Docker image. You can do that by issuing the following commands:

> cd docker/
> docker build .

Then after this you can:

> cd ..
> docker compose up

You may have to run these commands as root/superuser.

Running docker-compose up results in incorrect file system permissions on /code/node_modules directory

After running docker-compose up, on Ubuntu, Fedora or other Linux-based systems, you may observe that the directory /code/node_modules is owned by root:root instead of app:app.

You may also observe the errors:

  • error Error: EACCES: permission denied, unlink '/code/node_modules/.yarn-integrity'
  • EACCES permission denied mkdir /code/node_modules

when the web container is building.

The root cause of this error is this line in the docker-compose.yaml, which is intended to set the GID and UID to a non-root user.

user: '${UID:-1000}:${GID:-1000}'

(These settings also appear under the bundler config. The bundler is only used for dataset releases, so we're only concerned about the web config here.)

However, in some cases, the UID and GID of your system may not be 1000:1000, which causes the error.

The workaround is to first identify the UID and GID of the current user, then edit the line in docker-compose.yaml to have that UID and GID:

> id -u
2000

> id -g
2000

(in docker-compose.yaml)

user: '${UID:-2000}:${GID:-2000}'

You should then be able to run docker-compose up successfully.

Linting Typescript code contributions

We're using ESLint (with Typescript) and Prettier to lint the project.

Install ESlint and Prettier extentions into your code editor or you can run this for all files with:

yarn lint

For now this is not automatically checked on Pull Requests or blocking while we fix existing issues.

Authentication

If you want to work with login-related features (Profile, Dashboard, Goals, ...) you'll need to set up authentication:

  1. Create an Auth0 account.
  2. Click "Applications" from the dashboard. Create a new one, or use the default application.
  3. On "Applications" still, next to your application, click the "three dots" icon, then Settings.
  4. Add http://localhost:9000/callback to the "Allowed Callback URLs" list.
  5. Copy the Auth0 application settings into your configuration file. These are found in the same Settings tab as the "Allowed Callback URLs".

For Docker, in .env-local-docker:

CV_FXA_DOMAIN="<domain_here>"
CV_FXA_CLIENT_ID="<client_id_here>"
CV_FXA_CLIENT_SECRET="<client_secret_here>"
  1. You can add more login options to your app from the "Connections" tab
  2. Restart your local Common Voice instance
  3. You will now be able to create a new user by clicking on "Log in" on your Common Voice instance and then switching over to the "Sign Up" tab on the login dialog.

Database migrations

We use db-migrate for running database migrations.

To add a migration run: yarn migrate:local create <MIGRATION_NAME>.

At the moment you manually have to change the migration file extension to .ts. A migration has to expose the following API:

export const up = async function (db: any): Promise<any> {
  return null
}

export const down = async function (): Promise<any> {
  return null
}

Migrations are always run when the server is started.

To migrate up or down manually run: yarn migrate:local [up|down].

This will execute the most recently added migration in the migrations folder.

Localization

We're using Fluent to localize strings. You can find examples all over the frontend code. Strings that appear in the english message files, can then be translated on Pontoon. Some things to note regarding string changes are documented on MDN.

Testing using Jest

We write unit tests using Jest. Test files should be named function_or_service.test.ts where function_or_service.ts is the name of the Typescript file.

To run a unit test, or all unit tests in the application, first enter the Docker container and open a bash shell, then run yarn test:

docker exec -it common-voice bash
$ yarn test

To run a single unit test, run yarn [path to test file].

Submitting an Issue

Did you notice a bug? Do you have a feature request? Please file an issue here on GitHub.

Something Else?

Want to talk about something but can't find a home for it here? Head to our Discourse Category to discuss everything from feedback and ideas to questions and random musings.