EXTRAS.md

June 2, 2025 · View on GitHub

Getting Google OAuth API credential file and token.pickle

NOTES:

  • Authentication methods have changed; token.pickle can no longer be generated using the bot or Replit. You will need an operating system with a local web browser (e.g., Termux on Android, or a desktop OS).
  • Windows users should install Python 3 and pip. Instructions can be found via a Google search or in Wiszky's Telegraph tutorial.
  • The link generated by generate_drive_token.py must be opened in a local web browser.
  1. Visit the Google Cloud Console.
  2. Go to the OAuth Consent Screen tab, fill in the required information, and save.
  3. Go to the Credentials tab and click Create Credentials -> OAuth client ID.
  4. Choose Desktop app as the Application type and click Create.
  5. Publish your OAuth consent screen app to prevent token.pickle from expiring. (Usually, an option like "Publish App" under the OAuth consent screen).
  6. Use the download button (usually a JSON icon) next to your newly created OAuth client ID to download your credentials.
  7. Move the downloaded file to the root directory of your mirrorbot instance and rename it to credentials.json.
  8. Visit the Google APIs Library.
  9. Search for "Google Drive API" and enable it for your project.
  10. Finally, run the script to generate the token.pickle file for Google Drive:
pip3 install google-api-python-client google-auth-httplib2 google-auth-oauthlib
python3 generate_drive_token.py

Generating rclone.conf

  1. Install Rclone from the Official Site.
  2. Create new remote(s) using the rclone config command.
  3. Copy rclone.conf from your system’s Rclone configuration directory into the repository root directory.

Upload Configuration

  • RCLONE_PATH, similar to GDRIVE_ID, defines a default path for mirrors.

  • Additionally, the DEFAULT_UPLOAD variable selects the default tool, either Rclone (rc) or the Google API Python client (gd).

  • If DEFAULT_UPLOAD is set to 'rc', you must provide RCLONE_PATH as the default, or use rcl in commands to select the destination path for each new task.

  • If DEFAULT_UPLOAD is set to 'gd', you must provide GDRIVE_ID with the Google Drive folder or Team Drive ID.

  • rclone.conf can be added to the repository root directory (like token.pickle) before deployment, or uploaded as a private file using the /bsetting command.

  • If rclone.conf is uploaded via /usetting or placed in rclone/{user_id}.conf, then any Rclone path specified in commands or RCLONE_PATH must be prefixed with mrcc:.

  • When manually specifying a path to use a user-specific rclone.conf (added via /usetting), prefix the path with mrcc:.

In summary, the up: parameter in commands can have the following values:

  • gd: Upload to the GDRIVE_ID specified in your configuration.
  • rc: Upload to the RCLONE_PATH specified in your configuration.
  • rcl: Select an Rclone path at the time of the task.
  • rclone_path: A specific Rclone path, e.g., myremote:path (uses owner's rclone.conf) or mrcc:myremote:path (uses user's rclone.conf).

The UPSTREAM_REPO variable can be used to edit or add any file in the repository.

  • You can add a private or public repository link to fetch or overwrite all files from it.
  • You can skip adding private files like token.pickle or the accounts folder before deploying; simply fill UPSTREAM_REPO with a private repository link if you want to fetch all files, including private ones.
  • If you add private files during deployment AND you've also specified a private UPSTREAM_REPO containing those same private files, the files from the UPSTREAM_REPO will overwrite those added during deployment.
  • Furthermore, if you are using a database for private files, those files will override any added before deployment or from a private UPSTREAM_REPO.
  • If you fill UPSTREAM_REPO with the official repository link, be careful: if there are changes in requirements.txt in the official repository, your bot might not start after a restart. In this case, you need to deploy again with updated code to install the new requirements or simply change UPSTREAM_REPO to your fork's link with the older (compatible) updates.
  • If you fill UPSTREAM_REPO with your fork's link, also be careful when fetching commits from the official repository, as changes there might affect your bot.
  • Changes from your UPSTREAM_REPO will take effect only after a restart of the bot.

BitTorrent Seed Configuration

Using the -d argument (or equivalent seeding options in commands) alone will lead to using global options for Aria2c or qBittorrent.

QBittorrent

Global options: GlobalMaxRatio and GlobalMaxSeedingMinutes in qbittorrent.conf. A value of -1 means no limit, but you can cancel seeding manually.

NOTE: Do not change MaxRatioAction.

Aria2c

Global options: --seed-ratio (a value of 0 means no limit) and --seed-time (a value of 0 means no seeding after download completion) in aria.sh.

Using Service Accounts for Uploading (to avoid user rate limits)

For Service Accounts to work, you must set USE_SERVICE_ACCOUNTS = "True" (as a string) in your configuration file or environment variables. NOTE: Using Service Accounts is primarily recommended when uploading to a Team Drive.

1. Generate Service Accounts. (What is a Service Account?)

Let us create only the Service Accounts that we need. Warning: Abuse of this feature is not the aim of this project, and we do NOT recommend creating many projects. Just one project with 100 SAs allows for plenty of use. It's also possible that excessive abuse might get your projects banned by Google.

NOTE: If you have created SAs in the past using this script, you can re-download the keys by running:

python3 gen_sa_accounts.py --download-keys $PROJECTID

NOTE: One Service Account can upload/copy approximately 750 GB per day. One Google Cloud project can create up to 100 Service Accounts, allowing for a potential daily upload capacity of 75 TB.

NOTE: Google Drive users can typically copy 2TB/DAY from files created by a single uploader account. If you encounter a userRateLimitExceeded error, it doesn't necessarily mean your limit was exceeded, but rather that the file creator's daily sharing/download limit (2TB/DAY) for that specific file has been reached for others.

Two methods to create Service Accounts:

List your projects ids:

python3 gen_sa_accounts.py --list-projects

Enable services automatically by this command:

python3 gen_sa_accounts.py --enable-services $PROJECTID

Create Service Accounts to current project:

python3 gen_sa_accounts.py --create-sas $PROJECTID

Download Service Accounts as accounts folder:

python3 gen_sa_accounts.py --download-keys $PROJECTID
Method 2: Create Service Accounts in New Project
python3 gen_sa_accounts.py --quick-setup 1 --new-only

A folder named accounts will be created which will contain keys for the Service Accounts.

2. Add Service Accounts

Two methods to add service accounts:

Mount accounts folder:

cd accounts

Grab emails from all accounts into an emails.txt file that will be created in the accounts folder:

For Windows using PowerShell:

$emails = Get-ChildItem .\**.json |Get-Content -Raw |ConvertFrom-Json |Select -ExpandProperty client_email >>emails.txt

For Linux:

grep -oPh '"client_email": "\K[^"]+' *.json > emails.txt

Navigate out of the accounts folder:

cd ..

Then add the emails from emails.txt to a Google Group. After that, add this Google Group to your Shared Drive and promote the group to "Manager". Finally, you can delete the emails.txt file from the accounts folder.

Method 2: Add Them To Team Drive Directly

Run:

python3 add_to_team_drive.py -d SharedTeamDriveSrcID

Create Database

  1. Go to https://mongodb.com/ and sign-up.
  2. Create Shared Cluster.
  3. Press on Database under the Deployment header; your created cluster will be listed there.
  4. Press Connect for your cluster, choose Allow Access From Anywhere, and press Add IP Address (without editing the IP unless necessary). Then, create a database user.
  5. After creating the user, press Choose a connection method, then select Connect your application. Choose Python as your driver and version 3.12 or later.
  6. Copy your connection string. Replace <password> with the password of the user you created. Then, press Close.

Multi-Drive List Configuration

To use a list from multiple Team Drives or folders, run driveid.py in your terminal and follow the prompts. It will generate a list_drives.txt file, or you can simply create list_drives.txt in the working directory and fill it according to the format below:

DriveName folderID/tdID or `root` IndexLink(if available)
DriveName folderID/tdID or `root` IndexLink(if available)

Example:

TD1 root https://example.dev
TD2 0AO1JDB1t3i5jUk9PVA https://example.dev

yt-dlp and Aria2c Authentication Using .netrc File

For using your premium accounts in yt-dlp or for accessing password-protected Index Links with Aria2c, create a .netrc file in the repository root with the following format:

Note: Create a file named .netrc (starting with a dot). This file will be hidden on Unix-like systems, so ensure your file explorer is set to show hidden files if you need to edit it after creation.

Format:

machine host login username password my_password

Using Aria2c you can also use built in feature from bot with or without username. Here example for index link without username.

machine example.workers.dev password index_password

Where host is the name of the extractor (e.g., instagram, twitch, or the hostname of the protected index). Multiple accounts for different hosts can be added, each on a new line.

yt-dlp: Authentication using cookies.txt file. It's recommended to CREATE THIS FILE IN AN INCOGNITO/PRIVATE BROWSER TAB to avoid exporting personal cookies.