Incremental Fine-Tuning (IFT) Pipeline

April 7, 2026 · View on GitHub

The Incremental Fine-Tuning (IFT) Pipeline implements an approach for automated drug design using chemical language models trained on Structure-Activity Relationship (SAR) series. By fine-tuning models on compounds with increasing potency, it generates novel molecules with desired activity profiles. This documentation provides installation instructions and a guide for running the complete pipeline, from pretraining through to final compound generation.

Background
Quick start on Demo
System Requirements
- Hardware Requirements
- Software Requirements
Installation
- Post-Installation Steps
Pipeline Usage
License
Support

Background

Automating structural optimization of drug molecules for on-target potency by machine learning is an open challenge in chemistry. Here, we capitalize on the ability of chemical language models (CLMs) to learn from sequential data and design new molecules with desired properties. We establish a training strategy mimicking the learning trajectory of a drug discovery program. Incremental CLM fine-tuning with increasingly potent template molecules from a given structure-activity relationship (SAR) series successfully bias the model to design highly active analogues. Prospective application of this technique enables the data-driven development of PPARγ and RORγ modulators exceeding known ligands of the given chemotypes in potency without external scoring. Our results reveal an ability of CLMs to capture SAR patterns and long-range dependencies, and to exploit SAR knowledge in designing analogues with improved on-target activity de novo corroborating their applicability to structural optimization of drug molecules.

Quick Start on Demo

For a quick demo using already prepared data and models, run:

# Clone repository
git clone https://github.com/AkMerk/incremental-clm --depth 1
cd incremental-clm

# Create and activate virtual environment
# virtualenv needs python3-virtualenv and python3.10 installed on the system
# alternatively use uv, conda, ... to create the virtual environment
python3 -m virtualenv ift_venv --python=python3.10
. ift_venv/bin/activate

# Install package
pip install .

cd example_workspace
ift setup demo_run sar/prepared.csv "PMID: 14695833" 0 values 4 2
ift run all demo_run pparg_2l_rnd.h5

For a complete execution of the entire pipeline, refer to the provided shell script:

cd example_workspace
./run.sh

Expected runtime: ~20 minutes (for quick demo) or > 200 minutes (for complete pipeline)

System Requirements

Hardware Requirements

Operating System: Linux Ubuntu (20.04/22.04/24.04 LTS) or macOS Sequoia (15)
Processor: x86_64 (Linux) or arm64 (macOS)
GPU: NVIDIA CUDA-compatible (Linux) or Metal (macOS) with ~16 GB VRAM

Software Requirements & Dependencies

Python 3.10
TensorFlow 2.11 (Linux) or 2.13 (macOS)
System utilities: git, sqlite3, sed, pigz, mawk, pv
Additional Python packages listed in requirements.txt

Benchmarked on 8 cores, 24 GB RAM, 125Mbit/s internet

Installation

Expected time: ~5-10 minutes

# Clone repository
git clone https://github.com/AkMerk/incremental-clm --depth 1
cd incremental-clm

# Create and activate virtual environment
python3 -m virtualenv ift_venv --python=python3.10
. ift_venv/bin/activate

# Install package
pip install .

ift --help # Should show the available commands for the IFT pipeline

Post-Installation Steps

macOS

# test gpu
pip install pytest && pytest tests/test_gpu.py # should pass with 3 passed, 1 skipped

Linux

# set environment variables for CUDA (required on every new shell)
export PYTHON_SITE=$(python3 -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')
export LD_LIBRARY_PATH="$(find $PYTHON_SITE -type d -path "*/nvidia/*/lib" | tr '\n' ':')${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"

# test gpu
pip install pytest && pytest tests/test_gpu.py # should pass with 3 passed, 1 xfailed

Pipeline Usage

Each command supports --help flag for detailed parameter information. All commands but ift init must be run from within the workspace directory.

1. Initialize Workspace

Create and configure a new workspace.

ift init ./workspace

Creates a new workspace containing a settings.toml configuration file that can be customized following the included comments.

Note: For quick testing, use provided example_workspace directory and skip to Pretraining

2. Download Data

Download and preprocess required datasets from public sources.

# Required for pretraining
ift download chembl_db             # ChEMBL database (4.3 GB, ~5 min)

# Optional for SAR filtering
# BindingDB mirror is not very reliable.
# Recommendation:
# Download it manually from https://www.bindingdb.org/rwd/bind/BDB-mySQL_All_202508_dmp.zip
# and continue with the download command afterwards
# curl -L https://www.bindingdb.org/rwd/bind/BDB-mySQL_All_202508_dmp.zip -o downloads/BDB-mySQL_All_202508_dmp.zip
# unzip -p downloads/BDB-mySQL_All_202508_dmp.zip BDB-mySQL_All_202508.dmp > downloads/BDB-mySQL_All_202508.dmp.d817889fd71c389c1891e75953b045d6a90a802d5858c03ab03d645c156307d7
ift download binding_db

# Optional for compound filtering
ift download chembl_compounds    # ChEMBL compounds (200 MB, ~4 min)
ift download surechembl_compounds # SureChEMBL compounds (1.9 GB, ~42 min)

# To download all files
ift download all

Downloads required datasets from ChEMBL and SureChEMBL to the downloads directory.

3. Prepare Data

Filter, clean and split datasets into training and validation sets.

# Prepare base dataset
ift prepare chembl              # ChEMBL data (~7 min)

# Prepare target-specific data
ift prepare <target>            # Target dataset (~2 min)

Processes the downloaded data and creates training splits in data/training/<target>/*.txt, where targets must be defined in the settings.toml under [TARGETS].

4. Pretrain Model

Train and evaluate the base chemical language model.

# Train model on prepared dataset
ift pretrain train --
    <target>                # Target dataset
    <layers>                # LSTM layers: 2 or 3
    <augmentation>          # 0: canonical, 10: augmented, -1: randomized SMILES

# Evaluate and visualize training progress
ift pretrain eval -- <target> <layers> <augmentation>

Evaluates and visualizes model performance in pretraining/<layers>l/x<augmentation>/<target>/scores/plot.png. Use this plot to select the optimal epoch from pretraining/<layers>l/x<augmentation>/<target>/models/*.h5 for subsequent fine-tuning with .

Pretraining Run Times

# LSTM layers	Augmentation	# Epochs	Train Time (macOS, Metal)	Train Time (Linux, CUDA)
2	randomized	120	~10 min/epoch	~3 min/epoch
2	canonical (0x)	40	~10 min/epoch	~3 min/epoch
2	augmented (10x)	40	~90 min/epoch	~30 min/epoch
3	randomized	120	~30 min/epoch	~9 min/epoch
3	canonical (0x)	40	~30 min/epoch	~9 min/epoch
3	augmented (10x)	40	~5 hours/epoch	~80 min/epoch

Note: additional ~1 min/target for generating comparison data

Test Time (macOS, Metal)	Test Time (Linux, CUDA)
~1.5 min/epoch	~1 min/epoch

5. SAR data

Prepare Structure-Activity Relationship (SAR) series data for fine-tuning.

Required Format: CSV file with columns:

identifier: Unique identifier for each SAR series
smiles: SMILES string of the molecule
value: Activity value (e.g., pEC50, higher = more active)

identifier,smiles,value
PMID:123456789,CCO,1.2
PMID:123456789,CCN,2.4

Warning: Activity values must be in increasing order within each series (e. g., pEC50)

Demo Data

For testing, use provided example: example_workspace/prepared/sar.csv

Automatic SAR data generation

Generate SAR data from public databases (ChEMBL and BindingDB):

See ./sar_draft.toml for further information and as example.

ift download chembl_db             # ChEMBL SQLite (4.3 GB, ~5 min)
ift download bindingdb_db          # BindingDB data (800 MB, ~5 min)

# Adjust the SAR configuration file in ./sar_draft.toml

# Extract target data (e.g., PPARγ)
ift sar
    <sar config>                      # Path to the SAR configuration
    <chembl like>                     # SQL-flavored query for ChEMBL's pref_name
    --bindingdb-like <bindingdb like> # SQL-flavored query for BindingDB's display_name

Generates filtered and merged SAR data in sar/merged.csv. See ./sar_draft.toml for configuration options and filtering criteria.

Note: The extraction process may require multiple manual interventions to resolve ambiguous assay selections. See comments in ./sar_draft.toml.

6. Setup Incremental Fine-Tuning

Configure the incremental fine-tuning process.

ift setup
    <run tag>              # Unique identifier for this run
    <sar csv file>         # Path to SAR data CSV
    <identifier>           # SAR series identifier to use
    <augmentation>         # Augmentation strategy
    <splitting method>     # How to split SAR series
    <number of boxes>      # Number of incremental steps
    <layers>               # Number of LSTM layers

Creates a runs/<run tag>/config.json file containing the fine-tuning configuration and the already split dataset.

For detailed parameter options, use ift setup --help

7. Run Incremental Fine-Tuning

Execute the fine-tuning process step by step or use the combined command.

# Train models incrementally (~3-9 min)
ift run train
    <run tag>
    <pretrained model path>  # Model from step 4

# Sample new molecules (~2-19 min)
ift run sample <run tag>

# Evaluate results (~1-4 min)
ift run eval
    <run tag>
    [
    --pretrained-model <pretrained model>    # Optional to remove pretraining artifacts
    --naive-perp-threshold <perplexity threshold>
    ]

The pipeline saves trained models in runs/<identifier>/<run tag>/runs/<repetition>/models/*.h5, generates samples in sampled.json, and evaluates them against the holdout set in evaluated.json.

Recommended: Use combined command for complete pipeline:

ift run all
    <run tag>
    <pretrained model>
    [--naive-perp-threshold <naive perplexity threshold>]

8. Score Runs

Compare and evaluate multiple training runs.

ift score "<glob pattern>"    # Glob pattern to match run tags, e.g. "run*" or "pparg_*"
ift score --mode adjusted "<glob pattern>" # to exclude pretraining artifacts (specified by --naive-perp-threshold during run eval)
ift score --top-n <n> "<glob pattern>" # to only consider the top n samples

Outputs a combined CSV file with scores at scores/<timestamp>.csv (~1 min).

Warning: Use ift setup ... --rep to differentiate runs with identical parameters

9. Final Application

Generate optimized compounds using the incremental fine-tuning method.

Note: Currently only available for randomized training mode (augmentation = -1)

# 1. Download filtering datasets
ift download chembl_compounds
ift download surechembl_compounds

# 2. Setup final run (set holdout-frac to 0.0 to use all SAR data)
ift setup <final run tag> <sar csv> <id> -1 values <boxes> <layers> --holdout-frac 0.0

# 3. Run final generation & filtering
ift final \
    [--filter-smis <path_to_exclude.smi>] \
    -- \
    <final run tag> \
    <pretrained model> \
    <perplexity threshold> \
    <output filename>

Trains multiple instances of the model to generate novel compounds, which are then filtered based on perplexity scores and known compounds from ChEMBL/SureChEMBL databases (18-50 min on Linux).
Use --filter-smis <path_to_exclude.smi> to provide a list of SMILES to exclude from final output (e.g., to prevent overlap with a previous all run).
Results are saved to finals/<final run tag>/<output filename> and a detailed runs/<final run tag>/filtered_samples.csv.

License

MIT License

Support

For issues and questions, please open a GitHub issue.