List of processes

April 10, 2024 · View on GitHub

Introduction: Quick Start, Tutorial, Flowchart, Outputs structure
Install: Dependencies, Containers, References, Test datasets
Inputs: Data, Design, Parameters
1. Preprocessing: ATAC reads, ATAC peaks, mRNA
2. Differential Analysis: ATAC, mRNA, Split
3. Enrichment Analysis: Enrichment, Figures, Tables

List of processes

Preprocessing
Quality Controls

Preprocessing

ATAC_reads__merging_reads

Description

Samples sequenced multiple times (that have the same id) are merged.

Parameters

params.save_fastq_type: Saving only the last, none or all fastq files. Options: 'none', 'last', 'all'. Default: 'none'.

Outputs

Merged reads: Processed_Data/1_Preprocessing/ATAC__reads__fastq_merged/${sample}_R{1,2}_merged.fastq.gz if params.save_fastq_type = 'all'.

ATAC_reads__trimming_reads

Description

ATAC-Seq adaptors are trimmed using Skewer, then they are compressed in parallel by PIGZ.

Parameters

params.pigz__nb_threads: number of threads used for parallel compression. Default: 6.
params.save_fastq_type: Saving only the last, none or all fastq files. Options: 'none', 'last', 'all'. Default: 'none'.

Outputs

Log files:
- ${sample}_skewer_trimming.log
- ${sample}_pigz_compression.log
Trimmed reads: *_R{1,2}_trimmed.fastq if params.save_fastq_type = 'all'

Output folders

Processed_Data/1_Preprocessing/ATAC__reads__fastq_trimmed.

ATAC_reads__aligning_reads

Description

Reads are aligned to the reference genome by Bowtie2 and the resulting SAM files are converted to BAM with SAMtools. SAMtools is also used to count the number of aligned reads per category.

Parameters

params.bowtie2__nb_threads: number of threads used by Bowtie2. Default: 6.

Outputs

Bowtie 2 alignment metrics: ${sample}_bowtie2_align_metrics.txt
Number of aligned reads per category: ${sample}.qc
Aligned reads: ${sample}.bam if params.save_bam_type = 'all'

Output folders

Processed_Data/1_Preprocessing/ATAC__reads__fastq_trimmed.

ATAC_reads__removing_low_quality_reads

Low quality reads with these attributes are filtered: unmapped, mate unmapped, no primary alignment, low MAPQ (quality score). Reads are sorted and the number of aligned reads per category is determined with SAMtools.

Parameters

params.sam_MAPQ_threshold: MAPQ threshold. Default: 30.

Outputs

Number of aligned reads per category: ${sample}__filter_LQ.qc
Aligned reads: ${sample}__filter_LQ.bam if params.save_bam_type = 'all'

Output folders

Processed_Data/1_Preprocessing/ATAC__reads__bam_no_lowQ.

ATAC_reads__marking_duplicated_reads

Description

Picard is used to mark duplicated reads.

Parameters

params.memory_picard: maximum memory used by Picard. Default: '20G'.

ATAC_reads__removing_duplicated_reads

Description

Duplicated reads are removed with SAMtools, an index is build, and the number of aligned reads per category is determined with SAMtools.

Outputs

Number of aligned reads per category: ${sample}__dup_rem.qc
Aligned reads: ${sample}__dup_rem.bam if params.save_bam_type = 'all'

Output folders

Processed_Data/1_Preprocessing/ATAC__reads__bam_no_lowQ_dupli.

ATAC_reads__removing_reads_in_mitochondria_and_small_contigs

Description

Samtools is used to remove reads mapping to the mitochondrial chromosome or to small contigs. This is defined as regions (contigs and chromosomes) with less than 5 genes (see References). Number of aligned reads per chromosomes and per category are also computed with SAMtools.

Outputs

Number of aligned reads per category: ${sample}__no_mito.qc
Number of reads per chromosome before and after filtering: .txt file)
Aligned reads: ${sample}__no_mito.bam if params.save_bam_type = 'all' or 'last'

Output folders

Processed_Data/1_Preprocessing/ATAC__reads__bam_no_lowQ_dupli_mito.

ATAC_reads__converting_bam_to_bed_and_adjusting_for_Tn5

Description

This process converts bam files to bed files, and use the insert length as the score. It then adjusts for the shift of the transposase to account for the 9 bp offset of the transposase. This is done by shifting reads on the + strand by +4 base pairs and reads on the - strand by -5 base pairs. Finally, it keeps only the 5' end of each reads (and thus each read becomes 1 base pair long), and create a sorted and indexed bam file from the final adjusted bed file. The bam file is sent to DiffBind] for Differential Binding analysis. The bed file is sent for custom quality controls processes (see below), for computing and plotting saturation curve, and for calling macs2 peaks.

Parameters

params.save_1bp_bam: Saving the 1 base pair reads after all filtering steps and tn5-shift adjustement

Outputs

Aligned reads: Processed_Data/1_Preprocessing/ATAC__reads__bam_asBed_atacShift/${sample}__1bp_shifted_reads.bam if params.save_1bp_bam = true.

Quality Controls

ATAC_QC_reads__running_fastqc

Description

FastQC is runned to perform standard quality controls checks on sequenced data.

Parameters

params.fastqc__nb_threads: number of threads used by FastQC. Default: 2.

Outputs

Reads quality control reports: *_R{1,2}_fastqc.html

Output folders

Processed_Data/1_Preprocessing/ATAC__reads__fastqc_raw for raw reads
Processed_Data/1_Preprocessing/ATAC__reads__fastqc_trimmed for trimmed reads.

ATAC_QC_reads__computing_bigwig_tracks_and_plotting_coverage

Description

Coverage profiles (i.e. bigwig files) and plots are generated by DeepTools.

Parameters

params.do_bigwig: enable or disable this process. Default: true.
params.deeptools__nb_threads: number of threads used by DeepTools. Default: 6.
params.deeptools__binsize_bigwig_creation: size of the bins for the creation of the bigwig file. Smaller values increase computation time. Default: 10.
params.deeptools__binsize_bigwig_correlation: size of the bins for computing correlation between samples. Smaller values increase computation time. Default: 10000.
params.deeptools__nb_of_1_bp_samples: number of 1 bp sites to sample for the coverage plots. Default: 10000.
params.deeptools__normalization_method: normalization method to use when creating BigWig files. See here for options. Default: 'none'.

Outputs

BigWig files: Processed_Data/1_Preprocessing/ATAC__reads__bigwig_raw/${sample}.bw.
Coverage plots:
- Figures_Individual/1_Preprocessing/ATAC__reads__coverage/${sample}__reads_coverage.pdf
- Figures_Merged/1_Preprocessing/ATAC__reads__coverage.pdf

ATAC_QC_reads__computing_and_plotting_bigwig_tracks_correlations

Description

First, the coverage matrix of all samples is computed with DeepTools's function multiBigwigSummary. Then PCA and Correlations plots are generated also with DeepTools, while excluding blacklisted regions. The PCA is computed with filtering genes to keep only the top (100, 1000, 5000) most variables rows in the matrix, and with or without genomic DNA control (i.e. input) if present.
Correlation matrix is computed for spearman and pearson correlation, with or without outliers, and with or without genomic DNA control (i.e. input) if present. If an input control is included (genomic DNA), then two sets of plots will be made: with or without the input.

Parameters

params.deeptools__binsize_bigwig_creation: size of the bins in the coverage matrix. Smaller values increase computation time. Default: 10000.

Outputs

PCA plots:
- Figures_Individual/1_Preprocessing/ATAC__reads__PCA/pca_top{100,1000,5000}_${gDNA_PRESENT}_pca.pdf
- Figures_Merged/1_Preprocessing/ATAC__reads__PCA.pdf.
Correlation plots:
- Figures_Individual/1_Preprocessing/ATAC__reads__correlations/{pearson,spearman}_correlation_heatmap_{with,without}_outliers_${gDNA_PRESENT}_cor.pdf
- Figures_Merged/1_Preprocessing/ATAC__reads__correlations.pdf.

ATAC_QC_reads__plotting_insert_size_distribution

Description

Insert size plots are made with Picard.

Parameters

params.memory_picard: maximum memory used by Picard. Default: '20G'.

Outputs

insert size plots:
- Figures_Individual/1_Preprocessing/ATAC__reads__insert_size/${sample}__insert_size.pdf
- Figures_Merged/1_Preprocessing/ATAC__reads__insert_size.pdf.

ATAC_QC_reads__sampling_aligned_reads

Description

This process takes as input non-filtered aligned reads (coming straight out of Bowtie2).
These are filterd by keeping only mapped reads (flag 0x4) that are not secondary (flag 0x100, multimappers) or supplementary (flag 0x800, chimeric entries) and transformed to the sam format with SAMtools. This sam file is sent to estimating_library_complexity.
Then a certain number of reads are randomly sampled and a bam file is created, sorted and indexed. This bam file is sent to measuring_overlap_with_genomic_regions and gathering_all_stat.
Finally, SAMtools is used again to determine the number of total pairs and the number of aligned pairs in the original, non-sampled bam file. This information is sent to gathering_all_stat.

Parameters

params.nb_sampled_aligned_reads: Number of aligned reads to sample. Default: 1000000.

ATAC_QC_reads__measuring_overlap_with_genomic_regions

Description

The overlap of sampled aligned reads with various annotated regions (exons, genes, intergenic regions, introns and promoters) is computed with BEDTools.

ATAC_QC_reads__estimating_library_complexity

Description

Picard is used to estimate library complexity of our set of sampled aligned reads.

Parameters

params.memory_picard: maximum memory used by Picard. Default: '20G'.

ATAC_QC_reads__sampling_trimmed_reads

Description

BBMap is used to sample trimmed reads.

Parameters

params.nb_sampled_trimmed_reads: Number of trimmed reads to sample. Default: 1000000.

ATAC_QC_reads__aligning_sampled_reads

Description

Sampled trimmed reads are aligned to the reference genome under study (worm, fly, mouse or human) and to the contaminant specie's reference genome (i.e. E. coli OP50 strain for all species currently). Then, the number of aligned reads per category is determined with SAMtools for both organisms.

Parameters

params.botwie2__nb_threads: number of threads used by Bowtie2. Default: 6.

ATAC_QC_reads__gathering_all_stat

Description

This process gather all reads statistics from previous processes (starting with ATAC_QC_reads__) for a given sample, computes a few more stastistics, and creates a summary file.

Here is a description/summary of how the reported statistics were generated:

Number of paired end reads:
- Raw Pairs: number of total reads in the original bam file (after Bowtie2 alignment, without any filtering step)
- Aligned Pairs: number of aligned reads in the original bam file (after Bowtie2 alignment, without any filtering step)
- Final Pairs: number of reads in the final bed file (1 base pair reads after all filtering steps)
Percentage of sampled trimmed reads that aligned to:
- the reference genome under study
- the contaminant specie's genome.
Sampled aligned reads are used to estimate:
- The percentage of read that overlap with the mitochondrial genome and with genomic annotations (promoters, exons, introns, intergenic regions and genic regions)
- Library complexity

MultiQC report: ATAC__multiQC.html. Example.

Output folders

Figures_Individual/1_Preprocessing
Figures_Merged/1_Preprocessing