4-RNA-seq

A slurm based schema for RNA-seq analysis to execute on linux clusters.
A Pipeline:4-RNA-seq and a minimal RNA-seq cook book to explain each step is freely available here.

The purpose of this project to develop a easily customizable commandline based schema. Additionally it has basic linux scripts for file manipulation which is key to execute command line pipeline.

Installation

4-RNA-seq is required to download every new project.
Download
For each experiment 4-RNA-seq pipeline needs to be downloaded separately. Let us download it to a directory named "myProjectDir" with following commands

git clone https://github.com/vondoRishi/4-RNA-seq myProjectDir

From now on myProjectDir is the project space

Prepare the workspace
Make a directory "rawreads" inside "myProjectDir" and copy fastq(.gz) files there.

mkdir myProjectDir/rawReads
cp sourceDir/*fastq.gz myProjectDir/rawReads

RNA-seq pipeline

This pipeline and workflow is optimized for Puhti server. The old pipeline for Taito.csc server is moved at "Taito" branch due to decommisioning of the old server.

Please read the server documentation before start using. It may need to customized according to that.

The objective of this documentation is to make execution faster and reproducible as much as possible. The project folder ( should be in /scratch/<project> path and in this example "myProjectDir" ) should contain these folders before starting

scripts : contains all scripts to run in taito server
OUT : contains output files from all scripts
ERROR : contains error files from all scripts
commands : contains actual commands { will be required in future to find the project specific parameters }
rawReads : should contain sequencing reads generated by the sequencing machine. Folder name could be anything.

Additional info : Library type, sequencing platform Input: Reference Genome (DNA sequences) fasta and annotation file (GTF) Run “ls -lrth” after every step to find the last modified file

Dependency

Need to install afterqc by the user.

Multiqc ( run almost after all the commands) { installation guide}
AfterQC { installation guide .}

Execution

Before execution please define the project in the variables of 4-rna-seq.config file. These values will be used by different scripts of this pipeline. First define PROJAPPL variable as used for installing MultiQC

cd myProjectDir

QC and Filtering

Start QC ( quality checking) with Fastqc and Multiqc. The scripts/fastqc.sh executes first Fastqc and then Multiqc internally.

Input : directory rawReads with fastq or fastq.gz files
Execution: Replace <project> and <email_id>.

sbatch -A <project> -D $PWD --mail-user <email_id> scripts/fastqc.sh rawReads  # Don't use "rawReads/"

Output : directory rawReads/rawReads.html and other files

ls -lrth rawReads/

Filter/trimminging with
a) AfterQC (Define the "AfterQC" variable in 4-rna-seq.config before using)
Input : directory rawReads with fastq or fastq.gz files
Execution :

sbatch -A <project> -D $PWD --mail-user <email_id> scripts/afterqc_batch.sh rawReads

Output : directory good, bad and QC

b) ALERT for "single end reads" users!! AfterQC can not trim adapters from single end reads. Hence use Trimmomatic to cut adapters [ check for trimming parameters ] [ Tips for filename ]
Input : directory good with fastq or fastq.gz files
Execution:

sbatch -A <project> -D $PWD --mail-user <email_id> scripts/trimmo.sh good trimmed_reads

Output : directory trimmed_reads

{ Run step 1 review effect of trimming }

Sortmerna.sh [ We can also execute this at the very beginning (optional) ]
Sometimes ribosomal or any other unwanted RNAs may present in the library. Sortmerna could be used to filterout them.
Input: good
Execution:
```
sbatch -A <project> -D $PWD --mail-user <email_id> scripts/sortmerna.sh good sortMeRna   
```
Output: sortMeRna, the folder contains many different types of file. Fastq/fq files starting with non_Rna will be used in downstream analysis. Files with .log will be used by multiqc to summarize. The "rRna" fastq/fq and ".sam" files are removed by default from sortMeRna before next step. To retailn these files comment out "rm -rf $2/rRna_*{fastq,fq}"

Execution:
```
sbatch -A <project> -D $PWD --mail-user <email_id> scripts/compress_fastq.sh sortMeRna  
```
Output: sortMeRna

Now summarize the presence of rRNA.
Execution:
```
sbatch -A <project> -D $PWD --mail-user <email_id> scripts/fastqc.sh sortMeRna  
```
Output: sortMeRna

Alignment

Depending upon the library preparation kit the parameters of alignment software need to set. Here are few examples of different popular library kits. [ please report any missing library type and parameters]

To align to a reference genome

STAR:
Confirm the parameters in file 4-rna-seq.config
- "maxReadLength" to maximum read length
- "genome_file" to path to reference genome
- "gene_annotation" path to gtf file
Input: folder which contains the filtered reads; ex. good or sortMeRna
Execution:
```
sbatch -A <project> -D $PWD --mail-user <email_id> scripts/star.sh good star_alignment   
```
Output: star_alignment (contains bam files and quality report star_alignment.html)

Counting

htseq-count
[ STAR can also give count values of htseq-count’s default parameter but htseq-count will be used separately]
Confirm the parameters in file 4-rna-seq.config
- "stranded" depending upon the library type
- "gene_annotation" path to gtf file
Input: star_alignment
Execution:
```
sbatch -A <project> -D $PWD --mail-user <email_id> scripts/star_htseq-count.sh star_alignment star_count   
```
Output: count values at star_count/htseq_*txt and quality report at star_count.html

Final report

Till now we have generated multiqc reports for every command or folder. Now to summarize all in one place execute. Edit multiqc configuration file if requires

sbatch -A <project> -D $PWD --mail-user <email_id> scripts/multiqc_slurm.sh

EXTRA

Alignment read viewer

Need to sort (uncomment for tophat output bams) and index.

sbatch -A <project> -D $PWD --mail-user <email_id> scripts/samtools_index.sh bam_directory

Concatenating fastq files

There are, some times, multiple copies of same sample from multiple runs. It will be easier to concatenate all these multiple copies in single fastq.gz files before starting any workflow or 4-RNA-seq. Let us assume there are two samples control_1 and treated_1 and they have two copies from two separate runs, run_1 and run_2. Therefore, in the project directory there should have two sub-directories

run_1
- control_1_run2019.fastq.gz
- treated_1_run2019.fastq.gz
run_2
- control_1_run2020.fastq.gz
- treated_1_run2020.fastq.gz

Additionally another file, such as, sample_names.txt containing each sample names in a separate line. In this case the sample file should look like this

control_1  
treated_1

Now we can use

sbatch -A <project> -D $PWD --mail-user <email_id> scripts/cat.gz.sh sample_names.txt rawReads

cat.gz.sh will search any fastq.gz files matching with names given in sample_names.txt in ALL sub-directories and concatenate them. The output files can be found in rawReads directory.

Compressing fastq files

sbatch -A <project> -D $PWD --mail-user <email_id> scripts/compress_fastq.sh old_data

Cufflink

sbatch scripts/cuffdiff_batch.sh Derm Ctrl Fgf20 star-genome_annotated

rmurray2 / 4-RNA-seq