This repository has the code and data required to reproduce the figures for the article:
MethNet: a robust approach to identify regulatory hubs and their distal targets in cancer
To run the code you need to:
- If not available in your system, install snakemake
- Edit the configuration file
config/config.yaml
to match your computer setup - Execute the command
snakemake --profile config/config.yaml
from the directory where theSnakefile
is
This command should generate 3 html files under ./data/figures/
with all the
programmatically generated figures of the article.
The expected running times and other resources for each rule
is specified in the resources
field
and refer to the whole pipeline (all 24 TCGA studies). For testing/development users can specify fewer studies.
The pipeline was developed using:
- slurm executor on NYU's UltraViolet HPC cluster
- conda version
23.3.1
- mamba version
1.4.2
- snakemake version
8.5.3
annot/
: annotation for genes and CREshugo_hg19_pancan.tsv
: gene coordinates from xenahubshugo_hg19_pancan_augmented.tsv
: genes with annotations (biotype)intergenic_cre.txt
: (temporary file)meth-cluster_annotated.csv
: potential CRE with chromatin annotation datameth-cluster_loops.csv
: number of loops per potential CRE aggregated fromBhattacharyya_loops
meth-cluster_TFBS.csv
: transcription factors binding each potential CREmeth-cluster_beta_summary.csv
: mean and standard deviation of methylation status for each CpG cluster.methyl450k_hg19_ilmn12.csv
: annotation for CpG probe based on Illumina's manifest filewgEncodeBroadHmm_state_count.csv
: ChrommHMM states downloaded from Genome Browser.
experiments/
: (INPUT) data generated for this studyBhattacharyya_loops.csv.gz
: loops matrix compiled from FitHiChIP dataHiCUP_summary_report.tsv
: QC report for promoter-capture HiCpcChIP_loops.tsv
: loops recovered from promoter-capture HiCperturb_seq_results.h5
: CellRanger output from the analysis of perturb-seq experiment.protospacer_umi_thresholds.csv
: CellRanger output, threshold to detect sgRNA
figures/
: figures uses in the article (final output)genomic-networks/
: cre-gene adjacency network and CpG-clusteringabsolute_clusters_probes.tsv
gene_cgi_overlaps.csv
pcgene-cluster_complete_200bp_network_1e6.tsv
pcgene-probe450_distances_list.rds
pcgene-probe450_network_1e6.tsv
matrices/
:methyl450k
: methylation data (beta values) for each TCGA cancer. Rows = CpG probes, Columns = TCGA samplesmeth_clusters
: methylation data (beta values) for each TCGA cancer. Rows = TCGA samples, Columns = CpG clustersrnaseq
: gene expression data for each tCGA cancer. Rows = TCGA Samples, Columns = Genes
models_enet/
: intermediate results directory to store the glmnet models for each TCGA cancerpancan/
: matrices and clinical data for all TCGA cancers combined (PanCancer)results/
: main output of the pipelinemodels_enet_hyperparams.csv
: hyper-parameters (alpha and lambda) for each elastic-net modelmodels_enet_params.csv
: coefficients for each elastic-net modelmodels_enet_samples.csv
: TCGA patients used to fit each elastic-net models for each TCGA cancer.models_enet_clusters.csv
: CpG clusters used to fit each elastic-net models for each TCGA cancer.intercepts.csv
: intercept coefficients from elastic-net regression models.associations.csv
: all associations recovered from elastic-net regression models.methnet.csv
: aggregated associations across multiple cancers.cluster_score.csv
: score for intergenic CpG clusterscre_cox_glance.csv
: statistics of Cox-PH models fitted for each intergenic CpG clustercre_cox_params.csv
: coefficients of Cox-PH models fitted for each intergenic CpG clusterinteraction_examples.csv
: methylation and gene expression data to generate figure S2 and S3perturb-seq_results.csv
: interactions experimentally recovered from the perturb-seq assay
xenahubs/
: xenahub clones for GDC (gdc/
) and PanCancerAtlas (pancanatlas/
) hubs.scripts/download_xenahubs.sh
will clone both hubs but the pipeline only requires the following files:gdc/genomicMatrix/TCGA-{cancer}/Xena_Matrices/TCGA-{cancer}.methylation450.tsv
(where{cancer}
is a wildcard for all the TCGA projects required)pancanatlas/genomicMatrix/EB++AdjustPANCAN_IlluminaHiSeq_RNASeqV2.geneExp.xena.tsv
pancanatlas/clinicalMatrix/Survival_SupplementalTable_S1_20171025_xena_sp.tsv
pancanatlas/probeMap/hugo_gencode_good_hg19_V24lift37_probemap.tsv hugo_hg19_pancan.tsv