UpalabdhaD / GATK_parallel

Development of short tutorials for UNM's Center for Advanced Research Computing

QuickBytes

Quickbytes are tutorials designed to help CARC users.

• Linux-Intro

• Running jobs

  • Logging in
  • SSH keys and Config file
  • Transferring data
  • PBS/TORQUE
  • Sample PBS script
  • Submitting jobs
  • Check running jobs
  • Managing modules
  • Intro to Slurm
  • Converting PBS to Slurm
  • Intro to Slurm accounting at CARC
  • GNU Parallel

• Applications software

  • MATLAB
    • Running MATLAB jobs at CARC
    • Parallel MATLAB Server
    • Parallel MATLAB batch submission
    • Using GPUs with MATLAB
    • MATLAB Deep Learning on Xena
  • JupyterHub
    • JupyterHub: Parallel Processing with MPI
    • Conda python environments for JupyterHub
    • Using Julia in JupyterHub
  • Conda
    • General intro to Conda
    • Intro to Conda with example
    • Anaconda pip channels
  • R
    • R Programming in HPC
    • R at CARC
    • Running R in Parallel with Future
    • Gurobi optimizer with R
  • Machine Learning
    • Tensorflow
    • Machine Learning Conda Environments (including PyTorch K40 GPU libraries)
    • Tensorflow with multiple GPUs
    • Parallelization with JupyterHub using Dask and SciKit-learn
  • Bioinformatics
    • Genomic variant calling with GATK
    • Genome evaluation with QUAST and BUSCO
    • Single genome demographic history with PSMC
    • Stacks for RAD-Seq Data
    • Metabarcoding with QIIME2, Mothur, and USEARCH
    • BEAST at CARC
    • Population genetic simulations with msprime (backwards time)
  • Computational Chemistry
    • Orca on Wheeler and Taos
  • Computational Immunology
    • SimCov on Wheeler
  • Astronomy
    • CASA Radio Astronomy
  • Paraview
    • Paraview Gibbs and Wheeler
    • Paraview Hopper
  • Docker and Singularity
  • Haskell at CARC
  • X11 Forwarding
  • Spark
  • CuPy - Cuda in Python
  • Installing Perl Libraries to Your Home Directory

• Workshop Slides
  These slides were used in CARC workshops. They are provided for your convenience. Github does a poor job rendering PDFs. We recomment downloading the presentations.

  Current workshop schedule

  • Beginner Introduction: Basic Linux and Interactive SLURM
  • Intermediate Introduction: High Performance Computing and Batch SLURM
  • Biology 4/519: Biodiversity Informatics
  • Chemistry 567/Physics 581/Electrical and Computer Engineering 595: Quantum Computing for Quantum Chemistry
  • Computer Science 4/542: Introduction to Parallel Processing
  • Computer Science 523: Complex Adaptive Systems
  • Computer Science/Math 471: Introduction to Scientific Computing
  • Environmental Sciences 352: Global Climate Change
  • Earth and Planetary Sciences 400/522: Computational Methods for Geoscience
  • CARC Annual Meeting: Machine Learning
  • PBS, Modules, GNU Parallel, Conda, and MPI
  • Intermediate Intro to SLURM with NAMD