Installing mamba within your GEP environment is recommended, as it makes conda package managing/installing quicker. To do so, run the following command after the GEP environment is installed and activated:
```
conda install -c conda-forge mamba
```
If you do not install mamba, you will need to make sure the following flag is included in your snakemake command at the time of execution: `--conda-frontend=conda`
***Note** *If you already have a snakemake (or suitable Python) installation and would like to avoid installing again, ensure that all of the above software are in your `PATH`. If you do this instead of installing from the provided GEP environment (`installGEP.yaml`), you will still need at least the base conda installed/activated - as it's required to handle software dependencies of all the tools used within the workflow itself*
<br>
...
...
@@ -308,7 +317,7 @@ keep-going: False
snakemake --profile SUBMIT_CONFIG/local/
```
#### SLURM
#### HPC (SLURM)
Modify the slurm snakemake parameters in the file located at [GEP/SUBMIT_CONFIG/**slurm**/config.yaml](GEP/SUBMIT_CONFIG/slurm/config.yaml). You can set your desired `partition` and `qos` under `default-resources:`, as well as any snakemake parameters.
...
...
@@ -318,8 +327,31 @@ Modify the slurm snakemake parameters in the file located at [GEP/SUBMIT_CONFIG/
snakemake --profile SUBMIT_CONFIG/slurm/
```
##### Running using singularity container (only on HPC)
GEP has been containerized using docker. Within this container, snakemake will install the necessary software stacks as contained conda environments (separate to those installed during *normal* execution). All you have to do is make sure singularity is installed (often installed by your respective HPC IT team), and in your PATH:
e.g.
```
module load Singularity
```
And executed by adding `--use-singularity` and `--singularity-args "--bind <path1>,<path2> --workdir <userTempDirectory>" --contain`
Where the `--bind` paths are any location/s in your system that you require singularity to be able to ***recursively*** access, commonly your /scratch/username/ directory where both your raw data and results are/will be stored. Multiple paths can be given separated by a comma.
The `--workdir` is the singularity tmp directory. Often on HPC systems this will be automatically set as `/tmp/...`, but this can lead to issues during initial installation of the container as this default `tmp` directory may be too small. Instead, provide a path to a new directory to be used as the `tmp` for snakemake + singularity.
The `--contain` flag ensures that the container is completely kept separate from the host systems files so that the packages being installed are not shared with those that already exist (or are in the host system's conda cache).
For those not aware, you can run GEP with `nohup` and `&` added to the command (both local and slurm)
For those not aware, you can run GEP with `nohup` and `&` added to the command (both local and slurm, and with or without singularity). `nohup` will run the snakemake command in a way that cannot be interrupted when you lose connection to the server/cluster. The trailing `&` will simply run the command in the background of your current terminal, allowing you to freely use the terminal instance in the meantime.
