# PGAM on High Performance Computing: Efficient estimation of neural tuning during naturalistic behavior
----------------
Here we provide a singulaity image that allows to easily deploy the PGAM based estimation of neural tuning fuctions in HPC settings. The image is the singularity counterpart of the dokcer image https://hub.docker.com/r/edoardobalzani87/pgam version 1.0.
# Table of contents
----------
* **Resources**
* **Running the ".sif" image in a singularity container**
* **SLURM job script for parallelized PGAM fits**
# Resources
The image facilitates setting up the PGAM library, available on github at https://github.com/BalzaniEdoardo/PGAM;
We refer to the paper,
https://proceedings.neurips.cc/paper/2020/hash/94d2a3c6dd19337f2511cdf8b4bf907e-Abstract.html,
for additional details on the theory and implementation of the PGAM parameter estimation.
We refer to the jupyter notebook https://github.com/BalzaniEdoardo/PGAM/blob/master/PGAM%20Tutorial.ipynb
for a practical introduction to the model implementation, and the parameter selection.
For NYU greene users, see https://sites.google.com/nyu.edu/nyu-hpc/hpc-systems/greene/software/singularity-with-miniconda for a tutorial on how to work with singularity container and miniconda, where to find overlays and "sif" images.
# Running the ".sif" image in a singularity container
In this session we will show how to run a script that imports the **GAM_library** through a singularity container.
Setting up the container requires uploading the **pgam_1.0.sif** image into your HPC folders, and an singularity overlay file system.
For NYU greene HPC users, overlays can be copied and unzipped from the folder **/scratch/work/public/overlay-fs-ext3**, while the PGAM image is readily available at **/scratch/work/public/singularity/pgam_1.0.sif**.
Let's suppose that we have the following script named *import_pgam.py* which simply imports the pgam libary,
```python
import GAM_library as gml
print('\n################################')
print('Succesfully imported GAM_library!\n\n')
```
In order you can run the code via the singularity container with the following,
```sh
singularity exec --overlay \
<path to overlay>:ro \
<path to sif>/pgam_1.0.sif \
/bin/bash -c "python <path to script>/import_pgam.py"
```
## SLURM job script for parallelized PGAM fits
Let's suppose that we have a script *my_fit.py* that takes as an impput a job id, and fits the corresponding neuron.
Below an example of SLURM job script, **run-job.sbatch**, that runs 100 fits,
*run-job.sbatch*:
```sh
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=1
#SBATCH --time=1:00:00
#SBATCH --mem=8GB
#SBATCH --job-name=fit-pgam
#SBATCH --array=1-100
module purge
singularity exec --overlay \
<path to overlay>:ro \
<path to sif>/pgam_1.0.sif \
/bin/bash -c "python <path to script>/my_fit.py $IID"
```
The PGAM repo provides a standardized pipeline for fitting PGAMs which is described in the tutorial https://github.com/BalzaniEdoardo/PGAM/blob/master/Standardized_PGAM_Fit_with_examples.ipynb.
The script
https://github.com/BalzaniEdoardo/PGAM/blob/master/demo/fit_from_config.py could replace *my_fit.py* in **run-job.sbatch**, provided that the configuration files and inputs are structured appropriately.
