Recent Changes - Search:
Main /

JobSubmission

Two cluster resources are available to members of the gruop. The first is a group cluster named "watt" abailable to only those a part of Knotts Group. The second is the Fulton Supercomputer available to account holders. The supercomputer is much faster than watt, but has walltime constraints.

Specific commands will vary between scheduler, but all scripts need to specify:

  • Walltime- Max time job will need to run
  • CPU usage specifications- How many cores you will need across all nodes
  • Memory- How much memory (RAM) you will need per core
  • Job name

Submitting jobs to either cluster requires you to first

  • Compile your code to create an executable.
  • Create the input files/folders needed to run the program.
  • Create the submission script.
  • Submit the job to the cluster.

Compiling the code and generating the input file are discussed on other pages. This page details creating the submission script and submitting your job.

FSL Supercomputer Submission

The FSL supercomputer uses a SLURM scheduler. The job submission script is a shell script. It can have any desired name but is commonly something like submit.sbatch. The scheduler read the script's comments for job parameters and executes the following commands. \\ There are many different ways to properly set up a job, listed below are the basics. Following the instructions is a list of additional command you can use for job-specific purposes.

  1. Take a copy of the submit_to_supercomputer_template.sbatch file script and put it in the same folder as your INPUT and executable
    • The template is located in fslg_knotts/thecode_scripts/
  2. Rename the script to something more job-specific
  3. Open the script in vim and change the settings to match your simulation parameters
    • #SBATCH --time=01:00:00 Note: sets the walltime
      • For reference I have found that a ~1250 amino acid protein runs about 1,000,000 steps/hr, estimate high, adjust as needed
    • #SBATCH --ntasks=1 Note: for RepX ntasks should equal your total number of boxes
    • #SBATCH --mem-per-cpu=128M Note: this should be good for most situations, adjust when needed
    • #SBATCH -J "job-name" Note: you use this name to reference this job while it is running.
  4. Make sure the executable command references the right executable and (if applicable) input files.
  5. In bash window, submit the job when ready
    • $ sbatch YourScriptName.sbatch

You can see the status of your jobs using:
$ squeue -u username-on-fsl

Watt Submission

Watt uses a job scheduler named torque. The basics of using torque are below. You can find a lot more by searching the web. The job submission script is a shell script. It can have any desired name but is commonly something like submit.sub or submit.sh. \\ There are many different ways to properly set up a job, listed below are the basics. Following the instructions is a list of additional command you can use for job-specific purposes.

  1. Take a copy of the submit_to_watt_template.sub file script and put it in the same folder as your INPUT and executable
    • The template is located in SomePath/TBD/
  2. Rename the script to something more job-specific
  3. Open the script in vim and change the settings to match your simulation parameters
    • #PBS -l nodes=1:ppn=1,pmem=500mb,walltime= --time=01:00:00
      • nodes: refers to the number of physical machines that you want. See Computer Resources for a description of the types of machines available on watt.
      • ppn: refers to the number of processors requested on this machine. More specifically, this is the number of cores you want. The total number of cores requested will be nodes*ppn.
      • For example, nodes in the hex queue have two, hex core processors for a total of 24 core. If you wanted to run a job on 36 processors, you could do so by setting nodes=3:ppn=12.
      • pmem=100mb is the amount of memory you are requesting. For our jobs, memory usually isn't the limitation. Our jobs usually require much less than 1 GB. All the machines on watt have at least 1 GB per core.
      • walltime is how long your job will run in HH:MM:SS. You want to give your very best prediction of how long the jobs will run. If you don't give it a long enough time, your job will terminate before the simulation is over. It is better to over predict than underestimate.
    • #PBS -N name-of-job Note: you use this name to reference this job while it is running.
    • #PBS -q hex Note: specifies that the job should be run on the hex queue. This can be batch, dual, quad, hex, or gpu
  4. Make sure the executable command references the right executable and (if applicable) input files.
  5. In bash window, submit the job when ready
    • $ qsub YourScriptName.sbatch

You can see the status of your jobs using:
$ qstat -u username-on-watt