-----------------------------------------------------
README: Running Q-Chem with a job scheduler
-----------------------------------------------------

Users in most distributed computing environment are required to submit their jobs through a batch job scheduler. This is especially true for most parallel computers. We provide below some example scripts for PBS, a common scheduler, and Loadleveler on IBM-SP.

NOTE: These are just examples and should not be used as is. They are just a starting point for you to be able to create your own customized scripts.

1. Sample PBS batch job request file

#PBS -l nodes=4
#!/bin/csh
source $HOME/cshrc_mpi       # In order to get correct MPI enviornment
# please define your MPI path in this file
# If you have it in your .cshrc file then
# your don't need this line
#gmpiconf $PBS_NODEFILE   # For those of you with myrinet card there
# may be a similar file to set up the environment
cat $PBS_NODEFILE               # The nodes allocated must be in this file. Look at it
# just in case.
set NN = `cat $PBS_NODEFILE | wc -l`
setenv QC /home/user/qchem
setenv QCAUX $QC/aux
setenv ONEEXE -DONEEXE
setenv QCLOCALSCR $PBSTMPDIR
setenv QCSCRATCH /scratch
cd $QC/samples/
qchem -np $NN DFT_glutamine.in DFT_glutamine.out   # Run qchem on NN processors

2. Sample loadleveler scripts for IBM_SP users

In order to run Q-Chem on an IBM/SP a LoadLeveler script is required. The following is an example for a LoadLeveler script. This should NOT be taken as it is. It most likely won't work, because many parameters depend on the setup of your system. It is strongly suggested you consult your system administrator for how to run q-chem on your system.

A loadLeveler script for Parallel QCHEM with dynamic load balancing (the default distribution, uses IBM LAPI library):

#!/usr/bin/csh
#@ job_name = ppkjob
#@ account_no = mp54
#@ output = $(jobid).output.sp
#@ error = $(jobid).error.sp
#@ job_type = parallel
#@ environment = COPY_ALL;ONEXE=-DONEEXE;
#@ network.LAPI = csss,not_shared,us
#@ node_usage = not_shared
#@ class = regular
#
#@ notification = never
#@ node = 1
#@ tasks_per_node = 2
#@ wall_clock_limit= 01:00:00
#
#@ queue
setenv QC $HOME/qchem
setenv QCAUX $HOME/aux2
setenv QCSH "/usr/bin/csh -f"
setenv QCSCRATCH /tmp/ppko
setenv QCLOCALSCR $SCRATCH
echo "for the starting node"
echo $SCRATCH

env | grep MP_

cd $QC/samples
$QC/bin/qchem -np 2 test.in test.out_lapi
foreach node ($LOADL_PROCESSOR_LIST)
echo $node
echo $SCRATCH
end

A loadleveler script for Parallel QCHEM with MPI only (static load balancing):

#!/usr/bin/csh
#@ job_name = ppkjob
#@ account_no = mp54
#@ output = ppk.out_MPI
#@ error = ppk.err_MPI
#@ job_type = parallel
#@ environment = COPY_ALL;ONEXE=-DONEEXE;
#@ network.MPI = css0,not_shared,us
#@ node_usage = not_shared
#@ class = debug
#
#@ notification = never
#@ node = 1
#@ tasks_per_node = 2
#@ wall_clock_limit= 00:30:00
#
#@ queue
setenv QCAUX $HOME/aux
setenv QCSH "/usr/bin/csh -f"
setenv QCSCRATCH /tmp/ppko
setenv QCLOCALSCR $SCRATCH
setenv QC /scratch/scratchdirs/ppko/qchem.par
echo "for the starting node"
echo $SCRATCH

env | grep MP_

cd $QC/samples

$QC/bin/qchem -np 2 test.in test.out_mpi
foreach node ($LOADL_PROCESSOR_LIST)
echo $node
echo $SCRATCH
end