Difference between revisions of "AWP-ODC-FDQ"
From SCECpedia
Jump to navigationJump to search| Line 2: | Line 2: | ||
== PBS Script == | == PBS Script == | ||
| + | maechlin@h2ologin3:~/fdq_awpodc> more fdq_bw.pbs | ||
| + | |||
| + | #!/bin/bash | ||
| + | ### | ||
| + | ### PBS script for submitting FDQ on Blue Waters | ||
| + | ### | ||
| + | ### Set the number of nodes | ||
| + | ### Set the number of PEs per node | ||
| + | #PBS -l nodes=4:ppn=1:xk | ||
| + | ### | ||
| + | ### Set the wallclock time | ||
| + | ### | ||
| + | #PBS -l walltime=01:30:00 | ||
| + | ### | ||
| + | ### Set the job name | ||
| + | ### | ||
| + | #PBS -N chino_hills_gpu | ||
| + | ### | ||
| + | ### Set the job stdout and stderr | ||
| + | ### | ||
| + | #PBS -e $PBS_JOBID.err | ||
| + | #PBS -o $PBS_JOBID.out | ||
| + | ### | ||
| + | ### Set the Queue | ||
| + | ### | ||
| + | #PBS -q normal | ||
| + | ### | ||
| + | ### Set the Allocation | ||
| + | ### | ||
| + | #PBS -A jmz | ||
| + | ### | ||
| + | ### Set the Email (Beginning, End, Abort) | ||
| + | ### | ||
| + | #PBS -m bea | ||
| + | #PBS -M maechlin@usc.edu | ||
| + | |||
| + | ### | ||
| + | ### Load specific modules | ||
| + | ### | ||
| + | module swap PrgEnv-cray PrgEnv-gnu | ||
| + | module load cudatoolkit | ||
| + | module unload darshan | ||
| + | |||
| + | cd $PBS_O_WORKDIR | ||
| + | |||
| + | now=`date` | ||
| + | fname="O.$now.tmp" | ||
| + | echo "STARTING $now" >> "$fname" | ||
| + | aprun -np 4 ./pmcl3d --NX 224 --NY 224 -Z 1024 -x 2 -y 2 \ | ||
| + | --TMAX 20.0 --DH 200.0 --DT 0.01 \ | ||
| + | --NSRC 1 --NST 91 \ | ||
| + | --MEDIASTART 0 \ | ||
| + | --READ_STEP 91 \ | ||
| + | --NTISKP 10 --WRITE_STEP 10 \ | ||
| + | --FL 0.005 --FH 5.0 --FP 2.5 \ | ||
| + | --NEDZ 1 --INSRC FAULTPOW --INVEL mesh256 \ | ||
| + | --NSKPX 2 --NSKPY 2 \ | ||
| + | >> "$fname" | ||
| + | echo "ENDING `date`" >> "$fname" | ||
== Changing Default Blue Waters Environment == | == Changing Default Blue Waters Environment == | ||
Revision as of 23:57, 1 July 2015
AWP-ODC-FDQ is a version of the wave propagation code AWP-ODC that contains frequency dependent-Q physics modules. Currently we have a GPU version of this code.
Contents
PBS Script
maechlin@h2ologin3:~/fdq_awpodc> more fdq_bw.pbs
#!/bin/bash ### ### PBS script for submitting FDQ on Blue Waters ### ### Set the number of nodes ### Set the number of PEs per node #PBS -l nodes=4:ppn=1:xk ### ### Set the wallclock time ### #PBS -l walltime=01:30:00 ### ### Set the job name ### #PBS -N chino_hills_gpu ### ### Set the job stdout and stderr ### #PBS -e $PBS_JOBID.err #PBS -o $PBS_JOBID.out ### ### Set the Queue ### #PBS -q normal ### ### Set the Allocation ### #PBS -A jmz ### ### Set the Email (Beginning, End, Abort) ### #PBS -m bea #PBS -M maechlin@usc.edu ### ### Load specific modules ### module swap PrgEnv-cray PrgEnv-gnu module load cudatoolkit module unload darshan cd $PBS_O_WORKDIR now=`date` fname="O.$now.tmp" echo "STARTING $now" >> "$fname" aprun -np 4 ./pmcl3d --NX 224 --NY 224 -Z 1024 -x 2 -y 2 \ --TMAX 20.0 --DH 200.0 --DT 0.01 \ --NSRC 1 --NST 91 \ --MEDIASTART 0 \ --READ_STEP 91 \ --NTISKP 10 --WRITE_STEP 10 \ --FL 0.005 --FH 5.0 --FP 2.5 \ --NEDZ 1 --INSRC FAULTPOW --INVEL mesh256 \ --NSKPX 2 --NSKPY 2 \ >> "$fname" echo "ENDING `date`" >> "$fname"
Changing Default Blue Waters Environment
Default software modules are Cray. Change these to GNU
module unload PrgEnv-cray module load PrgEnv-gnu module load cudatoolkit module unload darshan
Makefile
CC = cc
CFLAGS = -O3 -Wall
GFLAGS = nvcc -O4 -Xptxas -dlcm=ca -maxrregcount=255 -use_fast_math --ptxas-options=-v -arch=sm_35
INCDIR = -I/opt/nvidia/cudatoolkit/5.5.20-1.0402.7700.8.1/include
OBJECTS = command.o pmcl3d.o grid.o source.o mesh.o cerjan.o swap.o kernel.o io.o
LIB = -lm -ldl -L/opt/nvidia/cudatoolkit/5.5.20-1.0402.7700.8.1/lib64 -lcudart -lmpich
pmcl3d: $(OBJECTS)
$(CC) $(CFLAGS) $(INCDIR) -o pmcl3d $(OBJECTS) $(LIB)
pmcl3d.o: pmcl3d.c
$(CC) $(CFLAGS) $(INCDIR) -c -o pmcl3d.o pmcl3d.c
command.o: command.c
$(CC) $(CFLAGS) $(INCDIR) -c -o command.o command.c
io.o: io.c
$(CC) $(CFLAGS) $(INCDIR) -c -o io.o io.c
grid.o: grid.c
$(CC) $(CFLAGS) $(INCDIR) -c -o grid.o grid.c
source.o: source.c
$(CC) $(CFLAGS) $(INCDIR) -c -o source.o source.c
mesh.o: mesh.c
$(CC) $(CFLAGS) $(INCDIR) -c -o mesh.o mesh.c
cerjan.o: cerjan.c
$(CC) $(CFLAGS) $(INCDIR) -c -o cerjan.o cerjan.c
swap.o: swap.c
$(CC) $(CFLAGS) $(INCDIR) -c -o swap.o swap.c
kernel.o: kernel.cu
$(GFLAGS) $(INCDIR) -c -o kernel.o kernel.cu
clean:
rm *.o
