AWP-ODC-FDQ - Revision history

Maechlin at 22:42, 2 July 2015

2015-07-02T22:42:58Z

Maechlin at 20:22, 2 July 2015

2015-07-02T20:22:39Z

Maechlin at 19:26, 2 July 2015

2015-07-02T19:26:56Z

Maechlin at 19:20, 2 July 2015

2015-07-02T19:20:18Z

Maechlin at 18:50, 2 July 2015

2015-07-02T18:50:06Z

Maechlin at 00:32, 2 July 2015

2015-07-02T00:32:17Z

Maechlin at 00:31, 2 July 2015

2015-07-02T00:31:00Z

Maechlin at 00:18, 2 July 2015

2015-07-02T00:18:49Z

Maechlin at 00:11, 2 July 2015

2015-07-02T00:11:21Z

Maechlin at 23:57, 1 July 2015

2015-07-01T23:57:58Z

@@ Line 212: / Line 212: @@
    - BLOCK_SIZE_Z divides NZ
    - BLOCK_SIZE_Y * BLOCK_SIZE_Z = 512
 == See Also ==

@@ Line 214: / Line 214: @@
 == See Also ==
 *[[High-F Project]]
 *[[Main Page]]

@@ Line 54: / Line 54: @@
    --MEDIASTART 0 --READ_STEP 91 \
    --NTISKP 10 --WRITE_STEP 10 \
-   --FAC 1.0 --Q0 150.0 -EX 0.6 --FP 2.5 \
+   --FAC 1.0 --Q0 150.0 --EX 0.6 --FP 2.5 \
    --INSRC FAULTPOW \
    --NSKPX 2 --NSKPY 2 \

← Older revision		Revision as of 19:20, 2 July 2015
Line 59:		Line 59:
	>> "$fname"		>> "$fname"
	echo "ENDING `date`" >> "$fname"		echo "ENDING `date`" >> "$fname"
		+
		+	== Current Result ==
		+
		+	maechlin@h2ologin1:~/fdq_awpodc> more *.err
		+	_pmiu_daemon(SIGCHLD): [NID 23046] [c7-0c0s3n0] [Thu Jul 2 14:10:29 2015] PE RA
		+	NK 0 exit signal Segmentation fault
		+	[NID 23046] 2015-07-02 14:10:29 Apid 25377581: initiated application termination
		+	maechlin@h2ologin1:~/fdq_awpodc> more *.out
		+	----------------------------------------
		+	Begin Torque Prologue on nid27634
		+	at Thu Jul 2 14:10:24 CDT 2015
		+	Job Id: 1950564.nid11293
		+	Username: maechlin
		+	Group: PRAC_jmz
		+	Job name: chino_hills_gpu
		+	Requested resources: neednodes=4:ppn=1:xk,nodes=4:ppn=1:xk,walltime=01:30:00
		+	Queue: normal
		+	Account: jmz
		+	End Torque Prologue: 0.024 elapsed
		+	----------------------------------------
		+	maechlin@h2ologin1:~/fdq_awpodc> more *.tmp
		+	STARTING Thu Jul 2 14:10:26 CDT 2015
		+
		+	rank=0) RS=91, RSG=91, NST=91, IF=1
		+
		+	0 = (0,0)) NX,NY,NZ=224,224,1024
		+	nxt,nyt,nzt=112,112,1024
		+	rec_N=(112,112,1)
		+	rec_nxt,=56,56,1
		+	NBGX,SKP,END=(1:2:223),(1:2:223),(1:1:1)
		+	rec_nbg,ed=(0,110),(0,110),(0,0)
		+	disp=0
		+
		+	rank=1) RS=91, RSG=91, NST=91, IF=1
		+
		+	1 = (0,1)) NX,NY,NZ=224,224,1024
		+	nxt,nyt,nzt=112,112,1024
		+	rec_N=(112,112,1)
		+	rec_nxt,=56,56,1
		+	NBGX,SKP,END=(1:2:223),(1:2:223),(1:1:1)
		+	rec_nbg,ed=(0,110),(0,110),(0,0)
		+	disp=25088
		+
		+	rank=3) RS=91, RSG=91, NST=91, IF=1
		+
		+	3 = (1,1)) NX,NY,NZ=224,224,1024
		+	nxt,nyt,nzt=112,112,1024
		+	rec_N=(112,112,1)
		+	rec_nxt,=56,56,1
		+	NBGX,SKP,END=(1:2:223),(1:2:223),(1:1:1)
		+	rec_nbg,ed=(0,110),(0,110),(0,0)
		+	disp=25312
		+
		+	rank=2) RS=91, RSG=91, NST=91, IF=1
		+
		+	2 = (1,0)) NX,NY,NZ=224,224,1024
		+	nxt,nyt,nzt=112,112,1024
		+	rec_N=(112,112,1)
		+	rec_nxt,=56,56,1
		+	NBGX,SKP,END=(1:2:223),(1:2:223),(1:1:1)
		+	rec_nbg,ed=(0,110),(0,110),(0,0)
		+	disp=224
		+	filetype size (supposedly=rec_nxtnytnztWS4=125440) =125440
		+	rank=0, x_rank_L=-1, x_rank_R=2, y_rank_F=-1, y_rank_B=1
		+	Before inisource
		+	After inisource. Time elapsed (seconds): 0.003165
		+	rank=0, source rank, npsrc=1
		+	rank=1, x_rank_L=-1, x_rank_R=3, y_rank_F=0, y_rank_B=-1
		+	rank=2, x_rank_L=0, x_rank_R=-1, y_rank_F=-1, y_rank_B=3
		+	rank=3, x_rank_L=1, x_rank_R=-1, y_rank_F=2, y_rank_B=-1
		+	Before inimesh
		+	tau: 5.420455e-03,1.111455e+00; 3.223022e+00,1.321751e-01; 9.346188e+00,4.558041
		+	e-02; 1.571835e-02,3.832840e-01
		+	After inimesh. Time elapsed (seconds): 0.045935
		+	Application 25377581 exit codes: 139
		+	Application 25377581 exit signals: Killed
		+	Application 25377581 resources: utime ~0s, stime ~1s, Rss ~414872, inblocks ~789
		+	, outblocks ~1277
		+	ENDING Thu Jul 2 14:10:30 CDT 2015

@@ Line 50: / Line 50: @@
    fname="O.$now.tmp"
    echo "STARTING $now" >> "$fname"
-   aprun -n 4 -S 1 ./pmcl3d --NX 224 --NY 224 -Z 1024 -x 2 -y 2 \
+   aprun -n 4 -N 1 ./pmcl3d --NX 224 --NY 224 -Z 1024 -x 2 -y 2 \
    --TMAX 20.0 --DH 200.0 --DT 0.01 \
-  --NSRC 1 --NST 91 \
+   --MEDIASTART 0 --READ_STEP 91 \
    --NTISKP 10 --WRITE_STEP 10 \
-   --FL 0.005 --FH 5.0 --FP 2.5 \
+   --FAC 1.0 --Q0 150.0 -EX 0.6 --FP 2.5 \
-   --NEDZ 1 --INSRC FAULTPOW --INVEL mesh256 \
+   --INSRC FAULTPOW \
    --NSKPX 2 --NSKPY 2 \
    >> "$fname"
    echo "ENDING `date`" >> "$fname"
 == Changing Default Blue Waters Environment ==

@@ Line 113: / Line 113: @@
 == Location of Code ==
 == Notes about FDQ Version ==
 Note that there are some structural changes from the cpu code. The GPU code doesn't use the input parameter file anymore, all parameters are specified in the run script (if different from default values defined in the code). Also see the instructions below.

← Older revision		Revision as of 23:57, 1 July 2015
Line 2:		Line 2:

	== PBS Script ==		== PBS Script ==
		+	maechlin@h2ologin3:~/fdq_awpodc> more fdq_bw.pbs
		+
		+	#!/bin/bash
		+	###
		+	### PBS script for submitting FDQ on Blue Waters
		+	###
		+	### Set the number of nodes
		+	### Set the number of PEs per node
		+	#PBS -l nodes=4:ppn=1:xk
		+	###
		+	### Set the wallclock time
		+	###
		+	#PBS -l walltime=01:30:00
		+	###
		+	### Set the job name
		+	###
		+	#PBS -N chino_hills_gpu
		+	###
		+	### Set the job stdout and stderr
		+	###
		+	#PBS -e $PBS_JOBID.err
		+	#PBS -o $PBS_JOBID.out
		+	###
		+	### Set the Queue
		+	###
		+	#PBS -q normal
		+	###
		+	### Set the Allocation
		+	###
		+	#PBS -A jmz
		+	###
		+	### Set the Email (Beginning, End, Abort)
		+	###
		+	#PBS -m bea
		+	#PBS -M maechlin@usc.edu
		+
		+	###
		+	### Load specific modules
		+	###
		+	module swap PrgEnv-cray PrgEnv-gnu
		+	module load cudatoolkit
		+	module unload darshan
		+
		+	cd $PBS_O_WORKDIR
		+
		+	now=`date`
		+	fname="O.$now.tmp"
		+	echo "STARTING $now" >> "$fname"
		+	aprun -np 4 ./pmcl3d --NX 224 --NY 224 -Z 1024 -x 2 -y 2 \
		+	--TMAX 20.0 --DH 200.0 --DT 0.01 \
		+	--NSRC 1 --NST 91 \
		+	--MEDIASTART 0 \
		+	--READ_STEP 91 \
		+	--NTISKP 10 --WRITE_STEP 10 \
		+	--FL 0.005 --FH 5.0 --FP 2.5 \
		+	--NEDZ 1 --INSRC FAULTPOW --INVEL mesh256 \
		+	--NSKPX 2 --NSKPY 2 \
		+	>> "$fname"
		+	echo "ENDING `date`" >> "$fname"

	== Changing Default Blue Waters Environment ==		== Changing Default Blue Waters Environment ==