AWP ODC GPU

Yifeng Cui, Kim Olsen, Efecan Poyraz, and others on SCEC projects have developed SCEC's first major GPU software. This group is running a full scale Chino Hills validation ground motion simulation, at size of 3500x2500x1500 problem size, 16m resolution, we used 250 GPUs on Keeneland, the initial test went well. We are seeing 55x speedup comparing to the CPU code for the same setting, we know Keeneland may have some scaling issue when using high number of nodes, plus this simulation has serious I/O - around 0.5TB, requiring frequent data transfer from GPU to CPU through PCIe. However, we are pretty happy with the performance getting so far. One issue with current GPU code is that we don't have validation prints about velocity max/min value, for that the simulation results couldn't be validated at run time. We are working with our visualization expert to analyze the initial results, and also work with seismologist Kim Olsen. Once everything is set, we'll make reservation next week or so for a full size run which possibly take 5 hours to complete.