View source for Visualizing AWP-ODC Output

This page documents a procedure to visualize AWP-ODC velocity output, using remote Paraview.  So far, this procedure has only been tested on Titan using Paraview 4.4.0.

== Configure the simulation ==

When running the AWP-ODC simulation, make sure it is configured to produce velocity output.

Take note of the following parameters:
*NX, NY, NZ (and NBGX, NEDX, NGBY, NEDY, NGBZ, NEDZ if specified)
*WRITE_STEP
*NSKPX, NSKPY, NSKPZ
*NTISKP

They'll be needed when producing the output.

== Create a Paraview XDMF file ==

Once the simulation is complete, we must create a configuration file which tells Paraview how the velocity data is laid out in memory.

For this, run the script [http://source.usc.edu/svn/cybershake/import/trunk/AWP-ODC/visualizer/create_timeseries_file.py create_timeseries_file.py].
<pre>
Usage: ./create_timeseries_file.py <X dim in output> <Y dim in output> <grid decimation> <NT in sim> <DT of sim> <NTISKP> <timesteps output per file> <prefix> <output file>
</pre>

Based on this, a XDMF (XML) file is produced which can serve as input to Paraview.

== Create traces ==

Often, other visual references are needed to make the visualization meaningful.  The approach outlined uses a Cartesian grid projection to display the velocity results.  This means that to display other data, we have to convert the data from (lat, lon) into (X index, Y index).

To do this, use the code get_grid_values.c, in https://source.usc.edu/svn/cybershake/import/trunk/PreCVM/GenGrid_py/src/get_grid_values.c .  It takes a path to a gridfile generated from PreCVM; the modellat, modellon, and modelrot; an input CSV file in lat,lon format; a path to the output file; the grid spacing; and the xshift and yshift from the model_params file.  geoproj should always be set to 1, which is Rob's great circle projection.  Here's a sample invocation:

<pre>
./get_grid_values gridfile=gridfile_fwd modellat=35.70115 modellon=-119.75587 modelrot=-30.0 infile=CA_trace.csv outfile=ca_trace_grid.csv grid_spacing_km=0.1 xshift=-149.95001 yshift=-134.95001 geoproj=1
</pre>

The directory http://source.usc.edu/svn/cybershake/import/trunk/AWP-ODC/visualizer/ contains trace files for California, CCA, USGS, and cvms4/cvms4.26/cvms5 models in lat,lon format, ready for use with get_grid_values.

== Use local Paraview to connect to remote nodes ==

The easiest way to visualize the data is to leave it on the remote system, and use the client/server functionality of Paraview.  In this mode, Paraview connects to a remote system and uses the remote cluster to drive the rendering.

Follow the instructions at [https://www.olcf.ornl.gov/tutorials/running-paraview-on-titan/ Titan's Paraview guide], under "Interactive Mode", to create a connection.

== Load datasets ==

Once the job starts on Titan, you can begin loading data.

=== Velocity data ===

To load velocity data, go to File->Open and select the XDMF file you previously made.  It will appear in the 'Pipeline Browser' window in the upper left.  You may need to click on the eye next to it to get it to display. Additionally, the data is rendered on the Y/Z plane (there is a reason I had to set it up this way, but now I forget why), so click on the '-X' icon to the right of the Slice pull-down in the toolbar.  You should see the rectangle defined by your simulation region.  You can change the timestep you're looking at by changing the value in the text box to the right of "Time:".

=== Traces ===

Loading traces is more involved.  For each trace:

#Go to File->Open and select the .csv trace file you created earlier.  The .csv file should appear in the Pipeline Browser window on the left.
#On the Properties tab below the Pipeline Browser, de-select 'Have Headers' and click 'Apply'.  A spreadsheet will appear.
#Go to Filters->Alphabetical->Table to Points.  This will open a new filter, below your .csv file in the Pipeline Browser.
#In the Properties tab, change the Y Column pull-down to "Field 1", and the Z column pull-down to "Field 2".  Click "Apply".  You have now converted the .csv into a series of points.  You can close the spreadsheet.  If you click on the eye next to 'TableToPoints1' in the Pipeline Browser, you will see your outline appear, but as a series of points.  We will next convert it to line segments.
#Go to Filters->Alphabetical->Programmable Filter.  This filter allows you to write your own VTK code, in Python, to do whatever you like.  We will write code which will convert the series of points into line segments.
#Copy and paste the following into the 'Script' section of the Properties tab:
 pdi = self.GetPolyDataInput()
 pdo = self.GetPolyDataOutput()
 numPoints = pdi.GetNumberOfPoints()
 pdo.Allocate()
 for i in range(0, numPoints-1):
   points = [i, i+1]
   pdo.InsertNextCell(3,2,points)
#Click 'Apply'.  It may take a moment, depending on how many points you have.  You should now see the outline of the object.  If you don't, try toggling the eye.
#To change the color of the object, click on "ProgrammableFilter1", scroll down in the Properties tab, and under Coloring click the Edit button and pick a new color.  You can also change the Line Width in the Styling section.

Repeat this until all your traces have been loaded.

== Modify visuals ==

You may need to make additional modifications to your plot to make it look as you like.

=== Add timestamp ===

To display the timestamp on the plot, go to Filters->Temporal->Annotate Time Filter.  Click 'Apply' in the Properties tab to make it appear.  You can click and drag it on the Layout to move it around.  If you want to show fewer decimal places, change the 'Format' section in the Properties tab.

=== Change plot colors ===

I'm not sure how the default plot colors are selected, but they're usually not symmetric around 0 which isn't what we want.

To change the colors, click on the XDMF file in the Pipeline Browser.  The Color Map Editor will appear on the right side.

#Change the max and min to be symmetric around 0.  To determine what the max and min should be:
##Select a timestep about 10% of the way through the simulation.
##Click on the Information tab (next to the Properties tab).  It will show you what the Data Range is for velocity, an easy way to figure out how extreme your values are.
##I usually select a max and min about 10% of the global max and min.  I find this gives a good balance between too much saturation and not losing the small values, but you may prefer other settings.
#Click on the 'Rescale to Custom Data Range' button.  It's 2 buttons to the left of the pull-down that says 'X Velocity, with what looks like an arrow with a C.  Set your new max and min.
#If you want to change the color scheme, click on the 'Choose Preset' button in the Color Map Editor on the right.  It's the one with a heart on it.  Select a new color scheme.
#If you want to alter how quickly the color changes from neutral to red or blue, you can click on the horizontal bar (not the triangle part, that controls opacity) to add new reference points.  Then, you can edit the 'Color transfer function values' table to set new R,G,and B values for it.

=== Edit color legend ===

If you want to alter the color legend, click on the 'Edit Color Legend Parameters' button, near the top right of the Color Map Editor.  It's the button with an 'e' on it.

== Save state ==

Before starting your animation, save a state file.  That way, if there are any issues, you don't have to reload everything again.  Go to File->Save State.

In the future, you should reconnect before you try to load it.  It may take 15-20 seconds to load once you confirm the file paths.

== Create series of snapshots ==

The way that Paraview creates animations is by creating a series of snapshots, which you can then render into a movie.  To save a series of snapshots, go to File->Save Animation.  Check over the Animation Settings.  Note that you can set a custom Timestep range, so if you aren't able to finish rendering the snapshots in the length of your Titan job, you can resume from where you left off.

'''If the connection terminates in the middle, or your local machine goes to sleep, you may end up with rendered frames in which the content wasn't updated.  Check your last frames to be sure they have the correct timestamp.'''

I think the speed of rendering is a function of your internet connection, the number of remote nodes, and the image size.  It typically takes me 90-120 minutes to render a 2000-timestep animation at ~1280x800.  I use PNG files.

== Create movie ==

On Linux, you can use ffmpeg to string together the PNGs to create a movie, with a command like:
 ffmpeg -framerate <fps> -i fileprefix.%04d.png <output movie.mp4>