Capstone Week 30

Total Time Spent: 10hrs

Investigating GNUplot (2hr):
Initially, I had no idea how GNUplot (my graphing library of choice) worked.  My first goal was to figure out how to get the program to work from my C++ program.  After some inital googling, I found two C++ interfaces for the program, gnuplot-cpp and gnuplot-iostream.  The first is a simple C++ mask for the command line interface for GNUplot, while the second provided a significantly more involved interface.  Both documentation bodies indicated that gnuplot-cpp was the easier of the two to learn.  Unfortunately, this still meant that I had to learn the command line interface for GNUplot.

gnuplot-cpp provides an interactive command line interface, as well as the ability to run batch files.  It seemed that the thing to do was to learn the batch files, and have the C++ main program call them.  Upon further review, it turns out the library provided a cmd(string cmd) method that essentially dumped  a cmd to the command line through unix pipes.  The advantage here is that data can be written to a file, then pass that filename to the batch file/command line for execution.  My data comes out of a netcdf file as a series of z values, separated by spaces and newlines (for columns and rows).  This is super convenient.

gnuplot-iostream required me to reformat the data into a trio of x,y,z arrays.  This is rather unpleasant as it requires me to think a little bit and preform some data manipulation.  Additionally, the api calls required didn’t look like they lent themselves to generating the “map” style graphs I want.

I made the not-so-difficult call and went with gnuplot-cpp.
Ubuntu Partition Resizing: (1hr)
I’ve been using my current Ubuntu box for quite a while, and have several environments configured for testing (R, C++, python) and a bajillion libraries installed.  Long story short, I ran out of space on my partition.  I had tried installing doxygen so that I could generate the documentation for my gnuplot-cpp api, but the installation failed when I ran out of disk space.

Initial googling showed that it was a simple matter of running VBoxManage on my disk and expanding the partition.  It didn’t work out that way.  When I tried running the command, I recieved an error stating that the Vitrual Box communication server wasn’t running.  Google indicated that I needed to run the program with administrative privlidges.  Since I was running this from the command line, I figured elevating the command prompt would solve my problems.  Running the command with elevated permissions gave me the same error.  Oddly, elevating the VBoxManage.exe to run as admin, and then running it from a standard command prompt worked.  Sort of.  This time I got an error indicating that VirtualBox couldn’t resize a partition with the given format.  Further googling revealed that this was due to the fact that I had specified a fixed-sized partition when I created the drive.  GOOD JOB PAST-GREG…
Anyway, I found the following two articles which satisfactorily fixed my issue:

The basic idea was to clone the fixed size drive into a dynamically sized drive, then run GParted (a linux disk utility on a recovery disk iso) to move the swap space/secondary partition and expand the primary partition.resize1

GNUpart implementation: (4hrs)
Looking into the GNUpart-cpp documentation (which was AMAZING by the way), I was able to figure out that I wanted to make a heat map using an image file format.  I worked off the examples given and was able to get a batch file working.  Essentially, I had two files, one for data and one for commands.  The data format, as explained earlier, was a simple whitespace separated matrix.  The command file is a series of simple statements that tells the program what to do.  For the most part, the program takes care of all the heavy lifting (but at the same time, the fine grained control).  For example, I can define a color pallate as a series of hex color values, and the system will evenly distribute those colors along the range of data file as a gradient.  This is useful for most cases, except when I want to provide certain levels of contrast.  For example, by passing the program three colors, it’ll make a pretty gradient, but makes it very difficult to see a hard contour for the shoreline (it gets lost in the smooth gradient).  One oddity I noticed was that the library flipped the y axis, so that the last line in the file was the top line in the graph.

Test Data!

Test Data!

With a working example in hand, I proceeded to try and implement the batch file in C++.  At first, I tried using the dedicated methods for setting various options (setTitle, setSize..etc), but then I took a look at the source code and realized it was simply masking calls to the cmd() function.  At that point, I gave up trying to use those functions and ran cmd() with formatted strings.

When I tried running my batch program from my C++ program using the cmd() function, I received a runtime error stating that x11 was an invalid line ending. As it turned out, I needed to install gnuplot-x11.  Fixing this error let my program compile and run, but it kept hanging (with no program output at all).  The cause was that the cmd() function wasn’t returning error messages from the gnuplot command line application.  In order to see those, I had to point my terminal at my C++ program’s executable and run it explicitly.  This revealed a myriad of errors with the commands I was passing the cmd() function.  Apparently the unix pipes only flow in one direction…
One notably strange error, was that setting the size of the resulting graph (a gif img) caused the program to hang indefinitely, despite it being a copy paste from the example code.  This sort of sucks as my high resolution maps (1000 cells x 1000 cells) get squished into a 400×600 graphic, losing a lot of fidelity in the process.
Additionally, I had to append semicolons to my command strings (which I didn’t have to do in my batch files).
NetCDF/GNUplot Integration: 2hrs
After getting my C++ version of GNUplot working, I tried graphing my NetCDF data to test it out.  I ran into errors almost immediately.  The first time I ran the program, I got an error message back that the data file was empty.  Turns out, I had chosen a section of the graph dominated by NAN values.  I replaced all the NAN values with 0 (a value Martin and Kevin had agreed was a good default), and then I got a blank graph back.  After choosing a better location, I found that my image was flipped upside down.  I had intentionally flipped the graph over the y axis because during testing of my batch file, the test data files were inverted by GNUplot.  So apparently, flipping an inverted image gives you another upside down image… Lovely.

When I finally managed to get a right side up image, I found that the dataset I had been using was a mesh of bathymetry and topography.  This was problematic as it gave me an image that didn’t have a clear definition of the shoreline (the gradient continued from sea to mountains).  To resolve this, I downloaded the bahtymetry-only data set, which meant that the shoreline (0 meters deep) was the biggest value on the map, and therefore the top color I defined in my palatte.  I messed around with the color palatte to get an acceptable view of the bathymetry.  It’s rather difficult to make the program give a good representation, but once I figure out how to draw contour lines, it should get better.