Compiling Condor on Summit

This page is to gather notes on compiling Condor on Summit. I compiled v8.8.2 with blahp for glite support.

Summit is a Power PC system, so few, if any, precompiled binaries exist. Additionally, many libraries installed on Summit are in unconventional locations, making linking them in difficult. I found it was simpler to install user-space copies of many of the libraries Condor requires.

Pre-built

A built copy of Condor for Summit, v8.8.2, is available on Summit at /ccs/home/callag/condor-8.8.2-summit.tgz . Feel free to download it. You will probably still have to include many of the dependencies in your LD_LIBRARY_PATH.

Steps

I compiled this guide after I completed the install, so there's a chance I left something out. These are the steps as I best remember them.

1. Download Condor source from the website. I used v8.8.2.
2. Unzip the Condor source.
3. Load the following modules:
   1. module load cmake
   2. module load curl
   3. module load sqlite
4. Install the following dependencies:
   1. PCRE
   2. UUID
   3. Globus Toolkit (I used 6.0).
   4. Munge
   5. Boost. I installed v1.66, since that's one of the versions Condor seemed to expect in the externals/bundles/boost directory. Note that Summit does have several versions of Boost which can be loaded via modules, but I couldn't get Condor to pick them up correctly.
   6. I tried to install PAM also, but there seemed to be an error in the autoconf setup and I couldn't build it. I was able to get Condor to build without it.
5. Make the following edits to condor-8.8.2/CMakeLists.txt . (If this file doesn't exist, run the cmake command a few steps below, which will create the file). I'm sure there are other ways to get this into cmake, but this seemed to work.
   1. In line 30, add

      set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -I<path/to/pcre/include> -I</path/to/uuid/include> -I</path/to/globus_toolkit/include> -I</path/to/munge/include> -I</path/to/boost/include>")

      You may also need to add the path to sqlite3.h, if it's not picked up from the module.
   2. On the next line, add

      set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} <all the entries you added to CMAKE_C_FLAGS>")

   3. On the next line, add

      set(CMAKE_LD_FLAGS "${CMAKE_LD_FLAGS} -L<path/to/pcre/lib> -L</path/to/uuid/lib> -L</path/to/globus_toolkit/lib> -L</path/to/munge/lib> -L</path/to/boost/lib>")

6. Add Globus libraries to your LD_LIBRARY_PATH:

   export LD_LIBRARY_PATH=</path/to/globus_toolkit/lib>:$LD_LIBRARY_PATH

7. BLAHP support is required to get glite to compile (which is needed for rvGAHP). I was unable to get Condor to install blahp using the default configuration. Condor is supposed to initially look for libraries on the system, but when it couldn't find blahp it didn't try to install it itself. To force Condor to build BLAHP itself, I did the following:
   1. In condor-8.8.2/externals/bundles/blahp/1.16.5.1/CMakeLists.txt (if this file doesn't exist, run the cmake command down lower and make these changes when the file is created), edit line 28 to say

      if ( NOT BLAHP_PROPER )

   2. In condor-8.8.2/CMakeCache.txt, line 29, add

      BLAHP_PROPER:BOOL=OFF

8. Run the following command:

   (cmake -DUW_BUILD:BOOL=TRUE -D_VERBOSE:BOOL=TRUE -DBUILDID:STRING=SUMMIT -DCLIPPED:BOOL=ON -DWITH_BLAHP:BOOL=TRUE -DWITH_CREAM:BOOL=OFF -DWANT_MAN_PAGES:BOOL=FALSE -DCMAKE_INSTALL_PREFIX:PATH=<path/to/install/dir> && make && make install) 2>&1 | tee build.log

   It will fail when it gets to the blahp build, because it can't find the Globus includes.
9. Running the cmake command created the Makefile for BLAHP. Edit condor-8.8.2/externals/bundles/blahp/1.16.5.1/blahp-prefix/src/blahp/src/Makefile:
   1. In line 327, add

      -I</path/to/globus_toolkit/include

   2. In line 502, change

      -L/lib

      to

      -L</path/to/globus_toolkit/lib>

      Keep all the -l<library> entries the same.
   3. In line 503, change it to

      -L</path/to/globus_toolkit

10. Run the cmake command again. If you run into issues with includes or libraries, go back to condor-8.8.2/CMakeLists.txt and make changes to the include and library paths to point to them.
11. Once everything is built, change to the install prefix and run

    ./condor_configure --install --make-personal-condor --prefix=</path/to/condor/install/directory>

Good luck!