Installation
How to download the pipeline routines:
The routines are stored in a Subversion repository that facilitates keeping track of changes and ensuring you always have the latest version available:
The latest, stable released version is version 1.0. You can download this version via SVN with the following command:
svn export http://pubsvn.galev.org/podi/release/v1.0 (directory)
(directory) can either specify a new directory to hold the reduction scripts, or you can specify . (dot) to put the scripts in the current directory.
If you are feeling adventurous, you can also try the bleeding-edge development version. Just keep in mind that this represents both the “latest and greatest”, but might also be unstable and/or incomplete. To be on the safe side please let me know if you are planning to use this version and I can tell you what works and what doesn’t. As another caveat, I try to keep the documentation up-to-date, but my development version likely has added new or removed old features that are not (well) described in the documentation.
svn co http://pubsvn.galev.org/podi/trunk (directory)
However, while I do make any attempt to test code changes for usability before checking in my changes, it can occasionally happen that the trunk version is not running perfectly in all cases – You have been warned, proceed at your own risk.
To browse changes etc there is a web-based SVN explorer here.
Checking your installation
The pipeline package comes with a small testing tool that checks if all required packages can be found. To use this tool, run.
/your/directory/podi_testinstall.py
If everything works out, you’ll get the following output. If there are some missing packages, these will be listed in the output and you have to install them. Check here for a list of packages and their respective websites.
Testing if all packages are installed checking standard packages ... Found working version of os! Found working version of sys! Found working version of math! Found working version of time! Found working version of types! Found working version of ctypes! Found working version of itertools! checking multi-processor packages ... Found working version of multiprocessing! Found working version of Queue! Found working version of threading! Found working version of subprocess! checking numerical processing packages ... Found working version of numpy! Found working version of scipy! Found working version of scipy.stats! Found working version of scipy.optimize! Found working version of scipy.interpolate! Found working version of scipy.ndimage! Found working version of bottleneck! checking plotting packages ... Found working version of matplotlib! Found working version of Image! Found working version of ImageDraw! checking astronomy-related packages ... Found working version of pyfits! Found working version of ephem! Found working version of astLib! Found working version of pywcs! Found working version of podi_sitesetup! Checking cython-optimized package for pODI Found working version of podi_cython! * sigma_clip_mean : True * sigma_clip_median : True * lacosmics : True All routines found Check done! Do you want to run the sitesetup assistant (y/N)?
Configuring QuickReduce via the config assistant
After installing the pipeline there are some settings that are not set just but that are necessary for proper operation. These are stored in a file podi_sitesetup.py for use during execution. However, it is advisable to not change this file by hand, but rather use the sitesetup assistant. If you select “y” in the question at the end of podi_testinstall.py you’ll be taken through a set of questions – most of the default parameters are very reasonable numbers, so only change them if you know what you are doing. Other ones, however, ask for e.g. the path to your SourceExtractor executable, so these you do have to specify. Also note that due to the ongoing development of the code some options are added from time to time, so it is recommended to run this configuration tool after each update – it’ll use the current settings as defaults, so you won’t have to do a lot if you run it for a second time.
A part of the configuration is shown below:
Do you want to run the sitesetup assistant (y/N)? y Starting auto-configuration! ---- Maximum number of CPU cores available for use in reduction Default: 7 Answer: ---- Directory holding the local 2MASS catalog in FITS format Default: /nas/wiyn/podi_calibrations/2mass_fits/ Answer: ---- Type of astrometric reference catalog (leave unchanged) Default: 2mass_nir Answer: ---- Source of SDSS catalog (choose from local, web, stripe82) Default: local Answer: ---- If local SDSS catalog, specify source directory Default: /nas/wiyn/podi_calibrations/sdss_photcalib/ Answer: ---- Directory of the UCAC4 directory - specify none if it does not exist Default: /nas/wiyn/podi_calibrations/ucac4b Answer: ---- Scratch-directory for temporary files (the faster the better) Default: /scratch/ Answer: ---- Path to SourceExtractor executable (Hint: run 'which sex' in another terminal) Default: /usr/local/bin/sex Answer: ---- Command to re-direct stdout and stderr to /dev/null Default: >& /dev/null Answer: ---- Delete SourceExtractor temporary files after use (choose from True (recommended) or False) Default: True Answer: ----
ODI and Cython: New since 2014-01-23:
Starting on the above date, the pODI pipeline contains a optimized package developed in C and Cython that needs to be compiled before it can be used. To do so, run the following in the directory that contains all the other .py files as well:
python setup.py build_ext --inplace
This typically just works out of the box, but if it fails for some reason it’s almost always one or both of the following reasons. Once you fixed each one of them go back to the pODI source directory and re-run the above command to see if you done installing dependencies.
Typical problem #1: No Cython
This problem shows up when you run the above command and you get a python error message saying something like “import Cython” failed.
In this case, you need to install cython first. On most systems you can simply run
sudo pip install Cython
Typical problem #2: Missing GSL library
Once or If you have Cython installed, you might run into something like the following error message, indicating that you do not have the GSL library installed:
Unable to find gsl_sort.h
The GSL, or GNU Scientific Library, handles most of the heavy lifting under the hood, asving me the trouble of coming up with my own sort- and statistics routines. GSL is relatively easy to install:
Download it first:
wget ftp://ftp.gnu.org/gnu/gsl/gsl-1.16.tar.gz
and then unpack and install it:
tar xvf gsl-1.16.tar.gz cd gsl-1.16 ./configure make make install
Installing GSL on a Mac
If you have Xcode installed, you should be able to follow the instructions above.
Otherwise GSL is (or so the internet says, I don’t have a Mac (go Linux!) so can’t confirm) also available via Mac Ports. In this case you need to run “sudo port install gsl” and at some point enter your root password. If you don’t have Mac Ports yet you can get it here.
Keeping up-to-date
Note: this only works in the trunk/development branches
Above command is only needed once to checkout the repository. Once you have done that, you can obtain the latest version via
svn update
Note here, that svn is clever enough to merge changes you might have made to the code with potential updates.
If you made changes and you want to undo them, there are two ways: Either you delete the file and run “svn update” or you can run
svn revert (filenames)
to restore them to the version in the repository.
To get some information about what has changed in the past, run
svn log
If you made some changes to the routines and want to know exactly which lines were changed, run
svn diff
Also note that above link gives you read-only access to the files. If you make changes that you would like to share, please send me (contacts) an email and attach the changed files, and I’ll check them in for you.
For more commands and infos on what you can do with svn, check out the SVN book