Podd 1.5 to 1.6 Database Conversion¶

In Analyzer 1.5 and earlier, many (but not all) detector classes employed fixed-format database files. These are problematic for a number of reasons: they are prone to errors, hard to extend, inefficient with time stamps, not readily transferable to non file-based database systems, etc. To address this problem, all modules (detectors, spectrometers, physics modules etc.) in Analyzer version 1.6 use the LoadDB database API. LoadDB expects database files in free-form key/value format.

To help users convert their existing databases to the new format, which could be a non-trivial and potentially very time-consuming task, we have written an automatic conversion tool called dbconvert. To build this tool, proceed as follows

• Ensure that the analyzer has been built properly
• Point the environment variable ANALYZER to the top-level installation directory. For example, if you have installed the distribution in ~/analyzer-1.6.0, do
  

  % export ANALYZER=~/analyzer-1.6.0
  

  
  or for tcsh users (JLab CUE)
  

  % setenv ANALYZER ~/analyzer-1.6.0
  

• Now build dbconvert
  

  % cd $ANALYZER/utils
  % make
  % cd ..
  

Once built, the tool allows you to convert an entire directory tree of database files to a new directory tree of free-format files. Before starting, make sure you have a backup of your old databases. Assuming the old databases are stored under $DB_DIR, back them up and write-protect them:

% cp -av $DB_DIR DB_backup
% chmod -R -w DB_backup

Now you can convert the files in $DB_DIR simply like this

./utils/dbconvert $DB_DIR newDB

where newDB is the name of the output directory. That directory will be created if it does not exist. Treat it like scratch space; the name can always be changed later. If the target directory does exists, all database files in it will be erased after asking for confirmation.

dbconvert has to make assumptions about what detector each database files belongs to. To do so, it uses a mapping from typical detector names to detector types. For example, databases with detector name "R.s1" are interpreted as belonging to THaScintillator or derived classes. You can inspect the full name mapping in the source code of dbconvert.cxx in function DefaultMap(). To change this map, you currently have to edit the source and recompile; we are planning to make the mapping user-configurable in a later version. This is important to know in case you are using custom names for some detectors in your experiment. dbconvert is unlikely to know about them. To have these files converted automatically, you will need to edit the list.

Of course, if you have written an entirely new custom detector with its own database format, dbconvert will not have a conversion function. In that case, no worries. In general, you do not need to convert databases for your own custom-written modules. Even with Analyzer 1.6, you can do whatever you want in the ReadDatabase functions of your own modules, and things will continue to work as before. Using the new database system is strongly encouraged, however, for all new development. Eventually, we might use different database backends, and your old-style code would not be able to take advantage of those new features.

Caution: dbconvert is still under development at this time (Feburary 2018). Bugs may cause incorrect conversions. Additional features may be added that improve the conversion results (formatting, version indicators, etc.) It goes without saying that you must keep backup copies of your original database files so you will be able to re-run the conversion at a later time. You should also use version control with your converted databases so you have a record of any edits you made. Do spot-checks of your converted databases for apparent errors and file a bug report if necessary.

Below is a summary of options currently understood by dbconvert:

Usage: dbconvert [options] SRC_DIR DEST_DIR
 Convert Podd 1.5 and earlier database files under SRC_DIR to Podd 1.6 and 
 later format, written to DEST_DIR
Options:
 -h, --help                   show this help message
 -v, --verbose                increase verbosity
 -d, --debug                  print extensive debug info
 -p, --preserve-subdirs       preserve subdirectory structure
     --no-preserve-subdirs    don't preserve subdirectory structure (negates 
                                -p)
     --no-clean               don't erase all existing files from DEST_DIR if 
                                it exists
     --no-verify              don't ask for confirmation before deleting any 
                                files
 -s, --subdirs ARG            split output files into new set of sudirectories
                                given in ARG. ARG is a comma-separated list 
                                of directory names which must be in YYYYMMDD 
                                format. Overrides -p.
 -z, --input-timezone ARG     interpret all validity timestamps in the input 
                                that don't have a timezone indicator to be in 
                                timezone ARG. ARG may be the name of a 
                                system timezone file (e.g. 'US/Eastern'), 
                                which supports DST, or a fixed offset 
                                specified as [+|-]nnnn (e.g. +0100 for central
                                European standard time). The default is to use
                                the current timezone at run time.
     --output-timezone ARG    write all output timestamps relative to timezone
                                ARG, specified in the same way as 
                                --input-timezone. The default is to use the 
                                current timezone at run time.

As mentioned before, additional features are expected to be added over the next several months as we test and improve the program.