Wire Cell Toolkit Configuration
(top)
Table of Contents
1. What is here
This repository holds a reference set of Wire Cell Toolkit (WCT) configuration files. These are largely in the form of Jsonnet but some in FHiCL format are as examples to integrate with applications based on the art event processing framework.
The goal of any main Jsonnet file is to produce an ordered array of
component configuration objects. Each object names a WCT C++
component type and an optional instance name and may also include
additional attributes which are passed to the C++ object instance to
configure it. A special configuration object, typically the last in
the sequence, names the WCT “app” which is usually a Pgrapher or
TbbFlow component that provide data flow processing (DFP) graph
execution. The main configuration element of this component is a list
of “edges” connecting two data flow graph nodes by stating their
type:instance name pairs.
Configuration is complex and more so as it spans multiple detectors
and for each detector variations in term of detector model, job
intention and other factors. To manage this complexity a number of
structural conventions are observed at the directory, file and content
level. The Jsonnet files at top cfg/ directory provide general
utilities. Sub-directories cfg/* define some scope for conventions,
some of which are described in the next section.
For more information on configuration see the WCT configuration manual as well as news posts tagged with config.
2. Contents overview
This describes what is available. Be wary is may not always be up to date with the reality on the ground.
2.1. General support files
Files at top level provide general support and utilities. Primarily there is the file:
This holds a Jsonnet version of the WCT system of units (which is essentially identical to CLHEP’s). It also includes a number of Jsonnet functions to form some common configuration data structures.
This holds some functions to assist in doing vector arithmetic in Jsonnet.
This holds functions to support building a processing graph for use by
the Pgrapher WCT app component. Some details are here.
2.2. Structure layers
Across all types of detectors and jobs which are supported by WCT we define a layered configuration structure. The top layer is most generic and the bottom most specific. An Jsonnet API is defined for each layer. Users defining new “main” Jsonnet files, or improving legacy forms, should read the document layers/README.html.
2.3. Legacy forms
The original configuration structure which does not allow for detector-independent job configuration definition is held under pgrapher/. See it’s README for more info on the conventions it follows. New detector configuration should follow the structured layers form introduced above. In some cases, legacy detector configuration may used in an adapter that provides structured layers forms.
3. Tests
As with other WCT subpackages, this provides a /test/ sub-directory with various tests that used through the WCT testing framework. In particular the cfg sub-package has many tests written in Jsonnet and the Jsonnet test support is particularly relevant.
4. JQ tricks
The jq tool is like grep for JSON. Here are some useful tricks to operate on a mongo big JSON file.
4.1. WCT JSON Config Files
Find an element in the configuration sequence by type
$ jq '.[]| select(.type|contains("Drifter"))' wct.json