Default Config Dumper
WCT is very configurable. It provides a way for the expert that wrote
the C++ component to provide sane defaults and a way that a user may
override or augment those defaults. WCT helps such intrepid users to
discover the defaults with the ConfigDumper WCT app component.
Quick review of WCT config
The configuration section of the WCT manual has details but for here, it's worth giving a quick review of WCT configuration. Ultimately, a WCT job is configured by providing a configuration sequence. This is simply an array of data structures. Each structure corresponds to a component instance. This is a C++ object instance of a WCT component class which inherits from one or more WCT interface base classes. At it's top level, each configuration data structure follows a simply object schema consisting of three attributes:
type- name the WCT component type (usually similar but not always identical to the corresponding C++ class name).
name- an optional instance name to differentiate multiple instances of the same type.
data- an optional and arbitrary instance configuration data structure.
In principle the configuration sequence can be provided in a variety of forms. In practice, just one form is supported: JSON, possibly compiled from Jsonnet. The user thus provides one or more files that ultimately produce a simple configuration sequence array and WCT uses that to instantiate and configure component objects.
The job finally runs by locating any configured component objects which happen to implement the interface IApplication. Any and all "apps" that are found are executed and through their own configuration (their instance configuration structure specified by the data attribute) they make use of other configured component objects.
Discovering Component Configuration
Okay, so what, you want to configure some component but you don't know
what parameters it honors. As always, the best way to know what code
does is to read it. Short of that, WCT provides a special "app"
component called ConfigDumper that can dispaly the default
configuration which is hard-coded into the components' C++
implementation.
You can run this app without any configuration:
$ wire-cell -p WireCellApps -a ConfigDumper
[{"data":{"components":[],"filename":"/dev/stdout"},"name":"","type":"ConfigDumper"},{"data":{"filename":"/dev/stdout","nodes":[]},"name":"","type":"NodeDumper"}]
Or, for something a bit more pretty:
$ wire-cell -p WireCellApps -a ConfigDumper | json_pp
[
{
"type" : "ConfigDumper",
"name" : "",
"data" : {
"components" : [],
"filename" : "/dev/stdout"
}
},
{
"type" : "NodeDumper",
"name" : "",
"data" : {
"filename" : "/dev/stdout",
"nodes" : []
}
}
]
The output is printed to stdout and the stderr logging is omitted
here for clarity. You may replace the json_pp with your favorite
JSON pretty-printer.
By default ConfigDumper dumps the configuration about all
configurable components provided by the WCT plugins listed on the
command line. At a minimum the WireCellApps plugin must be listed
as it is required for WCT to find ConfigDumper itself. You can add
additional plugins and their components will also be dumped. Here,
WireCellGen is added:
$ wire-cell -p WireCellGen -p WireCellApps -a ConfigDumper | json_pp
[
...
{
"name" : "",
"type" : "PlaneImpactResponse",
"data" : {
"plane" : 0,
"tick" : 500,
"nticks" : 10000,
"field_response" : "FieldResponse",
"other_responses" : []
}
},
...
]
Only one of about three dozen instance configuration data structures
is shown. It is the one for PlaneImpactResponse which was featured
in a recent post.
Of course, as shown above, ConfigDumper itself takes some
configuration. Namely the user may set an output file and may limit
which components are dumped. This configuration can be provided in a
usual configuration file but in the next example we use the process
substitution feature of bash to do everything on the command line:
$ wire-cell -p WireCellSigProc -p WireCellGen -p WireCellApps \
-a ConfigDumper \
-c <(echo '[{type:"ConfigDumper",data:{filename:"dump.json",components:["FieldResponse","PlaneImpactResponse"]}}]')
$ cat dump.json |json_pp
[
{
"type" : "FieldResponse",
"data" : {
"filename" : ""
},
"name" : ""
},
{
"name" : "",
"data" : {
"plane" : 0,
"other_responses" : [],
"field_response" : "FieldResponse",
"tick" : 500,
"nticks" : 10000
},
"type" : "PlaneImpactResponse"
}
]
Using Configuration Dumps
After running the ConfigDumper the user may wish to provide a
variant configuration which is influenced by what was discovered. A
few guidelines and caveats are useful here.
JSON vs Jsonnet
The ConfigDumper output is in JSON while humans are likely happier
writing in the Jsonnet data language. By design, the former is a
strict subset of the latter so it is possible to simply copy-paste a
snippet of ConfigDumper output into a Jsonnet file and then modify
as desired. However, JSON should be seen as a "lossy" format and some
manual "up-conversion" to Jsonnet is recomended. In particular:
- quoting attribute keys is "ugly" (admittedly, this is purely subjective).
- literal numerical quantities in JSON are implicitly expressed in the WCT system of units while it is much more clear and less error prone if they carry explicit units.
- it is best to have one component refer to another not by literal
strings but via the
tn()function provided bywirecell.jsonnet.
As an example, one might write a configuration which is merely
influenced by the dump of PlaneImpactResponse's default
configuration above in more "proper" Jsonnet like:
local wc = import "wirecell.jsonnet";
local fr0 = {
type: "FieldResponse",
data: {
filename: "ub-10-half.json.bz2",
}
};
local pir = {
type : "PlaneImpactResponse",
data : {
plane : 0,
field_response : wc.tn(fr0),
tick : 0.5*wc.us,
nticks : 9595,
},
};
[fr0, pir, ...]
Missing or Empty Default Attributes
A user configuration object is "deeply merged" in to the default configuration object by recursively copying leaf attributes. Any part of the default structure which does not exist in the user structure will be retained. The result is what is passed to the C++ component instance referred to by the configuration object itself.
In some cases, the default configuration will have placeholder
attributes with empty or otherwise "bogus" value. This can be seen in
FieldResponse where the filename attribute is the empty string.
Left unset this will lead to a runtime error.
In other cases, there may be attributes which are simply not mentioned
in the default configuration at all and the user must understand what
is required. For example the simulation has a component called
MultiDuctor which implements a facade patter by applying a chain of
rules that determine which "real" ductor to use to service any given
energy deposition. It's default configuration is:
$ wire-cell -p WireCellSigProc -p WireCellGen -p WireCellApps \
-a ConfigDumper \
-c <(echo '[{type:"ConfigDumper",data:{components:["MultiDuctor"]}}]') \
| json_pp
[
{
"name" : "",
"data" : {
"continuous" : false,
"chain" : [],
"readout_time" : 5000000,
"first_frame_number" : 0,
"anode" : "AnodePlane",
"start_time" : 0
},
"type" : "MultiDuctor"
}
]
The default chain is an empty string. It must be set to list of a
list of data structures that associate rules to ductors. The
structure of chain can only (currently) be understood by reading the
code or existing configuration produced by others.
Seeing what is available
One can also use the ConfigDumper on all plugins and without any
restriction on the component list in order to generate a list of all
components. This is particularly useful for someone to do before
adding a new component as there may already be one that does what is
needed. To generate the list one can simply run the dumper and browse
through the giant JSON text that is produced. However the immensely
useful jq tool provides some useful features here.
$ wire-cell -p WireCellPgraph -p WireCellTbb -p WireCellSio \
-p WireCellSigProc -p WireCellGen -p WireCellApps \
-a ConfigDumper|json_pp|wc -l
790
$ wire-cell -p WireCellPgraph -p WireCellTbb -p WireCellSio \
-p WireCellSigProc -p WireCellGen -p WireCellApps \
-a ConfigDumper|jq '.[].type'|wc -l
60
$ wire-cell -p WireCellPgraph -p WireCellTbb -p WireCellSio \
-p WireCellSigProc -p WireCellGen -p WireCellApps \
-a ConfigDumper|jq '.[].type'|sort | tr -d '"'
AnodePlane
BirksRecombination
BlipSource
BoxRecombination
CelltreeFrameSink
CelltreeSource
ChannelSelector
ConfigDumper
DepoFanout
DepoFramer
DepoMerger
Diffuser
Drifter
Ductor
ElecResponse
EmpiricalNoiseModel
FieldResponse
FourDee
FrameFanin
FrameMerger
FrameSummer
HfFilter
HistFrameSink
JsonDepoSource
L1SPFilter
LfFilter
MagnifySink
MagnifySource
MipRecombination
Misconfigure
MultiDuctor
NodeDumper
NoiseSource
NominalChannelResponse
NumpyDepoSaver
NumpyFrameSaver
OmniChannelNoiseDB
Omnibus
OmnibusNoiseFilter
OmnibusPMTNoiseFilter
OmnibusSigProc
PerChannelResponse
Pgrapher
PlaneImpactResponse
RCResponse
Random
SilentNoise
StaticChannelStatus
TbbDataFlowGraph
TbbFlow
TrackDepos
Truth
TruthSmearer
VagabondX
WireParams
WireSchemaFile
WireSource
mbADCBitShift
mbOneChannelNoise
mbOneChannelStatus
Summary
It's always best to read the source code to learn what configuration a
component expects, and more importantly, what it does with the
configuration it gets. With that said, the WCT ConfigDumper "app"
provides an easy way to discover or simply remind oneself what is the
default configuration of some component.
