abstract type AbstractSetupData

Subtypes wrap SetupConfig for specific experiments.

struct ChannelId <: DataSelector

Represents a LEGEND data channel.

Example:

ch = ChannelId(1083204)
# ch = ChannelId(98) # with old channel numbering
ch.no == 1083204
string(ch) == "ch1083204"
ChannelId("ch1083204") == ch

struct DataCategory <: DataSelector

Represents a LEGEND data category (related to a DAQ/measuring mode) like "cal" or "phy".

Example:

category = DataCategory(:cal)
category.label == :cal
string(category) == "cal"
DataCategory("cal") == category

struct DataPartition <: DataSelector

Represents a LEGEND data-taking partition.

Example:

partition = DataPartition(1)
partition.no == 1
string(partition) == "partition01"
DataPartition("partiton01") == partition

struct DataPeriod <: DataSelector

Represents a LEGEND data-taking period.

Example:

period = DataPeriod(2)
period.no == 2
string(period) == "p02"
DataPeriod("p02") == period

struct DataRun <: DataSelector

Represents a LEGEND data-taking run.

Example:

r = DataRun(6)
r.no == 6
string(r) == "r006"
DataRun("r006") == r

abstract type DataSelector

Abstract type for data selectors like ExpSetup, DataTier, DataPeriod, DataRun, DataCategory, Timestamp and FileKey.

struct DataTier <: DataSelector

Represents a LEGEND data tier like "raw, "dsp", etc.

Example:

tier = DataTier(:raw)
tier.label == :raw
string(tier) == "raw"
DataTier("raw") == tier

struct DetectorId <: DataSelector

Represents a LEGEND detector id id.

Example:

detector = DetectorId(:V99000A)
detector.label == :V99000A
string(detector) == "V99000A"
DetectorId("V99000A") == detector

struct ExpSetup <: DataSelector

Represents a LEGEND experimental setup like "l200".

Example:

setup = ExpSetup(:l200)
setup.label == :l200
string(setup) == "l200"
ExpSetup("l200") == setup

struct FileKey <: DataSelector

Represents a LEGEND file key.

Example:

filekey = FileKey("l200-p02-r006-cal-20221226T200846Z")

See also read_filekeys and write_filekeys.

struct LegendData <: AbstractSetupData

Provides access to LEGEND data and metadata.

Constructors:

• LegendData(setup_config::SetupConfig).

• LegendData(setup::Symbol) - requires the $LEGEND_DATA_CONFIG environment variable to be set.

Examples:

config_filename = "/path/to/config.json"
config = LegendDataConfig(config_filename)
l200 = LegendData(config.setups.l200)

filekey = FileKey("l200-p02-r006-cal-20221226T200846Z")

or simply (if $LEGEND_DATA_CONFIG is set):

l200 = LegendData(:l200)

LegendData has the (virtual) properties metadata and tier.

The full path to "tier" data files can be retrieved using

(data::LegendData)[tier::Symbol, filekey::FileKey]
(data::LegendData).tier[tier::Symbol, filekey::AbstractString]

Example:

l200.tier[:raw]
l200.tier[:raw, FileKey("l200-p02-r006-cal-20221226T200846Z")]

LegendData comes with an extension for SolidStateDetectors:

l200 = LegendData(:l200)
SolidStateDetector(l200, :V99000A)

struct LegendDataConfig

Data configuration multiple experimental setups.

Contains a single field setups::PropertyDict{Symbol,SetupConfig}.

Can be read from a config file via LegendDataConfig(config_filename[s]), or simply LegendDataConfig() if the environment variable $LEGEND_DATA_CONFIG is set. $LEGEND_DATA_CONFIG may be a list of colon-separated config filenames, which are applied/merged in reverse order (analog to the order of prioritiy in $PATH and similar).

Example:

config = LegendDataConfig("/path/to/config.json")
setup = config.setups.l200
data_path(setup, "tier", "raw", "cal", "p02", "r006", "l200-p02-r006-cal-20221226T200846Z-tier_raw.lh5")

See also SetupConfig.

struct LegendReport

Represents a LEGEND report.

Don't instantiate directly, use lreport()

struct LegendDataManagement.LegendTierData

Constructors:

(data::LegendData).tier

LegendDataManagement.LegendTierData(data::LegendData)

The path to data directories and files can be accessed via getindex on tier_data::LegendTierData:

tier_data[]
tier_data[tier::DataTierLike]
tier_data[tier::DataTierLike, category::DataCategoryLike]
tier_data[tier::DataTierLike, category::DataCategoryLike, period::DataPeriodLike]
tier_data[tier::DataTierLike, category::DataCategoryLike, period::DataPeriodLike, run::DataRunLike]
tier_data[tier::DataTierLike, category::DataCategoryLike, period::DataPeriodLike, run::DataRunLike, ch::ChannelIdLike]

tier_data[tier::DataTierLike, filekey::FileKeyLike]
tier_data[tier::DataTierLike, filekey::FileKeyLike, ch::ChannelIdLike]

Examples:

```julia l200 = LegendData(:l200)

filekey = FileKey("l200-p02-r006-cal-20221226T200846Z") isfile(l200.tier[:raw, filekey])

isdir(l200.tier[:raw, :cal]) isdir(l200.tier[:raw, :cal, "p02"]) isdir(l200.tier[:raw, :cal, "p02", "r006"]) isdir(l200.tier[DataTier(:raw), DataCategory(:cal), DataPeriod(2), DataRun(6)])

struct LegendDataManagement.NoSuchPropsDBEntry

Indicates that a given property (path) of a LegendDataManagementPropsDB[@ref] does not exist.

Supports PropDicts.writeprops(missing_props::NoSuchPropsDBEntry, props::PropDicts.PropDict) to create the missing directories and file for the property path.

struct PeriodSelLike = Tuple{<:DataPeriodLike, <:DataCategoryLike}

Represents a LEGEND period selection for a specific category.

@enum ProcessStatus

May be succeeded, pending, or failed.

struct LegendDataManagement.PropsDB

A PropsDB instance, e.g. myprops, presents an on-disk directory containing JSON files or sub-directories (that contains JSON files in leaf directories) as a dictionary of properties.

PropsDB supports Base.keys and Base.getindex as well as Base.propertynames and Base.getproperty to access it's contents. getindex and getproperty will return either another PropsDB or a PropDicts.PropDict, depending on whether the accessed property is stored as a sub-directory or a JSON file. We recommend to use getproperty where the properties/keys of the PropDict are more or less standardized and where they may be arbitrary (see examples below).

The contents of PropsDB may be time- and category-dependent, determined by the presence of a "validity.json" file. In this case, use myprops(sel::LegendDataManagement.ValiditySelection) or myprops(filekey::FileKey) to select the desired time and category. The selection can be made at some point during traversal of properties or at the leaf PropsDB (see the examples below).

Examples:

l200 = LegendData(:l200)

propertynames(l200.metadata.hardware)
l200.metadata.hardware.detectors.germanium

keys(l200.metadata.hardware.detectors.germanium.diodes)
l200.metadata.hardware.detectors.germanium.diodes[:V99000A]

diodes = l200.metadata.hardware.detectors.germanium.diodes
diodes[keys(diodes)]

sel = ValiditySelection("20221226T194007Z", :cal)
filekey = FileKey("l200-p02-r006-cal-20221226T194007Z")
data.metadata.hardware(sel).configuration.channelmaps
data.metadata.hardware.configuration.channelmaps(filekey)

Use code should not instantiate PropsDB directly, use LegendDataManagement.AnyProps(path::AbstractString) instead, which may return a PropsDB or a PropDicts.PropDict depending on what on-disk content path points to.

struct RunCategorySelLike = Tuple{<:DataPeriodLike, <:DataRunLike}

Represents a LEGEND run selection for a specific category.

struct RunSelLike = Tuple{<:DataPeriodLike, <:DataRunLike}

Represents a LEGEND run selection.

struct SetupConfig

Data configuration for an experimental setup.

Supports

data_path(setup, path_components)

Examples:

data_path(setup, "tier", "raw", "cal", "p02", "r006", "l200-p02-r006-cal-20221226T200846Z-tier_raw.lh5")

struct Timestamp <: DataSelector

Represents a LEGEND timestamp.

Example:

julia timestamp = Timestamp("20221226T200846Z") timestamp.unixtime == 1672085326 string(timestamp) == "20221226T200846Z"`

struct LegendDataManagement.ValiditySelection

Representy validiy selection for a LegendDataManagement.PropsDB[@ref].

LegendDataManagement.AnyProps = Union{LegendDataManagement.PropsDB,PropDicts.PropDict}

Properties stored either in a directory managed via [LegendDataManagement.PropsDB][@ref] or loaded from one or several files into a PropDicts.PropDict.

Constructors:

LegendDataManagement.AnyProps(base_path::AbstractString; override_base::AbstractString = "")

const AnyValiditySelection = Union{ValiditySelection,FileKey}

Anything that can be used in time/category-based data selection.

ChannelIdLike = Union{ChannelId, Integer, AbstractString}

Anything that can represent a data channel, like ChannelId(1083204) or "ch1083204".

DataCategoryLike = Union{DataCategory, Symbol, AbstractString}

Anything that can represent a data category, like DataCategory(:cal), :cal or "cal".

DataPartitionLike = Union{DataPartition, Symbol, AbstractString}

Anything that can represent a data partition, like DataPartition(2) or "partition02".

DataPeriodLike = Union{DataPeriod, Symbol, AbstractString}

Anything that can represent a data period, like DataPeriod(2) or "p02".

DataRunLike = Union{DataRun, Symbol, AbstractString}

Anything that can represent a data run, like DataRun(6) or "r006".

DataTierLike = Union{DataTier, Symbol, AbstractString}

Anything that can represent a data tier, like DataTier(:raw), :raw or "raw".

DetectorIdLike = Union{DetectorId, Symbol, AbstractString}

Anything that can represent a detector id.

ExpSetupLike = Union{ExpSetup, Symbol, AbstractString}

Anything that can represent a setup label, like ExpSetup(:l200), :l200 or "l200".

FileKeyLike = Union{FileKey, AbstractString}

Anything that can represent a file key, like FileKey("l200-p02-r006-cal-20221226T200846Z") or "l200-p02-r006-cal-20221226T200846Z".

const LJlExprLike = Union{Expr, Symbol, Integer, AbstractFloat}

Anything that can represent a parsed LEGEND Julia expression.

TimestampLike = Union{Timestamp, AbstractString, Integer}

Anything that can represent a timestamp, like Timestamp("20221226T200846Z") or "20221226T200846Z".

analysis_runs(data::LegendData)

Return cross-period analysis runs.

function atomic_fcreate(
    body, filenames::AbstractString...;
    create_dirs::Bool = true, overwrite::Bool = true, delete_on_error::Bool=true
)

Creates filenames in an atomic fashion.

Creates temporary files in the same directories as filenames, then calls body(temporary_filenames...). If body returns successfully, the files temporary_filenames are renamed to filenames. If body throws an exception, the temporary files are either deleted (if delete_on_error is true) or left in place (e.g. for debugging purposes).

If create_dirs is true, directories are created if necessary.

If all of files already exist and overwrite is false, takes no action (or, if the file is created by other code running in parallel, while body is running, does not overwrite it).

Throws an error if only some of the files exist and overwrite is false.

Returns nothing.

Example:

atomic_fcreate("foo.txt", "bar.txt") do foo, bar
    write(foo, "Hello")
    write(bar, "World")
end

bad_filekeys(data::LegendData)

Get the list of filekeys to ignore for data.

channelinfo(data::LegendData, sel::AnyValiditySelection; system::Symbol = :all, only_processable::Bool = false)
channelinfo(data::LegendData, sel::RunCategorySelLike; system::Symbol = :all, only_processable::Bool = false)

Get all channel information for the given LegendData and ValiditySelection.

data_path(setup::SetupConfig, path_components::AbstractString...)

Get the full absolute path for the given path_components as configured for setup.

Converts between "/" and "\" inside of path_components if necessary (for Windows compatibility).

data_path(setup::AbstractSetupData, path_components::AbstractString...)

Get the full absolute path for the given path_components as configured for setup.

data_path(tier_data::LegendTierData, path_components::AbstractString...)

Get the full absolute path for the given path_components relative to tier_data.

data_path(pd::LegendDataManagement.PropsDB)

Return the path to the data directory that contains pd.

dataprod_config(data::LegendData)

Get the Julia data production configuration for data.

Use dataprod_config(data)(valsel::AnyValiditySelection) to also set the time/category validity selection for the configuration.

Examples:

l200 = LegendData(:l200)
filekey = FileKey("l200-p02-r006-cal-20221226T200846Z")
dataprod_config(l200)(filekey)

or

l200 = LegendData(:l200)
vsel = ValiditySelection("20221226T200846Z", :cal)
dataprod_config(l200)(vsel)

dataprod_parameters(data::LegendData)

Get the Julia data production parameters data.

Examples:

l200 = LegendData(:l200)
dataprod_config(l200)

LegendDataManagement.dataprod_pars_aoe_window(data::LegendData, sel::AnyValiditySelection, detector::DetectorId)

Get the A/E cut window for the given data, validity selection and detector.

get_aux_cal_propfunc(data::LegendData, sel::AnyValiditySelection, detector::DetectorId)

Get the aux calibration function for the given data, validity selection and the aux channel referred to by detector.

get_aux_evt_chdata_propfunc(data::LegendData, sel::AnyValiditySelection)

Get the aux channel data output PropertyFunction.

get_aux_evt_chsel_propfunc(data::LegendData, sel::AnyValiditySelection)

Get the aux channel selection PropertyFunction.

get_aux_evt_levelname_propfunc(data::LegendData, sel::AnyValiditySelection)

Get the aux event level name.

LegendDataManagement.get_ged_aoe_cut_propfunc(data::LegendData, sel::AnyValiditySelection, detector::DetectorId; pars_type::Symbol=:ppars, pars_cat::Symbol=:aoe)

Get the A/E cut propfuncs for the given data, validity selection and detector.

get_ged_cal_propfunc(data::LegendData, sel::AnyValiditySelection, detector::DetectorId)

Get the HPGe calibration function for the given data, validity selection and detector.

Note: Caches configuration/calibration data internally, use a fresh data object if on-disk configuration/calibration data may have changed.

get_ged_evt_chdata_propfunc(data::LegendData, sel::AnyValiditySelection)

Get the Ge-detector channel data output PropertyFunction.

get_ged_evt_chsel_propfunc(data::LegendData, sel::AnyValiditySelection)

Get the Ge-detector channel selection PropertyFunction.

get_ged_evt_hitchsel_propfunc(data::LegendData, sel::AnyValiditySelection)

Get the hit Ge-detector channel selection PropertyFunction.

get_ged_evt_kwargs(data::LegendData, sel::AnyValiditySelection)

Get the Ge-detector evt kwargs.

get_ged_psd_propfunc(data::LegendData, sel::AnyValiditySelection, detector::DetectorId; pars_type::Symbol=:ppars, pars_cat::Symbol=:aoe)

Get the HPGe psd calibration function for the given data, validity selection and detector.

Note: Caches configuration/calibration data internally, use a fresh data object if on-disk configuration/calibration data may have changed.

get_ged_qc_cuts_propfunc(data::LegendData, sel::AnyValiditySelection, detector::DetectorId)

Get the Ge-detector QC cut definitions for the given data and validity selection.

get_ged_qc_is_baseline_propfunc(data::LegendData, sel::AnyValiditySelection)

Get a PropertyFunction that returns true for events that fullfill the is_baseline definition.

get_ged_qc_is_physical_propfunc(data::LegendData, sel::AnyValiditySelection, detector::DetectorId)

Get a PropertyFunction that returns true for events that fullfill the is_physical definition.

get_ged_qc_istrig_propfunc(data::LegendData, sel::AnyValiditySelection, detector::DetectorId)

Get the Ge-detector trigger condition for the given data and validity selection.

get_partition_combined_periods(data::LegendData, period::DataPeriodLike; chs::Vector{ChannelIdLike}=ChannelIdLike[])

Get a list periods which are combined in any partition for the given period and list of channels.

LegendDataManagement.get_setup_config(data::AbstractSetupData)::SetupConfig

Must be specialized for each subtype of AbstractSetupData.

get_spm_cal_propfunc(data::LegendData, sel::AnyValiditySelection, detector::DetectorId)

Get the LAr/SPMS calibration function for the given data, validity selection and detector.

get_spms_evt_chdata_propfunc(data::LegendData, sel::AnyValiditySelection)

Get the Ge-detector channel data output PropertyFunction.

get_spms_evt_chsel_propfunc(data::LegendData, sel::AnyValiditySelection)

Get the SiPM channel selection PropertyFunction.

get_uncertainties(x::Unitful.Quantity{<:Measurements.Measurement{<:Real}})
get_uncertainties(x::Unitful.Quantity{<:Real})
get_uncertainties(pd::PropDict)
get_uncertainties(A::AbstractArray)

Get the uncertainty of a Unitful.Quantity or Measurements.Measurement object or a PropDict or an array of Unitful.Quantity or Measurements.Measurement objects.

get_values(x::Unitful.Quantity{<:Measurements.Measurement{<:Real}})
get_values(x::Unitful.Quantity{<:Real})
get_values(pd::PropDict)
get_values(A::AbstractArray)

Get the value of a Unitful.Quantity or Measurements.Measurement object or a PropDict or an array of Unitful.Quantity or Measurements.Measurement objects.

is_analysis_cal_run(data::LegendData, (period::DataPeriodLike, run::DataRunLike))

Return true if run is an analysis run for data in period.

ATTENTION: This is only valid for cal runs.

is_analysis_phy_run(data::LegendData, (period::DataPeriodLike, run::DataRunLike))

Return true if run is an analysis run for data in period.

ATTENTION: This is only valid for phy runs.

is_analysis_run(data::LegendData, (period::DataPeriodLike, run::DataRunLike, cat::DataCategoryLike))

Return true if run is an cat analysis run for data in period.

is_lrun(data::LegendData, runsel::RunSelLike)

Return true if runsel is a valid run for data and therefore appears in the metadata.

livetime(data::LegendData, runsel::RunSelLike)

Get the livetime for data in physics data taking of run in period.

ljl_propfunc(expr::LJlExprLike)
ljl_propfunc(expr_string::AbstractString)

Compiles a PropertyFunctions.PropertyFunction from a LEGEND Julia expression.

See also parse_lpyexpr.

ljl_propfunc(expr_map::AbstractDict{Symbol,<:LJlExprLike})
ljl_propfunc(expr_map::AbstractDict{Symbol,<:AbstractString})
ljl_propfunc(expr_map::PropDict)

Compiles a map between output field-names and LEGEND Julia expressions to a single PropertyFunctions.PropertyFunction.

The generated function will return NamedTuples with the same property names as the keys of expr_map.

lpy_propfunc(expr_string::AbstractString)::PropertyFunctions.PropertyFunction

Generate a PropertyFunctions.PropertyFunction from a LEGEND Python expression.

See also parse_lpyexpr and ljl_propfunc.

lreport()
lreport(contents...)

Generate a LEGEND report, e.g. a data processing report.

Use lreport!(rpt, contents...) to add more content to a report.

Example:

using LegendDataManagement, StructArrays, IntervalSets, Plots

tbl = StructArray(
    col1 = rand(5), col2 = ClosedInterval.(rand(5), rand(5).+1),
    col3 = [rand(3) for i in 1:5], col4 = ProcessStatus.(rand(-1:1, 5)),
    col5 = [:a, :b, :c, :d, :e], col6 = ["a", "b", "c", "d", "e"],
    col7 = [:(a[1]), :(a[2]), :(a[3]), :(a[4]), :(a[5])]
)

rpt = lreport(
    "# New report",
    "Table 1:", tbl
)
lreport!(rpt, "Figure 1:", stephist(randn(10^3)))
lreport!(rpt, "Figure 2:", histogram2d(randn(10^4), randn(10^4), format = :png))

show(stdout, MIME"text/plain"(), rpt)
show(stdout, MIME"text/html"(), rpt)
show(stdout, MIME"text/markdown"(), rpt)

writelreport("report.txt", rpt)
writelreport("report.html", rpt)
writelreport("report.md", rpt)

See LegendDataManagement.lreport_for_show! for how to specialize the behavior of show for specific report content types.

lreport!(rpt::LegendReport, contents...)

Add more content to report rpt. See lreport for an example.

LegendDataManagement.lreport_for_show!(rpt::LegendReport, mime::MIME, content)

Add the contents of content to rpt in a way that is optimized for being displayed (e.g. via show) with the given mime type.

show(output, mime, rpt) first transforms rpt by converting all contents of rpt using lreport_for_show!(rpt::LegendReport, mime, content).

Defaults to lreport!(rpt, content), except for tables (Tables.istable(content) == true), which are converted to Markdown tables by default for uniform appearance.

lreport_for_show! is not inteded to be called by users, but to be specialized for specific types of content content. Content types not already supported will primarily require specialization of

lreport_for_show!(rpt::LegendReport, ::MIME"text/markdown", content::SomeType)

In some cases it may be desireable to specialize lreport_for_show! for MIME types like MIME"text/html" and MIME"text/plain" as well.

map_datafiles(
    f_process, f_open, data::LegendData,
    category::DataCategoryLike, filekeys::AbstractVector{<:FileKey}
)

Processes all filekeys in data for category.

Opens the files using f_open and processes them using f_process.

Returns a @NamedTuple{result::Dict{Filekey}, failed::Dict{Filekey}, success::Bool}:

• values(result) contains the results f_process(f_open(filename)) for all filenames referred to by category and filekeys.
• values(result) contains the error where processing failed.
• success equals isempty(failed)

parse_ljlexpr(expr_string::AbstractString)::LJlExprLike

Parse an LEGEND Julia expression and return a Julia syntax tree.

parse_lpyexpr(expr_string::AbstractString)::LJlExprLike

Parse an expression compatible with the LEGEND Python software and return a Julia syntax tree.

partitioninfo(data::LegendData, ch::ChannelId)
partitioninfo(data::LegendData, ch::ChannelId, part::DataPartitionLike)
partitioninfo(data::LegendData, ch::ChannelId, period::DataPeriodLike)

Return cross-period data partitions.

process_ljlexpr(expr::LJlExprLike, f_varsubst = identity)::LJlExprLike

Verify that expr is a valid LEGEND Julia expression and return it, with modifications if necessary.

Optionally substitute variables in expr using f_varsubst.

pydataprod_config(data::LegendData)

Get the Python data production configuration for data.

Use pydataprod_config(data)(valsel::AnyValiditySelection) to also set the time/category validity selection for the configuration.

Examples:

l200 = LegendData(:l200)
filekey = FileKey("l200-p02-r006-cal-20221226T200846Z")
pydataprod_config(l200)(filekey)

or

l200 = LegendData(:l200)
vsel = ValiditySelection("20221226T200846Z", :cal)
pydataprod_config(l200)(vsel)

pydataprod_parameters(data::LegendData)

Get the Julia data production parameters data.

Examples:

l200 = LegendData(:l200)
dataprod_config(l200)

read_filekeys(filename::AbstractString)::AbstractVector{FileKey}

Reads a list of FileKey from a text file, one file key per line.

Ignores empty lines. # may be used to start a comment in the file.

read_ldata(data::LegendData, selectors...; kwargs...)
read_ldata(f, data::LegendData, selectors...; kwargs...)
read_ldata(columns::NTuple{<:Any, Symbol}, data::LegendData, selectors::Tuple; kwargs...)
read_ldata(column::Symbol, data::LegendData, selectors::Tuple; kwargs...)

Read lh5 data from disk for a given set of selectors. After reading in, a PropertyFunction f can be applied to the data. If a tuple of Symbols is given, the properties from the tuple are selected. If the n_evts kwarg is provided, a random selection with n_evts number of events per file is performed. ch can be either a ChannelId or a DetectorId.

Examples

dsp = read_ldata(l200, :jldsp, filekey, ch)
dsp = read_ldata((:e_cusp, :e_trap, :blmean), l200, :jldsp, filekey, ch)
dsp = read_ldata(:e_cusp, l200, :jldsp, filekey, ch)
dsp = read_ldata(l200, :jldsp, :cal, :p03, :r000, ch)

dsp = read_ldata(l200, :jldsp, :cal, DataPartition(1), ch)
dsp = read_ldata(l200, :jldsp, :cal, DataPeriod(3), ch)
dsp = read_ldata(l200, :jldsp, :cal, runinfo(l200)[1:3], ch)

dsp = read_ldata(l200, :jldsp, filekey, ch; n_evts=1000)

readlprops(filename::AbstractString)
readprops(filenames::Vector{<:AbstractString})

Read a PropDict from a file and parse it to Unitful.Quantity and Measurements.Measurement objects.

Returns

• pd::PropDict with all :val fields converted to Unitful.Quantity objects and all :val fields converted to Measurements.Measurement objects.

runinfo(data::LegendData)::Table
runinfo(data::LegendData, runsel::RunSelLike)::NamedTuple
runinfo(data::LegendData, filekey::FileKey)::NamedTuple

Get the run information for data.

search_disk(::Type{<:DataSelector}, path::AbstractString)

Search on-disk data for data categories, periods, runs, and filekeys.

Examples:

l200 = LegendData(:l200)

search_disk(DataCategory, l200.tier[:raw])
search_disk(DataPeriod, l200.tier[:raw, :cal])
search_disk(DataRun, l200.tier[:raw, :cal, "p02"])
search_disk(FileKey, l200.tier[DataTier(:raw), :cal, DataPeriod(2), "r006"])

start_filekey(data::LegendData, runsel::RunCategorySelLike)

Get the starting filekey for data in period, run, category.

LegendDataManagement.tmp_filename(fname::AbstractString)

Returns a temporary filename, based on fname, in the same directory.

Does not create the temporary file.

write_filekeys(filename::AbstractString, filekeys::AbstractVector{<:FileKey})

Writes a list of FileKey to a text file, one file key per line.

writelprops(f::IO, p::PropDict; write_units::Bool=true, write_errors::Bool=true, mutliline::Bool=true, indent::Int=4)
writelprops(filename::AbstractString, p::PropDict; multiline::Bool=true, indent::Int=4)
writelprops(db::PropsDB, key::Union{Symbol, DataSelector}, p::PropDict; kwargs...)

Write a PropDict to a file and strip it to :val and :unit fields and :val and :err fields.

writelreport(filename::AbstractString, rpt::LegendReport)
writelreport(filename::AbstractString, mime::MIME, rpt::LegendReport)

Write lreport rpt to file filename.

channel2detector(data::LegendData, sel::Union{AnyValiditySelection, RunCategorySelLike}, channel::ChannelIdLike)

Get the DetectorId for a given ChannelId input:

• data, e.g. LegendData(:l200)`
• runsel: runselection, e.g. (DataPeriod(3), DataRun(0), :cal)
• channel: ChannelId e.g. ChannelId(1080005)`

output:

• DetectorId of corresponding channel

create_logtbl(result)

Create a log table for a given result which can be added in a report.

create_metadatatbl(filekey::FileKey)
create_metadatatbl(filekey::FileKey, part::DataPartitionLike)

Create a metadata table for a given filekey which can be added in a report.

create_pars(pd::PropDict, result::Dict{ChannelInfo, ChannelResult}) -> PropDict

Create a PropDict from a result of the parallel processing

create_validity(result) -> StructArray

Create a StructArray from a result of the parallel processing

data_starttime(data::LegendData, runsel::Union{AnyValiditySelection, RunCategorySelLike})

Extract startime as DateTime from file for a given run selection Input: * data: LegendData, e.g. LegendData(:l200) * runsel: runselection, e.g. (DataPeriod(3), DataRun(0), :cal)

detector2channel(data::LegendData, sel::Union{AnyValiditySelection, RunCategorySelLike}, detector::DetectorIdLike)

Get the ChannelId for a given detectorId input:

• data, e.g. LegendData(:l200)`
• runsel: runselection, e.g. (DataPeriod(3), DataRun(0), :cal)
• detector: DetectorID e.g. DetectorId(:P00573A)OR ChannelID e.g.ChannelId(1080005)``

output:

• ChannelId of corresponding detector

get_det_type(data::LegendData, det::DetectorIdLike)

Looks up the detector type for a given DetectorID.

get_hitchfilename(data::LegendData, setup::ExpSetupLike, period::DataPeriodLike, run::DataRunLike, category::DataCategoryLike, ch::ChannelIdLike)
get_hitchfilename(data::LegendData, filekey::FileKey, ch::ChannelIdLike)

Get the filename for the hitch data for a given channel.

get_mltrainfilename(data::LegendData, period::DataPeriodLike, category::DataCategoryLike)
get_mltrainfilename(data::LegendData, filekey::FileKey)

Get the filename for the machine learning training data.

get_partitionfilekeys(data::LegendData, part::DataPartitionLike, tier::DataTierLike, category::DataCategoryLike; only_good::Bool=true)

Get filekeys for a given partition.

Arguments

• data::LegendData: data object
• part::DataPartitionLike: partition to be searched in
• tier::DataTierLike: tier
• category::DataCategoryLike: category
• only_good::Bool=true: only get good filekeys

Return

• Vector{FileKey}: filekeys

get_partitionvalidity(data::LegendData, ch::ChannelIdLike, part::DataPartitionLike, cat::DataCategoryLike=:cal) -> Vector{@NamedTuple{period::DataPeriod, run::DataRun, validity::String}}

Get partition validity for a given channel and partition.

get_peaksfilename(data::LegendData, setup::ExpSetupLike, period::DataPeriodLike, run::DataRunLike, category::DataCategoryLike, ch::ChannelIdLike)
get_peaksfilename(data::LegendData, filekey::FileKey, ch::ChannelIdLike)

Get the filename for the peaks data for a given channel.

get_plottitle(setup::ExpSetupLike, period::DataPeriodLike, run::DataRunLike, category::DataCategoryLike, det::DetectorIdLike, process::String; additiional_type::String="")
get_plottitle(filekey::FileKey, det::DetectorIdLike, process::String; kwargs...)
get_plottitle(setup::ExpSetupLike, partition::DataPartitionLike, category::DataCategoryLike, det::DetectorIdLike, process::String; additiional_type::String="")

Get the title for a plot.

get_pltfilename(data::LegendData, setup::ExpSetupLike, period::DataPeriodLike, run::DataRunLike, category::DataCategoryLike, ch::ChannelIdLike, process::Symbol)
get_pltfilename(data::LegendData, filekey::FileKey, ch::ChannelIdLike, process::Symbol)
get_pltfilename(data::LegendData, partition::DataPartitionLike, setup::ExpSetupLike, category::DataCategoryLike, ch::ChannelIdLike, process::Symbol)

Get the filename for the plot file for a given setup, period, run, category, channel and process.

get_pltfolder(data::LegendData, period::DataPeriodLike, run::DataRunLike, category::DataCategoryLike, process::Symbol)
get_pltfolder(data::LegendData, filekey::FileKey, process::Symbol)
get_pltfolder(data::LegendData, partition::DataPartitionLike, category::DataCategoryLike, process::Symbol, ch::ChannelIdOrDetectorIDLike)

Get the folder for the plot files for a given period, run, category and process.

get_preportfilename(data::LegendData, setup::ExpSetupLike, period::DataPeriodLike, category::DataCategoryLike, process::Symbol)

Get the filename for the log file for a given setup, period, category and process.

get_preportfolder(data::LegendData, period::DataPeriodLike, category::DataCategoryLike)

Get the preport folder for the log files for a given period and category.

get_rreportfilename(data::LegendData, setup::ExpSetupLike, period::DataPeriodLike, run::DataRunLike, category::DataCategoryLike, process::Symbol)
get_rreportfilename(data::LegendData, filekey::FileKey, process::Symbol)

Get the filename for the log file for a given setup, period, run, category and process.

get_rreportfolder(data::LegendData, period::DataPeriodLike, run::DataRunLike, category::DataCategoryLike)

Get the rreport folder for the log files for a given period, run and category.

get_totalTimer(result::Vector)

Get the total timer from a result vector.

load_partition_ch(open_func::Function, flatten_func::Function, data::LegendData, partinfo::StructVector, tier::DataTierLike, cat::DataCategoryLike, ch::ChannelIdLike; data_keys::Tuple=(), n_evts::Int=-1, select_random::Bool=false)
load_partition_ch(open_func::Function, flatten_func::Function, data::LegendData, part::DataPartition, tier::DataTierLike, cat::DataCategoryLike, ch::ChannelIdLike; kwargs...)

Load data for a channel from a partition.

Arguments

• open_func::Function: function to open a file
• flatten_func::Function: function to flatten data
• data::LegendData: data object
• partinfo::StructVector: partition info
• tier::DataTierLike: tier
• cat::DataCategoryLike: category
• ch::ChannelIdLike: channel
• data_keys::Tuple=(): data keys, empty tuple selects all keys
• n_evts::Int=-1: number of events, -1 selects all events
• select_random::Bool=false: select events randomly

Return

• Table: data table with flattened events

load_raw_evt(open_func::Function, data::LegendData, ch::ChannelIdLike, data_hit::Table, sel_evt::Int)
load_raw_evt(open_func::Function, data::LegendData, ch::ChannelIdLike, data_hit::Table, sel_evt::UnitRange{Int})

Load data for a channel from a hitch file for a given selected event index or index range.

Arguments

• open_func::Function: function to open a file
• data::LegendData: data object
• ch::ChannelIdLike: channel
• data_hit::Table: hitch data
• sel_evt::Int/UnitRange{Int}: selected event index

Return

• Table: data table of raw events

load_run_ch(open_func::Function, flatten_func::Function, data::LegendData, filekeys::Vector{FileKey}, tier::DataTierLike, ch::ChannelIdLike; check_filekeys::Bool=true)

Load data for a channel from a list of filekeys in a given tier.

Arguments

• open_func::Function: function to open a file
• flatten_func::Function: function to flatten data
• data::LegendData: data object
• filekeys::Vector{FileKey}: list of filekeys
• tier::DataTierLike: tier to load data from
• ch::ChannelIdLike: channel to load data for
• check_filekeys::Bool=true: check if filekeys are valid

savelfig(save_func::Function, p, data::LegendData, setup::ExpSetupLike, period::DataPeriodLike, run::DataRunLike, category::DataCategoryLike, ch::ChannelIdLike, process::Symbol; kwargs...)
savelfig(save_func::Function, p, data::LegendData, filekey::FileKey, ch::ChannelIdLike, process::Symbol; kwargs...)
savelfig(save_func::Function, p, data::LegendData, partition::DataPartitionLike, setup::ExpSetupLike, category::DataCategoryLike, ch::ChannelIdLike, process::Symbol; kwargs...)

Save a lplot.

writevalidity(props_db::LegendDataManagement.PropsDB, filekey::FileKey; category::Symbol=:all)
writevalidity(props_db::LegendDataManagement.PropsDB, filekey::FileKey, part::DataPartitionLike; category::Symbol=:all)

Write validity for a given filekey.