SiPMs, events and stability

process_sipm_optimization_phy(processing_config, l200::LegendData, period::DataPeriod, run::DataRun;
    reprocess::Bool=false, timeout::Real=0)

Optimize the energy filters of the SiPM channels (spms) of one run from phy data and extract the trigger thresholds.

For each processable SiPM and each filter listed under e_filter in the optimization configuration, the DSP is run on a grid of filter window lengths. The window length with the best one photoelectron peak is selected (fit_sipm_wl), and the gain and the one photoelectron resolution are determined there. At the selected window length, the baseline distributions of the threshold waveforms are fitted (fit_sipm_threshold) and their sigma is taken as the trigger threshold.

Inputs

• raw tier of the phy filekeys of the run listed in the runinfo (excluding filekeys marked unprocessable), fields waveform_bit_drop and timestamp.
• jlaux tier fields timestamp and aux_trig of the pulser channel named by dataprod_config(l200).qc(filekey).pulser.puls_detector; waveforms within puls_ts_window of a pulser trigger are excluded.
• dataprod_config(l200).sipm(filekey): the default section (DSP settings) merged with the per-detector section, and its optimization subsection (max_wvfs, e_filter with e_grid_wl, kwargs and threshold).
• l200.par.rpars.sipmopt[period, run] (existing results, unless reprocess is set).

Outputs

• l200.par.rpars.sipmopt[period, run]: per detector and filter, the window length wl, gain, res_1pe and the trigger thresholds trig_threshold; the validity entry is written for (period, run).
• Plots per detector and filter: wl_sweep_<filter>, wl_sweep_calibration_<filter>, and per threshold type trigger_threshold_<threshold type>, all saved only at the debug plot level.
• If error_plots is on, a failed trigger threshold extraction saves error_trigger_threshold_extraction_<filter>_<threshold type>.
• Run report sipm_optimization_phy.

Keywords

• reprocess: discard the existing results of the run and optimize all filters of all detectors again. Otherwise, filters that already have results are skipped and only missing ones are optimized.
• timeout: seconds allowed per detector in parallel (0 = no limit).

Returns

nothing.

p_process_sipm_optimization_phy(processing_config, l200::LegendData, period::DataPeriod;
    reprocess::Bool=false, timeout::Real=0, only_first_period::Bool=true)

Optimize the energy filters of the SiPM channels (spms) per partition from phy data and extract the trigger thresholds, for all partitions that contain period.

The method is the same as in process_sipm_optimization_phy, but the waveforms of all runs of a detector's partition are used and the results are valid for the whole partition.

Inputs

• raw tier of the phy filekeys listed in the runinfo for all runs of each detector's partition (excluding filekeys marked unprocessable), fields waveform_bit_drop and timestamp, limited to n_evts events per detector.
• jlaux tier fields timestamp and aux_trig of the pulser channel named by dataprod_config(l200).qc(filekey).pulser.puls_detector; waveforms within puls_ts_window of a pulser trigger are excluded.
• dataprod_config(l200).sipm(filekey): the default section (DSP settings) merged with the per-detector section, and its optimization subsection using p_default merged with p (n_evts, max_wvfs, e_filter with e_grid_wl, kwargs and threshold).
• l200.par.ppars.sipmopt[det, part] (existing results, unless reprocess is set).

Outputs

• l200.par.ppars.sipmopt[det, part]: per filter, the window length wl, gain, res_1pe and the trigger thresholds trig_threshold, written for each detector as soon as it is finished; the validity of all partitions is written once at the end.
• Plots per detector and partition: per filter wl_sweep_<filter> and wl_sweep_calibration_<filter>, per threshold type trigger_threshold_<threshold type>. All of these plots are saved only at the debug plot level.
• If error_plots is on, a failed trigger threshold extraction saves error_trigger_threshold_extraction_<filter>_<threshold type>.
• Partition report sipm_optimization_phy.

Keywords

• reprocess: discard the existing results and optimize all filters of all detectors again. Otherwise, filters that already have results are skipped and only missing ones are optimized.
• timeout: seconds allowed per detector and partition in parallel (0 = no limit).
• only_first_period: if true, skip a detector whose partition starts in an earlier period than period; it is processed together with that earlier period. run_processing sets this to true when the previous period is also being processed.

Returns

true if at least one detector was skipped because of only_first_period, otherwise false.

process_sipm_calibration_phy(processing_config, l200::LegendData, period::DataPeriod, run::DataRun;
    reprocess::Bool=false, timeout::Real=0)

Calibrate the SiPM channels (spms) of one run from phy data.

For each processable SiPM and each energy type listed in the calibration configuration, the pulser-tagged events and the events failing the energy-type qc cut are removed, a simple calibration of the amplitude spectrum is made, the photoelectron peaks are fitted, and a polynomial calibration curve is fitted to the peak positions. Control plots are saved for each step.

Inputs

• dsp tier (DataTier(:jldsp)) of the phy data of the run, per SiPM.
• jlaux tier fields timestamp and aux_trig of the pulser channel named by dataprod_config(l200).qc(filekey).pulser.puls_detector; events within puls_ts_window of a pulser trigger are excluded.
• dataprod_config(l200).sipm(filekey).calibration: the default section merged with the per-detector section. It sets energy_types (each with qc, simple.kwargs, fit.min_pe, fit.max_pe, fit.kwargs) and pol_order.
• l200.par.rpars.sipmcal[period, run] (existing results, unless reprocess is set).

Outputs

• l200.par.rpars.sipmcal[period, run]: per detector and energy type, the simple calibration constants, the noise thresholds, the peak fit result (fit) and the calibration curve (cal); the validity entry is written for (period, run).
• Plots per detector and energy type: pe_uncalibrated_<energy type>, sipm_simple_calibration_<energy type>, sipm_peak_fits_<energy type>, sipm_calibration_curve_<energy type>. pe_uncalibrated_<energy type> and sipm_simple_calibration_<energy type> are saved only at the debug plot level.
• If error_plots is on, a failed simple calibration saves error_simple_calibration_<energy type>.
• Run report sipm_calibration_phy.

Keywords

• reprocess: discard the existing results of the run and calibrate all energy types of all detectors again. Otherwise, energy types that already have results are skipped and only missing ones are calibrated.
• timeout: seconds allowed per detector in parallel (0 = no limit).

Returns

nothing.

p_process_sipm_calibration_phy(processing_config, l200::LegendData, period::DataPeriod;
    reprocess::Bool=false, timeout::Real=0, only_first_period::Bool=true)

Calibrate the SiPM channels (spms) per partition from phy data, for all partitions that contain period.

The method is the same as in process_sipm_calibration_phy, but the data of all runs of a detector's partition are combined and the results are valid for the whole partition.

Inputs

• dsp tier (DataTier(:jldsp)) of the phy data of all runs of each detector's partition.
• jlaux tier fields timestamp and aux_trig of the pulser channel named by dataprod_config(l200).qc(filekey).pulser.puls_detector over the same runs; events within puls_ts_window of a pulser trigger are excluded.
• dataprod_config(l200).sipm(filekey).calibration, using the p_default section merged with the per-detector section p. It sets energy_types (each with qc, simple.kwargs, fit.min_pe, fit.max_pe, fit.kwargs) and pol_order.
• l200.par.ppars.sipmcal[det, part] (existing results, unless reprocess is set).

Outputs

• l200.par.ppars.sipmcal[det, part]: per energy type, the simple calibration constants, the noise thresholds, the peak fit result (fit) and the calibration curve (cal), written for each detector as soon as it is finished; the validity of all partitions is written once at the end.
• Plots per detector, partition and energy type: pe_uncalibrated_<energy type>, sipm_simple_calibration_<energy type>, sipm_peak_fits_<energy type>, sipm_calibration_curve_<energy type>. pe_uncalibrated_<energy type> and sipm_simple_calibration_<energy type> are saved only at the debug plot level.
• If error_plots is on, a failed simple calibration saves error_simple_calibration_<energy type>.
• Partition report sipm_calibration_phy.

Keywords

• reprocess: discard the existing results and calibrate all energy types of all detectors again. Otherwise, energy types that already have results are skipped and only missing ones are calibrated.
• timeout: seconds allowed per detector and partition in parallel (0 = no limit).
• only_first_period: if true, skip a detector whose partition starts in an earlier period than period; it is processed together with that earlier period. run_processing sets this to true when the previous period is also being processed.

Returns

true if at least one detector was skipped because of only_first_period, otherwise false.

process_evt_phy(processing_config, l200::LegendData, period::DataPeriod, run::DataRun;
    reprocess::Bool=false, timeout::Real=0, subsystems::Vector{Symbol}=[:geds, :spms, :pmts, :aux])

Build the event tier (jlevt) of the phy run from the jldsp files, combining the listed subsystems.

For each phy filekey of the run listed in the runinfo, the jldsp file must exist. The DSP data of the detectors of all subsystems are read, calibrated and combined into one event table by calibrate_all, including the cut flags. Per file the log counts forced triggers, pulser triggers and physical triggers (events with geds.is_valid_qc and more than one entry in geds.trig_e_det). If the PMT output is not empty and :pmts is in subsystems, a separate PMT file (jlpmt tier) is written.

Inputs

• jldsp tier of the phy filekeys of the run listed in the runinfo.
• Channel info of every system in subsystems and the calibration parameters used by calibrate_all.

Outputs

• jlevt tier: one file per phy filekey, table jlevt.
• jlpmt tier: one file per filekey with PMT data, table jlpmt.
• Run report :evt_phy (Main Log, Metadata, Results table with one row per filekey, Total Timing).

Keywords

• reprocess: recreate the jlevt file (and the jlpmt file) of each filekey, removing existing ones. If false, existing jlevt files are kept and only their trigger counts are read for the report.
• timeout: seconds allowed per filekey in parallel (0 = no limit). Failed filekeys are retried (retry=true).
• use_cache: write each output file to a local cache directory first and move it to its place when done; otherwise write it next to its final location (default: true)
• subsystems: systems included in the event table, default [:geds, :spms, :pmts, :aux].

Returns

nothing.

p_process_skm_phy(processing_config, l200::LegendData, period::DataPeriod;
    reprocess::Bool=false, timeout::Real=0, only_first_period::Bool=true)

Create the skimmed event tier (jlskm) of the phy analysis runs of period, one file per run.

For each run, all jlevt files of the run (except filekeys listed as bad) are read and only events that are neither pulser nor forced triggers are kept, and that have ged_pmt.is_valid_muon, geds.is_valid_qc, geds.is_valid_trig, geds.is_valid_hit, geds.multiplicity == 1 and geds.max_e_cusp_ctc_cal > 500.0u"keV". The report lists the fraction of the selected events that pass the PSD cut, the LAr cut, and both. The total exposure of the period is logged.

Inputs

• jlevt tier of the phy filekeys listed in the runinfo for each analysis run in period (run info phy.is_analysis_run).
• Channel info of the geds system for the phy analysis runs of period, merged over the runs by channelinfo(l200, period, :phy), and the bad-filekey list from bad_filekeys(l200).

Outputs

• jlskm tier: one file per run (named by the start filekey of the run), table jlskm.
• Partition report :skm_phy (Main Log, Metadata, Results table with one row per run).

Keywords

• reprocess: recreate the jlskm file of each run, removing existing ones. If false, existing files are kept and only the survival fractions are read for the report.
• timeout: seconds allowed per run in parallel (0 = no limit).
• use_cache: write each output file to a local cache directory first and move it to its place when done; otherwise write it next to its final location (default: true)
• only_first_period: accepted for a uniform call signature; it has no effect in this processor.

Returns

Always false: no run is skipped, so run_processing does not wait for this step in the previous period.

process_stability_plots_phy(processing_config, l200::LegendData, period::DataPeriod, run::DataRun;
    reprocess::Bool=false, timeout::Real=0)

Plot the stability of the HPGe detectors (geds) over one run from phy data.

For each processable detector, the following time-dependent quantities are plotted: baseline, baseline sigma, the e_10410 energy estimator, the pulser rate, and the gain stability. The gain stability compares the relative change of e_10410 of the pulser channel and of the detector for pulser-triggered events, scaled to keV at Qbb. Each plot is made and logged separately; a failing plot does not stop the others.

Inputs

• dsp tier (DataTier(:jldsp)) of the phy data of the run, per detector, fields timestamp, blmean, blsigma and e_10410; events with non-finite values are dropped.
• jlaux tier fields e_10410 and aux_trig of the pulser channel named by dataprod_config(l200).qc(filekey).pulser.puls_detector.
• dataprod_config(l200).qc(filekey).stability_plots: n_ref (number of events for the reference value), n_smooth (smoothing window of 20 * n_smooth seconds) and Qbb.

Outputs

• Plots per detector: stability_time_vs_bl, stability_time_vs_blsigma, stability_time_vs_e10410, stability_e10410_hist. If pulser-triggered events exist: stability_time_vs_e10410_trig and stability_pulser_rate. If more than n_ref finite pulser-triggered events exist: stability_gain_stability. Only stability_gain_stability is saved below the debug plot level.
• Run report stability_plots_phy.

No parameters are written.

Keywords

• reprocess: only logged; plots are always remade and overwritten.
• timeout: seconds allowed per detector in parallel (0 = no limit).

Returns

nothing.

p_process_stability_plots_phy(processing_config, l200::LegendData, period::DataPeriod;
    reprocess::Bool=false, timeout::Real=0, only_first_period::Bool=true)

Plot the stability of the HPGe detectors (geds) per partition from phy data, for all partitions that contain period.

The plots are the same as in process_stability_plots_phy, but the data of all runs of a detector's partition are shown together. Each plot is made and logged separately; a failing plot does not stop the others.

Inputs

• dsp tier (DataTier(:jldsp)) of the phy data of all runs of each detector's partition, fields timestamp, blmean, blsigma and e_10410; events with non-finite values are dropped.
• jlaux tier fields e_10410 and aux_trig of the pulser channel named by dataprod_config(l200).qc(filekey).pulser.puls_detector, over the same runs.
• dataprod_config(l200).qc(filekey).stability_plots: n_ref (number of events for the reference value), n_smooth (smoothing window of 20 * n_smooth seconds) and Qbb.

Outputs

• Plots per detector and partition: stability_time_vs_bl, stability_time_vs_blsigma, stability_time_vs_e10410, stability_e10410_hist. If pulser-triggered events exist: stability_time_vs_e10410_trig and stability_pulser_rate. If more than n_ref finite pulser-triggered events exist: stability_gain_stability. Only stability_pulser_rate and stability_gain_stability are saved below the debug plot level.
• Partition report stability_plots_phy.

No parameters are written.

Keywords

• reprocess: only logged; plots are always remade and overwritten.
• timeout: seconds allowed per detector and partition in parallel (0 = no limit).
• only_first_period: if true, skip a detector whose partition starts in an earlier period than period; it is processed together with that earlier period. run_processing sets this to true when the previous period is also being processed.

Returns

true if at least one detector was skipped because of only_first_period, otherwise false.