Pulse shape discrimination

process_aoe_optimization_cal(processing_config, l200::LegendData, period::DataPeriod, run::DataRun;
    reprocess::Bool=false, timeout::Real=0)

Optimize the window length of the A/E filters for the HPGe detectors (geds) of one cal run.

For each detector with usability == :on and low_aoe_status of :valid or :present, and for each filter type listed in the aoe_filter table of the configuration, the processor runs a short DSP routine on the Tl208 DEP and SEP waveforms for a grid of window lengths, applies the configured QC expression, and evaluates the SEP survival fraction after a simple A/E cut set on the DEP. It stores the selected window length together with its survival fraction.

Inputs

• First cal filekey of the run (start_filekey(l200, (period, run, :cal))).
• Windowed and presummed waveforms of the Tl208DEP_Bi212FEP and Tl208SEP selections from the jlpks tier.
• l200.par.rpars.pz[period, run] (decay times) and l200.par.rpars.fltopt[period, run] (energy filter parameters).
• dataprod_config(l200).dsp(filekey): the default section merged with the detector section for the DSP settings, including the window length grids a_grid_wl_<filter type>; and aoe_optimization (default merged with the detector section) with qc and the aoe_filter table. Each filter entry sets dep, dep_window, sep, sep_window, sep_rel_cut, min_aoe_quantile, max_aoe_quantile, min_aoe_offset, max_aoe_offset, dep_cut_search_fit_func and sep_cut_search_fit_func.
• The QC evaluator of the filekey (load_qc_evaluator).

Outputs

• l200.par.rpars.aoeopt[period, run]: per detector and filter type, the selected window length (wl), SEP survival fraction (sf) and the number of DEP and SEP events used (n_dep, n_sep), plus the matching validity entry.
• One plot per detector and filter type of the window length sweep, named <filter type>_sweep.
• If error_plots is on, a failed DSP of DEP and SEP saves error_dsp_dep_sep_<filter type> and a failed window length optimization error_window_length_optimization_<filter type>.
• Run report aoe_optimization with the processing status, a metadata table and one result row per detector and filter type.

Keywords

• reprocess: Discard the existing aoeopt parameters of the run and optimize all detectors and filter types again. Otherwise, filter types that already have results are kept and only missing ones are computed.
• timeout: Seconds allowed per detector in parallel; 0 means no limit.

Returns

The return value is not used by run_processing.

p_process_aoe_optimization_cal(processing_config, l200::LegendData, period::DataPeriod;
    reprocess::Bool=false, timeout::Real=0, max_wvfs::Int=15000, only_first_period::Bool=true)

Optimize the window length of the A/E filters for every detector and cal partition that contains period.

This is the partition-level counterpart of process_aoe_optimization_cal. The Tl208 DEP and SEP waveforms of all cal runs in a detector's partition are combined for one optimization per detector, partition and filter type. Detectors with low_aoe_status of :valid or :present are processed.

Inputs

• Channel info of the cal analysis runs of period, merged over the runs by channelinfo(l200, period, :cal), and the start filekey of the first run of each partition (selects the configuration).
• Windowed and presummed waveforms of the Tl208DEP_Bi212FEP and Tl208SEP selections from the jlpks tier of the cal runs in the partition. The configuration entry n_evts sets how many events are read per run, and select_random whether they are chosen randomly.
• l200.par.ppars.pz[det, part] (decay times) and l200.par.ppars.fltopt[det, part] (energy filter parameters).
• dataprod_config(l200).dsp(filekey): the default section merged with the detector section for the DSP settings; aoe_optimization.p_default merged with aoe_optimization.p[det] for qc, n_evts, select_random and the aoe_filter table (same entries as in process_aoe_optimization_cal).
• The QC evaluator of the filekey (load_qc_evaluator).

Outputs

• l200.par.ppars.aoeopt[det, part]: per filter type, the selected window length (wl), SEP survival fraction (sf) and the number of DEP and SEP events used (n_dep, n_sep), plus the validity of ppars.aoeopt for period.
• One plot per detector, partition and filter type, named aoe_sweep_<filter type>, saved only at the debug plot level.
• If error_plots is on, a failed DSP of DEP and SEP saves error_dsp_dep_sep_<filter type> and a failed window length optimization error_window_length_optimization_<filter type>.
• Partition report aoe_optimization with the processing status, a metadata table and one result row per detector, partition and filter type.

Keywords

• reprocess: Discard the existing ppars.aoeopt parameters of each detector and partition and optimize all filter types again. Otherwise, filter types that already have results are kept.
• timeout: Seconds allowed per detector and partition in parallel; 0 means no limit.
• max_wvfs: Maximum number of DEP waveforms and of SEP waveforms used per detector and partition (default 15000). If more are loaded, max_wvfs of them are selected at random, without repetition and in their original order.
• only_first_period: If true, a detector whose partition starts in an earlier period is skipped and logged as skipped. run_processing sets it to true when the previous period is also being processed.

Returns

true if at least one detector was skipped because of only_first_period, otherwise false. run_processing uses this to wait for the same step in the previous period.

process_aoe_calibration_cut_cal(processing_config, l200::LegendData, period::DataPeriod, run::DataRun;
    reprocess::Bool=false, timeout::Real=0)

Calibrate A/E and generate the A/E cuts for the HPGe detectors (geds) of one cal run.

For each detector with usability == :on and low_aoe_status of :valid or :present, the A/E of every type listed in aoe_funcs is normalized and the Compton bands (compton_bands) are fitted. The fitted mean and width are described with energy-dependent correction functions, optionally refined by a combined fit (use_combined_fit). If apply_ctc is set, a charge trapping correction against the effective drift time (qdrift_expression) follows. For each classifier in aoe_classifiers, the low A/E cut is searched on the DEP, the high cut is set from sigma_high_sided, and survival fractions are computed for the configured peaks and the Qbb window with the low cut alone and with both cuts (ds).

Inputs

• First cal filekey of the run.
• l200.par.rpars.ecal[period, run]: the calibration function of the energy type e_type.
• jlqcs tier (single pulse, non-pulser events) and jldsp tier of the cal run.
• dataprod_config(l200).psd(filekey).aoe, default merged with the detector section: e_type, aoe_funcs, aoe_classifiers, compton_bands, compton_window, p_value, use_combined_fit, apply_ctc, qdrift_expression, sigma_high_sided, the DEP cut search settings (dep, dep_window, dep_cut_search_interval, dep_bin_width_window, dep_cut_search_rtol, dep_cut_search_maxiters, dep_cut_search_target_sf, dep_cut_search_fixed_position, dep_cut_search_fit_func) and the survival fraction settings (aoe_peaks, aoe_peaks_names, aoe_peaks_windows_left, aoe_peaks_windows_right, aoe_peaks_bin_width_window, aoe_peaks_fit_funcs, qbb, qbb_window).

Outputs

• l200.par.rpars.aoe[period, run]: per detector, one entry per A/E type (func, ctc, correction, apply_ctc, use_combined_fit) and one entry per classifier (lowcut, highcut, and peaks and qbb survival fractions with the low cut and with both cuts ds), plus the matching validity entry. The classifier name is the A/E type name followed by _classifier.
• Plots per detector: aoe_uncalibrated_<type>, compton_bands_mu_<type>, compton_bands_sigma_<type>, aoe_ctc_<type> (if apply_ctc), aoe_normalized_<type>, aoe_normalized_<classifier>, aoe_energy_afterAoE_zoom_<classifier> and aoe_peaks_ds_sf_<classifier>. Only aoe_normalized_<classifier> is saved below the debug plot level.
• If error_plots is on, a failed Compton band fitting saves error_compton_band_fitting_<type> and a failed cut generation error_cut_generation_<classifier>.
• Run report aoe_calibration_cut with the processing status, a metadata table, and one row per detector and A/E type combining the calibration and the cut results.

Keywords

• reprocess: Discard the existing aoe parameters of the run and process all detectors, A/E types and classifiers again. Otherwise, entries that already exist are kept and only missing ones are computed.
• timeout: Seconds allowed per detector in parallel; 0 means no limit.

Returns

The return value is not used by run_processing.

p_process_aoe_calibration_cut_cal(processing_config, l200::LegendData, period::DataPeriod;
    reprocess::Bool=false, timeout::Real=0, only_first_period::Bool=true)

Calibrate A/E and generate the A/E cuts for every detector and cal partition that contains period.

This is the partition-level counterpart of process_aoe_calibration_cut_cal: the cal data of all runs in a detector's partition are combined and the A/E normalization, Compton band fits, optional charge trapping correction and cut search run once per detector and partition. Detectors with low_aoe_status of :valid or :present are processed. In addition, the Compton-band A/E is plotted against time over the partition.

Inputs

• l200.par.rpars.ecal[period, run] of every run in the partition: the calibration function of the energy type e_type.
• jlqcs tier (single pulse, non-pulser events) and jldsp tier of all cal runs in the partition.
• dataprod_config(l200).psd(filekey).aoe of the first run of the partition, p_default merged with p[det], with the same entries as described for process_aoe_calibration_cut_cal.

Outputs

• l200.par.ppars.aoe[det, part]: per A/E type (func, ctc, correction, apply_ctc, use_combined_fit) and per classifier (lowcut, highcut, peaks and qbb survival fractions), plus the validity of ppars.aoe for period. The file is written only if the detector has results.
• Plots per detector and partition: aoe_uncalibrated_<type>, aoe_stability_<type>, compton_bands_mu_<type>, compton_bands_sigma_<type>, aoe_ctc_<type> and aoe_compton_after_ctc_<type> (if apply_ctc), aoe_normalized_<type>, aoe_normalized_<classifier>, aoe_energy_afterAoE_zoom_<classifier> and aoe_peaks_sf_<classifier>. Only aoe_normalized_<classifier> is saved below the debug plot level.
• If error_plots is on, a failed Compton band fitting saves error_compton_band_fitting_<type> and a failed cut generation error_cut_generation_<classifier>.
• Partition report aoe_calibration_cut with the processing status, a metadata table, and one row per detector, partition and A/E type.

Keywords

• reprocess: Discard the existing ppars.aoe parameters of each detector and partition and process all A/E types and classifiers again. Otherwise, entries that already exist are kept.
• timeout: Seconds allowed per detector and partition in parallel; 0 means no limit.
• only_first_period: If true, a detector whose partition starts in an earlier period is skipped and logged as skipped. run_processing sets it to true when the previous period is also being processed.

Returns

true if at least one detector was skipped because of only_first_period, otherwise false. run_processing uses this to wait for the same step in the previous period.

p_process_aoe_cut_cal(processing_config, l200::LegendData, period::DataPeriod;
    reprocess::Bool=false, timeout::Real=0, only_first_period::Bool=true)

Generate the A/E cuts from the combined cal data of every detector and partition that contains period.

The cal data of all runs in a detector's partition are calibrated with the run-level parameters (rpars, category aoe) through calibrate_ged_detector_data. For each A/E type in aoe_funcs, the low A/E cut of its classifier <type>_classifier is searched on the DEP of the combined data, the high cut is set from sigma_high_sided, and survival fractions are computed for the configured peaks and the Qbb window with the low cut alone and with both cuts (ds). Detectors with low_aoe_status of :valid or :present are processed.

Inputs

• jlqcs tier (single pulse, non-pulser events) and jldsp tier of all cal runs in the partition, calibrated with the run-level A/E parameters l200.par.rpars.aoe (and the parameters calibrate_ged_detector_data reads for the energy).
• dataprod_config(l200).psd(filekey).aoe of the first run of the partition, p_default merged with p[det]: e_type, aoe_funcs (only the keys), sigma_high_sided, the DEP cut search settings (dep, dep_window, dep_cut_search_interval, dep_bin_width_window, dep_cut_search_rtol, dep_cut_search_maxiters, dep_cut_search_target_sf, dep_cut_search_fixed_position, dep_cut_search_fit_func) and the survival fraction settings (aoe_peaks, aoe_peaks_names, aoe_peaks_windows_left, aoe_peaks_windows_right, aoe_peaks_bin_width_window, aoe_peaks_fit_funcs, qbb, qbb_window).

Outputs

• l200.par.ppars.aoecut[det, part]: per classifier, lowcut, highcut, and peaks and qbb survival fractions with the low cut and with both cuts ds, plus the validity of ppars.aoecut for period.
• Plots per detector and partition: aoe_normalized_runcal_<classifier>, aoe_energy_afterAoE_zoom_runcal_<classifier> and aoe_peaks_sf_runcal_<classifier>. Only aoe_normalized_runcal_<classifier> is saved below the debug plot level.
• If error_plots is on, a failed cut generation saves error_cut_generation_<classifier> and a failed survival fraction step error_survival_fractions_<classifier>.
• Partition report aoe_cut with the processing status, a metadata table and one result row per detector, partition and classifier.

Keywords

• reprocess: Discard the existing ppars.aoecut parameters of each detector and partition and process all classifiers again. Otherwise, classifiers that already have results are kept.
• timeout: Seconds allowed per detector and partition in parallel; 0 means no limit.
• only_first_period: If true, a detector whose partition starts in an earlier period is skipped and logged as skipped. run_processing sets it to true when the previous period is also being processed.

Returns

true if at least one detector was skipped because of only_first_period, otherwise false. run_processing uses this to wait for the same step in the previous period.

process_lq_calibration_cut_cal(processing_config, l200::LegendData, period::DataPeriod, run::DataRun;
    reprocess::Bool=false, timeout::Real=0)

Calibrate LQ and generate the LQ cut for the HPGe detectors (geds) of one cal run.

For each detector with usability == :on and lq_status of :valid or :present, the energy-corrected LQ of every type in lq_funcs is normalized and corrected for its dependence on the effective drift time (qdrift_expression) using DEP events. The corrected LQ is then normalized with the DEP peak and its sidebands. For each classifier in lq_classifiers, the high cut is the configured value high_cut_sigma of the normalized LQ. Survival fractions are computed for the configured peaks and the Qbb window with that cut.

Inputs

• First cal filekey of the run.
• l200.par.rpars.ecal[period, run]: the calibration function of the energy type e_type, and the DEP resolution from the Tl208DEP peak fit of the same energy type.
• jlqcs tier (single pulse, non-pulser events) and jldsp tier of the cal run.
• dataprod_config(l200).psd(filekey).lq, default merged with the detector section: e_type, lq_funcs, lq_classifiers, qdrift_expression, dep_mu, the drift time correction settings (ctc_dep_edgesigma, ctc_lq_precut_relative_cut, ctc_driftime_cutoff_method, lq_outlier_sigma, dt_eff_outlier_sigma, ctc_dt_eff_low_quantile, ctc_dt_eff_high_quantile, pol_fit_order, ctc_uncertainty), the normalization settings (dep_sideband_sigma, cut_truncation_sigma, cut_uncertainty) and the cut and survival fraction settings (high_cut_sigma, lq_peaks, lq_peaks_names, lq_peaks_windows_left, lq_peaks_windows_right, lq_peaks_fit_funcs, qbb, qbb_window).

Outputs

• l200.par.rpars.lq[period, run]: per detector, one entry per LQ type (the drift time correction and normalization results, with func the full LQ expression, and the mean, standard deviation and median of the energy-corrected LQ) and one entry per classifier (highcut, peaks, qbb), plus the matching validity entry. The classifier name is the LQ type name followed by _classifier.
• Plots per detector: lq_ctc_DEP_<type>, lq_ctc_<type>, lq_dep_normalization_<type>, lq_sidebands_<type>, lq_sideband_position_<type>, lq_classalized_<classifier>, lq_energy_after_<classifier> and lq_cut_fraction_<classifier>. Only lq_classalized_<classifier> is saved below the debug plot level.
• If error_plots is on, a failed drift time correction saves error_drift_time_correction_<type>, a failed LQ normalization error_lq_normalization_<type> and a failed peak survival fraction step error_peak_survival_fractions_<classifier>.
• Run report lq_calibration_cut with the processing status, a metadata table, and one row per detector and LQ type combining the calibration and the cut results.

Keywords

• reprocess: Discard the existing lq parameters of the run and process all detectors, LQ types and classifiers again. Otherwise, entries that already exist are kept and only missing ones are computed.
• timeout: Seconds allowed per detector in parallel; 0 means no limit.

Returns

The return value is not used by run_processing.

p_process_lq_calibration_cut_cal(processing_config, l200::LegendData, period::DataPeriod;
    reprocess::Bool=false, timeout::Real=0, only_first_period::Bool=true)

Calibrate LQ and generate the LQ cut for every detector and cal partition that contains period.

This is the partition-level counterpart of process_lq_calibration_cut_cal: the cal data of all runs in a detector's partition are combined, and the drift time correction, normalization, cut and survival fractions are computed once per detector and partition. Detectors with lq_status of :valid or :present are processed.

Inputs

• l200.par.rpars.ecal[period, run] of every run in the partition: the calibration function of the energy type e_type.
• l200.par.ppars.ecal[det, part]: the DEP resolution from the Tl208DEP peak fit of the same energy type.
• jlqcs tier (single pulse, non-pulser events) and jldsp tier of all cal runs in the partition.
• dataprod_config(l200).psd(filekey).lq of the first run of the partition, p_default merged with p[det], with the same entries as described for process_lq_calibration_cut_cal.

Outputs

• l200.par.ppars.lq[det, part]: per LQ type the drift time correction and normalization results, and per classifier highcut, peaks and qbb, plus the validity of ppars.lq for period.
• Plots per detector and partition: lq_ctc_DEP_<type>, lq_ctc_<type>, lq_dep_normalization_<type>, lq_sidebands_<type>, lq_sideband_position_<type>, lq_classalized_<classifier> and lq_energy_after_<classifier>. Only lq_classalized_<classifier> is saved below the debug plot level.
• If error_plots is on, a failed drift time correction saves error_drift_time_correction_<type>, a failed LQ normalization error_lq_normalization_<type> and a failed peak survival fraction step error_peak_survival_fractions_<classifier>.
• Partition report lq_calibration_cut with the processing status, a metadata table, and one row per detector, partition and LQ type.

Keywords

• reprocess: Discard the existing ppars.lq parameters of each detector and partition and process all LQ types and classifiers again.
• timeout: Seconds allowed per detector and partition in parallel; 0 means no limit.
• only_first_period: If true, a detector whose partition starts in an earlier period is skipped and logged as skipped. run_processing sets it to true when the previous period is also being processed.

Returns

true if at least one detector was skipped because of only_first_period, otherwise false. run_processing uses this to wait for the same step in the previous period.

process_psd_efficiencies_cal(processing_config, l200::LegendData, period::DataPeriod, run::DataRun;
    reprocess::Bool=false, timeout::Real=0, use_partition_pars::Bool=true)

Compute the survival fractions of the combined A/E and LQ cuts for the HPGe detectors (geds) of one cal run.

For each detector with usability == :on and low_aoe_status of :valid or :present, and for each classifier in the psd_classifiers table, the processor applies the A/E and LQ cuts to the cal data of the run and fits the configured peaks and the Qbb window. Survival fractions are evaluated for four selections: the low A/E cut (low), the low and high A/E cuts (ds), the low A/E cut and the LQ cut (low_lq), and all three cuts (lq_ds).

Inputs

• First cal filekey of the run.
• l200.par.rpars.ecal[period, run]: the calibration function of the energy type e_type.
• A/E and LQ parameters of the run: the aoe and lq parameters from ppars if use_partition_pars is true, otherwise from rpars. The low A/E cut is lowcut of the A/E classifier and the LQ cut is highcut of the LQ classifier.
• jlqcs tier (single pulse, non-pulser events) and jldsp tier of the cal run.
• dataprod_config(l200).psd(filekey).psd, default merged with the detector section: e_type, psd_classifiers (per classifier name, a list of the A/E classifier, the high A/E cut and the LQ classifier), psd_peaks, psd_peaks_names, psd_peaks_windows_left, psd_peaks_windows_right, psd_peaks_bin_width_window, psd_peaks_fit_funcs, qbb and qbb_window.

Outputs

• l200.par.rpars.psd[period, run]: per detector and classifier, cuts (lowcut, highcut, lq) and the peaks and qbb survival fractions for the four selections, plus the matching validity entry.
• Run report psd_efficiencies with the processing status, a metadata table and one result row per detector and classifier.
• At the debug plot level, per detector and classifier, the survival fractions of the A/E peaks, aoe_peaks_ds_sf_<classifier>.
• If error_plots is on, a failed A/E data loading saves error_aoe_data_loading_<classifier> and a failed survival fraction step error_survival_fraction_<classifier>.

Keywords

• reprocess: Discard the existing psd parameters of the run and process all detectors and classifiers again. Otherwise, classifiers that already have results are kept.
• timeout: Seconds allowed per detector in parallel; 0 means no limit.
• use_partition_pars: If true (default), read the A/E and LQ cuts from the partition parameters (ppars); if false, from the run parameters (rpars).

Returns

The return value is not used by run_processing.

p_process_psd_efficiencies_cal(processing_config, l200::LegendData, period::DataPeriod;
    reprocess::Bool=false, timeout::Real=0, only_first_period::Bool=true)

Compute the survival fractions of the combined A/E and LQ cuts for every detector and cal partition that contains period.

The cal data of all runs in a detector's partition are calibrated with calibrate_ged_detector_data, which applies the A/E and LQ cut flags (<A/E classifier>_low_cut and <LQ classifier>_high_cut) from the parameters selected by get_ged_evt_kwargs. The high A/E cut is taken from the psd_classifiers table of the configuration. Peaks and the Qbb window are fitted for five selections: the low A/E cut (low), the high A/E cut (high), the low and high A/E cuts (ds), the low A/E cut and the LQ cut (low_lq), and all three cuts (lq_ds). Detectors with low_aoe_status of :valid or :present are processed.

Inputs

• jlqcs tier (single pulse, non-pulser events) and jldsp tier of all cal runs in the partition.
• Calibration and cut parameters as read by calibrate_ged_detector_data with the keyword arguments of get_ged_evt_kwargs(l200, filekey).
• dataprod_config(l200).psd(filekey).psd of the first run of the partition, default merged with the detector section: e_type, psd_classifiers (per classifier name, a list of the A/E classifier, the high A/E cut and the LQ classifier), psd_peaks, psd_peaks_names, psd_peaks_windows_left, psd_peaks_windows_right, psd_peaks_bin_width_window, psd_peaks_fit_funcs, qbb and qbb_window.

Outputs

• l200.par.ppars.psd[det, part]: per classifier, cuts (highcut is the configured high A/E cut; lowcut and lq are stored as NaN) and the peaks and qbb survival fractions for the five selections, plus the validity of ppars.psd for period. The file is written only if the detector has results.
• Partition report psd_efficiencies with the processing status, a metadata table and one result row per detector, partition and classifier.
• At the debug plot level, per detector and classifier, the survival fractions of the A/E peaks, aoe_peaks_sf_<classifier>.
• If error_plots is on, a failed survival fraction step saves error_survival_fraction_<classifier>.

Keywords

• reprocess: Discard the existing ppars.psd parameters of each detector and partition and process all classifiers again. Otherwise, classifiers that already have results are kept.
• timeout: Seconds allowed per detector and partition in parallel; 0 means no limit.
• only_first_period: If true, a detector whose partition starts in an earlier period is skipped and logged as skipped. run_processing sets it to true when the previous period is also being processed.

Returns

true if at least one detector was skipped because of only_first_period, otherwise false. run_processing uses this to wait for the same step in the previous period.