lib.Calibration.Qfluxjumps module

class lib.Calibration.Qfluxjumps.DT(thr_count=600, thr_amp=200000.0, tol=100.0, depth=True, depth_number=0)

Bases: object

Methods

amplitude_filter(filpred, filindex, filcount)	Filter level transitions based on amplitude threshold.
calculate_levels(tt, todarray, nc[, consec])	Calculate level transitions using decision tree analysis.
calculate_start_end2(todarray, valini, ...)	Calculate start and end indices for level transitions.
change_values(xc, xcf[, max_gap])	Merge consecutive transitions that are close together.
count_filter(predunique, index, count)	Filter levels based on count threshold.
define_model(tt, todarray, num)	Define and fit a Decision Tree Regressor model.
uniqueindex(ypred)	Find unique predicted values and count their occurrences.

calculate_start_end

amplitude_filter(filpred, filindex, filcount)

Filter level transitions based on amplitude threshold.

Identifies transitions between consecutive levels that exceed the amplitude threshold (thr_amp).

Parameters:

• filpred (ndarray) – Filtered unique predicted values.

• filindex (ndarray) – Filtered indices.

• filcount (ndarray) – Filtered counts (unused but kept for consistency).

Returns:

• ampnew: List of amplitudes between consecutive levels.

• valini: List of initial level values.

• valfin: List of final level values.

• indexini: List of initial level indices.

• indexfin: List of final level indices.

Return type:

Tuple containing

calculate_levels(tt, todarray, nc, consec=True)

Calculate level transitions using decision tree analysis.

This is the main method that performs the complete level calculation pipeline: 1. Fits a decision tree model to identify discrete levels 2. Finds unique levels and filters by count 3. Identifies transitions between levels based on amplitude 4. Calculates start and end indices for each transition 5. Optionally merges consecutive transitions

Parameters:

• tt (ndarray) – 1D array of time values.

• todarray (ndarray) – 1D array containing the time-ordered data.

• nc (int) – Number of flux jumps (used to determine tree depth if depth=True).

• consec (bool) – If True, merge consecutive transitions that are close together (default: True).

Returns:

• xc_unique: List of start indices for level transitions.

• xcf_unique: List of end indices for level transitions.

Return type:

Tuple containing

calculate_start_end(todarray, valini, valfin, indexfin)

calculate_start_end2(todarray, valini, valfin, indexini, indexfin)

Calculate start and end indices for level transitions.

Determines the start and end positions of transitions between levels by finding where the signal matches the initial and final level values within the tolerance range.

Parameters:

• todarray (ndarray) – 1D array containing the time-ordered data.

• valini (list) – List of initial level values.

• valfin (list) – List of final level values.

• indexini (list) – List of initial level indices.

• indexfin (list) – List of final level indices.

Returns:

• start: Array of start indices for each transition.

• end: Array of end indices for each transition.

Return type:

Tuple containing

change_values(xc, xcf, max_gap=10)

Merge consecutive transitions that are close together.

Groups transitions that are within max_gap samples of each other into a single transition region.

Parameters:

• xc (Union[ndarray, list]) – Array or list of start indices.

• xcf (Union[ndarray, list]) – Array or list of end indices.

• max_gap (int) – Maximum gap between transitions to consider them consecutive (default: 10).

Returns:

• xc2: List of merged start indices.

• xcf2: List of merged end indices.

Return type:

Tuple containing

count_filter(predunique, index, count)

Filter levels based on count threshold.

Removes levels that don’t have enough occurrences (below thr_count).

Parameters:

• predunique (ndarray) – Array of unique predicted values.

• index (ndarray) – Array of first occurrence indices.

• count (ndarray) – Array of counts for each unique value.

Returns:

• filtered_pred: Filtered unique predicted values.

• filtered_index: Filtered indices.

• filtered_count: Filtered counts.

Return type:

Tuple containing

define_model(tt, todarray, num)

Define and fit a Decision Tree Regressor model.

Fits a Decision Tree Regressor to the TOD to identify discrete levels in the signal.

Parameters:

• tt (ndarray) – 1D array of time values.

• todarray (ndarray) – 1D array containing the TOD.

• num (int) – Number of flux jumps (used to determine tree depth).

Returns:

Predicted values from the decision tree model.

Return type:

ypred

uniqueindex(ypred)

Find unique predicted values and count their occurrences.

Parameters:

ypred (ndarray) – Array of predicted values from the decision tree.

Returns:

• predunique: Array of unique predicted values.

• index: Array of first occurrence indices for each unique value.

• count: Array of counts for each unique value.

Return type:

Tuple containing

class lib.Calibration.Qfluxjumps.badIV(directory, n_times, TES_number=256, filename='IVdict2025.yaml')

Bases: object

Methods

load_bad_iv()	Load TES with bad IV curves from a directory.
select_badIV()	Select TES indices whose bad IV occurrence count is >= n_times.

load_bad_iv()

Load TES with bad IV curves from a directory.

select_badIV()

Select TES indices whose bad IV occurrence count is >= n_times.

Parameters:

• bad_iv_counts (dict[int, int]) – Dictionary {TES_index: number_of_bad_IVs}.

• n_times (int) – Minimum number of bad IV occurrences.

Returns:

Sorted list of TES indices satisfying n_fallas >= n_times.

Return type:

list[int]

lib.Calibration.Qfluxjumps.compute_residual_jumps_with_z(signal, jump_starts, jump_ends, window=100, robust=False)

Compute residual jump amplitudes and z-scores.

Parameters:

• signal (array-like) – Corrected TOD.

• jump_starts (array-like) – Start indices of jumps.

• jump_ends (array-like) – End indices of jumps.

• window (int) – Samples used before and after jump.

• robust (bool) – If True, use median and MAD for robustness.

Returns:

• residuals (np.ndarray)

• z_scores (np.ndarray)

• rms_residual (float)

lib.Calibration.Qfluxjumps.compute_residual_metrics_from_results(results, todarray, window=100, robust=False, use_dt=True, use_corrected=True)

Compute residual jump metrics for all TES using the results dictionary.

Parameters:

• results (dict) – Results dictionary produced by the flux jump analysis. Expected keys include: - ‘TES_yes’, ‘TES_yes_dt’ - ‘jump_data’, ‘dt_jump_data’ - ‘corrected_data’ (dict of corrected TOD per TES index)

• todarray (np.ndarray) – Original TOD array of shape (n_TES, n_samples).

• window (int) – Samples used before and after each jump.

• robust (bool) – If True, use median and MAD for robustness.

• use_dt (bool) – If True, use DT-refined jumps (‘dt_jump_data’ and ‘TES_yes_dt’). If False, use Haar jumps (‘jump_data’ and ‘TES_yes’).

• use_corrected (bool) – If True, use corrected TOD from results[‘corrected_data’] when available; otherwise fall back to the original TOD in todarray.

Returns:

• metrics_per_tes (dict) – Dictionary keyed by TES index: {idx: {‘residuals’: np.ndarray,

  ’z_scores’: np.ndarray, ‘sigma’: np.ndarray}}

• global_rms_residual (float) – RMS of all residuals concatenated over TES (np.nan if none).

class lib.Calibration.Qfluxjumps.correction(region_off=5, region_amp=10, change_mode='const')

Bases: object

Methods

calculate_amplitude(todarray, xc, xcf, nc)	Calculate the amplitude offset for each flux jump.
changesignal_init(tod_new, xc, xcf)	Replace jump regions with noise using initial statistics.
changesignal_noise(tod_new, xc, xcf)	Replace jump regions with noise using local statistics.
constrained_realization(tod_new, xini, xfin)	Replace jump region with constrained realization based on power spectrum.
correct_TOD(todarray, offset, xc, xcf, nc)	Apply complete correction to the TOD array.
move_offset(todarray, offset, xc, xcf, nc)	Remove offset from the TOD after each flux jump.

calculate_amplitude(todarray, xc, xcf, nc)

Calculate the amplitude offset for each flux jump.

For each jump, computes the difference between the median value after the jump and the median value before the jump. This offset represents the amplitude of the discontinuity.

Parameters:

• todarray (ndarray) – 1D array containing the TOD.

• xc (ndarray) – Array of start indices for each jump.

• xcf (ndarray) – Array of end indices for each jump.

• nc (int) – Number of jumps.

Returns:

Array of offset amplitudes for each jump.

Return type:

offset

changesignal_init(tod_new, xc, xcf)

Replace jump regions with noise using initial statistics.

Uses the standard deviation computed from the beginning of the data and the mean computed before each jump to generate replacement noise.

Parameters:

• tod_new (ndarray) – TOD array with offsets removed.

• xc (ndarray) – Array of start indices for each jump.

• xcf (ndarray) – Array of end indices for each jump.

Returns:

Corrected TOD array with jump regions replaced by noise.

Return type:

y_cor

changesignal_noise(tod_new, xc, xcf)

Replace jump regions with noise using local statistics.

For each jump, computes the mean and standard deviation from the region immediately before the jump, then generates replacement noise using these local statistics.

Parameters:

• tod_new (ndarray) – TOD array with offsets removed.

• xc (ndarray) – Array of start indices for each jump.

• xcf (ndarray) – Array of end indices for each jump.

Returns:

Corrected TOD array with jump regions replaced by noise.

Return type:

y_cor

constrained_realization(tod_new, xini, xfin)

Replace jump region with constrained realization based on power spectrum.

Uses the power spectrum from regions before and after the jump to generate a synthetic signal that maintains the statistical properties while smoothly connecting the edges.

Parameters:

• tod_new (ndarray) – TOD array with offsets removed.

• xini (int) – Start index of the jump region.

• xfin (int) – End index of the jump region.

Returns:

TOD array with jump region replaced by constrained realization.

Return type:

y

correct_TOD(todarray, offset, xc, xcf, nc)

Apply complete correction to the TOD array.

This is the main correction method that: 1. Calculates jump amplitudes (if not provided) 2. Removes offsets from the data 3. Replaces jump regions based on the selected change_mode

Parameters:

• todarray (ndarray) – 1D array containing the time-ordered data.

• offset (ndarray) – Array of offset amplitudes for each jump.

• xc (ndarray) – Array of start indices for each jump.

• xcf (ndarray) – Array of end indices for each jump.

• nc (int) – Number of jumps.

Returns:

Corrected TOD array with flux jumps removed.

Return type:

tod_corr

move_offset(todarray, offset, xc, xcf, nc)

Remove offset from the TOD after each flux jump.

Applies the calculated offset correction to the data after each jump. The offsets are applied in reverse order (from last to first).

Parameters:

• todarray (ndarray) – 1D array containing the TOD.

• offset (ndarray) – Array of offset amplitudes for each jump.

• xc (ndarray) – Array of start indices for each jump.

• xcf (ndarray) – Array of end indices for each jump.

• nc (int) – Number of jumps.

Returns:

Copy of todarray with offsets removed after each jump.

Return type:

tod_new

class lib.Calibration.Qfluxjumps.fluxjumps(thr, window_size)

Bases: object

Methods

change_values(xc, xcf[, max_gap])	Merge consecutive jumps that are close together.
clusters(todarray, jumps)	Cluster detected jump indices to identify distinct flux jump events.
find_candidates(tod_haar, thr)	Find flux jump candidates using threshold comparisons.
haar_function(todarray)	Apply Haar filter to detect discontinuities in the TOD array.
initial_start_end(nc, idx_jumps, tod_haar, ...)	Determine the initial start and end indices of each flux jump cluster.
jumps_detection(todarray[, consec, nc_cond])	Main method to detect flux jumps in the TOD array.
unique(xc, xcf)	Remove duplicate start and end indices.

change_values(xc, xcf, max_gap=10)

Merge consecutive jumps that are close together.

Groups jumps that are within max_gap samples of each other into a single jump region, which is useful for handling multiple closely-spaced jumps as a single discontinuity.

Parameters:

• xc (Union[ndarray, list]) – Array or list of start indices.

• xcf (Union[ndarray, list]) – Array or list of end indices.

• max_gap (int) – Maximum gap between jumps to consider them consecutive (default: 10).

Returns:

• xc2: List of merged start indices.

• xcf2: List of merged end indices.

Return type:

Tuple containing

clusters(todarray, jumps)

Cluster detected jump indices to identify distinct flux jump events.

Uses DBSCAN clustering to group nearby jump indices into clusters, where each cluster represents a distinct flux jump event.

Parameters:

• todarray (ndarray) – 1D array of the TOD.

• jumps (ndarray) – Boolean array indicating jump candidate locations.

Returns:

• nc: Number of clusters (flux jump events) found.

• idx_jumps: Array of indices where jumps occur.

• clust: DBSCAN cluster object, or None if no clusters found.

Return type:

Tuple containing

find_candidates(tod_haar, thr)

Find flux jump candidates using threshold comparisons. Returns boolean mask indicating jump locations.

Parameters:

• tod_haar (ndarray) – Array of Haar filter values.

• thr (ndarray) – Array of threshold values to test.

Returns:

• jumps: Boolean array where True indicates a flux jump candidate.

Return type:

Tuple containing

haar_function(todarray)

Apply Haar filter to detect discontinuities in the TOD array.

Parameters:

todarray (ndarray) – 1D array containing the time-ordered data.

Returns:

Array of the same size as todarray with Haar filter values.

Return type:

tod_haar

initial_start_end(nc, idx_jumps, tod_haar, thr_used, clust)

Determine the initial start and end indices of each flux jump cluster.

For each cluster, finds where the Haar filter values drop below a fraction of the threshold to determine the actual start and end of the jump.

Parameters:

• nc (int) – Number of clusters.

• idx_jumps (ndarray) – Array of indices where jumps occur.

• tod_haar (ndarray) – Array of Haar filter values.

• thr_used (float) – Threshold value used for jump detection.

• clust (DBSCAN) – DBSCAN cluster object.

Returns:

• xc: Array of start indices for each flux jump.

• xcf: Array of end indices for each flux jump.

Return type:

Tuple containing

jumps_detection(todarray, consec=True, nc_cond=False)

Main method to detect flux jumps in the TOD array.

This method performs the complete flux jump detection pipeline: 1. Applies Haar filter to detect discontinuities 2. Finds jump candidates using thresholds 3. Clusters jumps into distinct events 4. Optionally re-thresholds if too many jumps detected 5. Determines start and end positions of jumps 6. Optionally merges consecutive jumps

Parameters:

• todarray (ndarray) – 1D array containing the TOD.

• consec (bool) – If True, merge consecutive jumps that are close together (default: True).

• nc_cond (bool) – If True, apply higher threshold if more than MAX_JUMPS_BEFORE_RETHRESHOLD jumps are detected (default: False).

Returns:

• nc_unique: Number of unique flux jumps detected.

• xc_unique: Array of start indices (empty array if no jumps).

• xcf_unique: Array of end indices (empty array if no jumps).

Return type:

Tuple containing

unique(xc, xcf)

Remove duplicate start and end indices.

Parameters:

• xc (ndarray) – Array of start indices.

• xcf (ndarray) – Array of end indices.

Returns:

• nc_unique: Number of unique jump pairs.

• xc_unique: Array of unique start indices.

• xcf_unique: Array of unique end indices.

Return type:

Tuple containing

lib.Calibration.Qfluxjumps.load_results(load_file, load_format='pickle')

Load previously saved analysis results.

lib.Calibration.Qfluxjumps.plot_corrections(tt, todarray, results, DT=True)

lib.Calibration.Qfluxjumps.plot_jump_detections(tt, todarray, results, DT=True)

lib.Calibration.Qfluxjumps.plot_no_jumps(tt, todarray, results)

lib.Calibration.Qfluxjumps.save_results(results, output_dir='./results_15_26_08', save_format='pickle', dataset_name='15.26.08')

Save analysis results to disk in pickle format.