RFI Detection and Mitigation for Dithered PRF mode by hb6688 · Pull Request #235 · isce-framework/isce3

hb6688 · 2026-03-15T20:30:49Z

This PR makes the following changes:

Separately computes sample covariance matrix for dithered-PRF and constant-PRF modes of operations
Skip threshold blocks when there are not enough valid samples for dithered-PRF mode

rad-eng-59 · 2026-03-16T22:18:35Z

python/packages/isce3/signal/compute_evd_cpi.py

+        else:
+            cov[i, i] = 0.0


Suggested change

else:

cov[i, i] = 0.0

rad-eng-59 · 2026-03-16T23:01:32Z

python/packages/isce3/signal/compute_evd_cpi.py

+        Gap-excluded sample covariance matrix.
+    """
+
+    num_pulses, num_rng_samples = data.shape


Add a check to see if x percent of total number of samples is above min required CPI length per theory and if not issue a warning.

FYI, for fixed PRF, make sure to check the number of valid samples (fixed over all CPI pulses) meets the min requirement and they are not all zeros.

rad-eng-59 · 2026-03-16T23:15:55Z

python/packages/isce3/signal/compute_evd_cpi.py

+
+    # compute number of CPIs
+    num_pulses, num_rng_samples = raw_data.shape
+    num_cpi = num_pulses // cpi_len


Suggested change

num_cpi = num_pulses // cpi_len

if cpi_len > num_pulses:

raise ValueError(f'"cpi_len" {cpi_len} is larger than number of pulses {num_pulses}!')

num_cpi = num_pulses // cpi_len

rad-eng-59 · 2026-03-17T00:19:09Z

python/packages/isce3/signal/compute_evd_cpi.py

+        # Eigenvalues.  Zero Eigenvalues are replaced by a small epsilon, 1e-30 to 
+        # ensure log values of such are greater than zero.
+        eig_val_sort_db = 10*np.log10(np.maximum(np.abs(eig_val_sort), 1e-30))
+        usable_rank = int(np.count_nonzero(eig_val_sort_db > 0))


Adding some user-defined rank computation per max expected dynamic range of eigen values.

Suggested change

usable_rank = int(np.count_nonzero(eig_val_sort_db > 0))

# do we need to peak normalize this?

eig_val_norm = np.copy(eig_val_sort)

if eig_val_sort[0] > eig_val_sort[-1]:

eig_val_norm /= eig_val_sort[0]

# set to max possible dynamic range (16 bits ~ 96 dB ~ 1e-9) or better to ENOB + floating-point precision ( < 8 bit ~ 50 dB ~ 1e-5)

usable_rank = np.sum(~np.isclose(eig_val_norm, 0, atol=1e-9))

rad-eng-59 · 2026-03-17T00:36:41Z

python/packages/isce3/signal/compute_evd_cpi.py

+        eig_val_sort_db = 10*np.log10(np.maximum(np.abs(eig_val_sort), 1e-30))
+        usable_rank = int(np.count_nonzero(eig_val_sort_db > 0))
+
+        if usable_rank < noise_ev_idx + 1:


Replace noise_ev_idx with min_cpi_rank and then before for loop add error check min_cpi_rank <= cpi_len and issue a warning if not 1 < min_cpi_rank < cpi_len - 1 (recommended range in docstring).

Suggested change

if usable_rank < noise_ev_idx + 1:

if usable_rank < min_cpi_rank:

rad-eng-59 · 2026-03-17T00:39:34Z

python/packages/isce3/signal/compute_evd_cpi.py

+    off_diag_overlap_ratio: float = 0.1,
+    diag_valid_ratio: float = 0.05, 


At the runconfig, it is assumed that the num_range_sample_block >= 250, then here are the suggested default percentage.

Suggested change

off_diag_overlap_ratio: float = 0.1,

diag_valid_ratio: float = 0.05,

off_diag_overlap_ratio: float = 0.25,

diag_valid_ratio: float = 0.2,

Tyler-g-hudson

I looked over your error strings and added some commentary on how to make them more descriptive. In one of them, I think there is also a small logic error which could lead to an edge case.

Tyler-g-hudson · 2026-03-17T00:49:40Z

python/packages/isce3/signal/compute_evd_cpi.py

+        raise ValueError(
+            "Minimum number of samples in a range block to estimate Sample Covariance"
+            f" Matrix is {rng_samples_min}! Current number of samples per range block"
+            f" is {num_rng_samples}!"
+        )


This block assumes that a user with no knowledge of RFI mitigation is going to understand the significance of these numbers and how they are calculated. In general, it's most helpful to the user to be able to identify at a glance:

What code is throwing this error (e.g. start the ValueError string with something like 'RFI detection:')

How the errored values were acquired. In this case, it looks like these numbers are coming from the shape of the data and the CPI length; Some quick instruction regarding setting an appropriate CPI length for this data would be useful here e.g. CPI length must be no more than half of the number of range pixels.

Tyler-g-hudson · 2026-03-17T00:51:05Z

python/packages/isce3/signal/compute_evd_cpi.py

+        raise ValueError(
+            f"Coherent Processing Interval length exceeds total number of pulses {num_pulses}!"
+        )


Suggested change

raise ValueError(

f"Coherent Processing Interval length exceeds total number of pulses {num_pulses}!"

)

raise ValueError(

f"Coherent Processing Interval length (cpi_len) exceeds total number of pulses {num_pulses}!"

)

Tyler-g-hudson · 2026-03-17T00:52:28Z

python/packages/isce3/signal/compute_evd_cpi.py

+        mask_valid_cpi = np.ones(raw_data.shape, dtype=bool)
+
+    if mask_valid_cpi.shape != raw_data.shape:
+        raise ValueError(f"mask shape {mask_valid_cpi.shape} != data shape {raw_data.shape}")


Suggested change

raise ValueError(f"mask shape {mask_valid_cpi.shape} != data shape {raw_data.shape}")

raise ValueError(f"Valid CPI mask shape {mask_valid_cpi.shape} != data shape {raw_data.shape}")

Tyler-g-hudson · 2026-03-17T00:53:04Z

python/packages/isce3/signal/compute_evd_cpi.py

+        mask_valid_cpi = np.ones(data.shape, dtype=bool)
+
+    if mask_valid_cpi.shape != data.shape:
+        raise ValueError(f"mask shape {mask_valid_cpi.shape} != data shape {data.shape}")


Suggested change

raise ValueError(f"mask shape {mask_valid_cpi.shape} != data shape {data.shape}")

raise ValueError(f"Valid CPI mask shape {mask_valid_cpi.shape} != data shape {data.shape}")

Tyler-g-hudson · 2026-03-17T00:54:25Z

python/packages/isce3/signal/rfi_detection_evd.py

-    if detect_threshold <= 0:
-        raise ValueError("Detection threshold must be a positive value!")
+    if (not np.isfinite(detect_threshold)) or (detect_threshold <= 0):
+        #raise ValueError("Detection threshold must be a positive value!")


It looks like this ValueError is no longer needed

Suggested change

#raise ValueError("Detection threshold must be a positive value!")

Tyler-g-hudson · 2026-03-17T00:57:16Z

python/packages/isce3/signal/rfi_process_evd.py

+    if noise_ev_idx >= cpi_len:
+        raise ValueError(
+            f"noise_ev_idx must be less than {cpi_len - 1}, got {noise_ev_idx}."
+    )


The logic here and the error statement say two different things; the error output says it needs to be less than cpi_len - 1 but the logic only checks if it's less than cpi_len.

Suggested change

if noise_ev_idx >= cpi_len:

raise ValueError(

f"noise_ev_idx must be less than {cpi_len - 1}, got {noise_ev_idx}."

)

if noise_ev_idx >= (cpi_len - 1):

raise ValueError(

f"noise_ev_idx must be less than cpi_len - 1: {cpi_len - 1}, got {noise_ev_idx}."

)

Tyler-g-hudson · 2026-03-17T00:57:38Z

python/packages/isce3/signal/rfi_process_evd.py

+
+    # Verify Mask shape
+    if mask_valid.shape != raw_data.shape:
+        raise ValueError(f"mask shape {mask_valid.shape} != data shape {raw_data.shape}")


Suggested change

raise ValueError(f"mask shape {mask_valid.shape} != data shape {raw_data.shape}")

raise ValueError(f"Valid CPI mask shape {mask_valid.shape} != data shape {raw_data.shape}")

Updates for Dithered PRF mode RFI detection and mitigation

b2fb817

hb6688 requested review from Tyler-g-hudson, bhawkins, hfattahi and rad-eng-59 March 15, 2026 20:30

restore previous updates from development branch.

e7f69a5

hfattahi added this to the R05.01.3 milestone Mar 16, 2026

rad-eng-59 reviewed Mar 17, 2026

View reviewed changes

Tyler-g-hudson reviewed Mar 17, 2026

View reviewed changes

-        usable_rank = int(np.count_nonzero(eig_val_sort_db > 0))
+        # do we need to peak normalize this?
+        eig_val_norm = np.copy(eig_val_sort)
+        if eig_val_sort[0] > eig_val_sort[-1]:
+            eig_val_norm /=  eig_val_sort[0]
+        # set to max possible dynamic range (16 bits  ~ 96 dB ~ 1e-9) or better to ENOB + floating-point precision ( < 8 bit ~ 50 dB ~ 1e-5)
+        usable_rank = np.sum(~np.isclose(eig_val_norm, 0, atol=1e-9))

	if usable_rank < noise_ev_idx + 1:
	if usable_rank < min_cpi_rank:

		off_diag_overlap_ratio: float = 0.1,
		diag_valid_ratio: float = 0.05,

	raise ValueError(f"mask shape {mask_valid_cpi.shape} != data shape {raw_data.shape}")
	raise ValueError(f"Valid CPI mask shape {mask_valid_cpi.shape} != data shape {raw_data.shape}")

Conversation

hb6688 commented Mar 15, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Tyler-g-hudson left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants