composite score for denoising 1D signal

Question

I am suing different methods such as wavelet(different parameters) , FFT to denoise 1 D single . I using skimage.metrics to evaluate denoising as shown in snippet below. (here signal is original noisy signal)

import skimage.metrics as M
def get_denoise_metrics(signal, clean_signal):
  data_range = (min(signal), max(signal))
  d = {
         'normalized_root_mse' : [np.round(M.normalized_root_mse(signal, clean_signal), 4)],
         'peak_signal_noise_ratio' :  [round(M.peak_signal_noise_ratio(signal, clean_signal, data_range =data_range[1]), 4)],
         'structural_similarity' : [np.round(M.structural_similarity(signal, clean_signal, data_range =data_range[1]), 4)],
  }
  return d

Since I have 3 metrices for each denoising method (total no. of methods are greater than 10) I am using , How can I create a composite score so based on that I select based method .

Answer 1

All depends on what you want to achieve, there is no best answer, because for someone RMSE will be most important, for others peak SNR

That being said, I give you three propositions:

Simple ranking

Score methods and create ranking for each metric separately.

composite_score = place_in_ranking_metric1 + place_in_ranking_metric2 + ...

Arbitrary weights

Use your knowledge and intuition to find arbitrary weights.

composite_score = w1 * score_metric1 + w2 * score_metric2 + ...

Normalized, "Floating-point" ranking

If you think it through this should work better than simple ranking. For each method normalize score based on best result.

score_metric1 = score_metric1 / best_score_metric1
score_metric2 = score_metric2 / best_score_metric1
...
composite_score = score_metric1 + score_metric2 + ...

Sometimes, you may want to transform some scores using either log, exp, sqrt or pow. Or some other way. It is, again, very arbitrary.