How to get the best parameters from Trails object in .pkl file

Question

I have one issue when I tried to get the best parameters from Trails object in python file combined_trials01.pkl.

import pickle
import os
from hyperopt import Trials
import pickle
import os
from hyperopt import space_eval

directory_path = r'\data\python_related\result'

output_file_path = os.path.join(directory_path, 'combined_trials01.pkl')


combined_trials = Trials()  


for i in range(21):  
    file_name = f'comm_trials_binary_test26_{i}.pkl'
    file_path = os.path.join(directory_path, file_name)

   
    if os.path.exists(file_path):
        with open(file_path, 'rb') as f:
            data = pickle.load(f)  
            
            for trial in data.trials:
                combined_trials.insert_trial_doc(trial)  
            combined_trials.refresh()  


with open(output_file_path, 'wb') as f:
    pickle.dump(combined_trials, f)

print(f"All trials combined and saved to: \n{output_file_path}")


if os.path.exists(output_file_path):
    with open(output_file_path, 'rb') as f:
       
        combined_trials = pickle.load(f)
       
        for trial in combined_trials:
            print(trial)
          
else:
    print(f"File {output_file_path} not found.")
print("--------------------------------------------------------------")




output_file_path = r'\data\python_related\result\combined_trials01.pkl'

if os.path.exists(output_file_path):
    with open(output_file_path, 'rb') as f:
        
        combined_trials = pickle.load(f)

      

        for trials in combined_trials['trials']:
           
            best_trial = min(trials, key=lambda x: x['result']['loss'])
           
            best_params = space_eval(trials.space, best_trial['misc']['vals'])
            
          
          
            print("The best is :\n",best_trial)

else:
    print(f"File {output_file_path} not found.")

I don't know how to get the items (information of each line in Trials object )in each Trials object in pkl file and get the best parameters.

And I got the error information:

Traceback (most recent call last):
  File "testcombine01.py", line 63, in <module>
    for trials in combined_trials['trials']:
                  ~~~~~~~~~~~~~~~^^^^^^^^^^
  File "\Python\Python311\site-packages\hyperopt\base.py", line 351, in __getitem__
    raise NotImplementedError("")
NotImplementedError

Could you please provide any hint or suggestions?

Answer 1

Your error is due to treating the Trials object as a dictionary, however it does not support direct item access using []. You should access it via the trials attribute of the object instead.