I am getting an intermittent error where shelve does not load all the keys stored inside it when I iterate through as shown in the code below. kc was a debug variable to see what was going on--I have 93 keys saved, and only 89 load.
I have a separate code written where I write sets of variables that are defined based on an external input file--regardless of the input file that is given, the same variables are stored, and each respective variable is of the same type as it was for any different external input file.
The code I'm using to bring the shelved variables into scope is below:
import shelve as sv
# Use this to open working file with 93 keys
sv_in = "ProperlyWorking.in" # drop the .db extension
# Use this to open the file that only loads 89 keys
sv_in = "MissedValuesonLoad.in"
wrdict = sv.open(sv_in,writeback=True)
kc = 0
for d in wrdict:
globals()[d] = wrdict[d]
kc += 1
wrdict.close()
Edit: Please see the files in this link that contain the .db files to load with shelve. With further troubleshooting, I was unable to reproduce the error with the code given here and also noticed that trying to load the .db file AGAIN in the exact same script resulted in all 93 keys loading. It was only the first load that failed, and I really have no idea what's going on. I just added the load function twice as a workaround for now.
With other external input files, all 93 keys load with no problems and I have been using this code for almost a year with zero issues across several of those external input files. There have been no changes to the file structure or the shelve code...I just have different numbers on each variable and over time have added more keys to be saved (actually, I recently reduced the number to reduce clutter, but that was 2 months ago and had no issues until this one specific file).
I know the keys actually exist in the shelve .db file because I listed them out at the time of saving them, and once I found out which 4 were missing, I just manually loaded them in (as a stop-gap) with:
globals()["var"] = wrdict["var"].
Has anyone seen this issue where iterating through the shelve file does not actually load everything contained in the file? I'm a little new to using shelve and I haven't found anything in the documentation indicating any way to see silent errors (if that is what's happening). I've tried wiping the python cache and deleting any leftover .db files just in case, but that had no effect. Thank you!
Machine is a 2019 Intel Macbook Pro 15", MacOS 12.6.1, Python 3.10.2