Unable to add data to a numpy file

Question

while (True):
    if not paused:
        screen = grab_screen(region=(0, 40, 800, 640))
        screen = cv2.cvtColor(screen,
                              cv2.COLOR_BGR2GRAY)  # Removing color can reduce the complexity to train the neural network model
        screen = cv2.resize(screen, (160, 120))  # resizing so that it would be easy to train in a CNN model
        keys = key_check()
        output = keys_to_output(keys)
        training_data.append([screen, output])

        if len(training_data) % 100 == 0:  # to save training data after every 1000 records
            print(len(training_data))
            np.save(file_name, training_data)

Error:

  File "create_training_data.py", line 63, in main
    np.save(file_name, training_data)  # saving 1000 records of training data to a file
  File "<__array_function__ internals>", line 200, in save
  File "C:\Users\Administrator\PycharmProjects\GameAutomation\venv\lib\site-packages\numpy\lib\npyio.py", line 521, in save
    arr = np.asanyarray(arr)
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 2 dimensions. The detected shape was (100, 2) + inhomogeneous part.

I am trying to grab screen data along with keys recorded and adding to a list and after 100 such data records I am trying to save it to a file. But I am getting some kind of error.

Answer 1

You are trying to save a list of numpy array [np.array([1,2,4]), np.array([1,2,3])] which have different shapes as well (?), So concatenating the data will not work.

I recommend you to simply save both values to different files, this should make the shape homogeneous and saving them in array-like structures will work.

PS: You can re-create a small problem with actual data, so we can give you more concrete advice.

Answer 2

• Numpy arrays require all elements to have the same shape. error you're encountering is because of training_data, it contains elements with different shapes, convert training_data into a numpy array before saving it to a file.

if len(training_data) % 100 == 0:
    print(len(training_data))
    # Convert training_data to a numpy array
    training_data_array = np.array(training_data)
    np.save(file_name, training_data_array)