I used the following roberta sentiment model to and fine tuned it to my text data.
roberta_model = TFAutoModelForSequenceClassification.from_pretrained('cardiffnlp/twitter-roberta-base-sentiment')
tokenizer = AutoTokenizer.from_pretrained('cardiffnlp/twitter-roberta-base-sentiment')
tokenized_data = tokenizer(list(df['text']), padding=True, return_tensors='np')
tokenized_data = dict(tokenized_data)
def get_model(roberta_model):
input_ids = Input(shape=(None, ), dtype='int32', name='input_ids')
attention_mask = Input(shape=(None, ), dtype='int32', name='attention_mask')
roberta_outputs = roberta_model(input_ids, attention_mask=attention_mask)[0]
outputs = Dense(32, activation='relu')(roberta_outputs)
outputs = Dense(3, activation='softmax')(outputs)
model = tf.keras.Model(inputs=[input_ids, attention_mask], outputs = outputs)
return model
model = get_model(roberta_model)
model.compile(optimizer=Adam(3e-5), metrics = ['accuracy'], loss='sparse_categorical_crossentropy')
history = model.fit(tokenized_data, y, validation_split=0.2, epochs = 5)
When I save the model using:
model.save('/content/mymodel')
and reload the model using:
model = keras.models.load_model('/content/mymodel')
It shows this error:
ValueError: The two structures don't have the same nested structure.
I have tried saving it in h5 and keras format. The above mentioned issue is the current issue. Sometimes before it gave me an error of str object is not callable.
How can I fix this issue?