UFuncTypeError: ufunc ‘clip’ did not contain a loop with signature matching types (dtype(‘<U32’), dtype(‘<U32’), dtype(‘<U32’)) -> dtype(‘<U32’)

Question

I am using the Deep Pavlov framework to work with Bert Classifier simply because the language I need to predict staff is Russian. Basically, I am trying to solve a multi-class classification problem. According to the Deep Pavlov, we can easily change some configs on config file. I took this config file https://github.com/deepmipt/DeepPavlov/blob/master/deeppavlov/configs/classifiers/rusentiment_convers_bert.json and trained it, and it took me around 13 hours to finish and it turned out to be that my model is overfitting.

I made some changes, particularly these:

"weight_decay_rate": 0.001,
"learning_rate_drop_patience": 1,
"learning_rate_drop_div": 2.0,
"load_before_drop": True, 
"min_learning_rate": 1e-03,
"attention_probs_keep_prob": 0.5,
"hidden_keep_prob": 0.5,

also, I increased the batch size, it was 16 before now:

"batch_size": 32

and added some metrics:

"log_loss",
"matthews_correlation",

Also changed validation_patience to 1 and added tensorboard func

"validation_patience": 1,
"tensorboard_log_dir": "logs/",

and that is it. these are all the changes I made to my model, and when i tried to train my model, it is giving me following error:

UFuncTypeError                            Traceback (most recent call last)
        /usr/local/lib/python3.7/dist-packages/numpy/core/fromnumeric.py in _wrapfunc(obj, method, *args, **kwds)
             60     try:
        ---> 61         return bound(*args, **kwds)
             62     except TypeError:

15 frames
UFuncTypeError: ufunc 'clip' did not contain a loop with signature matching types (dtype('<U32'), dtype('<U32'), dtype('<U32')) -> dtype('<U32')

During handling of the above exception, another exception occurred:

UFuncTypeError                            Traceback (most recent call last)
<__array_function__ internals> in clip(*args, **kwargs)

/usr/local/lib/python3.7/dist-packages/numpy/core/_methods.py in _clip_dep_invoke_with_casting(ufunc, out, casting, *args, **kwargs)
     83     # try to deal with broken casting rules
     84     try:
---> 85         return ufunc(*args, out=out, **kwargs)
     86     except _exceptions._UFuncOutputCastingError as e:
     87         # Numpy 1.17.0, 2019-02-24

UFuncTypeError: ufunc 'clip' did not contain a loop with signature matching types (dtype('<U32'), dtype('<U32'), dtype('<U32')) -> dtype('<U32')

At first, I thought it has something to do with a dataset, however, I did not change my dataset and it has run the first time I trained this model.

Answer 1

log_loss in DeepPavlov is just a wrapper over sklearn.metrics.log_loss: https://github.com/deepmipt/DeepPavlov/blob/master/deeppavlov/metrics/log_loss.py#L37

By default DeepPavlov uses chainer's out as y_pred in metrics computation and chainer's in_y as y_true.

To use log loss you can specify y_true as y or y_ids. And specify y_pred as y_pred_probas in log loss computation. This change will compute log loss for your case:

{
  "name": "log_loss",
  "inputs": [
    "y",
    "y_pred_probas"
  ]
}