Improving one class in a pre-trained tensorflow object detection api model

Question

I was trying to improve the phone mobile class in the pre-trained tf object detection API model (efficientdet_d0_coco17_tpu-32).

I have collected training data and created tfrecords where the label_map.pbtext is as the following:

item {
 id: 77
 name: 'mobile'
}

and updated the pipeline.config as the following:

  fine_tune_checkpoint: "checkpoint/ckpt-0" (of the pretrained model)
  fine_tune_checkpoint_version: V2
  fine_tune_checkpoint_type: "detection"

after training when I am trying to inference the new model, I am always getting 1 as detection_classes.

any idea what I am doing wrong? or continues learning is not possible here?