ValueError: Cannot convert a Tensor of dtype resource to a NumPy array

Question

I'm trying to isolate some user specific parameters by having matrix of parameters where each array would learn parameters specific to that user.

I want to index the matrix using the user id, and concatenate the parameters to the other features.

Lastly, have some fully-connected layers to get desirable outcome.

However, I keep getting this error on the last line of the code.

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-93de3591ccf0> in <module>
     20 # combined = tf.keras.layers.Concatenate(axis=-1)([le_param, le])
     21 
---> 22 net = tf.keras.layers.Dense(128)(combined)

~/anaconda3/envs/tam-env/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py in __call__(self, inputs, *args, **kwargs)
    793     # framework.
    794     if build_graph and base_layer_utils.needs_keras_history(inputs):
--> 795       base_layer_utils.create_keras_history(inputs)
    796 
    797     # Clear eager losses on top level model call.

~/anaconda3/envs/tam-env/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer_utils.py in create_keras_history(tensors)
    182     keras_tensors: The Tensors found that came from a Keras Layer.
    183   """
--> 184   _, created_layers = _create_keras_history_helper(tensors, set(), [])
    185   return created_layers
    186 

~/anaconda3/envs/tam-env/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer_utils.py in _create_keras_history_helper(tensors, processed_ops, created_layers)
    229               constants[i] = backend.function([], op_input)([])
    230       processed_ops, created_layers = _create_keras_history_helper(
--> 231           layer_inputs, processed_ops, created_layers)
    232       name = op.name
    233       node_def = op.node_def.SerializeToString()

~/anaconda3/envs/tam-env/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer_utils.py in _create_keras_history_helper(tensors, processed_ops, created_layers)
    229               constants[i] = backend.function([], op_input)([])
    230       processed_ops, created_layers = _create_keras_history_helper(
--> 231           layer_inputs, processed_ops, created_layers)
    232       name = op.name
    233       node_def = op.node_def.SerializeToString()

~/anaconda3/envs/tam-env/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer_utils.py in _create_keras_history_helper(tensors, processed_ops, created_layers)
    227           else:
    228             with ops.init_scope():
--> 229               constants[i] = backend.function([], op_input)([])
    230       processed_ops, created_layers = _create_keras_history_helper(
    231           layer_inputs, processed_ops, created_layers)

~/anaconda3/envs/tam-env/lib/python3.6/site-packages/tensorflow_core/python/keras/backend.py in __call__(self, inputs)
   3746     return nest.pack_sequence_as(
   3747         self._outputs_structure,
-> 3748         [x._numpy() for x in outputs],  # pylint: disable=protected-access
   3749         expand_composites=True)
   3750 

~/anaconda3/envs/tam-env/lib/python3.6/site-packages/tensorflow_core/python/keras/backend.py in <listcomp>(.0)
   3746     return nest.pack_sequence_as(
   3747         self._outputs_structure,
-> 3748         [x._numpy() for x in outputs],  # pylint: disable=protected-access
   3749         expand_composites=True)
   3750 

ValueError: Cannot convert a Tensor of dtype resource to a NumPy array.

Code to reproduce the error:

import tensorflow as tf

num_uids = 50
input_uid = tf.keras.layers.Input(shape=(1,), dtype=tf.int32)
params = tf.Variable(tf.random.normal((num_uids, 9)), trainable=True)

param = tf.gather_nd(params, input_uid)

input_shared_features = tf.keras.layers.Input(shape=(128,), dtype=tf.float32)
combined = tf.concat([param, input_shared_features], axis=-1)

net = tf.keras.layers.Dense(128)(combined)

There are few things I've tried:

1. I tried to use tf.keras.layers.Lambda to encapsulate tf.gather_nd and tf.concat.
2. I tried replacing tf.concat with tf.keras.layers.Concatenate.

Oddly enough if I specify the number of items and replace Input with tf.Variable, the code would work as expected:

import tensorflow as tf

num_uids = 50
input_uid = tf.Variable(tf.ones((32, 1), dtype=tf.int32))
params = tf.Variable(tf.random.normal((num_uids, 9)), trainable=True)

param = tf.gather_nd(params, input_uid)

input_shared_features = tf.Variable(tf.ones((32, 128), dtype=tf.float32))
combined = tf.concat([param, input_shared_features], axis=-1)

net = tf.keras.layers.Dense(128)(combined)

I'm using Tensorflow 2.1 with Python 3.6.10

Answer 1

I faced a similar issue when I was trying to use a TensorFlow table lookup (tf.lookup.StaticHashTable) in TensorFlow 2.x. I ended up solving it by keeping it inside a Custom Keras Layer. The same solution seems to have worked for this problem as well—at least until to the version that's mentioned in the question. (I tried using TensorFlow 2.0, 2.1, and 2.2 and it worked in all of these versions.)

import tensorflow as tf

num_uids = 50
input_uid = tf.keras.Input(shape=(1,), dtype=tf.int32)
input_shared_features = tf.keras.layers.Input(shape=(128,), dtype=tf.float32)

class CustomLayer(tf.keras.layers.Layer):
    def __init__(self,num_uids):
        super(CustomLayer, self).__init__(trainable=True,dtype=tf.int64)
        self.num_uids = num_uids

    def build(self,input_shape):
        self.params = tf.Variable(tf.random.normal((num_uids, 9)), trainable=True)
        self.built=True

    def call(self, input_uid,input_shared_features):
        param = tf.gather_nd(self.params, input_uid)
        combined = tf.concat([param, input_shared_features], axis=-1)
        return combined

    def get_config(self):
        config = super(CustomLayer, self).get_config()
        config.update({'num_uids': self.num_uids})
        return config

combined = CustomLayer(num_uids)(input_uid,input_shared_features)
net = tf.keras.layers.Dense(128)(combined)
model = tf.keras.Model(inputs={'input_uid':input_uid,'input_shared_features':input_shared_features},outputs=net)
model.summary()

Here's what model summary looked like:

Model: "model"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 1)]          0                                            
__________________________________________________________________________________________________
input_2 (InputLayer)            [(None, 128)]        0                                            
__________________________________________________________________________________________________
custom_layer (CustomLayer)      (None, 137)          450         input_1[0][0]                    
__________________________________________________________________________________________________
dense (Dense)                   (None, 128)          17664       custom_layer[0][0]               
==================================================================================================
Total params: 18,114
Trainable params: 18,114
Non-trainable params: 0

For more info you can refer to the tf.keras.layers.Layer documentation.

In case you want to refer to the table lookup problem and solution, here are the links:

• git issue
• Sample working code for TensorFlow lookup

Answer 2

While Jithin Jees's answer is very clear, Shown below is a slightly different workaround using Concatenate Operation:

import tensorflow as tf

num_uids = 50
#input_uid = tf.keras.layers.Input(shape=(1,), dtype=tf.int32, batch_size = 32)
#input_uid = tf.keras.layers.Input(shape=(1,), dtype=tf.int32)
#params = tf.Variable(tf.random.normal((num_uids, 9)), trainable=True)

#param = tf.gather_nd(params, input_uid)

indices = tf.keras.layers.Input(name='indices', shape=(), dtype='int32')
params = tf.Variable(params)

class GatherLayer(tf.keras.layers.Layer):
    def call(self, indices, params):
        return tf.gather(params, indices)

output = GatherLayer()(indices, params)

#input_shared_features = tf.keras.layers.Input(shape=(128,), dtype=tf.float32, batch_size = 32)
input_shared_features = tf.keras.layers.Input(shape=(128,), dtype=tf.float32)
combined = tf.concat([output, input_shared_features], axis=-1)

net = tf.keras.layers.Dense(128)(combined)

For more details, please refer this Github Issue.