Attaching two parts of a pandas data frame in a Tensor of type tensorflow.python.data.ops.dataset_ops.PrefetchDataset

Question

I want to run a Neural Network on my data and for preprocessing I want to create tensor including two element: The first element including inputs in the form of a array of shape (100,4) and the other one of shape of (100,1): I have created these two Tensors by using below codes:

```
dataset_train_X = tf.constant(Train_data.drop('target',axis = 1))
dataset_train_target = tf.constant(Train_data.target)
```

dataset_train_X is:

<tf.Tensor: shape=(43152, 4), dtype=float64, numpy=
array([[2.01, 8.23, 8.19, 4.77],
       [1.01, 6.57, 6.49, 3.92],
       [1.1 , 6.59, 6.54, 4.1 ],
       ...,
       [0.33, 4.49, 4.46, 2.7 ],
       [0.9 , 6.13, 6.03, 3.82],
       [1.14, 6.82, 6.79, 4.11]])>

and dataset_train_target is:
<tf.Tensor: shape=(43152,), dtype=float64, numpy=array([16231.,  4540.,  5729., ...,  1014.,  2871.,  6320.])>


now I want a Tensor of type like the mnist test and train form. For instance:

(<tf.Tensor: shape=(43152, 4), dtype=uint8, numpy=
 array(array([[2.01, 8.23, 8.19, 4.77],
       [1.01, 6.57, 6.49, 3.92],
       [1.1 , 6.59, 6.54, 4.1 ],
       ...,
       [0.33, 4.49, 4.46, 2.7 ],
       [0.9 , 6.13, 6.03, 3.82],
       [1.14, 6.82, 6.79, 4.11]])>, dtype=uint8)>, <tf.Tensor: shape=(43152), dtype=int64, numpy=array([16231.,  4540.,  5729., ...,  1014.,  2871.,  6320.])>

Can anyone help, please?

I have tried concat but there are some errors each time I alter my method....

Answer 1

MNIST dataset consists of two separate arrays, as you can see here:

from keras.datasets import mnist
train_dataset, test_dataset = mnist.load_data()

print(type(train_dataset)) # tuple

However, NumPy doesn't allow creating ndarrays from the union of ndarrays with different shapes, but it does when talking about python lists.

This is what you can do:

arr = [train_dataset[0], train_dataset[1]] # first possibility
arr = np.array([list(train_dataset[0]), list(train_dataset[1])]) # second possibility

You cannot convert these arrays to tensors, because only rectangular sequences are converted to tensors.

Answer 2

Finally, I've tried the below and it solved the issue:

train_tensors = tf.constant(Train_data.drop('target',axis = 1)),
tf.constant(Train_data.target) train_dataset =
tf.data.Dataset.from_tensor_slices(tensors).batch(1)

test_tensors = tf.constant(Test_data.drop('target',axis = 1)),
tf.constant(Test_data.target) test_dataset =
tf.data.Dataset.from_tensor_slices(test_tensors).batch(1)