Visualize trajectories with t-sne

39 views Asked by At

I work on a project where I want to compare the optimization algorithms (SGD, Adam, SGD with momentum). I want to start with 5 w0 start(initialize) point and do the training. I chose the mnist dataset for training and avoided building a complex model to see the comparasion. At the end I should have 5*3 trajectories. I want to visualize these 15 trajectories (2d) via t-sne and color the optimizer algorithms. I tried many ways but could not write the python code for it. Any help would apreciated. My model is:

`def create_mnist_model(optimizer):
    model = keras.Sequential([
        keras.layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),
        keras.layers.MaxPooling2D((2, 2)),
        keras.layers.Flatten(),
        keras.layers.Dense(64, activation='relu'),
        keras.layers.Dense(10)
    ])
    model.compile(optimizer=optimizer,
                  loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
                  metrics=['accuracy'])
    return model`

I trained each loop with 10 epoch and with batch_size=32. I saved the losses and append them as trajectories as array. For different initilize points I used seeds (I am not sure about it)

0

There are 0 answers