Laboratory

Hyperparameter

import numpy as np
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, BatchNormalization, Flatten
from tensorflow.keras.callbacks import EarlyStopping

from tensorflow.keras.datasets import mnist

(X_train, y_train), (X_valid, y_valid) = mnist.load_data()

2025-02-12 03:24:49.251451: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2025-02-12 03:24:49.251470: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.

(28, 28)

def create_model(optimizer='adam', dropout_rate=0):
    model = Sequential([
        Flatten(input_shape=X_train[0].shape),
        BatchNormalization(),
        Dropout(dropout_rate),
        Dense(64, activation='relu'),
        Dropout(dropout_rate),
        Dense(16, activation='relu'),
        Dropout(dropout_rate),
        Dense(10, activation='softmax')
    ])

    model.compile(
        optimizer=optimizer,
        loss='sparse_categorical_crossentropy',
        metrics=['accuracy']
    )

    return model

from keras.wrappers.scikit_learn import KerasClassifier

ks = KerasClassifier(build_fn=create_model, batch_size=100)

early_stopping_callback = EarlyStopping(
    patience=10,
    min_delta=0.01,
    restore_best_weights=True
)

RandomSearchCV

from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import uniform, randint

param_dist = {
    "dropout_rate": uniform(0.1, 0.5)
}

cv = RandomizedSearchCV(estimator=ks, param_distributions={}, n_iter=5)

cv.fit(
    X_train, y_train,
    validation_data=(X_valid, y_valid),
    epochs=20,
    callbacks=[early_stopping_callback]
)

/home/dopas/.micromamba/envs/python39/lib/python3.9/site-packages/sklearn/model_selection/_search.py:317: UserWarning: The total space of parameters 1 is smaller than n_iter=5. Running 1 iterations. For exhaustive searches, use GridSearchCV.
  warnings.warn(

Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 2.7280 - accuracy: 0.2438 - val_loss: 1.6369 - val_accuracy: 0.4102
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 1.3517 - accuracy: 0.5373 - val_loss: 1.1350 - val_accuracy: 0.6061
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 1.0056 - accuracy: 0.6469 - val_loss: 0.9184 - val_accuracy: 0.6835
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8010 - accuracy: 0.7240 - val_loss: 0.7211 - val_accuracy: 0.7739
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6206 - accuracy: 0.8016 - val_loss: 0.5818 - val_accuracy: 0.8283
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 0.5102 - accuracy: 0.8511 - val_loss: 0.5314 - val_accuracy: 0.8675
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 0.4050 - accuracy: 0.8874 - val_loss: 0.4176 - val_accuracy: 0.8978
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3314 - accuracy: 0.9087 - val_loss: 0.3588 - val_accuracy: 0.9171
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2747 - accuracy: 0.9245 - val_loss: 0.3204 - val_accuracy: 0.9215
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2379 - accuracy: 0.9341 - val_loss: 0.2740 - val_accuracy: 0.9303
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1902 - accuracy: 0.9477 - val_loss: 0.2411 - val_accuracy: 0.9419
Epoch 12/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1603 - accuracy: 0.9541 - val_loss: 0.2611 - val_accuracy: 0.9412
Epoch 13/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1487 - accuracy: 0.9581 - val_loss: 0.2246 - val_accuracy: 0.9522
Epoch 14/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1365 - accuracy: 0.9604 - val_loss: 0.2467 - val_accuracy: 0.9452
Epoch 15/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1255 - accuracy: 0.9642 - val_loss: 0.1986 - val_accuracy: 0.9522
Epoch 16/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1140 - accuracy: 0.9671 - val_loss: 0.2018 - val_accuracy: 0.9537
Epoch 17/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1012 - accuracy: 0.9696 - val_loss: 0.2058 - val_accuracy: 0.9506
Epoch 18/20
480/480 [==============================] - 1s 1ms/step - loss: 0.0967 - accuracy: 0.9712 - val_loss: 0.2086 - val_accuracy: 0.9513
Epoch 19/20
480/480 [==============================] - 1s 1ms/step - loss: 0.0919 - accuracy: 0.9731 - val_loss: 0.2008 - val_accuracy: 0.9533
Epoch 20/20
480/480 [==============================] - 1s 1ms/step - loss: 0.0899 - accuracy: 0.9732 - val_loss: 0.1930 - val_accuracy: 0.9562
120/120 [==============================] - 0s 1ms/step - loss: 0.2006 - accuracy: 0.9578
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 2.6636 - accuracy: 0.1631 - val_loss: 2.1203 - val_accuracy: 0.1974
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 2.0234 - accuracy: 0.2256 - val_loss: 1.9375 - val_accuracy: 0.2951
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 1.6755 - accuracy: 0.3522 - val_loss: 1.5443 - val_accuracy: 0.3811
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 1.4604 - accuracy: 0.3992 - val_loss: 1.3985 - val_accuracy: 0.4138
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 1.3017 - accuracy: 0.4487 - val_loss: 1.2579 - val_accuracy: 0.4899
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 1.1719 - accuracy: 0.4950 - val_loss: 1.1331 - val_accuracy: 0.5401
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 1.0588 - accuracy: 0.5546 - val_loss: 1.0148 - val_accuracy: 0.5958
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8631 - accuracy: 0.6696 - val_loss: 0.8207 - val_accuracy: 0.7014
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7251 - accuracy: 0.7218 - val_loss: 0.7308 - val_accuracy: 0.7295
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6590 - accuracy: 0.7394 - val_loss: 0.6808 - val_accuracy: 0.7418
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 0.5553 - accuracy: 0.7949 - val_loss: 0.5256 - val_accuracy: 0.8322
Epoch 12/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3991 - accuracy: 0.8917 - val_loss: 0.3859 - val_accuracy: 0.9203
Epoch 13/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2938 - accuracy: 0.9374 - val_loss: 0.3612 - val_accuracy: 0.9297
Epoch 14/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2388 - accuracy: 0.9467 - val_loss: 0.3013 - val_accuracy: 0.9418
Epoch 15/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2018 - accuracy: 0.9541 - val_loss: 0.3083 - val_accuracy: 0.9374
Epoch 16/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1772 - accuracy: 0.9580 - val_loss: 0.3098 - val_accuracy: 0.9359
Epoch 17/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1603 - accuracy: 0.9599 - val_loss: 0.2597 - val_accuracy: 0.9481
Epoch 18/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1385 - accuracy: 0.9642 - val_loss: 0.2590 - val_accuracy: 0.9441
Epoch 19/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1267 - accuracy: 0.9670 - val_loss: 0.2738 - val_accuracy: 0.9446
Epoch 20/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1145 - accuracy: 0.9695 - val_loss: 0.2703 - val_accuracy: 0.9501
120/120 [==============================] - 0s 885us/step - loss: 0.2475 - accuracy: 0.9473
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 2.8741 - accuracy: 0.1226 - val_loss: 2.2551 - val_accuracy: 0.1341
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 2.0621 - accuracy: 0.2234 - val_loss: 1.9488 - val_accuracy: 0.2546
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 1.7967 - accuracy: 0.2982 - val_loss: 1.6980 - val_accuracy: 0.3606
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 1.5276 - accuracy: 0.3872 - val_loss: 1.5046 - val_accuracy: 0.4349
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 1.4157 - accuracy: 0.4179 - val_loss: 1.4478 - val_accuracy: 0.4147
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 1.2332 - accuracy: 0.5029 - val_loss: 1.1411 - val_accuracy: 0.5742
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 1.0177 - accuracy: 0.5991 - val_loss: 1.0091 - val_accuracy: 0.6278
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8459 - accuracy: 0.6691 - val_loss: 0.8172 - val_accuracy: 0.7004
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7137 - accuracy: 0.7399 - val_loss: 0.6214 - val_accuracy: 0.8189
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 0.5140 - accuracy: 0.8461 - val_loss: 0.4881 - val_accuracy: 0.8626
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3713 - accuracy: 0.9033 - val_loss: 0.3577 - val_accuracy: 0.9250
Epoch 12/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2588 - accuracy: 0.9398 - val_loss: 0.2751 - val_accuracy: 0.9394
Epoch 13/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2056 - accuracy: 0.9513 - val_loss: 0.2614 - val_accuracy: 0.9389
Epoch 14/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1730 - accuracy: 0.9575 - val_loss: 0.2336 - val_accuracy: 0.9471
Epoch 15/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1523 - accuracy: 0.9609 - val_loss: 0.2408 - val_accuracy: 0.9413
Epoch 16/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1360 - accuracy: 0.9650 - val_loss: 0.1988 - val_accuracy: 0.9564
Epoch 17/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1217 - accuracy: 0.9687 - val_loss: 0.2068 - val_accuracy: 0.9538
Epoch 18/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1091 - accuracy: 0.9710 - val_loss: 0.1876 - val_accuracy: 0.9580
Epoch 19/20
480/480 [==============================] - 1s 1ms/step - loss: 0.0963 - accuracy: 0.9746 - val_loss: 0.1801 - val_accuracy: 0.9611
Epoch 20/20
480/480 [==============================] - 1s 1ms/step - loss: 0.0895 - accuracy: 0.9762 - val_loss: 0.1855 - val_accuracy: 0.9579
120/120 [==============================] - 0s 1ms/step - loss: 0.1992 - accuracy: 0.9578
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 2.4221 - accuracy: 0.2714 - val_loss: 1.5715 - val_accuracy: 0.4307
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 1.3463 - accuracy: 0.5196 - val_loss: 1.1462 - val_accuracy: 0.5895
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 0.9246 - accuracy: 0.6859 - val_loss: 0.8190 - val_accuracy: 0.7333
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6937 - accuracy: 0.7886 - val_loss: 0.7195 - val_accuracy: 0.8066
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 0.5075 - accuracy: 0.8615 - val_loss: 0.4847 - val_accuracy: 0.8814
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3660 - accuracy: 0.9045 - val_loss: 0.3919 - val_accuracy: 0.9167
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3010 - accuracy: 0.9219 - val_loss: 0.3597 - val_accuracy: 0.9217
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2615 - accuracy: 0.9334 - val_loss: 0.3270 - val_accuracy: 0.9261
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2147 - accuracy: 0.9445 - val_loss: 0.2859 - val_accuracy: 0.9412
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1791 - accuracy: 0.9539 - val_loss: 0.2682 - val_accuracy: 0.9460
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1660 - accuracy: 0.9572 - val_loss: 0.2573 - val_accuracy: 0.9483
Epoch 12/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1492 - accuracy: 0.9612 - val_loss: 0.2369 - val_accuracy: 0.9503
Epoch 13/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1305 - accuracy: 0.9657 - val_loss: 0.2324 - val_accuracy: 0.9520
Epoch 14/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1152 - accuracy: 0.9696 - val_loss: 0.2016 - val_accuracy: 0.9591
Epoch 15/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1097 - accuracy: 0.9709 - val_loss: 0.1995 - val_accuracy: 0.9576
Epoch 16/20
480/480 [==============================] - 1s 1ms/step - loss: 0.0906 - accuracy: 0.9756 - val_loss: 0.2008 - val_accuracy: 0.9591
Epoch 17/20
480/480 [==============================] - 1s 1ms/step - loss: 0.0850 - accuracy: 0.9769 - val_loss: 0.1974 - val_accuracy: 0.9591
Epoch 18/20
480/480 [==============================] - 1s 1ms/step - loss: 0.0782 - accuracy: 0.9786 - val_loss: 0.1929 - val_accuracy: 0.9593
Epoch 19/20
480/480 [==============================] - 1s 1ms/step - loss: 0.0754 - accuracy: 0.9793 - val_loss: 0.2116 - val_accuracy: 0.9555
Epoch 20/20
480/480 [==============================] - 1s 1ms/step - loss: 0.0703 - accuracy: 0.9804 - val_loss: 0.2025 - val_accuracy: 0.9598
120/120 [==============================] - 0s 902us/step - loss: 0.2083 - accuracy: 0.9559
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 3.0352 - accuracy: 0.2880 - val_loss: 1.6968 - val_accuracy: 0.3694
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 1.5141 - accuracy: 0.4482 - val_loss: 1.3746 - val_accuracy: 0.5330
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 1.2320 - accuracy: 0.5627 - val_loss: 1.1750 - val_accuracy: 0.6028
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 1.0138 - accuracy: 0.6489 - val_loss: 0.9347 - val_accuracy: 0.7061
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7729 - accuracy: 0.7579 - val_loss: 0.7612 - val_accuracy: 0.7846
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6059 - accuracy: 0.8353 - val_loss: 0.5687 - val_accuracy: 0.8845
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 0.4446 - accuracy: 0.8981 - val_loss: 0.4475 - val_accuracy: 0.9155
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3399 - accuracy: 0.9226 - val_loss: 0.3688 - val_accuracy: 0.9248
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2677 - accuracy: 0.9391 - val_loss: 0.3403 - val_accuracy: 0.9341
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2246 - accuracy: 0.9460 - val_loss: 0.3013 - val_accuracy: 0.9427
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1922 - accuracy: 0.9532 - val_loss: 0.2662 - val_accuracy: 0.9446
Epoch 12/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1797 - accuracy: 0.9555 - val_loss: 0.2358 - val_accuracy: 0.9486
Epoch 13/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1493 - accuracy: 0.9620 - val_loss: 0.2237 - val_accuracy: 0.9535
Epoch 14/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1379 - accuracy: 0.9645 - val_loss: 0.2344 - val_accuracy: 0.9527
Epoch 15/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1305 - accuracy: 0.9664 - val_loss: 0.2205 - val_accuracy: 0.9545
Epoch 16/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1161 - accuracy: 0.9696 - val_loss: 0.2239 - val_accuracy: 0.9531
Epoch 17/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1085 - accuracy: 0.9712 - val_loss: 0.2378 - val_accuracy: 0.9482
Epoch 18/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1029 - accuracy: 0.9731 - val_loss: 0.2276 - val_accuracy: 0.9502
Epoch 19/20
480/480 [==============================] - 1s 1ms/step - loss: 0.0926 - accuracy: 0.9742 - val_loss: 0.2067 - val_accuracy: 0.9601
Epoch 20/20
480/480 [==============================] - 1s 1ms/step - loss: 0.0878 - accuracy: 0.9763 - val_loss: 0.1983 - val_accuracy: 0.9588
120/120 [==============================] - 0s 927us/step - loss: 0.1977 - accuracy: 0.9586
Epoch 1/20
600/600 [==============================] - 1s 1ms/step - loss: 2.6521 - accuracy: 0.2839 - val_loss: 1.5831 - val_accuracy: 0.4175
Epoch 2/20
600/600 [==============================] - 1s 1ms/step - loss: 1.4159 - accuracy: 0.4525 - val_loss: 1.2985 - val_accuracy: 0.4833
Epoch 3/20
600/600 [==============================] - 1s 1ms/step - loss: 1.1339 - accuracy: 0.5563 - val_loss: 0.9877 - val_accuracy: 0.5960
Epoch 4/20
600/600 [==============================] - 1s 1ms/step - loss: 0.8578 - accuracy: 0.6771 - val_loss: 0.7066 - val_accuracy: 0.7852
Epoch 5/20
600/600 [==============================] - 1s 1ms/step - loss: 0.6046 - accuracy: 0.8125 - val_loss: 0.5431 - val_accuracy: 0.8546
Epoch 6/20
600/600 [==============================] - 1s 1ms/step - loss: 0.4766 - accuracy: 0.8651 - val_loss: 0.4385 - val_accuracy: 0.8884
Epoch 7/20
600/600 [==============================] - 1s 1ms/step - loss: 0.3541 - accuracy: 0.9035 - val_loss: 0.3210 - val_accuracy: 0.9157
Epoch 8/20
600/600 [==============================] - 1s 1ms/step - loss: 0.2649 - accuracy: 0.9268 - val_loss: 0.2931 - val_accuracy: 0.9216
Epoch 9/20
600/600 [==============================] - 1s 1ms/step - loss: 0.2128 - accuracy: 0.9428 - val_loss: 0.2349 - val_accuracy: 0.9422
Epoch 10/20
600/600 [==============================] - 1s 1ms/step - loss: 0.1798 - accuracy: 0.9506 - val_loss: 0.2284 - val_accuracy: 0.9430
Epoch 11/20
600/600 [==============================] - 1s 1ms/step - loss: 0.1568 - accuracy: 0.9575 - val_loss: 0.2208 - val_accuracy: 0.9480
Epoch 12/20
600/600 [==============================] - 1s 1ms/step - loss: 0.1358 - accuracy: 0.9634 - val_loss: 0.1947 - val_accuracy: 0.9536
Epoch 13/20
600/600 [==============================] - 1s 1ms/step - loss: 0.1272 - accuracy: 0.9653 - val_loss: 0.1936 - val_accuracy: 0.9548
Epoch 14/20
600/600 [==============================] - 1s 1ms/step - loss: 0.1138 - accuracy: 0.9687 - val_loss: 0.1868 - val_accuracy: 0.9581
Epoch 15/20
600/600 [==============================] - 1s 1ms/step - loss: 0.1055 - accuracy: 0.9703 - val_loss: 0.1952 - val_accuracy: 0.9547
Epoch 16/20
600/600 [==============================] - 1s 1ms/step - loss: 0.0961 - accuracy: 0.9729 - val_loss: 0.1992 - val_accuracy: 0.9556
Epoch 17/20
600/600 [==============================] - 1s 1ms/step - loss: 0.0897 - accuracy: 0.9744 - val_loss: 0.1943 - val_accuracy: 0.9576
Epoch 18/20
600/600 [==============================] - 1s 1ms/step - loss: 0.0819 - accuracy: 0.9768 - val_loss: 0.1912 - val_accuracy: 0.9576
Epoch 19/20
600/600 [==============================] - 1s 1ms/step - loss: 0.0784 - accuracy: 0.9774 - val_loss: 0.1962 - val_accuracy: 0.9600
Epoch 20/20
600/600 [==============================] - 1s 1ms/step - loss: 0.0768 - accuracy: 0.9777 - val_loss: 0.1862 - val_accuracy: 0.9612

RandomizedSearchCV(estimator=<keras.wrappers.scikit_learn.KerasClassifier object at 0x7931f4cd13d0>,
                   n_iter=5, param_distributions={})

RandomizedSearchCV(estimator=<keras.wrappers.scikit_learn.KerasClassifier object at 0x7931f4cd13d0>,
n_iter=5, param_distributions={})

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

print("Best parameters found: ", cv.best_params_)
best_model = cv.best_estimator_

Best parameters found:  {'optimizer': 'adam'}

GridSearchCV

from sklearn.model_selection import GridSearchCV

param_grid = {
    "optimizer": ['adam', 'rmsprop', 'sgd']
}

cv = GridSearchCV(estimator=ks, param_grid=param_grid)

cv.fit(
    X_train, y_train,
    validation_data=(X_valid, y_valid),
    epochs=20,
    callbacks=[early_stopping_callback]
)

/tmp/ipykernel_65582/1041889032.py:27: DeprecationWarning: KerasClassifier is deprecated, use Sci-Keras (https://github.com/adriangb/scikeras) instead.
  ks = KerasClassifier(build_fn=create_model, batch_size=100)
2025-02-12 03:24:51.522100: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2025-02-12 03:24:51.522119: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2025-02-12 03:24:51.522133: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (lt14): /proc/driver/nvidia/version does not exist
2025-02-12 03:24:51.522293: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 2.8914 - accuracy: 0.1791 - val_loss: 1.9815 - val_accuracy: 0.2577
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 1.8337 - accuracy: 0.2973 - val_loss: 1.6663 - val_accuracy: 0.3552
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 1.5131 - accuracy: 0.4230 - val_loss: 1.2843 - val_accuracy: 0.5396
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 1.1007 - accuracy: 0.6158 - val_loss: 0.8808 - val_accuracy: 0.6936
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8227 - accuracy: 0.7480 - val_loss: 0.6627 - val_accuracy: 0.8282
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6104 - accuracy: 0.8391 - val_loss: 0.5975 - val_accuracy: 0.8316
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 0.5080 - accuracy: 0.8662 - val_loss: 0.5195 - val_accuracy: 0.8544
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 0.4286 - accuracy: 0.8910 - val_loss: 0.4403 - val_accuracy: 0.8997
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3663 - accuracy: 0.9104 - val_loss: 0.4159 - val_accuracy: 0.9086
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3110 - accuracy: 0.9239 - val_loss: 0.3481 - val_accuracy: 0.9162
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2662 - accuracy: 0.9363 - val_loss: 0.3410 - val_accuracy: 0.9291
Epoch 12/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2359 - accuracy: 0.9430 - val_loss: 0.2921 - val_accuracy: 0.9363
Epoch 13/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1998 - accuracy: 0.9514 - val_loss: 0.2551 - val_accuracy: 0.9452
Epoch 14/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1824 - accuracy: 0.9545 - val_loss: 0.2425 - val_accuracy: 0.9460
Epoch 15/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1658 - accuracy: 0.9587 - val_loss: 0.2289 - val_accuracy: 0.9478
Epoch 16/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1498 - accuracy: 0.9626 - val_loss: 0.2018 - val_accuracy: 0.9574
Epoch 17/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1298 - accuracy: 0.9667 - val_loss: 0.2028 - val_accuracy: 0.9566
Epoch 18/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1217 - accuracy: 0.9693 - val_loss: 0.1803 - val_accuracy: 0.9578
Epoch 19/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1094 - accuracy: 0.9722 - val_loss: 0.1961 - val_accuracy: 0.9575
Epoch 20/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1056 - accuracy: 0.9720 - val_loss: 0.1803 - val_accuracy: 0.9590
120/120 [==============================] - 0s 858us/step - loss: 0.1925 - accuracy: 0.9563
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 3.5808 - accuracy: 0.3415 - val_loss: 1.4291 - val_accuracy: 0.5227
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 1.2466 - accuracy: 0.5896 - val_loss: 1.0411 - val_accuracy: 0.6566
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 0.9224 - accuracy: 0.7089 - val_loss: 0.8185 - val_accuracy: 0.7673
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7404 - accuracy: 0.7695 - val_loss: 0.7155 - val_accuracy: 0.7758
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6124 - accuracy: 0.8011 - val_loss: 0.6394 - val_accuracy: 0.8254
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 0.5217 - accuracy: 0.8217 - val_loss: 0.5883 - val_accuracy: 0.8165
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 0.4546 - accuracy: 0.8377 - val_loss: 0.5355 - val_accuracy: 0.8832
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3916 - accuracy: 0.8920 - val_loss: 0.5182 - val_accuracy: 0.8914
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3486 - accuracy: 0.9053 - val_loss: 0.4437 - val_accuracy: 0.9042
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2900 - accuracy: 0.9240 - val_loss: 0.3910 - val_accuracy: 0.9292
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2484 - accuracy: 0.9359 - val_loss: 0.3501 - val_accuracy: 0.9271
Epoch 12/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2138 - accuracy: 0.9446 - val_loss: 0.3692 - val_accuracy: 0.9358
Epoch 13/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1929 - accuracy: 0.9505 - val_loss: 0.3675 - val_accuracy: 0.9412
Epoch 14/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1675 - accuracy: 0.9561 - val_loss: 0.3416 - val_accuracy: 0.9443
Epoch 15/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1557 - accuracy: 0.9585 - val_loss: 0.3112 - val_accuracy: 0.9448
Epoch 16/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1487 - accuracy: 0.9605 - val_loss: 0.3048 - val_accuracy: 0.9364
Epoch 17/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1304 - accuracy: 0.9651 - val_loss: 0.2729 - val_accuracy: 0.9439
Epoch 18/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1143 - accuracy: 0.9687 - val_loss: 0.3112 - val_accuracy: 0.9479
Epoch 19/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1095 - accuracy: 0.9705 - val_loss: 0.2546 - val_accuracy: 0.9501
Epoch 20/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1044 - accuracy: 0.9717 - val_loss: 0.2411 - val_accuracy: 0.9490
120/120 [==============================] - 0s 946us/step - loss: 0.2420 - accuracy: 0.9459
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 2.7768 - accuracy: 0.2859 - val_loss: 1.7259 - val_accuracy: 0.3679
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 1.6659 - accuracy: 0.3798 - val_loss: 1.5691 - val_accuracy: 0.3944
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 1.3844 - accuracy: 0.4764 - val_loss: 1.2467 - val_accuracy: 0.5642
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 1.0794 - accuracy: 0.6186 - val_loss: 0.9056 - val_accuracy: 0.7038
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7928 - accuracy: 0.7319 - val_loss: 0.7438 - val_accuracy: 0.7511
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6715 - accuracy: 0.7647 - val_loss: 0.7007 - val_accuracy: 0.7607
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 0.5917 - accuracy: 0.7911 - val_loss: 0.6350 - val_accuracy: 0.7927
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 0.5051 - accuracy: 0.8328 - val_loss: 0.5344 - val_accuracy: 0.8431
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 0.4243 - accuracy: 0.8849 - val_loss: 0.4347 - val_accuracy: 0.9087
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3467 - accuracy: 0.9209 - val_loss: 0.3913 - val_accuracy: 0.9211
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2884 - accuracy: 0.9312 - val_loss: 0.3310 - val_accuracy: 0.9225
Epoch 12/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2368 - accuracy: 0.9415 - val_loss: 0.3062 - val_accuracy: 0.9347
Epoch 13/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1997 - accuracy: 0.9507 - val_loss: 0.2901 - val_accuracy: 0.9399
Epoch 14/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1725 - accuracy: 0.9563 - val_loss: 0.2459 - val_accuracy: 0.9435
Epoch 15/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1510 - accuracy: 0.9611 - val_loss: 0.2350 - val_accuracy: 0.9505
Epoch 16/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1359 - accuracy: 0.9642 - val_loss: 0.2287 - val_accuracy: 0.9549
Epoch 17/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1252 - accuracy: 0.9685 - val_loss: 0.2190 - val_accuracy: 0.9527
Epoch 18/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1170 - accuracy: 0.9687 - val_loss: 0.2306 - val_accuracy: 0.9533
Epoch 19/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1050 - accuracy: 0.9721 - val_loss: 0.2101 - val_accuracy: 0.9534
Epoch 20/20
480/480 [==============================] - 1s 1ms/step - loss: 0.0999 - accuracy: 0.9735 - val_loss: 0.2177 - val_accuracy: 0.9560
120/120 [==============================] - 0s 1ms/step - loss: 0.2063 - accuracy: 0.9556
Epoch 1/20
480/480 [==============================] - 1s 2ms/step - loss: 2.5524 - accuracy: 0.3559 - val_loss: 1.5160 - val_accuracy: 0.4838
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 1.4168 - accuracy: 0.4909 - val_loss: 1.2927 - val_accuracy: 0.5245
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 1.2003 - accuracy: 0.5737 - val_loss: 1.0746 - val_accuracy: 0.6434
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 0.9268 - accuracy: 0.6828 - val_loss: 0.8720 - val_accuracy: 0.7249
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7772 - accuracy: 0.7400 - val_loss: 0.7459 - val_accuracy: 0.7744
Epoch 6/20
480/480 [==============================] - 1s 2ms/step - loss: 0.6750 - accuracy: 0.7774 - val_loss: 0.6576 - val_accuracy: 0.7967
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 0.5690 - accuracy: 0.8166 - val_loss: 0.5560 - val_accuracy: 0.8308
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 0.4384 - accuracy: 0.8708 - val_loss: 0.4476 - val_accuracy: 0.8923
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3475 - accuracy: 0.9071 - val_loss: 0.3606 - val_accuracy: 0.9092
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2949 - accuracy: 0.9254 - val_loss: 0.3268 - val_accuracy: 0.9264
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2599 - accuracy: 0.9347 - val_loss: 0.3099 - val_accuracy: 0.9293
Epoch 12/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2246 - accuracy: 0.9440 - val_loss: 0.2851 - val_accuracy: 0.9360
Epoch 13/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1848 - accuracy: 0.9524 - val_loss: 0.2408 - val_accuracy: 0.9442
Epoch 14/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1572 - accuracy: 0.9591 - val_loss: 0.2231 - val_accuracy: 0.9496
Epoch 15/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1393 - accuracy: 0.9628 - val_loss: 0.1975 - val_accuracy: 0.9542
Epoch 16/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1257 - accuracy: 0.9661 - val_loss: 0.2198 - val_accuracy: 0.9504
Epoch 17/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1124 - accuracy: 0.9693 - val_loss: 0.2152 - val_accuracy: 0.9558
Epoch 18/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1083 - accuracy: 0.9698 - val_loss: 0.1769 - val_accuracy: 0.9592
Epoch 19/20
480/480 [==============================] - 1s 1ms/step - loss: 0.0940 - accuracy: 0.9740 - val_loss: 0.1950 - val_accuracy: 0.9559
Epoch 20/20
480/480 [==============================] - 1s 1ms/step - loss: 0.0968 - accuracy: 0.9726 - val_loss: 0.1812 - val_accuracy: 0.9602
120/120 [==============================] - 0s 950us/step - loss: 0.2214 - accuracy: 0.9488
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 3.3245 - accuracy: 0.1846 - val_loss: 2.0359 - val_accuracy: 0.2077
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 1.9854 - accuracy: 0.2391 - val_loss: 1.8600 - val_accuracy: 0.2882
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 1.6880 - accuracy: 0.3710 - val_loss: 1.5105 - val_accuracy: 0.4400
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 1.4073 - accuracy: 0.4752 - val_loss: 1.2940 - val_accuracy: 0.5145
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 1.1648 - accuracy: 0.5499 - val_loss: 1.0803 - val_accuracy: 0.5667
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 1.0055 - accuracy: 0.5993 - val_loss: 0.9730 - val_accuracy: 0.6260
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8825 - accuracy: 0.6503 - val_loss: 0.9118 - val_accuracy: 0.6656
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7781 - accuracy: 0.7123 - val_loss: 0.7761 - val_accuracy: 0.7435
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6832 - accuracy: 0.7648 - val_loss: 0.6676 - val_accuracy: 0.7845
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 0.5445 - accuracy: 0.8407 - val_loss: 0.5024 - val_accuracy: 0.8732
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3988 - accuracy: 0.8981 - val_loss: 0.3911 - val_accuracy: 0.9134
Epoch 12/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3084 - accuracy: 0.9257 - val_loss: 0.3224 - val_accuracy: 0.9304
Epoch 13/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2463 - accuracy: 0.9413 - val_loss: 0.3028 - val_accuracy: 0.9361
Epoch 14/20
480/480 [==============================] - 1s 1ms/step - loss: 0.2109 - accuracy: 0.9489 - val_loss: 0.2636 - val_accuracy: 0.9406
Epoch 15/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1847 - accuracy: 0.9548 - val_loss: 0.2429 - val_accuracy: 0.9468
Epoch 16/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1625 - accuracy: 0.9589 - val_loss: 0.2544 - val_accuracy: 0.9473
Epoch 17/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1499 - accuracy: 0.9621 - val_loss: 0.2411 - val_accuracy: 0.9447
Epoch 18/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1369 - accuracy: 0.9650 - val_loss: 0.2669 - val_accuracy: 0.9363
Epoch 19/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1258 - accuracy: 0.9668 - val_loss: 0.2514 - val_accuracy: 0.9449
Epoch 20/20
480/480 [==============================] - 1s 1ms/step - loss: 0.1134 - accuracy: 0.9701 - val_loss: 0.2294 - val_accuracy: 0.9514
120/120 [==============================] - 0s 924us/step - loss: 0.2413 - accuracy: 0.9514
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 2.7519 - accuracy: 0.3924 - val_loss: 1.4163 - val_accuracy: 0.5382
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 1.4105 - accuracy: 0.5388 - val_loss: 1.2557 - val_accuracy: 0.5795
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 1.2292 - accuracy: 0.5811 - val_loss: 1.0991 - val_accuracy: 0.6036
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 1.0706 - accuracy: 0.6237 - val_loss: 1.1248 - val_accuracy: 0.6442
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 0.9628 - accuracy: 0.6666 - val_loss: 1.0244 - val_accuracy: 0.6694
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8747 - accuracy: 0.6904 - val_loss: 0.9018 - val_accuracy: 0.6960
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8319 - accuracy: 0.7056 - val_loss: 0.9174 - val_accuracy: 0.7234
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7720 - accuracy: 0.7252 - val_loss: 0.9151 - val_accuracy: 0.7233
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7364 - accuracy: 0.7443 - val_loss: 0.8280 - val_accuracy: 0.7295
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6945 - accuracy: 0.7630 - val_loss: 0.8265 - val_accuracy: 0.7617
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6643 - accuracy: 0.7793 - val_loss: 0.8584 - val_accuracy: 0.7920
Epoch 12/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6174 - accuracy: 0.7974 - val_loss: 0.7772 - val_accuracy: 0.7914
Epoch 13/20
480/480 [==============================] - 1s 1ms/step - loss: 0.5839 - accuracy: 0.8116 - val_loss: 0.7815 - val_accuracy: 0.7960
Epoch 14/20
480/480 [==============================] - 1s 1ms/step - loss: 0.5400 - accuracy: 0.8411 - val_loss: 0.7302 - val_accuracy: 0.8830
Epoch 15/20
480/480 [==============================] - 0s 1ms/step - loss: 0.4754 - accuracy: 0.8962 - val_loss: 0.6860 - val_accuracy: 0.8987
Epoch 16/20
480/480 [==============================] - 0s 1ms/step - loss: 0.4465 - accuracy: 0.9064 - val_loss: 0.7874 - val_accuracy: 0.9116
Epoch 17/20
480/480 [==============================] - 1s 1ms/step - loss: 0.4011 - accuracy: 0.9110 - val_loss: 0.6430 - val_accuracy: 0.8984
Epoch 18/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3999 - accuracy: 0.9174 - val_loss: 0.6453 - val_accuracy: 0.9066
Epoch 19/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3766 - accuracy: 0.9220 - val_loss: 0.6824 - val_accuracy: 0.9135
Epoch 20/20
480/480 [==============================] - 1s 1ms/step - loss: 0.3654 - accuracy: 0.9242 - val_loss: 0.6774 - val_accuracy: 0.9197
120/120 [==============================] - 0s 892us/step - loss: 0.5788 - accuracy: 0.9158
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 2.4504 - accuracy: 0.3422 - val_loss: 1.5268 - val_accuracy: 0.4474
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 1.4233 - accuracy: 0.4900 - val_loss: 1.3790 - val_accuracy: 0.4998
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 1.2919 - accuracy: 0.5213 - val_loss: 1.3350 - val_accuracy: 0.5303
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 1.1990 - accuracy: 0.5392 - val_loss: 1.2118 - val_accuracy: 0.5375
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 1.1486 - accuracy: 0.5489 - val_loss: 1.2154 - val_accuracy: 0.5484
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 1.1059 - accuracy: 0.5728 - val_loss: 1.2285 - val_accuracy: 0.5739
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 1.0460 - accuracy: 0.6015 - val_loss: 1.0976 - val_accuracy: 0.5885
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 0.9958 - accuracy: 0.6120 - val_loss: 1.3047 - val_accuracy: 0.6245
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 0.9606 - accuracy: 0.6219 - val_loss: 1.0633 - val_accuracy: 0.6178
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 0.9417 - accuracy: 0.6305 - val_loss: 1.0764 - val_accuracy: 0.6146
Epoch 11/20
480/480 [==============================] - 0s 1ms/step - loss: 0.9259 - accuracy: 0.6342 - val_loss: 1.1188 - val_accuracy: 0.6103
Epoch 12/20
480/480 [==============================] - 1s 1ms/step - loss: 0.9025 - accuracy: 0.6404 - val_loss: 1.1207 - val_accuracy: 0.6482
Epoch 13/20
480/480 [==============================] - 0s 1ms/step - loss: 0.8918 - accuracy: 0.6433 - val_loss: 1.1242 - val_accuracy: 0.6252
Epoch 14/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8792 - accuracy: 0.6431 - val_loss: 1.2413 - val_accuracy: 0.6472
Epoch 15/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8712 - accuracy: 0.6540 - val_loss: 1.1232 - val_accuracy: 0.6451
Epoch 16/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8535 - accuracy: 0.6598 - val_loss: 1.1871 - val_accuracy: 0.6388
Epoch 17/20
480/480 [==============================] - 0s 1ms/step - loss: 0.8449 - accuracy: 0.6606 - val_loss: 1.2077 - val_accuracy: 0.6688
Epoch 18/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8293 - accuracy: 0.6758 - val_loss: 1.1782 - val_accuracy: 0.6793
Epoch 19/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8157 - accuracy: 0.6819 - val_loss: 1.1945 - val_accuracy: 0.6757
120/120 [==============================] - 0s 835us/step - loss: 0.9932 - accuracy: 0.6137
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 2.5621 - accuracy: 0.2704 - val_loss: 1.8347 - val_accuracy: 0.3176
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 1.7044 - accuracy: 0.3576 - val_loss: 1.5304 - val_accuracy: 0.4252
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 1.4693 - accuracy: 0.4457 - val_loss: 1.4143 - val_accuracy: 0.4907
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 1.3133 - accuracy: 0.4834 - val_loss: 1.3102 - val_accuracy: 0.4956
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 1.1775 - accuracy: 0.5436 - val_loss: 1.1184 - val_accuracy: 0.5868
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 1.0744 - accuracy: 0.5855 - val_loss: 1.1372 - val_accuracy: 0.6047
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 1.0253 - accuracy: 0.6012 - val_loss: 1.0412 - val_accuracy: 0.6106
Epoch 8/20
480/480 [==============================] - 0s 1ms/step - loss: 0.9840 - accuracy: 0.6132 - val_loss: 1.0844 - val_accuracy: 0.5863
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 0.9553 - accuracy: 0.6235 - val_loss: 1.0209 - val_accuracy: 0.6166
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 0.9441 - accuracy: 0.6334 - val_loss: 1.0941 - val_accuracy: 0.6287
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 0.9183 - accuracy: 0.6398 - val_loss: 1.0209 - val_accuracy: 0.6395
Epoch 12/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8967 - accuracy: 0.6451 - val_loss: 1.0313 - val_accuracy: 0.6391
Epoch 13/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8806 - accuracy: 0.6544 - val_loss: 1.1046 - val_accuracy: 0.6525
Epoch 14/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8578 - accuracy: 0.6625 - val_loss: 1.1376 - val_accuracy: 0.6602
Epoch 15/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8550 - accuracy: 0.6691 - val_loss: 1.0461 - val_accuracy: 0.6650
Epoch 16/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8197 - accuracy: 0.6753 - val_loss: 1.0604 - val_accuracy: 0.6649
Epoch 17/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8237 - accuracy: 0.6820 - val_loss: 1.1282 - val_accuracy: 0.6792
Epoch 18/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8163 - accuracy: 0.6924 - val_loss: 1.1923 - val_accuracy: 0.6803
Epoch 19/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7821 - accuracy: 0.6990 - val_loss: 1.1407 - val_accuracy: 0.6900
120/120 [==============================] - 0s 916us/step - loss: 1.0889 - accuracy: 0.6072
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 2.7644 - accuracy: 0.1947 - val_loss: 2.0768 - val_accuracy: 0.2244
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 1.9819 - accuracy: 0.2446 - val_loss: 1.9224 - val_accuracy: 0.2653
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 1.8320 - accuracy: 0.3207 - val_loss: 1.6775 - val_accuracy: 0.3620
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 1.6038 - accuracy: 0.4033 - val_loss: 1.5520 - val_accuracy: 0.4297
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 1.5232 - accuracy: 0.4200 - val_loss: 1.4504 - val_accuracy: 0.4339
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 1.4805 - accuracy: 0.4404 - val_loss: 1.4121 - val_accuracy: 0.4646
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 1.2693 - accuracy: 0.5332 - val_loss: 1.2018 - val_accuracy: 0.5578
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 1.0975 - accuracy: 0.6067 - val_loss: 1.1213 - val_accuracy: 0.6006
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 1.0104 - accuracy: 0.6226 - val_loss: 1.1118 - val_accuracy: 0.6265
Epoch 10/20
480/480 [==============================] - 1s 2ms/step - loss: 0.9673 - accuracy: 0.6340 - val_loss: 1.1221 - val_accuracy: 0.6272
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 0.9037 - accuracy: 0.6503 - val_loss: 1.0522 - val_accuracy: 0.6698
Epoch 12/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8394 - accuracy: 0.6937 - val_loss: 0.9550 - val_accuracy: 0.6807
Epoch 13/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8031 - accuracy: 0.7078 - val_loss: 0.9646 - val_accuracy: 0.7041
Epoch 14/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7489 - accuracy: 0.7296 - val_loss: 0.8833 - val_accuracy: 0.7271
Epoch 15/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7136 - accuracy: 0.7502 - val_loss: 0.8739 - val_accuracy: 0.7593
Epoch 16/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6858 - accuracy: 0.7663 - val_loss: 0.8684 - val_accuracy: 0.7811
Epoch 17/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6535 - accuracy: 0.7773 - val_loss: 0.9675 - val_accuracy: 0.7836
Epoch 18/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6339 - accuracy: 0.7812 - val_loss: 0.9092 - val_accuracy: 0.7517
Epoch 19/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6095 - accuracy: 0.7910 - val_loss: 1.0667 - val_accuracy: 0.8068
Epoch 20/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6095 - accuracy: 0.7978 - val_loss: 0.9706 - val_accuracy: 0.7895
120/120 [==============================] - 0s 968us/step - loss: 0.8270 - accuracy: 0.7798
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 2.7082 - accuracy: 0.2888 - val_loss: 1.7505 - val_accuracy: 0.3396
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 1.5574 - accuracy: 0.4318 - val_loss: 1.4637 - val_accuracy: 0.4486
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 1.4202 - accuracy: 0.4585 - val_loss: 1.3848 - val_accuracy: 0.4788
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 1.3238 - accuracy: 0.4895 - val_loss: 1.2386 - val_accuracy: 0.5217
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 1.1810 - accuracy: 0.5360 - val_loss: 1.1664 - val_accuracy: 0.5437
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 1.1188 - accuracy: 0.5560 - val_loss: 1.1303 - val_accuracy: 0.5587
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 1.0607 - accuracy: 0.5853 - val_loss: 1.1368 - val_accuracy: 0.6109
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 0.9822 - accuracy: 0.6290 - val_loss: 1.1597 - val_accuracy: 0.5958
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 0.9216 - accuracy: 0.6520 - val_loss: 0.9576 - val_accuracy: 0.6566
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8692 - accuracy: 0.6749 - val_loss: 0.9122 - val_accuracy: 0.6843
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8473 - accuracy: 0.6832 - val_loss: 0.8830 - val_accuracy: 0.6733
Epoch 12/20
480/480 [==============================] - 1s 1ms/step - loss: 0.8270 - accuracy: 0.6947 - val_loss: 0.9186 - val_accuracy: 0.7073
Epoch 13/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7799 - accuracy: 0.7046 - val_loss: 1.0920 - val_accuracy: 0.6714
Epoch 14/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7741 - accuracy: 0.7098 - val_loss: 0.9068 - val_accuracy: 0.7138
Epoch 15/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7585 - accuracy: 0.7134 - val_loss: 0.9038 - val_accuracy: 0.7043
Epoch 16/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7525 - accuracy: 0.7170 - val_loss: 0.8882 - val_accuracy: 0.7116
Epoch 17/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7337 - accuracy: 0.7177 - val_loss: 0.9358 - val_accuracy: 0.6877
Epoch 18/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7281 - accuracy: 0.7254 - val_loss: 1.0716 - val_accuracy: 0.7300
Epoch 19/20
480/480 [==============================] - 1s 1ms/step - loss: 0.7181 - accuracy: 0.7410 - val_loss: 0.8905 - val_accuracy: 0.7554
Epoch 20/20
480/480 [==============================] - 1s 1ms/step - loss: 0.6932 - accuracy: 0.7534 - val_loss: 0.9336 - val_accuracy: 0.7530
120/120 [==============================] - 0s 1ms/step - loss: 0.9109 - accuracy: 0.7528
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 87234.7578 - accuracy: 0.1113 - val_loss: 2.3014 - val_accuracy: 0.1136
Epoch 2/20
480/480 [==============================] - 0s 1ms/step - loss: 2.3016 - accuracy: 0.1123 - val_loss: 2.3010 - val_accuracy: 0.1136
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3014 - accuracy: 0.1123 - val_loss: 2.3009 - val_accuracy: 0.1136
Epoch 4/20
480/480 [==============================] - 0s 1ms/step - loss: 2.3013 - accuracy: 0.1123 - val_loss: 2.3008 - val_accuracy: 0.1136
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3013 - accuracy: 0.1123 - val_loss: 2.3008 - val_accuracy: 0.1136
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3013 - accuracy: 0.1123 - val_loss: 2.3008 - val_accuracy: 0.1136
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3013 - accuracy: 0.1123 - val_loss: 2.3008 - val_accuracy: 0.1136
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3013 - accuracy: 0.1123 - val_loss: 2.3008 - val_accuracy: 0.1136
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3013 - accuracy: 0.1123 - val_loss: 2.3008 - val_accuracy: 0.1136
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3013 - accuracy: 0.1123 - val_loss: 2.3008 - val_accuracy: 0.1136
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3013 - accuracy: 0.1123 - val_loss: 2.3008 - val_accuracy: 0.1136
120/120 [==============================] - 0s 1ms/step - loss: 3.0497 - accuracy: 0.1126
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 7.2950 - accuracy: 0.1086 - val_loss: 2.3017 - val_accuracy: 0.1135
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3016 - accuracy: 0.1117 - val_loss: 2.3013 - val_accuracy: 0.1135
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3014 - accuracy: 0.1117 - val_loss: 2.3012 - val_accuracy: 0.1135
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3013 - accuracy: 0.1117 - val_loss: 2.3011 - val_accuracy: 0.1135
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3013 - accuracy: 0.1117 - val_loss: 2.3011 - val_accuracy: 0.1135
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1117 - val_loss: 2.3011 - val_accuracy: 0.1135
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1117 - val_loss: 2.3010 - val_accuracy: 0.1135
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1117 - val_loss: 2.3010 - val_accuracy: 0.1135
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1117 - val_loss: 2.3010 - val_accuracy: 0.1135
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1117 - val_loss: 2.3010 - val_accuracy: 0.1135
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1117 - val_loss: 2.3010 - val_accuracy: 0.1135
120/120 [==============================] - 0s 952us/step - loss: 2.3017 - accuracy: 0.1151
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 110.8144 - accuracy: 0.1114 - val_loss: 2.3013 - val_accuracy: 0.1136
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3015 - accuracy: 0.1117 - val_loss: 2.3010 - val_accuracy: 0.1136
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3013 - accuracy: 0.1117 - val_loss: 2.3009 - val_accuracy: 0.1136
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3013 - accuracy: 0.1117 - val_loss: 2.3009 - val_accuracy: 0.1136
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3013 - accuracy: 0.1117 - val_loss: 2.3008 - val_accuracy: 0.1136
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1117 - val_loss: 2.3008 - val_accuracy: 0.1136
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1117 - val_loss: 2.3008 - val_accuracy: 0.1136
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1117 - val_loss: 2.3008 - val_accuracy: 0.1136
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1117 - val_loss: 2.3008 - val_accuracy: 0.1136
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1117 - val_loss: 2.3008 - val_accuracy: 0.1136
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1117 - val_loss: 2.3008 - val_accuracy: 0.1136
120/120 [==============================] - 0s 1ms/step - loss: 2.3016 - accuracy: 0.1150
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 9.8984 - accuracy: 0.1123 - val_loss: 2.3017 - val_accuracy: 0.1135
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3015 - accuracy: 0.1122 - val_loss: 2.3013 - val_accuracy: 0.1135
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3013 - accuracy: 0.1122 - val_loss: 2.3012 - val_accuracy: 0.1135
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1122 - val_loss: 2.3011 - val_accuracy: 0.1135
Epoch 5/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1122 - val_loss: 2.3011 - val_accuracy: 0.1135
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1122 - val_loss: 2.3010 - val_accuracy: 0.1135
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1122 - val_loss: 2.3011 - val_accuracy: 0.1135
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1122 - val_loss: 2.3011 - val_accuracy: 0.1135
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1122 - val_loss: 2.3010 - val_accuracy: 0.1135
Epoch 10/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1122 - val_loss: 2.3011 - val_accuracy: 0.1135
Epoch 11/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3012 - accuracy: 0.1122 - val_loss: 2.3011 - val_accuracy: 0.1135
120/120 [==============================] - 0s 939us/step - loss: 2.3017 - accuracy: 0.1131
Epoch 1/20
480/480 [==============================] - 1s 1ms/step - loss: 9.7886 - accuracy: 0.1139 - val_loss: 2.3015 - val_accuracy: 0.1135
Epoch 2/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3014 - accuracy: 0.1140 - val_loss: 2.3012 - val_accuracy: 0.1135
Epoch 3/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3011 - accuracy: 0.1140 - val_loss: 2.3011 - val_accuracy: 0.1135
Epoch 4/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3010 - accuracy: 0.1140 - val_loss: 2.3010 - val_accuracy: 0.1135
Epoch 5/20
480/480 [==============================] - 0s 1ms/step - loss: 2.3010 - accuracy: 0.1140 - val_loss: 2.3010 - val_accuracy: 0.1135
Epoch 6/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3010 - accuracy: 0.1140 - val_loss: 2.3010 - val_accuracy: 0.1135
Epoch 7/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3010 - accuracy: 0.1140 - val_loss: 2.3010 - val_accuracy: 0.1135
Epoch 8/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3010 - accuracy: 0.1140 - val_loss: 2.3010 - val_accuracy: 0.1135
Epoch 9/20
480/480 [==============================] - 1s 1ms/step - loss: 2.3010 - accuracy: 0.1140 - val_loss: 2.3010 - val_accuracy: 0.1135
Epoch 10/20
480/480 [==============================] - 0s 1ms/step - loss: 2.3010 - accuracy: 0.1140 - val_loss: 2.3010 - val_accuracy: 0.1135
Epoch 11/20
480/480 [==============================] - 0s 1ms/step - loss: 2.3010 - accuracy: 0.1140 - val_loss: 2.3010 - val_accuracy: 0.1135
120/120 [==============================] - 0s 935us/step - loss: 2.3020 - accuracy: 0.1060
Epoch 1/20
600/600 [==============================] - 1s 1ms/step - loss: 2.7009 - accuracy: 0.1527 - val_loss: 2.1169 - val_accuracy: 0.1903
Epoch 2/20
600/600 [==============================] - 1s 1ms/step - loss: 2.0917 - accuracy: 0.1965 - val_loss: 2.0771 - val_accuracy: 0.1956
Epoch 3/20
600/600 [==============================] - 1s 1ms/step - loss: 2.0599 - accuracy: 0.2015 - val_loss: 2.0534 - val_accuracy: 0.2011
Epoch 4/20
600/600 [==============================] - 1s 1ms/step - loss: 2.0409 - accuracy: 0.2064 - val_loss: 2.0769 - val_accuracy: 0.1911
Epoch 5/20
600/600 [==============================] - 1s 1ms/step - loss: 1.9538 - accuracy: 0.2402 - val_loss: 1.8549 - val_accuracy: 0.2501
Epoch 6/20
600/600 [==============================] - 1s 1ms/step - loss: 1.7357 - accuracy: 0.3199 - val_loss: 1.5059 - val_accuracy: 0.4154
Epoch 7/20
600/600 [==============================] - 1s 1ms/step - loss: 1.1947 - accuracy: 0.5837 - val_loss: 0.9818 - val_accuracy: 0.6618
Epoch 8/20
600/600 [==============================] - 1s 1ms/step - loss: 0.8230 - accuracy: 0.7465 - val_loss: 0.6804 - val_accuracy: 0.8367
Epoch 9/20
600/600 [==============================] - 1s 1ms/step - loss: 0.5241 - accuracy: 0.8632 - val_loss: 0.4124 - val_accuracy: 0.8920
Epoch 10/20
600/600 [==============================] - 1s 1ms/step - loss: 0.3225 - accuracy: 0.9134 - val_loss: 0.3285 - val_accuracy: 0.9135
Epoch 11/20
600/600 [==============================] - 1s 1ms/step - loss: 0.2356 - accuracy: 0.9359 - val_loss: 0.2794 - val_accuracy: 0.9328
Epoch 12/20
600/600 [==============================] - 1s 1ms/step - loss: 0.1967 - accuracy: 0.9467 - val_loss: 0.2616 - val_accuracy: 0.9396
Epoch 13/20
600/600 [==============================] - 1s 1ms/step - loss: 0.1718 - accuracy: 0.9528 - val_loss: 0.2322 - val_accuracy: 0.9476
Epoch 14/20
600/600 [==============================] - 1s 1ms/step - loss: 0.1533 - accuracy: 0.9582 - val_loss: 0.2136 - val_accuracy: 0.9491
Epoch 15/20
600/600 [==============================] - 1s 1ms/step - loss: 0.1370 - accuracy: 0.9625 - val_loss: 0.2265 - val_accuracy: 0.9508
Epoch 16/20
600/600 [==============================] - 1s 1ms/step - loss: 0.1257 - accuracy: 0.9651 - val_loss: 0.2214 - val_accuracy: 0.9505
Epoch 17/20
600/600 [==============================] - 1s 1ms/step - loss: 0.1112 - accuracy: 0.9689 - val_loss: 0.2214 - val_accuracy: 0.9545
Epoch 18/20
600/600 [==============================] - 1s 1ms/step - loss: 0.1016 - accuracy: 0.9712 - val_loss: 0.2034 - val_accuracy: 0.9586
Epoch 19/20
600/600 [==============================] - 1s 1ms/step - loss: 0.0952 - accuracy: 0.9732 - val_loss: 0.2020 - val_accuracy: 0.9583
Epoch 20/20
600/600 [==============================] - 1s 1ms/step - loss: 0.0861 - accuracy: 0.9749 - val_loss: 0.2102 - val_accuracy: 0.9581

GridSearchCV(estimator=<keras.wrappers.scikit_learn.KerasClassifier object at 0x7931f4cd13d0>,
             param_grid={'optimizer': ['adam', 'rmsprop', 'sgd']})

GridSearchCV(estimator=<keras.wrappers.scikit_learn.KerasClassifier object at 0x7931f4cd13d0>,
param_grid={'optimizer': ['adam', 'rmsprop', 'sgd']})

print("Best parameters found: ", cv.best_params_)
best_model = cv.best_estimator_

Best parameters found:  {'optimizer': 'adam'}

Tuning a Random Forest

Demo about using grid-search and random-search to tune the hyperparameters of a Random Forest network traffic classifier

Avoiding overfitting with regularization techniques

Demo about using L1/L2 normalization, Dropout and Earlystopping to avoid overfitting the training data

Lab: Implementation of a hyperparameter tuning process

Improve the Grid search by:
- tuning more hyperparameters (hidden units and dropout rate)
- adding early-stopping to speed up the tuning process and avoid overfitting
Implement a Randomized Search by taking inspiration from the Random Forest code available in the same folder. Add early-stopping and set the number of iterations to 20 or 30, to avoid waiting for a long time
Compare Grid Search with Randomized Search in terms of execution time and accuracy of the best model on the validation set
More details are available in the Jupyter notebook