{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"Αυτοκωδικοποιητές.ipynb","provenance":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","metadata":{"id":"8RwDm0n8n-7s"},"source":["# Αυτοκωδικοποιητές\n","\n","\n","\n","Όπως είπαμε και στη διάλεξη, οι **αυτοκωδικοποιητές** (*autoencoders*) είναι νευρωνικά δίκτυα τα οποία εκπαιδεύονται στο να αντιγράφουν την είσοδό τους στην έξοδό τους. Εσωτερικά αποτελούνται από κρυφό επίπεδο $\\mathbf{h}$, στο οποίο αναπαρίσταται (κωδικοποιείται) η είσοδος, ενώ το δίκτυο περιλαμβάνει δύο μέρη:\n","1. τη συνάρτηση **κωδικοποίησης** (*encoder function*) $\\mathbf{h} = f(x)$\n","1. τη συνάρτηση **αποκωδικοποίησης** (*decoder function*) $\\mathbf{r} = g(\\mathbf{h})$\n","\n","\n","Η εκπαίδευση τους πραγματοποιείται είτε μέσω τεχνικών οπίσθιας διάδοσης του σφάλματος (backpropagation) είτε μέσω **επανακυκλοφορίας** (recirculation), όπου συγκρίνεται η ενεργοποίηση των νευρώνων στην αρχική είσοδο και στην αναπαράσταση. Σε αυτό το notebook, η εκπαίδευση των δικτύων που θα δούμε βασίζεται στην πρώτη τεχνική.\n","\n","Οι αυτοκωδικοποιητές παρουσιάστηκαν για πρώτη φορά στα μέσα της δεκαετίας του 1980. Αν και η λειτουργία τους προσομοιάζει αρκετά με τους **απωλεστικούς αλγορίθμους συμπίεσης** (*lossy compresion algorithms*) όπως θα δούμε στη συνέχεια, χρησιμοποιούνται κυρίως σε προβλήματα **μείωσης διαστατικότητας** (*dimensionality reduction*) και **εξαγωγής χαρακτηριστικών** (*feature extraction*)."]},{"cell_type":"markdown","metadata":{"id":"ZfTvrH68vMLk"},"source":["## Εισαγωγή\n"]},{"cell_type":"markdown","metadata":{"id":"6IpSl0RghaJ4"},"source":["### Εισαγωγή Βιβλιοθηκών"]},{"cell_type":"markdown","metadata":{"id":"AgZqMTGEpfPa"},"source":["Στη συνέχεια εισάγουμε τις κλάσεις και τις μεθόδους που πρόκειται να χρησιμοποιήσουμε. Επίσης για να μπορούμε να αναπαράξουμε τα αποτελέσματα που θα λάβουμε, αρχικοποιούμε τη γεννήτρια ψευδοτυχαίων αριθμών χρησιμοποιώντας την ίδια σπορά (εδώ τον αριθμό $2021$)."]},{"cell_type":"code","metadata":{"id":"w4tmYQGMS374"},"source":["from tensorflow.keras import regularizers\n","from tensorflow.keras.datasets import mnist\n","from tensorflow.keras.layers import Input, Dense\n","from tensorflow.keras.models import Sequential\n","from sklearn.preprocessing import MinMaxScaler\n","from sklearn.metrics.pairwise import cosine_similarity\n","\n","import matplotlib.pyplot as plt\n","import numpy as np\n","import pandas as pd\n","\n","# Θέτουμε σπορά (seed) στη γεννήτρια ψευδο-τυχαίων αριθμών για να λάβουμε\n","# ίδια αποτελέσματα\n","\n","np.random.seed(2022)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"5LoZ5pqrheof"},"source":["### Ορισμός συναρτήσεων"]},{"cell_type":"markdown","metadata":{"id":"r7HHOIhyp7mQ"},"source":["Ορίζουμε δύο συναρτήσεις (μεθόδους) που πρόκειται να χρησιμοποιήσουμε για την παρουσίαση των αποτελεσμάτων, την *compare_digits()* και την *display_digits()*"]},{"cell_type":"code","metadata":{"id":"tBvkPPF-fr2V"},"source":["def compare_digits(test, decoded, n=10):\n"," plt.figure(figsize=(20, 4))\n"," for i in range(n):\n"," # display original\n"," ax = plt.subplot(2, n, i + 1)\n"," plt.imshow(test[i].reshape(28, 28))\n"," plt.gray()\n"," ax.get_xaxis().set_visible(False)\n"," ax.get_yaxis().set_visible(False)\n","\n"," # display reconstruction\n"," ax = plt.subplot(2, n, i + 1 + n)\n"," plt.imshow(decoded[i].reshape(28, 28))\n"," plt.gray()\n"," ax.get_xaxis().set_visible(False)\n"," ax.get_yaxis().set_visible(False)\n"," plt.show()\n"," \n"," \n","def display_digits(data, n=10):\n"," plt.figure(figsize=(20, 2))\n"," for i in range(n):\n"," ax = plt.subplot(1, n, i + 1)\n"," plt.imshow(data[i].reshape(28, 28))\n"," plt.gray()\n"," ax.get_xaxis().set_visible(False)\n"," ax.get_yaxis().set_visible(False)\n"," plt.show()"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"N-tUpSqUhkRK"},"source":["## Δεδομένα"]},{"cell_type":"markdown","metadata":{"id":"piEA2NarqaKm"},"source":["Για να κατανοήσουμε καλύτερα τη λειτουργία των αυτοκωδικοποιητών, θα χρησιμοποιήσουμε το (γνώστό σε προβλήματα βαθιάς μηχανικής μάθησης) dataset [MNIST,](http://yann.lecun.com/exdb/mnist/), το οποίο περιέχει $70.000$ σκαναρισμένες εικόνες ψηφίων που αντιστοιχούν στους αριθμούς $0$ ως $9$, όπως αυτοί έχουν γραφτεί από διαφορετικούς ανθρώπους.\n","\n","Η κάθε εικόνα έχει μέγεθος $28\\times28$ pixels και είναι ασπρόμαυρη. Επιπλέον, οι δημιουργοί του dataset τις έχουν χωρίσει σε δύο σύνολα, στο σύνολο εκπαίδευσης (που περιέχει $60.000$ εικόνες) και στο σύνολο ελέγχου (που περιέχει $10.000$ εικόνες). \n","\n","Μιας και το MNIST dataset παρέχεται (και) μέσω του keras, το φορτώνουμε μέσω αυτής της βιβλιοθήκης. Κατόπιν μετατρέπουμε την δισδιάστατη εικόνα σε ένα διάνυσμα $784$ χαρακτηριστικών και τέλος μετασχηματίζουμε γραμμικά τις τιμές των pixels από το $[0,255]$ στο $[0,1]$ μέσω της κλάσης *MinMaxScaler* του scikit-learn."]},{"cell_type":"code","metadata":{"id":"0gg1ABRRUfwz","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1642512628272,"user_tz":-120,"elapsed":1337,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"25b852fc-536e-404c-b1a1-1a7f800d3a11"},"source":["(x_train, _), (x_test, _) = mnist.load_data()\n","\n","x_train = x_train.reshape((len(x_train), np.prod(x_train.shape[1:])))\n","x_test = x_test.reshape((len(x_test), np.prod(x_test.shape[1:])))\n","\n","scaler = MinMaxScaler()\n","x_train = scaler.fit_transform(x_train)\n","x_test = scaler.fit_transform(x_test)\n","\n","print(x_train.shape)\n","print(x_test.shape)"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n","11493376/11490434 [==============================] - 0s 0us/step\n","11501568/11490434 [==============================] - 0s 0us/step\n","(60000, 784)\n","(10000, 784)\n"]}]},{"cell_type":"markdown","metadata":{"id":"VU7CeHUUhqak"},"source":["## Υποπλήρης αυτοκωδιοποιητής"]},{"cell_type":"markdown","metadata":{"id":"js2_LTwkvdVe"},"source":["Σύμφωνα με τα όσα είπαμε στη διάλεξη, κατά την εκπαίδευση των αυτοκωδικοποιητών δεν μας ενδιαφέρει τόσο η διαδικασία της αντιγραφής, όσο το να μάθει το δίκτυο να εξάγει και να αποτυπώνει στο επίπεδο κωδικοποίησης ($\\mathbf{h}$) χρήσιμες ιδιότητες του χώρου εισόδου ($\\mathbf{x}$). Ένας τρόπος για να αποτρέπουμε το δίκτυο από το να μάθει απλά να αντιγράφει την είσοδό του στην έξοδό του είναι να επιβάλλουμε τον περιορισμό η διάσταση της κωδικοποίησης να είναι μικρότερη της διάστασης της εισόδου ($\\dim(\\mathbf{h}) < \\dim(\\mathbf{x}))$, κατασκευάζοντας κατ' αυτόν τον τρόπο τους **υποπλήρεις αυτοκωδικοποιητές** (*underbomplete autoencoders*)\n","\n","Στο παράδειγμα που εξετάζουμε, η διάσταση της εισόδου είναι $28\\times28 = 784$ χαρακτηριστικά. Θα την περιορίσουμε (αυθαίρετα) στα $32$ (επιτυγχάνοντας ένα λόγο συμπίεσης $24.5$)"]},{"cell_type":"code","metadata":{"id":"M5HZ7q0YwoLe"},"source":["dim_x = 28*28\n","dim_h = 32"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"qrAfyx4nwsBs"},"source":["Επιλέγουμε για συνάρτηση κωδικοποίησης την [Rectified Linear Unit (ReLU)](https://en.wikipedia.org/wiki/Rectifier_%28neural_networks%29) και για συνάρτηση αποκωδικοποίησης τη [σιγμοειδή](https://en.wikipedia.org/wiki/Sigmoid_function). Έτσι η συνάρτηση κωδικοποίησης γίνεται $f(\\mathbf{x}) = \\max(0, \\mathbf{w}_e\\mathbf{x} + \\mathbf{b}_e)$ και η συνάρτηση αποκωδικοποίησης $g(\\mathbf{h}) = S(\\mathbf{w}_d\\mathbf{h} + \\mathbf{b}_d)$. Κατά τη διάρκεια της εκπαίδευσης (που πραγματοποιείται με τη μέθοδο Adam), το δίκτυο μαθαίνει τα βάρη ($\\mathbf{w}_e, \\mathbf{w}_d$) και τις πολώσεις ($\\mathbf{b}_e, \\mathbf{b}_d$). Τέλος, η συνάρτηση σφάλματος που βελτιστοποιείται είναι η [δυαδική διασταυρούμενη εντροπία](https://en.wikipedia.org/wiki/Cross_entropy)"]},{"cell_type":"code","metadata":{"id":"-1LBKWxYOPzg","executionInfo":{"status":"ok","timestamp":1642512839401,"user_tz":-120,"elapsed":242,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"452a08bc-d8b6-4f9a-82da-ed3437bd0fe7"},"source":["autoencoder = Sequential([\n"," Dense(dim_h, activation='relu', input_shape=(dim_x,), name=\"encoder\"),\n"," Dense(dim_x, activation='sigmoid', name=\"decoder\")\n","])\n","\n","autoencoder.summary()\n","\n","autoencoder.compile(optimizer='adam', loss='binary_crossentropy')"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Model: \"sequential_1\"\n","_________________________________________________________________\n"," Layer (type) Output Shape Param # \n","=================================================================\n"," encoder (Dense) (None, 32) 25120 \n"," \n"," decoder (Dense) (None, 784) 25872 \n"," \n","=================================================================\n","Total params: 50,992\n","Trainable params: 50,992\n","Non-trainable params: 0\n","_________________________________________________________________\n"]}]},{"cell_type":"markdown","metadata":{"id":"giNZHkyH39Qy"},"source":["Εκπαιδεύουμε το δίκτυο που ορίσαμε προηγουμένως για 20 εποχές και με batch size 64. Προσέξτε ότι, μιας και πρόκειται για αυτοκωδικοποιητή, η είσοδος και η έξοδος ταυτίζονται (x_train)."]},{"cell_type":"code","metadata":{"id":"2FrkksFgUr__","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1642512956747,"user_tz":-120,"elapsed":79039,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"b7f04c63-b83e-4e06-b389-e784e7d6d84f"},"source":["autoencoder.fit(x_train, x_train,\n"," epochs=20,\n"," batch_size=64,\n"," shuffle=True,\n"," validation_data=(x_test, x_test))"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Epoch 1/20\n","938/938 [==============================] - 6s 4ms/step - loss: 0.1905 - val_loss: 0.1332\n","Epoch 2/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1193 - val_loss: 0.1073\n","Epoch 3/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1029 - val_loss: 0.0976\n","Epoch 4/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0970 - val_loss: 0.0945\n","Epoch 5/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0951 - val_loss: 0.0933\n","Epoch 6/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0944 - val_loss: 0.0928\n","Epoch 7/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0940 - val_loss: 0.0926\n","Epoch 8/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0938 - val_loss: 0.0925\n","Epoch 9/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0936 - val_loss: 0.0924\n","Epoch 10/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0935 - val_loss: 0.0923\n","Epoch 11/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0934 - val_loss: 0.0925\n","Epoch 12/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0934 - val_loss: 0.0920\n","Epoch 13/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0933 - val_loss: 0.0920\n","Epoch 14/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0932 - val_loss: 0.0920\n","Epoch 15/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0932 - val_loss: 0.0920\n","Epoch 16/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0932 - val_loss: 0.0922\n","Epoch 17/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0931 - val_loss: 0.0920\n","Epoch 18/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0931 - val_loss: 0.0918\n","Epoch 19/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0931 - val_loss: 0.0919\n","Epoch 20/20\n","938/938 [==============================] - 4s 4ms/step - loss: 0.0930 - val_loss: 0.0920\n"]},{"output_type":"execute_result","data":{"text/plain":[""]},"metadata":{},"execution_count":7}]},{"cell_type":"markdown","metadata":{"id":"q5GLHnaV4agO"},"source":["Παρατηρούμε ότι δεν παρουσιάστηκαν φαινόμενα υπερπροσαρμογής στην εκπαίδευση. Το σφάλμα εκπαίδευσης ξεκίνησε από το $0.19$, έπεσε γρήγορα μετά τις πρώτες εποχές και τέλος σταθεροποιήθηκε γύρω στο $0.09$. Παρόμοια συμπεριφορά εμφάνισε και το σφάλμα επαλήθευσης.\n","\n","\n","Για να μπορέσουμε να εκτιμήσουμε την είσοδο και την έξοδο του αυτοκωδικοποητή που ορίσαμε προηγουμένως, κατασκευάζουμε τα αντίστοιχα δίκτυα. Προσέξτε ότι στον ορισμό τους δεν χρησιμοποιούμε την κλάση *Dense* όπως πριν, αλλά αναφερόμαστε απευθείας στα επίπεδα του αυτοκωδικοποιητή. Ο λόγος που το κάνουμε αυτό είναι γιατί θέλουμε να αντιγράψουμε τις τιμές των βαρών ($\\mathbf{w}_e, \\mathbf{w}_d$) και των πολώσεων ($\\mathbf{b}_e, \\mathbf{b}_d$) που προέκυψαν από την εκπαίδευση του δικτύου."]},{"cell_type":"code","metadata":{"id":"F4jdkEKAbTeR"},"source":["encoder = Sequential([\n"," autoencoder.layers[-2]\n","])\n","\n","decoder = Sequential([\n"," autoencoder.layers[-1]\n","])\n","\n","h = encoder.predict(x_test)\n","x_test_out = decoder.predict(h)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"8j2t0o4D64__"},"source":["Μέσω της μεθόδου *compare_digits()* που ορίσαμε στην αρχή του notebook, θα εμφανίσουμε στην πρώτη σειρά τα δέκα πρώτα ψηφία του συνόλου ελέγχου και στη δεύτερη σειρά, τα ίδια ψηφία όπως προκύπτουν από την έξοδο του αυτοκωδικοποιητή"]},{"cell_type":"code","metadata":{"id":"8vHdghvOUww_","colab":{"base_uri":"https://localhost:8080/","height":248},"executionInfo":{"status":"ok","timestamp":1642513084647,"user_tz":-120,"elapsed":1452,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"e92f4ae1-3f30-4a39-d23e-cd8f15c9c3ba"},"source":["compare_digits(x_test, x_test_out)"],"execution_count":null,"outputs":[{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAABG0AAADnCAYAAACkCqtqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3defxV0/7H8RVCaVJKpYmUUkmjTL9bdEmUqUwZM14ZriHzEJlu4SJDuteQChlylaErSekaS6WZSnMaNBAi+v7+8PDxXqvv2Z3v6Zzz3d9zXs+/Ptta33N2Z5+19z7b+qxPqYKCAgcAAAAAAIB42aG4dwAAAAAAAABb46ENAAAAAABADPHQBgAAAAAAIIZ4aAMAAAAAABBDPLQBAAAAAACIIR7aAAAAAAAAxNBORelcqlQp6oMXk4KCglLpeB2OYbFaU1BQUDUdL8RxLD6MxZzAWMwBjMWcwFjMAYzFnMBYzAGMxZxQ6Fhkpg2QPYuKewcAOOcYi0BcMBaBeGAsAvFQ6FjkoQ0AAAAAAEAM8dAGAAAAAAAghnhoAwAAAAAAEEM8tAEAAAAAAIghHtoAAAAAAADEEA9tAAAAAAAAYoiHNgAAAAAAADHEQxsAAAAAAIAY2qm4dwD56dprr7W4TJkyXtsBBxxgcbdu3RK+xhNPPGHxRx995LUNGTJke3cRAAAAAIBixUwbAAAAAACAGOKhDQAAAAAAQAzx0AYAAAAAACCGWNMGWTN8+HCLo9aqUVu2bEnYdvHFF1vcsWNHr238+PEWL168ONldRDFr2LChtz1nzhyLr7zySosHDBiQtX3KZ7vttpvF/fv3t1jHnnPOTZ482eLu3bt7bYsWLcrQ3gEAABSP3Xff3eI6deok9TfhPdFVV11l8YwZMyz+8ssvvX7Tpk1LZReRQ5hpAwAAAAAAEEM8tAEAAAAAAIgh0qOQMZoO5VzyKVGaEvPf//7X4n322cfr16VLF4vr16/vtfXo0cPie++9N6n3RfFr0aKFt63pcUuXLs327uS9GjVqWHzhhRdaHKYttmrVyuLjjjvOa3vssccytHdQLVu2tHjEiBFeW7169TL2vkcddZS3PXv2bIuXLFmSsffFtuk10jnnRo4cafFll11m8cCBA71+v/32W2Z3LAdVq1bN4pdeesniDz/80Os3aNAgixcuXJjx/fpDxYoVve3/+7//s3j06NEWb968OWv7BJQExx57rMVdu3b12tq3b2/xvvvum9TrhWlPdevWtXiXXXZJ+Hc77rhjUq+P3MVMGwAAAAAAgBjioQ0AAAAAAEAMkR6FtGrdurXFJ554YsJ+M2fOtDicbrhmzRqLN27caPHOO+/s9fv4448tbt68uddWpUqVJPcYcXLggQd62z/88IPFr732WrZ3J+9UrVrV2x48eHAx7QmK6uijj7Y4aop1uoUpOD179rT4tNNOy9p+4Hd67Xv88ccT9nv00Uctfvrpp722n376Kf07lmO0aoxz/j2NpiKtXLnS61dcKVFa4c85/1yv6a3z5s3L/I6VMBUqVPC2NeW+adOmFodVTEk1izddVqFXr14Wayq4c86VKVPG4lKlSm33+4ZVUoFkMdMGAAAAAAAghnhoAwAAAAAAEEM8tAEAAAAAAIihYl3TJiwBrXmEy5cv99o2bdpk8bBhwyz+5ptvvH7k4xYvLREc5n5qzreuv7BixYqkXvuaa67xtvfff/+Efd98882kXhPFT3PCtQytc84NGTIk27uTd6644gqLTzjhBK+tbdu2RX49LSXrnHM77PDn/xuYNm2axRMmTCjya8O3005/XsI7d+5cLPsQrpVx9dVXW7zbbrt5bbpGFTJDx1+tWrUS9nvhhRcs1vsrJLbHHntYPHz4cK+tcuXKFutaQpdffnnmdyyBW265xeK9997ba7v44ost5r55az169LD47rvv9tpq165d6N+Ea998++236d8xpI2eH6+88sqMvtecOXMs1t9CSB8tua7nauf8NVa1TLtzzm3ZssXigQMHWvy///3P6xeH8yQzbQAAAAAAAGKIhzYAAAAAAAAxVKzpUf369fO269Wrl9Tf6bTO77//3mvL5rSzpUuXWhz+WyZNmpS1/YiTUaNGWaxT1Zzzj9XatWuL/Nph+djSpUsX+TUQP40aNbI4TKcIp6Aj/f75z39arNNEU3XSSScl3F60aJHFp556qtcvTLPBtnXo0MHigw8+2OLwepRJYeljTVstW7as10Z6VPqF5d1vvvnmpP5OU08LCgrSuk+5qmXLlhaHU+zVnXfemYW92VqTJk28bU0pf+2117w2rq1b03SZhx56yOIqVap4/RKNlwEDBnjbmu6dyj0vkhOmwmiqk6a4jB492uv3888/W7xhwwaLw+uU3pe+8847XtuMGTMs/uSTTyyeMmWK1++nn35K+PpIni6n4Jw/xvReM/xOJOuggw6y+Ndff/Xa5s6da/HEiRO9Nv3O/fLLLym9dzKYaQMAAAAAABBDPLQBAAAAAACIIR7aAAAAAAAAxFCxrmmjJb6dc+6AAw6wePbs2V5b48aNLY7KK27Xrp3FS5YssThRib7CaB7b6tWrLdZy1qHFixd72/m6po3S9StS1bt3b4sbNmyYsJ/mkha2jfi67rrrLA6/M4yjzHjrrbcs1pLcqdLSphs3bvTa6tata7GWnf3000+9fjvuuON270euC/O5tWzz/PnzLb7nnnuytk/HH3981t4LW2vWrJm33apVq4R99d7m7bffztg+5Ypq1ap52yeffHLCvueff77Fet+YabqOzbvvvpuwX7imTbgeJJy79tprLdYS7skK12nr1KmTxWHZcF3/JpNrYOSqqHVmmjdvbrGWeg59/PHHFuvvyoULF3r96tSpY7GuZepcetYBxNb0eUCvXr0sDsdYhQoVCv37ZcuWedsffPCBxV9//bXXpr9BdG3Ftm3bev30nNC5c2evbdq0aRZr2fB0Y6YNAAAAAABADPHQBgAAAAAAIIaKNT1q7NixkdsqLNX2h7Dc6IEHHmixTnNq06ZN0vu1adMmi7/88kuLw5QtnSqlU9OxfY477jiLtXTmzjvv7PVbtWqVxTfeeKPX9uOPP2Zo77C96tWr5223bt3aYh1vzlEaMV3+8pe/eNv77befxTq9N9mpvuH0T52erKUznXPuiCOOsDiqHPHf/vY3i5944omk9iPf3HLLLd62ThHXqfhhilq66bUv/G4xXTy7olJ2QmEaAaI98MAD3vaZZ55psd5fOufcyy+/nJV9Ch1++OEW77nnnl7bs88+a/HQoUOztUslhqbuOufceeedV2i/L774wtteuXKlxR07dkz4+hUrVrRYU6+cc27YsGEWf/PNN9ve2TwX3v8///zzFms6lHN+enBUyqAKU6JUuPwF0u/JJ5/0tjWtLap8tz43mD59usU33XST109/14cOOeQQi/U+9Omnn/b66fMFPQc459xjjz1m8auvvmpxulNlmWkDAAAAAAAQQzy0AQAAAAAAiKFiTY9Kh3Xr1nnb48aNK7RfVOpVFJ16HKZi6VSs4cOHp/T62Jqmy4RTIpV+5uPHj8/oPiF9wnQKlc2qG7lO09BefPFFry1quqnSal465fOOO+7w+kWlI+prXHTRRRZXrVrV69evXz+Ld911V6/t0UcftXjz5s3b2u2c0q1bN4vDigXz5s2zOJuV1jTNLUyHev/99y1ev359tnYpb/3f//1fwrawKk1UeiK2VlBQ4G3rd3358uVeWyYrAJUpU8bb1qn/l156qcXh/vbs2TNj+5QLNN3BOefKly9vsVabCe9Z9Pp0+umnWxymZNSvX9/i6tWre22vv/66xcccc4zFa9euTWrf80G5cuUsDpdA0GUU1qxZ47Xdf//9FrNUQnyE93VatemCCy7w2kqVKmWx/i4IU+f79+9vcarLKVSpUsVirWLap08fr58u0xKmVmYLM20AAAAAAABiiIc2AAAAAAAAMcRDGwAAAAAAgBgq8WvaZEK1atUsfvzxxy3eYQf/GZeWoyYPNXX/+c9/vO2jjjqq0H7PPfectx2Wv0XJ0KxZs4Rtuq4Jts9OO/15ek92DZtwbajTTjvN4jBvPFm6ps29995r8YMPPuj1K1u2rMXh92DkyJEWz58/P6X9KKm6d+9usX5GzvnXp0zTNZJ69Ohh8W+//eb1u+uuuyzOt/WHskVLlGocCnP8p06dmrF9yjfHHnust63l1HUtp3ANhmTpOirt27f32tq1a1fo37zyyispvVe+2mWXXbxtXRPon//8Z8K/0/LBzzzzjMV6rnbOuX322Sfha+haK5lcD6kkO+GEEyy+4YYbvDYtw61l751zbsOGDZndMaQkPI/17t3bYl3Dxjnnli1bZrGuLfvpp5+m9N66Vk3t2rW9Nv1t+dZbb1kcrmOrwv0dMmSIxZlcy4+ZNgAAAAAAADHEQxsAAAAAAIAYIj2qEL169bJYy9KG5cXnzp2btX3KNTVq1LA4nN6tU1Y1JUOn3Tvn3MaNGzO0d0g3nc593nnneW1TpkyxeMyYMVnbJ/xOS0WHJWJTTYlKRNOcNMXGOefatGmT1vcqqSpWrOhtJ0qFcC711ItUaLl2TbebPXu212/cuHFZ26d8lexYyeb3Ixc9/PDD3naHDh0srlmzptempdd16nzXrl1Tem99jbCUt1qwYIHFYclpRNNy3SFNfwtT+BNp3bp10u/98ccfW8y9bOGiUj/1vnHp0qXZ2B1sJ01Rcm7r1Gr166+/WnzQQQdZ3K1bN69fo0aNCv37n376ydtu3LhxobFz/n3unnvumXCf1MqVK73tbKWFM9MGAAAAAAAghnhoAwAAAAAAEEOkRznnDj30UG87XKX8D7qSuXPOzZgxI2P7lOteffVVi6tUqZKw39ChQy3Ot6oxuaRjx44WV65c2WsbPXq0xVqVAekTVr5TOvU003TKf7hPUfvYp08fi88666y071echBVN9tprL4tfeOGFbO+OqV+/fqH/netg9kWlYaSjchF+N3nyZG/7gAMOsPjAAw/02jp16mSxVkVZvXq112/w4MFJvbdWI5k2bVrCfh9++KHF3CMVTXg+1VQ2TUEMUzC0AuaJJ55ocVhtRsdi2HbhhRdarMd61qxZSe17PghTYZSOt9tvv91re/311y2mYl58vPfee962plLrbwTnnKtTp47FjzzyiMVRqaKabhWmYkVJlBK1ZcsWb/u1116z+IorrvDaVqxYkfT7bQ9m2gAAAAAAAMQQD20AAAAAAABiiIc2AAAAAAAAMcSaNs65zp07e9ulS5e2eOzYsRZ/9NFHWdunXKT5wi1btkzY7/3337c4zFVFydS8eXOLw5zUV155Jdu7kxcuueQSi8Pc3OLSpUsXi1u0aOG16T6G+6tr2uS677//3tvWnHxdU8M5f32otWvXpnU/qlWr5m0nWl9g4sSJaX1fFO6www6z+IwzzkjYb8OGDRZTCje91q1bZ3FY2l63r7/++u1+r3322cdiXQvMOf+ccO211273e+Wrd99919vWsaPr1oTrzCRaVyN8vV69eln8xhtveG0NGjSwWNfH0Ot2vqtatarF4T2Brv122223eW233HKLxQMHDrRYy6w756+bMm/ePItnzpyZcJ+aNGnibevvQs630cIy3LoeVKVKlbw2XVtW15399ttvvX6LFy+2WL8T+pvDOefatm1b5P0dNGiQt33TTTdZrOtVZRMzbQAAAAAAAGKIhzYAAAAAAAAxlLfpUWXKlLFYS8c559wvv/xisabnbN68OfM7lkPCUt46tUxT0EI69Xfjxo3p3zFkRfXq1S0+/PDDLZ47d67XT8voIX00FSmbdEqzc87tv//+Fus5IEpYJjefzr3hFGIt43vyySd7bW+++abFDz74YJHfq2nTpt62pmTUq1fPa0uUEhCX1Ltcp9fTHXZI/P/bxowZk43dQYZpykc49jT9KjxXInlhSukpp5xisaZtV6xYMeFrDBgwwOIwLW7Tpk0WjxgxwmvT9I+jjz7a4vr163v98rmM+/3332/x1VdfnfTf6fnx0ksvLTROFx1/urTDaaedlvb3ymVhupGOj1Q899xz3nZUepSmpOv37Nlnn/X6aUnx4sJMGwAAAAAAgBjioQ0AAAAAAEAM8dAGAAAAAAAghvJ2TZvevXtbHJaeHT16tMUffvhh1vYp11xzzTXedps2bQrt95///Mfbpsx3bjj33HMt1vLBb7/9djHsDbLl5ptv9ra17GmUhQsXWnzOOed4bVrWMd/o+TAs/Xvsscda/MILLxT5tdesWeNt69oZe+yxR1KvEeZ9IzMSlVwP1wJ48skns7E7SLPu3bt722effbbFuuaCc1uXvUV6aMluHW9nnHGG10/HnK49pGvYhPr27ettN27c2OKuXbsW+nrObX0tzCe6rsnw4cO9tueff97inXbyf8rWrl3b4qj1v9JB1/DT74yWHXfOubvuuiuj+wHnrrvuOouLsqbQJZdcYnEq91HZxEwbAAAAAACAGOKhDQAAAAAAQAzlTXqUTiN3zrlbb73V4u+++85ru/POO7OyT7ku2RJ9l112mbdNme/cULdu3UL/+7p167K8J8i0t956y+L99tsvpdeYNWuWxRMnTtzufcoVc+bMsVhL0jrn3IEHHmjxvvvuW+TX1rK2ocGDB3vbPXr0KLRfWKIc6VGrVi1vO0zR+MPSpUu97UmTJmVsn5A5xxxzTMK2N954w9v+/PPPM707eU9TpTROVXie1HQfTY/q0KGD169y5coWhyXKc52WWA7Paw0bNkz4d0ceeaTFpUuXtrhPnz5ev0RLNqRK05dbtWqV1tdG4S644AKLNSUtTJlTM2fO9LZHjBiR/h3LEGbaAAAAAAAAxBAPbQAAAAAAAGIop9OjqlSpYvEjjzzite24444W69R+55z7+OOPM7tj8Oj0T+ec27x5c5FfY8OGDQlfQ6dHVqxYMeFrVKpUydtONr1Lp3Bef/31XtuPP/6Y1GvkouOOO67Q/z5q1Kgs70l+0qm6URUUoqblDxo0yOKaNWsm7Kevv2XLlmR30dOlS5eU/i6fTZ06tdA4HRYsWJBUv6ZNm3rbM2bMSOt+5KtDDjnE2040hsPqiyiZwvPwDz/8YPEDDzyQ7d1Bhr300ksWa3rUqaee6vXT5QNYuiE5Y8eOLfS/azqxc3561K+//mrxM8884/X717/+ZfHf//53ry1R2ioyo23btt62nhvLlSuX8O902Q2tFuWccz///HOa9i7zmGkDAAAAAAAQQzy0AQAAAAAAiCEe2gAAAAAAAMRQzq1po2vVjB492uK9997b6zd//nyLtfw3su+LL77Y7td4+eWXve0VK1ZYvOeee1oc5gunW6NGjbztv/71rxl9vzg57LDDvO3q1asX057AOeeeeOIJi/v165ewn5aTjVqPJtm1apLtN3DgwKT6oXjomkiFbf+BNWwyQ9fkC61Zs8bihx9+OBu7gwzQtRX0PsU551atWmUxJb5zj14n9fp8/PHHe/1uv/12i1988UWv7csvv8zQ3uWmd955x9u+++67LdYS0RdeeKHXb99997W4ffv2Sb3X0qVLU9hDbEu49mH58uUL7adrgjnnrxv1v//9L/07liXMtAEAAAAAAIghHtoAAAAAAADEUM6lR9WvX9/iVq1aJeyn5Zw1VQrpE5ZSD6d9plP37t1T+jst8xeV1jFy5EiLJ02a5LVp2sCHH36Y0n7kghNPPNHb1lTFKVOmWDxhwoSs7VM+GzFihMW9e/f22qpWrZqx9129erW3PXv2bIsvuugiizWFEfFTUFAQuY3MOvrooxO2LV682OINGzZkY3eQAZoeFY6vN998M+HfaUrA7rvvbrF+L1ByTJ061eLbbrvNa+vfv7/F99xzj9d21llnWfzTTz9laO9yh96LOOeXXT/llFMS/l2HDh0Stv32228W65i94YYbUtlFFELPd9ddd11SfzNs2DBv+/3330/nLhUbZtoAAAAAAADEEA9tAAAAAAAAYoiHNgAAAAAAADFU4te0qVu3rrcdlnT7Q7img5a5RWacdNJJ3rbmIpYuXTqp12jSpInFRSnX/fTTT1u8cOHChP1effVVi+fMmZP06+N3ZcuWtbhz584J+73yyisWaw4wMmfRokUWn3baaV7bCSecYPGVV16Z1vfVMprOOffYY4+l9fWRHbvuumvCNtZPyAy9Lur6fKFNmzZZvHnz5ozuE4qHXid79OjhtV111VUWz5w50+Jzzjkn8zuGjHruuee87Ysvvtji8J76zjvvtPiLL77I7I7lgPC69fe//93icuXKWdy6dWuvX7Vq1SwOf08MGTLE4j59+qRhL+GcfzxmzZplcdRvRx0DemxzCTNtAAAAAAAAYoiHNgAAAAAAADFU4tOjtISsc87VqVOn0H7jx4/3tilfmn39+vXbrr8/44wz0rQnSBedmr9u3TqvTcukP/zww1nbJ2wtLLOu25pSGp5Pu3TpYrEez0GDBnn9tOy9TmVFyXXeeed52+vXr7e4b9++2d6dvLBlyxaLJ02a5LU1bdrU4nnz5mVtn1A8LrjgAovPP/98r+2pp56ymLGYW1avXu1td+zY0eIwNef666+3OEyhw7atXLnSYr3X0VLqzjnXrl07i++44w6vbdWqVRnau/x2xBFHWFyrVi2Lo367a9qophDnEmbaAAAAAAAAxBAPbQAAAAAAAGKoVFHShEqVKhWLnKLDDjvM4rfeestr0xWnVdu2bb3tcOpx3BUUFJTadq9ti8sxzFOTCwoKWm+727ZxHIsPYzEnMBa3YdSoUd72gw8+aPG4ceOyvTuFyuWxWLNmTW/7rrvusnjy5MkW50B1trwdi3ovq5WAnPNTWJ944gmvTVORf/nllwztXdHk8liMizFjxnjbhx56qMVa8Wg7UpTzdizmklwYi9OmTbO4WbNmCfv179/fYk0XzAGFjkVm2gAAAAAAAMQQD20AAAAAAABiiIc2AAAAAAAAMVQiS34ffvjhFidaw8Y55+bPn2/xxo0bM7pPAADkCi2Biuxbvny5t92zZ89i2hNkysSJEy3WErdAYU4++WRve/r06RY3aNDA4u1Y0waIhcqVK1tcqtSfS/SEJdYfeuihrO1THDDTBgAAAAAAIIZ4aAMAAAAAABBDJTI9KoqWCTvyyCMtXrt2bXHsDgAAAACk7LvvvvO269atW0x7AmTWgw8+WGjct29fr9+KFSuytk9xwEwbAAAAAACAGOKhDQAAAAAAQAzx0AYAAAAAACCGShUUFCTfuVSp5DsjrQoKCkptu9e2cQyL1eSCgoLW6XghjmPxYSzmBMZiDmAs5gTGYg5gLOYExmIOYCzmhELHIjNtAAAAAAAAYoiHNgAAAAAAADFU1JLfa5xzizKxI4iUzrp+HMPiw3Es+TiGuYHjWPJxDHMDx7Hk4xjmBo5jyccxzA2FHscirWkDAAAAAACA7CA9CgAAAAAAIIZ4aAMAAAAAABBDPLQBAAAAAACIIR7aAAAAAAAAxBAPbQAAAAAAAGKIhzYAAAAAAAAxxEMbAAAAAACAGOKhDQAAAAAAQAzx0AYAAAAAACCGeGgDAAAAAAAQQzy0AQAAAAAAiCEe2gAAAAAAAMQQD20AAAAAAABiiIc2AAAAAAAAMcRDGwAAAAAAgBjioQ0AAAAAAEAM8dAGAAAAAAAghnhoAwAAAAAAEEM8tAEAAAAAAIghHtoAAAAAAADEEA9tAAAAAAAAYoiHNgAAAAAAADG0U1E6lypVqiBTO4JoBQUFpdLxOhzDYrWmoKCgajpeiONYfBiLOYGxmAMYizmBsZgDGIs5gbGYAxiLOaHQschMGyB7FhX3DgBwzjEWgbhgLALxwFgE4qHQschDGwAAAAAAgBjioQ0AAAAAAEAM8dAGAAAAAAAghnhoAwAAAAAAEENFqh4FxEmpUn8ukL7DDv7zx99++y3bu5P39Hg451xBAQvPAwAAAMD2YKYNAAAAAABADPHQBgAAAAAAIIZIj0JaaYrMzjvv7LU1aNDA4nPOOcfiww47zOu34447Wrxx40aLS5cu7fVbvXq1xZMnT/bahg0bZvGiRX+Wu49K2QnTexSpPtuWic9I09403rJlS8L35lhlxk47/Xm5CMfKr7/+ajGfPwAgH+i1kGsfEtHvSXj/FN7PAokw0wYAAAAAACCGeGgDAAAAAAAQQzy0AQAAAAAAiCHWtEFa7brrrhaHa9XceeedFjdt2tTismXLev0SrS0T5n3+/PPPFtevX99rGzt2rMVLliyxOKoUOPnImRNVnr1y5coWt2zZ0murU6eOxVOmTLF43rx5Xr8ffvjB4vAYky8cTY9NuXLlvDY9Hh07drS4evXqXj9dU2rcuHFe21dffWUxxyK9Eo0rXRfMOf/cpuMjHccjai0wcvezT78Hej12zrnddtvNYl0vTq+lznGcUhG1ZoXisy35Eh3f8L+n+54yvHdS+fa90mtc1HUm05+LrrVZoUIFixs3buz122effSwO71H198rKlSst5jcJQsy0AQAAAAAAiCEe2gAAAAAAAMQQ6VFIqzJlylh88skne201a9Ys9G9+/PFHb/u7776zeNOmTQnfS0sQz5w502vTlKhMT4+kVPi2RZXk3mWXXSxu3ry517bXXntZvGbNGovnz5/v9dNjzGdeNPr9LV++vNemx+PYY4+1WFPawn6LFi3y2sJUNqROz3nO+WNHU9vCNLcNGzZYrOfXzZs3e/2ixo5+T3Rq+s477+z1020tBe+cn4ZDmfjM0HTj7t27e20HHXSQxePHj7d41KhRXj9NN+XY/EnHQJUqVbw2vVbpd1vHm3P+dUzHQ/g5p/K5h/ciej9WqVIlr03vu3Qf8y3NJhnh55roni8TYyUq7S7ZpQRyQfhv1VQx/Z4751/XotKBtU1fP0wr1aUeLr/8cq/twAMPtFjHWHitVmE6qqaX6+trarlzW1+vkX+YaQMAAAAAABBDPLQBAAAAAACIoWJNj4qa6heukp4o/SGbU3ej0mBC+TKlOJwC2KhRI4urVq3qtS1evNjiESNGWPzvf/87YT/9zFu0aOH1u+222ywO0xmVmB4AACAASURBVDp0RfdMHwtNFYiqToXC6XcoPI5a4WT27NmF/nfn+Ny3h46xBg0aeG29e/e2eM8997Q4PD9rqkDXrl29tokTJ1r8/fffb9/O5qGo62LFihUtPuCAAywO06NmzZplcTh2kqXnUd2nMO1i9913t1jTbJxzbvXq1RbnayW/dKfTht8Jna5/6623em16rDSlYPTo0du9H7ko/Gzr1atncd++fb02Tf/+8ssvLX7xxRe9fuvXr7dY0x2iPvMwrSPRWAzTRNq2bWtx+/btvba5c+da/M4771i8du3ahO+VT6LSZapVq2ax3mt+++23Xj9NO0vHPUpUilCUXEiXCr+HmoIYLrGgEqX1Oucfx/POO8/iK6+80uun9zdF+R2YiH5nnPPH6e23327xjTfe6PXT1HPueYsm2TTDqOcLcTgXMtMGAAAAAAAghnhoAwAAAAAAEEM8tAEAAAAAAIihrKxpo/limstXu3Ztr5+umRC2/fLLLxZrTvCKFSu8fppDqvlnYe5nVH6b5j3qa+y2225eP80p1Fz9cD/CsqclnX5euo6Cc8799a9/tXjGjBle25gxYyz+9NNPLdZjG0XLeDvnXJ06dSzW745zzl166aUW33DDDRbnQm5vrtF1Fho2bOi1TZ8+3eJly5ZZTOnD9NG87kGDBnltuk5DVP68rqVw2mmneW0VKlSwWHPFw/z/OOQLx114DTryyCMt7tSpk8Xhufezzz6zONl1NEKJcvnDdaj0mvDNN994bXpd3LRpk8W5fl6Out9IJNljE47LU0891eK6desm/DtdpyNcewi/0zLezjl3xx13WKxjzzn/+zxp0iSLw7GYqJx6suWco4RrWem5uEmTJl6b3ue+++67he5TvtHPXNdkPOWUU7x+3bt3t1ivfRMmTPD6Pf300xbPmzfPa0v2vjdKot9W6XjtuNPvaXj90O/2zjvvbLGut+acc126dLFYS23vscceXr9Ea54454/75cuXF/q+zvnXyXBs629aXQ8r/H2b69fJRKLW9dPPuXLlyhaHa6Cee+65Fh9++OFem35fdP0/XXvVOedGjhxpcXhstIx7Jo8TM20AAAAAAABiiIc2AAAAAAAAMZSV9CidzqQlSjW9xTnnOnbsaHG7du28Np0C9dNPP1kcTlXTVAtNmQmnlev0JZ2y7Zyf6qTvVbZsWa+fToEdPHiw1zZz5kyLNY2qpE491elpOs0vnAY2fvx4i7V0t3POLVy4MOHfJePMM8/0tnXqdzhlrlmzZhano0RflFxLfwulezp/WO7w0EMPtThMi3z77bctjirriKLR6aC33HKLxfvuu6/XL1FKVFQpxPBcq1OQdcr5E0884fXTssN63s13Ov722Wcfr03Lq2tqhE6xds65lStXWpyOqbv6/dFj6px/3V23bp3XptfCfJrqHZUGs73C8aYpyuH41c/8rbfesph00z/pZxamXWvJ7/Cz1RLajzzyiMVr1qzx+iV7nYwqPav0+9SoUSOvrXnz5haH191Vq1ZZvHHjxqT2KdeEY1F/P9x9990Wn3HGGV4/TS3U+7/wXKipx8OHD/faxo0bZ3HUb5qoNCCVr+dW57Yu5a2/F/UYawqLc/5vFP3dF55TNd3sv//9r9d2zz33WPz1119brGlzzvnfLU3jCd9v2rRpCfe3pP5+TEY4Fnfa6c/HE/p5HXHEEV4/vb9s06aNxWGKm/5+D78v+rnqObNx48Zev549e1qsv3Wdc27AgAEWL1iwwOKwNPv2HkNm2gAAAAAAAMQQD20AAAAAAABiiIc2AAAAAAAAMZSVNW00v1LXpQjz9bRfWA5W+0aV09ac41122cViXUvHOb/E2uTJk702ze/t0KGDxVWqVPH6aY7chx9+6LV98cUXLpdoHt73339vcbh2goo6vsnSXMarrrrKa9Ocxw0bNnhtN99883a9LxJLZY0b/Zsw1/SEE06wWI+pc8599dVXFqd6HBOV243KHc91mrd7/vnnWxxV1ltzc8M1MPQ8HL6Gfua6Vlm4/sIrr7xi8T/+8Q+vTddkyTd6nenWrZvX1rBhQ4t1TQ0t8e2cn5Of6voqelx17Qa9Rjrnl4mfP3++1xa1dgOiz0+J+jVo0MBr0zLV4Wvofc/YsWOTeq8oye5vSRWuaaNrT2ipX+f8cs9a6j7Tn4veI/3tb3/z2nRNFS1l65xzb775psX5UCK6MOG6I7169bK4R48eCfvptVB/q4THWsvCh+tj6Gf+/vvvWxy1vlR4D5TP97Z6PQrXoNH7Ef0dEq6LqKXuTzzxRIvDktD6ey78bZfoeP3www/etq5tFXWflexaVrlAP4dwnR9dm+28886zOFzXT9cv0vEQrlujx0PXV3XOX4NGr6d7772310/vWevXr++1LVu2zOInn3zSYr3mOuf/m1MZv8y0AQAAAAAAiCEe2gAAAAAAAMRQVtKjdIqXTo+ePXu210/TnsJ0F53apNNSw3QKnWJVq1Yti8NprpoSFZZkrF69usUtW7a0WKd9O+enXIVToHKhzHci+u8JpxumY7q0Th/Tsoua7uac/5mfc845XtvUqVO3ez9QuETHONnp/C1atPDa9ttvP4uXLl3qtYXbqUiUApJP34tw7Lz66qsJ25ROMx4zZozF8+bN8/rVqFHDYi2H6pw/tVzPp2G6qZZVDUtF33fffRbnenni8Pt6wAEHWKyfX0hLOIfpZIm+61HpUeHf6HlZp/2H5XDXrl1r8TPPPOO1henM+Sjqc032nKR/c/zxx3ttOv7C6ddaZlhLPhdFsummuUCn3jvnXPny5S0O0781JSqlae9BykTU56nlg5977jmLW7du7fXTKftahtw5/9oalqXNZfo563IKzvnnMu0XXo/efvtti3VphHAsajpwmE7Rtm3bQl8j169v6aJpoDounXNuyZIlFus1JxxTmqKmKTPh9VN/t6bjGpavaW3hOU5/o4f3EZqqqOlv4bOBDz74wOKRI0daHN7bTJkyxeIwPUr7aoqp/v50zn/eEP5b9t13X4uj0k0p+Q0AAAAAAJCDeGgDAAAAAAAQQ1lJj1KJqhA551ciCqegJTudTNOoli9fnvDv9fXDaVSrV6+2WCt3hKlYOo1Rp+MVZX9zTTqmSOsK3VqZRKt6Oefc0KFDLX7nnXe8tnz9/LMhldXtS5cubfFRRx3ltWmaoaa1ORddmSGRqOpF+fS90M/huuuu89rq1KlT6N+E0zovueQSi0eMGGFxWK1BV/TXqcTO+av4a5rTIYcc4vXTdKmePXt6bQMGDLA4TEXNNWG6mo6XMEVXq6tpelR4HBONnaJUUNPrn1ZWCVMM9LoYTjPPp/GXrFRSTMuVK2dxly5dvH56nMIUnn//+98WJ5uGEd4fFaXiWEmk/76wAommnum9oXP++ezll1+2OLxv0dfX62J4f6nfhXCMDRw40GJNswkr1jz66KMWa3Ur5/K3YpR+5mG6qaaJaUWna6+91uunVfE0Va1Tp05eP00NDlPQNPWOtNFtCysA33XXXRaH6S4PPPCAxVHn1ETn3rAyHNet9AirsOl9nqZDOecfb106RX/3OefcQw89ZLE+UwiPe1SanN4rN2vWzOIw1T9RarBzfgplJpdHYaYNAAAAAABADPHQBgAAAAAAIIZ4aAMAAAAAABBDxbqmTZjjmY68Tn19fb2onMQw56xq1aoWV6tWLeFraKmxr7/+OvI14dN8wLAcu657oXnjkyZN8vrdf//9FkflZ0flIaayPku+STYnOKSfteZ9aw6+c37Z+BdeeMFrC9dHSea9okqn5tMx1jKYWsbQOf/z0vPkoEGDvH5DhgyxWM/X4doJunZJeJ7UtRp0XZyDDz7Y66frR+j3xTnnqlevbnEurmmjx0OvOc45V7duXYvD85yunaGltpP9nhdlPOiaQ1qGPBxvM2bMsFjXh8Pv0nE+1bz7sJSwvoaWdXbOuY8++iip90r0vuF2rq/1EN6j6vVIx4Nz/tpTN954o8Xheht6vtX1bsIS7Hofeu+993pte++9t8V6HD/55BOv37BhwywO1zfKV3q+0rWhnHNu9OjRFj/55JMWh/f3+r3XtWnCa5pe+8Jzt5aIz/V1olKlx+qee+7x2o455hiLp0+f7rX169evyO+lYz08ryV7Xs6n+8tk6eezxx57eG2tWrWyOFyzSP9O7/kmT57s9dNxpetVhWNKj2l4nBo0aGBxt27dEr5Govd1zv99msk1qphpAwAAAAAAEEM8tAEAAAAAAIihrKdHqXRMJYuavpRqiWCdHqXTG8Ppq1o+M9k0jnym6Q+1atWy+JFHHvH6tW7d2mKdFjd48GCvn6ZkhMdavxdati2c9siU4aJLZVq9Tj8MyxYvWrTIYk05dG7r6emJ6BgOzwnJvkau0anaYalFHQcLFiyw+KabbvL6JZrmGX6mUd8JLS08ceLEQvchFJa91nSpXJyOHJUeVblyZYtnzZrltU2ZMsXidH/Pw3Gk52VNWw2vfS+++KLFYelUbC3Z77Dei5x66qkWh6Wn9XswfPhwr01LokaJSjfNZDnTONB/U5h2MXfuXIsrVKjgtWk66plnnpnw9TUlatq0aRavWLHC66dpxJoi6Zx/fPT1evfu7fVj/G19HtNxFKY96TGIur/UlKi+fftaHKbMqTClWLd32203i8Nrbj6XA9d7xdNPP91r0/Gm10jn/GOcilw8rxUXHX/hfWjt2rUtjvrM9RqkKVXOObfXXntZrL/dw+vi2LFjLa5Ro4bX1qlTJ4t1LIb02vf55597be+++67FmUwbZqYNAAAAAABADPHQBgAAAAAAIIaKNT0qVTrdSlNunIteITrRa4TpGl26dLF43bp1Fg8dOtTrp1PTc72KQjroNLbnn3/eYq1E4pz/Wb7yyisWv/XWW16/qHSARBUuNFUD6RVOQ9aV3DW1Ihyzn332mcVr1qzZ7v0oSupOLgmnBHft2tVinc7tnHMbNmywuFevXhbrVPsoqX6menzDqfuaEhWuzB817TwX6PTfsHKWfi6a6ulcZtMfwqnMPXv2tFin7H/66adevwkTJljMdTF14flUK920a9cu4d9FVeNL9nhEVYjK9fOp/vu0wo9zzj3++OMWH3/88V6bVnTSa5xeB53z026effZZi8PrVsuWLRPuo44/rfCn6VsonJ5rwxQKTY1o3LhxoX/jnHNnn322xSeccELC99KxGH6X9J5Ix9v48eO9fosXL7Y4386nWvUyqrpQWEFPf2t89913Fqd67opahiNRW66fJ9NBf1+H9zJ6jtNUwo4dO3r96tSpY7HeJ4bn3TZt2iS1TzrGwnRirX4aLiWgS3mQHgUAAAAAAJBneGgDAAAAAAAQQzy0AQAAAAAAiKESs6ZNonVswrzBRLlkYd6h5q7eeOONXpvmJk+dOtXisOQ05RSjhWuX6Oes5SzDfOHly5dbPGjQIIuTXW8jpOtjkGeaPTrG2rdvb3E4Fr/44guLk11zKHyNZNeyymXhujWa06tr2Djn3Pz58y3Wzz+Vcu7b+rtkSxXra4TnVl0HIhdLfuu5MlzPR/O5w/Lauu7C2rVrC/0b5/zPKWq9DT0m5557rtfWoUOHQvddS2k6l3xZaRRN9erVLdZ7lJCOlSVLlqT0Xvp9yZUxlgpdk8Q559577z2Lde0m5/z1pvScF65Vo2P4559/LvRvnHPutddes7hFixZem45vvUeKWucvX4XfX73HCNcIO+eccyxu0qSJxbqelHP+tVbP1+GaQvr7ITzX6tqZJ554osXh+fSKK66wWNdnyVX6OZ1xxhkWh/ccuh1+trfffrvFL730ksXh/WWjRo0KbQvXadPS8OFab7vvvrvFX331lcXhscrX86j+u3V9Juecu/nmmy0+6KCDvDb9/bB06VKLw9+BRx55pMXdu3e3WI+Lc9HrEuk+6jUzXLfm1VdftTi8T8sWZtoAAAAAAADEEA9tAAAAAAAAYii26VHhVCZNodGpTMmW1gpTB3Q64lFHHeW16bQ2LcmoU+TC/cDWdDq3c/7UNZ2ir1OEnXPu8ssvt3j27NkWF+XzTnfJtaipdaqkfieSnToYJUxzq127tsU6vTt8r+nTpxf5vbC1cHp9rVq1ErbpOS7ZNE8ds+Gx1vGmJaqdc+6kk06y+LbbbkvYT4VTTzWlIBe/I/r5aXqoc34qm6ZDOeefU+vVq2fxqlWrvH469bthw4YWa1lN5/zvycUXX+y16XRlPR6aXudc/pWl/UO6rxFherFOAw/L36qJEydaHKb3JJLsvueDqNQwvVcJ71vClMTtpcckTAl4++23LdaUjKjvVlR6ST6lw2kKWXhd1NLRVatWtTi83mmp4uHDh1v81FNPef00ZVVTg51z7vDDD7e4QoUKFodpIlrSeObMmV5bLh4r/Sz0d1t4XdHvb3gdO/jggy3WEtGVKlXy+iX6XRmOZf3cw7QbTZfS70KfPn28fuk+P5QU+rmG16NPPvnE4smTJ3ttOk6jlj8YOXKkxcuWLbM4TG3SVMjwu6S/7U8++WSLw/EWh/RTZtoAAAAAAADEEA9tAAAAAAAAYoiHNgAAAAAAADEUqzVtNEcxzCHVbc1fTDaHd6+99vLatIxeWHpWyzq+8847FidbjjifaR6+fsbO+fmketzefPNNr9+oUaMsLs4cQv236Hck/G5qrmq4v7mYc5xI+Lk0a9bMYl0PY9GiRV4/zSeNWg9Dx3NRSk7ni7B8qZ7zwjKVmiev5TKjPleNw2Ndvnx5i7t27eq1PfLII4X2C+k6NkOGDPHakl23oaTS7/3q1au9tvHjx1v87bffem2tWrWyuFOnThaHx0fLcGv5988//9zrp+sJ6Jh1LvG6J+F6G7l4fLJFP+NwPOvaUHp8w3UCXnjhBYvDtR4SvVeIY5h91apV87bPOussi9evX++19e/f3+JU1/vS71Ac1mrIFj3XhutojBs3zmJdP0zX3XPOuX79+lmsa5CFn6NeWz/66COvTe979Hr85Zdfev3CY5/rdJ2hNWvWWBxe03S9IP2d5pxzzZs3t1jXCApfQ0WVEN9///0tDu+l9HeCrgM3ZswYr59uc379nY6XVM9Bet/47rvvWhz+/tTvla59E/bVdWzieF5kpg0AAAAAAEAM8dAGAAAAAAAghmKVHhUlquRXIjpt7eyzz/baNHUgLLF6//33W6ylcbFte++9t8Wnn36616bHQ8sM33fffV6/RFPSwunc+nphWk2yZWd1umRYDvCwww6z+IADDrD4s88+8/p9/PHHFmsagnP5NQ0yLFF76KGHWqzTTcOUDC2fGSUqPUrl02euwmm7ml6hpTOdc65GjRoWa2l2nXIc0s9c02icc+64446z+J577vHaypUrV+jrhcdJx9Hjjz/utYUlwHONnvPC0qATJkyweMqUKV6bpk5Vr17d4jBlRssT67T8MN1Ky5l269bNa9PX1/Nm5cqVHfzvczpKaIdjbL/99iv09ZcuXer1+/TTTwvdpyicT/+UqBR2Juh18cYbb/Ta6tata/HLL7/stc2bN8/iZPcx7Jdvx/UPeq7Vz9E556688kqLNbUwvP4kGuvhPZAuqTB16lSv7eqrr7ZYy3/PnTvX66fpp/mQFq6/uR566CGL9TeDc8598MEHFof3LXpP89hjj1msn7Nz/vjT460pxM755+KwTLzSlOIOHTp4bZq6k4vHrbjo0hVaZj1Mxdfvz9ChQ702/S4l+9uxuDDTBgAAAAAAIIZ4aAMAAAAAABBDsUqP0ilj4RSlVKb5NmnSxOIzzzzT66fTGF966SWvbfbs2Qn3A75wOugll1xisaZghPQ4tW7d2msLp6z+oVGjRt72HnvsYfGcOXO8tlWrVhX6XuE+acWVcLVxneqoFRp06qVz/lRKrXLjnJ/6E+fvUqrTNfWz1ePhXOJqNp988onXL6rCSTLvu622fJmKGqZyRlV7qlixosVa3emiiy7y+un3vkWLFhZfeumlXr927dpZHKZDJUo3WLx4sddPq6WEKav5JBwPOj0+THuKqq6h9NyjU/bDsaFt4Tm1TZs2Fut5v2rVql6/bKaXxFU6zqeakuucn7qmKR5h5ZQwRRepK8p1JlHf8Lqv/WrWrGlx586dvX6ahqGpFc6lds0M5evY1OMRpj0lm4arx1CPU/gd0OMUVtmbNGmSxVpRM0y/0bEenuMTHcOSfGw1jeXFF1+0OPw36bEK2+bPn2+x3tf/61//8vppVSi9Vw9fT8+9UfTvwrTVVOTrvWxIPwe9d3XOr8rVsmXLhK+h90qvvfaa15ZqBb7iwEwbAAAAAACAGOKhDQAAAAAAQAzx0AYAAAAAACCGYrWmjUo1r0zXUxg4cKDFe+65p9dvwYIFFj///PNeW66Xl02nMMdW14yJygfXUnv/+Mc/vLZbb73VYi1jHOb6ahnbFStWeG0LFy60WNfCCb8HUa+v34P169cXGjvnryEQ5i3HeR2bdNBjrGWfnXNur732slhzu7UkrXPp+YzinoeaDWGZSj2v3XDDDV6blgA/5JBDLA7L2SstIR6O+6ixruuk6Ljs0qWL12/JkiUWczz/pJ+Frm/gXOKxk+rnp68flljV99LXD/P99bsR7i+i6Wd3zDHHeG16zdQx9cwzz3j9UjmfMt7+FFW6Xbej1ptIFId/d/DBB1scrgmnf6frMWRCLq9DlYl1QXSc6vpe4W+HqPNfovtLLS/tnH+PGtJ7YL3HKsnHUPdd1xopCv0stIT6Nddc4/W77777LNbfBlHr8oV0f3XNucmTJyfslyzWtPmdjgH9Xe+cc61atbJYP69w3a+nnnrK4hkzZnhtJek+hZk2AAAAAAAAMcRDGwAAAAAAgBiKbXpUssKUlj59+ljctGnThH83YMAAi/O5vOz2Cqeg9e/f3+LDDz/ca6tevbrFOqU0nA4abieiJbnLly/vtTVo0KDQ94oqmRj+W7Rk39SpUy3WaXZhP52umg80ZaZHjx5em07n/+abbyxetmzZdr9vOE00ajp6vgjTIrSUd7du3by2xo0bW6zjI5wWnKyoKc0TJkyw+OKLL7ZY06HC10By0v2Z6XehbNmyXptOIdap/WGJab0mh99JjnE0TVs89NBDvTad+r1q1SqLFy9evN3vyzT8wkVdZ1L9zHbZZReLzzrrLIv1fsY5/zxar149r03HWLLlv8N7HxWV/lESRZXh1nNSsuenqDQ5vecrSmqivoaed6tVq+b10/NweH/59ddfW6yp+Szx8Ce9bk2fPt1r03uQWrVqWRx17Qu/IzpOtZT0nDlzUtrfXE5VLAr9HDRV+Pjjj0/YTz+vUaNGef369u1rcUlKhwox0wYAAAAAACCGeGgDAAAAAAAQQzy0AQAAAAAAiKESuaaN5rBpOWfnnOvevbvFut7G7NmzvX7Dhg2zONfLMmdSmHM5bdo0i/fff3+vrWHDhhZfdNFFFrdv397rp6UvNf87zMnWMt9hrq8e0x9++MHiMDdZ28aPH++1PfvssxbrGgLhmh1xzgfPxFoF+pp6rMI8YM2x1s9WyyIWRT7n96Zi3bp1Fnfs2NFrGzp0qMV/+ctfLNZ1iKKEx0LLat5xxx1e28iRIy1O9dgjM8Lzg47h8Hy7evVqi+fNm2fxggULvH76HWJthaLRdd8qVarktWmZb11HLR33L5xbiy7Vz0yPa+vWrS3WdU2c8+99unTp4rW9/vrrFm/YsCHhPiVbqjgX7oH136rrA4XXNP13h+cn3dZ1L8LPJ9l1hKLo8dY1GfU+2Tn/OxKWfn/vvfcs1mtw+O9inZTfhffugwcPtrh27doW69hzzrmVK1davHDhwoSvMXr0aIvDtd6S/dzz+fioihUrWqzr0ei6b875Y3PSpEkWn3rqqV6/kryOjWKmDQAAAAAAQAzx0AYAAAAAACCGSmR6lJbz69y5s9dWpkwZi3Xa6L333uv127RpU4b2Lr/p1D5Nj3HOuc8//9ziSy65ZLvfS6d8hlP5k01ZipoinAvTGTO9bzoFdODAgV7b7rvvbvFLL71kcapl0SnrnTqd3uucc0cffbTFOl3/iCOO8Po1aNDAYk0R/OSTT7x+OjU7nDrOsYqvqPTJN954w2vTaeHz58+3eNGiRV6/XJmGnC167dL7l7CUt6Y86DT8VD/vdJSvRtHpfYbeo1apUsXrp+dRTXUN26Lk0zVTx5GmUOy5555eP71nCdN1dSxpOmI60sfCMaa/Y6JKlOu/S1NUnfPvsTSlR9P+ncuN9Ld0CMfAxIkTLT799NMtrl+/vtdP01HDY6ApV3zOqQvTGDW9SZdh0HHpnF9aXe9rw365gpk2AAAAAAAAMcRDGwAAAAAAgBgqMelROkVQp0BddtllXj+dLqjT+adPn+71y/WpovlAjyFT8rNHP3ed3q3Vtpzzx2y6pxpj++gxWLt2rcWvvPJKcewOikl4HdSp3uPGjfPaNA1Kq5gsW7bM66fpj1xnt03HoqZE3XTTTV6//fbbz2JNTwwroqSC45Q9erzHjBljcZia+t1331ms6cXOcT0tjN4DagpUuBSC9gu/95p2ls0xoelv+p1wzj+/hmly69evt1j/zXwnChceU/1uaDVajZE5mgoYpof27NnT4goVKlgcXu8++ugji8O0wHTun3PxuE4y0wYAAAAAACCGeGgDAAAAAAAQQzy0AQAAAAAAiKFSRcnRKlWqVLEldOlaNWPHjrW4RYsWCf9m6NChFoclptOd+5ZpBQUFydWw3obiPIZwkwsKClqn44U4jsWHsZgTGIvbEFV6Vu8binP9hFwei+Hnr9s5tmZFXo1FPY46pmrUqOH1q1ixosXz5s3z2nTdqLjIhbGoxyYO61c455cv17Lezvn7qOuzJFsSvhB5NRZzVUkZi3r+23///b22CRMmWFypUiWLw9/uBx10kMWzZs1K6/4V85o2MFIEygAAAZ1JREFUhY5FZtoAAAAAAADEEA9tAAAAAAAAYihWJb91KlI4LalmzZoWlylTxuKoEm6jRo2yONV0qDhOlwQAIJPC652WykXmhZ8/9x+5QY+jjqmlS5d6/cJtZF4cx5imkIRpkb/88ovF25ESBRQL/T4vXLjQa9NS9/o7/MUXX/T6zZ49OzM752J6PijuHQAAAAAAAMDWeGgDAAAAAAAQQzy0AQAAAAAAiKFYrWmj+WNhLtnMmTMt1tJgO+64o9dPc+TSURYzjjltAAAAAHLX5s2bLQ7XrWEdG+SKjRs3etvNmjUrpj2JN2baAAAAAAAAxBAPbQAAAAAAAGKoqOlRa5xzizKxI0WhKUt5Mj2wbhpfKxbHME9xHEs+jmFu4DiWfBzD3MBxLPk4hhmiZeGzgONY8nEMc0Ohx7EUa7YAAAAAAADED+lRAAAAAAAAMcRDGwAAAAAAgBjioQ0AAAAAAEAM8dAGAAAAAAAghnhoAwAAAAAAEEM8tAEAAAAAAIghHtoAAAAAAADEEA9tAAAAAAAAYoiHNgAAAAAAADH0/41O+6Do3ruHAAAAAElFTkSuQmCC\n","text/plain":["
"]},"metadata":{"needs_background":"light"}}]},{"cell_type":"markdown","metadata":{"id":"F60Ql7sP7beI"},"source":["Απ' ότι φαίνεται, παρότι ο αυτοκωδικοποιητής χάνει λίγο σε λεπτομέρεια (εμφανέστερα στο 8ο ψηφίο που είναι ο αριθμός $9$) κατορθώνει να μάθει σε ικανοποιητικό βαθμό τα χαρακτηριστικά της εισόδου του συγκεκριμένου dataset.\n","\n","Ας δούμε την αντίστοιχη κωδικοποίηση των $10$ πρώτων ψηφίων ($32$ χαρακτηριστικά)"]},{"cell_type":"code","metadata":{"id":"WQ1OPOdC9FJ-","colab":{"base_uri":"https://localhost:8080/","height":427},"executionInfo":{"status":"ok","timestamp":1642513149850,"user_tz":-120,"elapsed":269,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"b521e34a-b72f-44e0-d9d2-44eb09324c9e"},"source":["pd.set_option('display.max_columns', dim_h) #display 32 columns\n","pd.DataFrame(data=h[0:10], columns=range(1,dim_h+1), index=range(1,11)).head(10)"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["\n","
\n","
\n","
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
1234567891011121314151617181920212223242526272829303132
10.2784594.9933598.9231703.7183536.8333892.9276234.94090210.9286578.5701512.3250598.4773641.6350625.9214343.5698707.34965310.7352095.4221805.24282610.89786714.6051568.0173915.3654475.61517810.9025049.8615972.8117316.2173705.0787272.6543992.3624865.4323619.119932
25.9360086.7399172.2890882.7298733.28610911.0994522.8582690.8246819.2463748.67549616.7886379.0622771.8584233.0140784.6474690.8092940.0000006.4357797.5814482.3647649.8815920.3366566.8429029.74045910.44737813.61576813.51972412.37904014.39960412.4297408.1650054.965208
36.4798186.43504610.3012085.9943831.3003225.3364102.8146657.6596482.4867524.0800531.4482375.6936104.2142015.0230212.9624030.3229862.3377033.6432363.7367693.2953154.1362412.8297584.8107003.0220565.9769985.7020854.5226036.0025660.4957714.6376295.3223053.069359
413.3747288.8564453.5237946.86353113.36077110.21588914.1688597.11636214.7039795.28269519.10854013.83817019.24585712.2447852.5397578.7234118.2065829.9421358.1565046.85207218.4348169.21383015.38207611.67151212.3278318.14784410.9028906.9818319.5730356.53855011.35824813.618340
53.92511410.4979850.2590505.4390447.4393973.1074387.3059688.96736011.0763228.1708242.0821583.7495835.5643022.8286692.2708559.9232618.5104966.4617796.9427507.9061984.2771201.9729927.1459793.6419549.0337324.4348896.7113532.7566256.2229736.67448411.0577878.129202
67.5274518.41068312.6114187.2094681.3932486.9597112.5819079.8341084.0842644.8889162.9550136.6884275.4840026.6466924.8581492.7993885.0036583.4410944.5802805.7798726.2611546.0152696.8215312.0754446.2624876.7574036.0886818.0723861.8588594.2137745.9947793.048912
73.70691010.0566386.0285757.78016210.7333335.2108622.5706152.2575837.2617775.1618157.4372403.7605565.3148661.1341328.05266510.5405982.1785438.1336706.04091919.1600745.6435026.71546611.1632144.6540819.8482245.8535835.1489604.6882818.83550510.6054337.6586593.553935
85.1418375.1605394.0729795.7973715.7374658.9081725.37397512.2227666.7446557.6348737.3590914.4896707.8228946.2102074.8078558.7570263.5225767.6651742.3125092.4385720.0000007.9932635.0945204.1433029.7600298.5250818.8761700.0000007.8592477.0388262.7699896.870513
93.60375319.4539996.21678413.1599010.0680020.4156797.1167935.07114210.59206210.3391854.4670417.42383619.4471246.3053399.27147810.3242628.8673573.5374437.9136617.56959710.6760638.8621887.2659968.28149014.2008146.7839439.0726555.1589409.74171810.86706514.6054087.576882
1010.99532910.9571308.3908049.6647119.1204107.1267835.52714321.2178179.4238842.46166813.7969886.36638510.8957128.25222010.8934959.3753557.70360210.2127804.27661319.12629310.8167466.2253989.72286211.71316111.9083153.41088411.7696189.2557418.3881833.6941739.5726939.145874
\n","
\n"," \n"," \n"," \n","\n"," \n","
\n","
\n"," "],"text/plain":[" 1 2 3 4 5 6 \\\n","1 0.278459 4.993359 8.923170 3.718353 6.833389 2.927623 \n","2 5.936008 6.739917 2.289088 2.729873 3.286109 11.099452 \n","3 6.479818 6.435046 10.301208 5.994383 1.300322 5.336410 \n","4 13.374728 8.856445 3.523794 6.863531 13.360771 10.215889 \n","5 3.925114 10.497985 0.259050 5.439044 7.439397 3.107438 \n","6 7.527451 8.410683 12.611418 7.209468 1.393248 6.959711 \n","7 3.706910 10.056638 6.028575 7.780162 10.733333 5.210862 \n","8 5.141837 5.160539 4.072979 5.797371 5.737465 8.908172 \n","9 3.603753 19.453999 6.216784 13.159901 0.068002 0.415679 \n","10 10.995329 10.957130 8.390804 9.664711 9.120410 7.126783 \n","\n"," 7 8 9 10 11 12 \\\n","1 4.940902 10.928657 8.570151 2.325059 8.477364 1.635062 \n","2 2.858269 0.824681 9.246374 8.675496 16.788637 9.062277 \n","3 2.814665 7.659648 2.486752 4.080053 1.448237 5.693610 \n","4 14.168859 7.116362 14.703979 5.282695 19.108540 13.838170 \n","5 7.305968 8.967360 11.076322 8.170824 2.082158 3.749583 \n","6 2.581907 9.834108 4.084264 4.888916 2.955013 6.688427 \n","7 2.570615 2.257583 7.261777 5.161815 7.437240 3.760556 \n","8 5.373975 12.222766 6.744655 7.634873 7.359091 4.489670 \n","9 7.116793 5.071142 10.592062 10.339185 4.467041 7.423836 \n","10 5.527143 21.217817 9.423884 2.461668 13.796988 6.366385 \n","\n"," 13 14 15 16 17 18 \\\n","1 5.921434 3.569870 7.349653 10.735209 5.422180 5.242826 \n","2 1.858423 3.014078 4.647469 0.809294 0.000000 6.435779 \n","3 4.214201 5.023021 2.962403 0.322986 2.337703 3.643236 \n","4 19.245857 12.244785 2.539757 8.723411 8.206582 9.942135 \n","5 5.564302 2.828669 2.270855 9.923261 8.510496 6.461779 \n","6 5.484002 6.646692 4.858149 2.799388 5.003658 3.441094 \n","7 5.314866 1.134132 8.052665 10.540598 2.178543 8.133670 \n","8 7.822894 6.210207 4.807855 8.757026 3.522576 7.665174 \n","9 19.447124 6.305339 9.271478 10.324262 8.867357 3.537443 \n","10 10.895712 8.252220 10.893495 9.375355 7.703602 10.212780 \n","\n"," 19 20 21 22 23 24 \\\n","1 10.897867 14.605156 8.017391 5.365447 5.615178 10.902504 \n","2 7.581448 2.364764 9.881592 0.336656 6.842902 9.740459 \n","3 3.736769 3.295315 4.136241 2.829758 4.810700 3.022056 \n","4 8.156504 6.852072 18.434816 9.213830 15.382076 11.671512 \n","5 6.942750 7.906198 4.277120 1.972992 7.145979 3.641954 \n","6 4.580280 5.779872 6.261154 6.015269 6.821531 2.075444 \n","7 6.040919 19.160074 5.643502 6.715466 11.163214 4.654081 \n","8 2.312509 2.438572 0.000000 7.993263 5.094520 4.143302 \n","9 7.913661 7.569597 10.676063 8.862188 7.265996 8.281490 \n","10 4.276613 19.126293 10.816746 6.225398 9.722862 11.713161 \n","\n"," 25 26 27 28 29 30 \\\n","1 9.861597 2.811731 6.217370 5.078727 2.654399 2.362486 \n","2 10.447378 13.615768 13.519724 12.379040 14.399604 12.429740 \n","3 5.976998 5.702085 4.522603 6.002566 0.495771 4.637629 \n","4 12.327831 8.147844 10.902890 6.981831 9.573035 6.538550 \n","5 9.033732 4.434889 6.711353 2.756625 6.222973 6.674484 \n","6 6.262487 6.757403 6.088681 8.072386 1.858859 4.213774 \n","7 9.848224 5.853583 5.148960 4.688281 8.835505 10.605433 \n","8 9.760029 8.525081 8.876170 0.000000 7.859247 7.038826 \n","9 14.200814 6.783943 9.072655 5.158940 9.741718 10.867065 \n","10 11.908315 3.410884 11.769618 9.255741 8.388183 3.694173 \n","\n"," 31 32 \n","1 5.432361 9.119932 \n","2 8.165005 4.965208 \n","3 5.322305 3.069359 \n","4 11.358248 13.618340 \n","5 11.057787 8.129202 \n","6 5.994779 3.048912 \n","7 7.658659 3.553935 \n","8 2.769989 6.870513 \n","9 14.605408 7.576882 \n","10 9.572693 9.145874 "]},"metadata":{},"execution_count":11}]},{"cell_type":"markdown","metadata":{"id":"YIbaD6b__Law"},"source":["Οι αυτοκωδικοποιητές μαθαίνουν κατανεμημένες αναπαραστάσεις (distributed representations) της εισόδου τους. Συνεπώς αναμένουμε οι αναπαραστάσεις όμοιων ψηφίων στο επίπεδο κωδικοποίησης να είναι περισσότερο \"κοντά\". Ας υπολογίσουμε την ομοιότητα συνημιτόνου για τις αναπαραστάσεις των $10$ πρώτων ψηφίων του συνόλου ελέγχου."]},{"cell_type":"code","metadata":{"id":"vWbLbxauqDTd","colab":{"base_uri":"https://localhost:8080/","height":363},"executionInfo":{"status":"ok","timestamp":1642513843330,"user_tz":-120,"elapsed":265,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"f8caba2f-50cc-4478-ebc1-254afd3e563b"},"source":["pd.DataFrame(data=cosine_similarity(h[0:10]), columns=range(1,11), index=range(1,11)).head(10)"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["\n","
\n","
\n","
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
12345678910
11.0000000.6791750.7644450.8239730.8478050.7993790.8643370.7927660.7996910.923981
20.6791751.0000000.7397150.8175250.7258640.7319160.7524400.7639060.7292280.735690
30.7644450.7397151.0000000.7949440.7721070.9835880.7535800.8106080.8139960.847206
40.8239730.8175250.7949441.0000000.8518670.8113960.8039230.8487610.8409470.879449
50.8478050.7258640.7721070.8518671.0000000.7890350.8596130.8558290.8862150.860500
60.7993790.7319160.9835880.8113960.7890351.0000000.7882380.8249460.8336660.878268
70.8643370.7524400.7535800.8039230.8596130.7882381.0000000.7956290.8274380.868047
80.7927660.7639060.8106080.8487610.8558290.8249460.7956291.0000000.8084670.859179
90.7996910.7292280.8139960.8409470.8862150.8336660.8274380.8084671.0000000.824825
100.9239810.7356900.8472060.8794490.8605000.8782680.8680470.8591790.8248251.000000
\n","
\n"," \n"," \n"," \n","\n"," \n","
\n","
\n"," "],"text/plain":[" 1 2 3 4 5 6 7 \\\n","1 1.000000 0.679175 0.764445 0.823973 0.847805 0.799379 0.864337 \n","2 0.679175 1.000000 0.739715 0.817525 0.725864 0.731916 0.752440 \n","3 0.764445 0.739715 1.000000 0.794944 0.772107 0.983588 0.753580 \n","4 0.823973 0.817525 0.794944 1.000000 0.851867 0.811396 0.803923 \n","5 0.847805 0.725864 0.772107 0.851867 1.000000 0.789035 0.859613 \n","6 0.799379 0.731916 0.983588 0.811396 0.789035 1.000000 0.788238 \n","7 0.864337 0.752440 0.753580 0.803923 0.859613 0.788238 1.000000 \n","8 0.792766 0.763906 0.810608 0.848761 0.855829 0.824946 0.795629 \n","9 0.799691 0.729228 0.813996 0.840947 0.886215 0.833666 0.827438 \n","10 0.923981 0.735690 0.847206 0.879449 0.860500 0.878268 0.868047 \n","\n"," 8 9 10 \n","1 0.792766 0.799691 0.923981 \n","2 0.763906 0.729228 0.735690 \n","3 0.810608 0.813996 0.847206 \n","4 0.848761 0.840947 0.879449 \n","5 0.855829 0.886215 0.860500 \n","6 0.824946 0.833666 0.878268 \n","7 0.795629 0.827438 0.868047 \n","8 1.000000 0.808467 0.859179 \n","9 0.808467 1.000000 0.824825 \n","10 0.859179 0.824825 1.000000 "]},"metadata":{},"execution_count":19}]},{"cell_type":"markdown","metadata":{"id":"G0kMBcPnNrrO"},"source":["Αναμένουμε το 3ο και το 6ο ψηφίο να είναι εγγύτερα (αριθμός $1$), όπως αντίστοιχα το 5ο και το 7ο (αριθμός $4$) και το 8ο και το 10ο (αριθμός $9$). Αυτό ισχύει για το 3ο και το 6ο ψηφίο (ομοιότητα $0,98$) και για το 8ο με το 10ο (ομοιότητα $0,85$). Ωστόσο το 5ο δεν είναι πιο όμοιο με το 7ο, αλλά με το 10ο . Επίσης, γενικότερα, ακόμα και για τα σημεία που δεν μοιάζουν καθόλου μεταξύ τους, οι ομοιότητες είναι υψηλές (η πλειοψηφία κυμαίνεται μεταξύ $0,70-0,98$).\n","\n","Καταλήγουμε στο συμπέρασμα ότι μπορεί ναι μεν η εκπαίδευση του αυτοκωδικοποιητή να ολοκληρώθηκε με χαμηλό σφάλμα εκπαίδευσης και γενίκευσης, ωστόσο η αναπαράσταση που έμαθε χρήζει βελτίωσης. Η ζητούμενη βελτίωση μπορεί να επιτευχθεί μεταβάλλοντας τα χαρακτηριστικά του αυτοκωδικοποιητή (μέγεθος κωδικοποίησης, συναρτήσεις (από)κωδικοποίησης, αλγόριθμος σφάλματος και εκπαίδευσης) αλλά και εξετάζοντας διαφορετικές αρχιτεκτονικές αυτοκωδικοποιητών, όπως θα δούμε παρακάτω.\n","\n","Πριν προχωρήσουμε, ας δούμε τη μέση τιμή των χαρακτηριστικών του υποπλήρη αυτοκωδικοποιητή "]},{"cell_type":"code","metadata":{"id":"tNnkzN_fqust","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1642513389501,"user_tz":-120,"elapsed":245,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"3566ce27-96e6-45d2-d377-530e2ea568d9"},"source":["print(h.mean())"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["7.7969995\n"]}]},{"cell_type":"markdown","metadata":{"id":"dGbZQUoKhy2M"},"source":["## Αραιός αυτοκωδικοποιητής"]},{"cell_type":"markdown","metadata":{"id":"nCNuE3oSS3WY"},"source":["Στον **αραιό αυτοκωδικοποιητή** (*sparse autoencoder*) επιβάλλουμε όρο ποινής αραιότητας $\\Omega(\\mathbf{h})$ του επιπέδου κωδικοποίησης $\\mathbf{h}$ στη διαδικασία μάθησης (παράμετρος *activity_regularizer* της κλάσης [Dense](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense)).\n","\n","Στη συγκεκριμένη περίπτωση επιλέγουμε $L_1$ κανονικοποίηση με βάρος $\\lambda=10^{-4}$, δηλαδή $\\Omega(\\mathbf{h}) = 10^{-4}||\\mathbf{h}||$"]},{"cell_type":"code","metadata":{"id":"l_ucBa5teUba","executionInfo":{"status":"ok","timestamp":1642513455152,"user_tz":-120,"elapsed":233,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"e3d0d167-d64e-45aa-f782-17cafa8b1148"},"source":["sparse_autoencoder = Sequential([\n"," Dense(dim_h, activation='relu', \n"," activity_regularizer=regularizers.l1(10e-4), \n"," input_shape=(dim_x,), name='encoder'),\n"," Dense(dim_x, activation='sigmoid', name='decoder')\n","])\n","\n","sparse_autoencoder.summary()\n","\n","sparse_autoencoder.compile(optimizer='adam', loss='binary_crossentropy')"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Model: \"sequential_4\"\n","_________________________________________________________________\n"," Layer (type) Output Shape Param # \n","=================================================================\n"," encoder (Dense) (None, 32) 25120 \n"," \n"," decoder (Dense) (None, 784) 25872 \n"," \n","=================================================================\n","Total params: 50,992\n","Trainable params: 50,992\n","Non-trainable params: 0\n","_________________________________________________________________\n"]}]},{"cell_type":"markdown","metadata":{"id":"lc4V1KadUYvq"},"source":["Όπως και προηγουμένως, εκπαιδεύουμε για $30$ εποχές με μέγεθος δέσμης $64$"]},{"cell_type":"code","metadata":{"id":"qleHxJHxexCH","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1642513591613,"user_tz":-120,"elapsed":123427,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"cda8ae56-9b76-4209-9921-59451adcda36"},"source":["sparse_autoencoder.fit(x_train, x_train,\n"," epochs=30,\n"," batch_size=64,\n"," shuffle=True,\n"," validation_data=(x_test, x_test))"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Epoch 1/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.2315 - val_loss: 0.1757\n","Epoch 2/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1651 - val_loss: 0.1541\n","Epoch 3/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1494 - val_loss: 0.1430\n","Epoch 4/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1409 - val_loss: 0.1363\n","Epoch 5/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1352 - val_loss: 0.1320\n","Epoch 6/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1316 - val_loss: 0.1288\n","Epoch 7/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1289 - val_loss: 0.1264\n","Epoch 8/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1268 - val_loss: 0.1246\n","Epoch 9/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1251 - val_loss: 0.1229\n","Epoch 10/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1237 - val_loss: 0.1219\n","Epoch 11/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1225 - val_loss: 0.1207\n","Epoch 12/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1214 - val_loss: 0.1197\n","Epoch 13/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1205 - val_loss: 0.1189\n","Epoch 14/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1197 - val_loss: 0.1180\n","Epoch 15/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1190 - val_loss: 0.1174\n","Epoch 16/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1184 - val_loss: 0.1167\n","Epoch 17/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1178 - val_loss: 0.1162\n","Epoch 18/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.1173 - val_loss: 0.1159\n","Epoch 19/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1168 - val_loss: 0.1154\n","Epoch 20/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1164 - val_loss: 0.1150\n","Epoch 21/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1160 - val_loss: 0.1145\n","Epoch 22/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1156 - val_loss: 0.1142\n","Epoch 23/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1153 - val_loss: 0.1141\n","Epoch 24/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.1150 - val_loss: 0.1136\n","Epoch 25/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.1147 - val_loss: 0.1135\n","Epoch 26/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.1144 - val_loss: 0.1130\n","Epoch 27/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1141 - val_loss: 0.1128\n","Epoch 28/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.1139 - val_loss: 0.1124\n","Epoch 29/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1136 - val_loss: 0.1122\n","Epoch 30/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1134 - val_loss: 0.1118\n"]},{"output_type":"execute_result","data":{"text/plain":[""]},"metadata":{},"execution_count":15}]},{"cell_type":"markdown","metadata":{"id":"wBspI15yWdbN"},"source":["Παρατηρούμε ότι ούτε σε αυτή την περίπτωση παρουσιάστηκαν φαινόμενα υπερπροσαρμογής στην εκπαίδευση. Το σφάλμα εκπαίδευσης ωστόσο είναι $0.11$, ελαφρώς μεγαλύτερο απ' ότι προηγουμένως ($0.09$). \n","\n","Όπως και πριν, ορίζουμε τα δίκτυα του κωδικοποιητή και του αποκωδικοποιητή."]},{"cell_type":"code","metadata":{"id":"QYXKlBHXfLlf"},"source":["sparse_encoder = Sequential([\n"," sparse_autoencoder.layers[-2]\n","])\n","\n","sparse_decoder = Sequential([\n"," sparse_autoencoder.layers[-1]\n","])\n","\n","h_sparse = sparse_encoder.predict(x_test)\n","x_test_out_sparse = sparse_decoder.predict(h_sparse)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"yxjYdct0XdxW"},"source":["Εμφανίζουμε στην πρώτη σειρά τα δέκα πρώτα ψηφία του συνόλου ελέγχου και στη δεύτερη σειρά, τα ίδια ψηφία όπως προκύπτουν από την έξοδο του αραιού αυτοκωδικοποιητή"]},{"cell_type":"code","metadata":{"id":"bPCOUKhHfRuT","colab":{"base_uri":"https://localhost:8080/","height":248},"executionInfo":{"status":"ok","timestamp":1642513678496,"user_tz":-120,"elapsed":1119,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"8209baef-f695-452e-eb47-ae3b2f540699"},"source":["compare_digits(x_test, x_test_out_sparse)"],"execution_count":null,"outputs":[{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAABG0AAADnCAYAAACkCqtqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd7wU9fX/8UNUIjYUBBtSBRsqAmJXLNHYiA27Kcau0ahR80uzJvl+LRhbNCZfa4xiFCtIsJEEiRpAUZoICooigiBKlNju7488PHl/DneHvcvu3rm7r+dfZ/zM3Tvs7Gd27vg557RqaGgwAAAAAAAA5MvXmvsAAAAAAAAAsCwe2gAAAAAAAOQQD20AAAAAAAByiIc2AAAAAAAAOcRDGwAAAAAAgBzioQ0AAAAAAEAOrdyUnVu1akV/8GbS0NDQqhyvwzlsVgsaGho6lOOFOI/Nh7lYE5iLNYC5WBOYizWAuVgTmIs1gLlYExqdi6y0AapndnMfAAAzYy4CecFcBPKBuQjkQ6NzkYc2AAAAAAAAOcRDGwAAAAAAgBzioQ0AAAAAAEAO8dAGAAAAAAAgh3hoAwAAAAAAkEM8tAEAAAAAAMghHtoAAAAAAADkEA9tAAAAAAAAcmjl5j4A1Kcf/ehHHrdp0yYZ23rrrT0+/PDDC77GTTfd5PE//vGPZOyuu+5a0UMEAAAAAKBZsdIGAAAAAAAgh3hoAwAAAAAAkEM8tAEAAAAAAMghatqgaoYOHepxVq0a9eWXXxYcO+WUUzzee++9k7G//vWvHr/55pvFHiKaWa9evZLtadOmeXz22Wd7fP3111ftmOrZ6quv7vGVV17psc49M7Px48d7PHjw4GRs9uzZFTo6AACA5rHOOut43Llz56J+Jt4TnXPOOR5PmjTJ4+nTpyf7TZw4sZRDRA1hpQ0AAAAAAEAO8dAGAAAAAAAgh0iPQsVoOpRZ8SlRmhLzl7/8xePu3bsn+x100EEe9+jRIxk79thjPf71r39d1O9F89t2222TbU2PmzNnTrUPp+5tsMEGHp900kkex7TFfv36eXzggQcmYzfeeGOFjg6qb9++Hg8bNiwZ69q1a8V+7z777JNsT5061eO33nqrYr8Xy6ffkWZmjzzyiMdnnnmmxzfffHOy3xdffFHZA6tBHTt29Pi+++7zeOzYscl+t9xyi8ezZs2q+HF9pW3btsn2brvt5vHIkSM9/uyzz6p2TEBLcMABB3g8aNCgZGzgwIEeb7LJJkW9Xkx76tKli8df//rXC/7cSiutVNTro3ax0gYAAAAAACCHeGgDAAAAAACQQ6RHoaz69+/v8SGHHFJwv8mTJ3sclxsuWLDA4yVLlnjcunXrZL/nnnvO42222SYZa9++fZFHjDzp06dPsv2vf/3L4wcffLDah1N3OnTokGzfcccdzXQkaKp9993X46wl1uUWU3BOOOEEj4866qiqHQf+Q7/7fvvb3xbc74YbbvD41ltvTcY++eST8h9YjdGuMWbpPY2mIs2bNy/Zr7lSorTDn1l6rdf01hkzZlT+wFqYtdZaK9nWlPvevXt7HLuYkmqWb1pW4YwzzvBYU8HNzNq0aeNxq1atVvj3xi6pQLFYaQMAAAAAAJBDPLQBAAAAAADIIR7aAAAAAAAA5FCz1rSJLaA1j/Cdd95JxpYuXerx3Xff7fG7776b7Ec+bvPSFsEx91NzvrX+wty5c4t67fPOOy/Z3mKLLQruO3z48KJeE81Pc8K1Da2Z2V133VXtw6k7Z511lscHH3xwMjZgwIAmv562kjUz+9rX/vv/BiZOnOjx3/72tya/NlIrr/zfr/D999+/WY4h1so499xzPV599dWTMa1RhcrQ+depU6eC+91zzz0e6/0VClt33XU9Hjp0aDLWrl07j7WW0A9+8IPKH1gBP/vZzzzu1q1bMnbKKad4zH3zso499liPf/nLXyZjG2+8caM/E2vfvP/+++U/MJSNXh/PPvvsiv6uadOmeax/C6F8tOW6XqvN0hqr2qbdzOzLL7/0+Oabb/b42WefTfbLw3WSlTYAAAAAAAA5xEMbAAAAAACAHGrW9Kgrrrgi2e7atWtRP6fLOj/66KNkrJrLzubMmeNx/LeMGzeuaseRJ48++qjHulTNLD1XCxcubPJrx/axq6yySpNfA/mz2WabeRzTKeISdJTfNddc47EuEy3VoYceWnB79uzZHh955JHJfjHNBsu3xx57eLzjjjt6HL+PKim2Pta01dVWWy0ZIz2q/GJ795/+9KdF/ZymnjY0NJT1mGpV3759PY5L7NWll15ahaNZ1pZbbplsa0r5gw8+mIzx3bosTZf5zW9+43H79u2T/QrNl+uvvz7Z1nTvUu55UZyYCqOpTpriMnLkyGS/f//73x4vXrzY4/g9pfelo0aNSsYmTZrk8fPPP+/xiy++mOz3ySefFHx9FE/LKZilc0zvNeNnoljbb7+9x59//nky9uqrr3o8ZsyYZEw/c59++mlJv7sYrLQBAAAAAADIIR7aAAAAAAAA5BAPbQAAAAAAAHKoWWvaaItvM7Ott97a46lTpyZjm2++ucdZecU77LCDx2+99ZbHhVr0NUbz2ObPn++xtrOO3nzzzWS7XmvaKK1fUarzzz/f4169ehXcT3NJG9tGfl1wwQUex88M86gyRowY4bG25C6VtjZdsmRJMtalSxePte3sCy+8kOy30korrfBx1LqYz61tm2fOnOnxr371q6od07e+9a2q/S4sa6uttkq2+/XrV3Bfvbd5/PHHK3ZMtaJjx47J9mGHHVZw3+9///se631jpWkdmyeffLLgfrGmTawHCbMf/ehHHmsL92LFOm3f/OY3PY5tw7X+TSVrYNSqrDoz22yzjcfa6jl67rnnPNa/K2fNmpXs17lzZ4+1lqlZeeoAYln6POCMM87wOM6xtdZaq9Gff/vtt5Ptv//97x6/8cYbyZj+DaK1FQcMGJDsp9eE/fffPxmbOHGix9o2vNxYaQMAAAAAAJBDPLQBAAAAAADIoWZNj3rqqacyt1Vs1faV2G60T58+Husyp+22267o41q6dKnH06dP9zimbOlSKV2ajhVz4IEHeqytM1u3bp3s995773n8//7f/0vGPv744wodHVZU165dk+3+/ft7rPPNjNaI5bL77rsn25tuuqnHury32KW+cfmnLk/W1plmZnvuuafHWe2ITzvtNI9vuummoo6j3vzsZz9LtnWJuC7Fjylq5abfffGzxXLx6spK2YliGgGyXX311cn2cccd57HeX5qZ/fnPf67KMUW77rqrx+utt14ydvvtt3v8xz/+sVqH1GJo6q6Z2fe+971G93v55ZeT7Xnz5nm89957F3z9tm3beqypV2Zmd999t8fvvvvu8g+2zsX7/z/96U8eazqUWZoenJUyqGJKlIrlL1B+v/vd75JtTWvLat+tzw1eeeUVj3/yk58k++nf9dFOO+3ksd6H3nrrrcl++nxBrwFmZjfeeKPHDzzwgMflTpVlpQ0AAAAAAEAO8dAGAAAAAAAgh5o1PaocFi1alGw/88wzje6XlXqVRZcex1QsXYo1dOjQkl4fy9J0mbgkUul7/te//rWix4TyiekUqppdN2qdpqHde++9yVjWclOl3bx0yecll1yS7JeVjqivcfLJJ3vcoUOHZL8rrrjC41VXXTUZu+GGGzz+7LPPlnfYNeXwww/3OHYsmDFjhsfV7LSmaW4xHWr06NEef/DBB9U6pLq12267FRyLXWmy0hOxrIaGhmRbP+vvvPNOMlbJDkBt2rRJtnXp/+mnn+5xPN4TTjihYsdUCzTdwcxszTXX9Fi7zcR7Fv1+Ovrooz2OKRk9evTweP3110/GHn74YY/3228/jxcuXFjUsdeDNdZYw+NYAkHLKCxYsCAZu+qqqzymVEJ+xPs67dp04oknJmOtWrXyWP8uiKnzV155pcelllNo3769x9rF9OKLL0720zItMbWyWlhpAwAAAAAAkEM8tAEAAAAAAMghHtoAAAAAAADkUIuvaVMJHTt29Pi3v/2tx1/7WvqMS9tRk4dauoceeijZ3meffRrd784770y2Y/tbtAxbbbVVwTGta4IVs/LK/728F1vDJtaGOuqoozyOeePF0po2v/71rz0eMmRIst9qq63mcfwcPPLIIx7PnDmzpONoqQYPHuyxvkdm6fdTpWmNpGOPPdbjL774Itnv8ssv97je6g9Vi7Yo1TiKOf4vvfRSxY6p3hxwwAHJtrZT11pOsQZDsbSOysCBA5OxHXbYodGfuf/++0v6XfXq61//erKtNYGuueaagj+n7YNvu+02j/VabWbWvXv3gq+htVYqWQ+pJTv44IM9/vGPf5yMaRtubXtvZrZ48eLKHhhKEq9j559/vsdaw8bM7O233/ZYa8u+8MILJf1urVWz8cYbJ2P6t+WIESM8jnVsVTzeu+66y+NK1vJjpQ0AAAAAAEAO8dAGAAAAAAAgh0iPasQZZ5zhsbalje3FX3311aodU63ZYIMNPI7Lu3XJqqZk6LJ7M7MlS5ZU6OhQbrqc+3vf+14y9uKLL3r8xBNPVO2Y8B/aKjq2iC01JaoQTXPSFBszs+22266sv6ulatu2bbJdKBXCrPTUi1Jou3ZNt5s6dWqy3zPPPFO1Y6pXxc6Van4+atG1116bbO+xxx4eb7jhhsmYtl7XpfODBg0q6Xfra8RW3ur111/3OLacRjZt1x1p+ltM4S+kf//+Rf/u5557zmPuZRuXlfqp941z5sypxuFgBWmKktmyqdXq888/93j77bf3+PDDD0/222yzzRr9+U8++STZ3nzzzRuNzdL73PXWW6/gMal58+Yl29VKC2elDQAAAAAAQA7x0AYAAAAAACCHSI8ys5133jnZjlXKv6KVzM3MJk2aVLFjqnUPPPCAx+3bty+43x//+EeP661rTC3Ze++9PW7Xrl0yNnLkSI+1KwPKJ3a+U7r0tNJ0yX88pqxjvPjiiz0+/vjjy35ceRI7mmy00UYe33PPPdU+HNejR49G/zvfg9WXlYZRjs5F+I/x48cn21tvvbXHffr0Sca++c1veqxdUebPn5/sd8cddxT1u7UbycSJEwvuN3bsWI+5R2qaeD3VVDZNQYwpGNoB85BDDvE4dpvRuRjHTjrpJI/1XE+ZMqWoY68HMRVG6Xy76KKLkrGHH37YYzrm5cfTTz+dbGsqtf6NYGbWuXNnj6+77jqPs1JFNd0qpmJlKZQS9eWXXybbDz74oMdnnXVWMjZ37tyif9+KYKUNAAAAAABADvHQBgAAAAAAIId4aAMAAAAAAJBD1LQxs/333z/ZXmWVVTx+6qmnPP7HP/5RtWOqRZov3Ldv34L7jR492uOYq4qWaZtttvE45qTef//91T6cunDqqad6HHNzm8tBBx3k8bbbbpuM6THG49WaNrXuo48+SrY1J19rapil9aEWLlxY1uPo2LFjsl2ovsCYMWPK+nvRuF122cXjY445puB+ixcv9phWuOW1aNEij2Nre92+8MILV/h3de/e3WOtBWaWXhN+9KMfrfDvqldPPvlksq1zR+vWxDozhepqxNc744wzPH7ssceSsZ49e3qs9TH0e7vedejQweN4T6C1337xi18kYz/72c88vvnmmz3WNutmad2UGTNmeDx58uSCx7Tlllsm2/p3IdfbbLENt9aDWnvttZMxrS2rdWfff//9ZL8333zTY/1M6N8cZmYDBgxo8vHecsstyfZPfvITj7VeVTWx0gYAAAAAACCHeGgDAAAAAACQQ3WbHtWmTRuPtXWcmdmnn37qsabnfPbZZ5U/sBoSW3nr0jJNQYt06e+SJUvKf2CoivXXX9/jXXfd1eNXX3012U/b6KF8NBWpmnRJs5nZFlts4bFeA7LENrn1dO2NS4i1je9hhx2WjA0fPtzjIUOGNPl39e7dO9nWlIyuXbsmY4VSAvKSelfr9Pv0a18r/P/bnnjiiWocDipMUz7i3NP0q3itRPFiSukRRxzhsaZtt23btuBrXH/99R7HtLilS5d6PGzYsGRM0z/23Xdfj3v06JHsV89t3K+66iqPzz333KJ/Tq+Pp59+eqNxuej809IORx11VNl/Vy2L6UY6P0px5513JttZ6VGakq6fs9tvvz3ZT1uKNxdW2gAAAAAAAOQQD20AAAAAAAByiIc2AAAAAAAAOVS3NW3OP/98j2Pr2ZEjR3o8duzYqh1TrTnvvPOS7e22267R/R566KFkmzbfteG73/2ux9o++PHHH2+Go0G1/PSnP022te1pllmzZnn8ne98JxnTto71Rq+HsfXvAQcc4PE999zT5NdesGBBsq21M9Zdd92iXiPmfaMyCrVcj7UAfve731XjcFBmgwcPTra//e1ve6w1F8yWbXuL8tCW3TrfjjnmmGQ/nXNae0hr2ESXXXZZsr355pt7PGjQoEZfz2zZ78J6onVNhg4dmoz96U9/8njlldM/ZTfeeGOPs+p/lYPW8NPPjLYdNzO7/PLLK3ocMLvgggs8bkpNoVNPPdXjUu6jqomVNgAAAAAAADnEQxsAAAAAAIAcqpv0KF1Gbmb285//3OMPP/wwGbv00kurcky1rtgWfWeeeWayTZvv2tClS5dG//uiRYuqfCSotBEjRni86aablvQaU6ZM8XjMmDErfEy1Ytq0aR5rS1ozsz59+ni8ySabNPm1ta1tdMcddyTbxx57bKP7xRblKI9OnTol2zFF4ytz5sxJtseNG1exY0Ll7LfffgXHHnvssWR7woQJlT6cuqepUhqXKl4nNd1H06P22GOPZL927dp5HFuU1zptsRyva7169Sr4c3vttZfHq6yyiscXX3xxsl+hkg2l0vTlfv36lfW10bgTTzzRY01JiylzavLkycn2sGHDyn9gFcJKGwAAAAAAgBzioQ0AAAAAAEAO1XR6VPv27T2+7rrrkrGVVlrJY13ab2b23HPPVfbAkNDln2Zmn332WZNfY/HixQVfQ5dHtm3btuBrrL322sl2selduoTzwgsvTMY+/vjjol6jFh144IGN/vdHH320ykdSn3SpblYHhaxl+bfccovHG264YcH99PW//PLLYg8xcdBBB5X0c/XspZdeajQuh9dff72o/Xr37p1sT5o0qazHUa922mmnZLvQHI7dF9Eyxevwv/71L4+vvvrqah8OKuy+++7zWNOjjjzyyGQ/LR9A6YbiPPXUU43+d00nNkvToz7//HOPb7vttmS/3//+9x7/8Ic/TMYKpa2iMgYMGJBs67VxjTXWKPhzWnZDu0WZmf373/8u09FVHittAAAAAAAAcoiHNgAAAAAAADnEQxsAAAAAAIAcqrmaNlqrZuTIkR5369Yt2W/mzJkea/tvVN/LL7+8wq/x5z//OdmeO3eux+utt57HMV+43DbbbLNk+xvf+EZFf1+e7LLLLsn2+uuv30xHAjOzm266yeMrrrii4H7aTjarHk2xtWqK3e/mm28uaj80D62J1Nj2V6hhUxlaky9asGCBx9dee201DgcVoLUV9D7FzOy9997zmBbftUe/J/X7+Vvf+lay30UXXeTxvffem4xNnz69QkdXm0aNGpVs//KXv/RYW0SfdNJJyX6bbLKJxwMHDizqd82ZM6eEI8TyxNqHa665ZqP7aU0ws7Ru1LPPPlv+A6sSVtoAAAAAAADkEA9tAAAAAAAAcqjm0qN69Ojhcb9+/Qrup+2cNVUK5RNbqcdln+U0ePDgkn5O2/xlpXU88sgjHo8bNy4Z07SBsWPHlnQcteCQQw5JtjVV8cUXX/T4b3/7W9WOqZ4NGzbM4/PPPz8Z69ChQ8V+7/z585PtqVOnenzyySd7rCmMyJ+GhobMbVTWvvvuW3DszTff9Hjx4sXVOBxUgKZHxfk1fPjwgj+nKQHrrLOOx/q5QMvx0ksvefyLX/wiGbvyyis9/tWvfpWMHX/88R5/8sknFTq62qH3ImZp2/Ujjjii4M/tscceBce++OILj3XO/vjHPy7lENEIvd5dcMEFRf3M3XffnWyPHj26nIfUbFhpAwAAAAAAkEM8tAEAAAAAAMghHtoAAAAAAADkUIuvadOlS5dkO7Z0+0qs6aBtblEZhx56aLKtuYirrLJKUa+x5ZZbetyUdt233nqrx7NmzSq43wMPPODxtGnTin59/Mdqq63m8f77719wv/vvv99jzQFG5cyePdvjo446Khk7+OCDPT777LPL+nu1jaaZ2Y033ljW10d1rLrqqgXHqJ9QGfq9qPX5oqVLl3r82WefVfSY0Dz0e/LYY49Nxs455xyPJ0+e7PF3vvOdyh8YKurOO+9Mtk855RSP4z31pZde6vHLL79c2QOrAfF764c//KHHa6yxhsf9+/dP9uvYsaPH8e+Ju+66y+OLL764DEcJs/R8TJkyxeOsvx11Dui5rSWstAEAAAAAAMghHtoAAAAAAADkUItPj9IWsmZmnTt3bnS/v/71r8k27Uur74orrlihnz/mmGPKdCQoF12av2jRomRM26Rfe+21VTsmLCu2WddtTSmN19ODDjrIYz2ft9xyS7Kftr3Xpaxoub73ve8l2x988IHHl112WbUPpy58+eWXHo8bNy4Z6927t8czZsyo2jGheZx44okef//730/G/u///s9j5mJtmT9/frK99957exxTcy688EKPYwodlm/evHke672OtlI3M9thhx08vuSSS5Kx9957r0JHV9/23HNPjzt16uRx1t/umjaqKcS1hJU2AAAAAAAAOcRDGwAAAAAAgBxq1ZQ0oVatWuUip2iXXXbxeMSIEcmYVpxWAwYMSLbj0uO8a2hoaLX8vZYvL+ewTo1vaGjov/zdlo/z2HyYizWBubgcjz76aLI9ZMgQj5955plqH06jankubrjhhsn25Zdf7vH48eM9roHubHU7F/VeVjsBmaUprDfddFMypqnIn376aYWOrmlqeS7mxRNPPJFs77zzzh5rx6MVSFGu27lYS2phLk6cONHjrbbaquB+V155pceaLlgDGp2LrLQBAAAAAADIIR7aAAAAAAAA5BAPbQAAAAAAAHKoRbb83nXXXT0uVMPGzGzmzJkeL1mypKLHBABArdAWqKi+d955J9k+4YQTmulIUCljxozxWFvcAo057LDDku1XXnnF4549e3q8AjVtgFxo166dx61a/bdET2yx/pvf/KZqx5QHrLQBAAAAAADIIR7aAAAAAAAA5FCLTI/Kom3C9tprL48XLlzYHIcDAAAAACX78MMPk+0uXbo005EAlTVkyJBG48suuyzZb+7cuVU7pjxgpQ0AAAAAAEAO8dAGAAAAAAAgh3hoAwAAAAAAkEOtGhoait+5Vavid0ZZNTQ0tFr+XsvHOWxW4xsaGvqX44U4j82HuVgTmIs1gLlYE5iLNYC5WBOYizWAuVgTGp2LrLQBAAAAAADIIR7aAAAAAAAA5FBTW34vMLPZlTgQZCpnXz/OYfPhPLZ8nMPawHls+TiHtYHz2PJxDmsD57Hl4xzWhkbPY5Nq2gAAAAAAAKA6SI8CAAAAAADIIR7aAAAAAAAA5BAPbQAAAAAAAHKIhzYAAAAAAAA5xEMbAAAAAACAHOKhDQAAAAAAQA7x0AYAAAAAACCHeGgDAAAAAACQQzy0AQAAAAAAyCEe2gAAAAAAAOQQD20AAAAAAAByiIc2AAAAAAAAOcRDGwAAAAAAgBzioQ0AAAAAAEAO8dAGAAAAAAAgh3hoAwAAAAAAkEM8tAEAAAAAAMghHtoAAAAAAADkEA9tAAAAAAAAcoiHNgAAAAAAADnEQxsAAAAAAIAc4qENAAAAAABADq3clJ1btWrVUKkDQbaGhoZW5XgdzmGzWtDQ0NChHC/EeWw+zMWawFysAczFmsBcrAHMxZrAXKwBzMWa0OhcZKUNUD2zm/sAAJgZcxHIC+YikA/MRSAfGp2LPLQBAAAAAADIIR7aAAAAAAAA5BAPbQAAAAAAAHKIhzYAAAAAAAA51KTuUUC5tGr13+LmK620UjL2ta/991ni559/7vGXX365wr+roYFi6Hmm5ypuZ507zisAIG/0fsas9PsY5F+8f1HcowBYUay0AQAAAAAAyCEe2gAAAAAAAOQQ6VFoMl0CGpeDtm7d2uM11lgjGevfv7/Hhx12mMc77bRTst8qq6zi8fPPP+/xqFGjkv1efPFFj+fOnZuMLV261OMvvvjC47hEVY9/5ZULT4dPP/200dczS5c7swR2+eJycU2Pi2N6fvR91rQ5M973Sik0P3SOmpl99tlnjcYAUG+y0mRQW/SeJd6/6H1JTIvjnqX2xFIPqtj0fj4XyMJKGwAAAAAAgBzioQ0AAAAAAEAO8dAGAAAAAAAgh6hpgybTvN211147GevZs6fHgwcPTsa0js16663nsdbBibp37+7xoYcemoxNnz7d4+uuuy4Ze+SRRzxevHixx7EejeaPxjz0mJ+MpinU1r1du3bJfn369PF42223Lfgaf//73z1+5ZVXkv2WLFniMS1Vl0/fV/2cr7nmmsl+2223ncc6n7t27Zrsp+fj97//fTI2Y8YMj2MtIqyYQucx69qVVeMrK5++UJ2OprS5JV+/PLJqsa2++uoet23btuDPLVy40ON//etfyX7xexLLp+ehTZs2HsfP/CeffOJxpd/nQt/BZuk1Qq/LtfD9mXVN0rFir0fx9QrV4YuvV+z5LXQdN8uuJaey6srVwjltilLOcbHi9bZDhw4e6981cb9FixZ5/PHHHydjWjNTx/79738n+9XbecSy+KsUAAAAAAAgh3hoAwAAAAAAkEOkR2GF6DJgM7NtttnG4wEDBiRj2gJcl/nFpdm6rct2dVmxWbqMcKONNkrGVl11VY91WWKxbffMCi9tzWpZTfpH47LaxGu61Oabb17wNV599VWPSV1bMYXSAmPq2oEHHujxvvvu63FMu+jVq5fH7733XjJ2ww03eMz8WDFxyfVaa63lsZ47vf6ZpdfK+fPnexyvqVnLr3XO6et//etfT/bTc7x06dKCY/q7SJtavkLX0JgyoamLMaV4ww039Hj48OEeP/nkk8l+mm7KufkvnQPrrrtuMjZw4ECPNUwTSX4AACAASURBVM102rRpyX5Tpkzx+MMPP/S41FQpPaaYaq7XhI033jgZ08/Q7NmzPdbrg1nLuWYXmh/xfiMrJb5QOlm8xun1L+teVtOUir33zGoNHu979HzrdSAeh6bf1Ap9L7Jabev7mfX9pq+haU5mZoMGDfL46KOPTsY222wzj/XvoTifP/roI48/+OCDZOzdd9/1eNiwYR6PGjUq2e/NN9/0OKbAoT7wlw8AAAAAAEAO8dAGAAAAAAAgh6qeHqVL0LKq2cdlgLpsUZdrxqWblazGn7WUMtIljbW2vDire5R2e4rnQrvIjB8/3uOHH3644H663FA72ZiZHXTQQR7369cvGRs9erTHuvSwHOLndrXVVvM4pgPE6u/1RD/3+lmIlfN1KX78zGiHk7feesvj+D7X2hyrJk252X333ZMxTa/QJcNxDug8PeKII5Kxv/zlLx5rlyk61BRH32u91piZbbXVVh5vvfXWHsdl4BMnTvRY51RU7BJ+PSbtnhGPce7cucmY/u6s5d21PJ+z7htUsZ234n/XbnzHHXdcMqadpTTl49lnn0320/SKWj4XyxPTDHfccUePf/CDHyRjm2yyicevv/66x3G+aXpUVncnlXUONC0mpnXsueeeHsfU4zfeeMNjTZOMqRvFppdUWznuv+Nr6Heh3ttuscUWyX69e/f2WO9L4jzStKRi51HcT//GiWlOerz6+Ykpk8WmabUkWelR+jnN2k/P8bnnnuvxySefnOyn6eDxb9Ni309NZdY0VTOzLl26NHrss2bNSvabN2+ex6RHNY2et3gfpedG31ftPBzHmmsesdIGAAAAAAAgh3hoAwAAAAAAkEM8tAEAAAAAAMihitS0iTl/mjutuWMbbLBBsp/m42p7aLM0z09zbt9///1kP22hmJXXqnmicUyPX2s1dOrUKdlP/10zZ85Mxt5++22PNQ81q51fnmnub6G8X7P0fGhLUTOzZ555xuNJkyZ5HOu+FKqdENuLn3POOR7Hc6Pt9SZMmOBxrKdSLP1MaDtPszQ/VVtnmtV3TRul5zTWodJzHOeD1jfS9zbmdreUeZQXOp+1DtUvfvGLZD+9Rme1Wddz2KNHj2Tskksu8fjCCy/0WM+tWctpLVtt+r7H65y2GdbzqHUzzNK6XlqvJNYVKqUtbayjofUe4jkeN26cx5ovTn2j/yil7kWcl/qZiLUTVKE8/qYcRy3SNsrbbLNNMnbBBRd4rLWDzNLrl843reNlltZwy6oRo9forPOhcyfWMtFaLHGe6j1rVg2jlvhZKLaOTaxpozWftB7Qaaedluyn82rq1Kkex/u9MWPGeBzbcBdbHyirHbiee/0OzqoJ2hLP5/Jk/XtV/Ltyr7328viYY47xWGvYxNcvtpV3PAb9u0H/hjJL60vddNNNHmttTrNlaznWi6zaX3rNW2eddTyONVBPPPFEj/v27ZuM6Wu+9957Hsd6q/fee6/HWsvKLK0LljVnV7TeLSttAAAAAAAAcoiHNgAAAAAAADlUkfSouHxJ04jat2/vcbdu3ZL9dClqTJ0q1FY5pkmsv/76HutS8rjcTX8uprTo62+88cYer7vuusl+urx76NChydgdd9zhsbZ8rIWlp7pU7Z133knGRo0a5XFstT1//nyPi10Or0vL4nK3nj17ehyXBWv7W/38lZoepf/muBRTl1PnqSVmuRTbWrNYej7MzHr16uVxTIHTlsGa+liL73M16fX0mmuu8Vivd2aFU9fiedfzEdvk7rzzzh7feOONHl9++eXJfv/85z891qWmZvWdPqPXNk09MjPbaaedPNZzoMt4zczmzJnjcVZqcLF0DsdWwjGlRGnr8RVdJtySZKVkqFLeB01zMjMbMGCAx1mtfzVVTVN2Sj2OlqzQ97sutzdL02fid5DeC+n9YGzbWygNNH4uip0fehwx/UPvWWN6zvTp0z3WlICWkioX369S7gni3yqaTnb22Wd7HK9phebwvvvum2zrd+Frr72WjGkJhWLvbeL3oKZj6XmL57AW75f03MV7Sj0/+p7Fuadz4KWXXir4ejp3br755mRs2LBhHi9YsKDga2yyySYex+uKplDqd3U93ffEv6v0HlVbous9j5nZbrvt5rHO386dOyf76bU7/i6dLxtttJHHRx55ZLLfpptu6vHf/va3ZOzJJ5/0WP/2jSlt+rtKOb+stAEAAAAAAMghHtoAAAAAAADkEA9tAAAAAAAAcqgiNW1inpbWj9Hc6VhfRHO/tI2aWdpKTcdi3qDmrenvjXm52ppaW/aZpTni/fv391jr5cT9Ys0czTXV96Ol5pbq+6f/Ns3LNUvzumO9oVL+7foen3nmmcmY5uvH3zVixAiPY75+KfTYY5t5/Ry31JZ8MUc7qwZDoRz3rNz3rLbF++yzj8exHoq+n/WU31tuMYf36KOP9lhzgmOOv9I5EK/dmqeb9RraDlzb55qZ3XfffR4//fTTyZheZ2q9NXg8Vx07dvT40EMPTca0HtTLL7/scfxOizUOvpJVXyXSNqWa9601i8zMOnTo4PGECROSMZ3PLfW7sBhZ19N4fvW6WWxrXp1jsdabXl/jcWjtjAcffNDjWp9Ty1OoRWu8d9DtWCPmxRdf9FjrYxT73pZaO0bveWNNFa1pE2swTJ482WO9nreUeVnqceqc0DbMZmaHH364x1tuuaXH8b5EPwdaxyT+jfDjH//Y4/i3yrPPPuvx//7v/3qsNU3M0n9n/DfHFuO1LF7L9P3UOWCWnh+tjxfvIXUO6Llq165dsp/WI9LzbZaek6w5rDU9s+ox5bWGVCXo33B6b2iWtujef//9PY7nplA79njd1fd/xowZydirr77qcffu3T3W+yszsx122MHjWHdXn1FofZs4R5tyz9UYVtoAAAAAAADkEA9tAAAAAAAAcqgi6VFZS/i0/bUuSTJL2/vGdBdd1qvLnnTJtlm6dErb18alUtOmTfM4pmJpqzBdshVbfuu/a8qUKcmYLsmrteVuxaZ7lfrv1qXfF154occbbrhhsp8uT37kkUeSMW1jXCg1oCmy/i2lthFvblkpUFnL+VWxKUu6DHKvvfZKxnR5/1tvvZWMaSpaOZZD19pcLJam2JiZ/fSnP/U4LttWen7ffPNNj//yl78k++ly5NiCWOetXpO1BaZZmv643nrrJWM33XSTx4sXLy54vLUgtmnecccdPe7bt28ypudO06N0qW6WpizVbd26tcdbb721x7ENuV4PY7tjHau1uZh1PdVraEwfLPQ+ZF3vNF3jsMMOS8Y0VSC+9j/+8Q+PdYl4rZ2LFaHvRbx3KHb5vY4Vm15cbNqcWXqNPeecczyO6ZP6ffrUU08lY4sWLfJY/121/lnIStXedtttG/0ZbYlulqY/PP/88x7HVEVtERxTrPQ6f8stt3gc06NUrZ+bLPF7UVPR4j2HXtt0Lsb3T//O1J+J801fI16Xiz0nLSXtsNz0vYzncLPNNvP4oosuSsYGDhzocZs2bTyOaan6t/zo0aM9Hj9+fLKf/o2uZTzicR1//PEex/utNdZYw+N43jXVUj8v5S7rwEobAAAAAACAHOKhDQAAAAAAQA5VJD0q0mVhmlKky0njdkyPKrR8My4p1creutwtLlHKqug/e/bsRo8p/sy8efM81s4Bcd9aXtJYjn9bXIqo6QDf+ta3PNaUM7M0RePnP/95MhY7PNWrUiuVl9I9Kus1dFnhfvvtl+ynS1tjNzKdi7U8jypBl3yecsopyZh2/9HzFK+TEydO9PiMM87wOFbf19fQpaxmZttvv73Hp512msdxaboudz7qqKOSsYceeshjTZWtlc9EVkcTvR7qPDJLv+/0PYpdbwotzY7vX1ZKhqb1aMeomMr2yiuvePz6668nY/XaAa7Y62nW51l/rn379h5rRwuz9J4ofg5+85vfeFxPnWdKFZfza4qLpguapZ1GNCVU00rN0rmo5yqm+qsuXbok20OGDPFYz3+8R7r11ls9jh3l6uUeNc43fZ9jyQOdL5r2dPvttyf7aZqhdo6N6RT6+YnnV8+9prPW8rloKj132pXQLE2zjyUKNC232OurfjfFvyuzrt+Kc7csfS/jfBs8eLDHu+66azKm9zp6fh9//PFkv6uuusrjN954w+PYyTcrBVRTigcNGuRx7FSl4mdE0031OhyfG6zoZ4SVNgAAAAAAADnEQxsAAAAAAIAc4qENAAAAAABADlWlpo3mcBWqb5O1XxxTxe7XlNz9DTbYwOMtttii4PE+8cQTHsc2ffXa3q1YWh9B2yKamf3whz/0WPMSH3300WS/66+/3uPYkpHc0v8otkZCU16jUG5o3K9QLuvmm2+e7Kct/O6///5krJT6JaXW8Wnpsq5jxx13XDKm809zbseNG5fsd8ghh3iscyyei6wW8VrvS891bDWu23rsZmn9m9dee83jWqmRou+Z1isxS+sPxVpvWtdLa1bE96Uc10M9Dq2zE8/3Cy+84HG8Ltfy92LWe6xj8dwU225Z56zel2j9lPga2g7VzGzChAlF/a56ptey2J5da5TE61fPnj09/u53v+vx9OnTC76GthSPtQ/0mnfSSSclY507d270eLWmg5nZ0KFDPc66365l8XtR3/9Yl0ivXVq3Rv+7WXretIabtjA2Sz8j8ThqsTZbuen8i/WCDjzwQI9nzpyZjMW6J8XIqpWq9Yj03De2jZS+lx07dkzGtDZerNen9wpvvfWWx3/605+S/bR9t/5MnG96HLFW2YUXXujxbrvt5nG8/us1OrYN1+9a/UxkPXsoBSttAAAAAAAAcoiHNgAAAAAAADlUlfQoVezy6HIsFyy21Zu2cTRLW9vqEtXYuvGxxx7zOLZaZLnjsnQpqi4jPe+885L9tM3aiBEjPH744YeT/fR8ZKVrcC4al5UiqLJSEIttEdytWzeP4zJITet48sknk7G4ZLyQcrQob+niUs599tnH49guU1MytK36kUcemew3b948j4tN/4j7aVvqhQsXFnyNrLQEbStdi+lvunR3nXXWScb0+0mXCZuZjRo1ymNNfyjHZz62qD3ggAM81pScd999N9nv6aef9ljT4cp1XC1dqddT/f785je/6fFqq62W7KdLs7UNvNmyrXGLUW/XU/33LV68OBnTz/raa6+djOn7tNNOO3ms58osnVeaPhjPjaaNazqUWXp91HvPG264Idlv/vz5HtdyamKWrPSomG6q9yL6vRjfO0172mabbTyOrdn1d2e9xvrrr+/x22+/nexXKynApdBr3t57752MbbXVVh5riprZstfEQgpdb+PfhGuttZbH77//fjJGelQ2vbeJ18x4r6P0fV2yZInH/fv3T/bbeOONPdbPS7x26zw69dRTkzG9Xuv1OX7X6WvG71YtkVLJOctKGwAAAAAAgBzioQ0AAAAAAEAOVT09qhxLa7M6lRSbEqXLS7fffvtkTKuS61Kp5557LtlvxowZHtfzEsZCYorD1ltv7fFFF13ksXbCMDN79dVXPdbuKNo1xmzZlLSs3/2VSiwRroVUrHjcWe9ToX9jXIasS1Z1SWOcK1rpX1NpmvK7ajFlpqliJ4wddtjB4zgfdCn+jTfe6HFcml3K5zn+jKbtfPDBB0X/nNLPUi2ea/0ei0uzNcUoLufXDiTleF/0cxJTMr797W97rMvP9XptlqYYFJveWOv0elpsN4l4PnUp+S677FJwP01BHD16dMHjyJL1WaqF77ss+m/S9FAzs2HDhnncp0+fZEyvUfodF1NTdQ5rZ6mYOqDpUfF91tSBp556yuP77rsv2a9e519WurSK80Gva5r2FLtCaTfM3Xff3eN4T6rX53hd17QO7e6o108zs48++qjg8dc6/fura9euyVjbtm091s5tZmmHUr2nifee+tnQdLXevXsn++ncnDx5cjKmKZPlTlFuqQrNuTg/Zs2a5bGmCJql164111zT42OOOSbZT8stZM113U/T3cyWfY7wlZjerZ2r/vjHPyZjOtezvu9XFCttAAAAAAAAcoiHNgAAAAAAADnEQxsAAAAAAIAcqnpNm1Jprpq27Iu5aIXar8VcN81dPeqoo5IxzZXUttKxxZe2/6rn/MVCYju3Sy+91ONdd93V4/jeacvYmTNnepzVPlY/E1G587rjZ0k/g7VS26jYz3NWm2bN5dfzHc/H+PHjPY41Owr9rqxzkNVSt9bo+7D66qsnY506dfI4fi712jVu3DiPy11zzCyt9dC9e3ePY0tppbnhZmZz5871uBbPp75nsfWv5kp37NgxGdN6YNpyMrZW1zmncyXWQdLXP/PMM5OxXr16eaznIOb4871YfN2aLPF6uuWWW3rcrVu3gj+ndVJef/31ZEyvjVnX06y6NfXUOjrW4HriiSc8Hjt2bDKm75Nev+I9aby2fSXWdNC53b59+2RM5/p5553ncax/Uq/zT2V9tmNraK1Po/cs+jeBWVp7b+LEiR4PHz482U/nrNbIMUvrahx++OEea40/s7SuYz3NPbPs+3q9f4j1oE455RSPtY6K1p8xM9too408Pvrooz3u27dvsp/OtzFjxiRj+v03adIkj2M9rHqqd6P/Pr331O8mM7Orr77aY51vZun1cL311mv0v8fX13mqdXDM0vvjeE3Q49X24pdddlmy3y233OJx1rW2kueXlTYAAAAAAAA5xEMbAAAAAACAHMptelRcvqTL5HS5fVx6WmhZUlxqrEsVd9ppp2Rs/vz5Ht92220ev/DCC8l+9bTcrVi6ZDEurx84cKDH2v7w/fffT/bTNDRd5h/fY/1dcRmlLoksx7nR36WtAaOY2tBSPhflOM54DrRtorZgXLRoUbKfLuGPaTx6XHpNKNSiL/5MPYlL6LUtabHi+1pKG+mYcrPjjjt6rK3f4+dFz1tsPa5tUGtxibj+m2Lbe11SHNsH77fffh5rC+J4TdXrkn5v6VJgs/QzpK9tlp5X/blp06Yl+2WlOCJbVurGoYce6rGmVsR7oGeeecbjmCZX6HralPSoWqdzMX6WdVvTFs0KL48v9v2L817Pa0wN/8Mf/uCxts1tyrWxls9x1r2CbsfvoE022cRjbckdX+Ott97y+O677/b4jTfeSPbr0aOHx6eddloypn93aCrHYYcdluynaXhZnzmV9b3dks613ndrqQqzNA1X/yY0S/++u+qqqzzWts9madpN1t+VmnYT76v22msvj/U+5f7770/2e+mllzyOfyfU4j3NVzQ1W8+Zmdnzzz/v8YQJE5IxnZt67xHvL/V8aHrjqaeemuynqfnx74z33nvPY02t09REs8LlV6qJlTYAAAAAAAA5xEMbAAAAAACAHOKhDQAAAAAAQA61mJo2uq15xbF9YqGWlrHew89//nOPYzu/oUOHenzHHXd4HPPxWlJuaLVoTvD3v//9ZExzRvW905bDZmlLPc2HjC2CNec/5hqWkiMa85a1RZzmvsa8WG0jGD+P5W43nmex1s/222/vsdZgiG3/9P0r9rwx9/5Dr3H6Hpul5yPW9NLPus6rOAf0fc5qba+v37Nnz2Tskksu8VhrssTfpXWo/vznPydjWuOmFvO/s2raaE2DeB61VpTG8RqlNWheeeUVj+O1V69fWe1W9boW6+cwN0un81nb0ZqlNeH0cxDrXDzyyCMeZ9UXyqq7Uq32pXlRqAZIqa3bS3nP9N7JLP3+XLp0aTI2atQoj/NQZyFviq1/F+ukPP300x7rd5zWsDEzu/baaz3W+5n4vaivP3v27GTsggsu8HjAgAEe672mWVpbJ947FXt/2ZLmsJ47/WzH+iL6ndatW7dkTN8zjWNNOP2O03MXa0jpd3JsL961a1eP9d4n1tnR+9xY+6hexHs33Y7XsWLrKWo9Gj2/8e8RnQPxHuuMM87w+PHHH/c4zuc8YKUNAAAAAABADvHQBgAAAAAAIIdymx4V6dKpYpfu6rLIE044IRnT5YiTJk1Kxq677jqPtT1xS1piWC1xuf7hhx/ucVbLYV22/eCDDyZjuhRYz2Fc8r/OOut4rEvk4s9ltTbVFCtNLzAz22233TzWf8trr72W7Pf3v//d45hCp8v/ajGtQ9/PeH607aLOnSlTpiT76TLXUpefl9KautbEz5fOsaz3VZcWa1tKszRlSV8jps5suummHt94443JmH4O9HoRl57qdfiuu+5Kxmq9jbS+F/qem6XpTHPmzEnG9Lqk57Fjx47JfrocWNts6vebWbo0f/DgwcmYvn5Wa2qUTr+3unTpkoxpirfO9Zji8frrr3vM9bS8yt0me9VVV/VY02XM0rT9J598MhnT60Cpx1HL97NZ93yaAqpzxSxNe7rmmmsa/Rmz4lPSdJ5Onjw5GRsyZIjH5513nscxraNPnz4ex7QOvX7rd0i8F2ip51rTlLRsgpnZ+PHjPdZSBmZpmtKgQYM83mWXXZL9NNVJ39tp06Yl++lnSP92NEvbtes9cO/evZP94v0xshXbzl7P4a9//WuPY3qxzo/hw4cnY5pumseUKMVKGwAAAAAAgBzioQ0AAAAAAEAO5TY9qhypJFo1/JxzzknGdAninXfemYy98847HrfUZYXV0rp162S7R48eHmdV7dflpbFa/gYbbOCxLh/eddddk/20680///nPZEyXOury/a222irZ7+CDD/ZYuzWYpSkg2r3m448/TvbTNK24fDV2fag1eo67d++ejG2xxRYe6zyaMGFCsl+5OyBkLe2vtfms/56YIqhpE507d07GdDnx6aef7rFeM83S1BxdNhrnyre//W2PN9xww2RMPyN6XdfrrJnZWWed5XH8t9TaeYv03xfnw0cffeRxvPZoRwpN24zpa7rMXJf6x/dVr8ux28nOO+/ssV739XptRmrNitC5Epd36/edpgtqxxuzZTuflFu5U4TyptTPr74XWe+RnuNOnTp5HNOzda7Hzjm1fl9RTjHdQa9xcayU0guFvt/idnw97Ug1Y8YMj/v165fsp9u6n1l6HdC02pi+1ZK6wenx6b8v/pv0ezH+raH365ryrWncZmkKk/5NGDtE9e3b1+PYpVO/a/UY43d1KWUAsKyYZjZy5EiPNRU/Xsfnzp3rsaYmmi17rvKMlTYAAAAAAAA5xEMbAAAAAACAHOKhDQAAAAAAQA7ltqZNqTTX/n/+53887tChQ7Kf5pMOGzYsGavF1syVEnMztYVibF2r7X41P/+73/1usp/WmdE2pzGX8cMPP/R41qxZydgHH3zgsdbpiLU9NHc15sVq3rjWjoj55Np2MeZG1mLuquaKah6wtlY0S2td6PsyderUZL9yzLdi6yzUWj0G/TfMnz8/GRs9erTHmpNtZrb++ut7rC1FNTYrnJOvczlux1ziQnV3TjzxxGS/cePGNfp76038XGbVI9CaDFltaLNqKxR6vTZt2iRjen3Uc6x1x8yy27ojm34vak2wSOsjPPbYY8lYsTXC0Lhyfy/E66HWwNh22209XnfddZP9dD7H+l9YVlabb5V1LSz23Be6l42twQsdn1n6OdA6KXrPa5bWOoptpLWuS7GfkZZ0fSi2Fk9W3SI9J3rdjHbffXePY10hvS+K7cX186T1xGKLdz1XaBqdY/p3vVl6bvQeJc7FX/3qVx7Hlu4tCSttAAAAAAAAcoiHNgAAAAAAADnU4tOj4jL9Pffc0+N9993XY20dZ2Z27bXXerxw4cIKHV3ti0stH3jgAY/j8u6BAwd6rKlOsUWwpjDpcre4PFKXKcbUKT0uXb6fteQ/fkbmzJnj8VNPPeXx448/nuynrZXroRWnLvPVNBtdXmqWpirqe6SpZmalLUfPSsHJUgspUYXEz95DDz3k8X777ZeM6XnTpaflEJcq6zw67rjjPB47dmyyXz2nRBUrazl/OVL/9JoaU4r1mqrLzDUV1Sz9PGW1nsV/6HeQpuv27Nkz2U/nlba0jW2A0XSVbIkc0671HB9yyCEet23bNtlP08tj6pTOsazrZrHXhJY4L+M9gL7PmrYdv9/0ezLe86msdGC9p9TjyEpRja+hnwO9L4376VhM758+fbrHmh6dlaZVb3R+xHsk/U7bfvvtG43N0r8v4v2Nlkd4/vnnPY5pqy2prXQe6LzStMDjjz8+2U/nvZ7rW2+9Ndnvtttua3S/loaVNgAAAAAAADnEQxsAAAAAAIAc4qENAAAAAABADrXImjaa89mtW7dk7JJLLvFY805ffvnlZL/f//73HrfEfN68iPmdmmN7+umnJ2O9evXyWOsNxVoo3bt391jPYTxP2j441iXSXFXNCY7t+rJa9A0fPtzjCRMmeBxbK2sec618lrJaZmqrSm2nHn9m8eLFHmtNIP3vpYrvc6218i5F/HdrHaFTTz01GfvDH/7g8W677eaxntssMSf4ww8/9Pj2229Pxn75y196/P777xc8XjRd1jwtRqy30bFjR4/jOdbaNW+88YbHWnvDLK0nEWsr1GsL8Kx2xDrntGZFrNOm9TemTJnicSXqV9Ra/ZM82WCDDTzu37+/x7Henn5OdD8zs5EjR3qsNTWyal7FsZZc16Exei1bc801PW7Xrl2yn86XeC9XqCZh/F7Ubb3+Zb2n8VqrtXb0+1Njs/S86f1wlngc9XrdjeL7orXZsuaKnuN58+YlYyNGjPD4nnvu8TjWGtPPFtfQ5VtttdU8Pu+88zzW+wuz9LP9wgsveHzuuecm+7WkVvdZWGkDAAAAAACQQzy0AQAAAAAAyKEWkx5VqC3m+eefn+ynKTi6pO3ee+9N9tNlcShdXOanS9B02a5Z2g5Pl7FddtllBV9flwjH5aWaJheXjeoSOl1mF2l6VGzJp8todVllrS0rbio957q8OLY41OXew4YN8zimU6zoMTS2jfQ9mTVrVjK2//77e7ztttt6fPTRRyf7derUyWNNbYpt759++mmPP/roO+rovQAABKtJREFUo4LHgRWT1ea2lJSWrCX7kyZNSsZ0GbIu/V6wYEHB11zR9K2WKuvfHcf0e0zbPsc00tdee83jV155peDrlZIqyhxtHnovkXUOYhtonad6vuO9ib5m1lhLlHUPUCjNySxNO4wpE3oPqGOxlbf+/aD7Zd0bxhQlvXd66aWXPI7pH7qtKc/xePXfH68JLf1crwh9L2L797lz53o8ZMgQjw844IBkP02301R/M7NnnnnGY/2bJ3626vkcFKLnpnXr1snYN77xDY/33HNPj+P7+Pbbb3t81llneZx1DShVHsowsNIGAAAAAAAgh3hoAwAAAAAAkEO5So/K6rCgXX9OOOEEjw899NBkP136+O6773qs3X/MWKrW3EpZth2Xnuryw6wOGnlY0tbSZL1PulRYu4VdffXVBV9DlyqybLT5aScaTVvUGPmQ9b1YSLFzKi7Znz17tsdDhw5Nxnr06OGxdu7TtB2zNOW0XruWZL3/8T3Ra6POP01HNEtTgLV7V0xD5nqaL1ldL7VD5T777JPsp93aRo8enYxph6Gs7pX19FnQ+wp9f2KaREwJVTo3C8VmpaXIx9fQjqdjxozxOHa00g658Zqgc19TtirRUa4W6H2PWToXNc1p7NixyX7aLSx+nmqxe2ylxPsXTVeLnaBPPvlkj/Xv+vjZ1jRuvX8ph3ityMP5ZaUNAAAAAABADvHQBgAAAAAAIId4aAMAAAAAAJBDzVrTJis/P+aSbbTRRh4PGjTIY23/bZa2RtR2eLENLepHHvIQa4m+n5rPG9tiAlhxOt/KfS2Lr6d1EcaNG5eMaVtaneuxTgDX26bR+hja5jvW4VOV/EygvOL50VqLV111lcda18QsrVfyz3/+MxnTulFYltaP+eSTTwruV+4aYU2htTnmzJnjsV4DzNLPi7aeNkv/bcW2Hq83pdTPjJ8ZrWPD9bZ08e96/fv9iCOOSMb69evnsba913sUM7OXX37Z43JcF7NqCOZhXrHSBgAAAAAAIId4aAMAAAAAAJBDzZoelbXMLC5D0mVP8+bNa/S/m6VLDrVlqbbIXN7vBgCg3uj3Ymxtisoj7an26b2tpr48/vjjBfeLKYgoj2rOsWLbB8eUDG13rCUfzNK24aSnVw7X4srQv9enTp2ajGl6sM4Bbc1uZnbzzTd7XI57Fj3XmmaZF6y0AQAAAAAAyCEe2gAAAAAAAOQQD20AAAAAAAByqFVTcvVatWqVi8S+lVf+byme1q1bJ2OF8oDz0KprRTQ0NBTXm3A58nIO69T4hoaG/uV4Ic5j82Eu1gTmYg1gLtYE5mINYC6WR6xpk9WWvAJ/1zAXawBzsSY0OhdZaQMAAAAAAJBDPLQBAAAAAADIoaa2/F5gZrMrcSBN8fnnnzca17AuZXytXJzDOsV5bPk4h7WB89jycQ5rA+ex5eMclkksWVHldtOcx5aPc1gbGj2PTappAwAAAAAAgOogPQoAAAAAACCHeGgDAAAAAACQQzy0AQAAAAAAyCEe2gAAAAAAAOQQD20AAAAAAAByiIc2AAAAAAAAOcRDGwAAAAAAgBzioQ0AAAAAAEAO8dAGAAAAAAAgh/4/FvT5/2b+DWAAAAAASUVORK5CYII=\n","text/plain":["
"]},"metadata":{"needs_background":"light"}}]},{"cell_type":"markdown","metadata":{"id":"q3z_r68rX7KS"},"source":["Απ' ότι φαίνεται, ο αραιός αυτοκωδικοποιητής εμφανίζει παρόμοια συμπεριφορά με τον υποπλήρη, χάνοντας λίγο σε λεπτομέρεια (εμφανέστερα στο 8ο ψηφίο που είναι ο αριθμός $9$), κατορθώνοντας όμως να μάθει σε ικανοποιητικό βαθμό τα χαρακτηριστικά της εισόδου του συγκεκριμένου dataset.\n","\n","Ας δούμε την αντίστοιχη κωδικοποίηση των $10$ πρώτων ψηφίων ($32$ χαρακτηριστικά)"]},{"cell_type":"code","metadata":{"id":"65s7RSVjqQkB","colab":{"base_uri":"https://localhost:8080/","height":427},"executionInfo":{"status":"ok","timestamp":1642513755591,"user_tz":-120,"elapsed":266,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"86b3158d-2824-4c47-f1ca-d607be24b55f"},"source":["pd.DataFrame(data=h_sparse[0:10], columns=range(1,dim_h+1), index=range(1,11)).head(10)"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["\n","
\n","
\n","
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
1234567891011121314151617181920212223242526272829303132
10.6283530.1155000.00.3077800.00.3469571.5138030.0000000.2323280.0350220.2595290.2026720.8547140.2455180.0989510.8706290.6652050.1549170.3049440.4605790.00.8619460.00.9767870.0000000.1189580.0000000.2157390.0000000.8501500.3072360.155699
20.0258030.0590350.00.3956650.00.7021750.3021830.5874670.8133150.4671451.6389290.0000000.0000000.7355060.0000000.0000000.0000000.1110650.2095420.1109830.00.5323690.00.0000000.2450611.2515930.4584200.0000000.3300410.9925720.0534060.616016
30.2038481.3009400.00.1250090.00.2513550.6874110.1501860.8658850.1081820.4246410.0636620.0279940.5063770.3291550.3778620.3662260.2077400.0166750.2256390.00.0372220.00.0167710.0000000.2692930.0000000.0598670.1663670.2114500.4068870.458270
41.4340670.1252830.00.2604630.00.2970930.3094430.4444810.2587440.4235340.8517651.0774770.3473141.2355020.1127920.6560410.1745930.3294190.4385250.0000000.00.4794910.00.6476661.5207940.1333921.0025240.0039851.0133010.6628220.1664190.753081
50.2819140.0891410.00.2594860.00.3561090.2619070.2674800.2663150.3158220.6482250.7609380.7151090.1169730.6166451.2474650.1589020.1875640.5484651.1168750.00.3921250.00.3140200.3603020.4704630.0627940.2789630.7530500.4325620.8406630.002456
60.0449601.5387530.00.1729540.00.1759700.7063920.2010090.8792620.1393740.4490980.1958850.0847010.3744620.2863060.2921710.4992790.3100380.1180960.0376340.00.0633870.00.1799180.0991630.3121660.0750950.1495290.0000000.1382390.4114810.362416
70.1399560.0164410.00.7015610.00.3298621.0006400.3738040.1732781.0327040.6201420.0761121.1960620.3376630.5640820.2030330.5378150.0000000.3436101.1959770.00.6737850.00.1243660.2276110.1592470.2491440.2457030.0374660.0670510.5890670.039906
80.2600100.0000000.00.6870850.00.2717710.3541930.2086930.6346940.1774950.0000000.3243960.0000000.0000000.2489990.8298260.5667790.4592250.0000000.1696570.00.7319860.00.0000000.0530530.9767060.4362510.0000000.8664020.0000000.6804460.086612
90.2305370.2544170.00.0000000.00.7519730.0034840.2930360.9526271.0547640.5147851.5989580.4563380.3559450.5020920.2296310.7302800.0000000.7464450.5778980.00.0000000.00.2230020.1375970.7484350.0000000.2442250.2131540.3220970.9607530.715349
100.1784460.6587070.00.1097600.00.0455331.6574050.0000000.0000000.1638870.0876060.4434640.3670920.1442660.4664490.2965450.0000000.2646870.3171840.0325510.01.0038600.00.9055480.4420440.0000000.4108870.2562780.4959270.3066711.1273460.593291
\n","
\n"," \n"," \n"," \n","\n"," \n","
\n","
\n"," "],"text/plain":[" 1 2 3 4 5 6 7 8 \\\n","1 0.628353 0.115500 0.0 0.307780 0.0 0.346957 1.513803 0.000000 \n","2 0.025803 0.059035 0.0 0.395665 0.0 0.702175 0.302183 0.587467 \n","3 0.203848 1.300940 0.0 0.125009 0.0 0.251355 0.687411 0.150186 \n","4 1.434067 0.125283 0.0 0.260463 0.0 0.297093 0.309443 0.444481 \n","5 0.281914 0.089141 0.0 0.259486 0.0 0.356109 0.261907 0.267480 \n","6 0.044960 1.538753 0.0 0.172954 0.0 0.175970 0.706392 0.201009 \n","7 0.139956 0.016441 0.0 0.701561 0.0 0.329862 1.000640 0.373804 \n","8 0.260010 0.000000 0.0 0.687085 0.0 0.271771 0.354193 0.208693 \n","9 0.230537 0.254417 0.0 0.000000 0.0 0.751973 0.003484 0.293036 \n","10 0.178446 0.658707 0.0 0.109760 0.0 0.045533 1.657405 0.000000 \n","\n"," 9 10 11 12 13 14 15 \\\n","1 0.232328 0.035022 0.259529 0.202672 0.854714 0.245518 0.098951 \n","2 0.813315 0.467145 1.638929 0.000000 0.000000 0.735506 0.000000 \n","3 0.865885 0.108182 0.424641 0.063662 0.027994 0.506377 0.329155 \n","4 0.258744 0.423534 0.851765 1.077477 0.347314 1.235502 0.112792 \n","5 0.266315 0.315822 0.648225 0.760938 0.715109 0.116973 0.616645 \n","6 0.879262 0.139374 0.449098 0.195885 0.084701 0.374462 0.286306 \n","7 0.173278 1.032704 0.620142 0.076112 1.196062 0.337663 0.564082 \n","8 0.634694 0.177495 0.000000 0.324396 0.000000 0.000000 0.248999 \n","9 0.952627 1.054764 0.514785 1.598958 0.456338 0.355945 0.502092 \n","10 0.000000 0.163887 0.087606 0.443464 0.367092 0.144266 0.466449 \n","\n"," 16 17 18 19 20 21 22 23 \\\n","1 0.870629 0.665205 0.154917 0.304944 0.460579 0.0 0.861946 0.0 \n","2 0.000000 0.000000 0.111065 0.209542 0.110983 0.0 0.532369 0.0 \n","3 0.377862 0.366226 0.207740 0.016675 0.225639 0.0 0.037222 0.0 \n","4 0.656041 0.174593 0.329419 0.438525 0.000000 0.0 0.479491 0.0 \n","5 1.247465 0.158902 0.187564 0.548465 1.116875 0.0 0.392125 0.0 \n","6 0.292171 0.499279 0.310038 0.118096 0.037634 0.0 0.063387 0.0 \n","7 0.203033 0.537815 0.000000 0.343610 1.195977 0.0 0.673785 0.0 \n","8 0.829826 0.566779 0.459225 0.000000 0.169657 0.0 0.731986 0.0 \n","9 0.229631 0.730280 0.000000 0.746445 0.577898 0.0 0.000000 0.0 \n","10 0.296545 0.000000 0.264687 0.317184 0.032551 0.0 1.003860 0.0 \n","\n"," 24 25 26 27 28 29 30 \\\n","1 0.976787 0.000000 0.118958 0.000000 0.215739 0.000000 0.850150 \n","2 0.000000 0.245061 1.251593 0.458420 0.000000 0.330041 0.992572 \n","3 0.016771 0.000000 0.269293 0.000000 0.059867 0.166367 0.211450 \n","4 0.647666 1.520794 0.133392 1.002524 0.003985 1.013301 0.662822 \n","5 0.314020 0.360302 0.470463 0.062794 0.278963 0.753050 0.432562 \n","6 0.179918 0.099163 0.312166 0.075095 0.149529 0.000000 0.138239 \n","7 0.124366 0.227611 0.159247 0.249144 0.245703 0.037466 0.067051 \n","8 0.000000 0.053053 0.976706 0.436251 0.000000 0.866402 0.000000 \n","9 0.223002 0.137597 0.748435 0.000000 0.244225 0.213154 0.322097 \n","10 0.905548 0.442044 0.000000 0.410887 0.256278 0.495927 0.306671 \n","\n"," 31 32 \n","1 0.307236 0.155699 \n","2 0.053406 0.616016 \n","3 0.406887 0.458270 \n","4 0.166419 0.753081 \n","5 0.840663 0.002456 \n","6 0.411481 0.362416 \n","7 0.589067 0.039906 \n","8 0.680446 0.086612 \n","9 0.960753 0.715349 \n","10 1.127346 0.593291 "]},"metadata":{},"execution_count":18}]},{"cell_type":"markdown","metadata":{"id":"oiBdMrTDY74I"},"source":["Παρατηρούμε ότι οι τιμές που λαμβάνουν τα χαρακτηριστικά είναι πιο χαμηλές σε σύγκριση με την περίπτωση του υποπλήρη αποκωδικοποιητή (κάποιες είναι και μηδενικές), πράγμα που οφείλεται στην επίδραση του όρου αραιότητας. \n","\n","Ας υπολογίσουμε την ομοιότητα συνημιτόνου για τις αναπαραστάσεις των 10 πρώτων ψηφίων του συνόλου ελέγχου."]},{"cell_type":"code","metadata":{"id":"N62az1DNrDEi","colab":{"base_uri":"https://localhost:8080/","height":363},"executionInfo":{"status":"ok","timestamp":1642232208919,"user_tz":-120,"elapsed":337,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"4562ba2c-8a82-4f34-a016-4725eb1bea9a"},"source":["pd.DataFrame(data=cosine_similarity(h_sparse[0:10]), columns=range(1,11), index=range(1,11)).head(10)"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["\n","
\n","
\n","
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
12345678910
11.0000000.4439200.5396380.5942580.6315030.5288080.7208120.5857080.4729400.740887
20.4439201.0000000.4798120.6185250.5230490.4292000.5413400.4854860.5707220.408226
30.5396380.4798121.0000000.4165070.4849190.9689400.4435840.4782210.4775140.575004
40.5942580.6185250.4165071.0000000.6417250.4117740.4928740.5245960.5878100.612528
50.6315030.5230490.4849190.6417251.0000000.4372440.7196400.6996240.7823370.617243
60.5288080.4292000.9689400.4117740.4372441.0000000.4281180.4429480.4742160.588981
70.7208120.5413400.4435840.4928740.7196400.4281181.0000000.5989690.6383770.658630
80.5857080.4854860.4782210.5245960.6996240.4429480.5989691.0000000.5743740.558564
90.4729400.5707220.4775140.5878100.7823370.4742160.6383770.5743741.0000000.476770
100.7408870.4082260.5750040.6125280.6172430.5889810.6586300.5585640.4767701.000000
\n","
\n"," \n"," \n"," \n","\n"," \n","
\n","
\n"," "],"text/plain":[" 1 2 3 4 5 6 7 \\\n","1 1.000000 0.443920 0.539638 0.594258 0.631503 0.528808 0.720812 \n","2 0.443920 1.000000 0.479812 0.618525 0.523049 0.429200 0.541340 \n","3 0.539638 0.479812 1.000000 0.416507 0.484919 0.968940 0.443584 \n","4 0.594258 0.618525 0.416507 1.000000 0.641725 0.411774 0.492874 \n","5 0.631503 0.523049 0.484919 0.641725 1.000000 0.437244 0.719640 \n","6 0.528808 0.429200 0.968940 0.411774 0.437244 1.000000 0.428118 \n","7 0.720812 0.541340 0.443584 0.492874 0.719640 0.428118 1.000000 \n","8 0.585708 0.485486 0.478221 0.524596 0.699624 0.442948 0.598969 \n","9 0.472940 0.570722 0.477514 0.587810 0.782337 0.474216 0.638377 \n","10 0.740887 0.408226 0.575004 0.612528 0.617243 0.588981 0.658630 \n","\n"," 8 9 10 \n","1 0.585708 0.472940 0.740887 \n","2 0.485486 0.570722 0.408226 \n","3 0.478221 0.477514 0.575004 \n","4 0.524596 0.587810 0.612528 \n","5 0.699624 0.782337 0.617243 \n","6 0.442948 0.474216 0.588981 \n","7 0.598969 0.638377 0.658630 \n","8 1.000000 0.574374 0.558564 \n","9 0.574374 1.000000 0.476770 \n","10 0.558564 0.476770 1.000000 "]},"metadata":{},"execution_count":17}]},{"cell_type":"markdown","metadata":{"id":"m6hlKn6GelyD"},"source":["Παρατηρούμε καταρχήν πως το εύρος των ομοιοτήτων συνημιτόνου είναι μεγαλύτερο σε σύγκριση με την προηγούμενη περίπτωση. Αυτό οφείλεται στην αραιότητα που επιβάλλει στις παραγόμενες αναπαραστάσεις ο όρος ποινής, κάνοντάς τες να λαμβάνουν τιμές πιο κοντά στο $0$.\n","\n","Παρότι το σφάλμα εκπαίδευσης ήταν ελαφρώς χειρότερο, παρατηρούμε ότι η αναπαράσταση που επιτύχαμε είναι καλύτερη. Για παράδειγμα, το πιο όμοιο στο 3ο ψηφίο είναι το 6ο (ομοιότητα $0,97$) και το αντίστοιχο ισχύει και για το 6ο ψηφίο (και τα δύο απεικονίζουν τον αριθμό $1$). Ωστόσο, αντίστοιχα συμπεράσματα δεν μπορούν να βγούν και για τα υπόλοιπα ζεύγη ψηφίων (5ο, 7ο και 8ο, 10ο), μιας και παρά την αύξηση του εύρους των ομοιοτήτων συνημιτόνου, πιο κοντινό στο 5ο ψηφίο είναι είναι το 9ο, ενώ στο 8ο το 5ο. Συνεπώς πρέπει να αναζητήσουμε καλύτερες υπερπαραμέτρους για τη συγκεκριμένη μεθοδολογία ή να προχωρήσουμε σε άλλες τεχνικές αυτοκωδικοποίησης\n","\n"]},{"cell_type":"code","metadata":{"id":"xpBrKcldhoHP","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1642514037958,"user_tz":-120,"elapsed":264,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"4a54c96c-c87a-48e1-89c6-2aa4d302350a"},"source":["print(h_sparse.mean())"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["0.35433093\n"]}]},{"cell_type":"markdown","metadata":{"id":"PS3upZqxhohQ"},"source":["Τέλος, παρατηρούμε ότι η μέση τιμή των χαρακτηριστικών του αραιού αυτοκωδικοποιητή είναι χαμηλότερη σε σύγκριση με του υποπλήρη ($0,35$ και $7,79$ αντίστοιχα)"]},{"cell_type":"markdown","metadata":{"id":"gQdSBZGOh4mD"},"source":["## Αυτοκωδικοιητής απαλοιφής θορύβου"]},{"cell_type":"markdown","metadata":{"id":"ajqqcs3PwhKU"},"source":["\n","\n","Σε αυτή την περίπτωση επιβάλλουμε στον αυτοκωδικοποιητή να μάθει τα χαρακτηριστικά της εισόδου $\\mathbf{x}$ , \"αλλοιώνοντας\" την μέσω της προσθήκης θορύβου κανονικής κατανομής με μέση τιμή $\\mu=0$ και τυπική απόκλιση $\\sigma = 1$: $\\mathcal{N}(0, 1)$. Επίσης ορίζουμε τον βαθμό συνεισφοράς του θορύβου (noise factor) στα αρχικά δεδομένα ίσο με $n_f = 0.3$. Έχουμε δηλαδή:\n","\n","$\\widetilde{\\mathbf{x}} = \\mathbf{x} + 0.3\\mathcal{N}(0, 1)$ "]},{"cell_type":"code","metadata":{"id":"VqPBlVbYiH03","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1642514110828,"user_tz":-120,"elapsed":2984,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"4c95b264-f3e8-4115-e911-5282ed885a14"},"source":["noise_factor = 0.3\n","x_train_noisy = x_train + noise_factor * np.random.normal(loc=0.0, scale=1.0, \n"," size=x_train.shape) \n","x_test_noisy = x_test + noise_factor * np.random.normal(loc=0.0, scale=1.0, \n"," size=x_test.shape) \n","\n","x_train_noisy = np.clip(x_train_noisy, 0., 1.)\n","x_test_noisy = np.clip(x_test_noisy, 0., 1.)\n","\n","print(x_train_noisy.shape)\n","print(x_test_noisy.shape)"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["(60000, 784)\n","(10000, 784)\n"]}]},{"cell_type":"markdown","metadata":{"id":"IIE7OrGhkqMq"},"source":["Ας δούμε τα $10$ πρώρα ψηφία του συνόλου ελέγχου μετά την προσθήκη του θορύβου"]},{"cell_type":"code","metadata":{"id":"lo-Ht040i8IJ","colab":{"base_uri":"https://localhost:8080/","height":129},"executionInfo":{"status":"ok","timestamp":1642514114878,"user_tz":-120,"elapsed":753,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"30cfcbfd-1e29-4b99-837e-060adc181933"},"source":["display_digits(x_test_noisy)"],"execution_count":null,"outputs":[{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAABG0AAABwCAYAAACkaY2RAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2dd7wdZbm2H9IDCRFCACUQhQ+lCIoKUqSoIByQIlV6B+nSOTRJqNIPvUnvTQTpVZAgQXoNTalSQgkkIYSE9f3BmTfXe2fPm7V31g4753dffz0rM3vWrJm3zeS5n3umRqMRxhhjjDHGGGOMMaZr0e3rPgFjjDHGGGOMMcYYMyV+aWOMMcYYY4wxxhjTBfFLG2OMMcYYY4wxxpguiF/aGGOMMcYYY4wxxnRB/NLGGGOMMcYYY4wxpgvilzbGGGOMMcYYY4wxXZAe7dm5e/fujZ49e0ZExOeff1673yyzzJJ9/ta3vpXiDz74IMUffvhhe76+KWadddYUf/LJJy0/Pundu3eK9Xr06tUrxX369Kk9xqRJk1I8duzY2v0ajcZMHTlHZaaZZkoe79/4xjeybV9++WWKO3rtBg4cWHuML774IsWzzTZbij/66KNsv9lnnz3FbCP9+vWr/d6ZZsovz6efftrkGdfDdvv222839Tf9+/fPPrNdTJgwYVSj0Rg0zScW+X1sz/l05Lp065a/22U7+fa3v51i7c+t7n8DBgxI8ejRo7Ntpb7I8//mN7+ZYo5FERHjx4+v/e4555wzIr76TZ999llL+uIss8zSqPpg9+7ds21vvPFG7d/16DF52Ob40Wg01SQiIu/7Y8aMSfHEiROz/WaeeeYUjxs3LsXsGxERH3/8cZv7ReTjH++N9lmOmfpb+Js5TvJvIiImTJgQTTLd+yLHtYjWzH+lcZRU7Tcib/dsPxERc8wxR5v7RdS3r9L4oPB+Mdb2r/2bVG1h0qRJ8eWXX7akL/bp06dRjZWjRo3q0DE4Pmk/KrV79sX3338/xfPNN1+23+uvv57i0vqC/VmvK+E6TdcenKtL9O3bN8WfffZZ7X6DBuVdjWPE2LFjW9YXe/fu3ajGLI5JCvtDRH7Pec30OgwePDjF7777braN7V77VTOw70XkY1lpLuVv0XvA+Z5jeUT+2ziev/baa9l+nDP/85//1J5HZ6xRlbq1oVI3b7UHjmulMW3uuedOMftvRET1vBQx5fqC/bZ0fLYDXbOzXfB4+pv5d9ov5Jp2yrzIcSIib6c6L7Lv8HfouMn7o3NQ3T0vnUcJ3kcd2zkvlsZUouMyz4v9Utc37M86hlVt4YsvvoiJEye2pC/26tWrUbUrfXYojQt8Dixd41Lf5P0urf3bs+6t0H7EPtHsnFZirrnmSrHOE/xd2vZlzdVmX2zXS5uePXumSeuVV16p3e8HP/hB9vmQQw5J8eWXX57iSy65pD1f3xTLLLNMim+//fba/Xjh9KY3O1jPO++8KX755ZezbZwEF1poodrvYkcYPnx47Xd1Bj//+c+zz5xUbr311g4dc80110zxHXfckW3ji4+VV145xddcc0223+qrr57iSy+9NMU/+clPsv14LTmoRkTce++9Ke7IAioiYqeddkox23CJpZdeOvv84osvpvg1XQ1NB5Zaaqns8913393uY+hLWLbZoUOHpvjKK6/M9utoG6pjxRVXTPGNN96YbSv1RQ7Cu+22W4p1/Hn22Wdrv3vTTTeNiIjLLrusHWdc5hvf+EZqY/py7fe//33t39W9GG3PBPPLX/4yxX/7299SrA+siyyySIr/+c9/pnjHHXfM9rv55ptTPGLEiGwbX+yxC+jChfvpApcLFI6TfIiKiHj11VejSaZ7X+S4FpGPbR2lNI6Sqv1GRFx44YUp1hc96667boovuuiibFvdf9SUxgeF8yLvHRdlEXl7Uqr2ry+VpoX+/fvHOuusExER5513XoeOsfzyy6dYr+vIkSNTrAvx3/zmNyk+/fTTU3zggQdm+/3ud79Lcd36IiLiwQcfTDH/EysiX89wrmLfjii/NCbf+973UvzEE0/U7rf++utnnx9//PEU/+Mf/2hZX5x55pnT2HbdddfV7sf+EBHxpz/9KcX8zyH9zxqOyyeffHK2jeNvR9rmeuutl33mWHnbbbfV/t1GG22U4ueeey7bxvmeY3lExFtvvZXiQw89NMXbbrtttt8OO+yQYs73Xwerrrpqiq+44ora/RZeeOEUP/roo00fn2v/0ksQsuWWW6b47LPPzrZxjHvmmWeybQsssECKSw/4//73v1O80korZdvuuuuuFPP+6hzMtf6f//znbJustztlXvzud7+bfX7yySdT/F//9V/ZNr505rjE/7iJyNdMOqY+9thjbZ4Hx6uI8phF+BCuL+Y4Ly622GIp/sc//lF7PJ3veO/4AkRf3PO5ZuONN862Vc+7bC/TSp8+fdIzhD47bL/99ikeNmxYtm2NNdZIsY5JROcdwmc6tl9dG7TjP+oSXP9G5HNFs3NaiS222CLFxx13XLaN/8HM5+WIfG0WNWvUmdrzlqrZ/1HcfPPNs89PP/10ivkQqwNh6S36T3/60xQ//PDDzZxGfP/730+xvl1+5513UqwPTFxw8QUUB5rpTav+F6N3796NaiLRge6FF15I8XLLLZdt4+djjz229vic6EqZC0RfxnDA5SKz9HZUB8EFF1ywzb/jJKffrQMIF8OcYHXQOPXUU1PMlwIREcsuu2yKhw8f/mij0ch/bAdpti/yxVNExJlnnjnN3/3jH/84xVwQ/fCHP8z244DHBxNdNDRL6X+LSOk82D5XWGGFbD/+Fi5UIyKOPvroFE+P/1Fce+21U6wLML6EfO+992qPz/8J0f8d0oVHRam98Prr8bjQKsFFCF/SROQPEaX/FCB6DE56usDlS7rNN9+8ZX2xb9++jWp8KL34K3H88ceneJ999sm2ccGiC9z99tsvxfwfX85vJfTFGR/ydUzld3NO1//Y4MOJZgjw/vM/TnS+54tWbePVeTz44IMxevTolmXaVG3pRz/6kW5L8QUXXJBt23rrrdvcplkTfBmqGaOrrLJKijk2cv0Skb9AOOKII1KsLw+Y1XLYYYfVfhfRFz8cO/igUELnPs6LU6FlfXHQoEGNaq4599xza/fjy86I+nWLrhdKVC/9IiJuuOGG2v247uI1O+GEE7L9+GKOc25EvnZu9j9i/vu//zv7zDmtbk5XtF1XY+zdd98dH374YcvnxT333DPbxpciHc2g4X8A8IVXRH2b3WqrrbLP8nCV0LFjySWXTPFDDz2UbWM74AtUjq0R+X+qKMwo4gt3Pb8NNtggxbr+4oPjTTfdNF3WqPy9+nKj2exRov85uddee6WYLzc6kpUxNfjCk+uq0lr74IMPzj5zPJ9nnnlSzDWRoi9t+BKzM9aoOm+VXhJyPuFzZYn5558/+8wEjNJ/lvI/Mffff/8Un3POOdl+/M9c/gdwRH0fO+igg7LPRx55ZO151KFzDf/TUceLl156KcXvvvtum33RNW2MMcYYY4wxxhhjuiB+aWOMMcYYY4wxxhjTBfFLG2OMMcYYY4wxxpguSKfUtCnxr3/9K8Xf+c53pvVwU0CtJPVhWhiOhUtLTiXNFnXSujgsvMcaKKw7EJHrcrWAKpkedTRKFa9Z64I6Sy3UVyosRajz0yLCdQVs9dqxboPWeqAOnxp11TL+/e9/TzH1xxF53RRWLNdaPSXdLc/r1ltvbZleeMCAAY2qzpDWeOoMVzbCei9PPfVUilWbTM0t9bZ/+ctfsv2oPdWK+PxtLGimdYVY+ExhccFf//rXKdZixiXOOuusiIg46qij4rXXXuv0vshCaW+++Wa2jfU+WDBUXWTYT0u1ang8FjuNyPW3rMuhNYVKhTepOS7t99vf/jbFWtS6WUpaedYIeeaZZ6Z7fanS+EUNuNaI4XijdZg43lI7rveRcyH3K2nmdVymqwVrCZXmSK0rdN9999XuS/bYY48U0zUpYvL4s/vuu8eLL77Ykr4411xzNarxSuvF83ppkeW661eqv6Y1t1jImUWEtT9zfub3an0ponVxWHuK9TzUxYK1AToK248WACWd1Re13hWvp9ZZqDtXbXu/+MUvUlyqW0PUmKOuNqLO46zjuN1222XbaPLAOjNarJV1U7jWicjHSq6HtaA562ZpseSqvZ544onxxhtvdPq82GzdLs75rJESka+PtEYPa5L87Gc/SzFrJEZMWduqDtbw0rZUB+sgRuRFsllgWeH4oG5KrB+m7VHW2y3rixxT77zzzmxbqTgt60yyuLvWBGLhbx1frr766jaPrYVfb7rpphTzGqkRBOfk//mf/8m2scgya+loMXfW6tR5kAW2mx1X1PSkKu4+YcKElrkq9ujRo1HVudPnHI4FWvSd4xALdbMtRzTfP3bZZZcUs1h/RD7O814suuii2X4c80smKawpqEWOac6gLlZcs7Am41//+tfa79L1Eddpb731lmvaGGOMMcYYY4wxxswo+KWNMcYYY4wxxhhjTBekZfIoST3PttVZeTOdKGLKtLNmGDp0aPb5D3/4Q5v7qZ3Yq6++mmJNIeb5M71UfxctFNVGj3aNTA/TlH3aNarEhylcrZJHdevWrVGlrjElNiKXSag8immFVRpexJQ+9v/v//2/FGtKKX8Pv3vnnXfO9qPdMdNBP//882w/2utuv/322TamIg4ZMiTFmvpOSmnMtL/VVEzKFzSFkPKxSZMmTRdJRsmCvA6miUbk91vTiyl1Wm211Zo6fom11lorxSo9Y8ok5RlMU4/IJThqWcp+y5TGwYMHZ/u9+OKLted4yCGHRETEeeedF2+//Xanp4GTSppVQekL5V6ahtm7d+8Ua9+pQ2USlLBS8qE2s5S10TY6IreV5hij7eqBBx5IcUnyQbbddtvsMyVvdbbm/0un9EVaJUdE/Oc//0mx9kWm2nJ+UokD06x1XK6DNuERuVyUfX2bbbbJ9itdM6atU8516aWXZvsdddRRKaY9fcTkfhSRy1W03XHc3HXXXbNtp512Woo7Qzas9sq8JmqZrenezaCyTI5/Jdj/uGZR2UuJTTfdNMW0UVU5Ba1IOY5ERBx++OEp5jhMmVdEPv/rOkFkLi3ri7PMMkujapsqE+ZYpnJ29g+VwBGub1TOznbD8VBT8SnF4nih6zGuFfm9EflcyLWsQomVyq8I53S1HqeFvNq/V1bSN998c3zwwQfTdV5UmaG2sfbu1x7Yfji3qpyiBCXLlCs3a0essP/R4jsit6Nn3AYt7YvVOMV2GJGvUTfbbLNsG/tLSU7N/qJSJMJnOH0m4TME7dNVylzqY5wXmy0PodLXqtSBfnfJ6lrbSbW2Gj58eIwePbrlfVHHIK4BdO1JqVnpHUPpvcFhhx3WZqzSXfZnll/RdSjvDUs8ROSyKsqjVE7HZ4SRI0dm2yjH4vO/SgPbgeVRxhhjjDHGGGOMMTMKfmljjDHGGGOMMcYY0wXxSxtjjDHGGGOMMcaYLki7atrMOeecjcrKmhrMiNyKVDXWamFaobUnVEveEajN5vdqzY7rr78+xe+99162jXUIWBuFNWwiIo4++uimzon6cP0u2vkussgi2bbKwu+f//xnfPLJJy3RKHbv3r1RaR/Hjh3boWOU7F5Zf4P6woi8tsxWW22VYta5aA+sL3DooYdm2/iZNqerr756tt+wYcNqj0+t5I477phi1YZTO/nJJ59k26j7fOGFF1qmF55tttkalS202u01C7XEWpeCGk+t4cNaJBdffHHt8VkzhrUU/vjHP2b78RqpBSfrnFBTS0u9iLwtqDUk6y6w9k2ztp1KZ9TRoI42IuKSSy5JsY4Z1M5S112yMVRtLsekY445JsWl2je0QNXvYh0W7c88X2rIFWrD1a732muvbfNvtI4G61aoRlrolJo2JZtshXaPJStsjpUXXnhhto19kfW0SvXiFlhggRRrPyJ6DzhXNVtzqFnUIpt9vVQnoDP6Iq3nIzpmP6+1b6hr1xobnD/uv//+2mNyTKa2Xvs272nJWrdZtN6Q1oqp0PZSsoIXWtYX+/Xr16iuL69XRH7Nhg8fnm3jOph1FirL4oqDDjqo9rs5pnLdqDUTaVXNc+TfRET069cvxaNGjcq28e9Yi09rb+gam9AWmXWidAxjfQat3Vi1jXfeeSc+//zzlvTFAQMGNKoaH6U5rVm0zhjHZL2urDNT2ZlHRDz11FNNfZeuPZZZZpkUa50UtgNZJ2b7sf1wfov4qpZQBZ93tt5662w/WkrzXkdM8bzWsr7Yt2/fRjWOc+6IyOvqaJ1J1nvheEjr+Yi8fqnWxdH1bIXWDb3jjjtSzPprWvPknHPOafN4EXldM9bY4xouImLzzTevPQbHlZNOOinFun5YcsklU6xjGOmMeVFhe9a6bKXr1Sx19W50rXz88cenuFTH8cgjj0xxaRxvtgZqCY45fJ+gaO1BrnU+/fRT17QxxhhjjDHGGGOMmVHwSxtjjDHGGGOMMcaYLki75FH9+vVrVKmYmpp18sknp1glE5r2WaFpo0zD1bR3pkkuvfTSKVYLbaYBMiVZJVq/+c1vUrzuuutm22jvTBmYpmryfH/1q19l25hSffnll6d4k002yfYrSQeq3/n000/HmDFjWpLu1rt370aVuqUWkyXr0LrUX4Vp7toONI27QtMIl1122RQzjfLpp5/O9uP91XR0Qvs5hemSpVTcNdZYI8VMm43IU1s1FY6pxi+//PJ0sfzubOpSedUSkNeCUjz+vR7j97//fbaN9qO0RdR+T0o257SaVJgGTsvhiIjFF188IiJeeumlGDduXEv6Yv/+/VMqv0ojmfbM8SiiNZKHjkCLQ5UeELWR5vlSWkg5VEQ+rmsKdx2a7lzqw5XcNCLi3XffnS59sU6uq+dDa8+VV155ms9J5/XSGFiHShU5tqvdMSnJuQjTqyntishlBSodq1KgJ0yYEF9++WWnp4HvsssuKR4/fny2janxJftSSocoJY/I58lf//rXKVYb1Tq45onI5YkqgWb6/ujRo1NM+W9ExD333JNiyktL0M49IrcGp51uRMS4ceP48WufF+vGfrUqpqRFU+c5r9GmWdu2zn8Vuh6mFEKlvBxXeP91TVeSKay33nop7qjEmrRKktGjR49GJQ3T38N+tcUWW2TbuNbieKp9lmUOXn/99WxbJTmPyNuzWsRzzdK9e/cUU6oWkUtuVK7BY7B/SN8owjXRZ599luLSGl2htHnixInTpS9SmkeJb0TeLrXcQKvhOuatt95KsY6bbIeUZUXkYypRG3LKVnnNI3LpOaVtp59+erYfn0NKkurOkEephPnjjz9uxVckVIa78MILp5jPXPpugPeKzyClNYqW02CZjDnnnDPFWpqA6HzH51iOK7xnEXmpll133TXbxnO+7bbbLI8yxhhjjDHGGGOMmVHwSxtjjDHGGGOMMcaYLki75FFMlSqlu6rbDNPqydChQ7PPTDvTtGqmjPGcNfWIsgKijiZMVaT0JyKvwF/nlBCRyzDuvvvubFuzEoYVV1wxxUypjZgsE3ryySdbJo/iPTzuuOOybfwNt912W7aN6YG890sssUS2H2VhzbYtTd2nHIQp3JqiylSy559/PtvGKuJ0LtJjMCXv3nvvbep8FabHMr02Ik/l+/Of/9wpqaeatshrVroHq622Wor1frMa/I033phtY1og05XVWaLuePPPP3+27fzzz0+xurzx/my55ZYp1vRfVpDX6vKE0juV1NFxoOSqMz0q85eguw0dztR1oiOo0wbTzNkvVR5FpxN1wOEYV5K2lqBshG5wKks98MADU8w014hcpvDFF1+0rC/27NmzUUmJVH63wQYbpFilFupCV6EyQ7ppqEvGAQcckGLOfeqSR9cRpmbr/Faa7+pQaQXlxiVZFh2t1O2qWaZHX6RkkzJwhRIodVIqpbITOklVksyKknyZHHzwwSmmRKnEPPPMk32mREj7UV3KOGVkEVOm9hdoWV/s1atXozpflS2UnCIJf8df/vKXbJu64JC6uVbdEtlnSy4jHEtKkt9mKa0F2Gd1DmZbVslB5X7SSodT9kVtl5SwdAYsm0DHKF0P0HGRcjfK+SPKTjR8JuHfXXHFFe097SnQtk45iMqSxZ2vU9aoXCNH5LKxNv4uxXQUViclrvlVwsn1LPsb71tEvh6kTEufbynnUt5///0UDxo0KMUqkeWzx5NPPplt4xg733zzpZiyzYjyfHrGGWdExFe/8bXXXmtJX+zTp0+jevYryY24rovI5UEc13QM4rOLyq04F/IZTp9H6GT57rvvpljdntmf+YwUkbvITZgwIcXqxkd0LaClHSqOPfbY7DPLejBuA8ujjDHGGGOMMcYYY2YU/NLGGGOMMcYYY4wxpgvilzbGGGOMMcYYY4wxXZAO17RRy29qMks2WWprSM4+++wUa+0R6ga5n9p/0RqRtQH23nvvbD/WalCtOGtuUNuoGkXWcVANOzWWrKWg9R6o9WTth4j8t7RKu9+/f/9GpRPVGjqkZMtMnS41hBG5vaK2LR6Ddm5qWbrYYou1+V133XVXtt8NN9yQYrUDZz2AZm1U1T6YuthSP6F1La1qI6ao89IyvfDgwYMblV3cWWedlW2jVe+jjz6abaOemVpm1V2yBgbrckTk/YM6bdZaicjrY9CKeamllsr269WrV4ofeOCBbBv13aydwfEgImLgwIEp1jGBtT5KWmr2de3PrL/SGXU0tN4J7SLVfpzsv//+KdbaCSUWXXTRFLOvaB0NtSSs4NgXkeuKaYsbkdeWYJ0GrX1GKiv0Co4zWvuBUMuvto5Cy/riHHPM0ajGGK3fxrG/rrZbRMTGG2+cYtbjiijbZg8ePDjF++67b4p33333bD9q4VnzhJbVEfm1pbY+Ykp73LbOISLiyCOPTDFr0UXkNSNYL4Ttpz20qi9269atUdUO0n70r3/9K8UnnXRSU8fTMfl3v/tdU3/HsZu1myLyuXWRRRZJsVoEs//pWozrKraJ9tScYN/Xc6xDbexlLm9ZX+zdu3ejqhOjNUSahbbNWnuC9dLuuOOObBv76c4775xird3FtRXHQJ3TuN4sWR/zftAKPCK3bmd9kIiIzTffPMWcI7leisjXTGpfTqZ3rTfW+oqY0tq74txzz80+b7/99ineaKONsm1XXXVVU+fI9eBFF12U4q222irbj5bcOn5yjmd9G10fsY4P71lEXge0VB/k2WefTTHn/ogp+vN0sfwuQet2rv+1pgptsrWv89rWrXkjIvbcc882/0bnT60zR3jvtL5kHVpT5aWXXkox+ynn0ogpbabr+LrrLm633XYpZlu89tprs/247tFaTmyXfD7h2KpsscUWKdY6RBwH/v3vf2fbPv300xQvueSSKdZnH7ZNrbFEuHaqWzdFROywww7Z53POOYcfXdPGGGOMMcYYY4wxZkbBL22MMcYYY4wxxhhjuiAdlke1sS3Fmt7NtEVa/9KeNSKXCGgaMtPYTjnllBRrGtsmm2yS4g8++CDFmoa03nrrpVhlQrS+ZioW08ojIlZaaaUUqzyHqbJMUWY6WETz9n6tSnfr2bNno0rh4zWIyCUImvrLFGzKiJgGF5HbuGo7YPphyZ6Y0gimhup9ogWxWuNRJsEU0hIqBTnttNNSzBTIzz77LNuP312SskQn2Smuv/762TZKe2iFXUJt79VSmzB9l6nkmpJ7zTXXpJiySLVN5b1Sa9M6i0NtW59//nnt+TYLv/uSSy7JtlXygY8//jgmTpz4taaeNgttJDkWRuRyMtoRc+yLyGUrtMRU+RilnZTARNRbwavEipJStZ1lX9fU4g7Ssr44ZMiQRiVT07Fm7bXXTvEtt9ySbaMcj5ayalV59NFHN3UelPE88sgj2TbKJjheqKTxyiuvrD0+526eux6DUg6mK0fkYzvt6ktoW2Nqc2ekgVOWHJHLuFRGxPtW+j20LFWJ7gILLJBijrsqH+T1P+6441K8wgorZPsddthhKa4smduCqd48B92mcyvXBlwLUALbTr52SUazcL3JtU5ExIsvvpjikvSTc6FK7utQ2QVlbhwb1fqYci7ai0fk0kW2ax17l1hiiRQvuOCC2TbKAFvVF/v06dOo+jx/W0TEiBEjav+O0kJa+Jbgui4iH69OPPHEFKsFNNcblFbwekdEPPjgg02dr0ro6lBLcZYIKMG1QKl8RbSwL/bs2bMx++yzt/mdlMTTYjki/43N/j7KqCJySTbnRR3bafnNe6WyT675dT3D51b2Px032de5/orI50VdgxHeR5YfiJi8BhsxYkR88sknLZ8X1daba0OVnVHWddRRR6VYbbG5X7NrekqDI/L5TuVMdegziEq866DNt8q09Pzr/p1SL5Z8iMhLT1x44YWWRxljjDHGGGOMMcbMKPiljTHGGGOMMcYYY0wXxC9tjDHGGGOMMcYYY7ogLatpQ50u9dsRU2r0K1Q7S/va5557LttGi0hq3NXicNZZZ00xdYOqHaYFntr+EVo+qp6dv5m1CyJyvSH1sKzREpHbPKrus6od8swzz8TYsWNbrlFUW93nn38+xbR8jshr0FAbXbLfbZbTTz89+7zLLrukmHpItd2jfpG1Bdrat0JtbNl+1L5ca9fUweuolt9nnnkmP3aKdl/tZNVullCHv8cee9Tu16yumG1brWepgWVthVKNDh2PWNOGVpXLL798th/rRKgVKFl99dVTzNpYEbktLesTREzW137wwQfxxRdfzBA1bYhaL7NWAOuHqZZb685UaF0ZHk/HWo6TrEeitUrq6hdF5P2WfVatG1lfpVSfJVrYF3v06NGo5h21l2X9GB1vafM5duzY2uOX6hGwz/H6aa0GQt009eAREZdeemmKZ5lllmxbXW2ie++9N/vMWm96T1mLg7Ug1J6d/Vnb2ne/+92I+KoGwejRo1veF0tW56zFEJHXY6AuXjXzZL/99ss+c46gje2rr76a7cc6M6VaKFxXqX0zz3fuuedO8bzzzpvtR2thWslHRPTo0SPFnGdLdeqmQsv64uyzz96oxnGtgVeq5XHQQSamSNkAACAASURBVAelmHUWtOYga8Tddttt2TbWd2MdR609QZq1kC3BGmKslxAR8fjjj6dY74/2uTq4tmV9J+XrthkeOnRoillbrD129lx/cA2gcxXb0vDhw1Os95pjqI617N9cR9ESPiJ/ftKaHSWbb8Ixn/NJRMSqq66a4qFDh7Z0jVqN/2pVve6666ZYn/VYl5TXT+dI1ik95phjsm28Lqy7dfvttzd17qyhGpHX6yqNc6V6q53N4osvHhFfrSvGjRvX8r6ocwRrlDYLn/EjIv74xz/W7stabZyDdE6rQ9cNRxxxRIo5VkdEbLbZZm0eg+N4RF6nk3NkRMSGG26YYvbFO++8s/Ycq5pPFTJ+uKaNMcYYY4wxxhhjzIyCX9oYY4wxxhhjjDHGdEE6LI9S+6+PPvooxSNHjsy2/eQnkzN8aA3GNNuI3EKsBNPMKZmIyK3emDaqqVI77bRTm8eLyFMVmQKltrm0AK9S0ypoF00plqZ20d6tJMfpjNTTKs28gumgKotgWjslUby3Efn91RTDa6+9NsVMC9Y0cEpp7rnnnhSrFR6lAbRDVZhSyvS29sCUf1okTw3+lnvvvbdlqaeDBg1qVJbtZ599dtN/V9cX28MBBxyQYsr91Cabqf5///vfa49HCaJaiqrVZkVJwqA257ThZJqzSmuIphBX48w///nPTrFT1DRJpnSr7Iz2vLSWVbt5WljSkjsib8Mcd7QtcYzmOMl7G5GnKqukgLIRSjDXXHPNbD+1QiYcSyjnKqGyV8qRHnvssZb1xb59+yaL2o5KRDgfiaQyY/vtt88+n3POOSmmnab2D0oSS3a4P/7xj1O8xhprZNuGDRvW5t9ou6N8VuVRddbtKnOmXaraUVP20hnzIq3tIybLlCOmTK/n7+NaipLAiFzOpGsAtvtTTz01xWptynR7rik05ZzrEp3HKZ3inDnXXHNl+zFdnG0sIrfX5RqLbSciX4tRlhqRS7OOO+64lkoVq9+sFtfsm81KZtqzPqbkg9IBnWe5ptQ5s1koS6M0RCVglEuV5jvOD9ttt122jedL6XrEZEnADjvsEC+88EJL+uLAgQMblaTlsssuq93vl7/8ZfZ5nnnmSfHFF1/coe+uu98qG2Z/5vysZQX4mfNPRL4ePvDAA1PM8SYib0u63tYSEy2gZX1xwIABjeo58Z133sm2PfXUU7V/xzUNxw2d93mvKOuNyOX9XP+1Z61MWHJAyxGwHAjHEZVnU7qtYwKfnwcNGpTi0npCJXvVnP/YY4/Fp59+2vJ5Udsex37KxyJyC/vScz1LGbB0g25bbbXVUqxrc46hjzzySIpVbsp7yPknIpdpUc6k4zOl4FoKg+dbkvpzrtWSHILlUcYYY4wxxhhjjDEzCn5pY4wxxhhjjDHGGNMF6TH1XSbzrW99K6VxazXwrbbaKsUqj3r66afbPJ6mTTHdSp0qDj/88BTzuzVli2lxdHpQpxxNxapjlVVWSbE6QjAVVSvx0xXqxBNPTPFee+2V7cfUOnWWao8Mp1lmmWWWlLr98MMPZ9uYmn3hhRdm2x599NEUMx1bj0E0nZFpYUxlVXmUtq0KlafxfrDtRORpikyvp8wrIpc9aaopq85TfqNQZqZSu6uvvrr276aFTz75pOlK+ISptpQqqFMMJQ+9e/fOtjG1nKnT6iRWl2qsDhSUFlI2V2K33XbLPrO6PNuxUkoRp4uCXtuqGnxHHT7aom/fvkkORte7iFxOps5PTLvlNdd7SOmoVrq/8cYbU0yZhMop2K/oqqJSJnVNInUpvtrfOJ5S2hWRSw9+9atfpbhy9aqgdIaOg23t2yrGjx/fIVkU3co0vZiwT9ABISKXtFJ6pvMbpWhsayo/oKyqJDGgO4JKMthmOM5HRHz55ZcppjxKpbqUCDB1PCLiRz/6UURMk1vRFMw///zJqZBp7BH5WMA07YjcQYhyHHV34lh73nnnZds47tx8880p1jGOUDaqzilkm222yT4z9Z5omvZrr72WYjrxROTzOt2otC3ReYNtMyJijjnmqD3naWHSpEmpXakjCNuLyqEo92dq/xJLLJHtRzcmhes1lWYRptxT5vTyyy/X/o3OwVzPVjLptth5551rtxGOMTpXEEqZIyb3Ya53p5UPP/ywVhZFSYjK5dURpkLnRboQqYMkoeTm/PPPz7ZRJsF1jsqAiEq/F1tssTb3Y5/SzyU5FB0DtdQC5Zol57lWwnlRXffIT3/60+wznyl43jpH8lprX6e8ls+ZKiXleoFjtK7/KIlS6esnn3yS4tL94XxX6usc99VxjGs6fW4qrW07yuyzz56eg1SOyLlFpbFcD1Imy/k/Ip9PVTbMe3jHHXekWKV16m5YUZIoaXvZYostUrzlllumWJ0e+QzL+x6RP49SYq3O0lwz6LaSO1+FM22MMcYYY4wxxhhjuiB+aWOMMcYYY4wxxhjTBfFLG2OMMcYYY4wxxpguSLtq2nz66adx//33R8SUlll1lokReS0B6t2HDx+e7ad1bEhdnRPVtfM8qBlVjX+li4/4yiKN0CaRmjvVoH7jG99IsWp6qR1nHRvVy5csDSsbYLUrnxa6d+8+haVpBbWUqvljTRvao6ketQR189S033DDDdl+aidcwfoNERFXXXVVitU+nprRE044IcVaY4b2bgqtwrU2QN15tfJelZgwYUKthpWWnVo/gbVBaC2sGnzqo1955ZVsW1X7ISLvO1pDhHp9aubnn3/+bD/aH6qlKPstbbG15gn7s2pNyd57751itouIsjWh6sxbwWeffVZrfUk9L/XzEbn2mtfkueeeq/0u6ooj8rH3oYceSvFCCy2U7dds7QnWTFE9OO836zlojTC2F9bDUDifaFuiRl1rYHUWPXr0SGOqthOOB7Qyjcj7i9bCIuxvanfJ/k3dt9KsNSxrj+gcz8+sjaUafx3P66DNMO9bRP47qzVHZzJmzJjUD1S7z9oZWp+P9ZVK15/tQK8r66pR766wngfr4JTQWhzN1jihHfEtt9ySbeMa64ILLkixjrvsw2pxqzburaJv376pn51++um1+2k/Yh0bonUWOBb//Oc/z7ZxbmW/1HmRNYhYd0HtiDm2a5/addddU8z6UnX1IyOmrPXG9Z6eYx2jRo3KPrP2WquYaaaZ0npOa/lwzaP9iHVcWOuyVD9E66Q88MADKWZ7Vot4Pd+KRRZZJNt2wAEHpFhrBdWNyRxTIvLnkRKlc2Rb0jmd85Kuq6aFnj17pnVMXf+KKNfFvO6661LMMUlZeOGFs8+s9ce1lNbWYn0fbWtkjTXWSDHrjukxWBOOdcciynXrhgwZUnuOhPOI1i3qDFhfqvSsqrV3WIeRdWz0+feKK65o6jw436nVOeFzoNbUPP7441O855571m4jH374YfaZNt8633Nbt26T82H03QhrMTVTw0Zxpo0xxhhjjDHGGGNMF8QvbYwxxhhjjDHGGGO6IDPV2fK2ufNMMzW1s6b3MX2MVsJqJUmLPbWgZCok0x379OmT7ce0J6YlaRoW07tXWGGFbBtT7mnJpen8Bx10UIpLkoxW2O01Go16/7J2MOusszYqSZNaxvGzprxTJsZUMpUyNZsuy1RdpslH5DI03kNNaTvjjDPa3C8ityynHb2mNDPdWW2Lt99++xTTJlevjcr8CjzaaDR+MvXdpk6pL9JesSTroVRRUy1p46vpmkx3pNWfSlqYhstUQpWQ8e9oqReRX2vaB6sshlIvtbWlFWgpDZz3WPvz4MGDI+KrcWnChAkt6Yule8h7o+fM8UpTdcnGG2+c4ssvvzzbNmbMmBQzpZSyuIj8HmpqfLNQaknL9NVXXz3bj7aLypJLLplilVo0C3/nxIkTW9YXe/fu3ajkhbRKVlT+MHTo0BRTyqXtl3OVQqnqaaedlmKVf4wbNy7FlBQzbT4il4Tq+VZ9ICKXYey4447Zfkx3L12PjlK1k5tvvjlGjRrVkr7Ys2fPRtXW1baX8oGRI0fWHoPWwipVKK0POF7Tvlll2/xuXn8d4ykV1bGWkm6mcKsVK8eYt956K9umctkKnRe51qH0MWIKyUqnzIuUkUbk14KStIiI/v37p5hyCr22lFyp/JL9g6h0YIcddkgx24VKU5uFa5/FF18827bpppumWMsP1I2jlDJH5GO2stxyy0VExBNPPBFjxoxpSV/s27dvo2ovKgGnVEhlw9dcc02bx+P1jsh/t86LlF689NJLKa6z51ZUSrjyyiunWK8rx8Zll102xfrsQ6mXSon4XFSyGye63hZpaqf0xaWXXrp2P1038vNaa62VYs51bXxX7Tb2y6q9VrB/l8pzlNhoo41STMkb59z2sOGGG6ZYyxZomQ9Stf/9998/XnnllU5fo1K2feSRR2bbKDFme9P9uBbRdQ+58sorU8xSHRH5XFiy0+YzoR6D/Wjs2LG1+/3iF79Isa6dKmv0iHzu1ufWk046KergHPDCCy+02RedaWOMMcYYY4wxxhjTBfFLG2OMMcYYY4wxxpguSLvkUX379m1U6YPq2kQWXXTR7DNThemWog4I6jrSDE888UT2mS4crAZOx5GI3BXgr3/9a7aNqVhMfWP6eURefb8VaDoXK0u3Sh41ePDgxh577NHmNl4Hdew47rjjUrzvvvs29V0qXWPaY7PtjuluTD9TWH09Ik8r3H///VNM2Ul7YDr1fffdl21jGjalOBERTz75ZIpHjx49XeRRROVGmnpbx3e/+90Uq2sXmXfeeVOsMg6mmdOlSx0WiKa59uvXL8WU9KgDQrPSndJ9pDyO0riIyePKyJEjY9y4cS1PPaX0KyJiv/32S7GmcD/yyCMppqxh4MCB2X5M3y/1N17LjjoucZykq5vCMU4leRz/mcoakbuxUKaqkgS6X6gEgg4W48eP75S+SCmmfqf+XvZNul2olIQSOKbn6vEJ0/Ij6mWBJQmJwrGN866mrdMxRaGjBtPHKU+J+MqpskIluJVL2ujRo2PixImdngZOdOyixKgkfabsQl1veA/ZP9TNhM6Hzc6fBx54YPaZ6d4ltyu2zdKcQUkYpVcRuSyIKecREb///e+z02qlVLGad15//fVsG8dKddD86KOPUsw1pK4viUpTVe5Zoa43F110UYrZH9TJkveYUp2IfMzecsstU3zxxRfXni/lshH5nFly0SxRrafuvffe+Oijj6ZrX6ScOSKXTnHbJpts0vR3s0TAySefnGJdAz377LMpprxI5cWUmat0jesZnmPJ6U8dieokOOyXEfmcos9udF564oknOmVe/N3vfpdtO+uss1KsDm11zyE6hrBkg7q70lGNa3BKUSMi+Cy00047pVjdc1m6oyQv4zpuwQUXzPajLI/3XuG6RWXO6n5bR6ueF5vti0rds7c6oVFGpM/TlAwSHdcpteX6SJ/r+e6Bjs4RubSSz4h8/tBjqNMtS4Pw+UnvISWxKkuWNYTlUcYYY4wxxhhjjDEzCn5pY4wxxhhjjDHGGNMF8UsbY4wxxhhjjDHGmC5Ij6nvMpnx48cnPeTWW2+dbbvgggtSLHaOU+jCKrSGDfXilW59alB/rGgdG/L++++nWGvrsD4DLeHaU8OGtucl7Ti1/KxhEzFZI059/7Ty0UcfJV3kPvvsk23TOjaE+l6i14RW3moBSk0n6+KwXk5ErjekxabaqNIimO0vIrdTLNWxYZujtjci4rrrrksx65+odp/62dI17Cy01s+tt96aYq2f0GxNm1JtC7LUUkulWGvaUCPM66z1GFgXgrVcIvLzv+eee1L84YcfZvuVatpQZ0xNqtbzoHU7rcwjInbeeeeIiBg2bFjt97SXXr16pd+nOl1ql1XLTR0s7SHVzl4/18E6Nmq7XaqRQFjHRmtZ0eqU22jnqWhtGNpjr7POOinWGjzUBKtuXK1FW0nVhrXGTKn2G+s3se/wt0ZEHHTQQSnWWkurrLJKig8//PAUs0aHMv/886eYNRwi8uuudYXq6nuUarlo7Sm2Beq+9dhsJ9SKf900uy6hTW9EPvfRDjUir3XENUDJArUE119HH310tk0tbyu0jhJtzrV2C+HYpOMU2yrrykXkdqk6lk8LEyZMmMImui20zgnXGeyLWjOL9bRY+zAiYo011mjqHLkeYf/V2jesUabfxdocHA91DqYtttZ4qFtXqvU4xzBt1x21SW4W1oKKyNdaWiuCv5XXpz0W5t/5zndSTHtltUfnddhxxx1TrDWvWLtJ1z18LtI1ZR0lG+njjz8+xbq25zVge4ko16CbFrp165bG8ZIlt7ZLwuej5ZdfvnY//b0TJ05MMecP9t+IvOYQn/W09g0Ri/T46U9/mmJasuvam+uRUu1L1onU82V/aGacm1YGDhyYag6xjlNEPm/pNqmTlGJdb3Bce+CBB7JtrO20xBJLpFjnYD5zsVbStttum+03adKkFOsz+TLLLJNi9kutQ8h3A3yHoHB+Of/887NtHH9Yy7BZnGljjDHGGGOMMcYY0wXxSxtjjDHGGGOMMcaYLki7LL+7d+/eqFKHmNockafyUp7RUVQy06zNNCU5lF28/fbb2X5M19t1112zbbSLZvpne34XLSVLqeqE6e0Rk60h33jjjRg/fnzLLdzU9pIphZo6yzRF2oEuueSS2X5XXHFF7XczFYypg5pmdtlll6WY6W4jR46sPXYrKKXR0gKQ9n8ReaqspnpLOmDL7BRnm222RpVmr5Is3h+1fmTqJdOxDznkkGy/E044IcVquVuS+9VB+++TTjop28aUQ4XWnbTKK6FyuFakcFcyj48//ji++OKLlvdFtcSkzEalI2yLbLMqEeSYp5alSy+9dIopBdM01zoofYuIOPPMM1Os15+SDEofSynTSrP7sq1qKr9IPDvF2lShDITzUUSepk5Lc6U0R/M3MoVfZW60z+Q4f8wxx2T7MR1b06+ZSk5ZpKa30wK8o9KmzTffPMWXXHJJtq0aE8aMGROTJk36Wq1N66CVbEQutShRkqcxlZz20mqXTrTfsN9zbqLMMiLiT3/6U4o1RZzQRlrn/jXXXDPFanfMMfm5557rlL6oUhWOgRtssEHt+ZRklLT7VZkE4T3RNH2ugyhNmW+++bL9mGKvMre68VBtvTn/6xhD++iS7KZZWmUz3L9//0ZV9kDHD8r2VNJHGR+fA1Qm26yshHJpyiciIo466qgUc61Ukvqdc8452WfKBznfX3755dl+LDmgrL322inW8gqE7VHl9LweI0aMaFlf7NWrV6Nai6qUjfIZ/b2c7zgOqa03r7XKmSi14e9VidUpp5yS4kMPPTTFlNpF5BJBlUwSjoEqwaQcXNsgpVl8HlL5NyXv+oxWyYbGjx/fKfOiliKpk0u3B/advffeO9tGGfHdd9+dYpUS8lmSa0/ts5yPlAMPPDDFfFZpT0kU9jHem0033TTbj2U3piLZt+W3McYYY4wxxhhjzIyCX9oYY4wxxhhjjDHGdEH80sYYY4wxxhhjjDGmC9Kumjat1n2XUP0Z7b8qC7KIKa34WGuHtT5Y5yMirwWhtra0Clf9J6mzelNooagWsNSYq10saZVeeNZZZ21UdRDUqpXWyKqXfOWVV1JMTaHuR1s1tWbjvrQH5XWMyK0uaZ1+44036s9JaO2WIUOGpJh6UbWQvOuuu2qPSYtx1v0YMWJE7d8o1F2//PLLLdML9+7du1FpLzvb+q9v377ZZ9b6obZ+scUWy/ZjzRz2KdXns/+pPR51zNT8q0U20VoGdTa9ak3drCV0q/piaTytq1VSQjXHjz/+eLPn0dR+P/rRj1JM2+iI/F7feeed2baVV145xaxpQzvUiPz+tqfeTbOI1XXL+uICCyzQOPbYYyMiYv3118+28Z6oBpz661LdJdZYU1tg1i3id9NCNCJi1KhRKeZYecstt2T7sSYG6w5E5DXdbrvtthRr7ZVma08RnQNYk0JrmVXX9Kqrror33nuvJQ2lV69ejUobrzVceO0U1tjgeWqdmTp75RJqzctaB9T49+7dO9uP4yu1+grrBmn74/powIAB2TbWGeN6hvOsnhfH/zZoWV8cMGBAo1pTaX2RZsdRsvvuu2efWQOjWVgHJyJivfXWa/P47Rm/r7/++jaPV0LrfnBcee2115o6RonOmBe5Pv7f70ixrrlZ84lrogMOOKDp72a/XXTRRVOsdu733HNPiln/pFTLSteePEf+Lq1tybFc6/+xFhX7rNqa87lI69vJHDVdar2xT2i9K9ZM5Ji65ZZbZvuxNqLW5yScn/QesN3z+HoP2IZ0rNR6YBVDhw6tPd/Ro0fXnm9Hqdbp48ePjy+//LIlfXGeeeZpVJb2Wl+H87w+m3Hs5bpu6623zvZ79NFHU1yyfmddGNY8Vdgvdf3C2jda54rffeSRR6ZYnx2+/PLLFHM9HJHPz2+88UaK+e4iolxbR3BNG2OMMcYYY4wxxpgZBb+0McYYY4wxxhhjjOmCtEweRWvTku2dHC/7zLRtTe9jWj3tZT/++OOmvkttt2jbTJvTiDw9ipbTajFIu+uS1XVHqeRK7733XkyYMGG6WpuqpS+vF21JmWIbkadja0p4ZRevqNUl/473d7/99sv2o2SLNmoKJVaaIs20bU19VqvOCm0HtPCk7KsNOiX1VCUZKqFoBpUn8LqorIPytQUXXDDFKu978MEHU0zLYZVHsc3Q5i8iYoEFFkgx5WpipR633357irVNcox75513Uqxp688//3yKVRJWtcMxY8bExIkTO70vVimpEVNKwSjz4b3h9dZtP/jBD0rn0cTZ5tAWMSLitNNOSzGlhBF5/37kkUdSzHTuiDzl/4gjjsi20RKd47/2xZdffnmq5/6/tKwv9uzZs1FJRktSGoUSoB49eqRYU3cpa1h11VWzbUz3Zmq+jrWcW5nqrVbU7777boqZ4hsR8dBDD6WYqcd6Dyizfeutt6IOtnGdxzkna7smnSHJoJw5Il87cJyJqLc012NQRrTKKqtk21ROWKHzEe1Haau+7rrrZvtRWqlSNUrynn322RSrZILSUZWN7r///m2eR0dtzqOFfbFPnz6NShqt42az8tdmpam//e1vs8+854cddljt37HPct2sMgtK81VOwTmUssUS/K6I5tfphJbDEZPXZLvvvnu8+OKLLemLgwYNalSSLy0nQMkdZTTtodlnFUo51E672WvHtZLKSygLp1RRra0vuOCCpr6LayfKRCLyNZDCdnbrrbd2yhpVbZr5nZScROTyy3PPPbfNf4/I10E6Z3IdcPrpp6dYJYLbbbdd4RdM5rHHHksxSy9E5OtGSur23XffbD/al6uVNGVapTIQlKqWJFatmhf79++fymncd9992TaOY1qigesUzpm65tPnfMJrxLWHjsl8Vj3jjDNSPGzYsGy/zTbbLMVqM3/IIYekmM8ZfHaMyMdaLRGgzyQVnE8i8vUSJZ1tYHmUMcYYY4wxxhhjzIyCX9oYY4wxxhhjjDHGdEHaJY9qNg1cU79OPfXUFLNSuLoNsPqyVhRnGhvTlVVaU+cqU/qdf/jDH7LPTGNjOjElAArT4iLy1OOOUqXYnn322fH222+3PA1c5SFMKVTHCFbR7tZt8rs+TW2kDIbXLiJPJ6b85he/+EW2H+8hnYDqXICmxqBBg1LMNqaoewNlYJQyUD4Xkadw0tksIk+Jff/99zsl9VTTtJlyp9XymVLJdFqVVDGldJ999sm20V2FUgitpr7rrrummGmGKsdhf9YUWDpolNzVTj755BSrKxFTlJmGzL9RSmmcnSHJYGX7iFzGxf4Wkbt0qUsQee+991I8xxxzZNso62CbZWpoRO4SRKccSiQi8pTupZZaKtvGe7jEEkvUnm+zMi22F16niNwBYosttsi2sZ9ceeWVLeuL/fr1a1T35Kmnnsq2jRs3rvbv6CLAcVOlKs2mx6+00kop1lRmMuecc6ZY3Ugo01JHDu4788wzp3jeeeet/a7lllsu+1wnddK5ginFKh+q5Kj//ve/Y/z48S3viypB4xhOty79zHFG50WOk+raWIe6AtF9hNdcpXD8O3XGbJaf/exnKa6TgLUHnVspUxgxYsR0cawh6krEuYVOo5QLKkypj8jHZa4V6WJaQteoTLenTFgpyRSahW1GXVw4Z5533nm1x2jVvDhgwIBGNR6q3JWfVRJPmXhpvc9ryT4bkUunON6p9I+yw9VWWy3Fum6gc4yOpx999FGKKU/Xdc4JJ5yQYl2L8e907iGcr/U5Rp7lpktfbFaCSDev9jjw1bkNcQyNmHKtWAfHUXXrYzvhuK9zH2X76kAlErWmzqlUFqFVfXHgwIGNaj7sjBIgpTVL3bjGNUpEvjbnmKnyJT5z8nsj8j5Xclojpf5M+OwYEfHSSy/VbqObVlgeZYwxxhhjjDHGGDPj4Jc2xhhjjDHGGGOMMV0Qv7QxxhhjjDHGGGOM6YK0q6ZNnz59GoMHD46I3G5Z0ToatFrUGgSEOuNS/YoSrNNC/a3qPampVj39xIkT2zy21sCgVXGpdgH17Kr/p9Uba4VE5DaonVFHQ229n3766RSrjp11EFgro4S2LdZm2GSTTVJ8/vnnZ/vpedUdjzZ/Wttj+eWXTzF1q6pprTteRH5PaYGqdVeI2pIfd9xx/Dhd9MK0qizVw2CdAbX1pk6bdU0i8lo1rH3DexqRW76y/2ldC9qXan0prZ1SwXpDEREXXnhhm/sprD2llsbUqWsdmerzxIkTW9oXqxoykyZN6tAxdt999xSrRTCvZcmClpb1apdOaMmtfYX1Y2iBGRHxt7/9LcVrrLFGilUDTAto1pWImNJytUItaFnfQ4/Puefll1/ulL6oVpJHHXVUirWWCbXxrFWgNbOItnv2W8asBRCRt3vWi9M+QL24tied1ys4HkTkY4LCOY3zvdplar27OjpjXtTfzbmF9e4icit6tlH2npgkMgAAFtVJREFUKeXEE0/MPu+1114pZl0wrVnAdQrrRKnunn1d51KOM+ecc06KdW3DcZe1ACLyeje8T2rry89XX311to2253feeWen9EVtr6yLoDV2dP6rYK2piIjhw4dP4xnm/Y/rA7WXJVrva5ZZZknx2LFja/+O/ZlWsxFT2mlXlK5bic7oi6y7FxExYsSIFLOGUES+fi3VP+G4q227VK+RsF+deeaZtfs1C+ui6JzGeVHbQWn+JxxrdWziM0i0eI1atT+tz3XVVVfV/t3BBx+c4iOOOKKp79L6daxBs8EGG6T4mmuuyfZjXaR11103xfqswTWN3gPWd6MduLaLnXfeOcVqPd7sfMf1ja7BqvMYP358TJo0qeV9sYTW+WG/4lpOn39Zz6h0DKJjN6/zLbfckmK1/C7VTGy2jk0J1obk+wsddznGaA1UvlMI17QxxhhjjDHGGGOMmXHwSxtjjDHGGGOMMcaYLkiPqe8ymc8//7woi6poNp1SLdEoI6IlXETZbpvQopHpVkx9i8jTx7fffvtsG9OtaBet9nB9+/Zt6pzeeuutNuOI3EqQcpLOonv37um3l9I6mS4WkafzleRR3/rWt2q3UbZDKL1SmOZKeUZEbu+rqdmU4zCN9pJLLsn2o52lpugzjZlplGpRzhRnPT5T2tXmr1Uw/TMil0Rpah4tMyUttnY/TbGv64tqgcc0XMrL9HspQ9OxY+WVV07xXXfdleKSfHLttdfOPlO2wFRyyqEUTY1ea621IqJspdwRKrnC3nvvnf07x9kBAwZk2zg2nnrqqbXHpkxFj8/xrySJIhxDL7roomwbZS8PPfRQto1pvIceemiK9R7SkrsksSEqK6PFqqbW0xqys6A9dEQuQTjooIOybZRBzTfffE0dX8fKOlkH562I/FozVmkwxyidA3h/OL42e68i8nGadrWl9HCdU6o+rOnh00KPHj1ijjnmiIhcchAx5VqE1Nl36/V///33U1ySbnCu2nDDDbNtN998c4r520eOHFl7PJ3jKe3lnH7WWWdl+9Wlpkfk/YhjqNrH65hDRo8eXbutVehcwvXbAw88UPt3f/zjH1OsKfbkmGOOyT4fcMABbe6n8zPnsTvuuCPFKo+itFIpSaLID37wgxRzrRNRL4fXNQOvo8paqnZTsm1uL/PPP3/67RxzIvI1ma6l68YQlYrSXplyioj6MUXXQDonV+g4/vrrr6d4oYUWyrZxbuB1pXQwIm9nKvnVNVcdHPNVflVaB04rVamJkhxK+82ll17a5n46z+yyyy4pVpkvufHGG1OscwllhrRi1vUfZT26jqBN/DrrrJPi0rpKreHZdo8//vgU61qbchrdVvUN9uvOhOMEnxcUPpNX9uEVzcqjKEFT63H2K85HpZIlCqXzPB7bREQuWdY2ze9ec801U6zvFyi5Vbktbcr1ebTCmTbGGGOMMcYYY4wxXRC/tDHGGGOMMcYYY4zpgviljTHGGGOMMcYYY0wXpF2W383af5Wg9eizzz5bux9rYEREXHnllSnebLPNUrzNNttk+7EuzCGHHJJitYRmXZa555679jyo1dPzLdn+0YKO9R5ofxuRaxtHjRqVbVthhRUiIuKxxx6LTz/9tCUWbt26dWtU11YtxktQ57fAAgukmDr7qUG7WloAah0Y6kdZl0Lv09ChQ1OsNTao/Xz00UdTTMvhiIhrr722mVPPUB2x6ozJf//3f6f46KOPbpmd4pxzztmoagioLptayEUWWSTbRo01rWe13s7dd9+d4lK9CbYLtXqmtvbNN99M8eqrr157PO0DVZ0Jhbr0iIhHHnkkxaorf+yxx1JcqiHFGh5a7+bDDz+MiK/0zF9++WXL7RS1JhN/t2pzaem7ww47pFiPwXGnshavYN2OfffdN8Wqw6bGn+1A5w1+F22QI/IaDq2gWS11qZ5TtNDatE+fPo1qzNLzqcbwiCnrPrCOBK2eFRlDsm2rrbZaim+77bYUlyyN2ba0vy211FIp1j5w++23p5h97Dvf+U62H2tg0HY+Ih+zWV+lVF+qRCtthitLUK0bVPX9iObrIn3/+9/PPrN+BetyROTjE8fr0vqIc5rW62PNHNqt6nk0C+9TRD4O0OKWdQEi8ro1WpuE67losc1w3TbWCtExqW4dzLpTEVPWpSKs0fTwww+nWG1uWX+DtXU4hkbkaxixgs3q3dBKWC2NtX+TFVdcMcVcl5bGTZ0fqnHgmWeeiTFjxrSkL/bv37/x4x//OCKmrP/EvqJrOdZ2YlvU+iG8lpz7FNY20jUq52Q+S2jdGtZQYW2ViHyMYG0stSamBbHO8aydxdpJWm+TFthToVP6Ymk+ahYdl0u1NdlP2Yd/9rOfZftxDuZ6VS25+d16D1gziesl1rqJyOdWztUREQMHDkyx1sWrg/V4IvI6V62cF6u4sm+voI38gw8+mG2rG0+1L5bmU15X1sXRa8c+wPur7wZY61Phbxs8eHCK+a4hIr+/2te51uE5ad07jltTeX625bcxxhhjjDHGGGPMjIJf2hhjjDHGGGOMMcZ0Qdotj6pS9/TvmAb49ttvN3U8tc1jOq1aZjJdjemDKjeiZRbR1DpacGrKZ7duk99lqa1gR6BlH2UnERF33nlnU8doVbrbvPPO26gkIprqS8s1TUtl2hlTChXatjPlLyK3iW32eKRfv37ZZ6bPaToj02Mr28GIiGWWWSbbj9Z+mrLJFFjK9TR9uM72tQ2mSxo4r/tuu+2WbWNqYck2u3R/mBJPu0i1en7uuedSTIlHKa1VoQyD9qW0do7IUzXZBvXzKaeckmIdfziuaKo66YzU0xKUlEbk14Tp4kwh/d/jp7g0zlep6BFTjgm0hGY70LTRkm0o08DHjx+fYpUSNSud5Viu49SCCy6YYk2PpeVvtLAvDhkypFGl3Ks8Qcf7r4v99tsvxbSqLN03bTOUE1544YUp1vFv+eWXT/G9995be/ySTKvErrvuGhFf/Y733nuv5X1RLb4pa1O5KdcRjBVagOrYxT7GMfT666/P9uPcRZmcSmeaheOIpnpTrlaaJzoKpc7vvPNOS+fFKtWd876ia77//Oc/be6nNuYl62LS7PqG7V7XS+x/KpnhuHLwwQenWG3OuQ7aeOONa4+x5JJLplhT9mlRq1K5SmY2ceLETpkXv/e972XbSvb2W2+9dYo57qiUmqUM1I6d4zfbiEpRFltssRRTWk4JTEQuY9T1K+8v+5vOrbynalXPdsFzLFnC6/jGZ61bb721pX2xepbSe1CShdTJxtoD2w2vkZ7H+eefn2LKzvfcc89sv1deeSXFOgfQmpro/eaYQGliRL4+LrH55punWK8hZbydIRtuz7sCwrlvxIgR2TZdsxKWK2C/5++MyGVoLL2gYwefEUWem5WN4H477rhjtt+pp56a4mHDhmXbDj300BTzGVHbQUkayPZZ1xedaWOMMcYYY4wxxhjTBfFLG2OMMcYYY4wxxpguSKe4RzHdPiJinXXWSTEdnVqBugBQmsVUMk0hVhcOQhcAymL+9a9/ZfsxpYryjIg8ffLcc8+t/S6ix2BacqvS3bp169aoZFCUKkTkFfJ57SJyaQpTFidNmpTtR3ceTY2nPIqoQ9Cxxx6bYkpY1O2Hqb9rr712to37ltyPStS5m6y66qrZfnRV2WWXXbJtbD8nnHBCS1NPNWW6oqNpjM2y3HLLpfhXv/pVitUphtfi9NNPn+bvpQOStrs6yVaJDTfcMPs8fPjwNr8rYnJ1+TfffDPGjx/fkr7YvXv35OSmDlF0aFPXNMow6CxRqszfrPPCt7/97ewzU7PpSKHfxVRWjosREVtuuWWKKV/df//9s/1KDkKUbtalI0fk7VHnBkoho5OkiuwbEVO6KhDKU+hEWEqV1rRqlQdXlJzEiEo9CeWNEblLEZ3imDo+NbbffvsUU5rA+9YeWjUvDhw4sFGN6yrBoHTkT3/6U7aNcxcl4pQtKupgwnlYHcYI51M6Meq9pcxc07vPPvvsFG+33XYpPu+882q/V53NKBmipIDzdkTe9jUdXWhZXxw8eHCSf6s0kfO0wj5HlxF1xJp55plTrGN2R6Bk6fLLL8+20SFKZauU4VByQMl4RN7W1C30pZdeSnGpDMBaa62VYnVAquaLt99+Oz7//POWy6PoHhqR31OdP+j2xLW03kPOaSr34njI7z7uuOOaOneFEgrKJyJy5z8+M6ljGdFxZYkllkgxpRU6B7O96DqcLmXDhg2bLhJ+oveY96skkyYqLaQDEx3C9Lrcf//9KWbf4boqIh8D1S3xuuuuSzHnNJVK8/j6jHzfffdFW6hjE505TzvttDb/JqJzJPxco0Tka2Q+H0bk0t477rgjxXwujshLF6jci1JAnTMJ5yC6Uyorr7xyinXO5PhXJ5VtFZQnlub7sHuUMcYYY4wxxhhjzIyDX9oYY4wxxhhjjDHGdEH80sYYY4wxxhhjjDGmC9KymjZ/+ctfUqz1RQjt7C699NJsW8kmkZp31kjYZJNNsv1oaUzNKLWAEXlNB1r2laC1YkTEEUcckWKtQ1CqmVPHbLPNln2u6lo899xzMXbs2JZrFHkdI3KNs9oh33rrrdP83ayzQJtTtWln3aOTTz45xaq7Z20dtXmm7pRtiZr0iLIunRaKtA/VWjKsn6N6ZNHQTne9sLLKKqukuFm7edbeiMjro1CT+fOf/zzb7/HHH08xa/vo/WYtGe0DH330UZvnpBpm6lXVlrXOSlrrt7ANac2czqgvNffcczeq2lHXXnttto2abLW1Zo0s6qZV98trd9JJJ037CYNSjRxtL6ojryjVz1Fo58qaEyVYmywi4uGHH+bHlvXFmWeeuVHV9NH6CbSI1BpKrANE+3OpvRMnnHBCivfee+9mzyn7XDfO0V41Ih9T9b7ts88+KS7VplPdekdgfQbVfVc1jcaMGROTJk1q+bzI+kkREWeccUaKt91229pjtEOrXgv7qdZgYY0h3hvWmorIx8Y333yz9rvmnXfeFL/xxhtNnyNrc7BmR6nNsW5GRMRf//pXfpzu8yJrK0VE3HPPPSnmGlX7AMcerVXImg9cL5xyyinZfieeeGKKWbNi8ODB2X6sY8N5uz2wvWo9pjpKtRXZZiImX7ff/OY38fTTT7e8LyqsBUI77Yh8jaH1GutQa2f2P9aV0xpuL7zwQoq5ntG6bLR0V+pqJqrl98ILL5xinVtZE4nPYGpHzJojF198cbZN6jRN976o45zWjKyD9cQqa/EK1ibi+pJ1xyJyu/Mnn3yyqe9l7aSIvLYS26DOI6w11iz6LM17rOtt1jxrZa23ai5mvdiI/HmxNM/wOuh6g/Unta/wmZr1vdQGnmMC76+uJ/nMqbWh6upg7rbbbtln1g/T2nesG8U6PtOAa9oYY4wxxhhjjDHGzCj4pY0xxhhjjDHGGGNMF6TD8qjzzz8/27bNNtvU/t0xxxyTYkpcNI2aMqU111wz23bTTTelmGnlmqJEmCqlNl6U6qjtH1NimdJ+9dVX137X+uuvn32m3IEpmO2RKSy++OIR8VUa2rhx41qeesp0sYj8Wqrda7MwnYzSgBJqJXfmmWemmHZ66623XrYfU+00ZW6RRRZJMX+LWtM1mxJJfvjDH2afn3jiidp95RxbKsn43ve+1+b3d0RKUpKqlGw3mdLdr1+/bD9KHNjuaYsYMaU1Yh08PqUIEblcR68H7Rt33XXXFKudJFOPL7rootrz6Aw7xRLVfa6gBIc2kpqiusMOO6T4nHPOybbxflPGUGcFrmg6MsdrtYBmynCzshHaOLb1uWL22WfPPn/44YdNHT9aKMlYZJFFGpVdLy1YW8UKK6yQYr3Hr776aopL/Z6SzmbnfNpKR0RceOGFKaY98e67757tR0t2lWUxpZuyRUqQInK7eqWSbzz99NMxZsyYlvTFnj17Nqq2pOM7U52vuOKKbBvlCZSPHX/88dl+XGOo1OXdd99NMeXAOsY1C/u2rnt4bwgleHoMpt1HRHzzm99MMduZjh0qFSGc43faaadOkWRQ3hyRj4e6DivJbupYaKGFss+UzJSgbJ9y/hKUcUREvP/++23up/KPZZddNsV33XVXto3yElreqjU4+7eOK9Wa7KabbopRo0a1pC/26tWrUZ2byt45f2yxxRbZNtqRq6UveeSRR1K85JJLZtu4ZulISQCVUVFmrbJX9rGSNJjye7VF5rjCZw6VTFLWpnbWQsv64myzzdZYaaWVIiLilltuybax3AUlJxH1a5Dlllsu+8x1hq49KTeu+96IfGynLEslW62A57TXXntl29h2+Tykcmu2ax3bOf5MjzVqJe2PiLjkkkuybX/7299SvOKKKzb1XSr9oxSM/UPHWcqo5p577hTrtSOcwyIiqnYakc/xKlnl8XUtS1kpJaXTgOVRxhhjjDHGGGOMMTMKfmljjDHGGGOMMcYY0wXxSxtjjDHGGGOMMcaYLki7atr06dOnUWkj6zSDERGHH3549nnMmDEppl3aHnvske1H7aZCi0ZqwKkFjMi1nLQPXmaZZbL9qJdTrSk/00pYLc+oKdRjLL/88imea665Utys7WLEZAvfJ554Ij799NOWaxT52yLyujxqkc7aEdRXa/sp6WWrGj0REU899VSKSxaTrHWw8847Z/uxPo1qmGnh1hEryBJq+d2OPtQyvXC/fv0aVe2FBx98ULelmH0vIqJ79+4pZh0qtS+tqwmksN5Kqb4UtcQ8h4i87gWtGiNyfTPrWpXQmjnXX399m+dR0qzz90fk16Az9MJqB836WaxbE5FbgG+55ZYp1nNmLYpS7YQBAwakuGRLyf3UEl3bWUcYOnRoilUjzfmGYxF/f0Req2KXXXbJtp1++un82LK++O1vf7tx8MEHR8SUVsKlOjOs+aXWks1CzTUtzrVts4/16tUrxbQVjoj4wx/+UPtdrPPE677ddttl+y244IIpZo2WiCl1+BVDhgzJPnP8Zu2biLwWVWf0xZINusK5hXP+nXfeme3Hmk86Tk6YMCHF1O7/9re/zfa78sora8+jDq0ToBbWFeuss072+YYbbmj3d2ntAo4JWseH9QtHjhw5XWyGacmqdTTqUNveuppAEbnV+gcffJBiHaPYj2jvfOmll2b70XJa64Q1C+2cWd8mYsoaNxX8HRF5/Ue1hq+eB958880YP358y/uijplcl5buBdGaiazvorU5ObdwPaw14Tjusp6ethedx6YVrf/HtQ5rJT3wwAPZfmPHjk3xVOrFtawv9u/fP61RH3rooWwba+JprZr77rsvxawvprVH77///hTL3J7N/axtxesQkbftUh2kEtVzWkTE3//+99r9OlIzR9fDpefuQw45JCK+qhv79ttvt6QvDhkypFGtEVhfNKJsa73RRhul+Kqrrkpxs/NRRP3z4lTON8Xs5xF5ny3NpXyHoPW9SnDt9/zzz6dYa/dedtllKeY7iogprMdd08YYY4wxxhhjjDFmRsEvbYwxxhhjjDHGGGO6IO21/H4/Il6b6o6m1QxpNBqDpr7b1PE9/FrxfZzx8T38v4Hv44yP7+H/DXwfZ3x8D/9v4Ps44+N7+H+DNu9ju17aGGOMMcYYY4wxxpjpg+VRxhhjjDHGGGOMMV0Qv7QxxhhjjDHGGGOM6YL4pY0xxhhjjDHGGGNMF8QvbYwxxhhjjDHGGGO6IH5pY4wxxhhjjDHGGNMF8UsbY4wxxhhjjDHGmC6IX9oYY4wxxhhjjDHGdEH80sYYY4wxxhhjjDGmC+KXNsYYY4wxxhhjjDFdkP8PrQ7MHwtO/GEAAAAASUVORK5CYII=\n","text/plain":["
"]},"metadata":{"needs_background":"light"}}]},{"cell_type":"markdown","metadata":{"id":"gqOXYgLpk7sk"},"source":["Το δίκτυο έχει την ίδια δομή με έναν υποπλήρη αυτοκωδικοποιητή"]},{"cell_type":"code","metadata":{"id":"0ajCzaOEluQD"},"source":["noisy_autoencoder = Sequential([\n"," Dense(dim_h, activation='relu', input_shape=(dim_x,), name='encoder'),\n"," Dense(dim_x, activation='sigmoid', name='decoder')\n","])\n","\n","\n","noisy_autoencoder.compile(optimizer='adam', loss='binary_crossentropy')"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"GkS5baBFlqTj"},"source":["Το εκπαιδεύουμε για ίδιο αριθμό εποχών"]},{"cell_type":"code","metadata":{"id":"bfFPAVpjl50-","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1642514303066,"user_tz":-120,"elapsed":143050,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"e4f4ba50-5e4c-46b7-e485-66a719835b31"},"source":["noisy_autoencoder.fit(x_train_noisy, x_train,\n"," epochs=30,\n"," batch_size=64,\n"," shuffle=True,\n"," validation_data=(x_test, x_test))"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Epoch 1/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.2038 - val_loss: 0.2737\n","Epoch 2/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1296 - val_loss: 0.2877\n","Epoch 3/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1161 - val_loss: 0.2898\n","Epoch 4/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1127 - val_loss: 0.2854\n","Epoch 5/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1098 - val_loss: 0.2763\n","Epoch 6/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1091 - val_loss: 0.2691\n","Epoch 7/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1086 - val_loss: 0.2491\n","Epoch 8/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1083 - val_loss: 0.2462\n","Epoch 9/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1080 - val_loss: 0.2338\n","Epoch 10/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1078 - val_loss: 0.2165\n","Epoch 11/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1076 - val_loss: 0.2048\n","Epoch 12/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1074 - val_loss: 0.1892\n","Epoch 13/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1072 - val_loss: 0.1825\n","Epoch 14/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1070 - val_loss: 0.1686\n","Epoch 15/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1069 - val_loss: 0.1585\n","Epoch 16/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1067 - val_loss: 0.1518\n","Epoch 17/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1066 - val_loss: 0.1434\n","Epoch 18/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1065 - val_loss: 0.1385\n","Epoch 19/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1065 - val_loss: 0.1344\n","Epoch 20/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1064 - val_loss: 0.1303\n","Epoch 21/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1063 - val_loss: 0.1256\n","Epoch 22/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1063 - val_loss: 0.1225\n","Epoch 23/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1063 - val_loss: 0.1213\n","Epoch 24/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1062 - val_loss: 0.1189\n","Epoch 25/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1062 - val_loss: 0.1173\n","Epoch 26/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1062 - val_loss: 0.1164\n","Epoch 27/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1061 - val_loss: 0.1148\n","Epoch 28/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1061 - val_loss: 0.1139\n","Epoch 29/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1061 - val_loss: 0.1130\n","Epoch 30/30\n","938/938 [==============================] - 4s 4ms/step - loss: 0.1061 - val_loss: 0.1123\n"]},{"output_type":"execute_result","data":{"text/plain":[""]},"metadata":{},"execution_count":24}]},{"cell_type":"markdown","metadata":{"id":"dVMG7frFlwVt"},"source":["Παρατηρούμε ότι ούτε σε αυτή την περίπτωση παρουσιάστηκαν φαινόμενα υπερπροσαρμογής στην εκπαίδευση, με το αντίστοιχο σφάλμα να είναι στα ίδια επίπεδα με τον αραιό αυτοκωδικοποιητή (και ελαφρώς χειρότερο απ' ότι στον υποπλήρη αυτοκωδικοποιητή)\n","\n","Ορίζουμε τα δίκτυα του κωδικοποιητή και του αποκωδικοποιητή."]},{"cell_type":"code","metadata":{"id":"nKgtSbaPmBn2"},"source":["noisy_encoder = Sequential([\n"," noisy_autoencoder.layers[-2]\n","])\n","\n","noisy_decoder = Sequential([\n"," noisy_autoencoder.layers[-1]\n","])\n","\n","h_noisy = noisy_encoder.predict(x_test_noisy)\n","x_test_out_noisy = noisy_decoder.predict(h_noisy)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"IyP8AB45mxd5"},"source":["Εμφανίζουμε στην πρώτη σειρά τα δέκα πρώτα ψηφία του συνόλου ελέγχου (με θόρυβο) και στη δεύτερη σειρά, τα ίδια ψηφία όπως προκύπτουν από την έξοδο του αραιού αυτοκωδικοποιητή"]},{"cell_type":"code","metadata":{"id":"WUqrololmLwZ","colab":{"base_uri":"https://localhost:8080/","height":248},"executionInfo":{"status":"ok","timestamp":1642514313807,"user_tz":-120,"elapsed":1434,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"8fe8da5c-8d05-471a-e331-d751e5885533"},"source":["compare_digits(x_test_noisy, x_test_out_noisy)"],"execution_count":null,"outputs":[{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAABG0AAADnCAYAAACkCqtqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydZ6BdVdW1J6RDQoQQQAkgIF1QVJAiRQVBkCIISgfpRSwoINUgTamfgEhApDdREaV3kCAgvYbeawiEFEJIcr4fvHvxrJG7F+fenBtOwnh+zZO17yl7r7Z35phjlkajEcYYY4wxxhhjjDGmvZj1k/4CxhhjjDHGGGOMMWZq/NDGGGOMMcYYY4wxpg3xQxtjjDHGGGOMMcaYNsQPbYwxxhhjjDHGGGPaED+0McYYY4wxxhhjjGlD/NDGGGOMMcYYY4wxpg3p2ZmDe/To0ejVq1dERLz//vu1x80+++zZ68997nMpfuutt1I8atSoznx8U8wxxxwpfvfdd1v+/qRPnz4p1vPRu3fvFPft27f2PSZPnpzicePG1R7XaDRm6cp3VGaZZZbk8f6Zz3wma5syZUqKu3ruBg0aVPseH3zwQYrnnHPOFL/99tvZcXPNNVeK2Uf69+9f+7mzzJKfnjFjxjT5jethv33llVea+psBAwZkr9kvJk6cOLLRaAye5i8W+XXszPfpynmZddb82S77yec///kU63hu9fgbOHBgikePHp21lcYiv/9nP/vZFHMuioiYMGFC7WfPM888EfHhb3rvvfdaMhZnn332RjUGe/TokbW9+OKLtX/Xs+dH0zbnj0ajqS4REfnYHzt2bIonTZqUHTfbbLOlePz48Snm2IiIeOeddzo8LiKf/3htdMxyztTfwt/MeZJ/ExExceLEaJLpPhY5r0W0Zv0rzaOk6r8Reb9n/4mImHvuuTs8LqK+f5XmB4XXi7H2fx3fpOoLkydPjilTprRkLPbt27dRzZUjR47s0ntwftJxVOr3HItvvvlmihdccMHsuBdeeCHFpf0Fx7OeV8J9mu49uFaX6NevX4rfe++92uMGD86HGueIcePGtWws9unTp1HNWZyTFI6HiPya85zpeRgyZEiKX3/99ayN/V7HVTNw7EXkc1lpLeVv0WvA9Z5zeUT+2zifP//889lxXDNfffXV2u/RHXtUpW5vqNStW52B81ppTptvvvlSzPEbEVHdL0VMvb/guC29P/uB7tnZL/h++pv5dzou5Jx2y7rIeSIi76e6LnLs8HfovMnro2tQ3TUvfY8SvI46t3NdLM2pROdlfi+OS93fcDzrHFb1hQ8++CAmTZrUkrHYu3fvRtWv9N6hNC/wPrB0jktjk9e7tPfvzL63QscRx0Sza1qJeeedN8W6TvB3ad+XPVeHY7FTD2169eqVFq2nn3669rgvfelL2euDDz44xRdccEGKzz333M58fFOsvPLKKb7mmmtqj+OJ04ve7GS9wAILpPipp57K2rgILrnkkrWfxYEwfPjw2s/qDr75zW9mr7moXHXVVV16zw022CDF1157bdbGBx9rrbVWiv/6179mx6233nopPu+881L8ta99LTuO55KTakTETTfdlOKubKAiInbfffcUsw+XWGmllbLXTzzxRIqf193QdGDFFVfMXt9www2dfg99CMs+O3To0BRfdNFF2XFd7UN1rLHGGim+/PLLs7bSWOQk/JOf/CTFOv888sgjtZ+91VZbRUTE+eef34lvXOYzn/lM6mP6cO1nP/tZ7d/VPRjtzALz7W9/O8W33HJLivWGdemll07x//73vxTvuuuu2XFXXHFFiu+6666sjQ/2OAR048LjdIPLDQrnSd5ERUQ888wz0STTfSxyXovI57auUppHSdV/IyLOOuusFOuDnk022STFZ599dtZW9x81pflB4brIa8dNWUTen5Sq/+tDpWlhwIABsfHGG0dExBlnnNGl91httdVSrOd1xIgRKdaN+Pe///0Un3LKKSk+4IADsuN22223FNftLyIibr/99hTzP7Ei8v0M1yqO7YjyQ2OyxBJLpPj++++vPe4HP/hB9vq+++5L8X//+9+WjcXZZpstzW1/+9vfao/jeIiI+POf/5xi/ueQ/mcN5+UTTzwxa+P825W+uemmm2avOVdeffXVtX/3wx/+MMWPPvpo1sb1nnN5RMTLL7+c4kMOOSTFO+64Y3bcLrvskmKu958E66yzToovvPDC2uOWWmqpFN9zzz1Nvz/3/qWHIGS77bZL8WmnnZa1cY57+OGHs7ZFF100xaUb/Oeeey7Fa665ZtZ2/fXXp5jXV9dg7vX/8Y9/ZG2y3+6WdXHxxRfPXj/wwAMp/u53v5u18aEz5yX+x01EvmfSOfXee+/t8Htwvoooz1mEN+H6YI7r4rLLLpvi//73v7Xvp+sdrx0fgOiDe97XbLHFFllbdb/L/jKt9O3bN91D6L3DzjvvnOLDDjssa1t//fVTrHMS0XWH8J6O/Vf3Bp34j7oE978R+VrR7JpWYtttt03xMccck7XxP5h5vxyR782iZo9qeZQxxhhjjDHGGGNMGzJLZ1KLmk0D32abbbLXDz30UIqZeaBPr0upj1//+tdTfOeddzbzNeKLX/xiijUl8LXXXkux/i83/5eMWUN8Ojy9aVXqaZ8+fRrV0399Ov3444+neNVVV83a+Pr3v/997fvzfydKchOiGTR8Ss7/GSyltOmT68UWW6zDv+P/TOhn61Nf/g8m/1dEn/SedNJJKWYmR0TEKquskuLhw4ff02g08h/bRZodi8wWiog49dRTp/mzv/rVr6aY/4v15S9/OTuOT6n5v8n6Pz3NUkrxJaXvwf65+uqrZ8fxt/B/FyMijjrqqBRPjzTwjTbaKMX6v2bMHHvjjTdq35/pq5rSq/9bVFHqLzz/+n7837ES/J8jZtZE5P/zW8rkJPoe/J8K/V9JZlZts802LRuL/fr1a1TzQylbq8Sxxx6b4l/+8pdZG/+XSf9Xct99900x0/S5vpXQbCdmZuicys/mmq7ZqPwfZZV18Poz21XXe2bHaR+vvsftt98eo0ePbpk8qupLX/nKV7QtxX/5y1+yth122KHDNpW6MINNZb5rr712ijk3cv8SkWd9HH744SnWjA9KkX7zm9/UfhbRbB3OHfzf3RK69nFd/BhaNhYHDx7cqNaa008/vfY4ZqhF1O9bdL9QosrUioi47LLLao/jvovn7LjjjsuOYzYV19yIfO/cbPbsr3/96+w117S6NV3Rfl3NsTfccEOMGjWq5eviz3/+86yNmSxdlT0xa5NZShH1fXb77bfPXsv/iCd07lhhhRVSfMcdd2Rt7AfMeuPcGpFnwiqUgTFLUr/fZpttlmLdf/F/+//1r39Nlz0qf69mpDQr+SWaUf6LX/wixcxI6YqU5uNglhr3VaW99kEHHZS95nw+//zzp5h7IkUzbZh51h17VF23SpldXE94X1likUUWyV5TNVPKcGfm+X777ZfiYcOGZccxA59Z+xH1Y+zAAw/MXh9xxBG136MOXWuYKa7zxZNPPpni119/vcOx6EwbY4wxxhhjjDHGmDbED22MMcYYY4wxxhhj2hA/tDHGGGOMMcYYY4xpQ7qlpk2JZ599NsULL7zwtL7dVFArSX2YVvOn20zJXrbZStxaF4duCayBwroDEbkuV11vyPSoo1GyKWOtC+os1V2hVA2cUOenzk91rkN67li3QWs9UIdPjbpqGf/zn/+kmPrjiLxuCm3mtFZPSXfL73XVVVe1TC88cODARlVnSGs8tcJKuATrvTz44IMpVm0yNbfU2/7zn//MjqP2VG0M+dtYhV7rCrFavUJHiO9973spVgeqEn/6058iIuLII4+M559/vtvHIqvbv/TSS1kb633Q5UWtfzlOS7Vq+H50qInI9besy6E1hUpuKdQcl4770Y9+lGJ1ImuWklaeNUIefvjh6V5fqjR/UQOuNWI432gdJs631I7rdeRayONKmnmdl2lFylpCpTVS6wrdfPPNtceSn/70pymm1XXER/PP3nvvHU888URLxuK8887bqOYrNfnj+VJnrLrzV6q/pjW36L5F5ycdz1yf+blaX4poXRzWnmI9D7UeZW2ArsL+o64tpLvGota74vnUOgt131X73re+9a0Ul+rWEHVTrauNqOs46zjutNNOWRudOVlnRh12WDeFe52IfK7kflhd6Fg3Sx2uqv56/PHHx4svvtjt62Kzdbu45rNGSkS+P9IaPaxJ8o1vfCPFrJEYMXVtqzpYw0v7Uh2sgxiRO5vRFUvh/KAW2Kwfpv1R9tstG4ucU6+77rqsreQoxDqTdOTTmkB0a9P55ZJLLunwvdWt51//+leKeY7UvZNr8v/7f/8va6MzFmvpqAMfa3XqOkhXtGbnFXWqrRz5Jk6cGFOmTGnJWOzZs2ejqnOn9zmcC9Spj/MQ3dXYlyOaHx977rlniumwGJHP87wWyyyzTHYc5/ySsy1rCqozFR011XqcexbWZPz3v/9d+1m6P+I+7eWXX3ZNG2OMMcYYY4wxxpgZBT+0McYYY4wxxhhjjGlDWiaPktTzrK3OypvpRBFTp501w9ChQ7PXhx56aIfHqZ3YM888k2JNIeb3Z3qp/i5aKKqNHu0amR6mKfu0a1SJD1O4WiWPmnXWWRtV6hpTYiNymYTKo5hWWKXhReR2yhERX/jCF1KsKaX8PfzsPfbYIzuOdsdMB33//fez42ivu/POO2dtTEVcaKGFUqyp76SUxkz7W03FpHxBUwgpH5s8efJ0kWSULMjrYJpoRH69Nb2YUqd11123qfcvseGGG6ZYpWdMmaQ8g2nqEbkERy1LOW6Z0jhkyJDsuCeeeKL2Ox588MEREXHGGWfEK6+80u1p4KSSZlVQ+kK5l6Zh9unTJ8U6dupQmQQlrJR8qM0sZW20jY7IbaU5x2i/uu2221JcknyQHXfcMXtNyVudrfn/0S1jkVbJERGvvvpqinUsMtWW65NKHJhmrfNyHbQJj8jlohzrP/7xj7PjSueMaeuUc5133nnZcUceeWSKaU8f8dE4isjlKtrvOG/utddeWdvJJ5+c4u6QDau9Ms+JWmZrunczqCyT818Jjj/uWVT2UmKrrbZKMW1UVU5BK1LOIxERv/3tb1PMeZgyr4h8/dd9gshcWjYWZ5999kbVN1UmzLlM5ewcHyqBI9zfqJyd/YbzoabiU4rF+UL3Y9wr8nMj8rWQe1mFEiuVXxGu6Wo9Tgt5tX+vrKSvuOKKeOutt6bruqgyQ+1jnT2uM7D/cG1VOUUJSpYpV27Wjljh+KPFd0RuR8+4A1o6Fqt5iv0wIt+jbr311lkbx0tJTs3xolIkwns4vSfhPQTt01XKXBpjXBebLQ+h0teq1IF+dsnqWvtJtbcaPnx4jB49uuVjUecg7gF070mpWekZQ+m5wW9+85sOY5Xucjyz/IruQ3ltWOIhIpdVUR6lcjreI4wYMSJroxyL9/8qDewElkcZY4wxxhhjjDHGzCj4oY0xxhhjjDHGGGNMG+KHNsYYY4wxxhhjjDFtSKdq2swzzzyNysqaGsyI3IpUNdZqYVqhtSdUS94VqM3m52rNjr///e8pfuONN7I21iFgbRTWsImIOOqoo5r6TtSH62fRznfppZfO2ioLv//973/x7rvvtkSj2KNHj0alfRw3blyX3qNk98r6G9QXRuS1ZbbffvsUs85FZ2B9gUMOOSRr42vanK633nrZcYcddljt+1Mrueuuu6ZYteHUTr777rtZG3Wfjz/+eMv0wnPOOWejsoVWu71moZZY61JQ46k1fFiL5Jxzzql9f9aMYS2F3/3ud9lxPEdqwck6J9TU0lIvIu8Lag3JugusfdOsbafSHXU0qKONiDj33HNTrHMGtbPUdZdsDFWbyznp6KOPTnGp9g0tUPWzWIdFxzO/LzXkCrXhatd76aWXdvg3WkeDdStUIy10S02bkk22QrvHkhU258qzzjora+NYZD2tUr24RRddNMU6joheA65VzdYcaha1yOZYL9UJ6I6xSOv5iK7Zz2vtG+ratcYG149bb7219j05J1Nbr2Ob17RkrdssWm9Ia8VUaH8pWcELLRuL/fv3b1Tnl+crIj9nw4cPz9q4D2adhcqyuOLAAw+s/WzOqdw3as1EWlXzO/JvIiL69++f4pEjR2Zt/DvW4tPaG7rHJrRFZp0oncNYn0FrN1Z947XXXov333+/JWNx4MCBjarGR2lNaxatM8Y5Wc8r68xUduYREQ8++GBTn6V7j5VXXjnFWieF/UD2idlx7D9c3yI+rCVUwfudHXbYITuOltK81hFT3a+1bCz269evUc3jXDsi8ro6WmeS9V44H9J6PiKvX6p1cXQ/W6F1Q6+99toUs/6a1jwZNmxYh+8Xkdc1Y4097uEiIrbZZpva9+C8csIJJ6RY9w8rrLBCinUOI92xLirsz1qXrXS+mqWu3o3ulY899tgUl+o4HnHEESkuzePN1kAtwTmHzxMUrT3Ivc6YMWNc08YYY4wxxhhjjDFmRsEPbYwxxhhjjDHGGGPakE7Jo/r379+oUjE1NevEE09MsUomNO2zQtNGmYarae9Mk1xppZVSrBbaTANkSrJKtL7//e+neJNNNsnaaO9MGZimavL7fuc738namFJ9wQUXpHjLLbfMjitJB6rf+dBDD8XYsWNbku7Wp0+fRpW6pRaTJevQutRfhWnu2g80jbtC0whXWWWVFDON8qGHHsqO4/XVdHRC+zmF6ZKlVNz1118/xUybjchTWzUVjqnGTz311HSx/O5u6lJ51RKQ54JSPP69vsfPfvazrI32o7RF1HFPSjbntJpUmAZOy+GIiOWWWy4iIp588skYP358S8bigAEDUiq/SiOZ9sz5KKI1koeuQItDlR4QtZHm96W0kHKoiHxe1xTuOjTduTSGK7lpRMTrr78+XcZinVxXvw+tPddaa61p/k66rpfmwDpUqsi5Xe2OSUnORZheTWlXRC4rUOlYlQI9ceLEmDJlSrenge+5554pnjBhQtbG1PiSfSmlQ5SSR+Tr5Pe+970Uq41qHdzzROTyRJVAM31/9OjRKab8NyLixhtvTDHlpSVo5x6RW4PTTjciYvz48Xz5ia+LdXO/WhVT0qKp81zXaNOsfVvXvwrdD1MKoVJeziu8/rqnK8kUNt100xR3VWJNWiXJ6NmzZ6OShunv4bjadtttszbutTif6phlmYMXXngha6sk5xF5f1aLeO5ZevTokWJK1SJyyY3KNfgeHB8yNopwT/Tee++luLRHVyhtnjRp0nQZi5TmUeIbkfdLLTfQariPefnll1Os8yb7IWVZEfmcStSGnLJVnvOIXHpOadspp5ySHcf7kJKkujvkUSphfuedd1rxEQmV4S611FIp5j2XPhvgteI9SGmPouU0WCZjnnnmSbGWJiC63vE+lvMKr1lEXqplr732ytr4na+++mrLo4wxxhhjjDHGGGNmFPzQxhhjjDHGGGOMMaYN6ZQ8iqlSpXRXdZthWj0ZOnRo9pppZ5pWzZQxfmdNPaKsgKijCVMVKf2JyCvw1zklROQyjBtuuCFra1bCsMYaa6SYKbURH8mEHnjggZbJo3gNjznmmKyNv+Hqq6/O2pgeyGu//PLLZ8dRFtZs39LUfcpBmMKtKapMJXvssceyNlYRp3ORvgdT8m666aamvq/C9Fim10bkqXz/+Mc/uiX1VNMWec5K12DddddNsV5vVoO//PLLszamBTJdWZ0l6t5vkUUWydrOPPPMFKvLG6/Pdtttl2JN/2UFea0uTyi9U0kdHQdKrjrTozJ/Cbrb0OFMXSe6gjptMM2c41LlUXQ6UQccznElaWsJykboBqey1AMOOCDFTHONyGUKH3zwQcvGYq9evRqVlEjld5tttlmKVWqhLnQVKjOkm4a6ZOy///4p5tqnLnl0HWFqtq5vpfWuDpVWUG5ckmXR0UrdrppleoxFSjYpA1cogVInpVIqO6GTVCXJrCjJl8lBBx2UYkqUSsw///zZa0qEdBzVpYxTRhYxdWp/gZaNxd69ezeq76uyhZJTJOHv+Oc//5m1qQsOqVtr1S2RY7bkMsK5pCT5bZbSXoBjVtdg9mWVHFTuJ610OOVY1H5JCUt3wLIJdIzS/QAdFyl3o5w/ouxEw3sS/t2FF17Y2a89FdrXKQdRWbK483XLHpV75IhcNtbB36WYjsLqpMQ9v0o4uZ/leON1i8j3g5Rp6f0t5VzKm2++meLBgwenWCWyvPd44IEHsjbOsQsuuGCKKduMKK+nf/zjHyPiw9/4/PPPt2Qs9u3bt1Hd+5XkRtzXReTyIM5rOgfx3kXlVlwLeQ+n9yN0snz99ddTrG7PHM+8R4rIXeQmTpyYYnXjI7oX0NIOFb///e+z1yzrwbgDLI8yxhhjjDHGGGOMmVHwQxtjjDHGGGOMMcaYNsQPbYwxxhhjjDHGGGPakC7XtFHLb2oySzZZamtITjvttBRr7RHqBnmc2n/RGpG1AfbZZ5/sONZqUK04a25Q26gaRdZxUA07NZaspaD1Hqj1ZO2HiPy3tEq7P2DAgEalE9UaOqRky0ydLjWEEbm9ovYtvgft3NSydNlll+3ws66//vrsuMsuuyzFagfOegDN2qiqfTB1saVxQutaWtVGTFXnpWV64SFDhjQqu7g//elPWRuteu+5556sjXpmaplVd8kaGKzLEZGPD+q0WWslIq+PQSvmFVdcMTuud+/eKb7tttuyNuq7WTuD80FExKBBg1KscwJrfZS01BzrOp5Zf6U76mhovRPaRar9ONlvv/1SrLUTSiyzzDIp5ljROhpqSVjBuS8i1xXTFjciry3BOg1a+4xUVugVnGe09gOhll9tHYWWjcW55567Uc0xWr+Nc39dbbeIiC222CLFrMcVUbbNHjJkSIp/9atfpXjvvffOjqMWnjVPaFkdkZ9bausjprbH7eg7REQcccQRKWYtuoi8ZgTrhbD/dIZWjcVZZ521UdUO0nH07LPPpviEE05o6v10Tt5tt92a+jvO3azdFJGvrUsvvXSK1SKY40/3YtxXsU90puYEx75+xzrUxl7W8paNxT59+jSqOjFaQ6RZaNustSdYL+3aa6/N2jhO99hjjxRr7S7urTgH6prG/WbJ+pjXg1bgEbl1O+uDRERss802KeYayf1SRL5nUvtyMr1rvbHWV8TU1t4Vp59+evZ65513TvEPf/jDrO3iiy9u6jtyP3j22WenePvtt8+OoyW3zp9c41nfRvdHrOPDaxaR1wEt1Qd55JFHUsy1P2Kq8TxdLL9L0Lqd+3+tqUKbbB3rPLd1e96IiJ///Ocd/o2un1pnjvDaaX3JOrSmypNPPplijlOupRFT20zX8UnXXdxpp51SzL546aWXZsdx36O1nNgveX/CuVXZdtttU6x1iDgPPPfcc1nbmDFjUrzCCiukWO992De1xhLh3qlu3xQRscsuu2Svhw0bxpeuaWOMMcYYY4wxxhgzo+CHNsYYY4wxxhhjjDFtSJflUR20pVjTu5m2SOtf2rNG5BIBTUNmGtsf/vCHFGsa25Zbbpnit956K8WahrTpppumWGVCtL5mKhbTyiMi1lxzzRSrPIepskxRZjpYRPP2fq1Kd+vVq1ejSuHjOYjIJQia+ssUbMqImAYXkdu4aj9g+mHJnpjSCKaG6nWiBbFa41EmwRTSEioFOfnkk1PMFMj33nsvO46fXZKyRDfZKf7gBz/I2ijtoRV2CbW9V0ttwvRdppJrSu5f//rXFFMWqbapvFZqbVpncah96/3336/9vs3Czz733HOztko+8M4778SkSZM+0dTTZqGNJOfCiFxORjtizn0RuWyFlpgqH6O0kxKYiHoreJVYUVKqtrMc65pa3EVaNhYXWmihRiVT07lmo402SvGVV16ZtVGOR0tZtao86qijmvoelPHcfffdWRtlE5wvVNJ40UUX1b4/125+d30PSjmYrhyRz+20qy+hfY2pzd2RBk5ZckQu41IZEa9b6ffQslQluosuumiKOe+qfJDn/5hjjknx6quvnh33m9/8JsWVJXNHMNWb30HbdG3l3oB7AUpgO8knLsloFu43udeJiHjiiSdSXJJ+ci1UyX0dKrugzI1zo1ofU85Fe/GIXLrIfq1z7/LLL5/ixRZbLGujDLBVY7Fv376Naszzt0VE3HXXXbV/R2khLXxLcF8Xkc9Xxx9/fIrVApr7DUoreL4jIm6//famvq9K6OpQS3GWCCjBvUCpfEW0cCz26tWrMddcc3X4mZTE02I5Iv+Nzf4+yqgickk210Wd22n5zWulsk/u+XU/w/tWjj+dNznWuf+KyNdF3YMRXkeWH4j4aA921113xbvvvtvydVFtvbk3VNkZZV1HHnlkitUWm8c1u6enNDgiX+9UzlSH3oOoxLsO2nyrTEu/f92/U+rFkg8ReemJs846y/IoY4wxxhhjjDHGmBkFP7QxxhhjjDHGGGOMaUP80MYYY4wxxhhjjDGmDWlZTRvqdKnfjphao1+h2lna1z766KNZGy0iqXFXi8M55pgjxdQNqnaYFnhq+0do+ah6dv5m1i6IyPWG1MOyRktEbvOous+qdsjDDz8c48aNa7lGUW11H3vssRTT8jkir0FDbXTJfrdZTjnllOz1nnvumWLqIdV2j/pF1hbo6NgKtbFl/1H7cq1dUwfPo1p+n3rqqXzZLdp9tZNVu1lCHf5Pf/rT2uOa1RWzb6v1LDWwrK1QqtGh8xFr2tCqcrXVVsuOY50ItQIl6623XopZGysit6VlfYKIj/S1b731VnzwwQczRE0botbLrBXA+mGq5da6MxVaV4bvp3Mt50nWI9FaJXX1iyLyccsxq9aNrK9Sqs8SLRyLPXv2bFTrjtrLsn6Mzre0+Rw3blzt+5fqEXDM8fxprQZC3TT14BER5513Xopnn332rK2uNtFNN92UvWatN72mrMXBWhBqz87xrH1t8cUXj4gPaxCMHj265WOxZHXOWgwReT0G6uJVM0/23Xff7DXXCNrYPvPMM9lxrDNTqoXCfZXaN/P7zjfffCleYIEFsuNoLUwr+YiInj17ppjrbKlO3dOY3LcAACAASURBVMfQsrE411xzNap5XGvglWp5HHjggSlmnQWtOcgacVdffXXWxvpurOOotSdIsxayJVhDjPUSIiLuu+++FOv10TFXB/e2rO+kfNI2w0OHDk0xa4t1xs6e+w/uAXStYl8aPnx4ivVacw7VuZbjm/soWsJH5PdPWrOjZPNNOOdzPYmIWGeddVI8dOjQlu5Rq/lfrao32WSTFOu9HuuS8vzpGsk6pUcffXTWxvPCulvXXHNNU9+dNVQj8npdpXmuVG+1u1luueUi4sN9xfjx41s+FnWNYI3SZuE9fkTE7373u9pjWauNa5CuaXXovuHwww9PMefqiIitt966w/fgPB6R1+nkGhkRsfnmm6eYY/G6666r/Y5VzacKmT9c08YYY4wxxhhjjDFmRsEPbYwxxhhjjDHGGGPakC7Lo9T+6+23307xiBEjsravfe2jDB9agzHNNiK3ECvBNHNKJiJyqzemjWqq1O67797h+0XkqYpMgVLbXFqAV6lpFbSLphRLU7to71aS43RH6mmVZl7BdFCVRTCtnZIoXtuI/PpqiuGll16aYqYFaxo4pTQ33nhjitUKj9IA2qEqTClleltnYMo/LZI/Dv6Wm266qWWpp4MHD25Ulu2nnXZa039XNxY7w/77759iyv3UJpup/v/5z39q348SRLUUVavNipKEQW3OacPJNGeV1hBNIa7mmf/973/dYqeoaZJM6VbZGe15aS2rdvO0sKQld0TehznvaF/iHM15ktc2Ik9VVkkBZSOUYG6wwQbZcWqFTDiXUM5VQmWvlCPde++9LRuL/fr1Sxa1XZWIcD0SSWXGzjvvnL0eNmxYimmnqeODksSSHe5Xv/rVFK+//vpZ22GHHdbh32i/o3xW5VF11u0qc6ZdqtpRU/bSHesire0jPpIpR0ydXs/fx70UJYERuZxJ9wDs9yeddFKK1dqU6fbcU2jKOfcluo5TOsU1c955582OY7o4+1hEbq/LPRb7TkS+F6MsNSKXZh1zzDEtlSpWv1ktrjk2m5XMdGZ/TMkHpQO6znJPqWtms1CWRmmISsAolyqtd1wfdtppp6yN35fS9YiPJAG77LJLPP744y0Zi4MGDWpUkpbzzz+/9rhvf/vb2ev5558/xeecc06XPrvueqtsmOOZ67OWFeBrrj8R+X74gAMOSDHnm4i8L+l+W0tMtICWjcWBAwc2qvvE1157LWt78MEHa/+OexrOG7ru81pR1huRy/u5/+vMXpmw5ICWI2A5EM4jKs+mdFvnBN4/Dx48OMWl/YRK9qo1/957740xY8a0fF3Uvse5n/KxiNzCvnRfz1IGLN2gbeuuu26KdW/OOfTuu+9OscpNeQ25/kTkMi3KmXR+phRcS2Hw+5ak/lxrtSSHYHmUMcYYY4wxxhhjzIyCH9oYY4wxxhhjjDHGtCE9P/6Qj/jc5z6X0ri1Gvj222+fYpVHPfTQQx2+n6ZNMd1KnSp++9vfppifrSlbTIuj04M65WgqVh1rr712itURgqmoWomfrlDHH398in/xi19kxzG1Tp2lOiPDaZbZZ589pW7feeedWRtTs88666ys7Z577kkx07H1PYimMzItjKmsKo/SvlWh8jReD/adiDxNken1lHlF5LInTTVl1XnKbxTKzFRqd8kll9T+3bTw7rvvNl0JnzDVllIFdYqh5KFPnz5ZG1PLmTqtTmJ1qcbqQEFpIWVzJX7yk59kr1ldnv1YKaWI00VBz21VDb6rDh8d0a9fvyQHo+tdRC4nU+cnpt3ynOs1pHRUK91ffvnlKaZMQuUUHFd0VVEpk7omkboUXx1vnE8p7YrIpQff+c53Uly5elVQOkPHwY6ObRUTJkzokiyKbmWaXkw4JuiAEJFLWik90/WNUjT2NZUfUFZVkhjQHUElGewznOcjIqZMmZJiyqNUqkuJAFPHIyK+8pWvRMQ0uRVNxSKLLJKcCpnGHpHPBUzTjsgdhCjHUXcnzrVnnHFG1sZ554orrkixznGEslF1TiE//vGPs9dMvSeapv3888+nmE48Efm6Tjcq7Ut03mDfjIiYe+65a7/ztDB58uTUr9QRhP1F5VCU+zO1f/nll8+OoxuTwv2aSrMIU+4pc3rqqadq/0bXYO5nK5l0R+yxxx61bYRzjK4VhFLmiI/GMPe708qoUaNqZVGUhKhcXh1hKnRdpAuROkgSSm7OPPPMrI0yCe5zVAZEVPq97LLLdngcx5S+Lsmh6BiopRYo1yw5z7USrovquke+/vWvZ695T8HvrWskz7WOdcpreZ+pUlLuFzhH6/6PkiiVvr777rspLl0frnelsc55Xx3HuKfT+6bS3rarzDXXXOk+SOWIXFtUGsv9IGWyXP8j8vVUZcO8htdee22KVVqn7oYVJYmS9pdtt902xdttt12K1emR97C87hH5/Sgl1uoszT2DtpXc+SqcaWOMMcYYY4wxxhjThvihjTHGGGOMMcYYY0wb4oc2xhhjjDHGGGOMMW1Ip2rajBkzJm699daImNoyq84yMSKvJUC9+/Dhw7PjtI4Nqatzorp2fg9qRlXjX+niIz60SCO0SaTmTjWon/nMZ1Ksml5qx1nHRvXyJUvDygZY7cqnhR49ekxlaVpBLaVq/ljThvZoqkctQd08Ne2XXXZZdpzaCVewfkNExMUXX5xitY+nZvS4445LsdaYob2bQqtwrQ1Q971aea1KTJw4sVbDSstOrZ/A2iC0FlYNPvXRTz/9dNZW1X6IyMeO1hChXp+a+UUWWSQ7jvaHainKcUtbbK15wvGsWlOyzz77pJj9IqJsTag681bw3nvv1VpfUs9L/XxErr3mOXn00UdrP4u64oh87r3jjjtSvOSSS2bHNVt7gjVTVA/O6816DlojjP2F9TAUrifal6hR1xpY3UXPnj3TnKr9hPMBrUwj8vGitbAIx5vaXXJ8U/etNGsNy9ojusbzNWtjqcZf5/M6aDPM6xaR/85qz9GdjB07No0D1e6zdobW52N9pdL5Zz/Q88q6atS7K6znwTo4JbQWR7M1TmhHfOWVV2Zt3GP95S9/SbHOuxzDanGrNu6tol+/fmmcnXLKKbXH6ThiHRuidRY4F3/zm9/M2ri2clzqusgaRKy7oHbEnNt1TO21114pZn2puvqREVPXeuN+T79jHSNHjsxes/Zaq5hlllnSfk5r+XDPo+OIdVxY67JUP0TrpNx2220pZn9Wi3j9vhVLL7101rb//vunWGsF1c3JnFMi8vuREqXvyL6kazrXJd1XTQu9evVK+5i68RVRrov5t7/9LcWck5Sllloqe81af9xLaW0t1vfRvkbWX3/9FLPumL4Ha8Kx7lhEuW7dQgstVPsdCdcRrVvUHbC+VOleVWvvsA4j69jo/e+FF17Y1PfgeqdW54T3gVpT89hjj03xz3/+89o2MmrUqOw1bb51vWfbrLN+lA+jz0ZYi6mZGjaKM22MMcYYY4wxxhhj2hA/tDHGGGOMMcYYY4xpQ2aps+Xt8OBZZmnqYE3vY/oYrYTVSpIWe2pByVRIpjv27ds3O45pT0xL0jQspnevvvrqWRtT7mnJpen8Bx54YIpLkoxW2O01Go16/7JOMMccczQqSZNaxvG1prxTJsZUMpUyNZsuy1RdpslH5DI0XkNNafvjH//Y4XERuWU57eg1pZnpzmpbvPPOO6eYNrl6blTmV+CeRqPxtY8/7OMpjUXaK5ZkPZQqaqolbXw1XZPpjrT6U0kL03CZSqgSMv4dLfUi8nNN+2CVxVDqpba2tAItpYHzGut4HjJkSER8OC9NnDixJWOxdA15bfQ7c77SVF2yxRZbpPiCCy7I2saOHZtippRSFheRX0NNjW8WSi1pmb7eeutlx9F2UVlhhRVSrFKLZuHvnDRpUsvGYp8+fRqVvJBWyYrKH4YOHZpiSrm0/3KtUihVPfnkk1Os8o/x48enmJJips1H5JJQ/b7VGIjIZRi77rprdhzT3Uvno6tU/eSKK66IkSNHtmQs9urVq1H1dbXtpXxgxIgRte9Ba2GVKpT2B5yvad+ssm1+Ns+/zvGUiupcS0k3U7jVipVzzMsvv5y1qVy2QtdF7nUofYyYSrLSLesiZaQR+bmgJC0iYsCAASmmnELPLSVXKr/k+CAqHdhll11SzH6h0tRm4d5nueWWy9q22mqrFGv5gbp5lFLmiHzOVlZdddWIiLj//vtj7NixLRmL/fr1a1T9RSXglAqpbPivf/1rh+/H8x2R/25dFym9ePLJJ1NcZ8+tqJRwrbXWSrGeV86Nq6yySor13odSL5US8b6oZDdOdL8t0tRuGYsrrbRS7XG6b+TrDTfcMMVc6zr4rNo2jsuqv1ZwfJfKc5T44Q9/mGJK3rjmdobNN988xVq2QMt8kKr/77fffvH00093+x6Vsu0jjjgia6PEmP1Nj+NeRPc95KKLLkoxS3VE5GthyU6b94T6HhxH48aNqz3uW9/6Vop171RZo0fka7fet55wwglRB9eAxx9/vMOx6EwbY4wxxhhjjDHGmDbED22MMcYYY4wxxhhj2pBOyaP69evXqNIH1bWJLLPMMtlrpgrTLUUdENR1pBnuv//+7DVdOFgNnI4jEbkrwL///e+sjalYTH1j+nlEXn2/FWg6FytLt0oeNWTIkMZPf/rTDtt4HtSx45hjjknxr371q6Y+S6VrTHtstt8x3Y3pZwqrr0fkaYX77bdfiik76QxMp7755puzNqZhU4oTEfHAAw+kePTo0dNFHkVUbqSpt3UsvvjiKVbXLrLAAgukWGUcTDOnS5c6LBBNc+3fv3+KKelRB4RmpTul60h5HKVxER/NKyNGjIjx48e3PPWU0q+IiH333TfFmsJ99913p5iyhkGDBmXHMX2/NN54LrvquMR5kq5uCuc4leRx/mcqa0TuxkKZqkoS6H6hEgg6WEyYMKFbxiKlmPqZ+ns5Nul2oVISSuCYnqvvT5iWH1EvCyxJSBTObVx3NW2djikKHTWYPk55SsSHTpUVKsGtXNJGjx4dkyZN6vY0cKJzFyVGJekzZRfqesNryPGhbiZ0Pmx2/TzggAOy10z3LrldsW+W1gxKwii9ishlQUw5j4j42c9+ln2tVkoVq3XnhRdeyNo4V6qD5ttvv51i7iF1f0lUmqpyzwp1vTn77LNTzPGgTpa8xpTqRORz9nbbbZfic845p/b7Ui4bka+ZJRfNEtV+6qabboq33357uo5FypkjcukU27bccsumP5slAk488cQU6x7okUceSTHlRSovpsxcpWvcz/A7lpz+1JGoToLDcRmRryl670bnpfvvv79b1sXddtsta/vTn/6UYnVoq7sP0TmEJRvU3ZWOatyDU4oaEcF7od133z3F6p7L0h0leRn3cYsttlh2HGV5vPYK9y0qc1b32zpadb/Y7FhU6u691QmNMiK9n6ZkkOi8Tqkt90d6X89nD3R0jsillbxH5P2Hvoc63bI0CO+f9BpSEquyZNlDWB5ljDHGGGOMMcYYM6PghzbGGGOMMcYYY4wxbYgf2hhjjDHGGGOMMca0IT0//pCPmDBhQtJD7rDDDlnbX/7ylxSLneNUurAKrWFDvXilW/84qD9WtI4NefPNN1OstXVYn4GWcJ2pYUPb85J2nFp+1rCJ+EgjTn3/tPL2228nXeQvf/nLrE3r2BDqe4meE1p5qwUoNZ2si8N6ORG53pAWm2qjSotg9r+I3E6xVMeGfY7a3oiIv/3tbylm/RPV7lM/WzqH3YXW+rnqqqtSrPUTmq1pU6ptQVZcccUUa00baoR5nrUeA+tCsJZLRP79b7zxxhSPGjUqO65U04Y6Y2pStZ4HrdtpZR4Rsccee0RExGGHHVb7OZ2ld+/e6fepTpfaZdVyUwdLe0i1s9fXdbCOjdpul2okENax0VpWtDplG+08Fa0NQ3vsjTfeOMVag4eaYNWNq7VoK6n6sNaYKdV+Y/0mjh3+1oiIAw88MMVaa2nttddO8W9/+9sUs0aHssgii6SYNRwi8vOudYXq6nuUarlo7Sn2Beq+9b3ZT6gV/6Rpdl9Cm96IfO2jHWpEXuuIe4CSBWoJ7r+OOuqorE0tbyu0jhJtzrV2C+HcpPMU+yrrykXkdqk6l08LEydOnMomuiO0zgn3GRyLWjOL9bRY+zAiYv3112/qO3I/wvGrtW9Yo0w/i7U5OB/qGkxbbK3xULevVOtxzmHar7tqk9wsrAUVke+1tFYEfyvPT2cszBdeeOEU015Z7dF5HnbdddcUa80r1m7SfQ/vi3RPWUfJRvrYY49Nse7teQ7YXyLKNeimhVlnnTXN4yVLbu2XhPdHq622Wu1x+nsnTZqUYq4fHL8Rec0h3utp7RsiFunx9a9/PcW0ZNe9N/cjpdqXrBOp35fjoZl5bloZNGhQqjnEOk4R+bqlbVInKcW63+C8dtttt2VtrO20/PLLp1jXYN5zsVbSjjvumB03efLkFOs9+corr5xijkutQ8hnA3yGoHB9OfPMM7M2zj+sZdgszrQxxhhjjDHGGGOMaUP80MYYY4wxxhhjjDGmDemU5XePHj0aVeoQU5sj8lReyjO6ikpmmrWZpiSHsotXXnklO47penvttVfWRrtopn925nfRUrKUqk6Y3h7xkTXkiy++GBMmTGi5hZvaXjKlUFNnmaZIO9AVVlghO+7CCy+s/WymgjF1UNPMzj///BQz3W3EiBG1790KSmm0tACk/V9Eniqrqd6SDtgyO8U555yzUaXZqySL10etH5l6yXTsgw8+ODvuuOOOS7Fa7pbkfnXQ/vuEE07I2phyqNC6k1Z5JVQO14oU7krm8c4778QHH3zQ8rGolpiU2ah0hH2RfVYlgpzz1LJ0pZVWSjGlYJrmWgelbxERp556aor1/FOSQeljKWVaafZY9lVN5ReJZ7dYmyqUgXA9isjT1GlprpTWaP5GpvCrzI32mZznjz766Ow4pmNr+jVTySmL1PR2WoB3Vdq0zTbbpPjcc8/N2qo5YezYsTF58uRP1Nq0DlrJRuRSixIleRpTyWkvrXbpRMcNxz3XJsosIyL+/Oc/p1hTxAltpHXt32CDDVKsdseckx999NFuGYsqVeEcuNlmm9V+n5KMkna/KpMgvCaaps99EKUpCy64YHYcU+xV5lY3H6qtN9d/nWNoH12S3TRLq2yGBwwY0KjKHuj8QdmeSvoo4+N9gMpkm5WVUC5N+URExJFHHpli7pVKUr9hw4Zlrykf5Hp/wQUXZMex5ICy0UYbpVjLKxD2R5XT83zcddddLRuLvXv3blR7UZWyUT6jv5frHechtfXmuVY5E6U2/L0qsfrDH/6Q4kMOOSTFlNpF5BJBlUwSzoEqwaQcXPsgpVm8H1L5NyXveo9WyYYmTJjQLeuiliKpk0t3Bo6dffbZJ2ujjPiGG25IsUoJeS/JvaeOWa5HygEHHJBi3qt0piQKxxivzVZbbZUdx7IbHyPZt+W3McYYY4wxxhhjzIyCH9oYY4wxxhhjjDHGtCF+aGOMMcYYY4wxxhjThnSqpk2rdd8lVH9G+6/Kgixiais+1tphrQ/W+YjIa0GorS2twlX/Seqs3hRaKKoFLDXmahdLWqUXnmOOORpVHQS1aqU1suoln3766RRTU6jH0VZNrdl4LO1BeR4jcqtLWqdffvnl+nMSWrtloYUWSjH1omohef3119e+Jy3GWffjrrvuqv0bhbrrp556qmV64T59+jQq7WV3W//169cve81aP9TWL7vsstlxrJnDMaX6fI4/tcejjpmaf7XIJlrLoM6mV62pm7WEbtVYLM2ndbVKSqjm+L777mv2ezR13Fe+8pUU0zY6Ir/W1113Xda21lprpZg1bWiHGpFf387Uu2kWsbpu2VhcdNFFG7///e8jIuIHP/hB1sZrohpw6q9LdZdYY01tgVm3iJ9NC9GIiJEjR6aYc+WVV16ZHceaGKw7EJHXdLv66qtTrLVXmq09RXQNYE0KrWVWndOLL7443njjjZZ0lN69ezcqbbzWcOG5U1hjg99T68zU2SuXUGte1jqgxr9Pnz7ZcZxfqdVXWDdI+x/3RwMHDszaWGeM+xmus/q9OP93QMvG4sCBAxvVnkrrizQ7j5K99947e80aGM3COjgREZtuummH79+Z+fvvf/97h+9XQut+cF55/vnnm3qPEt2xLnJ//H+fkWLdc7PmE/dE+++/f9OfzXG7zDLLpFjt3G+88cYUs/5JqZaV7j35Hfm7tLYl53Kt/8daVByzamvO+yKtbydr1HSp9cYxofWuWDORc+p2222XHcfaiFqfk3B90mvAfs/312vAPqRzpdYDqxg6dGjt9x09enTt9+0q1T59woQJMWXKlJaMxfnnn79RWdprfR2u83pvxrmX+7oddtghO+6ee+5Jccn6nXVhWPNU4bjU/Qtr32idK372EUcckWK9d5gyZUqKuR+OyNfnF198McV8dhFRrq0juKaNMcYYY4wxxhhjzIyCH9oYY4wxxhhjjDHGtCEtk0fR2rRkeyfvl71m2ram9zGtnvay77zzTlOfpbZbtG2mzWlEnh5Fy2m1GKTddcnquqtUcqU33ngjJk6cOF2tTdXSl+eLtqRMsY3I07E1Jbyyi1fU6pJ/x+u77777ZsdRskUbNYUSK02RZtq2pj6rVWeF9gNaeFL21QHdknqqkgyVUDSDyhN4XlTWQfnaYostlmKV991+++0ppuWwyqPYZ2jzFxGx6KKLpphyNbFSj2uuuSbF2ic5x7322msp1rT1xx57LMUqCav64dixY2PSpEndPharlNSIqaVglPnw2vB8a9uXvvSl0vdo4tvm0BYxIuLkk09OMaWEEfn4vvvuu1PMdO6IPOX/8MMPz9poic75X8fiU0899bHf/f9o2Vjs1atXo5KMlqQ0CiVAPXv2TLGm7lLWsM4662RtTPdmar7OtVxbmeqtVtSvv/56ipniGxFxxx13pJipx3oNKLN9+eWXow72cV3HuSZrvybdIcmgnDki3ztwnomotzTX96CMaO21187aVE5YoesR7Udpq77JJptkx1FaqVI1SvIeeeSRFKtkgtJRlY3ut99+HX6PrtqcRwvHYt++fRuVNFrnzWblr81KU3/0ox9lr3nNf/Ob39T+Hccs980qs6A0X+UUXEMpWyzBz4pofp9OaDkc8dGebO+9944nnniiJWNx8ODBjUrypeUEKLmjjKYzNHuvQimH2mk3e+64V1J5CWXhlCqqtfVf/vKXpj6LeyfKRCLyPZDCfnbVVVd1yx5VbZr5mZScROTyy9NPP73Df4/I90G6ZnIfcMopp6RYJYI77bRT4Rd8xL333ptill6IyPeNlNT96le/yo6jfblaSVOmVSoDQalqSWLVqnVxwIABqZzGzTffnLVxHtMSDdyncM3UPZ/e5xOeI+49dE7mveof//jHFB922GHZcVtvvXWK1Wb+4IMPTjHvM3jvGJHPtVoiQO9JKrieROT7JUo6O8DyKGOMMcYYY4wxxpgZBT+0McYYY4wxxhhjjGlDOiWPajYNXFO/TjrppBSzUri6DbD6slYUZxob05VVWlPnKlP6nYceemj2mmlsTCemBEBhWlxEnnrcVaoU29NOOy1eeeWVlqeBqzyEKYXqGMEq2rPO+tGzPk1tpAyG5y4iTyem/OZb3/pWdhyvIZ2A6lyAPo7BgwenmH1MUfcGysAoZaB8LiJP4aSzWUSeEvvmm292S+qppmkz5U6r5TOlkum0KqliSukvf/nLrI3uKpRCaDX1vfbaK8VMM1Q5DsezpsDSQaPkrnbiiSemWF2JmKLMNGT+jVJK4+wOSQYr20fkMi6Ot4jcpUtdgsgbb7yR4rnnnjtro6yDfZapoRG5SxCdciiRiMhTuldcccWsjddw+eWXr/2+zcq02F94niJyB4htt902a+M4ueiii1o2Fvv379+orsmDDz6YtY0fP7727+giwHlTpSrNpsevueaaKdZUZjLPPPOkWN1IKNNSRw4eO9tss6V4gQUWqP2sVVddNXtdJ3XStYIpxSofquSozz33XEyYMKHlY1ElaJzD6dalrznP6LrIeVJdG+tQVyC6j/CcqxSOf6fOmM3yjW98I8V1ErDOoGsrZQp33XXXdHGsIepKxLWFTqOUCypMqY/I52XuFeliWkL3qEy3p0xYKckUmoV9Rl1cuGaeccYZte/RqnVx4MCBjWo+VLkrX6sknjLx0n6f55JjNiKXTnG+U+kfZYfrrrtuinXfQOcYnU/ffvvtFFOervuc4447LsW6F+Pf6dpDuF7rfYzcy02XsdisBJFuXp1x4KtzG+IcGjH1XrEOzqPq1sd+wnlf1z7K9tWBSiRqTX2nUlmEVo3FQYMGNar1sDtKgJT2LHXzGvcoEfnenHOmypd4z8nPjcjHXMlpjZTGM+G9Y0TEk08+WdtGN62wPMoYY4wxxhhjjDFmxsEPbYwxxhhjjDHGGGPaED+0McYYY4wxxhhjjGlDOlXTpm/fvo0hQ4ZERG63rGgdDVotag0CQp1xqX5FCdZpof5W9Z7UVKueftKkSR2+t9bAoFVxqXYB9eyq/6fVG2uFROQ2qN1RR0NtvR966KEUq46ddRBYK6OE9i3WZthyyy1TfOaZZ2bH6feqez/a/Gltj9VWWy3F1K2qprXu/SLya0oLVK27QtSW/JhjjuHL6aIXplVlqR4G6wyorTd12qxrEpHXqmHtG17TiNzyleNP61rQvlTrS2ntlArWG4qIOOusszo8TmHtKbU0pk5d68hUrydNmtTSsVjVkJk8eXKX3mPvvfdOsVoE81yWLGhpWa926YSW3DpWWD+GFpgREbfcckuK119//RSrBpgW0KwrETG15WqFWtCyvoe+P9eep556qlvGolpJHnnkkSnWWibUxrNWgdbMItrvOW4ZsxZARN7vWS9OxwD14tqfdF2v4HwQkc8JUE+mbwAAIABJREFUCtc0rvdql6n17urojnVRfzfXFta7i8it6NlHOaaU448/Pnv9i1/8IsWsC6Y1C7hPYZ0o1d1zrOtaynlm2LBhKda9Dedd1gKIyOvd8DqprS9fX3LJJVkbbc+vu+66bhmL2l9ZF0Fr7Oj6V8FaUxERw4cPn8ZvmI8/7g/UXpZova/ZZ589xePGjav9O45nWs1GTG2nXVE6byW6Yyyy7l5ExF133ZVi1hCKyPevpfonnHe1b5fqNRKOq1NPPbX2uGZhXRRd07guaj8orf+Ec63OTbwHiRbvUav+p/W5Lr744tq/O+igg1J8+OGHN/VZWr+ONWg222yzFP/1r3/NjmNdpE022STFeq/BPY1eA9Z3ox249os99tgjxWo93ux6x/2N7sGq7zFhwoSYPHlyy8diCa3zw3HFvZze/7KeUek9iM7dPM9XXnllitXyu1Qzsdk6NiVYG5LPL3Te5RyjNVD5TCFc08YYY4wxxhhjjDFmxsEPbYwxxhhjjDHGGGPakJ4ff8hHvP/++0VZVEWz6ZRqiUYZES3hIsp224QWjUy3YupbRJ4+vvPOO2dtTLeiXbTaw/Xr16+p7/Tyyy93GEfkVoKUk3QXPXr0SL+9lNbJdLGIPJ2vJI/63Oc+V9tG2Q6h9EphmivlGRG5va+mZlOOwzTac889NzuOdpaaos80ZqZRqkU5U5z1/ZnSrjZ/rYLpnxG5JEpT82iZKWmxtcdpin3dWFQLPKbhUl6mn0sZms4da621Voqvv/76FJfkkxtttFH2mrIFppJTDqVoavSGG24YEWUr5a5QyRX22Wef7N85zw4cODBr49x40kkn1b43ZSr6/pz/SpIowjn07LPPztooe7njjjuyNqbxHnLIISnWa0hL7pLEhqisjBarmlpPa8jugvbQEbkE4cADD8zaKINacMEFm3p/nSvrZB1ctyLyc81YpcGco3QN4PXh/NrstYrI52na1ZbSw3VNqcawpodPCz179oy55547InLJQcTUexFSZ9+t5//NN99McUm6wbVq8803z9quuOKKFPO3jxgxovb9dI2ntJdr+p/+9KfsuLrU9Ih8HHEOVft4nXPI6NGja9taha4l3L/ddttttX/3u9/9LsWaYk+OPvro7PX+++/f4XG6PnMdu/baa1Os8ihKK5WSJIp86UtfSjH3OhH1cnjdM/A8qqyl6jcl2+bOssgii6TfzjknIt+T6V66bg5RqSjtlSmniKifU3QPpGtyhc7jL7zwQoqXXHLJrI1rA88rpYMReT9Tya/uuergnK/yq9I+cFqpSk2U5FA6bs4777wOj9N1Zs8990yxynzJ5ZdfnmJdSygzpBWz7v8o69F9BG3iN9544xSX9lVqDc++e+yxx6ZY99qU02hbNTY4rrsTzhO8X1B4T17Zh1c0K4+iBE2txzmuuB6VSpYolM7z/dgnInLJsvZpfvYGG2yQYn2+QMmtym1pU673oxXOtDHGGGOMMcYYY4xpQ/zQxhhjjDHGGGOMMaYN8UMbY4wxxhhjjDHGmDakU5bfzdp/laD16COPPFJ7HGtgRERcdNFFKd56661T/OMf/zg7jnVhDj744BSrJTTrssw333y134NaPf2+Jds/WtCx3gPtbyNybePIkSOzttVXXz0iIu69994YM2ZMSyzcZp111kZ1btVivAR1fosuumiKqbP/OGhXSwtArQND/SjrUuh1Gjp0aIq1xga1n/fcc0+KaTkcEXHppZc289UzVEesOmPy61//OsVHHXVUy+wU55lnnkZVQ0B12dRCLr300lkbNda0ntV6OzfccEOKS/Um2C/U6pna2pdeeinF6623Xu376Rio6kwo1KVHRNx9990pVl35vffem+JSDSnW8NB6N6NGjYqID/XMU6ZMabmdotZk4u9WbS4tfXfZZZcU63tw3qmsxStYt+NXv/pVilWHTY0/+4GuG/ws2iBH5DUcWkGzWupSPadoobVp3759G9Wcpd+nmsMjpq77wDoStHpWZA7J2tZdd90UX3311SkuWRqzb+l4W3HFFVOsY+Caa65JMcfYwgsvnB3HGhi0nY/I52zWVynVlyrRSpvhyhJU6wZVYz+i+bpIX/ziF7PXrF/BuhwR+fzE+bq0P+KapvX6WDOHdqv6PZqF1ykinwdoccu6ABF53RqtTcL9XLTYZriujbVCdE6q2wez7lTE1HWpCGs03XnnnSlWm1vW32BtHc6hEfkeRqxgs3o3tBJWS2Md32SNNdZIMfelpXlT14dqHnj44Ydj7NixLRmLAwYMaHz1q1+NiKnrP3Gs6F6OtZ3YF7V+CM8l1z6FtY10j8o1mfcSWreGNVRYWyUinyNYG0utiWlBrGs8a2exdpLW26QF9sfQLWOxtB41i87LpdqaHKccw9/4xjey47gGc7+qltz8bL0GrJnE/RJr3UTkayvX6oiIQYMGpVjr4tXBejwReZ2rVq6LVVzZt1fQRv7222/P2urmUx2LpfWU55V1cfTccQzw+uqzAdb6VPjbhgwZkmI+a4jIr6+Ode51+J207h3nrY+5f7bltzHGGGOMMcYYY8yMgh/aGGOMMcYYY4wxxrQhnZZHVal7+ndMA3zllVeaej+1zWM6rVpmMl2N6YMqN6JlFtHUOlpwasrnrLN+9CxLbQW7Ai37KDuJiLjuuuuaeo9WpbstsMACjUoioqm+tFzTtFSmnTGlUKFtO1P+InKb2Gbfj/Tv3z97zfQ5TWdkemxlOxgRsfLKK2fH0dpPUzaZAku5nqYP19m+dsB0SQPnef/JT36StTG1sGSbXbo+TImnXaRaPT/66KMppsSjlNaqUIZB+1JaO0fkqZrsg/r6D3/4Q4p1/uG8oqnqpDtST0tQUhqRnxOmizOF9P/eP8Wleb5KRY+Yek6gJTT7gaaNlmxDmQY+YcKEFKuUqFnpLOdynacWW2yxFGt6LC1/o4VjcaGFFmpUKfcqT9D5/pNi3333TTGtKkvXTfsM5YRnnXVWinX+W2211VJ800031b5/SaZVYq+99oqID3/HG2+80fKxqBbflLWp3JT7CMYKLUB17uIY4xz697//PTuOaxdlciqdaRbOI5rqTblaaZ3oKpQ6v/baay1dF6tUd677iu75Xn311Q6PUxvzknUxaXZ/w36v+yWOP5XMcF456KCDUqw259wHbbHFFrXvscIKK6RYU/ZpUatSuUpmNmnSpG5ZF5dYYomsrWRvv8MOO6SY845KqVnKQO3YOX+zj6gUZdlll00xpeWUwETkMkbdv/L6crzp2sprqlb17Bf8jiVLeJ3feK911VVXtXQsVvdSeg1KspA62VhnYL/hOdLvceaZZ6aYsvOf//zn2XFPP/10inUNoDU10evNOYHSxIh8f1xim222SbGeQ8p4u0M23JlnBYRr31133ZW16Z6VsFwBxz1/Z0QuQ2PpBZ07eI8o8tysbASP23XXXbPjTjrppBQfdthhWdshhxySYt4jaj8oSQPZP+vGojNtjDHGGGOMMcYYY9oQP7QxxhhjjDHGGGOMaUO6xT2K6fYRERtvvHGK6ejUCtQFgNIsppJpCrG6cBC6AFAW8+yzz2bHMaWK8oyIPH3y9NNPr/0sou/BtORWpbvNOuusjUoGRalCRF4hn+cuIpemMGVx8uTJ2XF059HUeMqjiDoE/f73v08xJSzq9sPU34022ihr47El96MSde4m66yzTnYcXVX23HPPrI3957jjjmtp6qmmTFd0NY2xWVZdddUUf+c730mxOsXwXJxyyinT/Ll0QNJ+VyfZKrH55ptnr4cPH97hZ0V8VF3+pZdeigkTJrRkLPbo0SM5ualDFB3a1DWNMgw6S5Qq8zfrvPD5z38+e83UbDpS6GcxlZXzYkTEdtttl2LKV/fbb7/suJKDEKWbdenIEXl/1LWBUsjoJqkix0bE1K4KhPIUOhGWUqU1rVrlwRUlJzGiUk9CeWNE7lJEpzimjn8cO++8c4opTeB16wytWhcHDRrUqOZ1lWBQOvLnP/85a+PaRYk4ZYuKOphwHVaHMcL1lE6Mem0pM9f07tNOOy3FO+20U4rPOOOM2s9VZzNKhigp4Lodkfd9TUcXWjYWhwwZkuTfKk3kOq1wzNFlRB2xZpttthTrnN0VKFm64IILsjY6RKlslTIcSg4oGY/I+5q6hT755JMpLpUB2HDDDVOsDkjVevHKK6/E+++/33J5FN1DI/JrqusH3Z64l9ZryDVN5V6cD/nZxxxzTFPfXaGEgvKJiNz5j/dM6lhGdF5ZfvnlU0xpha7B7C+6D6dL2WGHHTZdJPxErzGvV0kmTVRaSAcmOoTpebn11ltTzLHDfVVEPgeqW+Lf/va3FHNNU6k031/vkW+++eboCHVsojPnySef3OHfRHSPhJ97lIh8j8z7w4hc2nvttdemmPfFEXnpApV7UQqoaybhGkR3SmWttdZKsa6ZnP/qpLKtgvLE0nofdo8yxhhjjDHGGGOMmXHwQxtjjDHGGGOMMcaYNsQPbYwxxhhjjDHGGGPakJbVtPnnP/+ZYq0vQmhnd95552VtJZtEat5ZI2HLLbfMjqOlMTWj1AJG5DUdaNlXgtaKERGHH354irUOQalmTh1zzjln9rqqa/Hoo4/GuHHjWq5R5HmMyDXOaod81VVXTfNns84CbU7Vpp11j0488cQUq+6etXXU5pm6U/YlatIjyrp0WijSPlRrybB+juqRRUM73fXCytprr53iZu3mWXsjIq+PQk3mN7/5zey4++67L8Ws7aPXm7VkdAy8/fbbHX4n1TBTr6q2rHVW0lq/hX1Ia+Z0R32p+eabr1HVjrr00kuzNmqy1daaNbKom1bdL8/dCSecMO1fGJRq5Gh/UR15Ral+jkI7V9acKMHaZBERd955J1+2bCzONttsjaqmj9ZPoEWk1lBiHSDan0vtnTjuuONSvM8++zT7nbLXdfMc7VUj8jlVr9svf/nLFJdq06luvSuwPoPqvquaRmPHjo3Jkye3fF1k/aSIiD/+8Y8p3nHHHWvfoxNa9Vo4TrUGC2sM8dqw1lREPje+9NJLtZ+1wAILpPjFF19s+juyNgdrdpT6HOtmRET8+9//5svpvi6ytlJExI033phi7lF1DHDu0VqFrPnA/cIf/vCH7Ljjjz8+xaxZMWTIkOw41rHhut0Z2F+1HlMdpdqK7DMRH52373//+/HQQw+1fCwqrAVCO+2IfI+h9RrrUGtnjj/WldMabo8//niKuZ/Rumy0dFfqaiaq5fdSSy2VYl1bWROJ92BqR8yaI+ecc07WJnWapvtY1HlOa0bWwXpilbV4BWsTcX/JumMRud35Aw880NTnsnZSRF5biX1Q1xHWGmsWvZfmNdb9NmuetbLWW7UWs15sRH6/WFpneB50v8H6kzpWeE/N+l5qA885gddX95O859TaUHV1MH/yk59kr1k/TGvfsW4U6/hMA65pY4wxxhhjjDHGGDOj4Ic2xhhjjDHGGGOMMW1Il+VRZ555Ztb24x//uPbvjj766BRT4qJp1JQpbbDBBlnbv/71rxQzrVxTlAhTpdTGi1Idtf1jSixT2i+55JLaz/rBD36QvabcgSmYnZEpLLfcchHxYRra+PHjW556ynSxiPxcqt1rszCdjNKAEmold+qpp6aYdnqbbrppdhxT7TRlbumll04xf4ta0zWbEkm+/OUvZ6/vv//+2mPlO7ZUkrHEEkt0+PldkZKUpCol202mdPfv3z87jhIH9nvaIkZMbY1YB9+fUoSIXK6j54P2jXvttVeK1U6Sqcdnn3127ffoDjvFEtV1rqAEhzaSmqK6yy67pHjYsGFZG683ZQx1VuCKpiNzvlYLaKYMNysboY1jR68r5pprruz1qFGjmnr/aKEkY+mll25Udr20YG0Vq6++eor1Gj/zzDMpLo17SjqbXfNpKx0RcdZZZ6WY9sR77713dhwt2VWWxZRuyhYpQYrI7eqVSr7x0EMPxdixY1syFnv16tWo+pLO70x1vvDCC7M2yhMoHzv22GOz47jHUKnL66+/nmLKgXWOaxaObd338NoQSvD0PZh2HxHx2c9+NsXsZzp3qFSEcI3ffffdu0WSQXlzRD4f6j6sJLupY8kll8xeUzJTgrJ9yvlLUMYREfHmm292eJzKP1ZZZZUUX3/99Vkb5SW0vFVrcI5vnVeqPdm//vWvGDlyZEvGYu/evRvVd1PZO9ePbbfdNmujHbla+pK77747xSussELWxj1LV0oCqIyKMmuVvXKMlaTBlN+rLTLnFd5zqGSSsja1sxZaNhbnnHPOxpprrhkREVdeeWXWxnIXlJxE1O9BVl111ew19xm696TcuO5zI/K5nbIslWy1An6nX/ziF1kb+y7vh1RuzX6tczvnn+mxR62k/RER5557btZ2yy23pHiNNdZo6rNU+kcpGMeHzrOUUc0333wp1nNHuIZFRFT9NCJf41WyyvfXvSxlpZSUTgOWRxljjDHGGGOMMcbMKPihjTHGGGOMMcYYY0wb4oc2xhhjjDHGGGOMMW1Ip2ra9O3bt1FpI+s0gxERv/3tb7PXY8eOTTHt0n76059mx1G7qdCikRpwagEjci0n7YNXXnnl7Djq5VRryte0ElbLM2oK9T1WW221FM8777wpbtZ2MeIjC9/7778/xowZ03KNIn9bRF6XRy3SWTuC+mrtPyW9bFWjJyLiwQcfTHHJYpK1DvbYY4/sONanUQ0zLdy6YgVZQi2/OzGGWqYX7t+/f6OqvXD77bdrW4o59iIievTokWLWoVL70rqaQArrrZTqS1FLzO8Qkde9oFVjRK5vZl2rEloz5+9//3uH36OkWefvj8jPQXfohdUOmvWzWLcmIrcA32677VKs35m1KEq1EwYOHJjiki0lj1NLdO1nXWHo0KEpVo001xvORfz9EXmtij333DNrO+WUU/iyZWPx85//fOOggw6KiKmthEt1ZljzS60lm4Waa1qca9/mGOvdu3eKaSscEXHooYfWfhbrPPG877TTTtlxiy22WIpZoyViah1+xUILLZS95vzN2jcReS2q7hiLJRt0hWsL1/zrrrsuO441n3SenDhxYoqp3f/Rj36UHXfRRRfVfo86tE6AWlhXbLzxxtnryy67rNOfpbULOCdoHR/WLxwxYsR0sRmmJavW0ahDbXvragJF5Fbrb731Vop1juI4or3zeeedlx1Hy2mtE9YstHNmfZuIqWvcVPB3ROT1H9UavrofeOmll2LChAktH4s6Z3JfWroWRGsmsr6L1ubk2sL9sNaE47zLenraX3Qdm1a0/h/3OqyVdNttt2XHjRs3LsUfUy+uZWNxwIABaY96xx13ZG2siae1am6++eYUs76Y1h699dZbUyxre7b2s7YVz0NE3rdLdZBKVPdpERH/+c9/ao/rSs0c3Q+X7rsPPvjgiPiwbuwrr7zSkrG40EILNao9AuuLRpRtrX/4wx+m+OKLL05xs+tRRP394sd83xRznEfkY7a0lvIZgtb3KsG932OPPZZird17/vnnp5jPKCKmsh53TRtjjDHGGGOMMcaYGQU/tDHGGGOMMcYYY4xpQzpr+f1mRDz/sQeaVrNQo9EY/PGHfTy+hp8ovo4zPr6GMwe+jjM+voYzB76OMz6+hjMHvo4zPr6GMwcdXsdOPbQxxhhjjDHGGGOMMdMHy6OMMcYYY4wxxhhj2hA/tDHGGGOMMcYYY4xpQ/zQxhhjjDHGGGOMMaYN8UMbY4wxxhhjjDHGmDbED22MMcYYY4wxxhhj2hA/tDHGGGOMMcYYY4xpQ/zQxhhjjDHGGGOMMaYN8UMbY4wxxhhjjDHGmDbED22MMcYYY4wxxhhj2hA/tDHGGGOMMcYYY4xpQ/zQxhhjjDHGGGOMMaYN8UMbY4wxxhhjjDHGmDbED22MMcYYY4wxxhhj2hA/tDHGGGOMMcYYY4xpQ/zQxhhjjDHGGGOMMaYN8UMbY4wxxhhjjDHGmDbED22MMcYYY4wxxhhj2hA/tDHGGGOMMcYYY4xpQ/zQxhhjjDHGGGOMMaYN8UMbY4wxxhhjjDHGmDbED22MMcYYY4wxxhhj2hA/tDHGGGOMMcYYY4xpQ3p25uBZZpml0V1fxJRpNBqztOJ9fA0/UUY2Go3BrXgjX8dPDo/FmQKPxZkAj8WZAo/FmQCPxZkCj8WZAI/FmYIOx6IzbYyZfjz/SX8BY0xEeCwa0y54LBrTHngsGtMedDgW/dDGGGOMMcYYY4wxpg3xQxtjjDHGGGOMMcaYNsQPbYwxxhhjjDHGGGPaED+0McYYY4wxxhhjjGlD/NDGGGOMMcYYY4wxpg3xQxtjjDHGGGOMMcaYNsQPbYwxxhhjjDHGGGPakJ6f9BcwMxezzDJLinv2zLvXfPPNl+JVVlklxYssskjt+11zzTUpfvnll7O2Dz74IMWTJ0/O2vR1Re/evWvf4/3338/aGo1Gh7HCzyodNzPC6z3rrPXPgHlepkyZMs2fVeLTdg26k7pzrv/Oa6/nv24smukHr5fHhzHGdB3Op3Wxovsez8MzHz169EjxbLPNlrXx+r/33ntZW7P3GsY408YYY4wxxhhjjDGmDfFDG2OMMcYYY4wxxpg2xA9tjDHGGGOMMcYYY9oQ17QxHaLaXOosS3VMWMdmyJAhWdsGG2yQ4tVWWy3FX/3qV7PjqP3ceuutUzxmzJjsuIcffjjFt912W9Z28803p/jNN99M8ejRo2s/S6nTJ5dqt3waaFbDXfc3vXr1ytr69u2b4jnmmCNr47HUAb/77rvZcVqPiFgjPDV117BPnz7ZcXPOOWeKOWY333zz7DjWirryyiuztosuuijF77zzTop9XaaduuuoY4ywxpDWG+rKNdH5sDQntKK2lcnR8801WOsq8FpNmDAhxTp/+tp0nmbXxU+qfoV+p7o6V56XOwfHFGuaROTjqLvPq6/b9EGv8QILLJDi7373uyn+whe+kB334IMPpviee+7J2p577rkUjxs3LsW+pkb5dN99GmOMMcYYY4wxxrQpfmhjjDHGGGOMMcYY04bMkPKoZlNPTdcpncc6qVRExODBg1O89NJLZ21LLLFEihdffPEUT5o0KTtu1KhRKR47dmyKJ06cmB03cODAFC+66KJZG+VSfP/O2C7W/U6njn9EV8aiyikGDRqU4rXWWitrm2uuuVL8zDPPpPiWW27JjuM1LllMe374kLq+rfKoNdZYI8X77bdfihdbbLHsuA8++CDFKs255JJLpu3LfsopyVEpS5tnnnlSPN9882XHMeX6hRdeSDHn14jmxwe/h/aZAQMG1P4dJY6MdQ34tNKV+bSUrr/TTjtlbf3790/xtddem+LrrrsuO64kNzUfQhlaRN7v+/Xrl2Ldt7Dfc63SfUWprdm+wO+hcwLf44033kgxZXMR+dz+aUXnXZ5nxnqdOK91de/Bz+6q7O7Ttu+pO0/NngcdR7y/OP3007O2L3/5yymeffbZaz+L4/7VV1/N2i699NIUH3/88Sl+++23s+N872GcaWOMMcYYY4wxxhjThvihjTHGGGOMMcYYY0wb8onKozSFjWmAmmLPdDWmnWnqKVNKS6lwzbredJVPSzqiXiem4C655JJZG9MDL7jgghTfe++92XEvv/xyiukm9I1vfCM7bu21104xnW0iIu6+++4Uv/LKKyluxXUpuTCUpDkzKqWx0my6Jt9D08q/9KUvpZhuYRH5uGcKqWkdHBPq3rXbbruleKmllkqxjntKZJZbbrmsbeGFF04x3dssiek8mqb/mc98JsWUozJlOyKXP7z11lspVnlUV74HHcYiIhZccMEUjx8/PmtjWjhlF61wsZpRmdZUfu0TlDTuuOOOWRvnU0rrbrzxxqY+69MOnQ6/+MUvZm2UUHC8Pfvss9lxdXIjXg89TuVqdfJWyjMiIlZdddUUL7PMMlnb008/neJHHnkkxdx/RZRlWjMzHFcq+VRnoAq6AEXk8pbumNNKpQrqjpsZKe3JS//Oa8y19NBDD82O23777VOsY6zZe0n+HdfIiIjNNtssxa+99lqKzzrrrOw4rtcz+zVtljrXPpW4cY9KmXBE/hyB8lCVirbD/OdMG2OMMcYYY4wxxpg2xA9tjDHGGGOMMcYYY9oQP7QxxhhjjDHGGGOMaUOmS02bOstS1cKzBgptgCNyLTF1faq/ffPNN1PMmgmqSaUt5rzzzpu1UXf/+uuvp/jdd9/NjqMOTrX71CBTBzcz6BB5PWnJHJHXvaCuOyK34R4xYkSKS7az1IF+85vfzI6jpny22WbL2rbaaqsU33nnnSkuacNLlOwUWd9jZqxpo7+3Wb1wnd6aYzkiYvnll08xx2VErru/+eabU6xWiM2ed36PmWEstgLWUth7772ztq985SspZj8vacN17j7hhBNSvM8++6T4vvvuy45rhT3qzI7WvaD1+hZbbJFivQYcO12do3jN+T0GDx6cHcc1gPr8iHy9ntnWxWYp1UBoheX3+uuvn2Jdn3ntaQNvW+eO0fprrBW16aab1v7d1VdfnWJdq+rqSOl1LO05CPvM3HPPnbXtsMMOKda+cM4556SY+yKdH7q7/mM7wXXs85//fIp//etfZ8d9/etfTzHntGHDhmXHcd7V+4e6+a9Un6W0xyoxs+97SjX2OIZ1jHG87L777ilmDZuIqe8vCMcLY623SvQ+ZOTIkSnmtfK8/CGlmpi8Np/97GdTvOaaa2bHcb5mHU19f+5Zzj///Oy46667LsW8N4nI5/XS3mZax58zbYwxxhhjjDHGGGPaED+0McYYY4wxxhhjjGlDukUepRaUTGei1dayyy6bHff9738/xUyxjsjT2piCxrSyiDzFnilztEONyK1tx4wZk7VREvXSSy+lWNOhnnrqqRQPHz48a2OKVSnFbUZMVdQUQ/Lkk0+mWK/NCy+8kGKek5L8htdw3XX8Uj7eAAAgAElEQVTXzY7jNdRzzNettmnT95vZUxibTedvNq2XqasR+Xh+4IEHsrazzz47xY8++miKO2MX/WlK724WnpP11lsvxTvvvHN2nNpbVugYKKWDLr744ik+4ogjUnzGGWdkx91yyy0ppsw1YuaUHXYFvR7f/va3U0zphs69lKNSyqtrdQnOxZRE6drKNf7FF1/M2rjWzuzzJml2niTNSmJU+k3ZsL43z/k//vGPFHdmPv00obIL7kspTYyI+O9//5tiWmirLKbuupb2KaW+wDG88sorZ22cE55//vmsjXMC51sdl+1gc9td6Pw3//zzp/i8885L8de+9rXa9+jXr1+KV1pppayNlsGPPfZY1vbWW2+luK6cQkR+7UttpT4yI95nfBy8r1xkkUWyNsqeRo0alWKe84j8fLIshkoaOYb/97//ZW033nhjinmPqHbRAwcOrG175513Usx+0tVyDjMDHJvzzDNPijfccMPsuO9+97sppjxKbdW5Tuq8znWScqv99tsvO26PPfZIMctuROR7Wz4bUJlc6d63GZxpY4wxxhhjjDHGGNOG+KGNMcYYY4wxxhhjTBvihzbGGGOMMcYYY4wxbUi31LQp1SihDrFUp+C9996rfQ+i2jTaCVP/r1aptLtk3ZqIXAO56KKLplj1ws8++2yKVS+sVqcVM4MmkTpQrZ1ALajqMbtiy0ybb9WQU4ev9sHHHHNMikePHt3U53aVmVnz3RF141n7dl2dE62fwLFy0003ZW233npriputgVGqGTEzjL9WQGv1008/PcVaH4Pw/KttLdu05hWvN21UDz300Ow4asUPPPDArK3OKvrTALXdOgd+61vfSjFrK3DcRETcfffdKabGWscK11O9jqxVwzo266yzTnYca2XoOjiteu6ZAT2vfM3roVp49nsex7EcETHvvPN2eFxEvj4//PDDKe7MtWB/ZKzr+8xwfdVedokllkixWt1zv8MaFV09D83WNOKcesghh2THzTnnnClmjZaIvO4O99szw3UrUaoHxX3jiiuumGIds6zN9e9//zvFWreG77HWWmtlbVzv/vOf/6SY9x8Rzde7+bTB8fed73wna2NtzQcffDDFvO+LyM/1hRdemGK9n+P76TXm9eH10HpJzY5nzqMz+/Xl79Z6fdzbHHbYYSnWWjV8j9L9J+sI6f6Vr1kPSddWzhe673nuuedSfNxxx6VY6zOW1sxmcKaNMcYYY4wxxhhjTBvihzbGGGOMMcYYY4wxbch0kUcxPZ5pSGqhPWzYsBTTHi2iXhpBe66IXB7F1FCVR/Gz9fsut9xyKd5pp51SPN9882XHMd2qlMo8s6Xz8/fo7yZdTe1jStqvf/3rFKtVNC2gf/e732VtTGdkClpJOjOzXafugqnClFPoGOX153mmjCMiH1dMO44o9686NC21Lh1xZk89JZS2RET885//TPFcc82VYh0fTJu/4YYbUkyby4g8pZSSxoh6y9KFFlooO2755ZdPMa1qI3KZzadtnHLeW3fddbM2jp3HH388xRdddFF2HK1OOQZKlt86Tj/3uc+lmNa2tEGOyGU3mob8aRpzdajUQtPCK1RGyn7P66Z9gmNdxzP7Aa3fS2gf4ZxfJ9lSZqTrzt9BC+iIiBVWWCHFCy+8cNam81lnP0ulWKXUeY7FSy+9NMWUSkVEPPHEEynm/joil4rMSNdnWuE5X3bZZbM2ymzY73Ue23fffVN82WWXpVjP/6abbpriL37xi1nbkCFDUvzQQw+lWG2pP617FkXv4bbbbrsUU4YWEXH77benmNeuJC+jpFHlxSwDofNy3TVp9riImU/CX7cW6FoyxxxzpFjl8ttuu22KKUvS+4x77rknxeecc06KVeLGMhl6bTjGdthhhxTvtttu2XFcu3V/tMwyy3T4HUvlYrqCM22MMcYYY4wxxhhj2hA/tDHGGGOMMcYYY4xpQ7pFHqUwJY0VnbWqMl+XUuCZXqTuUUxZ4nFaSZqvNS2VKVCUaelnvfrqqynWVKyuVIX+/+3dbawV1fXH8WX6ghjbqgVBLKAt+IQVW/1bIojGCK2PiWmtMVLaxlap+FSriQ/VVluoibFN07SFBo1iVKRNBBVFKRBbKhYQE6UoAvIgYAURo6j0nf8XxtXfWtwZhnPPhTnnfD+v1mU2c8+dPXtmzmSvvVpRM6by5RQ3XXn7mGOO8TinysyZM8fjJUuWhG1F1Ww0fc4sngeNpOJ0grKUMj3Py8asjp08nf/888/3+KmnngrbqqbC6LTLnH6g2mHqaVV6HDTN0CxOC9f+zCkTt99+u8f33ntv4e/SSg5PPPFEYbsxY8Z4rKmnZnGqrLYzM3v22Wc9rlpFrFXlKcRaLUHTM8xitQStfLJmzZrQrur9SMdbrrShU4r1c/Tt2ze027hxo8ft3lfNoP2t96CyadV6Hxs3blxoV1bRT6vz5SniVT5f/rksdaNVr7Vl1dqOOuooj3v37h226TXrd7/7ncc6HsxiP+rzZU5h1TQZTT81M7v//vs91rSb3Kc33XSTx1u2bAnbWrV/ukvvi5rmaRbHjqbLXHnllaHdzJkzu9x3ToHS62T+nqGp4Bq3YxW2RulYyde5CRMmeJyrFGo6U9Xvkpp+lb8v6riq2h970m/t1sdFFQbzsiff/e53PdZ0t9xWx0ROXbv00ks91ncIe1JFVvte3yHk7/xl9Nm5rOJbdzHTBgAAAAAAoIZ4aQMAAAAAAFBDvLQBAAAAAACoob2ypk1Rydey8mtlyso068+aB1eWv533oSVwdb2VXPbv4Ycf9nj79u2l+0Sk+b1XXXVV2HbOOed02U7XEDIzW7Rokcd5PRrNUdTc81xeddOmTYX7wCfyuKw6rjSXVcuhXnHFFaGdli/N5S6LyoZnmqee12Do1HU1tCTtNddcE7bpMdLjo2sgmJlNnjzZY+3r3Bfvv/++x/l80d+la3FcdNFFoV3//v09zqVYdb0HvQ63Wy642a551FpCXe9NZmZPPvmkx4sXL/a46nole7L2mo6xfv36eZzz/9evX+8x98Fd5WOu61lov+V2Oo507YyyUtP//e9/w8/Tpk3zuOpaD/l6WrSOWbuMRf078r1Dnx/yONXr7S233OJxXv9E14A69NBDPd66dWtop887l112Wdg2YsQIj7V/XnvttdBu3rx5HrdL/3SXPlPq8TeLa6PMmDHD48cffzy00/NCr4XXXnttaHfQQQd5nNfwXLlypce6HkanrIdZhd73f/rTn4ZtetzzONXnkTI6dorWJDHjPtYdeoy1z8zMxo4d67GuaZjptfEnP/lJ2KZrdek1Lq8hVUa/I44aNcrjsu8cef3Hp59+2uOeHMPMtAEAAAAAAKghXtoAAAAAAADU0F5Jj1LNmKJZtg+dJld16m4uEaxlHnWa3PLly0M7LU/MlMbd06lmX/3qVz2+7rrrQjud1rZz506P8/HX6aU5HUCnMeu05TzdLZdqx67K0qPKpg9qitrFF1/ssZZxN4vpAXl6d9G4Kvu9+Vxoxyn8XclpDOPHj/c4l7rX4/Dvf//b46lTp4Z2RWk2+ThWTT/VEtV5+qp+/pzGqOdSWenGVqV/U54mfOyxx3qcp9jPnj3b42and+Zjq2mMn/vc5wo/k6actkv/dJcehzxWqqay6Ri44YYbPM5jW73xxhvh51WrVlX6XaosBb0d+1f/Jk31M4vXrzxO9fp17rnneqzT7c3MPvzwQ4912n8uya2p+aNHjw7bNDVLx/3vf//70I6U73Ivv/xy+FmP65w5cwr/n17/brzxRo/zs43SEuJm8VzS+11ehkGfgdpxvJU577zzPB40aFDYlr+3Kb2flj0r6rVMv2vsSTpUfu76VKN91Q7PN/q59fgccsghoZ2miua/VftAnyny/U6vwwMHDvRYU0jNYrppTp/71re+5bE+5+TvH/o+IJcenzt3buH/ayZm2gAAAAAAANQQL20AAAAAAABqaK+nRzVD2dS3qlN39f8dfPDBYdvRRx/tsU7Luvfee0M7nd6I3dt///09/uUvf+lxPv46tWzt2rUe33XXXaGdTifO09F06qROEc+rwuefO0kzpmEWTYM0i5UTtAJOHrPaPzkFruo0w2ZUpWt1uZrJWWed5XFZBZj77rvP454YD9rfeg3I6VFl51K7j1O9XuXroVYH0pQJszhNX/dRNeUm077q1atX2Pbtb3/bY03V0fQ6s13TABBVvR7lMaDngV5Py/Y/a9assC1Xk6qiLD2q3enzn5nZo48+6rGOh+y9994r3IemXGkFzHxe6JT9fK3UPtBnJK1g0tU+Ee99OV1Q76FaqU+rtZmZnXjiiR6fffbZHufrrqY6bdy4MWw74ogjPD7llFM8Xrp0aWin50uj1/VWovcdHWP5+UbvVQceeGDYNnToUI9feOEFj/PzpI6PqpVK83VZ74W6/1zRqpGxmD9Hq4/nnPqn96PcN3qN0ypvkyZNCu30fPnyl7/scdn3yvw8qeeW/t5c0W/+/PkeT5kyJWzTtqRHAQAAAAAAdBhe2gAAAAAAANQQL20AAAAAAABqqGXWtNHcPs3vzbm+Vdc+0By2MWPGhG1ajnrdunUe/+tf/wrtOim3uxE591PzdjUnOOdp6lo1P//5zz3Oub45Z1RpruSbb77pcVneZFWtnGdalqvbXbm/NadU1zLZvn17aDdt2rTCbUXHNv97UW5yJ9H1TczimkKZXieff/75pn6OfI5pScaxY8d63KdPn9BOc5M/+uijpn6mutOxk9et0eOiud1msSTqtm3bPM6lnrX0r17z8pjV+6KuiWRmdsEFF3isfbxs2bLQrt3XH+pJRc85ZrHs82c/+9nCfWjp2ocffrjbn6nTrqf69+br0J133unxzJkzwzYtI6tr2miJb7PYP/o8kkuI9+vXz2N9djKLZW8nT57ssV4DzNqjfHCz6bowb7/9dtim/a3HPN9LtW/0O8Lrr78e2um2vJ7U17/+dY9PP/10jwcMGBDa/fnPf/Y4lypuR3oPyiWii+T118aPH+/xkCFDPNZxaWa2Y8cOj3Xs5O8WOmbzWinDhg3zWL+jvPrqq6FdT65zUjf6jKHPHnl86DqxP/zhD8O2oufB/v37h3Y6NnVtI/3OYVZcmt0s9rc+O/3xj38M7R5//HGP33rrrbBNrys9ea1lpg0AAAAAAEAN8dIGAAAAAACghmqbHpWn2OuUuQMOOMBjnXplVjwtKU+N0tJgl19+edim0620nGkuZcp003I5/eH666/3WKeubd68ObS7+uqrPf7b3/7mcVm5w7LS71XLwJftU8+//Lv0HGzVc6LRlC/9f3mM6RR+TVHLUyS1TGlOVytK5ypLj+pUWubSLI6xfHx0um9Zieai41+WZpfLb15zzTUea3pUTufS6cMrVqwI27RUZDv2dVlajE7XHThwYNh26qmnejxo0CCPdbyZmW3YsMHjlStXepyn7H/xi1/0eOLEiWGbTgvX6cqaAmDWWdPAe5KWkjWLfa3nS74vahnpXGZYVU2VbcfxVlW+H7377rse53T5qqm8SvsgpxVqClS+RmtK64MPPuhxM8oMtzs9JtqfZjGVTcdfTpObPXu2x1pSOrfTZ6Kc4nbyySd7rNfdkSNHhnbTp0/3WNN5zNqzfz/zmc94rP2Rv+vpz/k+dsIJJ3isy13kFET9Wcd6Pi+0NHy+B2tp+L///e8eX3HFFaFdTs0q0m59qsc1p/f94Q9/8Pipp54K27RvdFzlMabfM3Wpkx/84AehnaaW52cs3edvf/tbj2fMmBHa6fjbV/3ETBsAAAAAAIAa4qUNAAAAAABADfHSBgAAAAAAoIZqtaaN5vfmfG79WXMZdW0Gs+I8s1xG9Uc/+pHHmk9qFkt5/eMf/yj8XSgv7ztp0qSwbfjw4V3+v1mzZoV2CxYs8LjqOjZla7JUzSkvW5PlS1/6ksc5f1bXDcj5lnXLT626Rkwjcp6olmvUfN5XXnkltNMy3/lzaH6zfva8bkanrqOhx+TII48s3JZpbm4+n/f095rFNZ907Q0zsyuvvNJjXS8s70PXdNB1jszitbduY6oZNO87l73Xdb3yuhd6X9Q1jHKJWi1tq6U0dd0Ms7ieQi4vrtdH7YP//Oc/oV079s/eomMil/XWNYv0eqfrPZmZ3X333YXbyu53qlOvp7tT9bmiKu2Pr33ta2HbJZdc4nFez0PXgtD1bvIaPCiXn9deeuklj7du3dplbGa2ZcsWj3UdofwMpGPsueeeC9t03RW97ub7sX53aXTtwVai97i5c+d63Lt379BuyZIlHuf+0e90us6Jlmo3i88tOnb0udMs3mf79u1buI/zzjvPYy3Vbha/S7ZjvxXRvzXfV3T85XUMq9I1MvXZ6cILLyz8P3mM6Xn2l7/8xeM6riHFTBsAAAAAAIAa4qUNAAAAAABADe3T9Kg81U+nFuqUM7M4ZU6nI1ZNfTn99NPDtlGjRnm8evXqsG3KlCkeazm/slSdTqVT7c3MrrrqKo8vuuiisE1LtWu6g05zNIvTFHV6aT4nNB0gTx/W86Vof2ax7LBOVzUzGz16tMeaeqJl/czi1Lq1a9daq2jGVD89ntq/ZrHE5bHHHutxPkb6OXL/6M86ZTV/9kbLurc6/VvzFOEyeq0dMGCAx5oaalY8dnJfn3jiiR7/7Gc/C9u0JKZek3M/rV+/3uN//vOfhZ+jHem04VyuW1Ncli5dGrbpdU5TU/XaaBavtzrlN6ch6zWwLEVG+yNfe9E4HR86bszitHxtt2nTptDuxRdf9LisD8vSJ8vGaScrS8lWeq3M1y49npoCN3HixNBuyJAhHuszhpnZypUrC/ePcnr8c4l0TfXUe2HZONLn0nw91etzvq5PnTrVYy0xnVNgdf85bacd+17/fi1nP2/evNBOj1lOd9Hneo3POuus0E5Tz/T7XU6L0eed3Md6HdD9nXbaaaGdpsdVXfahk669jf6tOiauv/56jwcPHly4/4ULF4Zt1113ncdalryOx5+ZNgAAAAAAADXESxsAAAAAAIAaqlX1KJ2KlKcI6nSyqlOW+vTp4/Gtt94atuk0trvuuitse+aZZzxu96ol3ZXTJM455xyPc/ULnTKs6Rl5GptWLdH9a7qSWZw2mqcP5ynjn+rfv3/4+bbbbvNYU+bMYmUVnYqZp9Rq9YY8BVanbdbh/Cn6DI1+Np3KOXDgwLDt3HPP9VhX8899UDYdtJEqYHU4zvuCVr4wi+lSeSxq6oWmM91xxx2h3bZt2zzW6ibf+973Qrthw4Z5nKsOKb2Or1q1KmzTqa1abcys/ftU/76cbqR9UEYrROVp9EXjKFc7WbRokcc6TdgsTgvXqg95ajpjsXF67A4//PCw7eCDD/ZYj+vy5ctDOx07OX1C91+WWkG/dU2PX07X1p81naasOp8+o37lK18p3J9WnjHb9RkEjclpT0VjIqfCafppWWXRN954w+OcEqPbZs+e3eW+s3zO6edtlxRx/exaDUifwXO7/PfqM/nNN9/s8YYNG0I7rSx14IEHepyfl/Tnqmml+Rmm7P+pVu67vSE/29x4440ef//73/c4P9ts3rzZ4wkTJoRt+uxU9+PPTBsAAAAAAIAa4qUNAAAAAABADfHSBgAAAAAAoIb26Zo2e1K2t2qemZagnj59usfHHXdcaKelU5999tmwjXVsqtNjZWa2ZcsWj3O+cNGaNtdee21oN27cOI/79evncS61pzmuOf9fy3LrWhxnnHFGaHf88cd7nHOJdd0Gza3NNI+5rLxnO9I+GTt2bNh2zDHHeKx5qDkfv6g8e1c/f6rq+ja7a9tOcrluzZMfP3582KbnupbBPPPMM0M7HcN6bS3Lz87jXtdGWbNmjcc//vGPQ7sVK1YU7qPdVV27qeo+ql6H8nFeu3atx1r+1iyuL6ZrBuTrsl7nO60fu0uP5Xe+852wTdfh0/7VcW5Wvt5JM84zfELPc7N4ruu6VPk467Vz6NChHmv/5v+n182e1qn3T7P4t5atX3TkkUd6fMopp3is62GaxXMiH0cdp3o9zWt26O/W8tVm8Tqg+yv7btVKGv3cetx1LZOHHnootDvppJM8HjJkiMe6nqXZrv2v9Fjrd4Zly5aFdq3aB3WgY1HXTTWLazJqP+X7oK7XmNc2aqVrHDNtAAAAAAAAaoiXNgAAAAAAADVU25LfmU6P0vjzn/98aPfAAw94fNppp3mcp2nPmTPHY52auLvPgSinR/3617/2WMs8m8WpwDr98JBDDgntNCVKlU35POGEE8K2Dz/80OPhw4d7nNOoNDUnp0AtWbLEY532qql1ZjFNK5frrbNGz3OdFq7lu7XEt1lMp9mxY4fH8+fPD+3K0suUlsysWj6xk+Rz75577vE4983RRx/tcVHaYpl87mjfaIqkmdnEiRM91r7fuHFjaEcZ2+ZpdCr2Bx984HGeQqzXbC1X27t379Bu06ZNHpelOXOf/YSOv6OOOsrjnKqoaRNaBn7x4sWhXdXjWpQKgmJ6zMpKeZf1gfb3qFGjPM5pMfrMmksQV1W1PHHVZ+9OGrPaT1oO2sxs0KBBHq9cudLjrVu3hnZVj5ce4/zcrNdkTb8xi8+ver7k7zud2odm8V6o3wvMzPr06eOxpqbmsVhUWt0s3u8eeeQRj5cvX164D+yenrO6xMW0adNCO/2eocc4pyrq/2vlMcBMGwAAAAAAgBripQ0AAAAAAEAN8dIGAAAAAACghlpmTZui/NJf/OIXoZ2Wr9XyX7ruiFlc+4aypI3Lffbiiy96/I1vfCNs03U0JkyY4PGpp54a2h122GEea15jzkfV9TFy2dM333zT45dfftnjvKaNlrXVz24W19yoWga+7rmSzfh82ieaa5/XBCpax2bhwoWhna6HUvb5NF81l1ut+nnr3j/NtHr1ao8vvvjisE3Xu9F84ZzLXXTs8noOOv4mTZoUtmnOf1kpXHStbF2K7q4Rk/+PXudeeumlsO2ggw7yeP369R7nUsX6s45tM+61Zrv2p66lcMYZZ3i8//77h3baV7qm0Hvvvdftz8RYrKYZx0nXhhozZozHeT0x/V3Dhg0L22bNmuWxrmuyJ4quHa1S8run12HSe2FeL27FihUev/POOx7nNR7Ljp2Oe7225jXCdI3HfP3U9Tjz8zE+oX2QvwdqOfCBAwcW7mPdunUez5w5M2z761//2uX+8ris6ziqizyedUzcfffdHuf1vfS4vvXWWx5fdtlloV0jaybmz1T2XWJv9S8zbQAAAAAAAGqIlzYAAAAAAAA1VKv0qKq+8IUveDxy5MiwTdMmdHqapkOZ7ZrKgebQFJY8bVtLaGucU110+rBOUc0l83Tqfd5WtZypTjfdV9Pd6q5surSmof3pT38K7TTNbcaMGR7rlN68vzKUC94zOiY0RdDMbMSIER5rP+UywyeffLLHWsIyp7itWbPG4zyVnFKX3dOTpZnL9penkmuZaR3DWnLTrPyail2Pud7vNNZSv2ZxHC1YsMDjPN6qouT3vnHooYd6rM+ymfZJTuvWFMSylNNGUrlbZczmz6nPkVVTSsvodSyX2tY0KE27KNt3Tj3WPtRxr2lT+Xfl9ChNqdO/P99zO/keXHYuPPjggx5Pnz7d4w0bNoR2muKtaf9m1dP7sSvtmwMOOCBs++Y3v+nx4MGDPc7nst4nf/WrX3n89ttvd/sz5e+muq1sTPXkecBMGwAAAAAAgBripQ0AAAAAAEANtUx6lE4tHDJkiMd5JWmduqZT3B566KHQjioW9ZGnmeWpqNi3yqZca5qhrqJvFvu12VNIO3m6bzNof2glmvvuuy+0yz9j32r2tNu8P60KtnTp0rBNr8s6JVmnjud9lKWtdqp8TPSZRafra8USs5huvHjxYo9zJbdG0C89J0+x16oo2qf5uUdTEDUdziyeM3o+dXI/Fj0TNJr6p8ey0VSIst+t92Dt+zzuNQVKK1Xl/6efke83/6P9k1ObHn300S7bcd/qGWWVE4cOHRq2XXjhhR5r6lROB9aqv08//bTHPf0dgepRAAAAAAAAcLy0AQAAAAAAqCFe2gAAAAAAANRQy6xpU5SfpmW9zWK+2+rVqz3etGlTaEeOItB9mjtNHjXQ2rR8bS4Tv3HjRo91nY6tW7cW7oP77O7pdVOfU/IaYVXX2EC95L7SNcQee+wxj4cPHx7arVq1yuNHHnkkbGu0zHsnavQa1OhaOEX7KFs/7N133/VY17rJ+3j//ffDNl1bLP8/7IpS6PWiZe+HDRsWtvXp08djXdM2r+G2YsUKj/VZpNFx34y1rHoSM20AAAAAAABqiJc2AAAAAAAANdQy6VFa9q5v374e55QMnT6o5dzytEIAAPA/Oh04T0PWqcfaLk/LJyWqcXrsSDdtT1rme+rUqR7PmDEjtFu7dq3HO3fu7PkPhkBTQHOqVNUy62Xb9Lq5ffv2Lv/dLPZ9LllNeg9aSR5Hmkr9/PPPh23HH3+8xzqOctr2b37zG48/+uijpnzOrn5vXTDTBgAAAAAAoIZ4aQMAAAAAAFBDvLQBAAAAAACoof32JGdrv/3222cJXpoLd9hhh3k8YMCA0G7dunUeb9u2zeNWz/38+OOPu19/0PZtH8KWffzxx//XjB3Rj/sOY7EtMBbbAGOxLTAW20C7jcWyct3N3r+WNO7Vq1dop+t07IU1NhiLbYCx2Ba6HIvMtAEAAAAAAKghXtoAAAAAAADU0J6W/N5mZht64oPsjk6J2rx5c5dxGzu8ifvaZ30I+rEN0IftgX5sffRhe6AfW1/b9WFPp2Ho/rXMdy75vZe1XT92oLbrww5KiVJd9uMerWkDAAAAAACAvYP0KAAAAAAAgBripQ0AAAAAAEAN8dIGAAAAAACghnhpAwAAAAAAUEO8tAEAAAAAAKghXtoAAAAAAADUED8lmYMAAAAjSURBVC9tAAAAAAAAaoiXNgAAAAAAADXESxsAAAAAAIAa+n9BEls6pF1stgAAAABJRU5ErkJggg==\n","text/plain":["
"]},"metadata":{"needs_background":"light"}}]},{"cell_type":"markdown","metadata":{"id":"t4uWOdSIm-EO"},"source":["Απ' ότι φαίνεται, ο αυτοκωδικοποητής απαλοιφής θορύβου κατορθώνει να γενικεύσει, μιας και μπορεί να αναπαριστά τα ψηφία χωρίς θόρυβο (ωστόσο όχι τόσο καλά όσο οι προηγούμενοι δύο).\n","\n","Ας δούμε την αντίστοιχη κωδικοποίηση των $10$ πρώτων ψηφίων ($32$ χαρακτηριστικά)"]},{"cell_type":"code","metadata":{"id":"tynTUs5amM3w","colab":{"base_uri":"https://localhost:8080/","height":427},"executionInfo":{"status":"ok","timestamp":1642514364060,"user_tz":-120,"elapsed":254,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"4b9b2fb2-401e-4449-da32-fbeca1bdd619"},"source":["pd.DataFrame(data=h_noisy[0:10], columns=range(1,dim_h+1), index=range(1,11)).head(10)"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["\n","
\n","
\n","
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
1234567891011121314151617181920212223242526272829303132
17.1241631.5579474.7170678.7389679.1843032.5125865.4914402.3509858.3797065.6690831.7899094.0419357.1773579.3260236.4912643.3016324.4053045.7884553.0279974.7848854.9197255.8701451.1081904.75635411.4732246.0163050.05.2968446.1739084.3551964.22422013.366877
27.2103823.2039524.8715704.6828889.8555830.0000006.1220056.4691802.8190231.4275137.6767654.3943876.8841214.0543199.90609010.2967704.0693396.5716325.1477279.1164964.8778239.8939251.50715517.6826824.4817223.2966340.010.2737955.9515373.6438453.2447845.132721
30.0000003.1339462.3816967.5789154.1774365.2547757.2721662.8490355.7298243.3228355.0684533.3480413.0356313.6002116.5604576.7352231.0902743.1545372.5297443.6999424.0995366.3054766.6940139.9768136.8270749.1901040.05.2366704.9915758.7839223.2282704.599844
414.4385224.5192537.1811538.0414273.4620298.86139413.10833517.11247312.3206237.6803619.9232511.5110409.31364412.56019710.36245510.86180912.4441976.3907334.2545058.7252827.01054713.9594575.56001810.3215548.5863993.1122470.012.6640753.0265609.8277064.8423709.247440
59.0471880.5974384.48244811.3297726.0681418.3594895.21534210.5735176.3782046.6362866.99456015.2816457.1366279.9794234.14968910.3205675.4958173.5764925.4791415.7393263.9127258.8530475.1123314.0956087.0887240.9865760.02.4815514.3893885.5035164.3267487.454122
61.6887534.6310272.25052610.1994574.2991365.9849688.5525904.1971839.9064463.8554393.4907662.6044444.4999903.2034057.4868928.6581401.7191342.5157191.9672773.8123194.9887606.8630318.70849411.4014637.85372611.0903870.08.58407710.13335210.8133015.8704975.402645
75.9652506.8209814.9346148.5228797.1189641.6346618.3503619.7284142.0699727.8931344.61613010.34479010.4874067.2315904.8425893.6651990.00000010.1606523.1667884.0484747.8350326.0175356.7148463.5314784.4334105.0217400.02.72878410.5937350.0000005.53747111.597893
82.7406040.7606765.3412149.1278725.5930858.8319700.0000008.2045916.9688214.7721855.2294454.6154714.6813517.7011850.9519797.9248593.1279867.3513232.0086466.7233735.4778965.1445474.0555556.4469913.6292454.7044150.06.0973297.8859437.0557062.9784524.841427
910.2712830.1868050.5137779.8128094.5104163.98046312.88964810.5378480.0000003.90230810.40768813.8491671.2319138.5421688.3905109.5885557.2226257.7970186.5852931.0946353.75638713.4902067.1136976.2917679.45250111.9778640.09.6528565.0500576.8099867.8574358.126180
103.8665386.3561637.8729479.6369095.9738602.8998379.5245416.52334313.9164678.9067925.5224565.0433686.78920712.3417847.6664227.8820333.8204823.9962203.07462410.0170663.0314115.3763376.6290180.5113335.95063310.2284960.010.21903410.39767811.2181623.14616711.696098
\n","
\n"," \n"," \n"," \n","\n"," \n","
\n","
\n"," "],"text/plain":[" 1 2 3 4 5 6 7 \\\n","1 7.124163 1.557947 4.717067 8.738967 9.184303 2.512586 5.491440 \n","2 7.210382 3.203952 4.871570 4.682888 9.855583 0.000000 6.122005 \n","3 0.000000 3.133946 2.381696 7.578915 4.177436 5.254775 7.272166 \n","4 14.438522 4.519253 7.181153 8.041427 3.462029 8.861394 13.108335 \n","5 9.047188 0.597438 4.482448 11.329772 6.068141 8.359489 5.215342 \n","6 1.688753 4.631027 2.250526 10.199457 4.299136 5.984968 8.552590 \n","7 5.965250 6.820981 4.934614 8.522879 7.118964 1.634661 8.350361 \n","8 2.740604 0.760676 5.341214 9.127872 5.593085 8.831970 0.000000 \n","9 10.271283 0.186805 0.513777 9.812809 4.510416 3.980463 12.889648 \n","10 3.866538 6.356163 7.872947 9.636909 5.973860 2.899837 9.524541 \n","\n"," 8 9 10 11 12 13 \\\n","1 2.350985 8.379706 5.669083 1.789909 4.041935 7.177357 \n","2 6.469180 2.819023 1.427513 7.676765 4.394387 6.884121 \n","3 2.849035 5.729824 3.322835 5.068453 3.348041 3.035631 \n","4 17.112473 12.320623 7.680361 9.923251 1.511040 9.313644 \n","5 10.573517 6.378204 6.636286 6.994560 15.281645 7.136627 \n","6 4.197183 9.906446 3.855439 3.490766 2.604444 4.499990 \n","7 9.728414 2.069972 7.893134 4.616130 10.344790 10.487406 \n","8 8.204591 6.968821 4.772185 5.229445 4.615471 4.681351 \n","9 10.537848 0.000000 3.902308 10.407688 13.849167 1.231913 \n","10 6.523343 13.916467 8.906792 5.522456 5.043368 6.789207 \n","\n"," 14 15 16 17 18 19 \\\n","1 9.326023 6.491264 3.301632 4.405304 5.788455 3.027997 \n","2 4.054319 9.906090 10.296770 4.069339 6.571632 5.147727 \n","3 3.600211 6.560457 6.735223 1.090274 3.154537 2.529744 \n","4 12.560197 10.362455 10.861809 12.444197 6.390733 4.254505 \n","5 9.979423 4.149689 10.320567 5.495817 3.576492 5.479141 \n","6 3.203405 7.486892 8.658140 1.719134 2.515719 1.967277 \n","7 7.231590 4.842589 3.665199 0.000000 10.160652 3.166788 \n","8 7.701185 0.951979 7.924859 3.127986 7.351323 2.008646 \n","9 8.542168 8.390510 9.588555 7.222625 7.797018 6.585293 \n","10 12.341784 7.666422 7.882033 3.820482 3.996220 3.074624 \n","\n"," 20 21 22 23 24 25 26 \\\n","1 4.784885 4.919725 5.870145 1.108190 4.756354 11.473224 6.016305 \n","2 9.116496 4.877823 9.893925 1.507155 17.682682 4.481722 3.296634 \n","3 3.699942 4.099536 6.305476 6.694013 9.976813 6.827074 9.190104 \n","4 8.725282 7.010547 13.959457 5.560018 10.321554 8.586399 3.112247 \n","5 5.739326 3.912725 8.853047 5.112331 4.095608 7.088724 0.986576 \n","6 3.812319 4.988760 6.863031 8.708494 11.401463 7.853726 11.090387 \n","7 4.048474 7.835032 6.017535 6.714846 3.531478 4.433410 5.021740 \n","8 6.723373 5.477896 5.144547 4.055555 6.446991 3.629245 4.704415 \n","9 1.094635 3.756387 13.490206 7.113697 6.291767 9.452501 11.977864 \n","10 10.017066 3.031411 5.376337 6.629018 0.511333 5.950633 10.228496 \n","\n"," 27 28 29 30 31 32 \n","1 0.0 5.296844 6.173908 4.355196 4.224220 13.366877 \n","2 0.0 10.273795 5.951537 3.643845 3.244784 5.132721 \n","3 0.0 5.236670 4.991575 8.783922 3.228270 4.599844 \n","4 0.0 12.664075 3.026560 9.827706 4.842370 9.247440 \n","5 0.0 2.481551 4.389388 5.503516 4.326748 7.454122 \n","6 0.0 8.584077 10.133352 10.813301 5.870497 5.402645 \n","7 0.0 2.728784 10.593735 0.000000 5.537471 11.597893 \n","8 0.0 6.097329 7.885943 7.055706 2.978452 4.841427 \n","9 0.0 9.652856 5.050057 6.809986 7.857435 8.126180 \n","10 0.0 10.219034 10.397678 11.218162 3.146167 11.696098 "]},"metadata":{},"execution_count":27}]},{"cell_type":"markdown","metadata":{"id":"Aub0yBDMnmV_"},"source":["Εδώ δεν υπάρχει ποινή αραιότητας, οπότε τα χαρακτηριστικά λαμβάνουν τιμές αντίστοιχες με τον υποπλήρη αυτοκωδικοποιητή. \n","\n","Ας υπολογίσουμε την ομοιότητα συνημιτόνου για τις αναπαραστάσεις των 10 πρώτων ψηφίων του συνόλου ελέγχου."]},{"cell_type":"code","metadata":{"id":"qVVEapqWmP5B","colab":{"base_uri":"https://localhost:8080/","height":363},"executionInfo":{"status":"ok","timestamp":1642514379100,"user_tz":-120,"elapsed":290,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"79dcf6ec-7e8f-4922-e886-08528d60d73b"},"source":["pd.DataFrame(data=cosine_similarity(h_noisy[0:10]), columns=range(1,11), index=range(1,11)).head(10)"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["\n","
\n","
\n","
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
12345678910
11.0000000.7896080.8234430.8387380.8360460.8211430.8538100.8304660.7978570.891888
20.7896081.0000000.8348130.8501150.7706920.8093310.7611680.8001740.8020630.743035
30.8234430.8348131.0000000.8261690.7864360.9796940.7625330.8568910.8465760.864586
40.8387380.8501150.8261691.0000000.8688940.8217000.7761600.8553050.8475900.858865
50.8360460.7706920.7864360.8688941.0000000.7616080.8496380.8845910.8730640.822533
60.8211430.8093310.9796940.8217000.7616081.0000000.7558740.8550990.8129850.878848
70.8538100.7611680.7625330.7761600.8496380.7558741.0000000.8132320.8125180.825472
80.8304660.8001740.8568910.8553050.8845910.8550990.8132321.0000000.7824340.867987
90.7978570.8020630.8465760.8475900.8730640.8129850.8125180.7824341.0000000.783937
100.8918880.7430350.8645860.8588650.8225330.8788480.8254720.8679870.7839371.000000
\n","
\n"," \n"," \n"," \n","\n"," \n","
\n","
\n"," "],"text/plain":[" 1 2 3 4 5 6 7 \\\n","1 1.000000 0.789608 0.823443 0.838738 0.836046 0.821143 0.853810 \n","2 0.789608 1.000000 0.834813 0.850115 0.770692 0.809331 0.761168 \n","3 0.823443 0.834813 1.000000 0.826169 0.786436 0.979694 0.762533 \n","4 0.838738 0.850115 0.826169 1.000000 0.868894 0.821700 0.776160 \n","5 0.836046 0.770692 0.786436 0.868894 1.000000 0.761608 0.849638 \n","6 0.821143 0.809331 0.979694 0.821700 0.761608 1.000000 0.755874 \n","7 0.853810 0.761168 0.762533 0.776160 0.849638 0.755874 1.000000 \n","8 0.830466 0.800174 0.856891 0.855305 0.884591 0.855099 0.813232 \n","9 0.797857 0.802063 0.846576 0.847590 0.873064 0.812985 0.812518 \n","10 0.891888 0.743035 0.864586 0.858865 0.822533 0.878848 0.825472 \n","\n"," 8 9 10 \n","1 0.830466 0.797857 0.891888 \n","2 0.800174 0.802063 0.743035 \n","3 0.856891 0.846576 0.864586 \n","4 0.855305 0.847590 0.858865 \n","5 0.884591 0.873064 0.822533 \n","6 0.855099 0.812985 0.878848 \n","7 0.813232 0.812518 0.825472 \n","8 1.000000 0.782434 0.867987 \n","9 0.782434 1.000000 0.783937 \n","10 0.867987 0.783937 1.000000 "]},"metadata":{},"execution_count":28}]},{"cell_type":"markdown","metadata":{"id":"6cU0IZ08oQjA"},"source":["Παρότι δεν υπάρχει ποινή αραιότητας, οι ομοιότητες συνημιτόνου είναι ακόμα μεγαλύτερες σε σύγκριση με τις δύο προηγούμενες περιπτώσεις. Επίσης, πάλι το 3ο και το 6ο ψηφίο είναι πιο όμοια μεταξύ τους, αλλά δεν ισχύει το ίδιο για τα υπόλοιπα ζεύγη ψηφίων (5ο, 7ο και 8ο, 10ο ).\n"]},{"cell_type":"markdown","metadata":{"id":"AxG4EkgPnkZS"},"source":["## Βαθύς Αυτοκωδικοποιητής"]},{"cell_type":"markdown","metadata":{"id":"xwxgXuyyyN-P"},"source":["\n","\n","Στους **βαθείς αυτοκωδικοποιητές** (*deep autoencoders*) οι συναρτήσεις (από)κωδικοποίησης δεν αποτελούνται από ένα μόνο επίπεδο αλλά από περισσότερα. Στο παράδειγμα που εξετάζουμε, συνάρτηση κωδικοποίησης αποτελείται από $3$ επίπεδα με διαστάσεις $128, 64$ και $32$ αντίστοιχα και με συνάρτηση ενεργοποίησης την ReLU. Ίδιο αριθμό επίπεδων (με ανάστροφη σειρά) έχει η συνάρτηση αποκωδικοποίησης, με το τελευταίο να έχει συνάρτηση ενεργοποίησης τη σιγμοειδή (για να \"σπάσει\" τη συμμετρία). Στη γενικότερη περίπτωση ωστόσο, δεν είναι υποχρεωτικό οι συναρτήσεις (απο)κωδικοποίησης να έχουν ίδιο αριθμό επιπέδων\n"]},{"cell_type":"code","metadata":{"id":"Rm88em1hn02l"},"source":["layer1_dim = 128\n","layer2_dim = 64\n","\n","deep_autoencoder = Sequential([\n"," Dense(layer1_dim, activation='relu', input_shape=(dim_x,)),\n"," Dense(layer2_dim, activation='relu'),\n"," Dense(dim_h, activation='relu'),\n"," \n"," Dense(layer2_dim, activation='relu'),\n"," Dense(layer1_dim, activation='relu'),\n"," Dense(dim_x, activation='sigmoid')\n","])\n","\n","\n","deep_autoencoder.compile(optimizer='adam', loss='binary_crossentropy')"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"_c0y9NNpvdyr"},"source":["Εκπαιδεύουμε για 30 εποχές"]},{"cell_type":"code","metadata":{"id":"fs9esJHupGJC","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1642514662636,"user_tz":-120,"elapsed":143024,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"0bc35626-239e-4187-99b3-8fa5b473fc78"},"source":["deep_autoencoder.fit(x_train, x_train,\n"," epochs=30,\n"," batch_size=64,\n"," shuffle=True,\n"," validation_data=(x_test, x_test))"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Epoch 1/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.1693 - val_loss: 0.1222\n","Epoch 2/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.1156 - val_loss: 0.1077\n","Epoch 3/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.1054 - val_loss: 0.1009\n","Epoch 4/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.1002 - val_loss: 0.0972\n","Epoch 5/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.0967 - val_loss: 0.0942\n","Epoch 6/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.0939 - val_loss: 0.0920\n","Epoch 7/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.0919 - val_loss: 0.0904\n","Epoch 8/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.0901 - val_loss: 0.0886\n","Epoch 9/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.0885 - val_loss: 0.0874\n","Epoch 10/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.0873 - val_loss: 0.0860\n","Epoch 11/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.0865 - val_loss: 0.0855\n","Epoch 12/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.0857 - val_loss: 0.0846\n","Epoch 13/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.0851 - val_loss: 0.0844\n","Epoch 14/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.0845 - val_loss: 0.0837\n","Epoch 15/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.0840 - val_loss: 0.0834\n","Epoch 16/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.0836 - val_loss: 0.0830\n","Epoch 17/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.0833 - val_loss: 0.0829\n","Epoch 18/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.0830 - val_loss: 0.0832\n","Epoch 19/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.0827 - val_loss: 0.0823\n","Epoch 20/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.0824 - val_loss: 0.0816\n","Epoch 21/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.0821 - val_loss: 0.0816\n","Epoch 22/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.0819 - val_loss: 0.0817\n","Epoch 23/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.0816 - val_loss: 0.0821\n","Epoch 24/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.0814 - val_loss: 0.0812\n","Epoch 25/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.0812 - val_loss: 0.0807\n","Epoch 26/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.0810 - val_loss: 0.0808\n","Epoch 27/30\n","938/938 [==============================] - 4s 5ms/step - loss: 0.0808 - val_loss: 0.0809\n","Epoch 28/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.0806 - val_loss: 0.0806\n","Epoch 29/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.0804 - val_loss: 0.0798\n","Epoch 30/30\n","938/938 [==============================] - 5s 5ms/step - loss: 0.0803 - val_loss: 0.0800\n"]},{"output_type":"execute_result","data":{"text/plain":[""]},"metadata":{},"execution_count":31}]},{"cell_type":"markdown","metadata":{"id":"4tPUROCBxG4M"},"source":["Παρατηρούμε ότι ούτε σε αυτή την περίπτωση παρουσιάστηκαν φαινόμενα υπερπροσαρμογής στην εκπαίδευση. Ωστόσο το σφάλμα εκπαίδευσης και επαλήθευσης είναι χαμηλότερο απ' ότι σε όλες τις άλλες περιπτώσεις\n","\n","Ορίζουμε τα δίκτυα του κωδικοποιητή και του αποκωδικοποιητή."]},{"cell_type":"code","metadata":{"id":"cR_WzjM6xFfz"},"source":["deep_encoder = Sequential([\n"," deep_autoencoder.layers[0],\n"," deep_autoencoder.layers[1],\n"," deep_autoencoder.layers[2]\n","])\n","\n","deep_decoder = Sequential([\n"," deep_autoencoder.layers[-3],\n"," deep_autoencoder.layers[-2],\n"," deep_autoencoder.layers[-1]\n","])\n","\n","h_deep = deep_encoder.predict(x_test)\n","x_test_out_deep = deep_decoder.predict(h_deep)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"BIsqM8x2yNja"},"source":["Εμφανίζουμε στην πρώτη σειρά τα δέκα πρώτα ψηφία του συνόλου ελέγχου και στη δεύτερη σειρά, τα ίδια ψηφία όπως προκύπτουν από την έξοδο του βαθύ αυτοκωδικοποιητή."]},{"cell_type":"code","metadata":{"id":"bFqWHHohpSkm","colab":{"base_uri":"https://localhost:8080/","height":248},"executionInfo":{"status":"ok","timestamp":1642514689241,"user_tz":-120,"elapsed":1057,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"c9e97661-a4b6-40b1-e153-cb2231f92838"},"source":["compare_digits(x_test, x_test_out_deep)"],"execution_count":null,"outputs":[{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAABG0AAADnCAYAAACkCqtqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3debzV4/bA8XWom5LmWSqioiJNklBkSqUQKcMtGa7Mlbj5kelyy5SpZC4kJDQiEgnJbdBAiialWTRIdH5/eFnWejp7dzrtvc/37P15/7W+Pc/Z+7G/+zvsr2c9Kys7O1sAAAAAAAAQLfvk9wAAAAAAAACwKx7aAAAAAAAARBAPbQAAAAAAACKIhzYAAAAAAAARxEMbAAAAAACACOKhDQAAAAAAQAQV2pPOWVlZ1AfPJ9nZ2VmJeB32Yb5al52dXT4RL8R+zD8ci2mBYzENcCymBY7FNMCxmBY4FtMAx2JayPFYZKYNkDpL83sAAESEYxGICo5FIBo4FoFoyPFY5KENAAAAAABABPHQBgAAAAAAIIJ4aAMAAAAAABBBPLQBAAAAAACIIB7aAAAAAAAARBAPbQAAAAAAACKIhzYAAAAAAAARxEMbAAAAAACACCqU3wNAZurdu7fGRYsWdW1HHnmkxueee27M1xg8eLDGn376qWsbPnz43g4RAAAAAIB8xUwbAAAAAACACOKhDQAAAAAAQATx0AYAAAAAACCCWNMGKTNy5EiN461VY+3cuTNm2xVXXKFx69atXduUKVM0XrZsWW6HiHxWq1Ytt/31119rfN1112n86KOPpmxMmWz//ffXeODAgRrbY09E5Msvv9S4U6dOrm3p0qVJGh0AAED+KF26tMbVqlXL1d+E90Q33HCDxnPnztV44cKFrt/s2bPzMkSkEWbaAAAAAAAARBAPbQAAAAAAACKI9CgkjU2HEsl9SpRNiXnnnXc0PuSQQ1y/du3aaVyzZk3X1rVrV43vvffeXL0v8t/RRx/ttm163IoVK1I9nIxXuXJljS+77DKNw7TFRo0aady2bVvX9vjjjydpdLAaNmyo8RtvvOHaatSokbT3PfXUU932ggULNF6+fHnS3he7Z6+RIiJvv/22xldffbXGQ4YMcf3++OOP5A4sDVWoUEHjV199VeNp06a5fkOHDtV4yZIlSR/XX0qWLOm2TzjhBI0nTpyo8Y4dO1I2JqAgOPPMMzVu3769a2vZsqXGhx56aK5eL0x7ql69usZFihSJ+Xf77rtvrl4f6YuZNgAAAAAAABHEQxsAAAAAAIAIIj0KCdW4cWONO3bsGLPfvHnzNA6nG65bt07jzZs3a/yPf/zD9fvss880Puqoo1xb2bJlczliREmDBg3c9pYtWzQePXp0qoeTccqXL++2X3jhhXwaCfbUaaedpnG8KdaJFqbgdO/eXePOnTunbBz4k732PfHEEzH7PfbYYxo/++yzrm3btm2JH1iasVVjRPw9jU1FWr16teuXXylRtsKfiD/X2/TWRYsWJX9gBUyJEiXctk25r1evnsZhFVNSzaLNLqvQs2dPjW0quIhI0aJFNc7Kytrr9w2rpAK5xUwbAAAAAACACOKhDQAAAAAAQATx0AYAAAAAACCC8nVNm7AEtM0jXLlypWv79ddfNX7ppZc0/vHHH10/8nHzly0RHOZ+2pxvu/7CqlWrcvXavXr1cttHHHFEzL7jxo3L1Wsi/9mccFuGVkRk+PDhqR5Oxrn22ms17tChg2tr2rTpHr+eLSUrIrLPPn//v4HZs2dr/NFHH+3xa8MrVOjvS3ibNm3yZQzhWhk33nijxvvvv79rs2tUITns8Ve1atWY/UaMGKGxvb9CbOXKldN45MiRrq1MmTIa27WErrnmmuQPLIZbb71V44MPPti1XXHFFRpz37yrrl27anzPPfe4toMOOijHvwnXvlm/fn3iB4aEsefH6667Lqnv9fXXX2tsfwshcWzJdXuuFvFrrNoy7SIiO3fu1HjIkCEaf/LJJ65fFM6TzLQBAAAAAACIIB7aAAAAAAAARFC+pkcNGDDAbdeoUSNXf2endf7yyy+uLZXTzlasWKFx+N8yY8aMlI0jSsaMGaOxnaom4vfVhg0b9vi1w/KxhQsX3uPXQPTUqVNH4zCdIpyCjsR76KGHNLbTRPPq7LPPjrm9dOlSjc8//3zXL0yzwe61atVK42OPPVbj8HqUTGHpY5u2WqxYMddGelTiheXd+/Xrl6u/s6mn2dnZCR1TumrYsKHG4RR7684770zBaHZVt25dt21TykePHu3auLbuyqbLPPzwwxqXLVvW9Yt1vDz66KNu26Z75+WeF7kTpsLYVCeb4jJx4kTXb/v27Rpv2rRJ4/A6Ze9L3333Xdc2d+5cjT///HONZ86c6fpt27Yt5usj9+xyCiL+GLP3muF3IreOOeYYjX///XfX9s0332g8depU12a/c7/99lue3js3mGkDAAAAAAAQQTy0AQAAAAAAiCAe2gAAAAAAAERQvq5pY0t8i4gceeSRGi9YsMC1HX744RrHyytu1qyZxsuXL9c4Vom+nNg8trVr12psy1mHli1b5rYzdU0by65fkVd9+vTRuFatWjH72VzSnLYRXTfddJPG4XeG4yg5xo8fr7EtyZ1XtrTp5s2bXVv16tU1tmVnp0+f7vrtu+++ez2OdBfmc9uyzYsXL9b4P//5T8rGdNZZZ6XsvbCr+vXru+1GjRrF7GvvbSZMmJC0MaWLChUquO1zzjknZt9LL71UY3vfmGx2HZtJkybF7BeuaROuBwmR3r17a2xLuOdWuE7b6aefrnFYNtyuf5PMNTDSVbx1Zo466iiNbann0Geffaax/V25ZMkS169atWoa27VMRRKzDiB2ZZ8H9OzZU+PwGCtRokSOf//DDz+47Y8//ljj77//3rXZ3yB2bcWmTZu6fvac0KZNG9c2e/ZsjW3Z8ERjpg0AAAAAAEAE8dAGAAAAAAAggvI1Per999+Pu22Fpdr+EpYbbdCggcZ2mlOTJk1yPa5ff/1V44ULF2ocpmzZqVJ2ajr2Ttu2bTW2pTP/8Y9/uH5r1qzR+JZbbnFtW7duTdLosLdq1Kjhths3bqyxPd5EKI2YKCeeeKLbrl27tsZ2em9up/qG0z/t9GRbOlNE5KSTTtI4Xjnif/3rXxoPHjw4V+PINLfeeqvbtlPE7VT8MEUt0ey1L/xuMV08teKl7ITCNALE98ADD7jtCy+8UGN7fyki8tprr6VkTKHjjz9e44oVK7q2559/XuMXX3wxVUMqMGzqrohIt27dcuw3Z84ct7169WqNW7duHfP1S5YsqbFNvRIReemllzT+8ccfdz/YDBfe/7/88ssa23QoEZ8eHC9l0ApToqxw+Qsk3pNPPum2bVpbvPLd9rnBV199pfG///1v18/+rg81b95cY3sf+uyzz7p+9vmCPQeIiDz++OMajxo1SuNEp8oy0wYAAAAAACCCeGgDAAAAAAAQQfmaHpUIGzdudNuTJ0/OsV+81Kt47NTjMBXLTsUaOXJknl4fu7LpMuGUSMt+5lOmTEnqmJA4YTqFlcqqG+nOpqG98sorri3edFPLVvOyUz7vuOMO1y9eOqJ9jcsvv1zj8uXLu34DBgzQeL/99nNtjz32mMY7duzY3bDTyrnnnqtxWLFg0aJFGqey0ppNcwvToT788EONf/rpp1QNKWOdcMIJMdvCqjTx0hOxq+zsbLdtv+srV650bcmsAFS0aFG3baf+X3XVVRqH4+3evXvSxpQObLqDiMgBBxygsa02E96z2OvTBRdcoHGYklGzZk2NK1Wq5Nreeustjc844wyNN2zYkKuxZ4LixYtrHC6BYJdRWLdunWu7//77NWaphOgI7+ts1aYePXq4tqysLI3t74IwdX7gwIEa53U5hbJly2psq5j279/f9bPLtISplanCTBsAAAAAAIAI4qENAAAAAABABPHQBgAAAAAAIIIK/Jo2yVChQgWNn3jiCY332cc/47LlqMlDzbs333zTbZ966qk59hs2bJjbDsvfomCoX79+zDa7rgn2TqFCf5/ec7uGTbg2VOfOnTUO88Zzy65pc++992r84IMPun7FihXTOPwevP322xovXrw4T+MoqDp16qSx/YxE/PUp2ewaSV27dtX4jz/+cP3uvvtujTNt/aFUsSVKbRwKc/xnzZqVtDFlmjPPPNNt23Lqdi2ncA2G3LLrqLRs2dK1NWvWLMe/ef311/P0XpmqSJEibtuuCfTQQw/F/DtbPvi5557T2J6rRUQOOeSQmK9h11pJ5npIBVmHDh00vvnmm12bLcNty96LiGzatCm5A0OehOexPn36aGzXsBER+eGHHzS2a8tOnz49T+9t16o56KCDXJv9bTl+/HiNw3VsrXC8w4cP1ziZa/kx0wYAAAAAACCCeGgDAAAAAAAQQaRH5aBnz54a27K0YXnxb775JmVjSjeVK1fWOJzebaes2pQMO+1eRGTz5s1JGh0SzU7n7tatm2ubOXOmxu+9917KxoQ/2VLRYYnYvKZExWLTnGyKjYhIkyZNEvpeBVXJkiXddqxUCJG8p17khS3XbtPtFixY4PpNnjw5ZWPKVLk9VlL5/UhHgwYNctutWrXSuEqVKq7Nll63U+fbt2+fp/e2rxGW8ra+++47jcOS04jPlusO2fS3MIU/lsaNG+f6vT/77DONuZfNWbzUT3vfuGLFilQMB3vJpiiJ7Jpabf3+++8aH3PMMRqfe+65rl+dOnVy/Ptt27a57cMPPzzHWMTf51asWDHmmKzVq1e77VSlhTPTBgAAAAAAIIJ4aAMAAAAAABBBpEeJyHHHHee2w1XK/2JXMhcRmTt3btLGlO5GjRqlcdmyZWP2e/HFFzXOtKox6aR169YalylTxrVNnDhRY1uVAYkTVr6z7NTTZLNT/sMxxRtj//79Nb7ooosSPq4oCSuaHHjggRqPGDEi1cNRNWvWzPHfuQ6mXrw0jERULsKfvvzyS7d95JFHatygQQPXdvrpp2tsq6KsXbvW9XvhhRdy9d62Gsns2bNj9ps2bZrG3CPtmfB8alPZbApimIJhK2B27NhR47DajD0Ww7bLLrtMY7uv58+fn6uxZ4IwFcayx9vtt9/u2t566y2NqZgXHR988IHbtqnU9jeCiEi1atU0fuSRRzSOlypq063CVKx4YqVE7dy5022PHj1a42uvvda1rVq1KtfvtzeYaQMAAAAAABBBPLQBAAAAAACIIB7aAAAAAAAARBBr2ohImzZt3HbhwoU1fv/99zX+9NNPUzamdGTzhRs2bBiz34cffqhxmKuKgumoo47SOMxJff3111M9nIxw5ZVXahzm5uaXdu3aaXz00Ue7NjvGcLx2TZt098svv7htm5Nv19QQ8etDbdiwIaHjqFChgtuOtb7A1KlTE/q+yFmLFi007tKlS8x+mzZt0phSuIm1ceNGjcPS9na7b9++e/1ehxxyiMZ2LTARf07o3bv3Xr9Xppo0aZLbtseOXbcmXGcm1roa4ev17NlT47Fjx7q2ww47TGO7Poa9bme68uXLaxzeE9i132677TbXduutt2o8ZMgQjW2ZdRG/bsqiRYs0njdvXswx1a1b123b34Wcb+MLy3Db9aBKlSrl2uzasnbd2fXr17t+y5Yt09h+J+xvDhGRpk2b7vF4hw4d6rb//e9/a2zXq0olZtoAAAAAAABEEA9tAAAAAAAAIihj06OKFi2qsS0dJyLy22+/aWzTc3bs2JH8gaWRsJS3nVpmU9BCdurv5s2bEz8wpESlSpU0Pv744zX+5ptvXD9bRg+JY1ORUslOaRYROeKIIzS254B4wjK5mXTuDacQ2zK+55xzjmsbN26cxg8++OAev1e9evXctk3JqFGjhmuLlRIQldS7dGevp/vsE/v/t7333nupGA6SzKZ8hMeeTb8Kz5XIvTCl9LzzztPYpm2XLFky5ms8+uijGodpcb/++qvGb7zxhmuz6R+nnXaaxjVr1nT9MrmM+/3336/xjTfemOu/s+fHq666Ksc4UezxZ5d26Ny5c8LfK52F6Ub2+MiLYcOGue146VE2Jd1+z55//nnXz5YUzy/MtAEAAAAAAIggHtoAAAAAAABEEA9tAAAAAAAAIihj17Tp06ePxmHp2YkTJ2o8bdq0lI0p3fTq1cttN2nSJMd+b775ptumzHd6+Oc//6mxLR88YcKEfBgNUqVfv35u25Y9jWfJkiUaX3LJJa7NlnXMNPZ8GJb+PfPMMzUeMWLEHr/2unXr3LZdO6NcuXK5eo0w7xvJEavkergWwJNPPpmK4SDBOnXq5LYvvvhije2aCyK7lr1FYtiS3fZ469Kli+tnjzm79pBdwyZ01113ue3DDz9c4/bt2+f4eiK7XgsziV3XZOTIka7t5Zdf1rhQIf9T9qCDDtI43vpfiWDX8LPfGVt2XETk7rvvTuo4IHLTTTdpvCdrCl155ZUa5+U+KpWYaQMAAAAAABBBPLQBAAAAAACIoIxJj7LTyEVE/u///k/jn3/+2bXdeeedKRlTusttib6rr77abVPmOz1Ur149x3/fuHFjikeCZBs/frzGtWvXztNrzJ8/X+OpU6fu9ZjSxddff62xLUkrItKgQQONDz300D1+bVvWNvTCCy+47a5du+bYLyxRjsSoWrWq2w5TNP6yYsUKtz1jxoykjQnJc8YZZ8RsGzt2rNv+3//+l+zhZDybKmXjvArPkzbdx6ZHtWrVyvUrU6aMxmGJ8nRnSyyH57VatWrF/LuTTz5Z48KFC2vcv39/1y/Wkg15ZdOXGzVqlNDXRs569OihsU1JC1PmrHnz5rntN954I/EDSxJm2gAAAAAAAEQQD20AAAAAAAAiKK3To8qWLavxI4884tr23Xdfje3UfhGRzz77LLkDg2Onf4qI7NixY49fY9OmTTFfw06PLFmyZMzXKFWqlNvObXqXncLZt29f17Z169ZcvUY6atu2bY7/PmbMmBSPJDPZqbrxKijEm5Y/dOhQjatUqRKzn339nTt35naITrt27fL0d5ls1qxZOcaJ8N133+WqX7169dz23LlzEzqOTNW8eXO3HesYDqsvomAKz8NbtmzR+IEHHkj1cJBkr776qsY2Per88893/ezyASzdkDvvv/9+jv9u04lFfHrU77//rvFzzz3n+j311FMaX3/99a4tVtoqkqNp06Zu254bixcvHvPv7LIbtlqUiMj27dsTNLrkY6YNAAAAAABABPHQBgAAAAAAIIJ4aAMAAAAAABBBabemjV2rZuLEiRoffPDBrt/ixYs1tuW/kXpz5szZ69d47bXX3PaqVas0rlixosZhvnCi1alTx22fcsopSX2/KGnRooXbrlSpUj6NBCIigwcP1njAgAEx+9lysvHWo8ntWjW57TdkyJBc9UP+sGsi5bT9F9awSQ67Jl9o3bp1Gg8aNCgVw0ES2LUV7H2KiMiaNWs0psR3+rHXSXt9Puuss1y/22+/XeNXXnnFtS1cuDBJo0tP7777rtu+5557NLYloi+77DLX79BDD9W4ZcuWuXqvFStW5GGE2J1w7cMDDjggx352TTARv27UJ598kviBpQgzbQAAAAAAACKIhzYAAAAAAAARlHbpUTVr1tS4UaNGMfvZcs42VQqJE5ZSD6d9JlKnTp3y9He2zF+8tI63335b4xkzZrg2mzYwbdq0PI0jHXTs2NFt21TFmTNnavzRRx+lbEyZ7I033tC4T58+rq18+fJJe9+1a9e67QULFmh8+eWXa2xTGBE92dnZcbeRXKeddlrMtmXLlmm8adOmVAwHSWDTo8Lja9y4cTH/zqYElC5dWmP7vUDBMWvWLI1vu+021zZw4ECN//Of/7i2iy66SONt27YlaXTpw96LiPiy6+edd17Mv2vVqlXMtj/++ENje8zefPPNeRkicmDPdzfddFOu/uall15y2x9++GEih5RvmGkDAAAAAAAQQTy0AQAAAAAAiCAe2gAAAAAAAERQgV/Tpnr16m47LOn2l3BNB1vmFslx9tlnu22bi1i4cOFcvUbdunU13pNy3c8++6zGS5Ysidlv1KhRGn/99de5fn38qVixYhq3adMmZr/XX39dY5sDjORZunSpxp07d3ZtHTp00Pi6665L6PvaMpoiIo8//nhCXx+psd9++8VsY/2E5LDXRbs+X+jXX3/VeMeOHUkdE/KHvU527drVtd1www0az5s3T+NLLrkk+QNDUg0bNsxtX3HFFRqH99R33nmnxnPmzEnuwNJAeN26/vrrNS5evLjGjRs3dv0qVKigcfh7Yvjw4Rr3798/AaOEiN8f8+fP1zjeb0d7DNh9m06YaQMAAAAAABBBPLQBAAAAAACIoAKfHmVLyIqIVKtWLcd+U6ZMcduUL029AQMG7NXfd+nSJUEjQaLYqfkbN250bbZM+qBBg1I2JuwqLLNut21KaXg+bdeuncZ2fw4dOtT1s2Xv7VRWFFzdunVz2z/99JPGd911V6qHkxF27typ8YwZM1xbvXr1NF60aFHKxoT80aNHD40vvfRS1/bMM89ozLGYXtauXeu2W7durXGYmtO3b1+NwxQ67N7q1as1tvc6tpS6iEizZs00vuOOO1zbmjVrkjS6zHbSSSdpXLVqVY3j/Xa3aaM2hTidMNMGAAAAAAAggnhoAwAAAAAAEEFZe5ImlJWVFYmcohYtWmg8fvx412ZXnLaaNm3qtsOpx1GXnZ2dtfteuxeVfZihvszOzm68+267x37MPxyLaYFjcTfGjBnjth988EGNJ0+enOrh5Cidj8UqVaq47bvvvlvjL7/8UuM0qM6WsceivZe1lYBEfArr4MGDXZtNRf7tt9+SNLo9k87HYlS89957bvu4447T2FY82osU5Yw9FtNJOhyLs2fP1rh+/fox+w0cOFBjmy6YBnI8FplpAwAAAAAAEEE8tAEAAAAAAIggHtoAAAAAAABEUIEs+X388cdrHGsNGxGRxYsXa7x58+akjgkAgHRhS6Ai9VauXOm2u3fvnk8jQbJMnTpVY1viFsjJOeec47a/+uorjQ877DCN92JNGyASypQpo3FW1t9L9IQl1h9++OGUjSkKmGkDAAAAAAAQQTy0AQAAAAAAiKACmR4Vjy0TdvLJJ2u8YcOG/BgOAAAAAOTZzz//7LarV6+eTyMBkuvBBx/MMb7rrrtcv1WrVqVsTFHATBsAAAAAAIAI4qENAAAAAABABPHQBgAAAAAAIIKysrOzc985Kyv3nZFQ2dnZWbvvtXvsw3z1ZXZ2duNEvBD7Mf9wLKYFjsU0wLGYFjgW0wDHYlrgWEwDHItpIcdjkZk2AAAAAAAAEcRDGwAAAAAAgAja05Lf60RkaTIGgrgSWdePfZh/2I8FH/swPbAfCz72YXpgPxZ87MP0wH4s+NiH6SHH/bhHa9oAAAAAAAAgNUiPAgAAAAAAiCAe2gAAAAAAAEQQD20AAAAAAAAiiIc2AAAAAAAAEcRDGwAAAAAAgAjioQ0AAAAAAEAE8dAGAAAAAAAggnhoAwAAAAAAEEE8tAEAAAAAAIggHtoAAAAAAABEEA9tAAAAAAAAIoiHNgAAAAAAABHEQxsAAAAAAIAI4qENAAAAAABABPHQBgAAAAAAIIJ4aAMAAAAAABBBPLQBAAAAAACIIB7aAAAAAAAARBAPbQAAAAAAACKIhzYAAAAAAAARxEMbAAAAAACACOKhDQAAAAAAQAQV2pPOWVlZ2ckaCOLLzs7OSsTrsA/z1brs7OzyiXgh9mP+4VhMCxyLaYBjMS1wLKYBjsW0wLGYBjgW00KOxyIzbYDUWZrfAwAgIhyLQFRwLALRwLEIREOOxyIPbQAAAAAAACKIhzYAAAAAAAARxEMbAAAAAACACOKhDQAAAAAAQATx0AYAAAAAACCCeGgDAAAAAAAQQTy0AQAAAAAAiKBC+T0ApK99993XbVeoUEHjDh06aNypUyfXr3Dhwjm+Rvny5V2/6dOna9y7d2/XtmrVqjyMGFHzj3/8I8d//+2331I8EhQrVkzj0qVLu7affvpJ4y1btqRsTAAAAAVFeF9rf/Ns27ZN4507d6ZsTCgYmGkDAAAAAAAQQTy0AQAAAAAAiCAe2gAAAAAAAEQQa9pgr2RlZbntcuXKady6dWvXdv3112tcs2ZNjQ844ICYr//HH3/EfC+7xk3RokVd20UXXaQxa2xEW5EiRTQ+4YQTXFuPHj00Hj58uMbjx493/cj9TYxwHarTTz9d45tvvlnjihUrun6TJk3S+NZbb3VtGzZsSOQQkQfhufMv2dnZe/16eX0NJN8++/j/L1eqVCmNf/31V423bt2asjEBQLqw10L7O6RSpUquX/v27TWuUqWKa7Pn6Xnz5mn88ccfu36LFi3au8GiwGOmDQAAAAAAQATx0AYAAAAAACCCSI/CXilUyH+FzjzzTI379evn2uzUbJuGEZbn/v777zW2pZ2PPPJI18+WHQ7Taho3bqzxlClTYv8HIN/ZqaHHHnusa7PfL1tWmpSM5AhLUXbo0EHjBg0aaGxLVIqIdOrUSePly5e7tv/+978ak8aWPMWLF9d4//33d207duzQ+Oeff9bYpp+K5O24ClPq7Hcj3N92HBzDyWH3xzXXXOParrzySo3t1Hv77yK7fi+wq/BcWatWLY1btGih8Q8//OD6LViwQONly5ZpbI8NkcQcHzZ1o1ixYq7NpqVv375d402bNrl+nLORacJrmv3tYn9biPh7pCOOOELjww8/PObrh9dne5za4693796u35IlSzT+/fffY74+0hczbQAAAAAAACKIhzYAAAAAAAARFKn0qLDSgcUUzWgKp/DaqfE2nUXET+ebPHmyxs8884zrZ6cA2mnaLVu2dP0GDRqksa1aJSLSs2dPjadOnZrj6yEa7FTUEiVKxGwjPSr5jjrqKLdt0x1tla/w87epAmeddZZre+uttzSeP39+QsaJP9nzrU0fPemkk1y/d999V+NZs2ZpnIgp1mGVDDuOL7/80rWtXbt2r98P8ZUsWVLjXr16uTZbtcRO0Q9TfbZt25ak0RVs9nM6++yzXdvll1+usU11svcfIiLfffddkurraVgAAB2LSURBVEa3a1pH1apVNbapGyL+nnrp0qUab968OWa/TGI/yzD9vmPHjhqPGTNG4zAV36b3I/+Fx4e9dtnfF/b3g4hI/fr1NQ4r1dp7IVuRL/ytYX/fhseUvY7bNOfwXmrixIkar1+/XpB5mGkDAAAAAAAQQTy0AQAAAAAAiCAe2gAAAAAAAERQyte0sbl7TZo0cW1t27bV+IwzznBttkyizRu165+I+NxcuwbGli1bXD+bsx0vZ9fmIYbrbcQq0yaSOeXYwv/Od955R+PatWu7tpkzZ2o8adIkjTds2BDzNe1n/MUXX7h+Gzdu1NiW/xbx35dMzcmOKrtPRfx6RDVr1nRttvz7t99+m9yBZShb+nXo0KGurXz58hrb/RZvTZuw1OVzzz2nsV0HYuXKla4f6xTtXrjum821v+CCC2L2s9fCRKzrZb8L5513nmuz5d/79Onj2ljTJvHC86m9jwrXG7J97XU3LDeNP4WfbeXKlTUO172wa1G8+uqrGo8ePdr1s2vaJHrNE7uekYjIfffdp3HZsmVdmz0v2/LvmXLvmpNChf7+SXTZZZdpbD9HEZH99ttP4/PPP1/jq6++2vUbNWqUxtyHpo49bu39TbNmzVw/u2aMXb/Prv0l4r8X4TnBXk9/+eUXjcePH+/62bWtwt8rdi3BefPmaRyu92l/8+BPdn/Y3+itWrVy/S699FKNq1Wr5tqWLVumsX2msHjxYtdv7NixOfYTSd15k5k2AAAAAAAAEcRDGwAAAAAAgAhKeXpUvLJnJ554osZ2OpqIn9bWvHnzmK9hp53ZKYzhdHGbLvXDDz+4tjVr1mhcr149je00OxFfGvGhhx5ybQMHDtQ4k6ab/vjjjxrbktwi/nOwn3+4D2OlSdhUDRE/xc2W2hMReeqpp3b7enllU/xE/PRIpsDuXngs2imqhx56qGsbNmyYxtu3b0/uwDJIqVKlNP7ggw80DsvChiUy/xJ+z+35Ovwbmyb58MMPa9yjRw/XL0wxxa5sCoaIyP/93/9pbI+dfv36uX52Km8izlG2/Psll1zi2g466CCNw1S5Tz75ZK/fG57dFyL+OxHeR9lrsE2FzKR7lD0RnsvOPfdcjcuUKePa5s6dq7FNiwnTehN9P2Kvp127dnVttlR1vHO2vX/KpDTVcP82btxY4759+2psf0uI+M/Opp3dcMMNrp9Nr5g1a5Zr415x79i0mHD/HHjggRo3bNhQY5vGLeLT7+fPn69xeK9ZsWJFjcPfGja18IknntD4f//7n+sXLtFh2f8Wey7OpGMxnnjLqtjf3vb+NSzNHi+9v06dOhrb33Phvu7evbvG48aNc22PP/64xqtWrcrx9RKBmTYAAAAAAAARxEMbAAAAAACACOKhDQAAAAAAQASlfE0bW+Jw4cKFrs2WIAzXj6lVq5bGNq8/LBFs22yeY5hXZnPk9t9/f9e2devWHPuFeZN225YTExG5//77JRPZ/Wvz+kR8HrDNLQ3XOLH5hjZv3OYMiogUK1ZM4zFjxri2cJ2ivWXHHn4PbN4jecq7F64JdM4552gcrtnx1VdfpWRM6S78zO2aC7bcZHgsxhKWCLbb4TFg1w2wayzcfPPNrp9di4M1Nv5m94ld/0lE5JhjjtHYlgq1efYifv/Y82tYvjS3OfT2+mzXRxLxueRhTjgSLyxPW7Vq1Zh9ben3559/PllDShsVKlRw2xdeeKHG9v5DxN9z2BKyyV6XwpYht9dSEX+fFa6x8eabb2qcSfct9npUvXp112bLfNu1ouKtu2ivVeG6i7fccovGw4cPd22TJk3S2P7mQM7Ca5Xddx07dnRtdg2aL774QuPp06e7fnYNTrseTXiPb0tJh2vv2TLf9ncm69HsmXC9oeOOO07jBx98UONwnbzw3vYv9reoiL8XsfssbLP3W+G9zWGHHaZxuH7VaaedprFd+8yumySy998LZtoAAAAAAABEEA9tAAAAAAAAIijl6VF2atCGDRtcm52uG5aqtGkTdqpiWO7STqGzU7jD6Ye2pJst3S3ip09ee+21Gt92222unx3HiBEjXFuYPpCJwhQHu08rVaqkcTjN2JaMtZ95/fr1XT87/a1Pnz6uLRFTE+00OTv28PvCNMg9U65cObdtpxyGUxqXL1+ekjGlu0aNGrltm6YULyXKfrftVOJHHnnE9bPlbsPjtFOnThrb495ORRfx1wNbGlwks8+ndqp37969XZtNRXrqqac0DqdwJ/ocZdNW7fhE/Hl/6tSpCX1f/Mne57Rr1861hdPMrWnTpmkcrwRtJrPX+jDdyJ6/wnLRse5REyE8R9t72xtvvFHjBg0auH6LFi3S+OKLL3ZtYYpAugrTaux+Ov74412bTRW216PwfPrzzz9rbO9RatSo4fq1aNFCY1t6WkSkV69eGttUNeSsefPmbvuaa67ROEwRtcslTJkyRePwN2es4zT8vRj+HfImPI/Z3wKPPvqoa+vQoYPGNgUqXOrEpqLa+8aRI0e6fvbvwmukXSLFnkPvuOMO18+ed8NnFAcffLDG9h44TI/aW8y0AQAAAAAAiCAe2gAAAAAAAERQytOj4rHTqsPUmlRWobBTuNasWRNzTDZdYPDgwckfWAFnPz87rb9r166u3+mnn66xTZ0Jp/i/9tprGtspcslgp1GSDrXn7BTlcEqyXan/hRdecG1huhRyz07fHDhwoGsLp/b/JTzHvfLKKxr37dtX49WrV7t+9piwVTFERGbNmqXxPffco3HdunVdP1s9KhSOP52F0/ltepk9H4r489748eM1zm16xp6cy+y4bJpWWGnDViiy10gkjp3ebStViPj9ZNPARUTuvffe5A4sDdgUl/bt27s2m8odTo+3+8ROo9+4caPrZ485ex4Oj3srrGJiz8UXXXSRxmH6wX333adxeM7OFOHnWrp0aY2bNWvm2tavX6/xRx99pPGSJUtcP5v2aVPmbMqTiL/GlS1b1rXZpRfGjh2rMZUT/2a/93fffbdrq127tsaffvqpa3v66ac1Do8/pJY9x4Wp1PZ6ZNOhRHxKlE2PD6uw9ezZU+PwehdLeJ606d72/B9Wg7PnknhVUu3r5bVCZyzMtAEAAAAAAIggHtoAAAAAAABEEA9tAAAAAAAAIihSa9pEhc1HPu+88zQOc5jfffddjTM1X3hP2JJra9eu1bhJkyauX7Vq1XL8m/nz57t+tsx3oktsivjcw3hrq9jvBfnIOYu3BsO2bds0Dktfsn5Q3lWtWlXjsAx3rO/2xIkTXb8rrrhC47AMZizh+mN2bYAJEyZoHK5pY0uxhmsDDBo0KMfxpqOSJUu6bbtmRXg8PP744xpv3rw5qeOy++ess86K2c+urZPbHHPsmQoVKmhcp04d12a/IytWrHBtc+bMSe7ACii7foIt+2w/ZxG/PoH9GxGRRo0aaWzX6bPloUVEVq1apbE9l9l/F/HreQwYMCDme9n7j7lz57p+lJLedf0KuwZXeN84efJkje2aieFxZNfYiFcOukiRIjHbbAlw+13iHvJvdj23o48+2rXZNURsWW8Rv64a8pc93k455RTX1qZNG43D86m9ji1atEhju06XiD8W47HnyerVq7u2Sy65RONu3bppHN6LWfZ3i4jIhx9+qLE9dyT6tykzbQAAAAAAACKIhzYAAAAAAAARRHqU7Fr+9sknn9TYlicOp/3baVo2jQc5s9Pd7NRfm44WsiUY77//fte2adOmBI5uV7lNzWHf756dZh6W2bTTDL///vuUjSndhKUFW7durXHRokVdm/1uf/vttxrbUs4iu57z8sKmyHz++ec5/ruInx4blrht3LixxtOmTdvrMUWN3Xe25KSIT3PbsmWLa3v//feTOi6rc+fOGtv9E6YH3HTTTRqT3pg49jty6qmnahweK3Y69jPPPOPacpvimGnsZzt79uwcYxGRcuXK5fg3IiJVqlTR+Pbbb9c4TM+xKVEzZszQ+IsvvnD9TjvtNI3DFHJ7z2pf77LLLnP9SLXZNT3BpkmE1ze7T226abwUDJtCEd7bxGP3of1eLV++PNevkY5sKv3VV1+tcfg7wd53h/c3dj9yDcpf8VJAw9/elt2H9h4oTI+aPn26xvZ8HZ53r7nmGo3t73oRn8ZoxxR+d+xvlYULF7q2xx57TONkXmeZaQMAAAAAABBBPLQBAAAAAACIINKjRKRmzZpu205LtVOlnn/+eddv2bJlSR1XurHT1dq2bavxwQcf7PrZ6XTDhw/X+OOPP3b9wunJiBa7f2zFqLJly7p+dr+uW7cu+QNLU+F00I4dO8Zss9Pmhw4dqnE4fTURK9/bKaZr1qzROF4VqPDYTveKUVblypXdtp26G1bF2LhxY9LGUaxYMbdtpxfb78+YMWNcvx9//DFpY8pkNn3wuuuu0zisammnZj/11FPJH1gasOcXe1935513un42hclW/xHxKS62AlWYumErEU2aNEnjMM3Qvl68NAL7GjNnzozZL1OF1zD7OYf3G7aqTO3atTUO96HdN3379o3Z75dfftE4rCRl96m9Hx48eHAO/xWZw6aixUtHtJ9fly5dXJut3rN06VKNw+9CIlKnSMWKz+5PW0lURGTEiBEa2yqZIv5YspUr7X2tiD927LEdnjNLly6tcVipyrJpd+H9lk3vtymwIj41KxnVjP/CTBsAAAAAAIAI4qENAAAAAABABPHQBgAAAAAAIIIydk0bm+928sknuzabg/fZZ59p3K9fP9eP/MU9Y8v22s8yzC+0eY92TZswvzBcpyO/8D3Imd2vthRpuN+eeOIJjSmfnndhDq8tkxh+ruvXr9fYrimUiFKF8XLPben3MMff+vnnn932okWL9npcUWbPIeHaFrY0evi52PXA7N/FWwPIHn/hujW2fLRd201E5MADD8zx9SdMmOD6JTOfO5PZfXPooYfG7GfXFErmmkfpyn63Fy9e7NpsWddQrOPKljAWEdmyZYvGdm2osKTxeeedp7Fda0VEZPXq1RpfcMEFGnMvsntr167VeM6cOa7Nlm2//vrrNQ7XvTzkkEM0tvvdrnkhIvLNN9/EfI26detqbEsQP/fcc65fWJY83dnvsD2XVapUyfWzn7tdf0hE5M0339TYrkNlf8+JiCxYsCDH97X3KSIi++23n8Z2jRwRkW+//VZj+93CrsLvsl0n74477nBt9nf5zTffrHH4Pfjuu+80tmtUHXHEEa6fXR8pZM/Dy5cv1zhcE86uwWP7iaTut0s0fvUCAAAAAADA4aENAAAAAABABGVselSZMmU0tlO0RPwULpvGs3nz5uQPLI2E5Q/Hjx+vsZ0KHE75HzRokMZ26ls47T6V0/Btyke8UuNhukEmf2eqVaumsZ12vGnTJtfPlixF3oXpUeG0fMumgNoppXmdXm+nKofva1M5brnlFo3DY8W+dzj1dNu2bXkaV0G0ZMkSt23TRWvUqOHa7JRiO9U7LN1up/+2atVKY5tyIyIyd+5cjY899ljXZvvaFBKbaidCikayHHnkkRrb1NPw83755Zc1JlVt7+zJd9lOj7elnnMrTB1YuXKlxvPmzXNtN954o8bh/RPis9e+KVOmuLYSJUpobFO6bWqoiL/WTp8+XeMePXq4fjY9MVyGwZYMbt68ucZnnHGG6zd69Ogc/isyw3XXXadx+DnYEs5hyv3hhx+usU2duvDCC10/e8za70WoUKG/fyqH72V/DwwcOFDj2267zfXjurgre31as2aNa7OpSK+88orG4XIa9jUqV66s8fvvv+/62f0UXhdtitt///tfjUeNGuX62dTW/NqfzLQBAAAAAACIIB7aAAAAAAAARBAPbQAAAAAAACIoY9a0CddXeeuttzS2pXFFRN555x2Np02bltyBpbFevXq5bZuDanMKv/rqK9dv6tSpGtu1GMK1ZOKtM5OIfEP7mjanNfwuWd27d3fbDz/88F6Po6AI98GZZ56psV3nJCzfbPNEkXfhd97m3dvvr4hI8eLFNY5XHjqWcP0cW3r6pJNOcm1XXHGFxvXq1dM4/L7YnPJHHnnEteVljAVVuEaMLTt56qmnujZbKva4447TOFxXyK7hZs9fK1ascP3selPhPg5z+f9i10RC4oSfv117z+4LWxJeRGTIkCEas45CwWFLfIuINGnSROOXXnrJtYWlpZE34ZqDr776qsZ2vZuwfLBdk9Guybd169aY7xXe59py1nb9v27durl+b7/9tsapKiscFfb3V6dOnVybXXukZMmSrq18+fIa2+tdkSJFXD+7Pkq8dcLs+TY8L1t2zb7wGB03blzM10d89vMK7wXtfWSLFi00PuSQQ2L2C9cBe+CBBzS2a+lE8b6TmTYAAAAAAAARxEMbAAAAAACACErr9CibEjBs2DDX1rRpU43D6cX9+/fX2KbnYPfsNEKbHiPip7ht2LBBY1sKXMSnTsWaki/i929Yrs++V27LnobvZadV1q1bV2M7lVXEp33NmjUrV++VjsJpoxdffLHGduppWOKbqaKJEU7ltOe1MBXJpkfVqlVLY3tcivjp2DblxqY5iYjcddddGtspqiK+tLc9xsJzqz12bNlikcz6joSfywcffKCxTR0V8fvBllYvW7as62dLhS9btkxjmw4g4j/ncB/XqVNHY1ue2E7zR+JUrFjRbdu0Q3schSlupKsVHLaM+7333uva7P3NTz/95NoyLU0mVeznakuuxzvHxbs22X0YpmLNmTNH4/r162sc3l/ac/d3332X6/dOB/bePSzPfsIJJ2gcXu9OOeUUja+++mqNDzvsMNdvv/3209ju+/A3of0tEK/kt2XfV8SXhg/LWyPvTjzxRI3t7/zw94j9Ls2cOdO1RT0lymKmDQAAAAAAQATx0AYAAAAAACCC0jo9qmHDhhq3adMmZr9PPvnEbc+bNy9pY0p3dkqarVgi4qen2amIFSpUcP0OOuggjW2qgE2zEBEpVaqUxuFq4LGm74dVVSpVqqRxy5YtXZutzFK7dm2NY02HFBG59dZbY7alu3AFf1tRyO77N9980/VL9ym+qRKmAY4dO1Zjm94n4qsoPP300xoPHDjQ9bPTsW1KaZcuXVw/e3yEx5g9XuzxHFYR69ixo8ZRn6KaSna/2vOaiMiMGTNyjPPKnr/D/dO6dWuN7VR/qr8lR9u2bd22naJvvxP2+BXZdWo/osUeY+3atdM4vK+w+9FWdcupL5Irtyn2IZu2v3r1atc2fPhwjY8++miN7X2TiMhFF12k8YABA1ybvR7kdYwFRXifaI8Pm8om4tNd7O+5Pn36uH4NGjTQ2P5e2bZtm+tnP9vw2LPpUvZauHbt2pjjRd6FacO2ulpYJdWyKWmXXnqpa4tX9S1qmGkDAAAAAAAQQTy0AQAAAAAAiCAe2gAAAAAAAERQ2q1pY0u49e7dW+NwnQVb9tSupYC9Y9eqWbp0qWuzJWntOjb/+te/XD+bb2jzv8N8RZujGOYk2vdesmSJxmeccYbrV6JECY3td0fEr6thS24uXrzY9bPlkz/++GPJVGGJQ7vmkP0swxK1SI4XX3xR4zCX2x5XtuT3Y4895vrZ48qeQ+36GiI+rzvM+bbr2Njy1VdddZXrt2rVqhz+K5BKdt2AwoULuzab12/z9Vl/KHHscXT55Ze7NnvM2rUsXnvtteQPDAljz512bbHQ999/r3FYvjYsO4zos9dBkdhrrYTX4KOOOkrjcJ1Iu06OPXdn+jqB9po0e/Zsjd944w3Xz65badfMDH8LbNiwQeNw/Uz728CWcR85cqTrF66Tg9yz57+JEye6Nvv7ywrXELK//exajQUNZ34AAAAAAIAI4qENAAAAAABABBX49KhwmqgtodikSRON7RQ2ET81n5KliWOn0L/wwguu7bjjjtPYTkUMpwjHmzJs2dJv4VQ422a/B/HKEYevYaev2rKqYYl4W+p648aNuRp7urCfZzid3362tkSwTWtD8nz99dcaT58+3bU1a9ZMY7ufwmnBdjvelGvbFqbLvPfeexrb8qXhORn5z34XqlevHrPNTkm2KbHYOzYFuHz58q7NHmPLly/XOCx3i2iz18zSpUtrHKZP2OMtLHNr75EyqexzOrHlwO21ety4ca6f3ff2HlpEZPz48Rrb70+YipXJ7Occfrbdu3fX+JhjjtE4TEe017tffvnFtf3www8ajxo1SuOpU6fGHAf2jE0RtMtshOwx0KtXL9dm0+QKMmbaAAAAAAAARBAPbQAAAAAAACKIhzYAAAAAAAARVODXtLGlo0VE7rrrLo3LlSun8ejRo12/SZMmJXdgkJdfftltf/755xrbcnhhjmKsdTTCtYfs69lYxK8tY/PGwzVn1q9fr7EtnygiMn/+fI1tyb8wbzyTS97aNRjC9aXsvpswYYLGmfx5pZJda6Rly5au7amnntK4c+fOGtv9KeLXVbD7M8yZt8fOQw895NqGDBmiMWUvo83u71KlSsXsZ9fRyPTysolkj6ulS5e6tsqVK2ts18BgrYSCxe4ve/8abz8efPDBbtuusWHvi7i2Fkz2fPrhhx+6Nrvuyumnn+7a7NpWs2bN0ji8R2Wtoz/ZtRVFRG666SaNhw4dqnGVKlVcP/ubZO3ata5t2LBhGk+ZMiXme3Gd3DN27a/rr79e43ANPXvOe/311zW297gi6fP5M9MGAAAAAAAggnhoAwAAAAAAEEEFMj3Klovu16+fa7PTSO00qhEjRrh+lMRLvnA62qJFizRu1KhRqoeDJLDH0dixY11btWrVNL7hhhtSNibsKpw2b6dc33PPPRqfffbZrl/Dhg01tsfzp59+6vrZVEibcihCSeiCxKbHheka9lifO3euxuky7TgKbHpaOA3fTre3KTFhWirHW7Rt3bpV402bNmlcuHBh1+/AAw/U2KZni4g0a9ZM48mTJ8fsZ79PYYoMx2102GN25syZrm3VqlUah0sJHHHEERovW7ZM45UrVyZ6iGlpzpw5Gl988cUad+nSxfWz90/h0hp2GQV7Xo53fNnjMty2f5fJx+g///lPjdu3b69xkSJFXL/FixdrPGDAAI3T9Tc+M20AAAAAAAAiiIc2AAAAAAAAEVQg06Ns2kXXrl1dm11x+vvvv9f4448/Tv7AgAxjp41eeeWVMfsxZT9a7P5YuHChxvfdd19+DAcRsX37do1vvPFG19amTRuNX3zxRY0zeQp3otnj8umnn3Ztdgr97bffrjHVowoWu4/vvvtujcOUlqOPPlrjcKq/TYmz+z88FqkaVPDYSlIiIsOHD9c4rCJmU3PWrVunMfs9d+zx8s0332hsz68iu6Yz5eb19qQf19Bd03zbtWun8f77769xeC585513NF6wYEGSRhcdzLQBAAAAAACIIB7aAAAAAAAARBAPbQAAAAAAACKoQK5pc+yxx2pcvHhx12ZzA1evXp3jvwNIPNatAQo2e52cMWOGawu3kVwTJkyIu42Cz5Zp7t+/f8x+4Zoa3M+mr3DfrlixQuMKFSq4tubNm2v8yiuvJHdgGYzjLfXsGk22lPqaNWtcv2HDhmmcCb9BmGkDAAAAAAAQQTy0AQAAAAAAiKACkx5VqNDfQy1RooTG++67r+u3adMmjbt166ZxWEYPAAAAiDLSMzJHWK578+bNGn/00UepHg6QEsWKFXPba9eu1XjatGkaz5492/XLhDLfFjNtAAAAAAAAIoiHNgAAAAAAABHEQxsAAAAAAIAIytqTXNmsrKzIJdYWKVLEbf/2228ap1MecHZ2dtbue+1eFPdhBvkyOzu7cSJeiP2YfzgW0wLHYhrgWEwLHItpgGMxLXAspoF0Oxb32efv+SXhmk9pLMdjkZk2AAAAAAAAEcRDGwAAAAAAgAja05Lf60RkaTIGklfbt2/P7yGkQvUEvlbk9mEGYT8WfOzD9MB+LPjYh+mB/VjwsQ/TA/ux4Eu7fZhBKVFWjvtxj9a0AQAAAAAAQGqQHgUAAAAAABBBPLQBAAAAAACIIB7aAAAAAAAARBAPbQAAAAAAACKIhzYAAAAAAAARxEMbAAAAAACACOKhDQAAAAAAQATx0AYAAAAAACCCeGgDAAAAAAAQQf8POXD0H+SofYgAAAAASUVORK5CYII=\n","text/plain":["
"]},"metadata":{"needs_background":"light"}}]},{"cell_type":"markdown","metadata":{"id":"97QR9a_vydVv"},"source":["Εκ πρώτης όψεως, ο βαθύς αυτοκωδικοποιητής έχει μάθει την καλύτερη αναπαράσταση σε σχέση με τους υπόλοιπους. \n","\n","Ας δούμε την αντίστοιχη κωδικοποίηση των $10$ πρώτων ψηφίων ($32$ χαρακτηριστικά)"]},{"cell_type":"code","metadata":{"id":"OxT7bX_zyyP6","colab":{"base_uri":"https://localhost:8080/","height":427},"executionInfo":{"status":"ok","timestamp":1642514737272,"user_tz":-120,"elapsed":254,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"cc48bbc8-032d-426a-c596-3dab30f22edd"},"source":["pd.DataFrame(data=h_deep[0:10], columns=range(1,dim_h+1), index=range(1,11)).head(10)"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["\n","
\n","
\n","
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
1234567891011121314151617181920212223242526272829303132
16.5421778.8577576.7039469.4056559.0733229.16424911.5979590.05.3558235.4218177.1219603.7411578.3065458.2447122.87702810.3118958.17995810.16504613.46103113.1589997.1920369.6879436.2123903.3382722.7483045.8644752.8622307.3177377.1408207.5126615.9221815.122150
26.6043217.08820812.1396209.8686698.9028259.8264353.6607140.06.4735655.5728598.2487698.5083818.8818421.4314383.4105479.3979953.4824278.4207025.3932716.9467439.3870287.6631563.2829919.58635715.3777186.7220573.1381586.3784393.6097985.9793486.2570989.722039
34.8349496.7723894.84310313.0764122.5748546.0357985.8437780.07.1549335.4975847.2846406.9450428.3661799.4391011.8332693.4143365.4379708.7869605.8542825.1773097.6257582.5969412.7231149.9512845.9815027.3985304.5314494.3358866.2297361.5243177.2262956.453763
43.0265076.9017344.3885835.6797835.7040038.8902866.7596680.04.7123114.4124369.8850131.55715012.6688722.2725644.1762894.6206865.6459493.0264736.2943716.4762272.58010110.3175544.1654705.74122012.0478894.5183916.3666945.5981125.7177399.4566866.9853106.148964
57.60863811.26228910.1076124.5438304.7497975.5403775.5609970.08.1216834.8338469.1513649.4747094.2860506.6870753.8246847.6376966.4668806.15975612.3337048.0471752.4316298.69259610.1338366.7002547.4791684.3624247.3238743.3319187.4962455.9763964.51366011.470777
66.3714615.6558775.43444213.5715461.8017496.3987726.6902690.07.3924595.1890898.3666267.6144959.6472879.9810621.9159864.7508716.2922938.5126505.7379347.3092267.1981642.7092462.4889559.0112066.6560597.6696995.1219945.6288455.5010852.0064647.6738126.394334
77.1624569.90231210.2489585.7964643.7480586.7454841.5910610.07.5182726.6921836.9704596.5322762.4351999.1372912.7631918.1371215.36763610.11264512.11757910.0085935.9251487.1312458.1674892.5193968.88002110.7663038.96172810.5581585.6609934.9088055.1379646.605459
85.0325866.3991649.9088695.3450179.5513358.4866087.8508840.09.3330157.1309858.57365111.0114303.5909325.3278094.4206175.4520845.8745895.6563805.0218476.8673634.2114288.2199955.0640356.3337356.6058965.8910285.0173753.44486711.4682551.8501514.3608395.017229
97.69826610.7033198.61818910.9827170.3273668.8717581.1522440.06.76117810.8579746.8065878.3766835.4297806.5593382.8806288.5247384.7085391.84990010.48854410.1229726.8402514.0236569.15283210.1139438.2735618.5165738.5300902.8272888.50596810.1305690.0000005.915468
103.0093206.4350774.67384210.8168427.6115479.72824911.3106840.05.1882004.03268010.5035226.0171857.0912967.2911363.91240510.5163946.4896296.3194499.7146108.6644985.6906104.6448626.5787083.0794366.2547747.40140112.59740811.5827816.4504787.4041049.8990314.481121
\n","
\n"," \n"," \n"," \n","\n"," \n","
\n","
\n"," "],"text/plain":[" 1 2 3 4 5 6 7 \\\n","1 6.542177 8.857757 6.703946 9.405655 9.073322 9.164249 11.597959 \n","2 6.604321 7.088208 12.139620 9.868669 8.902825 9.826435 3.660714 \n","3 4.834949 6.772389 4.843103 13.076412 2.574854 6.035798 5.843778 \n","4 3.026507 6.901734 4.388583 5.679783 5.704003 8.890286 6.759668 \n","5 7.608638 11.262289 10.107612 4.543830 4.749797 5.540377 5.560997 \n","6 6.371461 5.655877 5.434442 13.571546 1.801749 6.398772 6.690269 \n","7 7.162456 9.902312 10.248958 5.796464 3.748058 6.745484 1.591061 \n","8 5.032586 6.399164 9.908869 5.345017 9.551335 8.486608 7.850884 \n","9 7.698266 10.703319 8.618189 10.982717 0.327366 8.871758 1.152244 \n","10 3.009320 6.435077 4.673842 10.816842 7.611547 9.728249 11.310684 \n","\n"," 8 9 10 11 12 13 14 \\\n","1 0.0 5.355823 5.421817 7.121960 3.741157 8.306545 8.244712 \n","2 0.0 6.473565 5.572859 8.248769 8.508381 8.881842 1.431438 \n","3 0.0 7.154933 5.497584 7.284640 6.945042 8.366179 9.439101 \n","4 0.0 4.712311 4.412436 9.885013 1.557150 12.668872 2.272564 \n","5 0.0 8.121683 4.833846 9.151364 9.474709 4.286050 6.687075 \n","6 0.0 7.392459 5.189089 8.366626 7.614495 9.647287 9.981062 \n","7 0.0 7.518272 6.692183 6.970459 6.532276 2.435199 9.137291 \n","8 0.0 9.333015 7.130985 8.573651 11.011430 3.590932 5.327809 \n","9 0.0 6.761178 10.857974 6.806587 8.376683 5.429780 6.559338 \n","10 0.0 5.188200 4.032680 10.503522 6.017185 7.091296 7.291136 \n","\n"," 15 16 17 18 19 20 21 \\\n","1 2.877028 10.311895 8.179958 10.165046 13.461031 13.158999 7.192036 \n","2 3.410547 9.397995 3.482427 8.420702 5.393271 6.946743 9.387028 \n","3 1.833269 3.414336 5.437970 8.786960 5.854282 5.177309 7.625758 \n","4 4.176289 4.620686 5.645949 3.026473 6.294371 6.476227 2.580101 \n","5 3.824684 7.637696 6.466880 6.159756 12.333704 8.047175 2.431629 \n","6 1.915986 4.750871 6.292293 8.512650 5.737934 7.309226 7.198164 \n","7 2.763191 8.137121 5.367636 10.112645 12.117579 10.008593 5.925148 \n","8 4.420617 5.452084 5.874589 5.656380 5.021847 6.867363 4.211428 \n","9 2.880628 8.524738 4.708539 1.849900 10.488544 10.122972 6.840251 \n","10 3.912405 10.516394 6.489629 6.319449 9.714610 8.664498 5.690610 \n","\n"," 22 23 24 25 26 27 \\\n","1 9.687943 6.212390 3.338272 2.748304 5.864475 2.862230 \n","2 7.663156 3.282991 9.586357 15.377718 6.722057 3.138158 \n","3 2.596941 2.723114 9.951284 5.981502 7.398530 4.531449 \n","4 10.317554 4.165470 5.741220 12.047889 4.518391 6.366694 \n","5 8.692596 10.133836 6.700254 7.479168 4.362424 7.323874 \n","6 2.709246 2.488955 9.011206 6.656059 7.669699 5.121994 \n","7 7.131245 8.167489 2.519396 8.880021 10.766303 8.961728 \n","8 8.219995 5.064035 6.333735 6.605896 5.891028 5.017375 \n","9 4.023656 9.152832 10.113943 8.273561 8.516573 8.530090 \n","10 4.644862 6.578708 3.079436 6.254774 7.401401 12.597408 \n","\n"," 28 29 30 31 32 \n","1 7.317737 7.140820 7.512661 5.922181 5.122150 \n","2 6.378439 3.609798 5.979348 6.257098 9.722039 \n","3 4.335886 6.229736 1.524317 7.226295 6.453763 \n","4 5.598112 5.717739 9.456686 6.985310 6.148964 \n","5 3.331918 7.496245 5.976396 4.513660 11.470777 \n","6 5.628845 5.501085 2.006464 7.673812 6.394334 \n","7 10.558158 5.660993 4.908805 5.137964 6.605459 \n","8 3.444867 11.468255 1.850151 4.360839 5.017229 \n","9 2.827288 8.505968 10.130569 0.000000 5.915468 \n","10 11.582781 6.450478 7.404104 9.899031 4.481121 "]},"metadata":{},"execution_count":34}]},{"cell_type":"markdown","metadata":{"id":"YiR2Fu5hzC-n"},"source":["Ας υπολογίσουμε την ομοιότητα συνημιτόνου για τις αναπαραστάσεις των 10 πρώτων ψηφίων του συνόλου ελέγχου."]},{"cell_type":"code","metadata":{"id":"cY7x2EUGzEib","colab":{"base_uri":"https://localhost:8080/","height":363},"executionInfo":{"status":"ok","timestamp":1642514745284,"user_tz":-120,"elapsed":246,"user":{"displayName":"Γιώργος Αλεξανδρίδης","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GgfDSCm-2HDq--FfHt6Ft2fto7zbGbX0DAJENzBlw=s64","userId":"16422904839632337525"}},"outputId":"79b07b14-e13b-4861-93d4-d3e41374157e"},"source":["pd.DataFrame(data=cosine_similarity(h_deep[0:10]), columns=range(1,11), index=range(1,11)).head(10)"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["\n","
\n","
\n","
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
12345678910
11.0000000.8536780.8689180.8700190.8956860.8810900.9006170.8825760.8435260.927728
20.8536781.0000000.8856280.8965990.8791630.8900570.8745510.8932650.8620630.852048
30.8689180.8856281.0000000.8299850.8567410.9936230.8641750.8772060.8669890.878957
40.8700190.8965990.8299851.0000000.8656960.8431100.8275540.8496760.8286480.892536
50.8956860.8791630.8567410.8656961.0000000.8589270.9326820.9227910.9106180.871316
60.8810900.8900570.9936230.8431100.8589271.0000000.8735270.8750470.8681340.897078
70.9006170.8745510.8641750.8275540.9326820.8735271.0000000.8806300.9010680.900415
80.8825760.8932650.8772060.8496760.9227910.8750470.8806301.0000000.8608590.871938
90.8435260.8620630.8669890.8286480.9106180.8681340.9010680.8608591.0000000.840241
100.9277280.8520480.8789570.8925360.8713160.8970780.9004150.8719380.8402411.000000
\n","
\n"," \n"," \n"," \n","\n"," \n","
\n","
\n"," "],"text/plain":[" 1 2 3 4 5 6 7 \\\n","1 1.000000 0.853678 0.868918 0.870019 0.895686 0.881090 0.900617 \n","2 0.853678 1.000000 0.885628 0.896599 0.879163 0.890057 0.874551 \n","3 0.868918 0.885628 1.000000 0.829985 0.856741 0.993623 0.864175 \n","4 0.870019 0.896599 0.829985 1.000000 0.865696 0.843110 0.827554 \n","5 0.895686 0.879163 0.856741 0.865696 1.000000 0.858927 0.932682 \n","6 0.881090 0.890057 0.993623 0.843110 0.858927 1.000000 0.873527 \n","7 0.900617 0.874551 0.864175 0.827554 0.932682 0.873527 1.000000 \n","8 0.882576 0.893265 0.877206 0.849676 0.922791 0.875047 0.880630 \n","9 0.843526 0.862063 0.866989 0.828648 0.910618 0.868134 0.901068 \n","10 0.927728 0.852048 0.878957 0.892536 0.871316 0.897078 0.900415 \n","\n"," 8 9 10 \n","1 0.882576 0.843526 0.927728 \n","2 0.893265 0.862063 0.852048 \n","3 0.877206 0.866989 0.878957 \n","4 0.849676 0.828648 0.892536 \n","5 0.922791 0.910618 0.871316 \n","6 0.875047 0.868134 0.897078 \n","7 0.880630 0.901068 0.900415 \n","8 1.000000 0.860859 0.871938 \n","9 0.860859 1.000000 0.840241 \n","10 0.871938 0.840241 1.000000 "]},"metadata":{},"execution_count":35}]},{"cell_type":"markdown","metadata":{"id":"D30ptkWQy8M0"},"source":["Παρατηρούμε ότι σε αυτή την περίπτωση τα 2 ζεύγη όμοιων ψηφίων (3ο,-6ο, 5ο-7ο) είναι τα πιο όμοια μεταξύ τους, ενώ δεν ισχύει το ίδιο για το (8ο - 10ο). Για να βελτιωθεί περαιτέρω η απόδοση του συγκεκριμένου αυτοκωδικοποιητή, πρέπει να εξετάσουμε πιο βαθιά δίκτυα."]},{"cell_type":"markdown","metadata":{"id":"Lg3RIRB2p4YJ"},"source":["## Επιπρόσθετο υλικό"]},{"cell_type":"markdown","metadata":{"id":"AxOTzELUw4ly"},"source":["1. https://www.tensorflow.org/tutorials/generative/autoencoder"]}]}