diff --git a/Docker/README.md b/Docker/README.md new file mode 100644 index 0000000..2bd5884 --- /dev/null +++ b/Docker/README.md @@ -0,0 +1,3 @@ +# Containers +### Dedicated vs VM vs Containers +![image](/Docker/images/ss1.png) diff --git a/Docker/images/.empty b/Docker/images/.empty new file mode 100644 index 0000000..8b13789 --- /dev/null +++ b/Docker/images/.empty @@ -0,0 +1 @@ + diff --git a/Docker/images/ss1.png b/Docker/images/ss1.png new file mode 100644 index 0000000..ae5e720 Binary files /dev/null and b/Docker/images/ss1.png differ diff --git a/Tensorflow/Tensorflow Crash Course/TF.ipynb b/Tensorflow/Tensorflow Crash Course/TF.ipynb new file mode 100644 index 0000000..239cc43 --- /dev/null +++ b/Tensorflow/Tensorflow Crash Course/TF.ipynb @@ -0,0 +1,2372 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Framing\n", + "\n", + "- Example is a particular instance of data, x\n", + "- Labeled example has {features, label}: (x, y)\n", + " Used to train the model\n", + "- Unlabeled example has {features, ?}: (x, ?)\n", + " Used for making predictions on new data\n", + "- Model maps examples to predicted labels: y'\n", + " Defined by internal parameters, which are learned\n", + "\n", + "### Regression vs Classification\n", + "- A regression model predicts continuous values. For example, regression models make predictions that answer questions like the following:\n", + " What is the value of a house in California?\n", + " What is the probability that a user will click on this ad?\n", + "\n", + "- A classification model predicts discrete values. For example, classification models make predictions that answer questions like the following:\n", + " Is a given email message spam or not spam?\n", + " Is this an image of a dog, a cat, or a hamster?\n", + " \n", + "### Training \n", + "Training a model simply means learning (determining) good values for all the weights and the bias from labeled examples. In supervised learning, a machine learning algorithm builds a model by examining many examples and attempting to find a model that minimizes loss; this process is called empirical risk minimization." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Loss\n", + "Loss is the penalty for a bad prediction. That is, loss is a number indicating how bad the model's prediction was on a single example. If the model's prediction is perfect, the loss is zero; otherwise, the loss is greater. The goal of training a model is to find a set of weights and biases that have low loss, on average, across all examples.\n", + "\n", + "#### Mean square error (MSE) \n", + "It is the average squared loss per example over the whole dataset. To calculate MSE, sum up all the squared losses for individual examples and then divide by the number of examples:\n", + "\n", + "$\\frac{1}{N} \\sum_{(x,y \\in D)} (y-prediction(x))^2$\n", + "\n", + "Although MSE is commonly-used in machine learning, it is neither the only practical loss function nor the best loss function for all circumstances." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Reducing Loss\n", + "### Gradient Descent \n", + "Convex problems have only one minimum; that is, only one place where the slope is exactly 0. That minimum is where the loss function converges.\n", + "\n", + "Calculating the loss function for every conceivable value of $w_1$ over the entire data set would be an inefficient way of finding the convergence point. Let's examine a better mechanism—very popular in machine learning—called gradient descent.\n", + "\n", + "The first stage in gradient descent is to pick a starting value (a starting point) for $w_1$. The starting point doesn't matter much; therefore, many algorithms simply set $w_1$ to 0 or pick a random value.\n", + "\n", + "The gradient always points in the direction of steepest increase in the loss function. The gradient descent algorithm takes a step in the direction of the negative gradient in order to reduce loss as quickly as possible. To determine the next point along the loss function curve, the gradient descent algorithm adds some fraction of the gradient's magnitude to the starting point. \n", + "\n", + "The gradient descent then repeats this process, edging ever closer to the minimum.\n", + "\n", + "### Learning Rate\n", + "Gradient descent algorithms multiply the gradient by a scalar known as the learning rate (also sometimes called step size) to determine the next point.\n", + "\n", + "Hyperparameters are the knobs that programmers tweak in machine learning algorithms. Most machine learning programmers spend a fair amount of time tuning the learning rate. If you pick a learning rate that is too small, learning will take too long. Conversely, if you specify a learning rate that is too large, the next point will perpetually bounce haphazardly across the bottom of the well like a quantum mechanics experiment gone horribly wrong\n", + "\n", + "There's a Goldilocks learning rate for every regression problem. The Goldilocks value is related to how flat the loss function is. If you know the gradient of the loss function is small then you can safely try a larger learning rate, which compensates for the small gradient and results in a larger step size. \n", + "\n", + "### Stochastic Gradient Descent\n", + "In gradient descent, a batch is the total number of examples you use to calculate the gradient in a single iteration. So far, we've assumed that the batch has been the entire data set. A very large batch may cause even a single iteration to take a very long time to compute.\n", + "\n", + "By choosing examples at random from our data set, we could estimate (albeit, noisily) a big average from a much smaller one.\n", + "\n", + "Stochastic gradient descent (SGD) takes this idea to the extreme--it uses only a single example (a batch size of 1) per iteration. Given enough iterations, SGD works but is very noisy. The term \"stochastic\" indicates that the one example comprising each batch is chosen at random.\n", + "\n", + "Mini-batch stochastic gradient descent (mini-batch SGD) is a compromise between full-batch iteration and SGD. A mini-batch is typically between 10 and 1,000 examples, chosen at random. Mini-batch SGD reduces the amount of noise in SGD but is still more efficient than full-batch." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Tensorflow\n", + "TensorFlow APIs are arranged hierarchically, with the high-level APIs built on the low-level APIs. Machine learning researchers use the low-level APIs to create and explore new machine learning algorithms. tf.keras is the TensorFlow variant of the open-source Keras API.\n", + "\n", + "![tf image](https://developers.google.com/machine-learning/crash-course/images/TFHierarchyNew.svg)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "1. [Numpy Tutorial](https://colab.research.google.com/github/google/eng-edu/blob/master/ml/cc/exercises/numpy_ultraquick_tutorial.ipynb?utm_source=mlcc&utm_campaign=colab-external&utm_medium=referral&utm_content=numpy_tf2-colab&hl=en) Colab exercise, which provides all the NumPy information you need for this course.\n", + "2. [Pandas Tutorial](https://colab.research.google.com/github/google/eng-edu/blob/master/ml/cc/exercises/pandas_dataframe_ultraquick_tutorial.ipynb?utm_source=mlcc&utm_campaign=colab-external&utm_medium=referral&utm_content=pandas_tf2-colab&hl=en) exercise, which provides all the pandas information you need for this course." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Simple Linear Regression with Synthetic Data" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "import tensorflow as tf\n", + "from matplotlib import pyplot as plt\n", + "import tensorflow as tf" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Defined create_model and train_model\n" + ] + } + ], + "source": [ + "def build_model(my_learning_rate):\n", + " \"\"\"Create and compile a simple linear regression model.\"\"\"\n", + " # Most simple tf.keras models are sequential. \n", + " # A sequential model contains one or more layers.\n", + " model = tf.keras.models.Sequential()\n", + "\n", + " # Describe the topography of the model.\n", + " # The topography of a simple linear regression model\n", + " # is a single node in a single layer. \n", + " model.add(tf.keras.layers.Dense(units=1, \n", + " input_shape=(1,)))\n", + "\n", + " # Compile the model topography into code that \n", + " # TensorFlow can efficiently execute. Configure \n", + " # training to minimize the model's mean squared error. \n", + " model.compile(optimizer=tf.keras.optimizers.RMSprop(lr=my_learning_rate),\n", + " loss=\"mean_squared_error\",\n", + " metrics=[tf.keras.metrics.RootMeanSquaredError()])\n", + "\n", + " return model \n", + "\n", + "\n", + "def train_model(model, feature, label, epochs, batch_size):\n", + " \"\"\"Train the model by feeding it data.\"\"\"\n", + "\n", + " # Feed the feature values and the label values to the \n", + " # model. The model will train for the specified number \n", + " # of epochs, gradually learning how the feature values\n", + " # relate to the label values. \n", + " history = model.fit(x=feature,\n", + " y=label,\n", + " batch_size=batch_size,\n", + " epochs=epochs)\n", + "\n", + " # Gather the trained model's weight and bias.\n", + " trained_weight = model.get_weights()[0]\n", + " trained_bias = model.get_weights()[1]\n", + "\n", + " # The list of epochs is stored separately from the \n", + " # rest of history.\n", + " epochs = history.epoch\n", + " \n", + " # Gather the history (a snapshot) of each epoch.\n", + " hist = pd.DataFrame(history.history)\n", + "\n", + " # Specifically gather the model's root mean \n", + " #squared error at each epoch. \n", + " rmse = hist[\"root_mean_squared_error\"]\n", + "\n", + " return trained_weight, trained_bias, epochs, rmse\n", + "\n", + "print(\"Defined create_model and train_model\")" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Defined the plot_the_model and plot_the_loss_curve functions.\n" + ] + } + ], + "source": [ + "#@title Define the plotting functions\n", + "def plot_the_model(trained_weight, trained_bias, feature, label):\n", + " \"\"\"Plot the trained model against the training feature and label.\"\"\"\n", + "\n", + " # Label the axes.\n", + " plt.xlabel(\"feature\")\n", + " plt.ylabel(\"label\")\n", + "\n", + " # Plot the feature values vs. label values.\n", + " plt.scatter(feature, label)\n", + "\n", + " # Create a red line representing the model. The red line starts\n", + " # at coordinates (x0, y0) and ends at coordinates (x1, y1).\n", + " x0 = 0\n", + " y0 = trained_bias\n", + " x1 = my_feature[-1]\n", + " y1 = trained_bias + (trained_weight * x1)\n", + " plt.plot([x0, x1], [y0, y1], c='r')\n", + "\n", + " # Render the scatter plot and the red line.\n", + " plt.show()\n", + "\n", + "def plot_the_loss_curve(epochs, rmse):\n", + " \"\"\"Plot the loss curve, which shows loss vs. epoch.\"\"\"\n", + "\n", + " plt.figure()\n", + " plt.xlabel(\"Epoch\")\n", + " plt.ylabel(\"Root Mean Squared Error\")\n", + "\n", + " plt.plot(epochs, rmse, label=\"Loss\")\n", + " plt.legend()\n", + " plt.ylim([rmse.min()*0.97, rmse.max()])\n", + " plt.show()\n", + "\n", + "print(\"Defined the plot_the_model and plot_the_loss_curve functions.\")" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "my_feature = ([1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0])\n", + "my_label = ([5.0, 8.8, 9.6, 14.2, 18.8, 19.5, 21.4, 26.8, 28.9, 32.0, 33.8, 38.2])" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 1/10\n", + "1/1 [==============================] - 0s 2ms/step - loss: 971.8752 - root_mean_squared_error: 31.1749\n", + "Epoch 2/10\n", + "1/1 [==============================] - 0s 1ms/step - loss: 955.6766 - root_mean_squared_error: 30.9140\n", + "Epoch 3/10\n", + "1/1 [==============================] - 0s 1ms/step - loss: 944.0568 - root_mean_squared_error: 30.7255\n", + "Epoch 4/10\n", + "1/1 [==============================] - 0s 10ms/step - loss: 934.4051 - root_mean_squared_error: 30.5680\n", + "Epoch 5/10\n", + "1/1 [==============================] - 0s 3ms/step - loss: 925.8943 - root_mean_squared_error: 30.4285\n", + "Epoch 6/10\n", + "1/1 [==============================] - 0s 8ms/step - loss: 918.1411 - root_mean_squared_error: 30.3008\n", + "Epoch 7/10\n", + "1/1 [==============================] - 0s 1ms/step - loss: 910.9312 - root_mean_squared_error: 30.1816\n", + "Epoch 8/10\n", + "1/1 [==============================] - 0s 1ms/step - loss: 904.1323 - root_mean_squared_error: 30.0688\n", + "Epoch 9/10\n", + "1/1 [==============================] - 0s 1ms/step - loss: 897.6557 - root_mean_squared_error: 29.9609\n", + "Epoch 10/10\n", + "1/1 [==============================] - 0s 2ms/step - loss: 891.4384 - root_mean_squared_error: 29.8570\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEICAYAAAC0+DhzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAYh0lEQVR4nO3dfbRddZ3f8feXJJgYAiHmgSQQbsiCKMpIbAQUaxVMYWYooXSm1RmnactaWV3jjE9TR6hdq7PqsrLKzHTaOu0Ma7TSypI6yJMoRsyAzyDBKAEx8gwJkUQwiBg1Cd/+sfede+7d59yc3Nxz9nl4v9ba69699zn3/LZc7ye/32/v7y8yE0mSGh1VdwMkSb3HcJAkVRgOkqQKw0GSVGE4SJIqDAdJUkXt4RARMyJia0TcWu4viIjbI+Kh8uvxdbdRkoZN1P2cQ0S8H1gLHJuZF0XEfwGey8wrI+Jy4PjM/OBkP2PhwoU5MjLShdZK0uC49957f5yZi5qdm9ntxjSKiBOB3wQ+Ary/PLweeEv5/TXAncCk4TAyMsKWLVs600hJGlAR8USrc3UPK/0F8MfASw3HlmTmLoDy6+I6GiZJw6y2cIiIi4DdmXnvFN+/MSK2RMSWPXv2THPrJGm41dlzOBe4OCIeB64DzouITwHPRMRSgPLr7mZvzsyrM3NtZq5dtKjpkJkkaYpqC4fMvCIzT8zMEeDtwN9l5juBW4AN5cs2ADfX1ERJGlp1zzk0cyWwLiIeAtaV+5KkLqr1bqVRmXknxV1JZOazwPl1tkeSet1NW3dy1abtPL13H8vmz+EDF6zmkjXLp+3n90Q4SJLad9PWnVxxwzb27T8IwM69+7jihm0A0xYQvTisJEmaxFWbtv99MIzat/8gV23aPm2fYThIUp95eu++wzo+FYaDJPWZZfPnHNbxqTAcJKnPfOCC1cyZNWPcsTmzZvCBC1ZP22c4IS1JfWZ00tm7lSRJ41yyZvm0hsFEDitJkioMB0lSheEgSaowHCRJFYaDJKnCcJAkVRgOkqQKw0GSVGE4SJIqDAdJUoXlMyRpmnV6lbZuMBwkaRp1Y5W2bnBYSZKmUTdWaesGw0GSplE3VmnrBsNBkqZRN1Zp6wbDQZKmUTdWaesGJ6QlaRp1Y5W2bjAcJGmadXqVtm4wHCQNjUF4/qBbDAdJQ2FQnj/oFiekJQ2FQXn+oFsMB0lDYVCeP+gWw0HSUBiU5w+6xXCQNBQG5fmDbnFCWtJQGJTnD7qltnCIiNnAV4GXle24PjP/Y0QsAP4fMAI8DvzzzPxJXe2UNDgG4fmDbqlzWOmXwHmZ+VrgTODCiDgHuBzYnJmnApvLfUlSF9UWDln4Wbk7q9wSWA9cUx6/BrikhuZJ0lCrdUI6ImZExHeB3cDtmXk3sCQzdwGUXxfX2UZJGka1hkNmHszMM4ETgbMi4jXtvjciNkbElojYsmfPns41UpKGUE/crZSZeyPiTuBC4JmIWJqZuyJiKUWvotl7rgauBli7dm12rbGSOsK6R72ltp5DRCyKiPnl93OAtwE/AG4BNpQv2wDcXE8LJXXLaN2jnXv3kYzVPbpp6866mza06hxWWgrcERH3AfdQzDncClwJrIuIh4B15b6kAWbdo95T27BSZt4HrGly/Fng/O63SFJdrHvUeyyfIal21j3qPYaDpNpZ96j39MTdSpKGm3WPeo/hIKknWPeotzisJEmqsOcgaVI+nDacDAdJLY0+nDb6DMLow2mAATHgHFaS1JIPpw0vw0FSSz6cNrwMB0kt+XDa8DIcJLXkw2nDywlpSS35cNrwMhwkTcqH04aTw0qSpArDQZJUYThIkioMB0lSheEgSaowHCRJFYaDJKnCcJAkVRgOkqQKw0GSVGE4SJIqDAdJUoXhIEmqMBwkSRWW7Jb62E1bd7rWgjrCcJD61E1bd3LFDdvYt/8gADv37uOKG7YBGBA6Yg4rSX3qqk3b/z4YRu3bf5CrNm2vqUUaJIaD1Kee3rvvsI5Lh8NwkPrUsvlzDuu4dDhqC4eIOCki7oiIByPigYh4T3l8QUTcHhEPlV+Pr6uNUi/7wAWrmTNrxrhjc2bN4AMXrK6pRRokdfYcDgB/lJmvAs4B3hURpwOXA5sz81Rgc7kvaYJL1izno5eewfL5cwhg+fw5fPTSM5yM1rSo7W6lzNwF7Cq/fyEiHgSWA+uBt5Qvuwa4E/hgDU2Uet4la5YbBuqInriVNSJGgDXA3cCSMjjIzF0RsbjGpklT5jMI6me1h0NEHAN8FnhvZv40Itp930ZgI8CKFSs610BpCnwGQf2u1ruVImIWRTBcm5k3lIefiYil5fmlwO5m783MqzNzbWauXbRoUXcaLLXJZxDU72rrOUTRRfg48GBm/nnDqVuADcCV5deba2ieBlg3hnt8BkH9rs5hpXOB3wO2RcR3y2P/niIUPhMRlwFPAr9dU/s0gLo13LNs/hx2NgkCn0FQv6htWCkzv56ZkZm/lplnltsXMvPZzDw/M08tvz5XVxs1eLo13OMzCOp3tU9IS93UreGe0V6IdyupXxkOGirdHO7xGQT1M2sraag43CO1x56DhorDPVJ7DAcNHYd7pENzWEmSVGE4SJIqDAdJUoXhIEmqMBwkSRWT3q0UEZdOdr6hkqokaYAc6lbWfzLJuQQMB0kaQJOGQ2b+6241RJLUO9qac4iIJRHx8Yi4rdw/vSypLUkaQO1OSH8S2AQsK/d/CLy3Ew2SJNWv3XBYmJmfAV4CyMwDwMHJ3yJJ6lft1lZ6MSJeQTEJTUScAzzfsVZpaHVjCU9Jh9ZuOLyfYm3nVRHxDWAR8Fsda5WGUreW8JR0aG2FQ2Z+JyL+EbAaCGB7Zu7vaMs0dCZbwtNwkLqrrXCIiNnA7wNvohha+lpE/FVm/qKTjdNw6dYSnpIOrd0J6f8DvBr4H8DHgNOB/9upRmk4tVqqsxNLeEqaXLvhsDozL8vMO8ptI3BaJxum4eMSnlLvaDcctpZ3KAEQEWcD3+hMkzSsLlmznI9eegbL588hgOXz5/DRS89wvkGqwaEK722jmGOYBfzLiHiy3D8Z+H7nm6dh4xKeUm841IT0RV1phSSppxyq8N4TjfsRsRiY3dEWSZJq127hvYsj4iHgMeArwOPAbR1slySpRu1OSH8YOAf4YWauBM7HCWlJGljthsP+zHwWOCoijsrMO4AzO9guSVKN2q2ttDcijgG+ClwbEbuBA51rliSpTu32HNYD+4D3AV8EHmHyJUQlSX2s3cJ7LzbsXtOhtkiSesShHoJ7gXINh4mngMzMYzvSKklSrSYdVsrMeZl5bJNt3nQEQ0R8IiJ2R8T9DccWRMTtEfFQ+fX4I/0cSdLhaXfOoVM+CVw44djlwObMPBXYXO5Lkrqo3buVOiIzvxoRIxMOrwfeUn5/DXAn8MGuNUotuYSnNDxqDYcWlmTmLoDM3FWW7KiIiI3ARoAVK1Z0sXnDySU8peFS97DSlGXm1Zm5NjPXLlq0qO7mDLzJlvCUNHh6sefwTEQsLXsNS4HddTeo13VjuMclPKXh0os9h1uADeX3G4Cba2xLzxsd7tm5dx/J2HDPTVt3TuvnuISnNFxqDYeI+DTwLWB1ROyIiMuAK4F1ZRXYdeW+WujWcI9LeErDpe67ld7R4tT5XW1IH+vWcM/oMJV3K0nDoRfnHHQYls2fw84mQdCJ4R6X8JSGRy/OOegwONwjqRPsOfQ5h3skdYLhMAAc7pE03RxWkiRVGA6SpArDQZJUYThIkioMB0lSheEgSaowHCRJFYaDJKnCh+A6zKU1JfUjw6GDXFpTUr9yWKmDXFpTUr8yHDrIpTUl9SvDoYNcWlNSvzIcOsi1FiT1KyekO8i1FiT1K8Ohw1xrQVI/clhJklRhOEiSKgwHSVKF4SBJqjAcJEkVhoMkqcJwkCRVGA6SpArDQZJUYThIkioMB0lSheEgSaro2XCIiAsjYntEPBwRl9fdHkkaJj0ZDhExA/hL4NeB04F3RMTp0/5B+/bB3XfDj38MmdP+4yWpX/Vqye6zgIcz81GAiLgOWA98f1o/Zds2OOec4vtjj4VTToFVq8Zvp5wCJ50EM3v1fypJmn69+hdvOfBUw/4O4OzGF0TERmAjwIoVK6b2KaedBjffDI88Ao8+Wnzdtg1uuQX27x973cyZMDIyFhYTw2Pu3Kl9viT1qF4Nh2hybNy4T2ZeDVwNsHbt2qmNCc2fDxdfXD1+8CDs2DEWGI3bXXfB88+Pf/2SJc17HKtWweLFEM0uR5J6V6+Gww7gpIb9E4Gnu/bpM2bAyScX21vfWj3/3HNjYdEYIHfcAZ/61Pj5i2OOGQuKib2OFStg1qyuXZYktatXw+Ee4NSIWAnsBN4O/E69TWqwYEGxvf711XO/+AU8/ni1x/GDH8AXvgC//OXYa2fMKAKiWY9j1SqYN69rlyRJjXoyHDLzQET8AbAJmAF8IjMfqLlZ7Zk9G175ymKb6KWX4Omnm/c6/vZvix5Jo0WLmvc4Vq2CE05wuEpSx0QOwC2ca9euzS1bttTdjCO3d291nmN0/6mninAZNWdO88nxVauKyfOjj67tMiT1h4i4NzPXNjvXkz2HoTV/PrzudcU20a9+NTZcNTFAbr+9eGZj1FFHFbfftrq7av78rl2SpP5kOPSLo48ubr097bTquUz40Y+q8xyPPlrcqrtnz/jXL1jQep5j2bIiXCQNNcNhEETA0qXF9qY3Vc//9KdjvY3GXsfddxdzHQcPjr129mxYubL5PMfISHFe0sAzHIbBscfCmWcW20T798OTTzaf57jzTnjxxbHXRsDy5c17HKtWFT0SSQPBcBh2s2aN/XGfKBN2724+z/H5z8Mzz4x//fz5rUuQnHhiceuupL5gOKi1iOLp7yVL4I1vrJ5/8cXxoTH6/datcOONcODA2GuPPnqsBMnEXscppxR3X0nqGYaDpm7uXDjjjGKb6MCB4vbbZiVIvv51eOGF8a9ftqz1rbkLF/pMh9RlPueg7suEZ59tPs/xyCPFg4KN5s1rPc9hxVxpynzOQb0lougNLFwIZ59dPf/zn8Njj1V7HfffD5/7XPHMx6iZM4saWK0KH1oxV5oSw0G95+Uvh1e/utgmOngQdu5s3uP49reLp8wbNVbMndjrsGKu1JLDShoszz3XfJ7j0UeLMuyNv+9z57ae5zj5ZCvmauA5rKThMVoxd22T3/eJFXNHQ2T7drjtttYVc5vdnmvFXA04w0HD41AVc3ftat7j+Oxniwn0RgsXtp7nWLrU4Sr1PcNBgqKe1PLlxfbmN1fPP/9883mOb3wDrruuecXcViVIrJirPmA4SO047rjJK+Y+8UTzXseXvzy+Ym7EWMXcZhPlVsxVjzAcpCN19NFw6qnFNlFjxdyJE+WTVcxt1uuwYq66yHCQOulQFXNfeKH5Ak/33APXXz++Yu7LXlZUzG3W61i50oq5mlaGg1SnefPgta8ttokaK+ZODJDJKuY2uz13wQInyXVYDAepVx2qYu6ePc1LkNx2WzGU1ei441rPc1gxV00YDlI/iiie8F68GN7whur50Yq5E3sczSrmzprVeoGnlSuLJ9Y1dAwHaRAdqmLujh3Nex3f/GaxcmCjpUtb9zqsmDuwLJ8haUxjxdxmE+U7d45//bx5rRd4WrHCirk9zvIZktpzqIq5+/YVFXMn9jgeeABuvXXyirkTF3g65pjuXZcOm+EgqX1z5sDppxfbRKMVc5sVPrznHvjJT8a/fsmS1oUPlyxxuKpmDitJ6o6f/KT1Ak+TVcydGCBWzJ02DitJqt/xxxfVcltVzG1WguSHP4QvfrE4P+qoo8Yq5jbrdRx7bPeuaYAZDpLqN3s2rF5dbBNNrJjb2OM4VMXcib2OE06wBEmbDAdJva2dirnN7qz65jebV8xtVYJkZKQoUSLAcJDU7447DtasKbaJGivmTgyQzZuL9cpHNVbMbXZ77pBVzDUcJA2uQ1XMfeaZ5qXWP/c52L17/OuPP771PMfy5QM3XGU4SBpOEcUcxAknwLnnVs83Vsxt7HUcqmLuxF5Hn1bMNRwkqZnJKuYeODBWMXfiRPlXvgI/+9n4149WzG02Ud6jFXNrCYeI+G3gT4BXAWdl5paGc1cAlwEHgXdn5qY62ihJLc2cObYU7Lp14881VsydOM8xWcXcZvMcNVbMravncD9wKfDXjQcj4nTg7cCrgWXAlyPitMw8WP0RktSD2qmY21iCZDREvve9YnXA/fvHXjtrVnEXVbNexymndLRibi3hkJkPAkS1K7UeuC4zfwk8FhEPA2cB3+puCyWpQ+bOhde8ptgmOngQnnqqeQmSVhVz3/EO+LM/m/Zm9tqcw3Lgrob9HeWxiojYCGwEWLFiRedbJkmdNmNG0VMYGYHzzht/LhOee67a4zjppI40pWPhEBFfBk5ocupDmXlzq7c1Oda0+FNmXg1cDUVtpSk1UpL6RQS84hXFdtZZHf+4joVDZr5tCm/bATTG4InA09PTIklSu3rtqY1bgLdHxMsiYiVwKvDtmtskSUOnlnCIiH8aETuANwCfj4hNAJn5APAZ4PvAF4F3eaeSJHVfXXcr3Qjc2OLcR4CPdLdFkqRGvTasJEnqAYaDJKnCcJAkVRgOkqSKyOz/58ciYg/wxBH8iIXAj6epOXUalOsAr6UXDcp1gNcy6uTMXNTsxECEw5GKiC2Z2WTV8/4yKNcBXksvGpTrAK+lHQ4rSZIqDAdJUoXhULi67gZMk0G5DvBaetGgXAd4LYfknIMkqcKegySpYqjDISIujIjtEfFwRFxed3umKiJOiog7IuLBiHggIt5Td5uORETMiIitEXFr3W05EhExPyKuj4gflP9tmqwZ2R8i4n3l79b9EfHpiJhdd5vaFRGfiIjdEXF/w7EFEXF7RDxUfj2+zja2q8W1XFX+jt0XETdGxPzp+KyhDYeImAH8JfDrwOnAO8o1rPvRAeCPMvNVwDnAu/r4WgDeAzxYdyOmwX8DvpiZrwReS59eU0QsB94NrM3M1wAzKNZ67xefBC6ccOxyYHNmngpsLvf7wSepXsvtwGsy89eAHwJXTMcHDW04UKxN/XBmPpqZvwKuo1jDuu9k5q7M/E75/QsUf4SaLq/a6yLiROA3gb+puy1HIiKOBd4MfBwgM3+VmXvrbdURmQnMiYiZwMvpo0W4MvOrwHMTDq8Hrim/vwa4pKuNmqJm15KZX8rMA+XuXRSLpB2xYQ6H5cBTDfst16vuJxExAqwB7q63JVP2F8AfAy/V3ZAjdAqwB/jf5RDZ30TE3LobNRWZuRP4U+BJYBfwfGZ+qd5WHbElmbkLin9cAYtrbs90+TfAbdPxg4Y5HNper7pfRMQxwGeB92bmT+tuz+GKiIuA3Zl5b91tmQYzgdcB/ysz1wAv0j9DF+OU4/HrgZXAMmBuRLyz3lZpooj4EMUQ87XT8fOGORwGar3qiJhFEQzXZuYNdbdnis4FLo6IxymG+c6LiE/V26Qp2wHsyMzRHtz1FGHRj94GPJaZezJzP3AD8Maa23SknomIpQDl1901t+eIRMQG4CLgd3Oank8Y5nC4Bzg1IlZGxNEUE2y31NymKYmIoBjbfjAz/7zu9kxVZl6RmSdm5gjFf4+/y8y+/BdqZv4IeCoiVpeHzqdY/rYfPQmcExEvL3/XzqdPJ9cb3AJsKL/fANxcY1uOSERcCHwQuDgzfz5dP3dow6GcwPkDYBPFL/pnyjWs+9G5wO9R/Ev7u+X2G3U3SvwhcG1E3AecCfznmtszJWXv53rgO8A2ir8bffOEcUR8GvgWsDoidkTEZcCVwLqIeAhYV+73vBbX8jFgHnB7+f/9v5qWz/IJaUnSREPbc5AktWY4SJIqDAdJUoXhIEmqMBwkSRWGgzSJiHh3WVH1sJ46jYiRiPidTrVL6jTDQZrc7wO/kZm/e5jvGwEOOxzKasFS7QwHqYXyYaJTgFsi4kNlLf17ykJ668vXjETE1yLiO+U2WlbiSuAflg8lvS8i/lVEfKzhZ98aEW8pv/9ZRPyniLgbeENEvDMivl2+968NDNXBcJBayMx/S1Fv663AXIpyHq8v968qq6zuBtZl5uuAfwH89/LtlwNfy8wzM/O/HuKj5gL3Z+bZwLPlzzk3M88EDgKH22uRjtjMuhsg9Yl/TFEU8N+V+7OBFRTh8bGIGP1DftoUfvZBiqKJUNQt+gfAPUUZI+bQ50Xh1J8MB6k9AfyzzNw+7mDEnwDPUKz0dhTwixbvP8D4nnrjMpu/yMyDDZ9zTWZOy2pe0lQ5rCS1ZxPwh2VVUiJiTXn8OGBXZr5EUfxwdH7gBYpiaKMeB86MiKMi4iSKlQib2Qz8VkQsLj9nQUScPK1XIrXBcJDa82FgFnBfubj7h8vj/xPYEBF3UQwpvVgevw84EBHfi4j3Ad8AHqOoavqnFBVOKzLz+8B/AL5UVnO9HVjamUuSWrMqqySpwp6DJKnCcJAkVRgOkqQKw0GSVGE4SJIqDAdJUoXhIEmqMBwkSRX/H+52lI1aWzbEAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEGCAYAAABy53LJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxV9bnv8c+TkDDPJCEhRGYkDGGIQEXFAalQcShabbXH1p7S9ljr0Nvq8fZ0OOf2XI+1ttV6apXWiy22DsgRrVVQUaQqCMgcUGYCJGEKEOYkz/1jLTBihr1JdnaG7/v12q+s9Vv7t/az81Ke/NZvMndHREQkUgnxDkBERBoXJQ4REYmKEoeIiERFiUNERKKixCEiIlFpEe8A6kO3bt28V69eUdfL33+U/UdO0LVtMukdW2NW97GJiDRUS5cu3ePuKWeWN4vE0atXL5YsWRJ1vROl5Tzw6jqmL9xMZmZHfvuVkfTs0iYGEYqINDxmtrWycj2qqkZyiwR+dGU2j908ik17DvOFh99h3trCeIclIhJXShwRuGJId/52+4VkdW3DN59awv99JY+TZeXxDktEJC6UOCKU1bUNz3/7fL469hx+v2ATNz7+PrsOHI13WCIi9c6aw5Ijubm5fjZ9HFWZs2In/zprJS2TEvnVDcMZP+AzfUci0kScPHmS/Px8jh07Fu9QYqZVq1ZkZmaSlJT0qXIzW+ruuWe+v1l0jte1q3IyGJzRgdtmLuNrTy7mu5f0484JA0hM0LArkaYmPz+f9u3b06tXL6wJDq10d/bu3Ut+fj69e/eOqI4eVZ2lvintmP0v47h+VCaPvLmBm6cvouhQ0/2LRKS5OnbsGF27dm2SSQPAzOjatWtULSoljlponZzIA9fl8IvrhvHh9v184eGFvLdxb7zDEpE61lSTxinRfj8ljjpwfW5PXrztAtq3asFN09/n0fkbKC9v+n1HItI8KXHUkYHd2zPnuxdw5bAMfvHaer7+/z5g3+ET8Q5LRJqAdu3axTuET1HiqEPtWrbgNzcO5+fXDuG9jXv5wsPvsHTrvniHJSJSp5Q46piZcdOYc3jhX84nKTGBG37/Pk8s2ERzGPYsIvVn+fLljB07lmHDhnHttdeyf/9+AB5++GGys7MZNmwYN954IwBvv/02w4cPZ/jw4YwYMYJDhw7V6rM1jyOGDh47yQ+fW8mrawq4PDuNB6/LoWObpJorikiDkZeXx6BBgwD42UtrWLvzYJ3ePzujAz+ZMrja97Rr146SkpJPlQ0bNoxHHnmE8ePH8+Mf/5iDBw/y61//moyMDDZv3kzLli0pLi6mU6dOTJkyhXvvvZdx48ZRUlJCq1ataNHi07MxKn7PU6qaxxGzFoeZtTKzxWa2wszWmNnPwvLrw/NyM/tMQBXqX2Fm681sg5ndW6G8i5nNM7OPw5+dY/UdaqtDqyR+d/NIfnxlNvPXFXHlb99hZX5xvMMSkUbuwIEDFBcXM378eABuueUWFixYAAQJ5aabbuLPf/7z6eQwbtw47r77bh5++GGKi4s/kzSi5u4xeQEGtAuPk4BFwFhgEDAQeAvIraJuIrAR6AMkAyuA7PDaA8C94fG9wH/VFMuoUaM83pZu3eef+8/Xvf99r/iMdzd7eXl5vEMSkQisXbs23iF427ZtP3VeXFzsPXv2PH2+YcMGHzFihLu7l5aW+ptvvul33nmnDxgwwE+ePOnu7itXrvT777/fe/To4Xl5eZ/5jMq+J7DEK/k3NWYtjvBzT7WtksKXu3ueu6+vofpoYIO7b3L3E8BfgavDa1cDM8LjGcA1dRx6TIzM6szfvnch4/p15ccvruG7f/mQQ8dOxjssEWmEOnbsSOfOnXnnnXcA+NOf/sT48eMpLy9n+/btXHLJJTzwwAMUFxdTUlLCxo0bGTp0KPfccw+5ubmsW7euVp8f0yVHzCwRWAr0Ax5190URVu0BbK9wng+MCY/T3H0XgLvvMrPUuoo31jq3TeYPt5zH7xds4sG561m78yCPfmUk2Rkd4h2aiDRgR44cITMz8/T53XffzYwZM/j2t7/NkSNH6NOnD08++SRlZWXcfPPNHDhwAHfnrrvuolOnTvzbv/0b8+fPJzExkezsbCZNmlSreGKaONy9DBhuZp2A2WY2xN1XR1C1smmMUfXim9k0YBpAVlZWNFVjKiHB+M7FfRmZ1Ynb//Ih1/73P/j3qwfzpdyeTX52qoicnfLyyrdxeP/99z9TtnDhws+UPfLII3UaT70Mx3X3YoI+jSsirJIP9KxwngnsDI8LzSwdIPxZVMVnPu7uue6em5LS8FavHdOnK6/ccSHn9erCPbNW8b+eW8mRE6XxDktEpEaxHFWVErY0MLPWwAQg0gdrHwD9zay3mSUDNwJzwmtzgFvC41uAF+su6vrVrV1LZtw6mjsn9OeFD/O55tF/sKGoduOrRURiLZYtjnRgvpmtJEgE89z9ZTO71szygc8BfzOz1wDMLMPMXgFw91Lgu8BrQB7wrLuvCe97P3C5mX0MXB6eN1qJCcadEwbw1K2j2Vtygqt++w/+58Md8Q5LRCrwJj7fLdrvpwmADUjhwWPc/vSHLN6yjy+PzuInU7JplZQY77BEmrXNmzfTvn37Jru0uof7cRw6dOgz+3FoI6dGIK1DK57+5hh+Oe8jfvfWRlZsL+a/bxpJr25t4x2aSLOVmZlJfn4+u3fvjncoMXNqB8BIqcXRQL25rpC7nllBWbnzi+uGMWloerxDEpFmpt6XHJHaufTcNP72vQvol9qO78xcxr2zVmqHQRFpEJQ4GrDMzm149lufY9pFfXh+aT4X/+ItfvP6xxw+rmG7IhI/ShwNXHKLBO6bPIh5d49n/IAUfvX6R1z84Fs8vWgbpWWVTwoSEYklJY5Gone3tvzu5lHM+s75nNOlDffNXsXnf72AeWsLm/xQQRFpWJQ4GplR53TmuW9/jt9/dRTu8M2nlnDD4++zfLuWaxeR+qHE0QiZGZ8f3J3X7rqI/7hmCJt2l3DNo//gtqeXsXXv4XiHJyJNnIbjNgElx0t5fMEmnliwidLycm4acw7fu6w/Xdomxzs0EWnEqhqOq8TRhBQdPMavXv+YZz7YRtvkFnznkr7cOq63Zp+LyFnRPI5mILVDK/7vF4fy2p0XMaZPFx54dT2XPPgWzy3ZTll50/8DQUTqhxJHE9Q/rT3TbzmPv04bS2r7lvzg+ZV84eF3eGt9kUZgiUitKXE0YWP7dOV/bhvHb78ygiMnyvjakx9w8x8WsXrHgXiHJiKNmBJHE2dmXDksg9fvHs9PpmSzdudBrnxkIXc9s5z8/UfiHZ6INELqHG9mDhw9yWNvb+SPCzfjDl8b14vbLu5HxzZJ8Q5NRBoYjapS4viUncVHeWjeR8xalk+HVkl895J+/NP559CyhUZgiUig3kdVmVkrM1tsZivMbI2Z/Sws72Jm88zs4/Bn50rqDjSz5RVeB83szvDaT81sR4Vrk2P1HZqyjE6tefD6HF753oUM79mJn7+Sx2W/fJsXl++gXCOwRKQaMWtxWLBVVlt3LzGzJGAhcAfwRWCfu99vZvcCnd39nmrukwjsAMa4+1Yz+ylQ4u4PRhqLWhw1W/jxHv7zlTzW7jrIkB4duG/SIM7v1y3eYYlIHNV7i8MDJeFpUvhy4GpgRlg+A7imhltdBmx0960xCVQAuKB/N16+/QJ+dUMO+w+f5CvTF/G1JxezruBgvEMTkQYmpqOqzCzRzJYDRcA8d18EpLn7LoDwZ2oNt7kR+MsZZd81s5Vm9sfKHnWFnz3NzJaY2ZKmvOVjXUpIMK4dkckb3x/PfZPPZdnW/Uz+zTv88PkVFBzQJlIiEqiXznEz6wTMBm4HFrp7pwrX9rt7Vf/4JwM7gcHuXhiWpQF7CFov/wGku/ut1X2+HlWdnf2HT/Do/A089d5WEhLgGxf0ZtpFfenYWiOwRJqDuC454u7FwFvAFUChmaWHQaUTtEaqMglYdipphPcqdPcydy8HngBGxyzwZq5z22R+dGU2b3x/PJ8f3J1H529k3P1v8n9eXsvO4qPxDk9E4iSWo6pSwpYGZtYamACsA+YAt4RvuwV4sZrbfJkzHlOdSjqha4HVdRWzVK5nlzb85sYR/O17F3Dpuak8+e4WLnpgPnc/s5y8XeoDEWluYjmqahhB53ciQYJ61t3/3cy6As8CWcA24Hp332dmGcB0d58c1m8DbAf6uPuBCvf9EzCc4FHVFuBbp/pMqqJHVXVr+74j/PEfm3nmg+0cOVHGRQNS+NZFfTi/b1eCwXQi0hRoAqASR50rPnKCmYu28eQ/trCn5DiDMzow7aI+fGFoOi0StZqNSGN3VokjnENxv7v/IJbBxZoSR2wdO1nG7A938MQ7m9i0+zA9OrXmGxf05obzetK2ZYt4hyciZ+msWxxm9iZwmTfipokSR/0oL3dezyvk8QWbWLJ1Px1bJ3Hz2CxuOb8Xqe1bxTs8EYlSbRLHL4H+wHPA6Q2t3f2Fug4yVpQ46t/Srft5fMFG5q4tJCkxgakje/DPF/ahb0q7eIcmIhGqTeJ4spJir2nuREOixBE/m3aXMH3hZp5fms/JsnImDErjWxf1IbdXl3iHJiI1UOe4Ekdc7Sk5zlPvbuGp97dSfOQkI7M6Me2ivlyenUZigkZiiTREtWlxZAKPAOMIhsAuBO5w9/xYBBoLShwNx5ETpTy3JJ/pCzexfd9Rendryz9f2JupIzNplaQl3UUaktokjnnA08CfwqKbgZvc/fI6jzJGlDgantKycl5dU8Dv397Eqh0H6No2mVvO78VXx55D57bJ8Q5PRKhd4lju7sNrKmvIlDgaLnfnvU17eXzBJt5av5vWSYnccF5PvnFBb3p2aRPv8ESataoSRySD7PeY2c18svTHl4G9dRmcNF9mxvl9u3F+326sLzjE4ws2MXPRVp56bwuTh6bzrYv6MjSzY7zDFJEKImlxZAG/BT5H0MfxLkEfR6PZH0Mtjsal4MAxnvzHZp5etI1Dx0v5XJ+uTBvfh4sHpGhJE5F6VJuZ4zPc/eZYBhdrShyN08FjJ/nr4m38ceEWCg4eY2Bae755UR+uyskguYWWNBGJtdr0cbwGTHH3E7EKLtaUOBq3E6XlzFmxkycWbGJ94SG6d2jFTWOyuOG8nqR20Ix0kVipTeL4PTCSYDn0ijPHH6rrIGNFiaNpcHfe+mg3f3hnMws37CExwbh8UBo3jc1iXN9uJGg+iEidqk3n+M7wlQC0r+vARCJlZlwyMJVLBqayec9h/rJ4G88t2c6rawo4p2sbvjI6i+tGZdK1Xct4hyrSpKmPQxq1YyfLeG1NATPf38biLftITkzgiiHduWlMFqN7d1FnukgtnFWLw93Lwp38kqPt4zCzVsACoGX4Oc+7+0/MrAvwDNCLYCOmL7n7/krqbwEOAWVA6angI60vzUOrpESuHt6Dq4f34OPCQ8xctI1Zy/KZs2In/VLb8ZXRWUwdmUnHNtonXaSuxKyPw4I/9dq6e4mZJREuVQJ8Edjn7veb2b1AZ3e/p5L6W4Bcd99zRvkDkdSvSC2O5uXoiTJeWrmTmYu2sWJ7MS1bJDAlJ4ObxmQxvGcntUJEIlTvfRzh/h0l4WlS+HLgauDisHwG8BZQ7T/8Z6htfWniWicn8qXcnnwptyerdxzg6cXbePHDHTy/NJ9B6R24aUwW14zoQTttMiVyVs5qdVwza+HupRG8LxFYCvQDHnX3e8ys2N07VXjPfnfvXEndzcB+gmTze3d/PCyPqH5FanFIyfFSXly+gz+/v428XQdpm5zI1SN68JXRWQzpoZnpIpWJejiumS109wvC4z+5+1crXFvm7iOj+PBOwGzgdmBhhIkjw913mlkqMA+43d0XRJF4pgHTALKyskZt3dpoJrpLDLk7y7cXM3PRNl5asZPjpeXk9OzETWOymDIsg9bJWqFX5JSqEkd102/bVjgecub9ovlwdy8meKR0BVBoZulhUOlAURV1doY/iwiSzujwUqT1H3f3XHfPTUlJiSZcacLMjBFZnXnw+hwW3zeBn0zJ5vDxUn74/EpG/+fr/HTOGj4qPBTvMEUatOoSh1dxXNn5Z4SjsTqFx62BCcA6gk72W8K33QK8WEndtmbW/tQxMBFYHV6usb5IJDq2SeLr43oz766LeGbaWC49N5WnF21j4q8W8KXH3uPF5Ts4XloW7zBFGpzqegc7mdm1BMmlk5l9MSw3IJKHwunAjLCfIwF41t1fNrP3gGfN7BvANuB6CB5NAdPdfTKQBswOR7+0AJ5291fD+95fWX2Rs2VmjOnTlTF9uvLjK4/z/NJ8nl68jTv+upwubZO5blQmXx6dRe9ubWu+mUgzUF0fR2V7jZ/m7l+PSUQxoM5xiVZ5ufOPjXt4etE25q4tpKzcuaBfN74yJovLs9NIStQii9L0ac9xJQ45S4UHj/HsB9v56wfb2VF8lJT2Lbkhtyc3ju5JZmdtNiVNlxKHEofUUlm58/ZHRcx8fxvz1xfhwPl9uzJ1ZCZXDOlOm2TNC5GmRYlDiUPq0I7iozz7wXZe+DCf7fuO0jY5kUlD05k6MpMxvbtopV5pEpQ4lDgkBsrLnQ+27GPWsnxeWVVAyfFSMju35osjevDFkZn0Uoe6NGJnMwHwi5VeCLn7C3UUW8wpcUh9OHqijLlrC3h+aT4LN+zBHUad05mpIzP5wrB0OrbWQovSuJxN4jg1qioVOB94Mzy/BHjL3atNLA2JEofUt4IDx5j94Q5mLctnQ1EJLVskcHl2GlNHZXJhv2600KgsaQRqswPgy8A33X1XeJ5OsO6UEodIDdydlfkHTi/1XnzkJKntW3LNiB5MHZnJwO7aG00artokjtXuPqTCeQKwsmJZQ6fEIQ3B8dIy5q8r4vmlO3hrfRGl5c6QHh2YOjKTq3IytHOhNDi1SRy/BfoDfyFYauRGYIO73x6LQGNBiUMamr0lx5mzYiezluWzesdBWiQYl5ybytSRmVx6birJLfQoS+KvVqOqwqVHLgpPF7j77DqOL6aUOKQhW19wiFnL8pn94Q52HzpO5zZJXJWTwdRRmQzt0VEbT0nc1DZxnAP0d/fXzawNkOjujWYJUSUOaQxKy8p5Z8MeZi3NZ+7aQk6UltM/tR1TR2VyzfAedO/YKt4hSjNTm0dV3yTY16KLu/c1s/7AY+5+WWxCrXtKHNLYHDh6kr+t3MWsZfks3bqfBINx/bpx3ahMJmZ3174hUi9qkziWE+yFscjdR4Rlq9x9aEwijQElDmnMtuw5zAvL8pm1bAc7io/SvmULJg9NZ+qoTM7r1VmPsiRmarPn+HF3P3HqP04za0EE+3GISN3o1a0td08cyJ0TBrBoczBL/aWVO3lmyXayurTh2hE9uHp4Bn1S2sU7VGkmImlxPAAUA/9EsPXrvwBr3f1/xz68uqEWhzQ1R06U8urqAmYty+fdjXtxh8EZHbgqJ4MrczLo0al1vEOUJqA2j6oM+GeCXfgMeI1gw6VG0+pQ4pCmrODAMf62ahdzVuxkxfZiAHLP6cxVwzOYPDSdbpofImfprBJHbSb7mVkrYAHQkuCR2PPu/hMz6wI8A/QCtgBfcvf9Z9TtCTwFdAfKgcfd/TfhtZ8C3wR2h2+/z91fqS4WJQ5pLrbuPczLK3cxZ/lO1hceOt2pPiUng88P7q71siQqtWlxzAT+1d23RfmBBrR19xIzSwIWAncAXwT2ufv9ZnYv0Nnd7zmjbjqQ7u7Lwr3HlwLXuPvaMHGUuPuDkcaixCHN0fqCQ7y0YidzVuxk274jJCcmMH5gClflZDBhUJpGZkmNatM5ng6sMbPFwOFThe5+VXWVwkdZJeFpUvhy4Grg4rB8BvAWcM8ZdXcBu8LjQ2aWB/QA1kYQr4gAA7u3Z2D3gXx/4gBW5B/gpRU7eXnlTuatLaRNciITBqVxVU4GFw1I0Ux1iUokLY7xlZW7+9s13twskaC10I9gYcR7zKzY3TtVeM9+d+9czT16ETzyGuLuB8MWx9eAg8AS4PtnPuoK600jmH9CVlbWqK1bt9YUrkiTV1buLN68j5dW7uTvq3ax/8hJOrRqwaQh6Vw1PIOxfbqSqE2oJBTXjZzMrBMwm2BU1sJIE4eZtQPeBn5+av8PM0sD9hC0Xv6D4JHWrdV9vh5ViXzWybJyFm7Yw0vLd/LamgIOnyijW7uWXDksnSk56YzM0hyR5u6sH1WZ2VjgEWAQkAwkAofdvUOkH+7uxWb2FnAFUGhm6e6+K+zLKKric5OAWcDMiptGuXthhfc8AbwcaRwi8omkxAQuGZjKJQNTOXYyWLl3zoqdPL14G//v3S306NSaKTkZTMlJJzu9g5KInBZJH8dvCVbEfQ7IJZjP0b+mSmaWApwMk0ZrYALwX8Ac4Bbg/vDni5XUNeAPQJ67P3TGtfRTe4MA1wKrI/gOIlKNVknBnumThqZz6NhJ5q0tZM6KnTzxziYee3sjfVPaclVOD6bkpGuioUTUx7HE3XPNbKW7DwvL3nX382uoN4yg8zsRSACedfd/N7OuwLNAFrANuN7d95lZBsH8kMlmdgHwDrCKYDguhMNuzexPwHCCR1VbgG9VSCSV0qMqkbOz7/AJ/r46GN67eMs+3GFIjw5MGZbBlJwMMjTRsEmrzXDcBQSthelAAcFop6+5e04sAo0FJQ6R2is4cIyXV+7kpRU7WZF/AIDzenVmSo4mGjZVtUkc5xD0QyQBdwEdgf929w2xCDQWlDhE6tbWvYdPzxH5qLCEBIPz+3Zj8tB0Jg5OUxJpIuI6qirelDhEYmd9wSHmrNjB31buYsveIyQYjOndlclDu/P5wd1J7aB9RBqr2rQ4NlPJarju3qfuwostJQ6R2HN31hUc4u+rdvG3VbvYuPswZsG6WZOGpHPFkO7qE2lkapM4ulY4bQVcT7Cp04/rNsTYUeIQqX8fFx7ilVUF/H31LtYVBBuGjsjqxOQwifTs0ibOEUpN6vRRlZktdPcL6iSyeqDEIRJfG3eX8OrqAl5ZtYs1Ow8CMCyzI5OGpDNpSHd6dWsb5wilMrVpcYyscJpAMJfjOxpVJSJnY+vew/x9dQF/X11wehn47PQOTB7anSuGpNMvVfNEGoraJI75FU5LCeZOPOju6+s0whhS4hBpmPL3H+HVMIks3RosOTcgrR2ThqQzeWg6A9LaacZ6HGlUlRKHSINWcOAYr67exSurC/ggnGzYJ6Utk4ekM2lody17Ege1aXHcXd31M5cEaYiUOEQal6JDx3htTSF/X7WL9zftpdzhnK5twpZId4b26KgkUg9qkzieBs4jWGMKYArBMufbAdz9Z3Ubat1T4hBpvPaWHGfu2kL+vrqAdzfsobTc6dGpNZOGdGfS0HRG9OxEgpaCj4naJI65wFR3PxSetweec/crYhJpDChxiDQNxUdOMC9MIu98vJuTZU73Dq24Ykh3Jg9NZ9Q5nbWfSB2qTeJYB+S4+/HwvCWwwt3PjUmkMaDEIdL0HDh6kjfXFfLKqgLe/mg3J0rLSWnfkonZaXx+cHfG9umqnQ1rqTZbx/4JWGxms8PzawhWvRURiZuOrZO4dkQm147IpOR4KW+uK+Lvq3bxwrIdzFy0jfYtW3DJualMHJzGxQNTadcykn/uJBIRjaoK53JcSLD0yDvu/mGsA6tLanGINB/HTpax8OM9zF1bwOt5Rew7fILkxATG9evKxMHdmTAojZT2WoQxElE/qjKzNgQbMZ0MzwcCk4GtFXfkawyUOESap7JyZ+nW/by2poC5awvYvu8oZjAqqzMTB6cxMVuz1qtzNoljAfANd//YzPoBi4GZQDbwgbvfG8uA65ISh4icWoRx7ppC5q4tOL30yYC0dkzMDlbyHdJDc0UqOpvEscrdh4bH/0GwsOFtZpYMLD11rZoPbEUwbLclQV/K8+7+EzPrAjwD9CKYhf4ld99fSf0rgN8Q7CA43d3vD8sjql+REoeInGn7viPMWxskkcWb91HukN6xFROz05g4uDuje3chKbF5d66fTeKouFXsP4BfuPv/hOcralqrKtw3vK27l5hZErAQuAP4IrDP3e83s3uBzu5+zxl1E4GPgMuBfOAD4MvuvtbMHqip/pmUOESkOvsOn+DNdUW8tqaABR/t5nhpOR1bJ3FZ2Ll+0YAU2iQ3v871sxlVtdLMHgR2AP2AueGNOkXygR5kpJLwNCl8OXA1cHFYPgN4CzjzH/7RwAZ33xR+5l/DemsjrC8iErEubZO5blQm143K5MiJUt75eA9z1xTyxrpCXvhwBy1bJHBh/25MHNydy85NpWsz3+GwusTxTYIWQi9gorsfCcuzgQcjuXnYclhKkHgedfdFZpbm7rsA3H2XmaVWUrUH4cz0UD4wJjyOpD5mNg2YBpCVlRVJuCIitEluwecHB30epWXlLN6yj7lrCpm3tpDX84pIMMjt1eX0fJHmuK9IvSxyGLZSZgO3AwvdvVOFa/vdvfMZ778e+Ly7/3N4/lVgtLvfbmbFNdU/kx5ViUhtuTtrdh5k7poC5q4tPL051aD0DmG/SFqTW4ixNhMAa83di83sLeAKoNDM0sPWQjpQVEmVfKBnhfNMYGd4HEl9EZE6ZWYM6dGRIT06cvfEgWzdezjoXF9TyMNvfsxv3viYHp1aM3Fw0BLJPaczLZpo53rMWhxmlkIwD6TYzFoT9JH8FzAe2Fuhc7uLu//wjLotCDrHLyPoY/kA+Iq7rzGzX9RU/0xqcYhILO0pOc4beUESeWfDHk6UltOpTRIXD0hhQnbQud6hVVK8w4xave/HYWbDCDqvEwl2DnzW3f893MP8WSAL2AZc7+77zCyDYNjt5LD+ZODXYf0/uvvPw/JK61cXixKHiNSXw8dLefuj3byeV8j8dUXsP3KSFgnGmD5dmDAojQmD0hpNv0htFjkcAPwAOIcKj7bc/dK6DjJWlDhEJB7Kyp1l2/bzel4hb+QVsaEoGGg6IK0dl4VJZHjPTg12Rd/aJI4VwGMEo6PKTpW7+9K6DjJWlDhEpCHYsufw6SSyeMs+ysqdrm2TueTcVCYMSuPC/t1o24AWYz1xaKEAAA/ESURBVKxN4ljq7qNiFlk9UOIQkYbmwJGTvPVREW/kFfHW+iIOHislOTGBz/XtyoRBqVw2KI2MTq3jGmNtEsdPCUYuzQaOnyqvqV+hIVHiEJGG7GRZOUu2BI+0Xs8rZOveYNpcdnoHJgxKZUJ2GkMyOtb7Toe1SRybKyl2d+9TV8HFmhKHiDQW7s7G3SW8nlfEG3mFLN26n3KH1PYtuWxQKpedm8a4ft1onZwY81jqfVRVQ6LEISKN1b7DJ5i/rog31hWy4KM9lBwvpVVSAhf068Zlg9K47NxUUju0isln1ypxmNkQgqVGTkfn7k/VaYQxpMQhIk3B8dIyFm3axxt5wfInO4qPApCT2fH0KK1B6e3rbPZ6bR5V/YRgUcFs4BVgEsGyIdfVSWT1QIlDRJoad2d94SFeD9fQWpFfjDtkdGwVJJHsNMb26ULLFmf/SKs2iWMVkAN86O45ZpZGMFFvyllHU8+UOESkqSs6dIz564p4Pa+IhR/v4ejJMtomJ/LoTSO5eGCla8HWqDZrVR1193IzKzWzDgQjrBpNx7iISHOQ2r4VN5yXxQ3nZXHsZBnvbtzD63lFDOzevs4/K5LEsSRc3fYJgkmAJQTbyIqISAPUKimRS89N49Jz02Jy/xoTh7v/S3j4mJm9CnRw95UxiUZERBq8Gtf8tcDNZvZjd98CFJvZ6NiHJiIiDVEki8X/N/A54Mvh+SHg0ZhFJCIiDVokfRxj3H2kmX0I4O77zSw5xnGJiEgDFUmL42S4d7jD6Q2aymMalYiINFiRJI6HCRY4TDWznwMLgf+sqZKZ9TSz+WaWZ2ZrzOyOsDzHzN4zs1Vm9lI4xPfMugPNbHmF10EzuzO89lMz21Hh2uSovrGIiNRKJKOqZprZUoJtXA24xt3zIrh3KfB9d19mZu2BpWY2D5gO/C93f9vMbiXYJOrfzvjM9cBwgLC1s4MgeZ3yK3d/MIIYRESkjlWZOMysS4XTIuAvFa/VtKy6u+8CdoXHh8wsD+gBDAQWhG+bB7zGGYnjDJcBG919a3WfJyIi9aO6FsceIJ+g5QBBa+MUJ4rZ42bWCxgBLAJWA1cBLwLXAz1rqH4jFZJW6Ltm9k/AEoJWzf5KPnMaMA0gKysr0lBFRKQG1fVxPALsB14FbgH6uHvv8BVN0mgHzALudPeDwK3AbeHjr/bAiWrqJhMkmecqFP8O6EvwKGsX8MvK6rr74+6e6+65KSkpkYYrIiI1qDJxuPsdBP84Pwd8FfjQzB4ws96R3tzMkgiSxkx3fyG87zp3nxhuR/sXYGM1t5gELHP3wgpxFbp7mbuXEyyDosmIIiL1qNpRVR6YD/wQeAz4OjAhkhtbsCD8H4A8d3+oQnlq+DMB+FF436p8mTMeU5lZeoXTawkefYmISD2prnO8LXA1cAOQArwAjHT37RHeexxBS2WVmS0Py+4D+pvZbeH5C8CT4edlECzXPjk8bwNcDnzrjPs+YGbDCfpZtlRyXUREYqjK/TjM7DDwMcFf/BsIJwCecurRU2Og/ThERKJ3NvtxPEeQLM4NXxU5QWtBRESamSoTh7t/rR7jEBGRRiKSJUdEREROU+IQEZGoRLKRU8tIykREpHmIpMXxXoRlIiLSDFQ3j6M7waKErc1sBJ+sVdUBaFMPsYmISANU3XDczwNfAzKBhyqUHyKYyCciIs1QdcNxZwAzzGyqu8+qx5hERKQBi6SP4w0ze8jMloSvX5pZx5hHJiIiDVIkieMPBI+nvhS+DhKuLyUiIs1PjVvHAn3dfWqF859VWLRQRESamUhaHEfN7IJTJ2Y2Djgau5BERKQhi6TF8R2CTvKOBENy9xHsCCgiIs1QjYnD3ZcDOWbWITw/GPOoRESkwYpkyZGOZvYQ8CbwpkZViYg0b5H0cfyRsxhVZWY9zWy+meWZ2RozuyMszzGz98xslZm9dKolU0n9LeF7lpvZkgrlXcxsnpl9HP7sHMkXFRGRuhFJ4ujr7j9x903h62dAnwjqlQLfd/dBwFjgNjPLBqYD97r7UGA28INq7nGJuw8/Yweqe4E33L0/8EZ4LiIi9SRmo6rcfZe7LwuPDwF5BGtfDQQWhG+bB0yt/A5VuhqYER7PAK6Jsr6IiNRCvYyqMrNewAhgEbAauAp4Ebge6FlFNQfmmpkDv3f3x8PyNHffBUFyMrPUKj5zGjANICsrK5pwRUSkGjW2ONx9ubvnAMOAocB54c+ImFk7YBZwZzgi61aCx1ZLgfbAiSqqjnP3kcCk8P0XRfqZYdyPu3uuu+empKREU1VERKpRZeIwsw5m9q9m9lszu5ygg/yfgA0EneQ1MrMkgqQx091fAHD3de4+0d1HAX8BNlZW1913hj+LCPpCRoeXCs0sPbx/OlAUSSwiIlI3qmtx/ImgP2IV8E1gLsGjpWvc/eqabmxmRrDOVZ67P1ShPDX8mQD8CHiskrptzaz9qWNgIsEjLoA5fPKo7BaCR14iIlJPquvj6BOOfMLMpgN7gKywozsS44CvAqsqrG11H9DfzG4Lz18gHNprZhnAdHefDKQBs4PcQwvgaXd/NaxzP/CsmX0D2EaQzEREpJ5UlzhOnjpw9zIz2xxF0sDdF/LJroFn+k0l798JTA6PNwE5Vdx3L3BZpHGIiEjdqi5x5JjZqeVFjGAL2YPhsbt7pRP3RESkaatuB8DE+gxEREQah0gmAIqIiJymxCEiIlFR4hARkagocYiISFSUOEREJCpKHCIiEhUlDhERiYoSh4iIREWJQ0REoqLEISIiUVHiEBGRqChxiIhIVJQ4REQkKjFLHGbW08zmm1mema0xszvC8hwze8/MVpnZS2b2meXZq6obXvupme0ws+Xha3KsvoOIiHxWLFscpcD33X0QMBa4zcyygenAveHugrOBH0RR95Rfufvw8PVKDL+DiIicIWaJw913ufuy8PgQkAf0INjHfEH4tnnA1CjqiohInNVLH4eZ9QJGAIuA1cBV4aXrgZ5R1D3lu2a20sz+aGadq6g3zcyWmNmS3bt31yp+ERH5RMwTh5m1A2YBd7r7QeBWgkdPS4H2wIko6gL8DugLDAd2Ab+srK67P+7uue6em5KSUmffR0Skuatuz/FaM7Mkgn/4Z7r7CwDuvg6YGF4fAHwh0rph/cIK73kCeDlmX0BERD4jlqOqDPgDkOfuD1UoTw1/JgA/Ah6LtG54Lb3C6bUEj75ERKSexPJR1Tjgq8ClZwyd/bKZfQSsA3YCTwKYWYaZvVJDXYAHwqG8K4FLgLti+B1EROQM5u7xjiHmcnNzfcmSJfEOQ0SkUTGzpe6ee2a5Zo6LiEhUlDhERCQqShwiIhIVJQ4REYmKEoeIiERFiUNERKKixCEiIlFR4hARkagocYiISFSUOEREJCpKHCIiEhUlDhERiYoSh4iIREWJQ0REoqLEISIiUVHiEBGRqMRy69ieZjbfzPLMbI2Z3RGW55jZe+Eufi+ZWYcq6l9hZuvNbIOZ3VuhvIuZzTOzj8OfnWP1HURE5LNi2eIoBb7v7oOAscBtZpYNTAfudfehwGzgB2dWNLNE4FFgEpBNsN1sdnj5XuANd+8PvBGei4hIPYlZ4nD3Xe6+LDw+BOQBPYCBwILwbfOAqZVUHw1scPdN7n4C+CtwdXjtamBGeDwDuCY230BERCpTL30cZtYLGAEsAlYDV4WXrgd6VlKlB7C9wnl+WAaQ5u67IEhOQGoVnznNzJaY2ZLdu3fX9iuIiEgo5onDzNoBs4A73f0gcCvBY6ulQHvgRGXVKinzaD7X3R9391x3z01JSYk2bBERqUKLWN7czJIIksZMd38BwN3XARPD6wOAL1RSNZ9Pt0QygZ3hcaGZpbv7LjNLB4piFb+IiHxWLEdVGfAHIM/dH6pQnhr+TAB+BDxWSfUPgP5m1tvMkoEbgTnhtTnALeHxLcCLsfkGIiJSGXOP6glQ5Dc2uwB4B1gFlIfF9wH9gdvC8xeAf3V3N7MMYLq7Tw7rTwZ+DSQCf3T3n4flXYFngSxgG3C9u++rIZbdwNaz/CrdgD1nWbcp0u/jE/pdfJp+H5/WFH4f57j7Z571xyxxNBVmtsTdc+MdR0Oh38cn9Lv4NP0+Pq0p/z40c1xERKKixCEiIlFR4qjZ4/EOoIHR7+MT+l18mn4fn9Zkfx/q4xARkaioxSEiIlFR4hARkagocVSjqqXdm5uqlshv7sws0cw+NLOX4x1LvJlZJzN73szWhf+dfC7eMcWLmd0V/n+y2sz+Ymat4h1TXVPiqEINS7s3N1Utkd/c3UGw6rPAb4BX3f1cIIdm+nsxsx7A94Bcdx9CMIH5xvhGVfeUOKpW3dLuzUo1S+Q3W2aWSbDO2vR4xxJv4WZsFxEsMYS7n3D34vhGFVctgNZm1gJowyfr7DUZShxVq25p92brjCXym7NfAz/kk+V0mrM+wG7gyfDR3XQzaxvvoOLB3XcADxIsh7QLOODuc+MbVd1T4qharZd2b2oqWSK/WTKzK4Eid18a71gaiBbASOB37j4COEwz3Zkz3Mr6aqA3kAG0NbOb4xtV3VPiqFp1S7s3O5Utkd+MjQOuMrMtBI8wLzWzP8c3pLjKB/Ld/VQr9HmCRNIcTQA2u/tudz9JsJDr+XGOqc4pcVStuqXdm5Wqlshvrtz9X9090917Efx38aa7N7m/KiPl7gXAdjMbGBZdBqyNY0jxtA0Ya2Ztwv9vLqMJDhSI6UZOjZm7l5rZd4HX+GRp9zVxDitexgFfBVaZ2fKw7D53fyWOMUnDcjswM/wjaxPw9TjHExfuvsjMngeWEYxG/JAmuPSIlhwREZGo6FGViIhERYlDRESiosQhIiJRUeIQEZGoKHGIiEhUlDhE6oCZlZnZ8gqvOps5bWa9zGx1Xd1PpLY0j0Okbhx19+HxDkKkPqjFIRJDZrbFzP7LzBaHr35h+Tlm9oaZrQx/ZoXlaWY228xWhK9Ty1UkmtkT4T4Pc82sddy+lDR7ShwidaP1GY+qbqhw7aC7jwZ+S7CqLuHxU+4+DJgJPByWPwy87e45BOs9nVqtoD/wqLsPBoqBqTH+PiJV0sxxkTpgZiXu3q6S8i3Ape6+KVwossDdu5rZHiDd3U+G5bvcvZuZ7QYy3f14hXv0Aua5e//w/B4gyd3/T+y/mchnqcUhEntexXFV76nM8QrHZah/UuJIiUMk9m6o8PO98PhdPtlS9CZgYXj8BvAdOL2neYf6ClIkUvqrRaRutK6wcjAE+2+fGpLb0swWEfyh9uWw7HvAH83sBwS7551aTfYO4HEz+wZBy+I7BDvJiTQY6uMQiaGwjyPX3ffEOxaRuqJHVSIiEhW1OEREJCpqcYiISFSUOEREJCpKHCIiEhUlDhERiYoSh4iIROX/A/j0cYPYoTavAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "learning_rate=0.01\n", + "epochs=10\n", + "my_batch_size=12\n", + "\n", + "my_model = build_model(learning_rate)\n", + "trained_weight, trained_bias, epochs, rmse = train_model(my_model, my_feature, \n", + " my_label, epochs,\n", + " my_batch_size)\n", + "plot_the_model(trained_weight, trained_bias, my_feature, my_label)\n", + "plot_the_loss_curve(epochs, rmse)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Linear Regression with Real Data" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [], + "source": [ + "#@title Import relevant modules\n", + "import pandas as pd\n", + "import tensorflow as tf\n", + "from matplotlib import pyplot as plt\n", + "\n", + "# The following lines adjust the granularity of reporting. \n", + "pd.options.display.max_rows = 10\n", + "pd.options.display.float_format = \"{:.1f}\".format" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
longitudelatitudehousing_median_agetotal_roomstotal_bedroomspopulationhouseholdsmedian_incomemedian_house_value
0-114.334.215.05612.01283.01015.0472.01.566.9
1-114.534.419.07650.01901.01129.0463.01.880.1
2-114.633.717.0720.0174.0333.0117.01.785.7
3-114.633.614.01501.0337.0515.0226.03.273.4
4-114.633.620.01454.0326.0624.0262.01.965.5
\n", + "
" + ], + "text/plain": [ + " longitude latitude housing_median_age total_rooms total_bedrooms \\\n", + "0 -114.3 34.2 15.0 5612.0 1283.0 \n", + "1 -114.5 34.4 19.0 7650.0 1901.0 \n", + "2 -114.6 33.7 17.0 720.0 174.0 \n", + "3 -114.6 33.6 14.0 1501.0 337.0 \n", + "4 -114.6 33.6 20.0 1454.0 326.0 \n", + "\n", + " population households median_income median_house_value \n", + "0 1015.0 472.0 1.5 66.9 \n", + "1 1129.0 463.0 1.8 80.1 \n", + "2 333.0 117.0 1.7 85.7 \n", + "3 515.0 226.0 3.2 73.4 \n", + "4 624.0 262.0 1.9 65.5 " + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "training_df = pd.read_csv(filepath_or_buffer=\"https://download.mlcc.google.com/mledu-datasets/california_housing_train.csv\")\n", + "\n", + "# Scale the label.\n", + "training_df[\"median_house_value\"] /= 1000.0\n", + "\n", + "# Print the first rows of the pandas DataFrame.\n", + "training_df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
longitudelatitudehousing_median_agetotal_roomstotal_bedroomspopulationhouseholdsmedian_incomemedian_house_value
count17000.017000.017000.017000.017000.017000.017000.017000.017000.0
mean-119.635.628.62643.7539.41429.6501.23.9207.3
std2.02.112.62179.9421.51147.9384.51.9116.0
min-124.332.51.02.01.03.01.00.515.0
25%-121.833.918.01462.0297.0790.0282.02.6119.4
50%-118.534.229.02127.0434.01167.0409.03.5180.4
75%-118.037.737.03151.2648.21721.0605.24.8265.0
max-114.342.052.037937.06445.035682.06082.015.0500.0
\n", + "
" + ], + "text/plain": [ + " longitude latitude housing_median_age total_rooms total_bedrooms \\\n", + "count 17000.0 17000.0 17000.0 17000.0 17000.0 \n", + "mean -119.6 35.6 28.6 2643.7 539.4 \n", + "std 2.0 2.1 12.6 2179.9 421.5 \n", + "min -124.3 32.5 1.0 2.0 1.0 \n", + "25% -121.8 33.9 18.0 1462.0 297.0 \n", + "50% -118.5 34.2 29.0 2127.0 434.0 \n", + "75% -118.0 37.7 37.0 3151.2 648.2 \n", + "max -114.3 42.0 52.0 37937.0 6445.0 \n", + "\n", + " population households median_income median_house_value \n", + "count 17000.0 17000.0 17000.0 17000.0 \n", + "mean 1429.6 501.2 3.9 207.3 \n", + "std 1147.9 384.5 1.9 116.0 \n", + "min 3.0 1.0 0.5 15.0 \n", + "25% 790.0 282.0 2.6 119.4 \n", + "50% 1167.0 409.0 3.5 180.4 \n", + "75% 1721.0 605.2 4.8 265.0 \n", + "max 35682.0 6082.0 15.0 500.0 " + ] + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "training_df.describe()" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Defined the create_model and traing_model functions.\n" + ] + } + ], + "source": [ + "#@title Define the functions that build and train a model\n", + "def build_model(my_learning_rate):\n", + " \"\"\"Create and compile a simple linear regression model.\"\"\"\n", + " # Most simple tf.keras models are sequential.\n", + " model = tf.keras.models.Sequential()\n", + "\n", + " # Describe the topography of the model.\n", + " # The topography of a simple linear regression model\n", + " # is a single node in a single layer.\n", + " model.add(tf.keras.layers.Dense(units=1, \n", + " input_shape=(1,)))\n", + "\n", + " # Compile the model topography into code that TensorFlow can efficiently\n", + " # execute. Configure training to minimize the model's mean squared error. \n", + " model.compile(optimizer=tf.keras.optimizers.RMSprop(lr=my_learning_rate),\n", + " loss=\"mean_squared_error\",\n", + " metrics=[tf.keras.metrics.RootMeanSquaredError()])\n", + "\n", + " return model \n", + "\n", + "\n", + "def train_model(model, df, feature, label, epochs, batch_size):\n", + " \"\"\"Train the model by feeding it data.\"\"\"\n", + "\n", + " # Feed the model the feature and the label.\n", + " # The model will train for the specified number of epochs. \n", + " history = model.fit(x=df[feature],\n", + " y=df[label],\n", + " batch_size=batch_size,\n", + " epochs=epochs)\n", + "\n", + " # Gather the trained model's weight and bias.\n", + " trained_weight = model.get_weights()[0]\n", + " trained_bias = model.get_weights()[1]\n", + "\n", + " # The list of epochs is stored separately from the rest of history.\n", + " epochs = history.epoch\n", + " \n", + " # Isolate the error for each epoch.\n", + " hist = pd.DataFrame(history.history)\n", + "\n", + " # To track the progression of training, we're going to take a snapshot\n", + " # of the model's root mean squared error at each epoch. \n", + " rmse = hist[\"root_mean_squared_error\"]\n", + "\n", + " return trained_weight, trained_bias, epochs, rmse\n", + "\n", + "print(\"Defined the create_model and traing_model functions.\")" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Defined the plot_the_model and plot_the_loss_curve functions.\n" + ] + } + ], + "source": [ + "#@title Define the plotting functions\n", + "def plot_the_model(trained_weight, trained_bias, feature, label):\n", + " \"\"\"Plot the trained model against 200 random training examples.\"\"\"\n", + "\n", + " # Label the axes.\n", + " plt.xlabel(feature)\n", + " plt.ylabel(label)\n", + "\n", + " # Create a scatter plot from 200 random points of the dataset.\n", + " random_examples = training_df.sample(n=200)\n", + " plt.scatter(random_examples[feature], random_examples[label])\n", + "\n", + " # Create a red line representing the model. The red line starts\n", + " # at coordinates (x0, y0) and ends at coordinates (x1, y1).\n", + " x0 = 0\n", + " y0 = trained_bias\n", + " x1 = 10000\n", + " y1 = trained_bias + (trained_weight * x1)\n", + " plt.plot([x0, x1], [y0, y1], c='r')\n", + "\n", + " # Render the scatter plot and the red line.\n", + " plt.show()\n", + "\n", + "\n", + "def plot_the_loss_curve(epochs, rmse):\n", + " \"\"\"Plot a curve of loss vs. epoch.\"\"\"\n", + "\n", + " plt.figure()\n", + " plt.xlabel(\"Epoch\")\n", + " plt.ylabel(\"Root Mean Squared Error\")\n", + "\n", + " plt.plot(epochs, rmse, label=\"Loss\")\n", + " plt.legend()\n", + " plt.ylim([rmse.min()*0.97, rmse.max()])\n", + " plt.show() \n", + "\n", + "print(\"Defined the plot_the_model and plot_the_loss_curve functions.\")" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "WARNING:tensorflow:Falling back from v2 loop because of error: Failed to find data adapter that can handle input: , \n", + "Train on 17000 samples\n", + "Epoch 1/30\n", + "17000/17000 [==============================] - 1s 37us/sample - loss: 4238976.7001 - root_mean_squared_error: 2058.8784\n", + "Epoch 2/30\n", + "17000/17000 [==============================] - 0s 28us/sample - loss: 27841.4275 - root_mean_squared_error: 166.8575\n", + "Epoch 3/30\n", + "17000/17000 [==============================] - 0s 29us/sample - loss: 27085.9191 - root_mean_squared_error: 164.5780\n", + "Epoch 4/30\n", + "17000/17000 [==============================] - 1s 44us/sample - loss: 26178.0268 - root_mean_squared_error: 161.7962\n", + "Epoch 5/30\n", + "17000/17000 [==============================] - 0s 29us/sample - loss: 25479.9707 - root_mean_squared_error: 159.6245\n", + "Epoch 6/30\n", + "17000/17000 [==============================] - 1s 40us/sample - loss: 25069.7450 - root_mean_squared_error: 158.3343\n", + "Epoch 7/30\n", + "17000/17000 [==============================] - 1s 30us/sample - loss: 24137.9567 - root_mean_squared_error: 155.3640\n", + "Epoch 8/30\n", + "17000/17000 [==============================] - 1s 31us/sample - loss: 23649.1483 - root_mean_squared_error: 153.7828\n", + "Epoch 9/30\n", + "17000/17000 [==============================] - 1s 30us/sample - loss: 23069.3258 - root_mean_squared_error: 151.8859\n", + "Epoch 10/30\n", + "17000/17000 [==============================] - 0s 29us/sample - loss: 22312.3578 - root_mean_squared_error: 149.3732\n", + "Epoch 11/30\n", + "17000/17000 [==============================] - 0s 28us/sample - loss: 21987.5527 - root_mean_squared_error: 148.2820\n", + "Epoch 12/30\n", + "17000/17000 [==============================] - 1s 30us/sample - loss: 21397.6616 - root_mean_squared_error: 146.2794\n", + "Epoch 13/30\n", + "17000/17000 [==============================] - 0s 28us/sample - loss: 20829.8738 - root_mean_squared_error: 144.3255\n", + "Epoch 14/30\n", + "17000/17000 [==============================] - 1s 31us/sample - loss: 20307.0534 - root_mean_squared_error: 142.5029\n", + "Epoch 15/30\n", + "17000/17000 [==============================] - 0s 29us/sample - loss: 19983.5275 - root_mean_squared_error: 141.3631\n", + "Epoch 16/30\n", + "17000/17000 [==============================] - 0s 29us/sample - loss: 19492.3026 - root_mean_squared_error: 139.6148\n", + "Epoch 17/30\n", + "17000/17000 [==============================] - 0s 29us/sample - loss: 19072.7507 - root_mean_squared_error: 138.1042\n", + "Epoch 18/30\n", + "17000/17000 [==============================] - 1s 29us/sample - loss: 18565.0596 - root_mean_squared_error: 136.2536\n", + "Epoch 19/30\n", + "17000/17000 [==============================] - 0s 28us/sample - loss: 18184.6238 - root_mean_squared_error: 134.8503\n", + "Epoch 20/30\n", + "17000/17000 [==============================] - 1s 30us/sample - loss: 17809.0409 - root_mean_squared_error: 133.4505\n", + "Epoch 21/30\n", + "17000/17000 [==============================] - 1s 30us/sample - loss: 17551.6697 - root_mean_squared_error: 132.4827\n", + "Epoch 22/30\n", + "17000/17000 [==============================] - 1s 31us/sample - loss: 17086.2889 - root_mean_squared_error: 130.7146\n", + "Epoch 23/30\n", + "17000/17000 [==============================] - 1s 30us/sample - loss: 16906.1929 - root_mean_squared_error: 130.0238\n", + "Epoch 24/30\n", + "17000/17000 [==============================] - 1s 31us/sample - loss: 16583.3215 - root_mean_squared_error: 128.7762\n", + "Epoch 25/30\n", + "17000/17000 [==============================] - 1s 31us/sample - loss: 16313.6547 - root_mean_squared_error: 127.7249\n", + "Epoch 26/30\n", + "17000/17000 [==============================] - 1s 31us/sample - loss: 16168.5576 - root_mean_squared_error: 127.1556\n", + "Epoch 27/30\n", + "17000/17000 [==============================] - 1s 33us/sample - loss: 15808.9559 - root_mean_squared_error: 125.7337\n", + "Epoch 28/30\n", + "17000/17000 [==============================] - 1s 33us/sample - loss: 15615.2206 - root_mean_squared_error: 124.9609\n", + "Epoch 29/30\n", + "17000/17000 [==============================] - 1s 35us/sample - loss: 15426.3568 - root_mean_squared_error: 124.2029\n", + "Epoch 30/30\n", + "17000/17000 [==============================] - 1s 32us/sample - loss: 15329.1498 - root_mean_squared_error: 123.8109\n", + "\n", + "The learned weight for your model is 0.0118\n", + "The learned bias for your model is 132.5255\n", + "\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEHCAYAAABBW1qbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2dfZxVdZ34358ZLjj4NKjk4giCz2koyKgY1iaW+IBKPtbWZtauPVib1pK47ib2q6TY1nVrq3WrzTYzTI0QatEEM80nEBBJSfAJBkJEBhUGmIfP749z7uXOnXPOPWfuOfecO/fzfr3mNfd+73n43Dtzv5/v9/MoqophGIZhADSkLYBhGIaRHUwpGIZhGAVMKRiGYRgFTCkYhmEYBUwpGIZhGAUGpS1ApRx00EE6evTotMUwDMOoKZYuXfq6qg4vHa95pTB69GiWLFmSthiGYRg1hYi84jVu5iPDMAyjgCkFwzAMo4ApBcMwDKOAKQXDMAyjgCkFwzAMo0Di0Uci8jLwFtANdKlqq4gcAMwBRgMvA5ep6lYREeBW4FxgB/BxVX06aRmTYu6yNmYvXM2G9g4OaW5i+pRjmDa+xXfci3+eu5I7n1hHtyqNInz41JF8bdrYUPepVM64zzEMI/tUKyT1DFV9vej5DOBBVZ0lIjPc59cB5wBHuT+nAt93f9ccc5e1cf29K+no7Aagrb2D6+9dyZJX3uCepW19xoE+k+o/z13Jzx5/tfC8W7XwPK8Y/O7jdb0ocgadX+k9DcPILmmZjy4Ebncf3w5MKxr/qTo8DjSLyIg0BKyU2QtXFybNPB2d3dz5xDrP8dkLV/e5xp1PrPO8dvG43328rhdFzqDzK72nYRjZpRpKQYH7RWSpiFzljh2sqhsB3N/vcMdbgOKZcL071gsRuUpElojIks2bNycoev/Z0N7hOd7t07/C63i/Y4vH/e7jNx72uKDzK72nYRjZpRpKYZKqnoRjGrpaRN4bcKx4jPWZGVX1NlVtVdXW4cP7ZGlngkOamzzHG8XrLUKDCHOXtYU6tnjc7z5+42GPCzq/0nsahpFdElcKqrrB/f0a8CvgFGBT3izk/n7NPXw9MLLo9EOBDUnLmATTpxxDU66x11hTrpEPnzqyzzg4q//r713ZSzF8+NSRfY4rHfe7z/Qpx1QkZ9D5ld7TMIzskqhSEJG9RWTf/GPgLOBZYB5whXvYFcCv3cfzgI+Jw0RgW97MVGtMG9/CzReNpaW5CQFampu4+aKxfG3aWC6e4O2MLbbLz13WxuLne5vGGkX46MRRvaKP/O4T1uHbn/MrvadhGNlFkuzRLCKH4+wOwIl0+rmqfl1EDgTuAkYBrwKXquobbkjqd4GzcUJSr1TVwGp3ra2tWmsF8SbNWkSbj/1dgFsuH9crugeclbhNvIZhxIWILFXV1tLxRENSVfVF4ESP8S3AmR7jClydpExZIMghe0hzU2B0jykFwzCSxDKaU8DPISs49nqL7jEMIy1MKaSAl6NWgI9MHMW08S0W3WMYRmqYUkgBL0ftLZePKziQLbrHMIy0qPnOa7XKtPEtvv6B/LjVFjIMo9qYUsgoQUrDMAwjKcx8ZBiGYRQwpWAYhmEUMKVgGIZhFDCfgpE41pDHMGoHUwpGolhDHsOoLcx8ZCSKNeQxjNrClIKRKFaywzBqC1MKRqJYyQ7DqC1MKRiJYiU7DKO2MEezkShWssMwagtTCilj4ZqGYWQJUwopUg/hmvXwHg1jIGE+hRSph3DNeniPhjGQMKWQIvUQrlkP79EwBhKmFFKkHsI16+E9GsZAwpRCitRDuGY9vEfDGEiYozlF6iFcsx7eo2EMJERV05ahIlpbW3XJkiVpi2EYhlFTiMhSVW0tHTfzkWEYhlHAzEdG5rCEPsNID1MKRqawZDfDSBczHxmZwpLdDCNdTCkYmcKS3QwjXUwpGJnCkt0MI11MKRiZwpLdDCNdzNFsZApLdjOMdDGlkDIWftmXaeNb6v4zMIy0qIr5SEQaRWSZiMx3n48RkSdE5AURmSMig93xIe7zNe7ro6shX1rkwy/b2jtQ9oRfzl3WlrZohmHUKdXyKXwBeK7o+TeBW1T1KGAr8El3/JPAVlU9ErjFPW7AYuGXhmFkjcSVgogcCpwH/NB9LsBk4G73kNuBae7jC93nuK+f6R4/ILHwS8MwskY1dgr/DnwZ6HGfHwi0q2qX+3w9kDcgtwDrANzXt7nH90JErhKRJSKyZPPmzUnKnigWfmkYRtZIVCmIyFTgNVVdWjzscaiGeG3PgOptqtqqqq3Dhw+PQdJ0OONYb9n9xg3DMJIm6eijScAFInIusBewH87OoVlEBrm7gUOBDe7x64GRwHoRGQTsD7yRsIypsfh5712O37hhGEbSJLpTUNXrVfVQVR0NfAhYpKofARYDl7iHXQH82n08z32O+/oirfWGDwGYT8EwjKyRVkbzdcAXRWQNjs/gR+74j4AD3fEvAjNSkq8qmE/BMIysUbXkNVV9CHjIffwicIrHMTuBS6slU9pMn3JMrzLREFzSwRLdDMNIGstoTpEoJR0q6TNgysQwjLBYj+aMUjqRb9/VRXtHZ5/jWpqbeHTG5MDreO1Gbr5orCkGw6hj/Ho0R9opiMjpwFGq+j8iMhzYR1VfiktIw8FrV+BHOad0UNa0KYXKsB2YMRAJrRRE5EagFTgG+B8gB/wMJ+zUiBGvidyPck5pi3BKBmsbagxUokQffRC4ANgOoKobgH2TEKreCTthh+kzYBFOyRBn3aq5y9qYNGsRY2YsYNKsRVYQ0UiVKEpht5szoAAisncyIhl+E/awoTlampsQHF9CGL+ANa1Jhrh2YFYp18gaUXwKd4nIf+FkI/898Angv5MRq77xC1W98fzjI5smrGlNMhzS3OTp64m6AzOfj5E1QisFVf1XEfkA8CaOX+ErqvpAYpLVMXFP5Na0Jn6i5pj4YT4fI2tEij5ylYApgpjxi2KxiTy7xKW449pxGEZcRIk+eos9FUsH40QfbVfV/ZIQrF7IUhSLhVhGIw7FHdeOwzDiIor5qFekkYhMw6NUxUChWhNkVmzKWVJO9YT5fIys0e8yF6o6V0QGZMG6OCbIsEolKzblrCinesRMhUaWiGI+uqjoaQNOIltt18jwodIJMopSidOmXMnuJivKyTCMdImSp3B+0c8U4C2cnsoDjkonyCiJTV55BIKjSKIkMlUa725JboZhQDSfwpVJCpIlKl29+ymP/ETvtZKfvXA1be0dCHu2X1HMVpXubqZPOYbpv1xBZ8+ezV+uQQoOT3NCG0Z9UFYpiMh3CDATqeo/xCpRBqg0IsRPqeR3ANB3wp82voVJsxb1OS/sxB6L+ae0Q7b73JzQhlE/hNkpDLy61GWoNCLES6kU7wDylE74USf24tV7gwjdHmXQw+5uZi9cTWd37/M7u7Vg8jIntGHUB2WVgqreXg1BskYlESFeSsWv/HXxhB/FbFW6evdSCFF2N/3ZaZgT2jAGHlGij4bj9FY+DtgrP66q/h1eapC4bOelSsXLNAS9J/woZiu/8tqNIvSoRpa9nEKyrFvDqA+i5CncAcwBzgM+DVwBbE5CqLRI0nYeNOEXK6LmoTmGDGpgW0dnv/IbelR5adZ5scoHWNatYdQJUZTCgar6IxH5gqr+Hvi9iPw+KcHSIMkELj8/BfSecLfu6KQp18gtl48LvGcYU1OUXU8YP4pFHxnGwCeKUsg3CN4oIucBG4BD4xcpPeJM4Apb5G7SrEX9UkTlVvb92fUE+VEs69Yw6oMoyWtfE5H9gS8B/wj8ELg2EalSIq4EriiJZEE5DUGJZ9PGt3DzRWN9m+7E2RnMMIz6IYpSeEJVt6nqs6p6hqpOUNV5iUmWAnF1KYsyIQcpnHIZydPGtzB9yjEc0tzEhvYOZi9cXTjeylYYhtEfopiP/igiL+E4m+9V1a0JyZQacVWsDDMh581LpVnMxZQzIwWZiKxOv2EY/SFKmYujROQU4EPADSLyJ+AXqvqzxKRLgThs5+Um5NLJPKiqYNDKPmhHYnX6DcPoD1HMR6jqk6r6RZw+Cm8AdZnYVo5yZii/HAMvglb2QTuScj4HwzAML6Ikr+0HfBBnp3AE8CsGcJOdSihnhgpr1y+3si+3I7GIoWSw4oDGQCaKT2EFMBf4qqo+lpA8A4agCdlvMm9uyrH3kEGhJxszEVWfahcHNAVkVJsoSuFwVY8COy4i8h1V/XwMMg14zjh2OHc8/movX0JTrpGZFxwf6QtvrRyrTzU71Fl1WiMNojiay3VZm1ShLHXB3GVt3LO0rZdCEODiCf0z9ZiJqLpUM9TXWqQaaRDJ0WxUjtcXXYHFzw+oMlIDlmp2qLNcEyMNElUKIrKXiDwpIitEZJWI3OSOjxGRJ0TkBRGZIyKD3fEh7vM17uujk5QvDWrhiz53WRuTZi1izIwFkVqC1gNxJTiGwVqkGmkQp1Io7dsFsAuYrKonAuOAs0VkIvBN4BZVPQrYCnzSPf6TwFZVPRK4xT1uQJH1L3qlvZ4HOtUM9a2mAjKMPFEczQCIyN6qut3jpVtLB1w/xNvu05z7o8Bk4G/c8duBmcD3gQvdxwB3A98VEQnhz6gZsh4xlEU7dtYicKrlx7FAAiMNouQpvBunCN4+wCgRORH4lKp+FkBVf+JzXiOwFDgS+E9gLdCuql3uIeuB/H95C7DOvV6XiGwDDgReL7nmVcBVAKNGjQr7FjJBFr/oxZOun/ZNy7xV7xE4FkhgVJso5qNbgCnAFgBVXQG8t9xJqtqtquNwymyfArzT6zD3t5cJqs88paq3qWqrqrYOHz48pPjZIGur3lJzkR9pmbes2qthVJdI5iNVXSfSa94OV6vBObddRB4CJgLNIjLI3S0citObAZxdw0hgvYgMAvbHKadRk5QqgDOOHc49S9s8V71An2MXP785lPKoRNGEKbmRpnmrFhzzhjGQiKIU1rkmJHWjhf4BeC7oBLevc6erEJqA9+M4jxcDlwC/wGnr+Wv3lHnu88fc1xfVqj/By+xRmrAGzqr3pvtWsbOzp9exP3v81cIxQSaTSs0rQZOrQOq7Gav2ahjVJYr56NPA1Th2//U40URXlzlnBLBYRJ4BngIeUNX5wHXAF0VkDY7P4Efu8T8CDnTHvwjMiCBfpvDLR/Bi647Osqt1P5NJpeYVv8m1pbmJl2adx6MzJqdq3rIIHMOoLlEyml8HPhLl4qr6DDDeY/xFPIrpqepO4NIo98gqSZg3vK5ZqXkl69FQWXTMG8ZAJkr00beArwEdwP8BJwLXDLR+CnHhZ/YobajTlGtkyKAG2js6+xzrdc2w9wlrXqmFSdcicJIja4EPRvpE8SmcpapfFpEP4piPLsXxDZhS8MBrBZ5rEHKNwo7OHgCGDc1x4/nHA/Q5thS/1XscK32bdOuTeg/3NbyJ4lPIub/PBe5U1ZqNCqoG+czX5qZcYayzRwsKAWCn+9grS/ajE0eFypq1ZjpGf7FwX8OLKDuF+0TkeRzz0WfdyKKdyYg1cNjV1eP7Wj7yKL9S95rI89v7a+csL7TZLD3O61wzCxjlsHBfw4vQOwVVnQGcBrSqaiewHacsheFDmByArTs6fesK9bcOkdUvMsKQ9TpcRjqEVgoi8jHgfOAj7uNLgLOSEmwgEHbF5bdd99vez5y3quz10jALWHXV2sLCfQ0vopiPTi56vBdwJvA08NNYJUqZOM0ufpFBpUTdxrd3OLsLP7nSMAtk0WlpJrRgaiHyzKg+UfIUerXaFJH9gf+NXaIUiXti84oM8uKQ5ibPCSxIqQRVLd2/KecZ4pqkWSBr1VWzqKSyiEWeGaVU0k9hB3BUXIJkgbjNLqWRQcOG5sg19K7515Rr5Ixjh3v6AM441r/Yn9+qf+6yNrbv7uoznmuQRM0CWXNaWmSNYfSPKMlr97En76oRp9rpXUkIlRZJTGylKzGvHYHfBLb4+c0MG5pj647wq/7ZC1fT2d23oMY+ew1KdEWYtRpFWVNShlErRPEp/GvR4y7gFVVdH7M8qVKNic1ru37tnOWex25o7+CWy8dFSk7z9UN4KJY4yVq5jKwpKcOoFaKEpP4eeB7YFxgG7E5KqLRIKxrDb6LavylX2EU0uiXLyyWnpRVmmLUkOousMYz+EcV8dBkwG3gIp4TPd0RkuqrenZBsVSetaAy/khjbd3cVHMbdqoVJLUieNFfsWXJaWmSNYfQPCduuQERWAB9Q1dfc58OB36nqiQnKV5bW1lZdsmRJmiLEQqmvYcfuLk9fQktzE4/OmBzpWjYZZpQ1a2D+fHjgAbjnHthrr7QlMuoIEVmqqq2l41F8Cg15heCyhcqilwYclUzGpavsMTMWeB4XxlGapRW7UURnJzzyCCxY4CiD1W4k1HHHwauvwtFHpyufYRBNKfyfiCwE7nSfXw78Jn6RapNycfHlFEbp680Roo5sZ5BhXn8dfvtbRwksXAjbtsHgwXDGGXD11XDeeXD44WlLaRgFoiSvTReRi4FJOD6F21T1V4lJVmOUi4svpzBKX8+X2S4OL/XyDViSVsZQhZUrHSWwYAE89pgz9ld/BZdcAlOnwvvfD/vsU1WxbOFghCW0TyGrZMWnMGbGAs92m/k+x17hkXn/wKRZizxfb27KsfeQQYFfZL9z89fPn2OTQoJ0dMCiRXvMQuvWOeOtrY4SmDoVxo+HhnSsraULB3AWGFZivb6p2KcgIhcB3wTegTPXCaCqul9sUtYwQXHxfn6AtvaOwEl9W0cny28MrjkY5GPI7xqWvPIG9yxty9RuouaV1Pr1e5TAgw86imHvveGss+DGG+Hcc2HEiLSlBLJXgsTINlF8Ct8CzlfV55ISppbxCgUVnAm4UYRujx1Z/nU/wuQWlCu619HZzZ1PrOtzf6tLFJHubnjqqT1moeVuwuGYMfB3f+fsBv76r2HIkHTl9MCyu40oRFEKm0wh+FMcF9/W3tGrF7OfQggy3IXNLQhTdM/r/pDNukSZUgpvvgn33+8ogt/8BjZvhsZGmDQJvvUtRxEceyyIlL9Wilh2txGFskrBNRsBLBGROcBcYFf+dVW9NyHZao58KGiQSQgcX4FXFdM8LRHMKaXKKApWl8iDF15wlMD8+fDww9DVBcOGOeagqVNhyhTneQ2RtRIkRrYJs1M4v+jxDno31lGgbpWCn1283OS2q6vHt9BdmOS0UvLKyMuh6EeUSSFu+3+mVq67dzu5A3lF8MILzvjxx8OXvuQogokTYVCUTXW26E92d837fIx+U/Y/XVWvDHMhEbleVW+uXKTaIMguHsbOP2RQA025xthWb/kvcb5Okp/JCKLtRJKw/6e+ct28uXfuwJtvOrkDkyfDF77g5A6MHl0dWapElITGmvT5GLERZ4zcpTFeK/ME2cWnTzmGXGOwnXlbR2dsBeSKezKDvw8hT5RVXxJ9CapePE8VVqyAr38dTjsNDj4YrrjC2SFcfjnMnQtbtjiK4uqrB5xCiIr1oqhv4twTZ9vbFjNBdvFp41v4p3uf8exrkOeQ5qbYylF4fYnLHR/2vknZ/xMvxbFjh5M7kI8WWu9WeT/lFJg5c0/uQMadxGmQaZ9PGczsVTlxKoXazoKLiF/Ly/2bcsxd1saOzh7fc3ON8XZBi/pljXJ8puz/5Xj1VUcBLFjg5A7s3OlkDp91Fnz1q3DOOU5msRFITf3NizCzVzzEaT6qmyWXX8tLgO27u5g5b1Xg+YMaJNZ/Ur8vq98fxK9+0qRZixgzYwGTZi1i7rI2wLsvQT6/ovi4VOjudspI3HADnHgiHHYYfPaz8Nxz8KlPOeGkr7/uVCC98kpTCCGp1V4UZvaKhzh3Cr+M8VqZxq/lJUBntwaGmwJ0BOwi+oOf4/biCS29Mpnz4/2pn+SVf5HKSmzbNsc5PH++4wN4/XUnd+D002H2bMcsdMwxZhaqgFrtRVHLZq8sEaXMxXDg74HRxeep6ifc39+IW7iskrV/sqAvcethB5T9cpdLJgvKv0g86UwV/vznPb6BP/zByR044IA9uQNnnVVzuQNZpxbLr9eq2StrRNkp/Br4A/A7ILxXcwBSLuR02NAcOzt7fJ2/w4bmPMf9nGRhnGd+X+IwX+6wK6w4V2KB72n3bidxLF9baM0aZ3zsWJg+3QkZnTjR2SEYhkvqoc4DhChKYaiqXhfl4iIyEvgp8FdAD0657VtF5ABgDs6u42XgMlXdKiIC3Aqci5Mo93FVfTrKPatBUGmJplwjN55/PAAz563qY0rKNUrh9WL8TDjVKGYXdoUV10rM673O/t8/MGreRk5a+ajjC3jrLaeO0OTJcO21jiI47LBI9zHqi1o1e2WNKO04vwb8UVVDN9YRkRHACFV9WkT2BZYC04CPA2+o6iwRmQEMU9XrRORc4PM4SuFU4FZVPTXoHmmVzs6vdIsL3nklhZWuiM84djiLn9/c55+2XGmMUvqT+Rz0XsKUVo6rBPOkWYto27qD4197kclrnmTy2iWcuPHPNKBwyCF7yk1PnuxUHjUMI3b8SmdHUQpvAXvj1D3qpB+ls0Xk18B33Z/3qepGV3E8pKrHiMh/uY/vdI9fnT/O75pZ6acQhqBJ9Zo5yyNdS4CXZp0Xq2xhVlgVxYFv3w4PPsjPv/J9zlj7FCPe3kIPwooRR/PgkSez6IhT+M2PP2dOYsOoAhX3U1DVfSsUYDQwHngCODg/0buK4R3uYS3AuqLT1rtjvkqhlvBz6M6ct6ps1dRS4naehXUsRnZAvvLKHt/AokWwaxcXDhnKQ6PHs/iIk3no8Am8vrfjJG5pbjKFYBgpEykkVUSGAUcBe+XHVPXhEOftA9wDXKOqb4r/F9/rhT5zpYhcBVwFMGrUqPKCV0hcWZJ+DtlyIaxeTJ9yTDazN7u74fHH90QLrXT8Hxx5JHzmMzB1Kr/b93Bm3LfaHIIRyeTf2xhwRAlJ/TvgC8ChwHJgIvAYEGjYFpEcjkK4o6jM9iYRGVFkPnrNHV8PjCw6/VBgQ+k1VfU24DZwzEdh30N/8HKKXjNnOTPnrWLmBcf3+VJ6fXHB2SXEJWhzkxO9lJnszfb23rkDW7Y4VUXf8x749rcd/8DRRxcOvxDQ3GCb4CJg2bpGtYjiU1gJnAw8rqrjRORY4CZVvTzgHAFux3EqX1M0PhvYUuRoPkBVvywi5wGfY4+j+T9U9ZQguZL0Kcxd1saX7lrhW2Cu1Mnq5TPINQgIvsluuQYh1yiBZTF6Hd8ozL7kRN/+CXlnd6ITriqsXr2n3PQjjzg7hIMOckpJ5HMHmpvju2ed4xeIEGfAgVFfVOxTAHaq6k4RQUSGqOrzIlJuvz8J+FtgpYjkPan/BMwC7hKRTwKvsqfC6m9wFMIanJDUUGW7kyA/wQdVHO3o7OaaOcsLlVG9fAadPWWUbgQTugjMvuREwL+NZ34FGfuKctcuJ3cgrwhefNEZP+EEuO46RxGccorlDiSEZesa1SKKUlgvIs04ndceEJGteJh2ilHVR/Cf9s70OF6BqyPIlBhRKo+WTsRR6OzWwGqqefK7EqAwyXvRKBJfq8tNm5w2lPPnO7kDb78Ne+0FZ57pJJGdey5UwadjWLauUT2iRB990H04U0QWA/sD/5eIVBkg6gqsPwohChdP2FNqwu9epU17ign1flRh2bI9u4GnnnLGDz0UPvpRJ4Fs8mQYOrS/b8PoJ5ata1SLMD2a93Mjhg4oGs4vVfcB3khEspQpV8oiLA0CjQ0SajcQxJ1PrOOOx18NdFbffNFYX1+D74py+3b43e/2RAtt3OjYqSZOhK99zTELnXCChYqmjGXrGtWirKNZROar6lQReQknPLR4dlBVPTxJAcuRlKPZ02ncKOw9eFCkENJcg3D5KSO584l1ZTuiVUKjCD2qNA/N8fbOrl6+jD5Zxy+/vEcJLF7s+Av2289pSj91quMsHj48MVkNw0iffjuaVXWq+3tMEoJllaCV2dxlbUy/e0Wo1X9nj7LgmY30JKgQYE8Lzq07Osk1Cs1NObZ1dHJIcxNfPvMILtz+Esz4jqMMVrn9Ho4+2mk/ed55TunpwYMTldEwjOwTZqdwUtDraResS6vMxbib7o+0Yxg2NMfWHeWPLxfCWowADW7dpVJGyU4ePmH3ntyBrVud3IH3vtfZDZx3Xq/cAcMw6otKQlK/7f7eC2gFVuDMRyfglKw4PS4ha4ltEbOQVf0dwfkSFy0lyW4b2jt8J/18fPqYGQsKNzhyyzrOXOsUmJuw/k+gPY4Z6IILHEXwgQ/A/vtHfauGYdQRYcxHZwCIyC+Aq1R1pfv8XcA/JitedonqiN7W0cktl4/zrawKjiK4ds7yPqYq36iTXbuY9toqTnjmEc5c8ySjtm0CYNU7Dud7Ey/l2XGnc9t3PmO5A4ZhhCZKnsKxeYUAoKrPisi4BGSqCaZPOSZSZdNDmps8i8nNXdbWp++CXzvMDe0djG3YwU0NrzD+q/8FDzzALdu30zFoCI8edgI/mHgJiw4/mb/sdxDgRgSYQjAMIwJRlMJzIvJD4Gc41o6PAs8lIlUNMG18Czf8aiXbd4fLT9ixu4u5y9rK9ifIU0g4O3EE07o3Mm3nw/DgAsj7T0aOhI99DKZO5X1/7GZTV0Ofa1hik2EMTJIsjhhFKVwJfAanKB7Aw8D3Y5GiBpm7rI3dXeHqFYETFeS1+vdSCEN3d/Cel5cxee0SuPUj8Je/OHkCp50G3/iG4x9417sKuQPXjwgwMRXJG/afyKpxGkZ2Sbo4YpSM5p0i8gPgN6q6uuI71xilE+X2XV3l6xqV0NHZzU33rSr84YqzjEe2/4XJa5/izDVPcuq6lQzp7uKtvfaGC91IoXPOcQrOeVAusSnKP5FV4zSMbOPXl6VfpWw8iFI6+wJgNjAYGOP6E76qqhdULEXGKc1LqCTTeeuOTseMNPZgztn6Aics/wOT1y7h6C2vArD2gEO5/aTzWXTkyRx90RS+eklgRHCBoOY3Uf6Jkv6HMwyjMpIujhjFfHQjcArwEICqLne7qQ14brpvVcVlKgD273iLv35pKUOv/Dd45Wm+197O7oZBPDHyXdw5bgqLjjiZV8P5B3sAABMXSURBVIYdUjh+xYrXOOmItoon4yj/RFaN0zCyTdLFEaMohS5V3RbQNW3AEibpDJzEs8GDGvY4n1U56vVXOXPtU0xe+yQT2p6nUXt4fWgzXDaNJ48/jU/95QC2DvL+Y8a1Qo/yT2TVOA0j2yRdHDGKUnhWRP4GaBSRo4B/AP4YixQ1TEtzU287/jsPhIcegvnzabvjblrandyBZw8+gu+edhmLjjiZLceewCP/9H6unbWIrYOCV+BxrND9/onOOHY4k2Yt6iW/VeM0jGyTdHHEKErh88ANwC7g58BC4P/FIkXGaW7KeZa0aG7KOV2vNmxw+g7c9B90LbyfQTs76Bg0hD8fMZ7vnXYZvxszgU37Ok7iplwjN5/zTiDchB/HCt3rn+iMY4dzz9K2Pg7lmy8aW6i26lXzyaKSDCN9gnyIlRJFKRzn/gxyfy4ELsApdzGgmXnB8Uz/5YpCtJFoD+M3reUbg1+BCdfD0075p437v4P7j5vM4sNP5rFRY9mVG1IwKeGalIYM2pNPUC4rOtcosa3QS/+JvPoy5M1Vj86YbFFJhlGnRFEKd+CUtXgWCB+gPwCYNr6FQdvf5o8/+AXjnnmEM19ayoFvb4WGBjjtNFZ9fgbX7zqMZ5oP7dN3oLNH6SpKcGvv2JOv4GWq6UWChVWD2nl6YVFJhlEfRFEKm1X1vsQkySJr1zo9B+bPZ+pDDzG1s9NpRj/1bCeB7Oyz4cADucqnqXqe0rm9eEUO+DbG6ezRxCbdRp9Ce40+gQRxRSUlZYIy05ZhxEOkkFS3zMWDOH4FAFT13tilSovOTvjjH/e0o3z+eWf8ne+Ea65xksje/W7I5Xqd1h9ncFt7R6+yF351lNraOxgzY0HsE51fwx+/8TiikpIyQZlpyzDio2/BHH+uBMYBZwPnuz9TkxCqqmzZAj/7GXzoQ06Z6fe9D2691aktdOutsGYNc+94gEkHnM2Y377NpG//gbnL2gqnz13WRkNAmG5QAO/0u1cUVrhBKHsmuuJ7F8swadYixsxYwKRZizyPKaXFZzL3G58+5Riacr2L60WNSgoyQVVCUtc1jHokyk7hRFUdm5gk1ebnP4fvfQ8eewx6euDgg+Hiix2z0PvfD/vuCwSvQgGuv3el7+o63ych/7uUzm7lpvtW0R4yD8LLhu+VbT397hVA8Co5auhpHGFwSSXGWcKdYcRHFKXwuIgcp6p/SkyaarJuHezcCf/yL45ZaMIEx3FcQrlVqJeTuEEc23w+WinIX7x1RyctEXozlE50XtnWeWUTNGH3Z5KvNAwuqcQ4S7gzjPiIohROB64QkZdwfAoCqKrWZkjql78M111X9jC/1WbQJN6jROrJvH1XV58xv91F6UTnl20dJgs7yVhnL5JKjLOEO8OIjyhK4ezEpEiDkOU6onZY6w9eiXF75Rro6tFeu4Ban+iSysRMOsPTMOqJKKWzX0lSkKxSNpcgITo6e8g1CMOG5mjf0ek70QVlW4elmuGcSe1Oqr3rMYyBSpSdQl1Sugrtbz5ZU66Riye0sPj5zYXJt9wOpLNHGTp4EMu+cpbvMaXZ1uAU5pt5wfGh5LJwTsMwijGlEILiVeikMolqXjSKcPNFY/tMsmGuVe71Sk0nlqlsGEYxphQi0h9zUo+q5wQb9lqlvZ1LqcR0YuGchmEUEyV5zcCZgG++aCwtzU0ITrLXRyeO8k36Av/QyGnjW7h4Qktgghvgm7QWB36yWTinYdQntlPoB8Ur82InbXNTju27uzwjhrycuQD3LG0r66dI0pxj4ZyGYRRjSqECSp207R2dnhFDQC9ncFt7B9fMWU5TroGOznAFZ/PmnLgjhSyc0zCMYhJVCiLyY5z6SK+p6rvcsQOAOcBo4GXgMlXdKk6fz1uBc4EdwMdV9ekk5asULyetV8TQuJvu7xUdlCesQgDHnBMmUqg/SsPCOQ3DyJO0T+En9E16mwE8qKpH4VRcneGOnwMc5f5cBXw/YdkqJijbOV+Ybu6yNs88gijkzTnlSm7klUabGzobVESv3ulPEUHDqAcSVQqq+jDwRsnwhcDt7uPbgWlF4z9Vh8eBZhEZkaR8lRLkjM2biPxKYgex9+BGmptyBUd2Ppy1XKSQVQsNhylPw/AnDZ/Cwaq6EUBVN4rIO9zxFmBd0XHr3bGNpRcQkatwdhOMGjUqWWlLKDbP7N+UI9cofQrSRWHY0BxDBw8KZe4pV/jNwkvDYbkZhuFPlhzNXpGZnrOtqt4G3AbQ2tqaYNPK3gQ5lsMUoPPixvOPDz0RlYsUsmqh4TDlaRj+pJGnsClvFnJ/v+aOrwdGFh13KLChyrIFEuRYDspTCOKm+1aFNlt45UgUZ0rH0QinHrDcDMPwJ42dwjzgCmCW+/vXReOfE5FfAKcC2/JmpqwQ5Fj+6MRR3PH4q5FrI23d0Rmp1lBQpJCFl4bDcjMMw5+kQ1LvBN4HHCQi64EbcZTBXSLySeBV4FL38N/ghKOuwQlJvTJJ2fpDUBG7OU+t63exvDjt2RZeWh5Tnobhj2iEZjBZpLW1VZcsWVKVe81d1sa1c5aHnvwbRehR5ZDmJnbs7gr0Owjw0qzzYpEzCapZXtswjOQRkaWq2lo6niVHc+aZNr4lUohpj2phoi+nUPpjz67WRG3ltQ2jfjClEJEo/ZSLJ/pyCiXInu1XN6laE7WFcBpG/WBVUiPiFeGTaxByjb0jar0cl34RSs1NOd/J1S/R6qb7VlUtUc1COA2jfrCdQkT8nJTFY81Dc6jCtXOWM3vh6oJZxy/qpbRLWvHOoEGE7hK/T0dnt28PhiQmast/MIz6wRzNFRDGrAPOxJ/PJyjNiBahT0XVSnpCtzQ38eiMyZW/uSJKfQrQ+z0ZhlF7+DmaTSn4UM6J6zdR7pVr8IwyKo5EClIeQwY1hCqg19yUY1dXT9Um6v46tS1qyTCyiSmFCIRZGfenV3PxtfyUR9jzb75oLJDtWHvbYRhGdrGQ1AiEibapxHYf5BPwI+/GLp38szy5WtSSYdQephQ8CBNt4+d89TLrRKFBwKMfD81Dc70a9yRB3KYei1oyjNrDQlI9CFMwza/43MwLji8UrQuiuSnneb6XQgDHGZ0kSfQYsMJzhlF7mFLwwGvCB9i+q6swSQZVLJ02voVHZ0wOVAwijimlURzDUP58v3OSnkiTaNBjVVsNo/Yw85EHeZPJTfet6uUMbu/oXdHUr/hc3gwT5IjOX7dbtTBR5q+VRgXPJEw9VnjOMGoPUwo+TBvfwuyFq/tECJVzlHpF3JSj+JppTaRJJahZ1VbDqC1MKQTQn9Wzlxkm6r3SmEitx4BhGGA+hUD64ygNMhm1NDcxbGgu8jWrQbmuboZh1Ae2Uwgg6up57rI2BO/G0vnyE34JXVlYkZupxzAMUwoBRLXvz1642lMhCHtKY5vz1TCMLGNKoQxRVs9+vgald+axrcgNw8gq5lOIET+/QLlENsMwjKxgSiFGLFnLMIxax8xHMWL+AsMwah1TCjFj/gLDMGoZMx8ZhmEYBUwpGIZhGAVMKRiGYRgFTCkYhmEYBczRHBJrQG8YRj1gSiEEpfWK8l3JINs9kg3DMKJi5qMQJNGVzDAMI4uYUgiBNaA3DKNeMKUQAmtAbxhGvWBKIQRW08gwjHohc0pBRM4WkdUiskZEZqQtD1hXMsMw6odMRR+JSCPwn8AHgPXAUyIyT1X/lK5kVtPIMIz6IGs7hVOANar6oqruBn4BXJiyTIZhGHVD1pRCC7Cu6Pl6d6wXInKViCwRkSWbN2+umnCGYRgDnawpBfEY69P2WFVvU9VWVW0dPnx4FcQyDMOoD7KmFNYDI4ueHwpsSEkWwzCMuiNrSuEp4CgRGSMig4EPAfNSlskwDKNuENU+1plUEZFzgX8HGoEfq+rXyxy/GXiln7c7CHi9n+dWi6zLmHX5IPsyZl0+yL6MWZcPsifjYarax/6eOaVQTURkiaq2pi1HEFmXMevyQfZlzLp8kH0Zsy4f1IaMkD3zkWEYhpEiphQMwzCMAvWuFG5LW4AQZF3GrMsH2Zcx6/JB9mXMunxQGzLWt0/BMAzD6E297xQMwzCMIkwpGIZhGAXqVimkVaJbREaKyGIReU5EVonIF9zxmSLSJiLL3Z9zi8653pVztYhMqcZ7EJGXRWSlK8sSd+wAEXlARF5wfw9zx0VE/sOV4xkROanoOle4x78gIlfEJNsxRZ/TchF5U0SuSfszFJEfi8hrIvJs0Vhsn5mITHD/Jmvcc73KwkSVb7aIPO/K8CsRaXbHR4tIR9Fn+YNycvi91xhkjO3vKk5i7BOujHPESZKtVL45RbK9LCLL0/wMK0ZV6+4HJzFuLXA4MBhYARxXpXuPAE5yH+8L/Bk4DpgJ/KPH8ce58g0BxrhyNyb9HoCXgYNKxr4FzHAfzwC+6T4+F/gtTu2qicAT7vgBwIvu72Hu42EJ/C3/AhyW9mcIvBc4CXg2ic8MeBI4zT3nt8A5Mch3FjDIffzNIvlGFx9Xch1POfzeawwyxvZ3Be4CPuQ+/gHwmUrlK3n928BX0vwMK/2p151CaiW6VXWjqj7tPn4LeA6PSrBFXAj8QlV3qepLwBoc+dN4DxcCt7uPbwemFY3/VB0eB5pFZAQwBXhAVd9Q1a3AA8DZMct0JrBWVYOy2qvyGarqw8AbHveu+DNzX9tPVR9TZ8b4adG1+i2fqt6vql3u08dx6o35UkYOv/dakYwBRPq7uqvxycDd/ZUxSD73+pcBdwZdI+nPsFLqVSmEKtGdNCIyGhgPPOEOfc7dxv+4aNvoJ2vS70GB+0VkqYhc5Y4drKobwVFuwDtSlhGc+ljFX8IsfYYQ32fW4j5OUtZP4Kxa84wRkWUi8nsReU+R3H5y+L3XOIjj73og0F6kBOP+DN8DbFLVF4rGsvQZhqJelUKoEt2JCiCyD3APcI2qvgl8HzgCGAdsxNmGgr+sSb+HSap6EnAOcLWIvDfg2FRkdO3BFwC/dIey9hkGEVWmpD/LG4Au4A53aCMwSlXHA18Efi4i+yUthw9x/V2Tlv3D9F6gZOkzDE29KoVUS3SLSA5HIdyhqvcCqOomVe1W1R7gv3G2wEGyJvoeVHWD+/s14FeuPJvcrW9+C/xamjLiKKynVXWTK2umPkOXuD6z9fQ27cQmq+vMngp8xDVn4JpktriPl+LY6I8uI4ffe62IGP+ur+OY6QaVjFeMe82LgDlFcmfmM4xCvSqF1Ep0u3bHHwHPqeq/FY2PKDrsg0A+umEe8CERGSIiY4CjcJxUib0HEdlbRPbNP8ZxRj7rXj8fDXMF8OsiGT8mDhOBbe7WdyFwlogMc7f8Z7ljcdFrZZalz7CIWD4z97W3RGSi+z/0saJr9RsRORu4DrhAVXcUjQ8Xp2c6InI4zmf2Yhk5/N5rpTLG8nd1Fd5i4JK4ZQTeDzyvqgWzUJY+w0hU27OdlR+c6I8/42jvG6p439NxtorPAMvdn3OB/wVWuuPzgBFF59zgyrmaooiTpN4DTtTGCvdnVf7aODbZB4EX3N8HuOMC/Kcrx0qgtehan8BxAK4BroxRxqHAFmD/orFUP0McBbUR6MRZDX4yzs8MaMWZENcC38WtSFChfGtw7O/5/8UfuMde7P7tVwBPA+eXk8PvvcYgY2x/V/d/+0n3ff8SGFKpfO74T4BPlxybymdY6Y+VuTAMwzAK1Kv5yDAMw/DAlIJhGIZRwJSCYRiGUcCUgmEYhlHAlIJhGIZRwJSCYRiGUcCUglEXiEiziHy2zDGjReRvQlxrtBSVTjaMgYQpBaNeaAYClQJOqeOySiEKRSUVDKMmMKVg1AuzgCPcZiez3Z9n3UYnlxcd8x73mGvdHcEfRORp9+fdYW4kIh8XkV+KyH04lWbF634B4+9zq2reJSJ/FpFZIvIREXnSPe4I97hL3XNXiMjD8X9kRj1iqxijXpgBvEtVx4nIxcCngROBg4Cn3El1Bk4zl6kAIjIU+ICq7hSRo3BKHLSGvN9pwAmq+oZ7v3Ee93u3zzju2Dtxave/CPxQVU8Rp1Pf54FrgK8AU1S1TdyOaYZRKbZTMOqR04E71am8uQn4PXCyx3E54L9FZCVOnZzjItzjAVXNN2Pxu1+QHE+p05BpF059nPvd8ZU4Zi6AR4GfiMjf43QbM4yKsZ2CUY+E7W18LbAJZ9XeAOyMcI/tIe4XJMeuosc9Rc97cL+3qvppETkVOA9YLiLj1C3VbBj9xXYKRr3wFk5PbICHgctFpFFEhuP03X2y5BiA/YGN6tTx/1v6vxr3u5/feChE5AhVfUJVv4LTK2BkuXMMoxy2UzDqAlXdIiKPuqGkv8Upw7wCp4z5l1X1LyKyBegSkRU4pZC/B9wjIpfi1OHf7n31svwKx8dQej+/8WNDXne26+sQnDLLK/opn2EUsNLZhmEYRgEzHxmGYRgFzHxkGP1ERKYA3ywZfklVP5iGPIYRB2Y+MgzDMAqY+cgwDMMoYErBMAzDKGBKwTAMwyhgSsEwDMMo8P8B4CM466Icy9gAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEGCAYAAACZ0MnKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3df5RcdX3/8edrZnYm7Cy/WWlMiAG/0VPxC4gRFH98BaqgbUVtVX4IAfw21QMFq1+PYlsVWi3+bjm0cECjoAiigKaKP5BqqS2iAWMgUEtAkJUUQgAhiWSzO+/vH/fO7t3NzOZmsrMzu/N6nDNn7nzm/nhfJuSdz4/7+SgiMDMzy6PQ6QDMzGz2cNIwM7PcnDTMzCw3Jw0zM8vNScPMzHIrdTqAdtlvv/1i8eLFO33c2oefYp9qmfl7zpv+oMzMutjtt9/+WEQMTrXPnE0aixcvZtWqVTt93NK/+wGvecGz+Ps3H9KGqMzMupekB3e0j5unJhmoFNm8dbTTYZiZdSUnjUn6yyU2bx3pdBhmZl2pbUlD0gGSfijpHklrJZ2blu8j6SZJ96bve6flknSRpHWS1kg6PHOuZen+90pa1q6YAQYqJTY5aZiZNdTOPo0R4L0RcYek3YHbJd0EnA7cHBEXSvoA8AHg/cDrgCXp60jgEuBISfsAHwaWApGeZ2VEPNGOoKuVIo9tGm7Hqc1sFti2bRtDQ0M888wznQ6lbebNm8fChQvp6+vb6WPbljQiYj2wPt1+WtI9wALgBODV6W5XAD8iSRonAFdGMhnWTyTtJWl+uu9NEfE4QJp4jgeubkfc/ZUSmzduacepzWwWGBoaYvfdd2fx4sVI6nQ40y4i2LhxI0NDQxx44IE7ffyM9GlIWgy8CLgN2D9NKPXE8qx0twXAQ5nDhtKyZuWNrrNc0ipJqzZs2NBSrANlN0+Z9bJnnnmGfffdd04mDABJ7Lvvvi3XpNqeNCQNANcB746Ip6batUFZTFG+fWHEZRGxNCKWDg5OOdS4qWqlxJZhj54y62VzNWHU7cr9tTVpSOojSRhXRcT1afEjabMT6fujafkQcEDm8IXAw1OUt0W1UmTz8AieMt7MbHvtHD0l4PPAPRHxmcxXK4H6CKhlwDcz5aelo6heCvw2bb76HvBaSXunI61em5a1RbVSIgLXNsysYwYGBjodQlPtHD31cuBU4E5Jq9OyDwIXAtdKegfwa+At6Xc3Aq8H1gFbgDMAIuJxSX8L/Czd74J6p3g7VCvJf5LNwyNj22Zmlmjn6Kkf07g/AuDYBvsHcFaTc60AVkxfdM1Vy0WA5Knw3WfiimZmO/bggw9y5plnsmHDBgYHB/nCF77AokWL+NrXvsb5559PsVhkzz335JZbbmHt2rWcccYZDA8PU6vVuO6661iyZMm0xOF/Sk8yVtPwCCqznnf+v6zl7oenGr+z817w7D348B8fvNPHnX322Zx22mksW7aMFStWcM455/CNb3yDCy64gO9973ssWLCAJ598EoBLL72Uc889l1NOOYXh4WFGR6evud3TiEwy4KRhZl3o1ltv5eSTTwbg1FNP5cc//jEAL3/5yzn99NO5/PLLx5LDy172Mj72sY/x8Y9/nAcffJDddttt2uJwTWOS/nrz1LCThlmva6VGMFPqw2YvvfRSbrvtNr797W9z2GGHsXr1ak4++WSOPPJIvv3tb3Pcccfxuc99jmOOOWZaruuaxiT1msYmz3RrZl3kqKOO4pprrgHgqquu4hWveAUA9913H0ceeSQXXHAB++23Hw899BD3338/Bx10EOeccw5veMMbWLNmzbTF4ZrGJPU+jS1unjKzDtmyZQsLFy4c+/ye97yHiy66iDPPPJNPfvKTYx3hAO973/u49957iQiOPfZYDj30UC688EK+/OUv09fXx+/93u/xoQ99aNpic9KYpFqu1zScNMysM2q1WsPyf/3Xf92u7Prrr9+u7LzzzuO8886b9rjAzVPbqVYyQ27NzGwCJ41JSsUClVKBLe4INzPbjpNGA1UvxGTW0+b63HO7cn9OGg1UK0U/p2HWo+bNm8fGjRvnbOKor6cxb968lo53R3gD1XKJzZ6w0KwnLVy4kKGhIVpdk2c2qK/c1wonjQaqlZJrGmY9qq+vr6UV7XqFm6cacNIwM2vMSaOBgUrRzVNmZg04aTTQX3ZNw8ysESeNBgY85NbMrCEnjQaqlSJbhkfn7JA7M7NWOWk00F8uMVoLto40nv/FzKxXOWk0MD49upuozMyy2pY0JK2Q9KikuzJlX5W0On09IGl1Wr5Y0u8y312aOebFku6UtE7SRaqvPNJG49OjewSVmVlWOx/u+yJwMXBlvSAi3lbflvRp4LeZ/e+LiMManOcSYDnwE+BG4HjgO22Id0w1Xb3PNQ0zs4naVtOIiFuAxxt9l9YW3gpcPdU5JM0H9oiIWyPplb4SeON0xzpZvabhJV/NzCbqVJ/GK4FHIuLeTNmBkn4u6d8kvTItWwAMZfYZSssakrRc0ipJq3Zl3pixpOGahpnZBJ1KGicxsZaxHlgUES8C3gN8RdIeQKP+i6bjYCPisohYGhFLBwcHWw7OCzGZmTU24xMWSioBbwZeXC+LiK3A1nT7dkn3Ac8jqVlkp2JcCDzc7hjrS766pmFmNlEnahp/APxXRIw1O0kalFRMtw8ClgD3R8R64GlJL037QU4DvtnuAAfcp2Fm1lA7h9xeDdwKPF/SkKR3pF+dyPYd4K8C1kj6BfB14J0RUe9EfxfwOWAdcB9tHjkF0D/WPOWkYWaW1bbmqYg4qUn56Q3KrgOua7L/KuCF0xrcDlRKRfqKYpP7NMzMJvAT4U1UKyW2uHnKzGwCJ40mqmXPdGtmNpmTRhPVStF9GmZmkzhpNJE0T7lPw8wsy0mjCTdPmZltz0mjCTdPmZltz0mjiWql5GlEzMwmmTJpSCpK+vJMBdNNquWSnwg3M5tkyqQREaPAoKTyDMXTNZKahpOGmVlWnifCHwD+Q9JKYHO9MCI+066gusFApci20WB4pEa55FY8MzPIlzQeTl8FYPf2htM9+jMz3ZZLPVfRMjNraIdJIyLOB5C0e/IxNrU9qi5Qn+l209YR9q46aZiZQY7RU5JeKOnnwF3AWkm3Szq4/aF1Vn31Pj/gZ2Y2Lk9j/WXAeyLiORHxHOC9wOXtDavz6tOj+wE/M7NxeZJGNSJ+WP8QET8Cqm2LqEsMeJ1wM7Pt5OkIv1/S3wBfSj+/HfhV+0LqDvUlXz09upnZuDw1jTOBQeD69LUfcEY7g+oG1bHmKfdpmJnVTVnTSNft/mBEnDND8XSNqpunzMy2k+eJ8BfPUCxdZaxPw81TZmZj8jRP/VzSSkmnSnpz/bWjgyStkPSopLsyZR+R9BtJq9PX6zPfnSdpnaRfSjouU358WrZO0gd2+g5bVCkVKMg1DTOzrDwd4fsAG4FjMmVB0r8xlS8CFwNXTir/bER8Klsg6QXAicDBwLOBH0h6Xvr1PwGvAYaAn0laGRF354h7l0jyTLdmZpPk6dNYExGf3dkTR8Qtkhbn3P0E4JqI2Ar8StI64Ij0u3URcX8azzXpvm1PGpA0UbmmYWY2Lk+fxhum+ZpnS1qTNl/tnZYtAB7K7DOUljUrb0jSckmrJK3asGHDLgfaXy66T8PMLCNPn8Z/SrpY0islHV5/tXi9S4DnAocB64FPp+VqsG9MUd5QRFwWEUsjYung4GCLIY4bcPOUmdkEefo0jkrfL8iUBRP7OHKJiEfq25IuB76VfhwCDsjsupBkZl2mKG87r6lhZjZRnlluj56ui0maHxHr049vIpkEEWAl8BVJnyHpCF8C/JSkprFE0oHAb0g6y0+ernh2pL9c4vHNW2bqcmZmXa9p85Skf8hsnzvpuy/u6MSSrgZuBZ4vaUjSO4BPSLpT0hrgaOAvASJiLXAtSQf3d4GzImI0IkaAs4HvAfcA16b7zoiBStGz3JqZZUxV03hVZnsZ8I+Zz4fs6MQRcVKD4s9Psf9HgY82KL8RuHFH12sHN0+ZmU00VUe4mmz3jGql5KnRzcwypqppFNIhsYXMdj15FNseWReolktsHakxMlqjVPQ64WZmUyWNPYHbGU8Ud2S+azrsdS6pz3S7eXiUPXdz0jAza5o0ImLxDMbRlbIz3e65W1+HozEz6zz/83kK4+uEu1/DzAycNKY04IWYzMwmcNKYQn/ZCzGZmWU17dOQtM9UB0bE49MfTncZ8Op9ZmYTTDV66nbGJw1cBDyRbu8F/Bo4sO3RdVjVq/eZmU3QtHkqIg6MiINIpvD444jYLyL2Bf6IHS/ANCdUy+7TMDPLytOn8ZJ0Kg8AIuI7wP9pX0jdY2z0lJunzMyAfFOjPybpr4EvkzRXvZ1k+dc5b7e+IvI64WZmY/LUNE4CBoEb0tdgWjbnFQqiv6/o5ikzs1Se9TQeB86VNBARm2Ygpq5SrZT8cJ+ZWWqHNQ1JR0m6m2StCyQdKumf2x5Zl/BMt2Zm4/I0T30WOI60HyMifsHEtTbmtGql6D4NM7NUrifCI+KhSUU908hfLZfY7NX7zMyAfEnjIUlHASGpLOn/kSy92hO8ep+Z2bg8SeOdwFnAAmAIOCz93BOcNMzMxk2ZNCQVgVMj4pSI2D8inhURb4+IHT6nIWmFpEcl3ZUp+6Sk/5K0RtINkvZKyxdL+p2k1enr0swxL5Z0p6R1ki6SNKNLzw5Uim6eMjNLTZk0ImIUOKHFc38ROH5S2U3ACyPiEOC/gfMy390XEYelr3dmyi8BlgNL0tfkc7ZVf9k1DTOzujzNU/8h6WJJr5R0eP21o4Mi4hbg8Ull34+I+t/APwEWTnUOSfOBPSLi1ogI4ErgjTlinjbJcxqj1Go9scKtmdmU8kwjclT6fkGmLIBjdvHaZwJfzXw+UNLPgaeAv46If2e8H6VuKC1rSNJykloJixYt2sXwEvWFmLZsGx2bKt3MrFfleSL86Om+qKS/AkaAq9Ki9cCiiNgo6cXANyQdTDIV+3YhTRHrZcBlAEuXLp2WqkF2ISYnDTPrdbn+FpT0h8DBwLx6WURc0PyIKc+1jGR69WPTJiciYiuwNd2+XdJ9wPNIahbZJqyFwMOtXLdV9USxaesI+8/khc3MulCeaUQuBd4G/AXJv/zfAjynlYtJOh54P/CGiNiSKR9MR2oh6SCSDu/7I2I98LSkl6ajpk4DvtnKtVs1Pj26R1CZmeXpCD8qIk4DnoiI84GXAQfs6CBJVwO3As+XNCTpHcDFwO7ATZOG1r4KWCPpF8DXgXdmlpN9F/A5YB1wH/Cd/Le368YXYvIIKjOzPM1Tv0vft0h6NskcVDtc6jUiGk2f/vkm+14HXNfku1XAC3PE2RZVrxNuZjYmT9L4VvoQ3ieBO0g6oj/X1qi6iNcJNzMbl2f01N+mm9dJ+hYwLyJ+296wukc1HXK72X0aZmY7ThqSTmtQRkRc2Z6Quoubp8zMxuVpnnpJZnsecCxJM1VvJI2ym6fMzOryNE/9RfazpD2BL7Utoi5TLIh5fQXXNMzMyLkI0yRbSJ6j6BkDlRKb3KdhZparT+NfGJ+6owC8ALi2nUF1m2TSQtc0zMzy9Gl8KrM9AjwYEUPNdp6LPD26mVkiT5/Gv81EIN1soFL0E+FmZuRrnnqaxjPLCoiI2GPao+oy1UqJxzcPdzoMM7OOy9M89Vngf0hGTAk4Bdg9Ij7RzsC6SbVc4tePb9nxjmZmc1ye0VPHRcQ/R8TTEfFURFwC/Em7A+sm1UrRfRpmZuRLGqOSTpFUlFSQdArQU+NPq5WSp0Y3MyNf0jgZeCvwCPAoyXoaJ7czqG5TLZfYPDxCumaUmVnPyjN66gHghPaH0r2qlRK1gN9tGx1b/tXMrBc1rWlI+jNJS9JtSVoh6beS1kg6fOZC7LwBz3RrZgZM3Tx1LvBAun0ScChwEPAe4B/bG1Z3qdcu3BluZr1uqqQxEhHb0u0/Aq6MiI0R8QOg2v7Qukd9enQ/4GdmvW6qpFGTNF9SfTr0H2S+2629YXWXgTRpbBl285SZ9bapksaHgFUkTVQrI2ItgKT/A9yf5+RpP8ijku7KlO0j6SZJ96bve6flknSRpHWT+00kLUv3v1fSsp2/zV3TP9an4ZqGmfW2pkkjIr4FPAf4/Yj4s8xXq4C35Tz/F4HjJ5V9ALg5IpYAN6efAV5HMuX6EmA5cAkkSQb4MHAkcATw4XqimSkDbp4yMwN28JxGRIxExBOTyjZHxKY8J4+IW4DHJxWfAFyRbl8BvDFTfmUkfgLsJWk+cBxwU0Q8nsZyE9snoraqjjVPOWmYWW9rZRGmXbV/RKwHSN+flZYvAB7K7DeUljUr346k5ZJWSVq1YcOGaQu4Wk6ap7wQk5n1uk4kjWbUoCymKN++MOKyiFgaEUsHBwenLbB6TcN9GmbW63I93ixpAUn/xtj+adNTKx6RND8i1qfNT4+m5UPAAZn9FgIPp+WvnlT+oxav3ZK+YoFyqcBmN0+ZWY/Ls57Gx0k6vu9mfKLCAFpNGiuBZcCF6fs3M+VnS7qGpNP7t2li+R7wsUzn92uB81q8dsuqZc90a2aWp6bxRuD5EbF1Z08u6WqSWsJ+koZIRkFdCFwr6R3Ar0kmQAS4EXg9sA7YApwBEBGPS/pb4GfpfhdExOTO9barVkqeRsTMel6epHE/0AfsdNKIiJOafHVsg30DOKvJeVYAK3b2+tNpoOJ1ws3M8iSNLcBqSTeTSRwRcU7boupC/eWi+zTMrOflSRor01dPq1ZKPPWMk4aZ9bY862lcsaN9esFApcT//PaZTodhZtZReUZPLQH+HngBMK9eHhEHtTGurtNfdp+GmVmeh/u+QDIP1AhwNHAl8KV2BtWNBipFNnuWWzPrcXmSxm4RcTOgiHgwIj4CHNPesLpPNR095XXCzayX5ekIf0ZSAbhX0tnAbxifL6pnVCslRmrB1pEa8/qKnQ7HzKwj8tQ03g30A+cALwbeTvIkd0+pT1rohZjMrJflGT31MwBJERFntD+k7pSdtHCfarnD0ZiZdcYOaxqSXibpbuCe9POhkv657ZF1Ga8TbmaWr3nqH0gWQtoIEBG/AF7VzqC6kRdiMjPLuZ5GRDw0qajnGvYHKl6Iycwsz+iphyQdBYSkMkmH+D3tDav79Je9EJOZWZ6axjtJZp9dQLIg0mE0mY12Lhvw6n1mZrlGTz0GnDIDsXQ1L/lqZjZF0pB00VQH9uLU6ICnEjGznjZVTeOdwF3AtSRrdWtGIupSlVKBUkGuaZhZT5sqacwnWYr1bSSTFX4VuC4inpiJwLqNpLH5p8zMelXTjvCI2BgRl0bE0cDpwF7AWkmnzlRw3aZaLnrIrZn1tDxPhB9OMv/U24HvALfvygUlPV/S6szrKUnvlvQRSb/JlL8+c8x5ktZJ+qWk43bl+ruiWin54T4z62lTdYSfD/wRyTMZ1wDnRcQu/40ZEb8kGbaLpCLJrLk3AGcAn42IT02K4wXAicDBwLOBH0h6XkTM+D/5q5WSpxExs542VZ/G3wD3A4emr49JgqRDPCLikGm4/rHAfRHxYHruRk4AromIrcCvJK0DjgBunYbr75Rqpeg+DTPraVMljQNn4PonAldnPp8t6TRgFfDetNN9AfCTzD5Dadl2JC0HlgMsWrRo2oOtlkts3DQ87ec1M5stpuoIf3Cq165eOJ2S5A3A19KiS4DnkjRdrQc+Xd+1UXhNYr4sIpZGxNLBwcFdDXE7A26eMrMel2vCwjZ5HXBHRDwCEBGPRMRoRNSAy0maoCCpWRyQOW4hyXMjM67fzVNm1uM6mTROItM0JWl+5rs3kTxYCLASOFFSRdKBwBLgpzMWZUa1UvIT4WbW0/IMuT03T9nOkNQPvAa4PlP8CUl3SloDHA38JUBErCV5Kv1u4LvAWZ0YOQUwUC4xPFJj22itE5c3M+u4PFOjLwP+cVLZ6Q3KcouILcC+k8qaPjQYER8FPtrq9aZLf2bSwr36veSrmfWeqZ7TOAk4GThQ0srMV7uTruLXa+oLMW0eHmWv/g4HY2bWAVPVNP6TZBTTfoyPZAJ4GljTzqC6ladHN7Ne1zRppMNqHwReJml/4CXpV/dMx5Phs1E1Xb3Pw27NrFfl6Qh/C8lopbcAbwVuk/Sn7Q6sG9VrGls8aaGZ9ag8HeF/DbwkIh4FkDQI/AD4ejsD60bVtE/DNQ0z61V5ntMo1BNGamPO4+acevOU+zTMrFflqWl8V9L3GH8Q723Aje0LqXuNNU95enQz61E7TBoR8T5JbwZeQTIP1GURcUPbI+tCA5V6R7j7NMysN+WpaQD8B7CNZKLAjkzh0Q3m9RUoyM1TZta78oyeeitJovhTenz0lCSq5RKb3TxlZj0qT03jr/DoqTHVSsk1DTPrWR49tZOS6dHdp2FmvanV0VPfaV9I3W2g4uYpM+tdHj21k/rLXojJzHpXrtFTEXE96doXkoqSTomIq9oaWZcaqJT4zZPPdDoMM7OOaNo3IWkPSedJuljSa5U4G7ifZBRVT6pWSn64z8x61lQ1jS8BTwC3Av8XeB9QBk6IiNUzEFtX6i979JSZ9a6pksZBEfG/ASR9DngMWBQRT89IZF1qoFL0hIVm1rOmGjq7rb6Rrsn9q15PGJA0Tz2zrcZoLTodipnZjJsqaRwq6an09TRwSH1b0lO7emFJD0i6U9JqSavSsn0k3STp3vR977Rcki6StE7SGkmH7+r1WzU20637NcysBzVNGhFRjIg90tfuEVHKbO8xTdc/OiIOi4il6ecPADdHxBLg5vQzwOuAJelrOXDJNF1/p3nJVzPrZd32ZPcJwBXp9hXAGzPlV0biJ8BekuZ3IsD6Qkx+KtzMelEnk0YA35d0u6Tladn+EbEeIH1/Vlq+AHgoc+xQWjaBpOWSVklatWHDhrYE7YWYzKyX5Z0avR1eHhEPS3oWcJOk/5piXzUo264nOiIuAy4DWLp0aVt6qt08ZWa9rGM1jYh4OH1/FLgBOAJ4pN7slL7XJ0ocAg7IHL4QeHjmoh1XX4hp87Cbp8ys93QkaUiqStq9vg28FrgLWAksS3dbBnwz3V4JnJaOonop8Nt6M9ZM6x/r03BNw8x6T6eap/YHbpBUj+ErEfFdST8DrpX0DuDXwFvS/W8EXg+sA7YAZ8x8yInxJV+dNMys93QkaUTE/cChDco3Asc2KA/grBkIbYfqfRqef8rMelG3Dbntev19SfPUJg+5NbMe5KSxkwoFeU0NM+tZThot8PToZtarnDRaUC0X3TxlZj3JSaMF1YrX1DCz3uSk0QInDTPrVU4aLaiWi54a3cx6kpNGC5Kahvs0zKz3OGm0YMDNU2bWo5w0WtBfdtIws97kpNGCgUqRLdtGqXmdcDPrMU4aLahWSkTA77a5X8PMeouTRgv6vRCTmfUoJ40WDNTX1PBCTGbWY5w0WuB1ws2sVzlptKDqhZjMrEc5abTACzGZWa9y0mhBvU/DM92aWa9x0mhBv/s0zKxHzXjSkHSApB9KukfSWknnpuUfkfQbSavT1+szx5wnaZ2kX0o6bqZjnqzqIbdm1qNKHbjmCPDeiLhD0u7A7ZJuSr/7bER8KruzpBcAJwIHA88GfiDpeRHRsbahajkdcuvmKTPrMTNe04iI9RFxR7r9NHAPsGCKQ04AromIrRHxK2AdcET7I22uVCxQKRU8PbqZ9ZyO9mlIWgy8CLgtLTpb0hpJKyTtnZYtAB7KHDZEkyQjabmkVZJWbdiwoU1RJzzTrZn1oo4lDUkDwHXAuyPiKeAS4LnAYcB64NP1XRsc3nCmwIi4LCKWRsTSwcHBNkQ9zqv3mVkv6kjSkNRHkjCuiojrASLikYgYjYgacDnjTVBDwAGZwxcCD89kvI30l4secmtmPacTo6cEfB64JyI+kymfn9ntTcBd6fZK4ERJFUkHAkuAn85UvM0MVEp+uM/Mek4nRk+9HDgVuFPS6rTsg8BJkg4jaXp6APhzgIhYK+la4G6SkVdndXLkVF21UuLJLcOdDsPMbEbNeNKIiB/TuJ/iximO+Sjw0bYF1YJqpcjQE65pmFlv8RPhLaqWS2zx1Ohm1mOcNFpUrZQ8y62Z9ZxO9GnMCdVKkc1bR/jvR56mXCzQVyrQVxTlYoFyqUBfsUCpIJJ+fzOzucFJo0X7VivUAl772Vum3K9cTJJJpa9IuVig0lcYe6+UipntAuX0czlNQH3FJPmU69uliZ+LBU14lQqiIFEqpu+FAoUC9KVPsFdKRSqlAvP6kvd6DMWCE5uZ5eOk0aKTjljEc/bt55ltNYZHR9k2EgyP1hgeqbFtNHkNjwbDI+NlW0dGGR6psTUtq78/sXl4Qln9+G2j4+dsp1JBaRIpUiqMJ6RSMUlExUKSxIoF0VeY+F09sSWfk/3q2+VSYWyfcqmeGAtpIk2301clTYoFpUlQolAgs528FwvJdl963lImuTr5mbWfk0aLdisXOfb395+Ra0UEo7UYSyLb0kQyWkvKR9Pvs6+RzHZ9/60jSeLaOlJj67b0vV62rcYzI6OMjCbHjozW0vf6ucY/j9ZibN96gqt/V98eK0+PnwlSUqvqK4i+UmEsiY0ll0KBvpImlRe2S0ClgpLyBokw2S5kEhljiWxycqvX/kpj5xw/RzFzzVJh4vf144rFTJJOr2HWaU4as4DSJqdSEXaj2OlwdlqtltaY0uRVr3mN1bYyNbTRWlCLYLRGZnv8PZsUR9La2LZajW0jwUgtOdfIaPLdcD2JjdbYlt0/3Wd4pMbm4dG0PCnbVkvfR5PzZRPh6Awlv2akpFZYKkxMcOPJR9vV+oqFSYloUnNmMU1Yhcz39YRVVGafoibU+gr1ZCmltUMy2+Pnm1xz3T7mwlizqsTYuQsSIvmzX/9cyNY+M8nZ/YYzy0nD2q5QEPMKReb1zb6ElxUR47WtejJrUNPLJriRTG2vXgMbnVQry9bssgkxe/xImsS2jSa1volJbfsEV6/p1Wrwu9HR8dri6MTaaf28ozWoRYPrdjhR5lEQY/132VpePamNJbf0+wlNnWm5SJITaZISjCUypQlsYrJtnJDr/YnFbD28qVYAAAX0SURBVCIcS2715Fc/dzYhJtcpFpp/z1giTeJVGmf9M+m+u/UV+cND5k/1n2yXOGmY5SQpbdbqdCQzqzYhidSoBWPJshZBrUayPalmmG2yrDevNkpuo7UgIkla9fda/TNJsk6uR8Mk3ayJdmJMjCfz+vnS7yJzrVok14uAILm3IIlneKQ2IdnW77H+32W03rRbiyTmNN4J9zN2f0lZO+w3UGlr0lBE9/9LohWSNgAPtnj4fsBj0xhOp821+4G5d09z7X5g7t3TXLsf2P6enhMRU04RPmeTxq6QtCoilnY6juky1+4H5t49zbX7gbl3T3PtfqC1e/IT4WZmlpuThpmZ5eak0dhlnQ5gms21+4G5d09z7X5g7t3TXLsfaOGe3KdhZma5uaZhZma5OWmYmVluThoZko6X9EtJ6yR9oNPxTAdJD0i6U9JqSas6HU8rJK2Q9KikuzJl+0i6SdK96fvenYxxZzS5n49I+k36O62W9PpOxrgzJB0g6YeS7pG0VtK5afls/o2a3dOs/J0kzZP0U0m/SO/n/LT8QEm3pb/RVyWVd3gu92kkJBWB/wZeAwwBPwNOioi7OxrYLpL0ALA0ImbtQ0mSXgVsAq6MiBemZZ8AHo+IC9MEv3dEvL+TcebV5H4+AmyKiE91MrZWSJoPzI+IOyTtDtwOvBE4ndn7GzW7p7cyC38nJRN0VSNik6Q+4MfAucB7gOsj4hpJlwK/iIhLpjqXaxrjjgDWRcT9ETEMXAOc0OGYDIiIW4DHJxWfAFyRbl9B8j/0rNDkfmatiFgfEXek208D9wALmN2/UbN7mpUisSn92Je+AjgG+Hpanus3ctIYtwB4KPN5iFn8hyQjgO9Lul3S8k4HM432j4j1kPwPDjyrw/FMh7MlrUmbr2ZNU06WpMXAi4DbmCO/0aR7gln6O0kqSloNPArcBNwHPBkR9XWrc/2d56QxrtH8ynOh7e7lEXE48DrgrLRpxLrPJcBzgcOA9cCnOxvOzpM0AFwHvDsinup0PNOhwT3N2t8pIkYj4jBgIUnLyu832m1H53HSGDcEHJD5vBB4uEOxTJuIeDh9fxS4geQPy1zwSNruXG9/frTD8eySiHgk/Z+6BlzOLPud0nby64CrIuL6tHhW/0aN7mm2/04AEfEk8CPgpcBekuqznef6O89JY9zPgCXpaIIycCKwssMx7RJJ1bQTD0lV4LXAXVMfNWusBJal28uAb3Ywll1W/8s19SZm0e+UdrJ+HrgnIj6T+WrW/kbN7mm2/k6SBiXtlW7vBvwBST/ND4E/TXfL9Rt59FRGOnzuH4AisCIiPtrhkHaJpINIaheQrJ3yldl4T5KuBl5NMo3zI8CHgW8A1wKLgF8Db4mIWdG53OR+Xk3S5BHAA8Cf1/sDup2kVwD/DtwJ1Be0/yBJH8Bs/Y2a3dNJzMLfSdIhJB3dRZLKwrURcUH6d8Q1wD7Az4G3R8TWKc/lpGFmZnm5ecrMzHJz0jAzs9ycNMzMLDcnDTMzy81Jw8zMcnPSMGuRpNHMbKerp3NmZEmLs7PgmnWL0o53MbMmfpdOy2DWM1zTMJtm6RomH0/XL/ippP+Vlj9H0s3pZHc3S1qUlu8v6YZ0rYNfSDoqPVVR0uXp+gffT5/kNesoJw2z1u02qXnqbZnvnoqII4CLSWYZIN2+MiIOAa4CLkrLLwL+LSIOBQ4H1qblS4B/ioiDgSeBP2nz/ZjtkJ8IN2uRpE0RMdCg/AHgmIi4P5307n8iYl9Jj5Es7LMtLV8fEftJ2gAszE7fkE7HfVNELEk/vx/oi4i/a/+dmTXnmoZZe0ST7Wb7NJKdA2gU90FaF3DSMGuPt2Xeb023/5Nk9mSAU0iW3AS4GXgXjC2Us8dMBWm2s/wvF7PW7ZauhFb33YioD7utSLqN5B9mJ6Vl5wArJL0P2ACckZafC1wm6R0kNYp3kSzwY9Z13KdhNs3SPo2lEfFYp2Mxm25unjIzs9xc0zAzs9xc0zAzs9ycNMzMLDcnDTMzy81Jw8zMcnPSMDOz3P4/Aow10J9+HaYAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "# The following variables are the hyperparameters.\n", + "learning_rate = 0.01\n", + "epochs = 30\n", + "batch_size = 30\n", + "\n", + "# Specify the feature and the label.\n", + "my_feature = \"total_rooms\" # the total number of rooms on a specific city block.\n", + "my_label=\"median_house_value\" # the median value of a house on a specific city block.\n", + "# That is, you're going to create a model that predicts house value based \n", + "# solely on total_rooms. \n", + "\n", + "# Discard any pre-existing version of the model.\n", + "my_model = None\n", + "\n", + "# Invoke the functions.\n", + "my_model = build_model(learning_rate)\n", + "weight, bias, epochs, rmse = train_model(my_model, training_df, \n", + " my_feature, my_label,\n", + " epochs, batch_size)\n", + "\n", + "print(\"\\nThe learned weight for your model is %.4f\" % weight)\n", + "print(\"The learned bias for your model is %.4f\\n\" % bias )\n", + "\n", + "plot_the_model(weight, bias, my_feature, my_label)\n", + "plot_the_loss_curve(epochs, rmse)" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [], + "source": [ + "def predict_house_values(n, feature, label):\n", + " \"\"\"Predict house values based on a feature.\"\"\"\n", + "\n", + " batch = training_df[feature][10000:10000 + n]\n", + " predicted_values = my_model.predict_on_batch(x=batch)\n", + "\n", + " print(\"feature label predicted\")\n", + " print(\" value value value\")\n", + " print(\" in thousand$ in thousand$\")\n", + " print(\"--------------------------------------\")\n", + " for i in range(n):\n", + " print (\"%5.0f %6.0f %15.0f\" % (training_df[feature][10000 + i],\n", + " training_df[label][10000 + i],\n", + " predicted_values[i][0] ))" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "feature label predicted\n", + " value value value\n", + " in thousand$ in thousand$\n", + "--------------------------------------\n", + " 1960 53 156\n", + " 3400 92 173\n", + " 3677 69 176\n", + " 2202 62 158\n", + " 2403 80 161\n", + " 5652 295 199\n", + " 3318 500 172\n", + " 2552 342 163\n", + " 1364 118 149\n", + " 3468 128 173\n" + ] + } + ], + "source": [ + "predict_house_values(10, my_feature, my_label)" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "WARNING:tensorflow:Falling back from v2 loop because of error: Failed to find data adapter that can handle input: , \n", + "Train on 17000 samples\n", + "Epoch 1/20\n", + "17000/17000 [==============================] - 1s 32us/sample - loss: 38299.0813 - root_mean_squared_error: 195.7014\n", + "Epoch 2/20\n", + "17000/17000 [==============================] - 0s 28us/sample - loss: 19076.5906 - root_mean_squared_error: 138.1180\n", + "Epoch 3/20\n", + "17000/17000 [==============================] - 1s 57us/sample - loss: 15098.0711 - root_mean_squared_error: 122.8742\n", + "Epoch 4/20\n", + "17000/17000 [==============================] - 1s 36us/sample - loss: 14804.1275 - root_mean_squared_error: 121.6722\n", + "Epoch 5/20\n", + "17000/17000 [==============================] - 0s 28us/sample - loss: 14614.2507 - root_mean_squared_error: 120.8894\n", + "Epoch 6/20\n", + "17000/17000 [==============================] - 0s 29us/sample - loss: 14447.7099 - root_mean_squared_error: 120.1986\n", + "Epoch 7/20\n", + "17000/17000 [==============================] - 1s 52us/sample - loss: 14253.2030 - root_mean_squared_error: 119.3868\n", + "Epoch 8/20\n", + "17000/17000 [==============================] - 1s 45us/sample - loss: 14136.6367 - root_mean_squared_error: 118.8976\n", + "Epoch 9/20\n", + "17000/17000 [==============================] - 1s 32us/sample - loss: 14021.1195 - root_mean_squared_error: 118.4108\n", + "Epoch 10/20\n", + "17000/17000 [==============================] - 1s 30us/sample - loss: 13926.0850 - root_mean_squared_error: 118.0088\n", + "Epoch 11/20\n", + "17000/17000 [==============================] - 1s 56us/sample - loss: 13796.5251 - root_mean_squared_error: 117.4586\n", + "Epoch 12/20\n", + "17000/17000 [==============================] - 1s 44us/sample - loss: 13792.4641 - root_mean_squared_error: 117.4413\n", + "Epoch 13/20\n", + "17000/17000 [==============================] - 1s 45us/sample - loss: 13754.1693 - root_mean_squared_error: 117.2782\n", + "Epoch 14/20\n", + "17000/17000 [==============================] - 1s 38us/sample - loss: 13685.0884 - root_mean_squared_error: 116.9833\n", + "Epoch 15/20\n", + "17000/17000 [==============================] - 1s 40us/sample - loss: 13632.7631 - root_mean_squared_error: 116.7594\n", + "Epoch 16/20\n", + "17000/17000 [==============================] - 1s 61us/sample - loss: 13600.9446 - root_mean_squared_error: 116.6231\n", + "Epoch 17/20\n", + "17000/17000 [==============================] - 1s 31us/sample - loss: 13529.1607 - root_mean_squared_error: 116.3150\n", + "Epoch 18/20\n", + "17000/17000 [==============================] - 1s 44us/sample - loss: 13489.0530 - root_mean_squared_error: 116.1424\n", + "Epoch 19/20\n", + "17000/17000 [==============================] - 1s 41us/sample - loss: 13430.6068 - root_mean_squared_error: 115.8905\n", + "Epoch 20/20\n", + "17000/17000 [==============================] - 1s 29us/sample - loss: 13432.8901 - root_mean_squared_error: 115.9004\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de3zcdZ3v8dcnk8skzaWXpEmaNBQQdAHbWmsrVRFhD6CLdF1v0CognuWgsOB69ChHF5U9eERWXVlWWdDKotyPqBV18bryULlsgVJbQCmXtqH3C03bNJdJPueP32+m03SSTJP8ZiYz7+fj8XvMb76/38zvk+k0n3x/35u5OyIiIgBl+Q5AREQKh5KCiIikKCmIiEiKkoKIiKQoKYiISEp5vgMYj8bGRp8zZ85Rv27vwX427u7mVTNrqa6ITXxgIiIF7PHHH9/p7k2Zjk3qpDBnzhxWrVp11K97YuMe/uYbf+BfLl7IGa9pjiAyEZHCZWYbhjtWkrePWurjAGzd25vnSERECktJJoWmuirKDLZ29eQ7FBGRglKSSaEiVkZjbRVb9x7MdygiIgVlUrcpjEdLQ5ytXbp9JFKq+vv76ezspKeneO8YxONx2tvbqaioyPo1JZsUmuvjbNzVne8wRCRPOjs7qaurY86cOZhZvsOZcO7Orl276Ozs5Nhjj836dSV5+wigtSGuNgWREtbT08OMGTOKMiEAmBkzZsw46ppQySaF5vo4ew/2c7BvIN+hiEieFGtCSBrLz1eySSHVLVW1BRGRlJJNCq0NybEKSgoikh+1tbX5DuEIJZsUmpNJoUvdUkVEkiJLCma2wsy2m9natLJ5Zvawmf3RzH5sZvVpx642s/Vm9iczOzuquJI0qllECtGGDRs488wzmTt3LmeeeSYbN24E4L777uOUU05h3rx5nHbaaQCsW7eORYsWMX/+fObOnctzzz037utH2SX1NuAm4Pa0sm8Bn3D335rZJcAngX8ws5OA84GTgVnAL83sRHePrBV4SlU5dVXlbFObgkjJ+8KP1/H05q4Jfc+TZtXzuXeefNSvu+KKK7jwwgu56KKLWLFiBVdeeSU//OEPufbaa3nwwQdpa2vjlVdeAeDmm2/mqquuYvny5fT19TEwMP5fmZHVFNz9IWD3kOJXAw+F+78A3h3uLwXudvded38RWA8siiq2pJaGuNoURKSgPPzwwyxbtgyAD37wg/zud78D4E1vehMXX3wxt956a+qX/6mnnsoXv/hFrr/+ejZs2EB1dfW4r5/rwWtrgfOAHwHvBWaH5W3AI2nndYZlRzCzS4FLATo6OsYVTEtDnC2qKYiUvLH8RZ8ryW6lN998M48++ig/+clPmD9/PqtXr2bZsmUsXryYn/zkJ5x99tl861vf4owzzhjX9XLd0HwJcLmZPQ7UAX1heabOtJ7pDdz9Fndf6O4Lm5oyTgeeteb6ONtUUxCRArJkyRLuvvtuAO644w7e/OY3A/D888+zePFirr32WhobG9m0aRMvvPACxx13HFdeeSXnnXcea9asGff1c1pTcPdngbMAzOxE4K/CQ50cqjUAtAObo46ntSHOjv29JAYGKY+VbEcsEcmT7u5u2tvbU88//vGPc+ONN3LJJZdwww030NTUxHe+8x0APvnJT/Lcc8/h7px55pnMmzePL33pS3zve9+joqKClpYWrrnmmnHHlNOkYGYz3X27mZUBnwVuDg+tBO40s68SNDSfADwWdTzN9XEGBp2d+/toCbuoiojkyuDgYMbyX//610eU3X///UeUXX311Vx99dUTGlOUXVLvAh4GXm1mnWb2YeACM/sz8CxBTeA7AO6+DrgXeBr4D+DyKHseJWlUs4jI4SKrKbj7BcMc+vow518HXBdVPJm0pI9qnj3KySIiJaCkb6Qnk4LGKoiUJveM/VmKxlh+vpJOCtNrKqmIGVvUA0mk5MTjcXbt2lW0iSG5nkI8fnTtpSW7yA5AWZkxsy6umoJICWpvb6ezs5MdO3bkO5TIJFdeOxolnRRAo5pFSlVFRcVRrUhWKkr69hEk12pWUhARASUFWuqDmkKx3lcUETkaSgr1cQ72D9DVk8h3KCIieaekoG6pIiIpSgphUlC3VBERJYXUVBeaLVVEREmBmfVVgOY/EhEBJQWqymPMmFKppCAigpICEEyhrQFsIiJKCoBGNYuIJCkpECQFdUkVEVFSAIIeSLsO9NGbiHxdHxGRgqakwKFuqdu7evMciYhIfkW5HOcKM9tuZmvTyuab2SNmttrMVpnZorDczOxGM1tvZmvMbEFUcWXS3KBlOUVEINqawm3AOUPKvgx8wd3nA9eEzwHeDpwQbpcC34wwriO0pi/LKSJSwiJLCu7+ELB7aDFQH+43AJvD/aXA7R54BJhqZq1RxTZUc72SgogI5H6RnY8BD5rZPxEkpCVheRuwKe28zrBsy9A3MLNLCWoTdHR0TEhQ9fFyqitiun0kIiUv1w3NHwH+3t1nA38PfDsstwznZlzgwN1vcfeF7r6wqalpQoIyM1q12I6ISM6TwkXA/eH+fcCicL8TmJ12XjuHbi3lhEY1i4jkPilsBt4a7p8BPBfurwQuDHshvRHY6+5H3DqKkkY1i4hE2KZgZncBpwONZtYJfA74W+DrZlYO9BC2DQA/Bd4BrAe6gQ9FFddwmuvjbN/Xw+CgU1aW6W6WiEjxiywpuPsFwxx6fYZzHbg8qliy0doQp3/A2d3dR2NtVT5DERHJG41oDqlbqojIKEnBzGJm9r1cBZNPLRrAJiIyclJw9wGgycwqcxRP3iTnP1K3VBEpZdm0KbwE/N7MVgIHkoXu/tWogsqHproqYmWmmoKIlLRsksLmcCsD6qINJ39iZUZTbZVqCiJS0kZNCu7+BQAzqwue+v7Io8qTZi22IyIlbtTeR2Z2ipk9CawF1pnZ42Z2cvSh5V6rRjWLSInLpkvqLcDH3f0Ydz8G+J/ArdGGlR8a1SwipS6bpDDF3X+TfOLu/wlMiSyiPGquj7OvN8GB3kS+QxERyYtsksILZvYPZjYn3D4LvBh1YPnQ0hCMZFZjs4iUqmySwiVAE8HspvcDjeRhbqJcaKmvBmCbbiGJSIkasfeRmcWA/+3uV+YonrxKjmreoqQgIiUqmxHNR0xgV6w0qllESl02g9eeDEcz38fhI5rvH/4lk1N1ZYyG6gqNVRCRkpVNUpgO7CJYFCfJObSCWlFpqY/r9pGIlKxs2hTWuPvXchRP3mlUs4iUsmzaFM7LUSwFoaW+SgPYRKRkZdMl9Q9mdpOZvcXMFiS30V5kZivMbLuZrU0ru8fMVofbS2a2Ou3Y1Wa23sz+ZGZnj/HnGbeWhmp27O+lf2AwXyGIiORNNm0KS8LHa9PKnMPbGDK5DbgJuD31Ivf3J/fN7CvA3nD/JOB84GRgFvBLMzsxrKnkVEt9HHfYsa+XWVOrc315EZG8ymaW1LeN5Y3d/SEzm5PpmJkZ8D4OJZalwN3u3gu8aGbrgUXAw2O59nikj2pWUhCRUjPs7SMz++e0/auGHLttnNd9C7DN3Z8Ln7cBm9KOd4ZlmeK61MxWmdmqHTt2jDOMIyXXataoZhEpRSO1KZyWtn/RkGNzx3ndC4C70p5bhnM80wvd/RZ3X+juC5uamsYZxpFaG4LagbqlikgpGun2kQ2zPy5mVg78DYePlO4EZqc9bydY7S3nptVUUFlepm6pIlKSRqoplJnZNDObkbY/3cymA7FxXPMvgWfdvTOtbCVwvplVmdmxwAnAY+O4xpiZGc31WpZTRErTSDWFBuBxDtUSnkg7lvHWTjozuws4HWg0s07gc+7+bYJeRum3jnD3dWZ2L/A0kAAuz0fPo6TW+mqNVRCRkjRsUnD3OeN5Y3e/YJjyi4cpvw64bjzXnCjNDXHWdL6S7zBERHIum8FrJSc5qtl91AqRiEhRUVLIoLk+Tm9ikL0H+/MdiohITikpZKBuqSJSqoZtUwh7GQ3L3XdPfDiFIX1U81+01uc5GhGR3Bmp99HjBL2MDOgA9oT7U4GNwLGRR5cnGtUsIqVq2NtH7n6sux8HPAi8090b3X0GcC5FusBO0sy6OGZallNESk82bQpvcPefJp+4+8+At0YXUv5VlpcxY4rWVRCR0pPN1Nk7zeyzwPcIbid9gGB5zqLW0qBRzSJSerKpKVwANAE/CLemsKyotdTHVVMQkZKTzXoKu4GrzKzW3ffnIKaC0NIQ5/ENe/IdhohITo1aUzCzJWb2NMG8RJjZPDP7RuSR5VlLfZw93f309OdtCiYRkZzL5vbR14CzCdsR3P0pDl9roSiluqWqXUFESkhWI5rdfdOQoqL/8zk5qlntCiJSSrLpfbTJzJYAbmaVwJXAM9GGlX/po5pFREpFNjWFy4DLCdZM7gTmh8+LWvL2kWoKIlJKRqwpmFkM+KC7L89RPAWjLl7BlMqYagoiUlJGrCmEq58tzVEsBaelIa6GZhEpKdncPvq9md1kZm8xswXJbbQXmdkKM9tuZmuHlP+dmf3JzNaZ2ZfTyq82s/XhsbPH8LNMuJaGuKbPFpGSkk1D85Lw8dq0MgfOGOV1twE3AbcnC8zsbQQ1j7nu3mtmM8PykwjWbj4ZmAX80sxOzOc6zRC0KzzyfNHP6CEikpLNiOa3jeWN3f0hM5szpPgjwJfcvTc8Z3tYvhS4Oyx/0czWA4uAh8dy7YnSUh9n+75eBgadWJnlMxQRkZzIpqaAmf0VwV/x8WSZu187/CuGdSLwFjO7DugBPuHu/0XQs+mRtPM6w7K8am2Ikxh0du3vZWZ9fPQXiIhMctlMc3Ez8H7g7wgW2XkvcMwYr1cOTAPeCHwSuNfMLHzfoXyYeC41s1VmtmrHjh1jDCM7qW6pamwWkRKRTUPzEne/ENjj7l8ATgVmj/F6ncD9HngMGAQaw/L092wHNmd6A3e/xd0XuvvCpqamMYaRnZYGjVUQkdKSTVI4GD52m9ksoJ+xL8X5Q8IGajM7EagEdgIrgfPNrMrMjgVOAB4b4zUmTDIpqFuqiJSKbNoUHjCzqcANwBMEt3W+NdqLzOwu4HSg0cw6gc8BK4AVYTfVPuAid3dgnZndSzATawK4PN89jwAap1RRXmbqlioiJSOb3kf/GO5+38weAOLuvjeL1w23EM8Hhjn/OuC60d43l8rKjJl1WoFNRErHqEnBzC7MUIa7357p/GLTrFHNIlJCsrl99Ia0/ThwJsFtpJJICq0NcZ7dui/fYYiI5EQ2t4/+Lv25mTUA340sogLTXB/nt3+KtuuriEihyGqRnSG6CXoHlYSW+jgH+gbY19Of71BERCKXTZvCjzk0kKwMOAm4N8qgCkl6t9S6eEWeoxERiVY2bQr/lLafADa4e2dE8RSclnBU85a9PbxqZl2eoxERiVY2bQq/zUUghUqjmkWklGRz+2gfmechMsDdvX7CoyogyfmP1C1VREpBNrePvgZsJehxZMByoM7dvzziq4pEvCLGtJoKDWATkZKQTe+js939G+6+z9273P2bwLujDqyQNNfHdftIREpCNklhwMyWm1nMzMrMbDmQ93mJcqmlIa6agoiUhGySwjLgfcA2YDvBegrLogyq0LQ2xNm6tzffYYiIRC6b3kcvESyXWbKa6+Ps3N9LX2KQyvKxjPcTEZkchv0NZ2Z/a2YnhPtmZivMbK+ZrTGzBbkLMf+SYxW279MtJBEpbiP92XsV8FK4fwEwDzgO+Djw9WjDKizNWmxHRErESEkh4e7JCX/OBW53913u/ktgSvShFY7W1AA2tSuISHEbKSkMmlmrmSWny/5l2rHqaMMqLIemujg4ypkiIpPbSA3N1wCrgBiw0t3XAZjZW4EXchBbwWiorqCqvEy3j0Sk6A1bU3D3B4BjgL9w979NO7QKeP9obxw2TG8P12NOln3ezF42s9Xh9o60Y1eb2Xoz+5OZnT22HycaZhaOVdDtIxEpbiP2r3T3hLvvGVJ2wN33Z/HetwHnZCj/mrvPD7efApjZScD5wMnha75hZrFsfoBcaamPs1W3j0SkyEXW6d7dHwJ2Z3n6UuBud+919xeB9cCiqGIbC41qFpFSkI+RWFeEYx1WmNm0sKwN2JR2TmdYdgQzu9TMVpnZqh07crdMZkt9nG1dvbhnmjBWRKQ4ZJUUzKzNzJaY2WnJbYzX+yZwPDAf2AJ8JXmJDOdm/O3r7re4+0J3X9jU1DTGMI5eS0OcvsQge7q1LKeIFK9s1lO4nqBh+WkOTYTnwENHezF335b2vrcCD4RPO4HZaae2A5uP9v2jlN4tdfqUyjxHIyISjWzWU/hr4NXuPu6uN2bW6u5bwqfvApI9k1YCd5rZV4FZwAnAY+O93kRKH9V88qyGPEcjIhKNbJLCC0AFcFRJwczuAk4HGs2sE/gccLqZzSeoabwE/A8Ad19nZvcS1EYSwOXuXlDTcydrChrVLCLFLJuk0A2sNrNfkZYY3P3KkV7k7hdkKP72COdfB1yXRTx50VRXRZmhbqkiUtSySQorw62kVcTKaKytUrdUESlq2ayn8O+5CGQy0KhmESl22fQ+OgH4v8BJQDxZ7u7HRRhXQWqpj7NhV3e+wxARiUw24xS+QzC+IAG8Dbgd+G6UQRWqloa4ZkoVkaKWTVKodvdfAebuG9z988AZ0YZVmJrr43T1JDjYV1Ado0REJkw2SaHHzMqA58zsCjN7FzAz4rgKUqpbqhqbRaRIZZMUPgbUAFcCrwc+AFwUZVCF6tAKbEoKIlKcsul99F8AZubu/qHoQypcyVHNW7vUriAixWnUmoKZnWpmTwPPhM/nmdk3Io+sAGlUs4gUu2xuH/0zcDawC8DdnwLGOkvqpDalqpy6qnItyykiRSurqbPdfdOQopLtfqNuqSJSzLKZ5mKTmS0B3MwqCRqcn4k2rMKlUc0iUsyyqSlcBlxOsBJaJ8ECOZdHGVQha66Ps029j0SkSGXT+2gnsDwHsUwKrQ1xduzvJTEwSHksH6uZiohEZ9ikYGY3jvTC0abOLlbN9XEGBp2d+/toaYiP/gIRkUlkpJrCZQQro91LsDRmpnWUS076qGYlBREpNiMlhVbgvQTrMyeAe4Dvu/ueXARWqFrSRzXPHuVkEZFJZtib4u6+y91vdve3ARcDU4F1ZvbBbN7YzFaY2XYzW5vh2CfMzM2sMXxuZnajma03szVmtmBsP070DiUFdUsVkeKTzYjmBQTzH30A+BnweJbvfRtwTob3mw38N2BjWvHbgRPC7VKCqboL0vSaSipipm6pIlKUhk0KZvYFM3sc+DjwW2Chu3/Y3Z/O5o3d/SFgd4ZDXwP+F+BpZUuB2z3wCDDVzFqz/SFyqazMmFkX16hmESlKI7Up/APwAjAv3L5oZhA0OLu7zz3ai5nZecDL7v5U+F5JbUD6qOnOsGxLhve4lKA2QUdHx9GGMCFaG+KaKVVEitJISeHYibyQmdUAnwHOynQ4Q5lnKMPdbwFuAVi4cGHGc6LW3BDn6c1d+bi0iEikhk0K7r5hgq91PEGiSdYS2oEnzGwRQc0gvS9PO0E32ILUUh/n189sx90ZUuMREZnUcjYk193/6O4z3X2Ou88hSAQL3H0rsBK4MOyF9EZgr7sfceuoULTUxznYP0BXTyLfoYiITKjIkoKZ3QU8DLzazDrN7MMjnP5TgvaL9cCtwEejimsitE4NuqW+tPNAniMREZlYo859ZGZXufvXRysbyt0vGOX4nLR9ZxJNsrfk+EYqY2X84MmXmTd7ar7DERGZMNnUFDKtx3zxBMcxqUyfUsk5p7Tw/Sc6OdhXsktLiEgRGmmcwgVm9mPgWDNbmbb9hnAVtlK2bHEH+3oSPLCmYNvDRUSO2ki3j/5AME6gEfhKWvk+YE2UQU0Gi4+dzvFNU7jzsY28d6EmQRKR4jDS3Ecb3P0/3f1U4FmgLtw63b3ku92YGcsWH8OTG1/hmS0asyAixSGbuY/eCzxGMGPq+4BHzew9UQc2Gbx7QRuV5WXc+ejG0U8WEZkEsmlo/izwBne/yN0vBBYRTIFR8qbWVHLua1v54ZMv091X8pUnESkC2SSFMnffnvZ8V5avKwnLFnewrzfBj59Sg7OITH7Z/HL/DzN70MwuNrOLgZ8QDDYT4PXHTOPE5lrdQhKRojBqUnD3TwL/BswlmC31Fnf/VNSBTRZmxrJFHTzVuZe1L+/NdzgiIuOS7W2g3wO/AX4V7kuady1oJ15Rxp2PqbYgIpNbNr2P3kfQ++g9qPdRRg3VFZw7dxY/evJl9veqwVlEJq9sagqfQb2PRrVscQcH+gZYuVoNziIyean30QR53eypvKaljjse3UAwv5+IyOQz1t5HP4s2rMnHzFi+uIN1m7tY06kGZxGZnMba++h/RR3YZLT0dW1UV8TUPVVEJq2sbgO5+/3u/nF3/3tgpZktjziuSak+XsF582ax8qnNdPX05zscEZGjNtLU2fVmdrWZ3WRmZ4VLZV5BsELa+3IX4uSybHEHB/sH+NGTL+c7FBGRozZSTeG7wKuBPwL/Hfg5waR4S9196WhvbGYrzGy7ma1NK/tHM1tjZqvN7OdmNissNzO70czWh8cXjOunyqO57Q2cPKueOx7dqAZnEZl0RkoKx7n7xe7+b8AFwELgXHdfneV73wacM6TsBnef6+7zgQeAa8LytwMnhNulwDezvEbBCabU7uDZrft4ctMr+Q5HROSojJQUUjfF3X0AeNHd92X7xu7+ELB7SFn6wgNTgOSf0kuB2z3wCDDVzFqzvVahWTq/jSmVanAWkclnpKQwz8y6wm0fMDe5b2ZjXlXGzK4zs03Acg7VFNqATWmndYZlmV5/qZmtMrNVO3bsGGsYkaqtKue8+W08sGYzew+qwVlEJo+RVl6LuXt9uNW5e3nafv1YL+jun3H32cAdwBVhsWU6dZjX3+LuC919YVNT01jDiNzyxR309A/ygyc68x2KiEjW8jky+U7g3eF+J5C+0HE7MKnnizilrYG57Q3c+ZganEVk8shpUjCzE9Kenkew9jPASuDCsBfSG4G97r4ll7FFYfniDv68bT+Pb9iT71BERLISWVIws7uAh4FXm1mnmX0Y+JKZrTWzNcBZwFXh6T8lGP+wHrgV+GhUceXSO+fNoq6qXA3OIjJplEf1xu5+QYbibw9zrgOXRxVLvtRUlvPXr2vjnlWbuOadJzG1pjLfIYmIjEiznUZs2eIO+hKDfP8JjXAWkcKnpBCxv2it53UdU7lTU2qLyCSgpJADyxZ18PyOAzz24u7RTxYRySMlhRw4d+4s6uLlWsNZRAqekkIOVFfGePeCdn72x63sPtCX73BERIalpJAjyxZ30DcwyPcf1whnESlcSgo5cmJzHQuPmaYRziJS0JQUcmjZ4g5e3HmAh5/fle9QREQyUlLIoXe8tpWG6gruUIOziBQoJYUcilcEDc4/X7eVnft78x2OiMgRlBRybNni2fQPOPetUoOziBQeJYUce9XMOhYdO527HtvIHnVPFZECE9mEeDK8i5fM4aN3PMHr/vEXtE2t5pS2ek6Z1cApbQ2c3FbPzLp4vkMUkRKlpJAH73htK/dddipPbNjD2s1drH15Lw+u25Y6PrOuite2NXByWwOnzKrnlLYGWhvimGVaoE5EZOIoKeTJG+ZM5w1zpqee7+vp5+nNXazd3MW6l/eydvNefvOn7QyGQxqmT6nk5DBBvLatgZNa62mfVk15THcARWTiKCkUiLp4BYuPm8Hi42akyrr7EjyzZR/rNu9l7ct7WftyF7c+9AKJMFPEyoyW+jizp1cze1oNs6fX0D6tmtnTa5g9rYaZdVWUlal2ISLZU1IoYDWV5bz+mGm8/phpqbKe/gH+vG0fz2zpYtPug2za082m3d389s872L7v8G6ulbEy2qZVpxJF+7TDk8eMKZW6JSUih4ksKZjZCuBcYLu7nxKW3QC8E+gDngc+5O6vhMeuBj4MDABXuvuDUcU2mcUrYsxtn8rc9qlHHOvpH+DlVw6yaXc3m/YcpHNPN51h4li39sjJ+CrLy2htiNPaEGdWQzWtU+O0NFQzqyFOa0M1s6bGaaiuUOIQKSEW1Tw8ZnYasB+4PS0pnAX82t0TZnY9gLt/ysxOAu4CFgGzgF8CJ7r7wEjXWLhwoa9atSqS+IvR/t7EYYli694eNu/tYcsrB9myt4etXT0MDB7+faiuiAWJY2qYKBrCxDE1zuzpNbRNrSZeEcvTTyQiY2Fmj7v7wkzHolyj+SEzmzOk7OdpTx8B3hPuLwXudvde4EUzW0+QIB6OKr5SVFtVzmta6nlNS33G4wODzs79vWwOk0TyMUgeB/ndczvZvq+H9LxhRtCuEd6W6pheQ8eMajqmB8+baqtU0xCZRPLZpnAJcE+430aQJJI6w7IjmNmlwKUAHR0dUcZXcmJlRnN9nOb6OK8b5pzEwCDb9wWJY9OebjbuOsjG3UG7xu/X7+T7XT2HnR+vKGP2tJpUkugIt/bp1TTWVjGtppKYGsNFCkZekoKZfQZIAHckizKclvG+lrvfAtwCwe2jSAKUYZXHypg1tZpZU6tZmNalNinZrpFMFBt3dbNxd7A98sIuDvQdfkfQDBqqK5g+pZLpNZXB45RKpk2pZMaUSqbVVDK99vBjNZUx1T5EIpLzpGBmFxE0QJ/phxo0OoHZaae1A5tzHZuMX7wixvFNtRzfVHvEMXdnT3d/KmHsPtDHrgN97DnQx+5w27i7myc3vcKeA32prrdDVZWXBYmjppIZtWHiSEsm02sqmTalghlTqpg2pYJpNZVUaDyHSFZymhTM7BzgU8Bb3b077dBK4E4z+ypBQ/MJwGO5jE2iZ2apX97zZx/Zeyqdu9PVk2DP0MTR3Xd4WXeQSHYf6GNfT2LY96uPlx9RA2morqChuoL68DF9v766nIbqCqrK1YgupSXKLql3AacDjWbWCXwOuBqoAn4RVv8fcffL3H2dmd0LPE1wW+ny0XoeSXEzs9Qv6jmNU7J6TV9ikFe6g0Sx+0Afew70s/tAL7sP9LOn+1BtZPMrPax9uYuunn66+0b+msUryoIkET+UOJLJo7G2ksbaKmbUVqX2m8CWCn0AAAm/SURBVOqq1BtLJrXIuqTmgrqkynj1JQbp6uln78Fg6xryGOwnUvvp5+3rzVwzqa0qTyWJxtoqGuvS9muraKqrZMaUKspjlmobMYL2FQub14J9Uq1thh0qA8rMKI8ZleVlVMbK1MYiRyUvXVJFJoPK8rLUL+uj1dM/wK4Dfezc18vO/cmtjx1pz9fv2M8jL/bySnd/BNEfUhEzKmNlQZIIt4pYkDCqMpRVlpdRUxljSlU5UyrLqamKUVtVTk1lOVOS5VWx8HmwP6WqnKpyJaBip6QgMkbxihhtU6tpm1o96rn9A4PsPnAoYew+0EdiIKilO477oe52wX5QFhwPCtOPD7qTGHD6BgbpTQzSPzBIXyJtGwi3tLKe/kH29SToSwSvOdg3wIHeBAf6EgzTpn+EWJkFyaSynOrKGFXlZcQrYlRXxIhXlFFdGSNeHqMqrSw+ZD+5BUnKqIiVhdvo++q+HD0lBZEcqIiVpcaAFBp3pzcxyP7eBN29AxzoS4TJYoDu8DGZPLp7B4Lz+hIc7B+kp38gte3cn6Cnf4CD/QP09A/SG+4P14tsLMos6BZdGQsSUG1YowlqM8FWm/a8tqo8Ves5vCxIXKlaVFijUi1ISUGk5JlZ6q93juxJPG6JgUF6EkECOdg3QG8iSBp9A4P0JwbpH3D6B9P2BwbD7cj9xMAgfQNOX2KQg/1hsupNsL83wfZ9PRzYGSStA72JUTsRZFIRM6rKY6lEUZmeNCoOlaXXkGoqY8QrY9RUlFNdWUZ1ZXmqvLoiRnVl2nnJ8soYZUMS0KGaoWcoSz/Pw1jLIunUoKQgIpEqj5VRGyujtiq3v24GB53uMHHsT0seB3qDst7EQOpWWt/AIL39h99ySx5PlvWG276eBDsTh2pC3X3BY19iMKc/32VvPZ5Pv/01E/6+SgoiUpTKyoza8HZRcw6ul6wRdfcl6OkbpLs/wcG+oHaUnjySzwfTen6m9zoLnh9idvg5ybJMMyVPBCUFEZEJkK8a0UTT2H8REUlRUhARkRQlBRERSVFSEBGRFCUFERFJUVIQEZEUJQUREUmZ1FNnm9kOYMMYX94I7JzAcCZaoccHhR+j4hsfxTc+hRzfMe7elOnApE4K42Fmq4abT7wQFHp8UPgxKr7xUXzjU+jxDUe3j0REJEVJQUREUko5KdyS7wBGUejxQeHHqPjGR/GNT6HHl1HJtimIiMiRSrmmICIiQygpiIhIStEnBTM7x8z+ZGbrzezTGY5Xmdk94fFHzWxODmObbWa/MbNnzGydmV2V4ZzTzWyvma0Ot2tyFV94/ZfM7I/htVdlOG5mdmP4+a0xswU5jO3VaZ/LajPrMrOPDTkn55+fma0ws+1mtjatbLqZ/cLMngsfpw3z2ovCc54zs4tyGN8NZvZs+G/4AzPLuILLaN+HCOP7vJm9nPbv+I5hXjvi//cI47snLbaXzGz1MK+N/PMbN3cv2g2IAc8DxwGVwFPASUPO+Shwc7h/PnBPDuNrBRaE+3XAnzPEdzrwQB4/w5eAxhGOvwP4GcFiUW8EHs3jv/VWgkE5ef38gNOABcDatLIvA58O9z8NXJ/hddOBF8LHaeH+tBzFdxZQHu5fnym+bL4PEcb3eeATWXwHRvz/HlV8Q45/BbgmX5/feLdiryksAta7+wvu3gfcDSwdcs5S4N/D/f8HnGk2ZEXtiLj7Fnd/ItzfBzwDtOXi2hNoKXC7Bx4BpppZax7iOBN43t3HOsJ9wrj7Q8DuIcXp37N/B/46w0vPBn7h7rvdfQ/wC+CcXMTn7j9390T49BGgfaKvm61hPr9sZPP/fdxGii/83fE+4K6Jvm6uFHtSaAM2pT3v5Mhfuqlzwv8Ue4EZOYkuTXjb6nXAoxkOn2pmT5nZz8zs5JwGBg783MweN7NLMxzP5jPOhfMZ/j9iPj+/pGZ33wLBHwPAzAznFMpneQlB7S+T0b4PUboivL21Ypjbb4Xw+b0F2Obuzw1zPJ+fX1aKPSlk+ot/aB/cbM6JlJnVAt8HPubuXUMOP0FwS2Qe8C/AD3MZG/Amd18AvB243MxOG3K8ED6/SuA84L4Mh/P9+R2NQvgsPwMkgDuGOWW070NUvgkcD8wHthDcohkq758fcAEj1xLy9fllrdiTQicwO+15O7B5uHPMrBxoYGxV1zExswqChHCHu98/9Li7d7n7/nD/p0CFmTXmKj533xw+bgd+QFBFT5fNZxy1twNPuPu2oQfy/fml2Za8rRY+bs9wTl4/y7Bh+1xguYc3wIfK4vsQCXff5u4D7j4I3DrMdfP9+ZUDfwPcM9w5+fr8jkaxJ4X/Ak4ws2PDvybPB1YOOWclkOzl8R7g18P9h5ho4f3HbwPPuPtXhzmnJdnGYWaLCP7NduUovilmVpfcJ2iMXDvktJXAhWEvpDcCe5O3SXJo2L/O8vn5DZH+PbsI+FGGcx4EzjKzaeHtkbPCssiZ2TnAp4Dz3L17mHOy+T5EFV96O9W7hrluNv/fo/SXwLPu3pnpYD4/v6OS75buqDeC3jF/JuiV8Jmw7FqCLz9AnOC2w3rgMeC4HMb2ZoLq7Rpgdbi9A7gMuCw85wpgHUFPikeAJTmM77jwuk+FMSQ/v/T4DPjX8PP9I7Awx/++NQS/5BvSyvL6+REkqC1AP8Ffrx8maKf6FfBc+Dg9PHch8K20114SfhfXAx/KYXzrCe7HJ7+HyR55s4CfjvR9yFF83w2/X2sIftG3Do0vfH7E//dcxBeW35b83qWdm/PPb7ybprkQEZGUYr99JCIiR0FJQUREUpQUREQkRUlBRERSlBRERCRFSUFkBGY2MGQm1gmbedPM5qTPtClSCMrzHYBIgTvo7vPzHYRIrqimIDIG4bz415vZY+H2qrD8GDP7VThx26/MrCMsbw7XKXgq3JaEbxUzs1stWE/j52ZWnbcfSgQlBZHRVA+5ffT+tGNd7r4IuAn457DsJoKpxOcSTCp3Y1h+I/BbDybmW0AwohXgBOBf3f1k4BXg3RH/PCIj0ohmkRGY2X53r81Q/hJwhru/EE5quNXdZ5jZToIpGPrD8i3u3mhmO4B2d+9Ne485BOsnnBA+/xRQ4e7/J/qfTCQz1RRExs6H2R/unEx60/YHUDuf5JmSgsjYvT/t8eFw/w8Es3MCLAd+F+7/CvgIgJnFzKw+V0GKHA39VSIysuohi7D/h7snu6VWmdmjBH9cXRCWXQmsMLNPAjuAD4XlVwG3mNmHCWoEHyGYaVOkoKhNQWQMwjaFhe6+M9+xiEwk3T4SEZEU1RRERCRFNQUREUlRUhARkRQlBRERSVFSEBGRFCUFERFJ+f/wm+tohBPnHgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "feature label predicted\n", + " value value value\n", + " in thousand$ in thousand$\n", + "--------------------------------------\n", + " 2 53 188\n", + " 2 92 200\n", + " 2 69 194\n", + " 2 62 210\n", + " 1 80 185\n", + " 2 295 225\n", + " 2 500 209\n", + " 2 342 223\n", + " 4 118 288\n", + " 2 128 213\n", + " 2 187 223\n", + " 3 80 234\n", + " 2 112 224\n", + " 2 95 218\n", + " 2 69 210\n" + ] + } + ], + "source": [ + "# Define a synthetic feature named rooms_per_person\n", + "training_df[\"rooms_per_person\"] = training_df[\"total_rooms\"]/training_df[\"population\"] # write your code here.\n", + "\n", + "# Don't change the next line.\n", + "my_feature = \"rooms_per_person\"\n", + "\n", + "# Assign values to these three hyperparameters.\n", + "learning_rate = 0.06\n", + "epochs = 20\n", + "batch_size = 30\n", + "\n", + "# Don't change anything below this line.\n", + "my_model = build_model(learning_rate)\n", + "weight, bias, epochs, rmse = train_model(my_model, training_df,\n", + " my_feature, my_label,\n", + " epochs, batch_size)\n", + "\n", + "plot_the_loss_curve(epochs, rmse)\n", + "predict_house_values(15, my_feature, my_label)" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
longitudelatitudehousing_median_agetotal_roomstotal_bedroomspopulationhouseholdsmedian_incomemedian_house_valuerooms_per_person
longitude1.0-0.9-0.10.00.10.10.1-0.0-0.0-0.1
latitude-0.91.00.0-0.0-0.1-0.1-0.1-0.1-0.10.1
housing_median_age-0.10.01.0-0.4-0.3-0.3-0.3-0.10.1-0.1
total_rooms0.0-0.0-0.41.00.90.90.90.20.10.1
total_bedrooms0.1-0.1-0.30.91.00.91.0-0.00.00.0
population0.1-0.1-0.30.90.91.00.9-0.0-0.0-0.1
households0.1-0.1-0.30.91.00.91.00.00.1-0.0
median_income-0.0-0.1-0.10.2-0.0-0.00.01.00.70.2
median_house_value-0.0-0.10.10.10.0-0.00.10.71.00.2
rooms_per_person-0.10.1-0.10.10.0-0.1-0.00.20.21.0
\n", + "
" + ], + "text/plain": [ + " longitude latitude housing_median_age total_rooms \\\n", + "longitude 1.0 -0.9 -0.1 0.0 \n", + "latitude -0.9 1.0 0.0 -0.0 \n", + "housing_median_age -0.1 0.0 1.0 -0.4 \n", + "total_rooms 0.0 -0.0 -0.4 1.0 \n", + "total_bedrooms 0.1 -0.1 -0.3 0.9 \n", + "population 0.1 -0.1 -0.3 0.9 \n", + "households 0.1 -0.1 -0.3 0.9 \n", + "median_income -0.0 -0.1 -0.1 0.2 \n", + "median_house_value -0.0 -0.1 0.1 0.1 \n", + "rooms_per_person -0.1 0.1 -0.1 0.1 \n", + "\n", + " total_bedrooms population households median_income \\\n", + "longitude 0.1 0.1 0.1 -0.0 \n", + "latitude -0.1 -0.1 -0.1 -0.1 \n", + "housing_median_age -0.3 -0.3 -0.3 -0.1 \n", + "total_rooms 0.9 0.9 0.9 0.2 \n", + "total_bedrooms 1.0 0.9 1.0 -0.0 \n", + "population 0.9 1.0 0.9 -0.0 \n", + "households 1.0 0.9 1.0 0.0 \n", + "median_income -0.0 -0.0 0.0 1.0 \n", + "median_house_value 0.0 -0.0 0.1 0.7 \n", + "rooms_per_person 0.0 -0.1 -0.0 0.2 \n", + "\n", + " median_house_value rooms_per_person \n", + "longitude -0.0 -0.1 \n", + "latitude -0.1 0.1 \n", + "housing_median_age 0.1 -0.1 \n", + "total_rooms 0.1 0.1 \n", + "total_bedrooms 0.0 0.0 \n", + "population -0.0 -0.1 \n", + "households 0.1 -0.0 \n", + "median_income 0.7 0.2 \n", + "median_house_value 1.0 0.2 \n", + "rooms_per_person 0.2 1.0 " + ] + }, + "execution_count": 35, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Generate a correlation matrix.\n", + "training_df.corr()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Generalization\n", + "Generalization refers to your model's ability to adapt properly to new, previously unseen data, drawn from the same distribution as the one used to create the model.\n", + "\n", + "Machine learning's goal is to predict well on new data drawn from a (hidden) true probability distribution. Unfortunately, the model can't see the whole truth; the model can only sample from a training data set. If a model fits the current examples well, how can you trust the model will also make good predictions on never-before-seen examples?\n", + "\n", + "William of Ockham, a 14th century friar and philosopher, loved simplicity. He believed that scientists should prefer simpler formulas or theories over more complex ones. To put Ockham's razor in machine learning terms:\n", + "\n", + "```\n", + "The less complex an ML model, the more likely that a good empirical result is not just due to the peculiarities of the sample.\n", + "```\n", + "\n", + "A machine learning model aims to make good predictions on new, previously unseen data. But if you are building a model from your data set, how would you get the previously unseen data? Well, one way is to divide your data set into two subsets:\n", + "\n", + "- training set—a subset to train a model.\n", + "- test set—a subset to test the model.\n", + "\n", + "The following three basic assumptions guide generalization:\n", + "\n", + "1. We draw examples independently and identically (i.i.d) at random from the distribution. In other words, examples don't influence each other. (An alternate explanation: i.i.d. is a way of referring to the randomness of variables.)\n", + "2. The distribution is stationary; that is the distribution doesn't change within the data set.\n", + "3. We draw examples from partitions from the same distribution.\n", + "\n", + "### Splitting Data\n", + "\n", + "Make sure that your test set meets the following two conditions:\n", + "\n", + "1. Is large enough to yield statistically meaningful results.\n", + "2. Is representative of the data set as a whole. In other words, don't pick a test set with different characteristics than the training set.\n", + "\n", + "**Never train on test data.**\n", + "\n", + "### Validatiion Set\n", + "Dividing the data set into two sets is a good idea, but not a panacea. You can greatly reduce your chances of overfitting by partitioning the data set into the three subsets shown in the following figure:\n", + "![image](https://developers.google.com/machine-learning/crash-course/images/PartitionThreeSets.svg)\n", + "\n", + "Use the validation set to evaluate results from the training set. Then, use the test set to double-check your evaluation after the model has \"passed\" the validation set. The following figure shows this new workflow:\n", + "![alt](https://developers.google.com/machine-learning/crash-course/images/WorkflowWithValidationSet.svg)\n", + "\n", + "In this improved workflow:\n", + "\n", + "1. Pick the model that does best on the validation set.\n", + "2. Double-check that model against the test set.\n", + "This is a better workflow because it creates fewer exposures to the test set." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import pandas as pd\n", + "import tensorflow as tf\n", + "from matplotlib import pyplot as plt\n", + "\n", + "pd.options.display.max_rows = 10\n", + "pd.options.display.float_format = \"{:.1f}\".format\n", + "\n", + "train_df = pd.read_csv(\"https://download.mlcc.google.com/mledu-datasets/california_housing_train.csv\")\n", + "test_df = pd.read_csv(\"https://download.mlcc.google.com/mledu-datasets/california_housing_test.csv\")\n", + "\n", + "scale_factor = 1000.0\n", + "\n", + "# Scale the training set's label.\n", + "train_df[\"median_house_value\"] /= scale_factor \n", + "\n", + "# Scale the test set's label\n", + "test_df[\"median_house_value\"] /= scale_factor" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Defined the build_model and train_model functions.\n" + ] + } + ], + "source": [ + "#@title Define the functions that build and train a model\n", + "def build_model(my_learning_rate):\n", + " \"\"\"Create and compile a simple linear regression model.\"\"\"\n", + " # Most simple tf.keras models are sequential.\n", + " model = tf.keras.models.Sequential()\n", + "\n", + " # Add one linear layer to the model to yield a simple linear regressor.\n", + " model.add(tf.keras.layers.Dense(units=1, input_shape=(1,)))\n", + "\n", + " # Compile the model topography into code that TensorFlow can efficiently\n", + " # execute. Configure training to minimize the model's mean squared error. \n", + " model.compile(optimizer=tf.keras.optimizers.RMSprop(lr=my_learning_rate),\n", + " loss=\"mean_squared_error\",\n", + " metrics=[tf.keras.metrics.RootMeanSquaredError()])\n", + "\n", + " return model \n", + "\n", + "\n", + "def train_model(model, df, feature, label, my_epochs, \n", + " my_batch_size=None, my_validation_split=0.1):\n", + " \"\"\"Feed a dataset into the model in order to train it.\"\"\"\n", + "\n", + " history = model.fit(x=df[feature],\n", + " y=df[label],\n", + " batch_size=my_batch_size,\n", + " epochs=my_epochs,\n", + " validation_split=my_validation_split)\n", + "\n", + " # Gather the model's trained weight and bias.\n", + " trained_weight = model.get_weights()[0]\n", + " trained_bias = model.get_weights()[1]\n", + "\n", + " # The list of epochs is stored separately from the \n", + " # rest of history.\n", + " epochs = history.epoch\n", + " \n", + " # Isolate the root mean squared error for each epoch.\n", + " hist = pd.DataFrame(history.history)\n", + " rmse = hist[\"root_mean_squared_error\"]\n", + "\n", + " return epochs, rmse, history.history \n", + "\n", + "print(\"Defined the build_model and train_model functions.\")" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Defined the plot_the_loss_curve function.\n" + ] + } + ], + "source": [ + "def plot_the_loss_curve(epochs, mae_training, mae_validation):\n", + " \"\"\"Plot a curve of loss vs. epoch.\"\"\"\n", + "\n", + " plt.figure()\n", + " plt.xlabel(\"Epoch\")\n", + " plt.ylabel(\"Root Mean Squared Error\")\n", + "\n", + " plt.plot(epochs[1:], mae_training[1:], label=\"Training Loss\")\n", + " plt.plot(epochs[1:], mae_validation[1:], label=\"Validation Loss\")\n", + " plt.legend()\n", + " \n", + " # We're not going to plot the first epoch, since the loss on the first epoch\n", + " # is often substantially greater than the loss for other epochs.\n", + " merged_mae_lists = mae_training[1:] + mae_validation[1:]\n", + " highest_loss = max(merged_mae_lists)\n", + " lowest_loss = min(merged_mae_lists)\n", + " delta = highest_loss - lowest_loss\n", + " print(delta)\n", + "\n", + " top_of_y_axis = highest_loss + (delta * 0.05)\n", + " bottom_of_y_axis = lowest_loss - (delta * 0.05)\n", + " \n", + " plt.ylim([bottom_of_y_axis, top_of_y_axis])\n", + " plt.show() \n", + "\n", + "print(\"Defined the plot_the_loss_curve function.\")" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 1/30\n", + "136/136 [==============================] - 1s 8ms/step - loss: 40921.7539 - root_mean_squared_error: 202.2913 - val_loss: 49690.5586 - val_root_mean_squared_error: 222.9138\n", + "Epoch 2/30\n", + "136/136 [==============================] - 1s 5ms/step - loss: 23688.3301 - root_mean_squared_error: 153.9101 - val_loss: 29216.3457 - val_root_mean_squared_error: 170.9279\n", + "Epoch 3/30\n", + "136/136 [==============================] - 1s 8ms/step - loss: 12597.5850 - root_mean_squared_error: 112.2390 - val_loss: 15840.6221 - val_root_mean_squared_error: 125.8595\n", + "Epoch 4/30\n", + "136/136 [==============================] - 1s 4ms/step - loss: 7369.4302 - root_mean_squared_error: 85.8454 - val_loss: 9667.7637 - val_root_mean_squared_error: 98.3248\n", + "Epoch 5/30\n", + "136/136 [==============================] - 1s 4ms/step - loss: 6513.8804 - root_mean_squared_error: 80.7086 - val_loss: 9256.3809 - val_root_mean_squared_error: 96.2101\n", + "Epoch 6/30\n", + "136/136 [==============================] - 1s 5ms/step - loss: 6514.1553 - root_mean_squared_error: 80.7103 - val_loss: 9309.5977 - val_root_mean_squared_error: 96.4863\n", + "Epoch 7/30\n", + "136/136 [==============================] - 1s 4ms/step - loss: 6514.0376 - root_mean_squared_error: 80.7096 - val_loss: 9288.5332 - val_root_mean_squared_error: 96.3770\n", + "Epoch 8/30\n", + "136/136 [==============================] - 1s 7ms/step - loss: 6513.0557 - root_mean_squared_error: 80.7035 - val_loss: 9298.5352 - val_root_mean_squared_error: 96.4289\n", + "Epoch 9/30\n", + "136/136 [==============================] - 1s 4ms/step - loss: 6514.5352 - root_mean_squared_error: 80.7127 - val_loss: 9217.0439 - val_root_mean_squared_error: 96.0054\n", + "Epoch 10/30\n", + "136/136 [==============================] - 1s 4ms/step - loss: 6513.0737 - root_mean_squared_error: 80.7036 - val_loss: 9310.6855 - val_root_mean_squared_error: 96.4919\n", + "Epoch 11/30\n", + "136/136 [==============================] - 1s 5ms/step - loss: 6513.7534 - root_mean_squared_error: 80.7078 - val_loss: 9352.8506 - val_root_mean_squared_error: 96.7101\n", + "Epoch 12/30\n", + "136/136 [==============================] - 1s 7ms/step - loss: 6514.1660 - root_mean_squared_error: 80.7104 - val_loss: 9238.1992 - val_root_mean_squared_error: 96.1156\n", + "Epoch 13/30\n", + "136/136 [==============================] - 1s 6ms/step - loss: 6512.2290 - root_mean_squared_error: 80.6984 - val_loss: 9283.1416 - val_root_mean_squared_error: 96.3491\n", + "Epoch 14/30\n", + "136/136 [==============================] - 1s 4ms/step - loss: 6512.5464 - root_mean_squared_error: 80.7003 - val_loss: 9192.0459 - val_root_mean_squared_error: 95.8752\n", + "Epoch 15/30\n", + "136/136 [==============================] - 1s 4ms/step - loss: 6513.4199 - root_mean_squared_error: 80.7058 - val_loss: 9291.6582 - val_root_mean_squared_error: 96.3932\n", + "Epoch 16/30\n", + "136/136 [==============================] - 1s 6ms/step - loss: 6513.3047 - root_mean_squared_error: 80.7050 - val_loss: 9304.3906 - val_root_mean_squared_error: 96.4593\n", + "Epoch 17/30\n", + "136/136 [==============================] - 1s 5ms/step - loss: 6513.9692 - root_mean_squared_error: 80.7092 - val_loss: 9315.1504 - val_root_mean_squared_error: 96.5150\n", + "Epoch 18/30\n", + "136/136 [==============================] - 1s 5ms/step - loss: 6513.8237 - root_mean_squared_error: 80.7083 - val_loss: 9338.2305 - val_root_mean_squared_error: 96.6345\n", + "Epoch 19/30\n", + "136/136 [==============================] - 1s 6ms/step - loss: 6513.7568 - root_mean_squared_error: 80.7078 - val_loss: 9270.5557 - val_root_mean_squared_error: 96.2837\n", + "Epoch 20/30\n", + "136/136 [==============================] - 1s 7ms/step - loss: 6512.2217 - root_mean_squared_error: 80.6983 - val_loss: 9463.4424 - val_root_mean_squared_error: 97.2802\n", + "Epoch 21/30\n", + "136/136 [==============================] - 1s 7ms/step - loss: 6513.0322 - root_mean_squared_error: 80.7034 - val_loss: 9259.2979 - val_root_mean_squared_error: 96.2252\n", + "Epoch 22/30\n", + "136/136 [==============================] - 1s 6ms/step - loss: 6514.1646 - root_mean_squared_error: 80.7104 - val_loss: 9249.3379 - val_root_mean_squared_error: 96.1735\n", + "Epoch 23/30\n", + "136/136 [==============================] - 1s 6ms/step - loss: 6513.3789 - root_mean_squared_error: 80.7055 - val_loss: 9397.4609 - val_root_mean_squared_error: 96.9405\n", + "Epoch 24/30\n", + "136/136 [==============================] - 1s 5ms/step - loss: 6513.8096 - root_mean_squared_error: 80.7082 - val_loss: 9349.0381 - val_root_mean_squared_error: 96.6904\n", + "Epoch 25/30\n", + "136/136 [==============================] - 1s 5ms/step - loss: 6513.7441 - root_mean_squared_error: 80.7078 - val_loss: 9243.5498 - val_root_mean_squared_error: 96.1434\n", + "Epoch 26/30\n", + "136/136 [==============================] - 1s 5ms/step - loss: 6513.3188 - root_mean_squared_error: 80.7051 - val_loss: 9223.1143 - val_root_mean_squared_error: 96.0370\n", + "Epoch 27/30\n", + "136/136 [==============================] - 1s 6ms/step - loss: 6512.7412 - root_mean_squared_error: 80.7016 - val_loss: 9262.8340 - val_root_mean_squared_error: 96.2436\n", + "Epoch 28/30\n", + "136/136 [==============================] - 1s 8ms/step - loss: 6514.0166 - root_mean_squared_error: 80.7095 - val_loss: 9275.0068 - val_root_mean_squared_error: 96.3068\n", + "Epoch 29/30\n", + "136/136 [==============================] - 1s 5ms/step - loss: 6513.0972 - root_mean_squared_error: 80.7038 - val_loss: 9148.6162 - val_root_mean_squared_error: 95.6484\n", + "Epoch 30/30\n", + "136/136 [==============================] - 1s 8ms/step - loss: 6514.1665 - root_mean_squared_error: 80.7104 - val_loss: 9231.8604 - val_root_mean_squared_error: 96.0826\n", + "90.22956848144531\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEGCAYAAACO8lkDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3zcdZ3v8dcnaW7NvU0vuQBtoVDSC20JxQO4tILKuh5ALkIXlMIKiCy6eFZZPbuCsl09Hpb1wVHc5SbKIl28gOgiKCh0XRRoy62FIhUKpOm9NEnbpGmSz/nj95tkkkySSZqZycy8n4/HPOY337l9fvll5jPf3/dm7o6IiEi0nFQHICIi44+Sg4iIDKDkICIiAyg5iIjIAEoOIiIywIRUB3A4qqqqfMaMGakOQ0Qkraxdu3aXu08Z6jFpnRxmzJjBmjVrUh2GiEhaMbO3h3uMTiuJiMgASg4iIjKAkoOIiAyQ1m0OIpI8hw4dorGxkfb29lSHInEqLCykrq6OvLy8ET9XyUFE4tLY2EhpaSkzZszAzFIdjgzD3dm9ezeNjY3MnDlzxM/XaSURiUt7ezuTJ09WYkgTZsbkyZNHXdNTchCRuCkxpJfDOV7ZmRz2vgu/WQl73kx1JCIi41J2Joe292D1N2HbK6mORETisHv3bhYuXMjChQuZPn06tbW1Pbc7OjqGfO6aNWv47Gc/O+x7nHLKKWMS61NPPcVHP/rRMXmtVMrOBumy2uC6pSm1cYhIXCZPnsyLL74IwE033URJSQl/+7d/23N/Z2cnEybE/jpraGigoaFh2Pd45plnxibYDJGdNYeJk2BCIbRsSXUkIjJKK1as4POf/zzLli3jhhtu4LnnnuOUU05h0aJFnHLKKbz++utA31/yN910E1dccQVLly5l1qxZ3HbbbT2vV1JS0vP4pUuXcsEFFzBnzhwuueQSIitmPvroo8yZM4fTTjuNz372syOqITzwwAPMnz+fefPmccMNNwDQ1dXFihUrmDdvHvPnz+df/uVfALjtttuor69nwYIFXHzxxYf/xxqF7Kw5mEFZDTQrOYiMxld/voFXm1rG9DXra8q48X/OHdFz/vjHP/LEE0+Qm5tLS0sLq1evZsKECTzxxBN8+ctf5ic/+cmA52zcuJHf/va3tLa2ctxxx3HNNdcMGAfwwgsvsGHDBmpqajj11FP57//+bxoaGrj66qtZvXo1M2fOZPny5XHH2dTUxA033MDatWuprKzkQx/6EA8//DBHHHEEW7ZsYf369QDs3bsXgG984xu89dZbFBQU9JQlW3bWHCA4taTTSiJp7cILLyQ3NxeA5uZmLrzwQubNm8f111/Phg0bYj7nL/7iLygoKKCqqoqpU6eyffv2AY9ZsmQJdXV15OTksHDhQjZv3szGjRuZNWtWz5iBkSSH559/nqVLlzJlyhQmTJjAJZdcwurVq5k1axZvvvkm1113HY899hhlZWUALFiwgEsuuYR///d/H/R0WaJlZ80BgprD279PdRQiaWmkv/ATpbi4uGf7H/7hH1i2bBkPPfQQmzdvZunSpTGfU1BQ0LOdm5tLZ2dnXI+JnFoajcGeW1lZyUsvvcTjjz/Od77zHR588EHuuece/vM//5PVq1fzyCOPcPPNN7Nhw4akJ4ksrjnUQGsTdHenOhIRGQPNzc3U1gadTe69994xf/05c+bw5ptvsnnzZgD+4z/+I+7nnnzyyTz99NPs2rWLrq4uHnjgAU4//XR27dpFd3c3559/PjfffDPr1q2ju7ubd999l2XLlvHNb36TvXv3sm/fvjHfn+Fkcc2hFro7Yf9OKJ2W6mhE5DB98Ytf5LLLLuPWW2/lAx/4wJi/flFREbfffjtnnXUWVVVVLFmyZNDHPvnkk9TV1fXc/tGPfsTXv/51li1bhrvzkY98hHPOOYeXXnqJyy+/nO7wR+rXv/51urq6uPTSS2lubsbduf7666moqBjz/RmOHU5VKdUaGhp81Iv9bHwUVi2HK38DtSeObWAiGei1117j+OOPT3UYKbVv3z5KSkpwd6699lpmz57N9ddfn+qwhhTruJnZWncfsn9vdp9WAjVKi0jc7rzzThYuXMjcuXNpbm7m6quvTnVICZPdp5VAyUFE4nb99deP+5rCWMnemsPEyZCbr4FwIiIxZG9yyMkJTi2p5iAiMkD2JgfQQDgRkUFkeXKogebGVEchIjLuKDm0btVAOJFxbunSpTz++ON9yr71rW/xmc98ZsjnRLq6f+QjH4k5R9FNN93ELbfcMuR7P/zww7z66qs9t7/yla/wxBNPjCT8mMb71N5ZnhxqoasDDuxOdSQiMoTly5ezatWqPmWrVq2Ke36jRx99dNQDyfonh6997WuceeaZo3qtdKLkAOqxJDLOXXDBBfziF7/g4MGDAGzevJmmpiZOO+00rrnmGhoaGpg7dy433nhjzOfPmDGDXbt2AbBy5UqOO+44zjzzzJ5pvSEYw3DSSSdxwgkncP7553PgwAGeeeYZHnnkEb7whS+wcOFC/vSnP7FixQp+/OMfA8FI6EWLFjF//nyuuOKKnvhmzJjBjTfeyOLFi5k/fz4bN26Me1/Hy9Te2TvOAfoOhKtZmNpYRNLJL/9u7FdSnD4f/vwbMe+aPHkyS5Ys4bHHHuOcc85h1apVXHTRRZgZK1euZNKkSXR1dXHGGWfw8ssvs2DBgpivs3btWlatWsULL7xAZ2cnixcv5sQTgxkSzjvvPK688koA/v7v/567776b6667jrPPPpuPfvSjXHDBBX1eq729nRUrVvDkk09y7LHH8slPfpLvfve7/M3f/A0AVVVVrFu3jttvv51bbrmFu+66a9g/wXia2ls1B1DNQSQNRJ9aij6l9OCDD7J48WIWLVrEhg0b+pwC6u+//uu/+NjHPsbEiRMpKyvj7LPP7rlv/fr1vP/972f+/Pncf//9g075HfH6668zc+ZMjj32WAAuu+wyVq9e3XP/eeedB8CJJ57YM1nfcMbT1N7ZXXMongI5E5QcREZqkF/4iXTuuefy+c9/nnXr1tHW1sbixYt56623uOWWW3j++eeprKxkxYoVtLe3D/k6ZhazfMWKFTz88MOccMIJ3HvvvTz11FNDvs5w89JFpv0ebFrwkbxmKqb2TljNwczuMbMdZra+X/l1Zva6mW0ws29GlX/JzDaF9304UXH1kZMDpRoIJ5IOSkpKWLp0KVdccUVPraGlpYXi4mLKy8vZvn07v/zlL4d8jT/7sz/joYceoq2tjdbWVn7+85/33Nfa2kp1dTWHDh3i/vvv7ykvLS2ltbV1wGvNmTOHzZs3s2nTJgDuu+8+Tj/99MPax/E0tXciaw73At8GfhApMLNlwDnAAnc/aGZTw/J64GJgLlADPGFmx7p7VwLjC2iUtEjaWL58Oeedd17P6aUTTjiBRYsWMXfuXGbNmsWpp5465PMXL17MRRddxMKFCznqqKN4//vf33PfzTffzMknn8xRRx3F/PnzexLCxRdfzJVXXsltt93W0xANUFhYyPe+9z0uvPBCOjs7Oemkk/j0pz89ov0Zz1N7J3TKbjObAfzC3eeFtx8E7nD3J/o97ksA7v718PbjwE3uPuRSbYc1ZXfEj6+Aphfgsy8c3uuIZDhN2Z2e0mXK7mOB95vZs2b2tJmdFJbXAu9GPa4xLBvAzK4yszVmtmbnzp2HH1Gk5pDG61qIiIy1ZCeHCUAl8D7gC8CDFrQOxWohivlt7e53uHuDuzdMmTLl8CMqq4XOdjiw5/BfS0QkQyQ7OTQCP/XAc0A3UBWWHxH1uDogOQ0BPWMd1GNJZDjpvHJkNjqc45Xs5PAw8AEAMzsWyAd2AY8AF5tZgZnNBGYDzyUlIi36IxKXwsJCdu/erQSRJtyd3bt3U1hYOKrnJ6y3kpk9ACwFqsysEbgRuAe4J+ze2gFc5sF/2oawsfpVoBO4Nik9lUAD4UTiVFdXR2NjI2PS1idJUVhY2Kc31EgkLDm4+2AzYl06yONXAisTFc+gSqaC5armIDKMvLw8Zs6cmeowJEmye/oMgJxcKK1WzUFEJIqSA4TdWZUcREQilBxAo6RFRPpRcgAor9NAOBGRKEoOENQcDh2A9rGdD11EJF0pOUDvQLhmtTuIiECWJocDHZ2sfXsPzQcOBQUaCCci0kdWJoeN21o5/7u/Z+074XxKmkJDRKSPIZODmeWa2f9NVjDJUldRBMCW99qCgpLpYDmqOYiIhIZMDuEUFifaYOvqpamqkgLyc3PYsjdcTjB3QpAglBxERID4ps94AfiZmf0I2B8pdPefJiyqBMvJMaorCtmyt623UAPhRER6xJMcJgG7CWdTDTmQtskBoKa8iKb+yWHnxtQFJCIyjgybHNz98mQEkmy1lUX87o1dvQVltbDpyWAgXGadRRMRGbFheyuZWZ2ZPWRmO8xsu5n9xMxGNwfsOFJTUcT21nYOdQULdgcD4fbDwZbUBiYiMg7E05X1ewSL8dQQrOv887AsrdVVFOEO25rDRulyjXUQEYmIJzlMcffvuXtneLkXGIPFm1OrJtKdNdLuoEV/RER6xJMcdpnZpeGYh1wzu5SggTqt1Vb2G+ugKTRERHrEkxyuAD4ObAO2AheEZWmtujxYV7Wnx1LJdMB0WklEhGF6K5lZLvBP7n52kuJJmsK8XKpKCnpPK03ID5YM1WklEZG4RkhPMbP8JMWTVLUDBsLVquYgIkJ8g+A2A/9tZo/Qd4T0rYkKKllqK4vYuK21t6CsBnb/KXUBiYiME/G0OTQBvwgfWxp1SXuRUdIeWQFONQcRESC+NofZ7n5pkuJJqtrKItoPdbNnfweTSwqCmsPBZjjYCgUZkf9EREYlq9scImMdmiKzs2rRHxERINvbHHoGwh1gfl151CjpLTDluBRGJiKSWvEkh6bwEmlzyBi9ySFSc4isCKeag4hkt3hmZf1q/zIziyepjHsVE/OYmJ/bO0q6tDq4VnIQkSw3aJuDmf0uavu+fnc/l7CIksjMqKmIWtdhQgEUT4HmxtQGJiKSYkM1SBdHbc/rd1/GLHhQU1EUY0U41RxEJLsNlRx8kO1Yt9NWbUX/FeE01kFEZKi2gwoz+xhBAqkws/PCcgPKEx5ZktRWFLJ7fwdtHV0U5ecGyeHtZ1IdlohISg2VHJ4Gzo7a/p9R961OWERJFpm6u6m5jaOnlASnldr3Qsd+yC8e5tkiIplp0OSQqWtH91dT3ruuQ5AcogbCVc1OYWQiIqkTz9xKGa2n5rC336I/mrpbRLJY1ieHaWWF5Fis5KBGaRHJXglLDmZ2j5ntMLP1Me77WzNzM6uKKvuSmW0ys9fN7MOJiqu/vNwcppcV0qi1pEVEegza5hDVOykmd//pMK99L/Bt4Af9XvcI4IPAO1Fl9cDFwFygBnjCzI4NJ/5LuD4D4fIKYeJk1RxEJKsN1Vsp0jtpKnAK8Jvw9jLgKWDI5ODuq81sRoy7/gX4IvCzqLJzgFXufhB4y8w2AUuA3w8d/tiorSxi3Tvv9RZoIJyIZLlBTyu5++VhjyUH6t39fHc/n+DX/aiY2dnAFnd/qd9dtcC7Ubcbw7JYr3GVma0xszU7d+4cbSh91FQUsa25na7uqEV/mnVaSUSyVzxtDjPcfWvU7e3AsSN9IzObCPxv4Cux7o5RFnMUtrvf4e4N7t4wZcqUkYYRU21FEYe6nJ2tB4OCshq1OYhIVotndtWnzOxx4AGCL+yLgd+O4r2OBmYCL5kZQB2wzsyWENQUjoh6bB3BNOFJ0Tt1dxvTywuDmkPbHjjUBnlFyQpDRGTcGLbm4O5/DfwrcAKwELjD3a8b6Ru5+yvuPtXdZ7j7DIKEsNjdtwGPABebWYGZzQRmk8SZXyNjHbYM6LGkdgcRyU7xrsuwDmh19yfMbKKZlbp761BPMLMHgKVAlZk1Aje6+92xHuvuG8zsQeBVoBO4Nlk9lSB6udAYYx0mH52sMERExo1hk4OZXQlcBUwiODVUS1CTOGOo57n78mHun9Hv9kpg5XDxJEJJwQTKi/J6F/3RWAcRyXLxNEhfC5wKtAC4+xsE3VszSp+xDppCQ0SyXDzJ4aC7d0RuhEuEZsx6DhG10Yv+5E+Eokq1OYhI1oonOTxtZl8Giszsg8CPgJ8nNqzkq60o7LcinBb9EZHsFU9yuAHYCbwCXA08Cvx9IoNKhdrKIlrbO2lpPxQUaKyDiGSxIRukzSwHeNnd5wF3Jiek1IjusVQ2PS9IDk0vpDgqEZHUGLLm4O7dBIPWjkxSPCnTMxAuusfS/p3QeTCFUYmIpEY84xyqgQ1m9hywP1Lo7mcP/pT0UzvUWIdJM1MUlYhIasSTHL6a8CjGgaqSAvJzc2Ks66DkICLZZ9jk4O5PJyOQVMvJMaorCmna2x4UaAoNEcliw/ZWMrP3mdnzZrbPzDrMrMvMWpIRXLLVlBex5b0DwY2y6uBaPZZEJAvF05X128By4A2gCPhUWJZxaiuLemsOBaVQUK7kICJZKa41pN19E5Dr7l3u/j2CCfUyTk1FEdtb2+no7A4KtCKciGSpeBqkD5hZPvCimX0T2AoUJzas1KirKMIdtre0c8SkiVBeq5qDiGSleGoOnwBygb8m6Mp6BHB+IoNKlchAuMb3orqzquYgIlkont5Kb4ebbWR4t9bIoj9N0d1Z9+2Azg6YkJ/CyEREkiue9RzeIsYsrO4+KyERpVB1eSEQvSJcDeCwbxtUZPwgcRGRHvG0OTREbRcCFxIs/JNxCvNyqSopGDhKunmLkoOIZJV41pDeHXXZ4u7fAj6QhNhSos/U3WV1wbUapUUky8RzWmlx1M0cgppEacIiSrHayiI2bguXx46eX0lEJIvEc1rpn6O2O4HNwMcTEs04UFNexG827sDdscIyyC9VchCRrBNPb6VlyQhkvKitLKL9UDd79ncwuaRAi/6ISFaK57TS54e6391vHbtwUi8y1mHL3jYlBxHJWvEMgmsArgFqw8ungXqCdoeMa3sYsK5DudaSFpHsE0+bQxWw2N1bAczsJuBH7v6pRAaWKrUDRknXQus26DoEuXkpjExEJHniqTkcCXRE3e4AZiQkmnGgYmIeE/Nzo9Z1CAfCtW5LaVwiIskUT83hPuA5M3sovH0u8P3EhZRaZkZNRVHvaaVJ4UDw3W9AxRGpC0xEJIniGQS3ErgceA/YA1zu7l9PdGCpVFtR1DsQburc4Hr7htQFJCKSZIMmBzObaGZ5AO6+DniMYHbWjF9QuU/NoXgylFYrOYhIVhmq5vAYYduCmR0D/B6YBVxrZt9IfGipU1dZxO79HbR1dAUF0+bC9vWpDUpEJImGSg6V7v5GuH0Z8IC7Xwf8OfAXCY8shWoqgtlZm5rD2sO0ubDz9aDHkohIFhgqOURP0/0B4NcA7t4BdCcyqFSrrZgIwJZId9Zp86CrA3ZvSmFUIiLJM1RvpZfN7BZgC3AM8CsAM6tIRmCp1FNz2BtVc4Cg3WHq8SmKSkQkeYaqOVwJ7CJod/iQux8Iy+uBWxIcV0pNLyskx6IW/Zk8G3Ly1O4gIllj0JqDu7cBAxqe3f0Z4JlEBpVqE3JzmF4Wta7DhHyYMge2KTmISHaIZ4R0VqqpKOptc4Cwx5K6s4pIdkhYcjCze8xsh5mtjyr7v2a20cxeNrOHotsvzOxLZrbJzF43sw8nKq541VYW9fZWgiA5tDbBgT2pC0pEJEkSWXO4FzirX9mvgXnuvgD4I/AlADOrBy4G5obPud3MchMY27BqKorYuredru6w09Y0jZQWkewxbHIws2PN7E4z+5WZ/SZyGe557r6aYLqN6LJfuXtnePMPQLhIM+cAq9z9oLu/BWwCloxoT8ZYbUURnd3OztaDQcG0ecG1koOIZIF4Jt77EfCvwJ1A1xi+9xXAf4TbtQTJIqIxLBvAzK4CrgI48sgjxzCcvmp7Fv05wPTyQiiZChOr1GNJRLJCPMmh092/O5Zvamb/m2A96vsjRTEe5jHKcPc7gDsAGhoaYj5mLNRWRpJDOyceBZipUVpEskY8bQ4/N7PPmFm1mU2KXEb7hmZ2GfBR4BJ3j3y5NwLR82HXASldfq1nudDoHkvT58OO16B7LCtQIiLjTzw1h8vC6y9ElTnBJHwjYmZnATcAp0cNqgN4BPihmd0K1ACzgedG+vpjqaRgAuVFeb2jpCGoOXS2wZ43oWp26oITEUmwYZODu49qim4zewBYClSZWSNwI0HvpALg12YG8Ad3/7S7bzCzB4FXCU43XevuKf95XhO9rgNE9Vhar+QgIhktnpoDZjaPYNqMwkiZu/9gqOe4+/IYxXcP8fiVwMp44kmW2ooiGt+LquBUHQeWG7Q7zP1Y6gITEUmwYZODmd1IUAOoBx4lmLL7d8CQySET1FYU8uybu3sL8gqDGoMapUUkw8XTIH0BcAawzd0vB04gODWU8Wori2g92ElLe9Q6Dlr4R0SyQDzJoc3du4FOMysDdjCKxuh0FLPH0rS5sPcdaG9OUVQiIokXT3JYE86BdCewFlhHinsSJUtkIFzfHkvhSOkdr6UgIhGR5Iint9Jnws1/NbPHgDJ3fzmxYY0PvaOkYySHba/Ake9LQVQiIokXz9xKZmaXmtlX3H0zsNfMUjrvUbJUlRSQn5vTNzmU1UBhhRqlRSSjxXNa6XbgfwCRrqmtwHcSFtE4kpNjVFcU9m1zMAtqD0oOIpLB4kkOJ7v7tUA7gLu/B+QnNKpxpLaiqG+bAwSN0jtehe7u1AQlIpJg8SSHQ+HaCg5gZlOArPlWHDBKGoLk0LEP9r6dmqBERBIsnuRwG/AQMNXMVhIMgPunhEY1jtRWFLGj9SAdnVH5UGs7iEiGi6e30v1mtpZgIJwB57p71vTjrK0owh22Nrdx1OTioHDqHMCC5HD8R1Man4hIIgyaHPpNy70DeCD6PnfPisWUj55aAsAft+/rTQ75xTD5aI2UFpGMNVTNYRfBOguRZT2jF+QZ1ZTd6WjO9FLM4NWmFj5YP633jmlzg7EOIiIZaKg2h/8HvAc8RrCmwyx3nxlesiIxABQXTGDm5GJe3dpvuoxp82DPW3BwX2oCExFJoEGTg7t/DlhIsIb0J4AXzOybZjaq9R3S2fE1Zby6taVv4bS5gMPOjSmJSUQkkYbsreSB3wJfBP4VuBw4MxmBjSf11WW8u6eN5rZ+s7OC2h1EJCMNmhzMrNjM/tLMfkawjkMJsNjd70xadONEfU0ZAK9F1x7Kj4T8UnVnFZGMNFSD9A7gDYJeSpsIGqFPMrOTANz9p4kPb3yYWx0kh1ebWnjfrMlBYU4OTKtXchCRjDRUcvgRQUKYE16iOZA1yWFqWSFVJQWx2x3W/wTcgzmXREQyxKDJwd1XJDGOca++poxXm/onh3mw5h5o2QLldakJTEQkAeKZPkMIGqXf2NEaexqNbWqUFpHMouQQp/qaMg51OZt2RI1rmHp8cK0eSyKSYeJZ7KcgnrJMVx9plI5udygsg4qj1CgtIhknnprD7+Msy2gzq4opzMuJ3e6g5CAiGWaoifemA7VAkZktondupTJgYhJiG1dyc4w508vY0NR/Go258MdfwqF2yCtMTXAiImNsqK6sHwZWAHXArVHlrcCXExjTuDW3poxHXmrC3bFI19Vpc8G7g2k0ahamNkARkTEy1NxK33f3ZcAKd18WdTk7mwbARauvKaO1vZPG6DWlp88PrnVqSUQySDxtDk+a2a1mtia8/LOZlSc8snEoZqN05QzIm6jkICIZJZ7kcDfBqaSPh5cW4HuJDGq8mjO9jJxwbYceOblBl9btWttBRDLHsMuEAke7+/lRt79qZi8mKqDxrCg/l5lVxbGn0XjtF5pGQ0QyRjw1hzYzOy1yw8xOBdqGeHxGq68pj92dtW0P7NuemqBERMZYPMnhGuA7ZrbZzN4Gvg1cndiwxq/66jK27G2j+YDWdhCRzDVscnD3F939BGABMN/dF7n7y4kPbXyKrO2wIXrZ0Kn1wbUapUUkQ8QzfUa5md0K/Ab4TTb3VoKoHkvRp5YmToKyWiUHEckY8ZxWuodR9FYys3vMbIeZrY8qm2RmvzazN8Lryqj7vmRmm8zsdTP78Mh3JTmmlBYwtTTW2g6aRkNEMkc8yeFod7/R3d8ML18FZsXxvHuBs/qV/R3wpLvPBp4Mb2Nm9cDFwNzwObebWW6c+5B0sdd2mAs7X4fOjtQEJSIyhhLWW8ndVwN7+hWfA3w/3P4+cG5U+Sp3P+jubxEsS7okjthSor66jE079nGws6u3cNpc6D4Eu/6YusBERMZIsnsrTXP3rQDh9dSwvBZ4N+pxjWHZuFRfU0Znt/PG9qi1HSIL/+jUkohkgBH3VgJOCq/HUqyRYx7zgWZXRaby2Llz5xiHEZ+Y02hMPgZy89WdVUQywqDJwczKwkbib5vZBwkapT9JcMrn46N8v+1mVh2+fjWwIyxvBI6Ielwd0BTrBdz9DndvcPeGKVOmjDKMw3PU5GIm5uf2bXfInQBT5qjmICIZYaiaw33AccArwJXAr4ALgXPd/ZxRvt8jwGXh9mXAz6LKLzazAjObCcwGnhvleyRcsLZDaeyR0ttehvaW2E8UEUkTQ82tNMvd5wOY2V3ALuBId2+N54XN7AFgKVBlZo3AjcA3gAfN7K+AdwiSDe6+wcweBF4FOoFr3b0r5guPE3NrynnohS10dzs5OeFZsWPOgJd+CLfWw8K/hCVXQdUxqQ1URGQUhkoOPfNDuHuXmb0Vb2IIn7N8kLvOGOTxK4GV8b5+qtXXlHHfH96m8b02jpwcLow3/wKonAnP/RusuSe4PvoMOPlqOOaDkBNP+79Ikh3cBwdboLRaE0dKj6GSwwlmFjk/YgTLhbaE2+7uZQmPbhzrbZRu7k0OAHUnQt0d8KF/hLX3wvN3ww8/HiSNJVfCwkugqCI1QYsc3AfbXoGtL0LTC9D0Ytj92qF4KtQ1QO1iqG2AmkWJ/189sAd2bwou3Z2QXwIFpeF1SdTtYphQqOSVRIMmB3cft4PQxoPjppf2rO1w1rzqgQ8omQqnfxFOux5eewSe/Td4/Mvwm5VwwsXBKaepc3of397JPxIAAA4sSURBVN0FB1ujLi19r7sOBb2hcvOC65wJ4e38oDE8Nx9y8oL7zYLX8+7g0t0F3tVv24PbBaVQPAWKq6CwYnQfvoOt0Lod9m2D1m2wf2fwJXRoP3Qc6L3u2A+Hoq8PBF8IZdVQfgSU10HFkcF1+RFQccTIYuruDt9rP1gOTJwcrLdxuLq74cBuaN0axN3dFcTtXeF2+Dft7ux3u/91eDwG3NcdHN+ujn6XqLLO8Nq7oKAMiiqDy8RJvdv9LxAkgqYwEWwNE4F3B/eVTA8SwLzzgr/z1hehcQ28/mjvvlcdC7Un9l6mzYMJ+SP7+3V1wt63YdcbwfvvfiPcfgMO7Ir/dSw3SBgFZcHnq6wmmLamrDZquyaoAY00xrEW+Xx1dwX/g2Pxf5hk5h6zx2haaGho8DVr1qTs/T9469McNXkid112UnxPaHoRnrsDXvkxdB0MahOd7cGXa8e+4Z+faDkTgkQxsSpIFsVTehNHUSW0NwfTkrdu671u3RZ8IQ/2ennFwa++/InBinn5JX23zaClCZrfhebG4O8RLb+kN1EUTepNLj2Xfb3b/eOw3CD+kqlQOj24LpkOJdOgdFpwXTIt+BC3NIX70wQtW4Pr1m3h9tZggGOi5Rb0/gCYUND7Q6DnR0FBkPTam6HtvWCa+K44R+SXTAsSQfXC4LpmYfA3iaVtLzStgy1rYcu6IGHs39EbY8URwd/WcoIvPcvpt53bu71vB+x5s+/fr3gKTJ4NVZHLsb1dwTv2BT8sOlrD6/639wc/mPZth+Yt0LIlxmfHepNHaTXkFUX9oOr/4yr8QZWTF8R8qD3qR82B3h8xfbb3Q+fB3h8DkcQf/YMhusk0Z0LwP1w5I/ZlqNqZe3Cs9+8KfnTt39G7PfkYWDC6jqNmttbdG4Z8jJLD6H1u1Qs8/9YenvlSzGaUwe3fBeu+D1tfDn65F5RBYVm4Hd6Ovi4sC/7BIr8kuzujfllGyqK2IepDmtPvAxv1ocaCD1rPP17kEt4+sCvYjv7w5RWHX6zTB78umRbEPdJfb+7B+zW/A3vf7U0Ye98Nytr2hokmcintd7ukd9u7oxLZjqBWs29HcBmur0N+SfClUjq99wumrCa4nV8SHIuc3OA68jfu2Z7Q+8UY+Xv3uc4Z4r4R1trc4VBbmChiXLoOBSP3axYFtbPRcg+Ow5Y1QaJo3dq3Btq/FtRTS+oKEnokAVTNDr7QJk4afSyxtLcEyb0lTBY922GC72wPPy+d/T4rh2In/Zy88AdMcZBYIj9m8iYG/1t5RcEprp5jHTnuUZ+z6P+NjgNBzem9zcHlwO6+71dY0ZsoJhT2+yzuDD7vAxgsuAjO+7dR/cniSQ7xrAQng6ivLuNnLzbx3v4OKotH8EVYXAXv/1+JC2ysdRwIfqUWlgdf+oliBiVTgkvtiYl5j+6u4Dz3vkjtZ3vwZV1WHSaE6iAZpwOz4IsrfyKUJ3BCAbOgtlBxBMz9WOLeZ7QKwx9X0adp4+Xe+2OruyusZeSNfYzR2lv6JovIZfv64PRhyZTgx0j1gqAdKFL77anNTw0SbIJPVSk5HIbI2g6vbm3h1GOqUhxNAkW+gDJBTm5vAhrzgf6Sdsx6Ty0lS2EZTJ8fXMYx9a08DDHXdhARyQBKDodhckkB08sKB67tICKS5pQcDlPMtR1ERNKcksNhqq8uY9POfbQfGtezfYiIjIiSw2Gqrymjq//aDiIiaU7J4TBFT6MhIpIplBwO05GTJlJSMEHtDiKSUZQcDlNOjnF8dSkblBxEJIMoOYyB+uoyXtvaQnd3+k5FIiISTclhDNTXlLG/o4t39hxIdSgiImNCyWEM1FeXA2gwnIhkDCWHMTB7Wgm5OaZGaRHJGEoOY6AwL5djppSo5iAiGUPJYYxoGg0RySRKDmOkvrqMbS3t7N53MNWhiIgcNiWHMTI3am0HEZF0p+QwRo7X2g4ikkGUHMZIZXE+NeVa20FEMoOSwxhSo7SIZAolhzE0t6acP+3cx3Nv7Ul1KCIih0XJYQx94n8cxYyqYq6493lefHdvqsMRERk1JYcxVFVSwA8/9T4mFefzybufZf0WrfEgIulJyWGMTS8v5IdXnkxpYR6fuPtZXt/WmuqQRERGTMkhAeoqJ3L/p04mLzeHS+56ljd3aglREUkvSg4JMqOqmB9eeTLuzl/e+Szv7NZ03iKSPpQcEuiYqaX8+6dOpr2zi7+86w9s2duW6pBEROKi5JBgx1eXcd8VJ9PcdohL7vwD21vaUx2SiMiwlBySYH5dOfdevoQdrQe55K5n2aXJ+URknFNySJITj6rknhUn0fjeAS6961n2HuhIdUgiIoNKSXIws+vNbIOZrTezB8ys0MwmmdmvzeyN8LoyFbEl0vtmTebOTzbw5q79fOLu52hpP5TqkEREYjJ3T+4bmtUCvwPq3b3NzB4EHgXqgT3u/g0z+zug0t1vGOq1GhoafM2aNYkPeow9+dp2rr5vLWVFeZQWTkjoe1n0tlnM8uj/gMj/g/fcjvGaNvhr28CHx3z9WPfFK/IeZtb7flExRe9nn/eKsTMDSnyY+w9Tsj9vEYP9rSL3RRv2GPnA8ujnDLWH/Y/MYMdqsFjGymDvG7N0qBD7/S36f36GeqnBPo/9nz/Y33bZcVO56ey5QwQ3ODNb6+4NQz0msd9MQ79vkZkdAiYCTcCXgKXh/d8HngKGTA7p6ozjp3H3ipN4aF1jQt+n7z/YwHJ3j/ml3v/LP/qfONY/v8f4gPT/8MV6/Vj3DSc6acX8QPbE4lisV46jaKjYRxLnYM8b5vtwzA36t4raGPD3GuYYDfV/E+vxfd4zKq7o+5L19xos38QqHiw5Rcfb/28R68dT/9eP9XmMvF+8f9uZVcUxYxsrSU8O7r7FzG4B3gHagF+5+6/MbJq7bw0fs9XMpsZ6vpldBVwFcOSRRyYr7DF3+rFTOP3YKakOQ0QkpqS3OYRtCecAM4EaoNjMLo33+e5+h7s3uHvDlCn6chURSYRUNEifCbzl7jvd/RDwU+AUYLuZVQOE1ztSEJuIiJCa5PAO8D4zm2jBybUzgNeAR4DLwsdcBvwsBbGJiAipaXN41sx+DKwDOoEXgDuAEuBBM/srggRyYbJjExGRQEp6K7n7jcCN/YoPEtQiREQkxTRCWkREBlByEBGRAZQcRERkgKRPnzGWzGwn8Ha/4ipgVwrCSTTtV/rJ1H3TfqWf/vt2lLsPOVAsrZNDLGa2Zrg5Q9KR9iv9ZOq+ab/Sz2j2TaeVRERkACUHEREZIBOTwx2pDiBBtF/pJ1P3TfuVfka8bxnX5iAiIocvE2sOIiJymJQcRERkgIxJDmZ2lpm9bmabwmVGM4aZbTazV8zsRTNLv3VRQ2Z2j5ntMLP1UWVpv3b4IPt1k5ltCY/Zi2b2kVTGOBpmdoSZ/dbMXgvXfP9cWJ4Jx2ywfUvr42ZmhWb2nJm9FO7XV8PyER+zjGhzMLNc4I/AB4FG4Hlgubu/mtLAxoiZbQYa3D2tB+iY2Z8B+4AfuPu8sOybjHDt8PFmkP26Cdjn7rekMrbDEa6rUu3u68ysFFgLnAusIP2P2WD79nHS+LiFyyAUu/s+M8sDfgd8DjiPER6zTKk5LAE2ufub7t4BrCJYbU7GEXdfDezpV3wOwZrhhNfnJjWoMTDIfqU9d9/q7uvC7VaCdVdqyYxjNti+pTUP7Atv5oUXZxTHLFOSQy3wbtTtRjLgQEdx4FdmtjZcQzuT9Fk7HIi5dnia+mszezk87ZR2p16imdkMYBHwLBl2zPrtG6T5cTOzXDN7kWA1zV+7+6iOWaYkB4tRlv7ny3qd6u6LgT8Hrg1PY8j49l3gaGAhsBX459SGM3pmVgL8BPgbd29JdTxjKca+pf1xc/cud18I1AFLzGzeaF4nU5JDI3BE1O06oClFsYw5d28Kr3cADxGcRssUGbl2uLtvDz+k3cCdpOkxC89b/wS4391/GhZnxDGLtW+ZctwA3H0v8BRwFqM4ZpmSHJ4HZpvZTDPLBy4mWJM67ZlZcdhghpkVAx8C1g/9rLSSkWuHRz6IoY+RhscsbNy8G3jN3W+Nuivtj9lg+5bux83MpphZRbhdBJwJbGQUxywjeisBhF3OvgXkAve4+8oUhzQmzGwWQW0BgmVdf5iu+2ZmDwBLCaYP3k6wVOzDwIPAkYRrh7t7WjXuDrJfSwlOTTiwGbg6cs43XZjZacB/Aa8A3WHxlwnOzaf7MRts35aTxsfNzBYQNDjnEvz4f9Ddv2ZmkxnhMcuY5CAiImMnU04riYjIGFJyEBGRAZQcRERkACUHEREZQMlBREQGUHIQGYaZdUXN0vniWM76a2YzomdzFRkvJqQ6AJE00BZORyCSNVRzEBmlcJ2N/xPOn/+cmR0Tlh9lZk+Gk7c9aWZHhuXTzOyhcK79l8zslPClcs3sznD+/V+FI1tFUkrJQWR4Rf1OK10UdV+Luy8Bvk0wQp9w+wfuvgC4H7gtLL8NeNrdTwAWAxvC8tnAd9x9LrAXOD/B+yMyLI2QFhmGme1z95IY5ZuBD7j7m+EkbtvcfbKZ7SJYSOZQWL7V3avMbCdQ5+4Ho15jBsG0yrPD2zcAee7+j4nfM5HBqeYgcnh8kO3BHhPLwajtLtQWKOOAkoPI4bko6vr34fYzBDMDA1xCsFQjwJPANdCzIEtZsoIUGSn9QhEZXlG4slbEY+4e6c5aYGbPEvzQWh6WfRa4x8y+AOwELg/LPwfcYWZ/RVBDuIZgQRmRcUdtDiKjFLY5NLj7rlTHIjLWdFpJREQGUM1BREQGUM1BREQGUHIQEZEBlBxERGQAJQcRERlAyUFERAb4/5Eh9uLw9Ts0AAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "# The following variables are the hyperparameters.\n", + "learning_rate = 0.08\n", + "epochs = 30\n", + "batch_size = 100\n", + "\n", + "# Split the original training set into a reduced training set and a\n", + "# validation set. \n", + "validation_split=0.2\n", + "\n", + "# Identify the feature and the label.\n", + "my_feature=\"median_income\" # the median income on a specific city block.\n", + "my_label=\"median_house_value\" # the median value of a house on a specific city block.\n", + "# That is, you're going to create a model that predicts house value based \n", + "# solely on the neighborhood's median income. \n", + "\n", + "# Discard any pre-existing version of the model.\n", + "my_model = None\n", + "\n", + "# Invoke the functions to build and train the model.\n", + "my_model = build_model(learning_rate)\n", + "epochs, rmse, history = train_model(my_model, train_df, my_feature, \n", + " my_label, epochs, batch_size, \n", + " validation_split)\n", + "\n", + "plot_the_loss_curve(epochs, history[\"root_mean_squared_error\"], \n", + " history[\"val_root_mean_squared_error\"])" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "30/30 [==============================] - 0s 8ms/step - loss: 7066.7402 - root_mean_squared_error: 84.0639\n" + ] + } + ], + "source": [ + "x_test = test_df[my_feature]\n", + "y_test = test_df[my_label]\n", + "\n", + "results = my_model.evaluate(x_test, y_test, batch_size=batch_size)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Representation\n", + "### Feature Engineering\n", + "Integer and floating-point data don't need a special encoding because they can be multiplied by a numeric weight. As suggested in Figure 2, converting the raw integer value 6 to the feature value 6.0 is trivial:\n", + "![](https://developers.google.com/machine-learning/crash-course/images/FloatingPointFeatures.svg)\n", + "\n", + "Categorical features have a discrete set of possible values. For example, there might be a feature called street_name with options that include:\n", + "\n", + "```\n", + "{'Charleston Road', 'North Shoreline Boulevard', 'Shorebird Way', 'Rengstorff Avenue'}\n", + "```\n", + "\n", + "Since models cannot multiply strings by the learned weights, we use feature engineering to convert strings to numeric values.\n", + "\n", + "We can accomplish this by defining a mapping from the feature values, which we'll refer to as the vocabulary of possible values, to integers. Since not every street in the world will appear in our dataset, we can group all other streets into a catch-all \"other\" category, known as an OOV (out-of-vocabulary) bucket.\n", + "\n", + "However, if we incorporate these index numbers directly into our model, it will impose some constraints that might be problematic:\n", + "\n", + "- We'll be learning a single weight that applies to all streets. For example, if we learn a weight of 6 for street_name, then we will multiply it by 0 for Charleston Road, by 1 for North Shoreline Boulevard, 2 for Shorebird Way and so on. Consider a model that predicts house prices using street_name as a feature. It is unlikely that there is a linear adjustment of price based on the street name, and furthermore this would assume you have ordered the streets based on their average house price. Our model needs the flexibility of learning different weights for each street that will be added to the price estimated using the other features.\n", + "- We aren't accounting for cases where street_name may take multiple values. For example, many houses are located at the corner of two streets, and there's no way to encode that information in the street_name value if it contains a single index.\n", + "\n", + "To remove both these constraints, we can instead create a binary vector for each categorical feature in our model that represents values as follows:\n", + "\n", + "- For values that apply to the example, set corresponding vector elements to 1.\n", + "- Set all other elements to 0.\n", + "\n", + "The length of this vector is equal to the number of elements in the vocabulary. This representation is called a **one-hot encoding** when a single value is 1, and a **multi-hot encoding** when multiple values are 1.\n", + "\n", + "Figure 3 illustrates a one-hot encoding of a particular street: Shorebird Way. The element in the binary vector for Shorebird Way has a value of 1, while the elements for all other streets have values of 0.\n", + "\n", + "![](https://developers.google.com/machine-learning/crash-course/images/OneHotEncoding.svg)\n", + "\n", + "This approach effectively creates a Boolean variable for every feature value (e.g., street name). Here, if a house is on Shorebird Way then the binary value is 1 only for Shorebird Way. Thus, the model uses only the weight for Shorebird Way\n", + "\n", + "Suppose that you had 1,000,000 different street names in your data set that you wanted to include as values for street_name. Explicitly creating a binary vector of 1,000,000 elements where only 1 or 2 elements are true is a very inefficient representation in terms of both storage and computation time when processing these vectors. In this situation, a common approach is to use a sparse representation in which only nonzero values are stored. In sparse representations, an independent model weight is still learned for each feature value, as described above.\n", + "\n", + "### Good Features\n", + "- Avoid rarely used discrete feature values\n", + "- Prefer clear and obvious meanings\n", + "- Don't mix \"magic\" values with actual data\n", + "- Account for upstream instability\n", + "\n", + "### Cleaning Data\n", + "1. Scaling feature values\n", + " Scaling means converting floating-point feature values from their natural range (for example, 100 to 900) into a standard range (for example, 0 to 1 or -1 to +1). If a feature set consists of only a single feature, then scaling provides little to no practical benefit. If, however, a feature set consists of multiple features, then feature scaling provides the following benefits:\n", + "\n", + " - Helps gradient descent converge more quickly.\n", + " - Helps avoid the \"NaN trap,\" in which one number in the model becomes a NaN (e.g., when a value exceeds the floating-point precision limit during training), and—due to math operations—every other number in the model also eventually becomes a NaN.\n", + " - Helps the model learn appropriate weights for each feature. Without feature scaling, the model will pay too much attention to the features having a wider range.\n", + " You don't have to give every floating-point feature exactly the same scale. Nothing terrible will happen if Feature A is scaled from -1 to +1 while Feature B is scaled from -3 to +3. However, your model will react poorly if Feature B is scaled from 5000 to 100000.\n", + " \n", + "2. Handling extreme outliers\n", + "3. Binning\n", + " The following plot shows the relative prevalence of houses at different latitudes in California. Notice the clustering—Los Angeles is about at latitude 34 and San Francisco is roughly at latitude 38.\n", + " ![](https://developers.google.com/machine-learning/crash-course/images/ScalingBinningPart1.svg)\n", + " n the data set, latitude is a floating-point value. However, it doesn't make sense to represent latitude as a floating-point feature in our model. That's because no linear relationship exists between latitude and housing values. For example, houses in latitude 35 are not more expensive (or less expensive) than houses at latitude 34. And yet, individual latitudes probably are a pretty good predictor of house values.\n", + "\n", + " To make latitude a helpful predictor, let's divide latitudes into \"bins\" as suggested by the following figure:\n", + " ![](https://developers.google.com/machine-learning/crash-course/images/ScalingBinningPart2.svg)\n", + " Instead of having one floating-point feature, we now have 11 distinct boolean features (LatitudeBin1, LatitudeBin2, ..., LatitudeBin11). Having 11 separate features is somewhat inelegant, so let's unite them into a single 11-element vector. Doing so will enable us to represent latitude 37.4 as follows:\n", + " ```\n", + " [0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0]\n", + " ```\n", + " Thanks to binning, our model can now learn completely different weights for each latitude.\n", + " \n", + "4. Scrubbing\n", + " Clean out the bad data" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Feature Crosses\n", + "- Linear learners use linear models\n", + "- Such learners scale well to massive data e.g., Vowpal Wabbit, sofia-ml\n", + "- But without feature crosses, the expressivity of these models would be limited\n", + "- Using feature crosses + massive data is one efficient strategy for learning highly complex models\n", + " Foreshadowing: neural nets provide another\n", + "\n", + "![](https://developers.google.com/machine-learning/crash-course/images/LinearProblemNot.png)\n", + "\n", + "To solve the nonlinear problem shown in Figure 2, create a feature cross. A feature cross is a synthetic feature that encodes nonlinearity in the feature space by multiplying two or more input features together. (The term cross comes from cross product.) Let's create a feature cross named $x_3$ by crossing $x_1$ and $x_2$ :\n", + " \n", + "$x_3={x_1}x_2$\n", + "\n", + "Thanks to stochastic gradient descent, linear models can be trained efficiently. Consequently, supplementing scaled linear models with feature crosses has traditionally been an efficient way to train on massive-scale data sets.\n", + "\n", + "### Crossing One-Hot Vectors\n", + "Machine learning models do frequently cross one-hot feature vectors.\n", + "\n", + "Linear learners scale well to massive data. Using feature crosses on massive data sets is one efficient strategy for learning highly complex models. Neural networks provide another strategy." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Imported the modules.\n" + ] + } + ], + "source": [ + "import numpy as np\n", + "import pandas as pd\n", + "import tensorflow as tf\n", + "from tensorflow import feature_column\n", + "from tensorflow.keras import layers\n", + "\n", + "from matplotlib import pyplot as plt\n", + "\n", + "# The following lines adjust the granularity of reporting.\n", + "pd.options.display.max_rows = 10\n", + "pd.options.display.float_format = \"{:.1f}\".format\n", + "\n", + "tf.keras.backend.set_floatx('float32')\n", + "\n", + "print(\"Imported the modules.\")" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "# Load the dataset\n", + "train_df = pd.read_csv(\"https://download.mlcc.google.com/mledu-datasets/california_housing_train.csv\")\n", + "test_df = pd.read_csv(\"https://download.mlcc.google.com/mledu-datasets/california_housing_test.csv\")\n", + "\n", + "# Scale the labels\n", + "scale_factor = 1000.0\n", + "# Scale the training set's label.\n", + "train_df[\"median_house_value\"] /= scale_factor \n", + "\n", + "# Scale the test set's label\n", + "test_df[\"median_house_value\"] /= scale_factor\n", + "\n", + "# Shuffle the examples\n", + "train_df = train_df.reindex(np.random.permutation(train_df.index))" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "# Create an empty list that will eventually hold all feature columns.\n", + "feature_columns = []\n", + "\n", + "# Create a numerical feature column to represent latitude.\n", + "latitude = tf.feature_column.numeric_column(\"latitude\")\n", + "feature_columns.append(latitude)\n", + "\n", + "# Create a numerical feature column to represent longitude.\n", + "longitude = tf.feature_column.numeric_column(\"longitude\")\n", + "feature_columns.append(longitude)\n", + "\n", + "# Convert the list of feature columns into a layer that will ultimately become\n", + "# part of the model. Understanding layers is not important right now.\n", + "fp_feature_layer = layers.DenseFeatures(feature_columns)" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Defined the create_model, train_model, and plot_the_loss_curve functions.\n" + ] + } + ], + "source": [ + "def create_model(my_learning_rate, feature_layer):\n", + " \"\"\"Create and compile a simple linear regression model.\"\"\"\n", + " # Most simple tf.keras models are sequential.\n", + " model = tf.keras.models.Sequential()\n", + "\n", + " # Add the layer containing the feature columns to the model.\n", + " model.add(feature_layer)\n", + "\n", + " # Add one linear layer to the model to yield a simple linear regressor.\n", + " model.add(tf.keras.layers.Dense(units=1, input_shape=(1,)))\n", + "\n", + " # Construct the layers into a model that TensorFlow can execute.\n", + " model.compile(optimizer=tf.keras.optimizers.RMSprop(lr=my_learning_rate),\n", + " loss=\"mean_squared_error\",\n", + " metrics=[tf.keras.metrics.RootMeanSquaredError()])\n", + "\n", + " return model \n", + "\n", + "\n", + "def train_model(model, dataset, epochs, batch_size, label_name):\n", + " \"\"\"Feed a dataset into the model in order to train it.\"\"\"\n", + "\n", + " features = {name:np.array(value) for name, value in dataset.items()}\n", + " label = np.array(features.pop(label_name))\n", + " history = model.fit(x=features, y=label, batch_size=batch_size,\n", + " epochs=epochs, shuffle=True)\n", + "\n", + " # The list of epochs is stored separately from the rest of history.\n", + " epochs = history.epoch\n", + " \n", + " # Isolate the mean absolute error for each epoch.\n", + " hist = pd.DataFrame(history.history)\n", + " rmse = hist[\"root_mean_squared_error\"]\n", + "\n", + " return epochs, rmse \n", + "\n", + "\n", + "def plot_the_loss_curve(epochs, rmse):\n", + " \"\"\"Plot a curve of loss vs. epoch.\"\"\"\n", + "\n", + " plt.figure()\n", + " plt.xlabel(\"Epoch\")\n", + " plt.ylabel(\"Root Mean Squared Error\")\n", + "\n", + " plt.plot(epochs, rmse, label=\"Loss\")\n", + " plt.legend()\n", + " plt.ylim([rmse.min()*0.94, rmse.max()* 1.05])\n", + " plt.show() \n", + "\n", + "print(\"Defined the create_model, train_model, and plot_the_loss_curve functions.\")" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Train on 17000 samples\n", + "Epoch 1/30\n", + "17000/17000 [==============================] - 2s 106us/sample - loss: 16333.5866 - root_mean_squared_error: 127.8029\n", + "Epoch 2/30\n", + "17000/17000 [==============================] - 1s 67us/sample - loss: 13558.8470 - root_mean_squared_error: 116.4424\n", + "Epoch 3/30\n", + "17000/17000 [==============================] - 1s 70us/sample - loss: 13524.0235 - root_mean_squared_error: 116.2928\n", + "Epoch 4/30\n", + "17000/17000 [==============================] - 1s 71us/sample - loss: 13473.5717 - root_mean_squared_error: 116.0757\n", + "Epoch 5/30\n", + "17000/17000 [==============================] - 1s 81us/sample - loss: 13426.2199 - root_mean_squared_error: 115.8716\n", + "Epoch 6/30\n", + "17000/17000 [==============================] - 1s 69us/sample - loss: 13375.7435 - root_mean_squared_error: 115.6536\n", + "Epoch 7/30\n", + "17000/17000 [==============================] - 1s 71us/sample - loss: 13338.5112 - root_mean_squared_error: 115.4925\n", + "Epoch 8/30\n", + "17000/17000 [==============================] - 1s 80us/sample - loss: 13272.5055 - root_mean_squared_error: 115.2064\n", + "Epoch 9/30\n", + "17000/17000 [==============================] - 1s 75us/sample - loss: 13253.5646 - root_mean_squared_error: 115.1241\n", + "Epoch 10/30\n", + "17000/17000 [==============================] - 1s 60us/sample - loss: 13221.1494 - root_mean_squared_error: 114.9832\n", + "Epoch 11/30\n", + "17000/17000 [==============================] - 1s 64us/sample - loss: 13191.1990 - root_mean_squared_error: 114.8529\n", + "Epoch 12/30\n", + "17000/17000 [==============================] - 1s 74us/sample - loss: 13152.7171 - root_mean_squared_error: 114.6853\n", + "Epoch 13/30\n", + "17000/17000 [==============================] - 1s 56us/sample - loss: 13126.2001 - root_mean_squared_error: 114.5696\n", + "Epoch 14/30\n", + "17000/17000 [==============================] - 1s 73us/sample - loss: 13105.6793 - root_mean_squared_error: 114.4800\n", + "Epoch 15/30\n", + "17000/17000 [==============================] - 1s 64us/sample - loss: 13075.0472 - root_mean_squared_error: 114.3462\n", + "Epoch 16/30\n", + "17000/17000 [==============================] - 1s 58us/sample - loss: 13050.9815 - root_mean_squared_error: 114.2409\n", + "Epoch 17/30\n", + "17000/17000 [==============================] - 1s 55us/sample - loss: 13038.4157 - root_mean_squared_error: 114.1859\n", + "Epoch 18/30\n", + "17000/17000 [==============================] - 1s 67us/sample - loss: 13007.2251 - root_mean_squared_error: 114.0492\n", + "Epoch 19/30\n", + "17000/17000 [==============================] - 1s 75us/sample - loss: 12978.2992 - root_mean_squared_error: 113.9223\n", + "Epoch 20/30\n", + "17000/17000 [==============================] - 1s 60us/sample - loss: 12985.4248 - root_mean_squared_error: 113.9536\n", + "Epoch 21/30\n", + "17000/17000 [==============================] - 1s 54us/sample - loss: 12965.6588 - root_mean_squared_error: 113.8668\n", + "Epoch 22/30\n", + "17000/17000 [==============================] - 1s 64us/sample - loss: 12952.7086 - root_mean_squared_error: 113.8100\n", + "Epoch 23/30\n", + "17000/17000 [==============================] - 1s 76us/sample - loss: 12932.1628 - root_mean_squared_error: 113.7197\n", + "Epoch 24/30\n", + "17000/17000 [==============================] - 1s 56us/sample - loss: 12917.0542 - root_mean_squared_error: 113.6532\n", + "Epoch 25/30\n", + "17000/17000 [==============================] - 1s 56us/sample - loss: 12911.4840 - root_mean_squared_error: 113.6287\n", + "Epoch 26/30\n", + "17000/17000 [==============================] - 1s 65us/sample - loss: 12897.0609 - root_mean_squared_error: 113.5652\n", + "Epoch 27/30\n", + "17000/17000 [==============================] - 1s 72us/sample - loss: 12888.2173 - root_mean_squared_error: 113.5263\n", + "Epoch 28/30\n", + "17000/17000 [==============================] - 1s 57us/sample - loss: 12870.0053 - root_mean_squared_error: 113.4461\n", + "Epoch 29/30\n", + "17000/17000 [==============================] - 1s 55us/sample - loss: 12861.3957 - root_mean_squared_error: 113.4081\n", + "Epoch 30/30\n", + "17000/17000 [==============================] - 1s 66us/sample - loss: 12848.7744 - root_mean_squared_error: 113.3524\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEGCAYAAABhMDI9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfXRc9X3n8fd3HjQzerKMLdlIxtgGNxtgbYcY0pCUEkgL7Xah2zwCDSTZXZaeUOhhDydLkw0J3XSTkjYtzSYsSQilIdCkgSwJTROS05SmAVIbjGtwA8ZgLNtYsmxJ1uOMRt/9496RZ4Q0GkuaGUnzeZ0zZ+793Tsz38uY+er3cH8/c3dERERyItUOQEREFhYlBhERKaDEICIiBZQYRESkgBKDiIgUiFU7gLlYuXKlr1u3rtphiIgsKtu3bz/i7q3THV/UiWHdunVs27at2mGIiCwqZrav2HE1JYmISAElBhERKaDEICIiBRZ1H4OIyGxlMhk6OzsZGRmpdihlk0wmWbNmDfF4/KRep8QgIjWps7OTpqYm1q1bh5lVO5x55+709PTQ2dnJ+vXrT+q1akoSkZo0MjLCihUrlmRSADAzVqxYMasakRKDiNSspZoUcmZ7fUoMIiJSQIlBRKRKGhsbqx3ClJQYRESkgBKDiMgCsm/fPi655BI2bdrEJZdcwquvvgrAt771Lc455xw2b97MhRdeCMBzzz3H+eefz5YtW9i0aRMvvvjivMSg4aoiUvM+9d3neP5g/7y+51ntzdz2H88+6dfdcMMNXHPNNVx77bXcc8893HjjjXznO9/h9ttv5wc/+AEdHR309vYCcNddd3HTTTdx9dVXk06nyWaz8xK7agwiIgvIE088wVVXXQXABz7wAX76058C8La3vY0PfvCDfPnLX55IAG9961v54z/+Yz772c+yb98+UqnUvMSgGoOI1LzZ/GVfKbkhp3fddRdPPfUUjz76KFu2bGHHjh1cddVVvOUtb+HRRx/l0ksv5Stf+QoXX3zxnD9TNQYRkQXkggsu4MEHHwTg/vvv5+1vfzsAL730Em95y1u4/fbbWblyJfv372fv3r1s2LCBG2+8kcsvv5ydO3fOSwyqMYiIVMnQ0BBr1qyZ2L/55pu58847+fCHP8wdd9xBa2srX/va1wC45ZZbePHFF3F3LrnkEjZv3sxnPvMZvv71rxOPx1m9ejWf+MQn5iUuc/d5eaNq2Lp1q2uhHhGZjd27d/PGN76x2mGU3VTXaWbb3X3rdK9RU5KIiBRQYhARkQJKDCJSsxZzU3opZnt9SgwiUpOSySQ9PT1LNjnk1mNIJpMn/VqNShKRmrRmzRo6Ozvp7u6udihlk1vB7WQpMYhITYrH4ye9slmtUFOSiIgUUGIQEZECSgwiIlJAiUFERAooMYiISIGyJgYzu8fMusxsV17ZH5nZTjPbYWY/NLP2sNzM7E4z2xMeP7ecsYmIyNTKXWO4F7hsUtkd7r7J3bcA3wNy0wH+BrAxfFwHfKnMsYmIyBTKmhjc/XHg6KSy/PXzGoDcbYdXAPd54EmgxcxOLWd8IiLyelW5wc3MPg1cA/QB7wiLO4D9ead1hmWHJr32OoIaBWvXri17rCIitaYqnc/u/jF3Pw24H7ghLLapTp3itXe7+1Z339ra2lrOMEVEalK1RyV9A3hXuN0JnJZ3bA1wsOIRiYjUuIonBjPbmLd7OfBv4fYjwDXh6KRfBvrc/dDr3kBERMqqrH0MZvYAcBGw0sw6gduA3zSzNwDjwD7g+vD0vwN+E9gDDAEfKmdsIiIytbImBne/corir05zrgMfKWc8IiIys2r3MYiIyAKjxCAiIgWUGEREpIASg4iIFFBiEBGRAkoMIiJSQIlBREQKKDGIiEgBJQYRESlQNDGYWdTMvl6pYEREpPqKJgZ3zwKtZlZXoXgqYiw7zv6jQwylx6odiojIglPKXEmvAP9sZo8Ag7lCd/+zcgVVbjv29/Luu57g3g+dx0VvaKt2OCIiC0opieFg+IgATeUNpzI6lqcAONg7UuVIREQWnhkTg7t/CsDMmoJdHyh7VGXW1pQkGjEO9g5XOxQRkQVnxlFJZnaOmT0D7AKeM7PtZnZ2+UMrn2jEWN2cVGIQEZlCKcNV7wZudvfT3f104L8DXy5vWOXX0ZLigBKDiMjrlJIYGtz9H3I77v4ToKFsEVVIe0uSg31KDCIik5WSGPaa2f80s3Xh4+PAy+UOrNzaW1K81jdCdtyrHYqIyIJSSmL4MNAKPBQ+VrIE1mNub0mRyTpHBkarHYqIyIJSdFSSmUWBP3T3GysUT8V0tARDVg/0DrOqOVnlaEREFo5S7nx+c4Viqaj2lty9DOpnEBHJV8oNbs+Edz1/i8I7nx8qW1QV0N4S1BIOHFNiEBHJV0piOAXoAS7OK3OC/oZFqykZpykZU41BRGSSUvoYdrr75ysUT0UF9zJoWgwRkXyl9DFcXqFYKq69JaUag4jIJKU0Jf3MzL4A/A2FfQxPly2qCmlvSfL0q8eqHYaIyIJSSmK4IHy+Pa/MKexzWJTaW1L0DmUYHB2jIVHKfwoRkaWvlNlV31GJQKohdy/Dob5hzmxbEjOKi4jM2bR9DGb253nbN006dm8ZY6qY9omb3NQBLSKSU6zz+cK87WsnHdtUhlgqrkM3uYmIvE6xxGDTbC8ZbU0JLdgjIjJJscQQMbPlZrYib/sUMzsFiM70xmZ2j5l1mdmuvLI7zOzfzGynmT1sZi1h+TozGzazHeHjrjlfWQli0Qirm5Nal0FEJE+xxLAM2A5sA5qBp8P97ZS29vO9wGWTyh4DznH3TcALwK15x15y9y3h4/rSwp+79hat5CYikm/aUUnuvm4ub+zuj5vZukllP8zbfRJ491w+Yz60t6R45tXeaochIrJglLIeQ7l8GPh+3v56M3vGzP7RzH5luheZ2XVmts3MtnV3d885iPaWFIf6hhnXgj0iIkCVEoOZfQwYA+4Piw4Ba939TcDNwDfMrHmq17r73e6+1d23tra2zjmW3II93VqwR0QEqEJiMLNrgd8CrnZ3B3D3UXfvCbe3Ay8Bv1SJeDpy02+rn0FEBCjSxxCOPpqWux892Q8zs8uAjwK/6u5DeeWtwFF3z5rZBmAjsPdk33828hfsOXft8kp8pIjIglZsSoztBHMiGbAWOBZutwCvAuuLvbGZPQBcBKw0s07gNoJRSAngMTMDeDIcgXQhcLuZjQFZ4PrZJJ7Z0EpuIiKFio1KWg8Q3lPwiLv/Xbj/G8A7Z3pjd79yiuKvTnPut4FvlxLwfGtOxmlKxDioaTFERIDS+hjOyyUFAHf/PvCr5Qup8tpbUupjEBEJlTLX9BEz+zjwdYKmpd8lWOpzydBNbiIiJ5RSY7gSaAUeDh+tYdmSoZXcREROKGU9hqPATWbW6O4DFYip4tpbUhwbyjCUHqO+Tgv2iEhtm7HGYGYXmNnzwPPh/mYz+2LZI6ugE9NvqwNaRKSUpqTPA5cS9iu4+7MUrtWw6HUs15BVEZGcku58dvf9k4qyZYilanQvg4jICaU0qO83swsAN7M64EZgd3nDqqxVTQkipsQgIgKl1RiuBz4CdACdwJZwf8k4sWCP+hhERIrWGMwsCnzA3a+uUDxVE9zkNjTziSIiS1zRGoO7Z4ErKhRLVQX3MqjGICJSSlPSP5vZF8zsV8zs3Nyj7JFVmBbsEREJlNL5fEH4fHtemQMXz3841dPRkiSTdY4MjNLWnKx2OCIiVVPKnc/vqEQg1ZYbsnqgd1iJQURqWknzP5jZfwDOBiZ+Md399ulfsfi05939/Ka1VQ5GRKSKSpkS4y7gfcDvEyzU8x7g9DLHVXG6yU1EJFBK5/MF7n4NcMzdPwW8FTitvGFVXnMyRmMipnUZRKTmlZIYcr+UQ2bWDmSYYVnPxcjMtC6DiAil9TF8z8xagDuApwlGJH2lrFFVSXtLioN9SgwiUttKGZX0R+Hmt83se0DS3fvKG1Z1tLek2Nm5JC9NRKRkMyYGM7tmijLc/b7yhFQ9HS0pjg6mGU5nSdVFqx2OiEhVlNKUdF7edhK4hKBJaUkmBoCDfcOc0dpY5WhERKqjlKak38/fN7NlwF+XLaIqyh+yqsQgIrWqpIV6JhkCNs53IAtBe0tw/96BY+qAFpHaVUofw3cJRiJBkEjOAr5ZzqCqZVVzUgv2iEjNK6WP4XN522PAPnfvLFM8VRWPRlilBXtEpMaV0sfwj5UIZKEI1mVQjUFEalcpTUnHOdGUVHAIcHdvnveoqii4l6G32mGIiFRNKU1JnwdeIxiJZMDVQJO7/0k5A6uW9pYkP9g1wvi4E4lYtcMREam4UkYlXeruX3T34+7e7+5fAt5V7sCqpaMlRTo7zpHB0WqHIiJSFaUkhqyZXW1mUTOLmNnVQHamF5nZPWbWZWa78sruMLN/M7OdZvZwOAdT7titZrbHzH5hZpfO7nLmrn3ZiXUZRERqUSmJ4SrgvcBhoItgPYarSnjdvcBlk8oeA85x903AC8CtAGZ2FvB+gsWALgO+aGZVmZNC6zKISK0rZVTSK8AVJ/vG7v64ma2bVPbDvN0ngXeH21cAD7r7KPCyme0BzgeeONnPnasOJQYRqXHT1hjM7L+a2cZw28Kmob6wGejcefjsDwPfD7c7gP15xzrDsqnius7MtpnZtu7u7nkIo1BzKkZDXVQL9ohIzSrWlHQT8Eq4fSWwGdgA3Az8xVw+1Mw+RnCz3P25oilOm2qILO5+t7tvdfetra2tcwljuth0L4OI1LRiiWHM3TPh9m8B97l7j7v/CGiY7Qea2bXh+13t7rkf/04KlwtdAxyc7WfMVZAY1PksIrWpWGIYN7NTzSw31faP8o6lZvNhZnYZ8FHgcncfyjv0CPB+M0uY2XqCSfp+PpvPmA+qMYhILSvW+fwJYBsQBR5x9+cAzOxXgb0zvbGZPQBcBKw0s07gNoJRSAngMTMDeNLdr3f358zsm8DzBE1MH3H3GYfElsua5Sl6tGCPiNSoaRODu3/PzE4nuMv5WN6hbcD7Znpjd79yiuKvFjn/08CnZ3rfSshNv60Fe0SkFhW9j8HdxyYlBdx90N0HyhtWdZ24yU3NSSJSe2azUM+Sp5vcRKSWKTFMYfWyJGZoXQYRqUmlzK6KmXUAp+ef7+6PlyuoaotHI6xqSqrGICI1qZT1GD5L0Nn8PCcmz3NgySYGCDqglRhEpBaVUmP4beAN4TxGNaO9JcWuA33VDkNEpOJK6WPYC8TLHchC09GS4mBfsGCPiEgtKaXGMATsMLMfAxO1Bne/sWxRLQDtLSnSY+P0DKZpbUpUOxwRkYopJTE8Ej5qSv6QVSUGEaklpazH8FeVCGShmbj7uXeYzae1zHC2iMjSUcqopI3A/wbOApK5cnffUMa4qi63YI/WZRCRWlNK5/PXgC8RTG73DuA+4K/LGdRCsCwVp74uqum3RaTmlJIYUu7+Y8DcfZ+7fxK4uLxhVV9uwZ4DvUMznywisoSU0vk8YmYR4EUzuwE4ALSVN6yFQQv2iEgtKqXG8AdAPXAj8Gbgd4FryxnUQtGhBXtEpAaVMirpXwDMzN39Q+UPaeHoaEnSM5hmJJMlGdeCPSJSG2asMZjZW83seWB3uL/ZzL5Y9sgWAE2/LSK1qJSmpD8HLgV6ANz9WeDCcga1UJxIDOpnEJHaUdJ6DO6+f1JR1dZjrqQO1RhEpAaVMippv5ldALiZ1RF0Qu8ub1gLw6rm3II9SgwiUjtKqTFcD3wE6AA6gS3h/pJXF4vQ1pRQjUFEakopo5KOAFdXIJYFqb0lxU9e6ObWh3ayujnFqS1J2pcFz6cuS1JfV9IieCIii8a0v2pmdmexFy71abdzrjxvLfc9+QqPPX+YIwPp1x1flopz6rIk7S0pTl2WZENrIxvbGtm4qpHVzUnMrApRi4jMXrE/d68HdgHfBA4CNfkL997zTuO9550GwEgmy+H+EQ71jXCob5iDvcHza30jHOwdYfu+Y/QNZyZe25iIcUZbkCjODJ83tjWxZnmKSKQm/3OKyCJQLDGcCryHYL3nMeBvgG+7+7FKBLYQJeNRTl/RwOkrGqY87u70DKZ58fAAe7oH2HP4OC92DfD4C9387fbOifMSsQgbWhvZ0NrAGSsbOKOtkQ0rG1nf2kBjQk1TIlJd5j7z0pVm1gFcCdwMfNTdF8Tsqlu3bvVt27ZVO4yS9A1n2NM1wJ6u47x4eICXugfYe2SQ/UeHyF89dFVzgg0rg6SxobWRM1obOLOtkfZlqmWIyPwws+3uvnW646Wsx3AuQVL4NeD7wPb5C692LEvFefPpy3nz6csLykfHsrzaM8RL3QO81D3I3u5B9h4Z4Hs7DxU0S9XXRTlzokmqaaIfY83yeqJKGCIyj4p1Pn8K+C2CexYeBG5197FKBVYrErEoG1c1sXFVU0G5u3N0MM1L3YPs6Rrgxa7j7Oka4Gd7enjo6QN5r49MdHivX9lAR0uKjuUpOlqCkVOJmOZ4EpGTM21TkpmNA3uB3CD+3IkGuLtvKn94xS2mpqT51D8SNksdPpEwXjg8wMG+YfK/TjNobUzQsTxFe0uKNWHSOG15PWd3NNPWlJz+Q0RkyZpLU9L6MsQj86A5Gefctcs5d21hs1R6bJzX+kbo7B3iwLFhDvQOTzzvOtDHY88dJp0dnzi/oyXFltNagsfaFs5pX0aqTjUMkVo3bWJw931zeWMzu4egKarL3c8Jy94DfBJ4I3C+u28Ly9cRNFn9Inz5k+5+/Vw+vxbVxSKsXVHP2hX1Ux4fH3eODIzySs8QOzt7eWZ/Lzte7eXRfz0EQDRivGFVE1vWBsniTae1sKG1UX0YIjWmnGMj7wW+QLBGdM4u4HeA/zvF+S+5+5YyxlPzIhGjrTlJW3OS89efMlHefXyUZ/f3siN8fHfHQb7x1KsTxxsTMZqTMZpTcZqTcZomtk+UNaditDUlWdWcZPWyJMvr47q5T2SRKlticPfHw5pAflluTYdyfazMQmtTgneetYp3nrUKCGoWe48M8MyrvRzoHaZ/eIz+kQzHRzL0D4/xWv8IL3Qd5/jIGP3DmYLhtjl1sQirmhOsbg6TRZgwcomjrSlBa1NCU4qILEClDFe9yd3/YqayebDezJ4B+oGPu/s/TRPPdcB1AGvXrp3nEASCmsWZbU2c2dY047nuzmA6S+9QmsP9oxzuH+G1vpHgOdzedaCPH+0+zEhm/HWvb0zEJpJEW3OQMNqaErQ1J1jVlKRjeYpTl6Woi5U0Q7yIzINS/ly7FpicBD44RdlcHALWunuPmb0Z+I6Zne3u/ZNPdPe7gbshGJU0jzHILJgZjYkYjYkYa5ZP3bcBQQLpHx7jUP8wXf2jdB0fpev4CF39o3SH2zs7e+nqH2U4U7jcR8RgdXOSNcvrWbM8FT7qWXNKMMJq9bIk8agSh8h8KXYfw5XAVQR/yT+Sd6iJcDW3+eLuo8BouL3dzF4CfgmovbGoS5SZsaw+zrL6OP9u9fTnuTsDo2N0HQ9qH53HhsPHEJ3Hhnnq5aN8Z8dwQfNVxILmsBUNCVY01rGyMcHKxjpWNCZY0RDsr8jb1/rdIsUVqzH8jOAv+ZXAn+aVHwd2zmcQZtYKHHX3rJltADYS3EMhNcbMaErGaUrGOaO1ccpzJoblhsli/7EhDveP0DOQ5shgmpePDHJkYHTKpiuAZDzCslQ871FXsN9SHzyf0lDHqrB5q0Wd6VJDZhquug94q5mtAs4LD+0u5Q5oM3sAuAhYaWadwG3AUeAvgVbgUTPb4e6XEqwhfbuZjREsG3q9ux+d/WXJUjbTsNycofQYR46nOTI4Ss9Amp6BUXoG0/QNZ+gbytA7HGwf6B1m96F++oYzDIxO/U+7LhqhrTno/8gli7awY72tKcEpDXUTD9VIZLGbcRK98N6DzwE/Ibjr+VeAW9z9b8se3Qxq9c5nKZ9Mdpz+4Qx9wxl6BtN0hR3qh8P+kK7jIxOd7MdHpk4iqXh0Ikksb6jjlPo4pzQkOKUhqAml6qKk4lHq66J52zFS8XA/LNP9I1Iuc55ED/g4cJ67d4Vv2Ar8CKh6YhCZb/FoJOiLaEywobX4ucPpbNCBfjyokRwbSnN0MM2xwTRHh8LnwTQvHxng2OD0tZGpmMGKhgSrmoMaSvBITDzn7hlZ0VCnWXdl3pWSGCK5pBDqobS1okWWtFRd8fU5JhsdyzIwMsZwJstwOstQOjuxPZwJ99PB8YGRMboHRjncP8prfcGIralWEIxFjNZwuG9rYzDMt7UxQWtzsnC/KVHQxDU+7qSz42Sy42SyTiY7TnrsxH48aixLxWlOxTXiqwaVkhj+3sx+ADwQ7r8P+LvyhSSyNCViURKNs+9/SI+Nc2RglNf6R+jqD5q0XusfCYf7jnKwb4RnO/voGRxlqhbihrooWXcyWSc71V2J06iviwZJIhkPk0Vwx3uurL4u1ywWm2geq881j4XH6uuiNCRiSjKLxIyJwd1vMbPfAd5O0Mdwt7s/XPbIRKRAXSxCe0swU24xY9lxjg6m6Toe3CPSfXyU7oFRjg6miUWMeDQSPGJGXW47GiEeNepiEWKRCJnseNBJP5yZ6HPJPQ70jrD70PGinfXTaQiTzLL6OpalYnkjweomaijLUnEaE1Ea6mI0JHKPKI2JoB9Go8PKr9T5CP4ZyBBMvf3z8oUjInMVi0Ym5sQqt+y4h81gY5OaxILnXPlQOsvA6Bi9Q/lJJhha3DecoXcow+jY1MOL80UMGupi1CeCGkgyFiUWNWIRIxaNTDzHI0Y0TILB8QgNiaAW0xg+N+Q955JQfV2UZDxKIhahLhYhEYsSj1rNJaNSpsR4L3AHJ0Yl/aWZLYhRSSJSXdHIiTvf52okk52onQyMjjE4mmUwPcbgaPAYGA0SzcBorizL6FiWsXFnLK+fZDCdZSw7TnY8KBsbdzJj4wxlsgyNZgumni9VIhYJk0V0YjsRj9KUiNGcioX33sQmJplsCieWzJXX10VJxKLUxSLURSMk4uFzLLIgk04p3+bH0KgkESmzZDz4a73cNZ302DhD6TEG01mGRk88D4yOMZgeYzQzzuhYkGRGx7LBc3ac0cw46bzn4XSWgdEMh/pG+MXh4pNKFlMXDWondWHCKRzOHCMVj0z01+QPc37DqiYueeOqsvw30qgkEakpwY9wHS3F74+cFXdnKJ0NZyMem5iReDiTnUgyQcLJTz7BczqbZSQzznAmy0jY/NY3nOFwX5ahTNhUl84ylMniDpdvbq9qYphqVNL3yxKNiMgiZmYTHeanLivPZ7g7o2PjjM9wc/JcaFSSiMgiYmZln3alpB4jd38IeCgMKmpmV7v7/WWNTEREqmLavgIzazazW83sC2b26xa4gWDW0/dWLkQREamkYjWGvwaOAU8A/wW4BagDrnD3HRWITUREqqBYYtjg7v8ewMy+AhwhWGXteEUiExGRqig27DST23D3LPCykoKIyNJXrMaw2cxyay4bkAr3DXB3by57dCIiUnHFVnDTMlQiIjVIdzCLiEgBJQYRESmgxCAiIgWUGEREpIASg4iIFFBiEBGRAkoMIiJSQIlBREQKKDGIiEgBJQYRESmgxCAiIgWUGEREpIASg4iIFFBiEBGRAmVLDGZ2j5l1mdmuvLL3mNlzZjZuZlsnnX+rme0xs1+Y2aXliktERIorZ43hXuCySWW7gN8BHs8vNLOzgPcDZ4ev+aKZaT0IEZEqKFticPfHgaOTyna7+y+mOP0K4EF3H3X3l4E9wPnlik1ERKa3UPoYOoD9efudYdnrmNl1ZrbNzLZ1d3dXJDgRkVqyUBKDTVHmU53o7ne7+1Z339ra2lrmsEREas9CSQydwGl5+2uAg1WKRUSkpi2UxPAI8H4zS5jZemAj8PMqxyQiUpNi5XpjM3sAuAhYaWadwG0EndF/CbQCj5rZDne/1N2fM7NvAs8DY8BH3D1brthERGR6ZUsM7n7lNIcenub8TwOfLlc8IiJSmoXSlCQiIguEEoOIiBRQYhARkQJKDCIiUkCJQURECigxiIhIASUGEREpoMQgIiIFlBhERKSAEoOIiBRQYhARkQJKDCIiUkCJQURECigxiIhIASUGEREpoMQgIiIFlBhERKSAEoOIiBRQYhARkQJKDCIiUkCJQURECpi7VzuGWTOzbmDfHN5iJXBknsJZCHQ9C99Su6aldj2w9K5pqus53d1bp3vBok4Mc2Vm29x9a7XjmC+6noVvqV3TUrseWHrXNJvrUVOSiIgUUGIQEZECtZ4Y7q52APNM17PwLbVrWmrXA0vvmk76emq6j0FERF6v1msMIiIyiRKDiIgUqMnEYGaXmdkvzGyPmf2PasczH8zsFTP7VzPbYWbbqh3PyTKze8ysy8x25ZWdYmaPmdmL4fPyasZ4sqa5pk+a2YHwe9phZr9ZzRhPhpmdZmb/YGa7zew5M7spLF+U31OR61nM31HSzH5uZs+G1/SpsHy9mT0Vfkd/Y2Z1Rd+n1voYzCwKvAD8GtAJ/Atwpbs/X9XA5sjMXgG2uvuivDHHzC4EBoD73P2csOxPgKPu/pkwgS93949WM86TMc01fRIYcPfPVTO22TCzU4FT3f1pM2sCtgO/DXyQRfg9Fbme97J4vyMDGtx9wMziwE+Bm4CbgYfc/UEzuwt41t2/NN371GKN4Xxgj7vvdfc08CBwRZVjqnnu/jhwdFLxFcBfhdt/RfA/7aIxzTUtWu5+yN2fDrePA7uBDhbp91TkehYtDwyEu/Hw4cDFwN+G5TN+R7WYGDqA/Xn7nSzyfwwhB35oZtvN7LpqBzNPVrn7IQj+JwbaqhzPfLnBzHaGTU2LotllMjNbB7wJeIol8D1Nuh5YxN+RmUXNbAfQBTwGvAT0uvtYeMqMv3m1mBhsirKl0J72Nnc/F/gN4CNhM4YsPF8CzgC2AIeAP61uOCfPzBqBbwN/4O791Y5nrqa4nkX9Hbl71t23AGsIWkjeONVpxd6jFhNDJ3Ba3v4a4GCVYpk37n4wfO4CHib4B7HYHQ7bgXPtwV1VjmfO3P1w+D/uOPBlFtn3FLZbfxu432WoF88AAAKtSURBVN0fCosX7fc01fUs9u8ox917gZ8Avwy0mFksPDTjb14tJoZ/ATaGvfR1wPuBR6oc05yYWUPYeYaZNQC/Duwq/qpF4RHg2nD7WuD/VTGWeZH7AQ39JxbR9xR2bH4V2O3uf5Z3aFF+T9NdzyL/jlrNrCXcTgHvJOg7+Qfg3eFpM35HNTcqCSAcfvbnQBS4x90/XeWQ5sTMNhDUEgBiwDcW2zWZ2QPARQRTBB8GbgO+A3wTWAu8CrzH3RdNZ+4013QRQROFA68A/y3XPr/QmdnbgX8C/hUYD4v/kKBdftF9T0Wu50oW73e0iaBzOUrwh/833f328DfiQeAU4Bngd919dNr3qcXEICIi06vFpiQRESlCiUFERAooMYiISAElBhERKaDEICIiBZQYRGZgZtm8mTZ3zOeMvGa2Ln/2VZGFIDbzKSI1bzicYkCkJqjGIDJL4RoYnw3nv/+5mZ0Zlp9uZj8OJ2H7sZmtDctXmdnD4Vz5z5rZBeFbRc3sy+H8+T8M71gVqRolBpGZpSY1Jb0v71i/u58PfIHgbnrC7fvcfRNwP3BnWH4n8I/uvhk4F3guLN8I/B93PxvoBd5V5usRKUp3PovMwMwG3L1xivJXgIvdfW84Gdtr7r7CzI4QLACTCcsPuftKM+sG1uRPRRBO9/yYu28M9z8KxN39f5X/ykSmphqDyNz4NNvTnTOV/DlrsqjvT6pMiUFkbt6X9/xEuP0zgll7Aa4mWF4R4MfA78HEYirNlQpS5GToLxORmaXCFbFy/t7dc0NWE2b2FMEfWVeGZTcC95jZLUA38KGw/CbgbjP7zwQ1g98jWAhGZEFRH4PILIV9DFvd/Ui1YxGZT2pKEhGRAqoxiIhIAdUYRESkgBKDiIgUUGIQEZECSgwiIlJAiUFERAr8fx8WYgnPMyMfAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + ": Evaluate the new model against the test set:\n", + "3000/1 [================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 51us/sample - loss: 13147.7241 - root_mean_squared_error: 110.9105\n" + ] + }, + { + "data": { + "text/plain": [ + "[12301.132747395834, 110.91047]" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# The following variables are the hyperparameters.\n", + "learning_rate = 0.05\n", + "epochs = 30\n", + "batch_size = 100\n", + "label_name = 'median_house_value'\n", + "\n", + "# Create and compile the model's topography.\n", + "my_model = create_model(learning_rate, fp_feature_layer)\n", + "\n", + "# Train the model on the training set.\n", + "epochs, rmse = train_model(my_model, train_df, epochs, batch_size, label_name)\n", + "\n", + "plot_the_loss_curve(epochs, rmse)\n", + "\n", + "print(\"\\n: Evaluate the new model against the test set:\")\n", + "test_features = {name:np.array(value) for name, value in test_df.items()}\n", + "test_label = np.array(test_features.pop(label_name))\n", + "my_model.evaluate(x=test_features, y=test_label, batch_size=batch_size)" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "resolution_in_degrees = 1.0 \n", + "\n", + "# Create a new empty list that will eventually hold the generated feature column.\n", + "feature_columns = []\n", + "\n", + "# Create a bucket feature column for latitude.\n", + "latitude_as_a_numeric_column = tf.feature_column.numeric_column(\"latitude\")\n", + "latitude_boundaries = list(np.arange(int(min(train_df['latitude'])), \n", + " int(max(train_df['latitude'])), \n", + " resolution_in_degrees))\n", + "latitude = tf.feature_column.bucketized_column(latitude_as_a_numeric_column, \n", + " latitude_boundaries)\n", + "feature_columns.append(latitude)\n", + "\n", + "# Create a bucket feature column for longitude.\n", + "longitude_as_a_numeric_column = tf.feature_column.numeric_column(\"longitude\")\n", + "longitude_boundaries = list(np.arange(int(min(train_df['longitude'])), \n", + " int(max(train_df['longitude'])), \n", + " resolution_in_degrees))\n", + "longitude = tf.feature_column.bucketized_column(longitude_as_a_numeric_column, \n", + " longitude_boundaries)\n", + "feature_columns.append(longitude)\n", + "\n", + "# Convert the list of feature columns into a layer that will ultimately become\n", + "# part of the model. Understanding layers is not important right now.\n", + "buckets_feature_layer = layers.DenseFeatures(feature_columns)" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Train on 17000 samples\n", + "Epoch 1/35\n", + "17000/17000 [==============================] - 1s 75us/sample - loss: 52088.3953 - root_mean_squared_error: 228.2288\n", + "Epoch 2/35\n", + "17000/17000 [==============================] - 1s 44us/sample - loss: 44695.7943 - root_mean_squared_error: 211.4138\n", + "Epoch 3/35\n", + "17000/17000 [==============================] - 1s 52us/sample - loss: 38115.8960 - root_mean_squared_error: 195.2330\n", + "Epoch 4/35\n", + "17000/17000 [==============================] - 1s 71us/sample - loss: 32275.3864 - root_mean_squared_error: 179.6535\n", + "Epoch 5/35\n", + "17000/17000 [==============================] - 1s 81us/sample - loss: 27268.8241 - root_mean_squared_error: 165.1328\n", + "Epoch 6/35\n", + "17000/17000 [==============================] - 1s 83us/sample - loss: 23020.5447 - root_mean_squared_error: 151.7252\n", + "Epoch 7/35\n", + "17000/17000 [==============================] - 2s 91us/sample - loss: 19503.6763 - root_mean_squared_error: 139.6556\n", + "Epoch 8/35\n", + "17000/17000 [==============================] - 1s 82us/sample - loss: 16694.4257 - root_mean_squared_error: 129.2069\n", + "Epoch 9/35\n", + "17000/17000 [==============================] - 1s 79us/sample - loss: 14549.8839 - root_mean_squared_error: 120.6229\n", + "Epoch 10/35\n", + "17000/17000 [==============================] - 2s 99us/sample - loss: 12997.4644 - root_mean_squared_error: 114.0064\n", + "Epoch 11/35\n", + "17000/17000 [==============================] - 2s 96us/sample - loss: 11951.0478 - root_mean_squared_error: 109.3209\n", + "Epoch 12/35\n", + "17000/17000 [==============================] - 1s 77us/sample - loss: 11326.9246 - root_mean_squared_error: 106.4280\n", + "Epoch 13/35\n", + "17000/17000 [==============================] - 1s 88us/sample - loss: 10982.8821 - root_mean_squared_error: 104.7993\n", + "Epoch 14/35\n", + "17000/17000 [==============================] - 1s 63us/sample - loss: 10789.4561 - root_mean_squared_error: 103.8723\n", + "Epoch 15/35\n", + "17000/17000 [==============================] - 1s 84us/sample - loss: 10672.7024 - root_mean_squared_error: 103.3088\n", + "Epoch 16/35\n", + "17000/17000 [==============================] - 2s 99us/sample - loss: 10598.6237 - root_mean_squared_error: 102.9496\n", + "Epoch 17/35\n", + "17000/17000 [==============================] - 1s 80us/sample - loss: 10549.6242 - root_mean_squared_error: 102.7114\n", + "Epoch 18/35\n", + "17000/17000 [==============================] - 1s 81us/sample - loss: 10516.3436 - root_mean_squared_error: 102.5492\n", + "Epoch 19/35\n", + "17000/17000 [==============================] - 1s 77us/sample - loss: 10493.2744 - root_mean_squared_error: 102.4367\n", + "Epoch 20/35\n", + "17000/17000 [==============================] - 2s 99us/sample - loss: 10475.0834 - root_mean_squared_error: 102.3479\n", + "Epoch 21/35\n", + "17000/17000 [==============================] - 2s 92us/sample - loss: 10459.7770 - root_mean_squared_error: 102.2731\n", + "Epoch 22/35\n", + "17000/17000 [==============================] - 1s 75us/sample - loss: 10447.3347 - root_mean_squared_error: 102.2122\n", + "Epoch 23/35\n", + "17000/17000 [==============================] - 2s 93us/sample - loss: 10435.7485 - root_mean_squared_error: 102.1555\n", + "Epoch 24/35\n", + "17000/17000 [==============================] - 1s 80us/sample - loss: 10425.0168 - root_mean_squared_error: 102.1030\n", + "Epoch 25/35\n", + "17000/17000 [==============================] - 1s 74us/sample - loss: 10415.7046 - root_mean_squared_error: 102.0574\n", + "Epoch 26/35\n", + "17000/17000 [==============================] - 2s 89us/sample - loss: 10406.5007 - root_mean_squared_error: 102.0123\n", + "Epoch 27/35\n", + "17000/17000 [==============================] - 1s 79us/sample - loss: 10397.5849 - root_mean_squared_error: 101.9686\n", + "Epoch 28/35\n", + "17000/17000 [==============================] - 1s 76us/sample - loss: 10389.7101 - root_mean_squared_error: 101.9299\n", + "Epoch 29/35\n", + "17000/17000 [==============================] - 1s 83us/sample - loss: 10381.7003 - root_mean_squared_error: 101.8906\n", + "Epoch 30/35\n", + "17000/17000 [==============================] - 2s 94us/sample - loss: 10374.2799 - root_mean_squared_error: 101.8542\n", + "Epoch 31/35\n", + "17000/17000 [==============================] - 1s 83us/sample - loss: 10366.6883 - root_mean_squared_error: 101.8169\n", + "Epoch 32/35\n", + "17000/17000 [==============================] - 1s 80us/sample - loss: 10359.2651 - root_mean_squared_error: 101.7805\n", + "Epoch 33/35\n", + "17000/17000 [==============================] - 1s 83us/sample - loss: 10352.2298 - root_mean_squared_error: 101.7459\n", + "Epoch 34/35\n", + "17000/17000 [==============================] - 1s 69us/sample - loss: 10345.1334 - root_mean_squared_error: 101.7110\n", + "Epoch 35/35\n", + "17000/17000 [==============================] - 1s 68us/sample - loss: 10338.7028 - root_mean_squared_error: 101.6794\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3gc5bn+8e+zqi5yl5skV4zBNm4IY2wwwSTBEMAEQjVgAieEHAikESA9nHP4kZ4QDhAIhJBDQgnNtAChV4NsbIMLuNtylQ1uuEp6fn/sSF6EykrW7uxq7891zbWz78zO3ruXrWenva+5OyIiIgCRsAOIiEjqUFEQEZFaKgoiIlJLRUFERGqpKIiISK3ssAMciB49eviAAQPCjiEiklZmzZq1yd0L61uW1kVhwIABlJWVhR1DRCStmNnKhpbp8JGIiNRSURARkVoqCiIiUiutzymIiLTUvn37KC8vZ/fu3WFHSZj8/HyKi4vJycmJ+zUqCiKSkcrLyykoKGDAgAGYWdhxWp27s3nzZsrLyxk4cGDcr9PhIxHJSLt376Z79+5tsiAAmBndu3dv9p6QioKIZKy2WhBqtOTzqSiIiEgtFQURkZB07Ngx7AifoaIgIiK1VBRERFLIypUrOf744xk5ciTHH388q1atAuDBBx9kxIgRjBo1ikmTJgEwf/58xo0bx+jRoxk5ciSLFy8+4PfXJakikvF+/vh8Fqzd1qrbHNa3Ez89ZXizX3fFFVdw4YUXMn36dO666y6uvPJKHn30Ua6//nqeeeYZioqK2LJlCwC33XYbV111FdOmTWPv3r1UVVUdcG7tKYiIpJA333yT8847D4ALLriA1157DYCJEydy0UUXcccdd9T+8T/qqKO44YYb+MUvfsHKlStp167dAb+/9hREJOO15Bd9stRcVnrbbbcxc+ZMnnzySUaPHs2cOXM477zzOPLII3nyySc54YQT+POf/8zkyZMP6P0yck+hutqZvepj3D3sKCIinzJhwgTuu+8+AO69916OPvpoAJYuXcqRRx7J9ddfT48ePVi9ejXLli1j0KBBXHnllZx66qnMmzfvgN8/I4vCP2eVc/otb7Bo/fawo4hIBtu5cyfFxcW1029/+1tuuukm/vKXvzBy5Ej+9re/8Yc//AGAq6++msMOO4wRI0YwadIkRo0axf3338+IESMYPXo0ixYt4sILLzzgTJbOv5ZLS0u9JYPsfPTJXsb9z7+55JiBXHfioQlIJiKpbuHChRx6aNv//1/f5zSzWe5eWt/6Gbmn0K1DLpMOLuTxOWuprk7foigi0toysigATB3dl7Vbd1O28uOwo4iIpIyMLQqfP7QX7XKyeGzOmrCjiEhI0vnweTxa8vkytih0yMvmi8N78eR769hbWR12HBFJsvz8fDZv3txmC0PNeAr5+fnNel1G36cwdXRfHpuzllcXV3D8ob3CjiMiSVRcXEx5eTkVFRVhR0mYmpHXmiOji8IxQwrp2j6Hx+asVVEQyTA5OTnNGpEsU2Ts4SOAnKwIXxrZh+cWbOCTPZVhxxERCV1GFwWAqaOL2LWviucWbAg7iohI6BJWFMysxMxeNLOFZjbfzK4K2n9lZovMbJ6ZPWJmXWJec52ZLTGzD8zshERli3V4v64UdWmnq5BEREjsnkIl8F13PxQYD1xuZsOA54AR7j4S+BC4DiBYdg4wHJgC3GJmWQnMB0AkYpwyqi+vLN7E5h17Ev12IiIpLWFFwd3XufvsYH47sBAocvdn3b3mAP5bQM2p8anAfe6+x92XA0uAcYnKF+u0MX2pqnaeem9dMt5ORCRlJeWcgpkNAMYAM+ssuhh4OpgvAlbHLCsP2upu61IzKzOzsta6lOyQ3p0Y2quAx+asbZXtiYikq4QXBTPrCDwEfMvdt8W0/5DoIaZ7a5rqefln7ipx99vdvdTdSwsLC1st56mj+1K28mNWf7Sz1bYpIpJuEloUzCyHaEG4190fjmmfDpwMTPP9txOWAyUxLy8GkvbT/dRRfQGYMVd7CyKSuRJ59ZEBdwIL3f23Me1TgGuAU9099mf5DOAcM8szs4HAEODtROWrq6Rbe0r7d2WGDiGJSAZL5J7CROACYLKZzQmmk4CbgQLguaDtNgB3nw88ACwA/gVc7u4HPgp1M0wd3ZcPNmxn0frWHcBbRCRdZOQgOw3ZvGMP4254nq8dM4hrTzyk1bYrIpJKNMhOnLp3zGPSkB48PleD74hIZlJRqGPq6CLWbNnFrFUafEdEMo+KQh1fGKbBd0Qkc6ko1NEhL5svDOvFk/PWsa9Kg++ISGZRUajH1NF9+XjnPl5d3HYH3xARqY+KQj2OGVJIl2DwHRGRTKKiUI/c7AgnHdaHZ+dr8B0RySwqCg348pjo4DvPzF8fdhQRkaRRUWhAaf+ulHRrxyPv6iokEckcKgoNMDNOH1PMa0s2sX7r7rDjiIgkhYpCI748pgh3eFT3LIhIhlBRaMSAHh04vH9XHppVTjr3ESUiEq9Gi4KZZZnZ/yUrTCo6fWwRizfuYP5a9ZwqIm1fo0Uh6Lq60Mxyk5Qn5Zx8WF9ysyI8NLs87CgiIgmXHcc6K4DXzWwG8ElNY+zAOW1Z5/Y5fH5YT2bMWcsPTjqUnCwdcRORtiuev3BrgSeCdQtipoxx+phiNn+yV91eiEib1+Segrv/HMDMCqJPfUfCU6WYY4cW0q1DLg/NXsPkQ3qFHUdEJGGa3FMwsxFm9i7wPjDfzGaZ2fDER0sdOVkRTh3Vl+cWbGDrrn1hxxERSZh4Dh/dDnzH3fu7e3/gu8AdiY2Vek4fW8Teymqeem9d2FFERBImnqLQwd1frHni7i8BHRKWKEUdVtSZg3p25GFdhSQibVg8RWGZmf3YzAYE04+A5YkOlmrMjNPHFvHOio9ZtXln2HFERBIinqJwMVAIPBxMPYCvNvUiMysxsxfNbKGZzTezq4L2bmb2nJktDh67Bu1mZjeZ2RIzm2dmY1v+sRLjtNFFmKFO8kSkzWryjmbgB+5+pbuPDaZvuXs8o9pXAt9190OB8cDlZjYMuBZ43t2HAM8HzwFOBIYE06XArS37SInTt0s7JgzuzsPvqtsLEWmb4rmj+fCWbNjd17n77GB+O7AQKAKmAn8NVvsrcFowPxW4x6PeArqYWZ+WvHcinT6mmJWbdzJ7VTx1UUQkvcRz+OhdM5thZheY2ek1U3PexMwGAGOAmUAvd18H0cIB9AxWKwJWx7ysPGiru61LzazMzMoqKpJ/M9mUEb1pl5PFQ7N1CElE2p54ikI3YDMwGTglmE6O9w3MrCPwEPAtd2+sVzmrp+0zx2jc/XZ3L3X30sLCwnhjtJoOedlMGdGbJ+auZfe+qqS/v4hIIjV6R3NwTmGeu/+uJRs3sxyiBeFed384aN5gZn3cfV1weGhj0F4OlMS8vJhoFxsp5/SxRTzy7hpeWLSRkw5LuSNcIiItFs85hVNbsmEzM+BOYGGdzvNmANOD+enAYzHtFwZXIY0HttYcZko1Ewb3oFenPB7WISQRaWPi6SX1DTO7GbifT/eSOruJ100ELgDeM7M5QdsPgBuBB8zsEmAVcGaw7CngJGAJsJM4LnsNS1bEOG1MEXe+upzNO/bQvWNe2JFERFpFPEVhQvB4fUybEz3H0CB3f436zxMAHF/P+g5cHkeelHD6mGL+9PIyHp+7losmDgw7johIq4inl9TjkhEk3QztXcCIok48/O4aFQURaTMaPKdgZr+Pmb+qzrK7E5gpbZw+pph55Vv5cMP2sKOIiLSKxk40T4qZn15n2cgEZEk7U0f3JTtiPFi2uumVRUTSQGNFwRqYl0D3jnl8/tBePPLuGvZVVYcdR0TkgDVWFCJm1tXMusfMdzOzbkBWkvKlvDNLi9m0Yy8vLtrY9MoiIimusRPNnYFZ7N9LiL0EVb3BBY49uJDCgjwenFXOF4f3DjuOiMgBabAouPuAJOZIW9lZEU4fW8SfX11OxfY9FBbongURSV/x9H0kTTjz8BKqqp1HNc6CiKQ5FYVWcFDPjozt14UHylZrnAURSWsqCq3kzNISFm/cwdzyrWFHERFpscZuXuvW2JTMkOng5JF9yM+J8IDuWRCRNNbYnsIsoCx4rAA+BBYH87MSHy29FOTncNKIPjw+Zy279mqcBRFJTw0WBXcf6O6DgGeAU9y9h7t3JzrAzsMNvS6TfaW0mO17Knlm/vqwo4iItEg85xSOcPenap64+9PAsYmLlL7GD+xOSbd2PDhLh5BEJD3FUxQ2mdmPzGyAmfU3sx8SHZ5T6ohEjK+MLeH1JZtZ/dHOsOOIiDRbPEXhXKAQeCSYCoM2qccZhxdhBg/NLg87iohIs8UznsJHwFVm1tHddyQhU1or7tqeiYN78GBZOVdOHkIkor4ERSR9NLmnYGYTzGwBsCB4PsrMbkl4sjR2Zmkxa7bs4q1lOsomIuklnsNHvwNOIDiP4O5z+fRYC1LHCcN7U5CfrXsWRCTtxHVHs7vX/eumC/EbkZ+Txamj+vL0++vZtntf2HFEROIWT1FYbWYTADezXDP7HrAwwbnS3lmlJeyprObxuWvDjiIiErd4isJlwOVAEVAOjA6eN8rM7jKzjWb2fkzbaDN7y8zmmFmZmY0L2s3MbjKzJWY2z8zGtuzjpI6RxZ05uFdHHizTVUgikj4aLQpmlgVc4O7T3L2Xu/d09/PdPZ4zqHcDU+q0/RL4ubuPBn4SPAc4ERgSTJcCtzbjM6QkM+Os0hLmrN7C4g3bw44jIhKXRouCu1cBU1uyYXd/BfiobjPQKZjvDNQcW5kK3ONRbwFdzKxPS943lZw2pojsiPHgLO0tiEh6iOfw0etmdrOZHWNmY2umFr7ft4Bfmdlq4NfAdUF7ERB7Mrs8aPsMM7s0OPRUVlFR0cIYydGjYx7HHdKTh2eXs6+qOuw4IiJNiqcoTACGA9cDvwmmX7fw/b4BfNvdS4BvA3cG7fXd4VXvaDXufru7l7p7aWFhYQtjJM85R5SwacdenluwIewoIiJNiueO5uNa8f2mA1cF8w8Cfw7my4GSmPWK2X9oKa19bmhPirq04//eWslJh6X9ETERaePiuk/BzL5kZt83s5/UTC18v7Xs72F1MtHxGQBmABcGVyGNB7a6+7oWvkdKyYoY544r4Y2lm1laoV5CRCS1xdPNxW3A2cA3iR7mORPoH8fr/gG8CQw1s3IzuwT4GvAbM5sL3ED0SiOAp4BlwBLgDuA/m/9RUtdZR5SQHTH+PnNV2FFERBrV5OEjYIK7jzSzee7+czP7DXEMsuPuDfWkeng96zpx3PuQrnoW5HPC8N78c1Y5V58wlPycrLAjiYjUK57DR7uCx51m1hfYBwxMXKS2adr4fmzdtY8n5rWJo2Ii0kbFUxSeMLMuwK+A2cAK4L5EhmqLjhrUnUGFHbh35sqwo4iINKjJouDu/+XuW9z9IaLnEg5x9x8nPlrbYmZMO7I/767awvy1W8OOIyJSr3hONF9YMxE94Tw1mJdm+srYYvKyI9yrE84ikqLiOXx0RMx0DPAz4NQEZmqzOrfP4ZRRfXn03TVsV5faIpKC4jl89M2Y6WvAGCA38dHapvPH92fn3ioefXdN2FFERD4jrpvX6thJtDdTaYFRxZ0Z3rcT985cRfRKXBGR1BHPOYXHzWxGMD0BfAA8lvhobZOZcf74/ixav51ZKz8OO46IyKfEc/NabOd3lcBKd1df0Afg1FF9ueHJhdw7cxWlA7qFHUdEpFY85xRejpleV0E4cB3ysvny2CKenLeOjz7ZG3YcEZFa8Rw+2m5m2+qZtpvZtmSEbIumHdmfvVXV/HPW6qZXFhFJknhONP8OuJbooDfFwDXAf7t7gbt3avSV0qChvQs4YkBX7p25iupqnXAWkdQQT1E4wd1vcfft7r7N3W8Fzkh0sExw/vj+rNy8k9eXbgo7iogIEF9RqDKzaWaWZWYRM5sGVCU6WCaYMqI33Trk8n9vqT8kEUkN8RSF84CzgA3ARqLjKZyXyFCZIi87izNLi/n3wo2s37o77DgiInFdfbTC3ae6e49gOs3dVyQhW0aYNq4/VdXOfe+oPyQRCV+DRcHMvmZmQ4J5M7O7zGyrmc0zs7HJi9i29evenkkHF3Lf26vZV1UddhwRyXCN7SlcRXTsBIBzgVHAIOA7wB8SGyuzTD+qP+u37ebp99eHHUVEMlxjRaHS3Wu68jwZuMfdN7v7v4EOiY+WOY4b2pNBPTpw56vL1B+SiISqsaJQbWZ9zCwfOB74d8yydomNlVkiEeOrEwcwt3yr+kMSkVA1VhR+ApQRPYQ0w93nA5jZscCyxEfLLGccXkzndjnc+drysKOISAZrsCi4+xNEh988NBhHoUYZ0RHYGhWcmN5oZu/Xaf+mmX1gZvPN7Jcx7deZ2ZJg2QnN/yjprX1uNucd2Y9n5q9n9Uc7w44jIhmq0UtS3b3S3T+u0/aJu++IY9t3A1NiG8zsOGAqMNLdhxP0wGpmw4BzgOHBa24xs6x4P0RbMf2oAUTM+MvrK8KOIiIZqiWD7MTF3V8BPqrT/A3gRnffE6yzMWifCtzn7nvcfTmwBBiXqGypqnfnfL40sg8PlK3WcJ0iEoqEFYUGHAwcY2YzzexlMzsiaC8CYrsLLQ/aMs4lRw9kx55K7n9HvaeKSPLFVRTMrMjMJpjZpJqphe+XDXQFxgNXAw+YmQFWz7r1XptpZpeaWZmZlVVUVLQwRuoaWdyFcQO6cfcbK6jUzWwikmTxjKfwC+B14EdE/5BfDXyvhe9XDjzsUW8D1UCPoL0kZr1iYG19G3D329291N1LCwsLWxgjtV189EDKP97Fsws2hB1FRDJMPMNxngYMrTkPcIAeBSYDL5nZwUAusAmYAfzdzH4L9AWGAG+3wvulpS8M60W/bu2587XlnHRYn7DjiEgGiefw0TIgp7kbNrN/AG8CQ82s3MwuAe4CBgWXqd4HTA/2GuYDDwALgH8Bl7t7xnbPnRXczDZr5cfMWb0l7DgikkGsqW4VzOwhov0ePQ/U7i24+5WJjda00tJSLysrCztGQuzYU8lRNzzP5w7pyR/PHRN2HBFpQ8xslruX1rcsnj2FGcB/AW8As2ImSaCOedmcM66Ep95bx5otu8KOIyIZoslzCu7+12QEkc+aPmEAd762nHveWMF1Jx0adhwRyQDxXH00xMz+aWYLzGxZzZSMcJmuuGt7TjysD39/exWf7KkMO46IZIB4Dh/9BbgVqASOA+4B/pbIULLfJUcPZPvuSv45qzzsKCKSAeIpCu3c/XmiJ6VXuvvPiF5WKkkwtl9XxvTrwl9eX05VtcZaEJHEiqco7DazCLDYzK4wsy8DPROcS2JccvRAVmzeyfMLdTObiCRWPEXhW0B74ErgcOB8YHoiQ8mnTRnem6Iu7fizxloQkQRrsii4+ztBV9kfu/tX3f0Md38rCdkkkJ0V4asTB/D28o8oW1G341kRkdYTz9VHR5nZAmBh8HyUmd2S8GTyKdOO7E/3Drnc9MKSsKOISBsWz+Gj3wMnAJsB3H0u0NJeUqWF2uVm8bVJg3jlwwreXaVxnEUkMeLqOtvd63bun7H9EoXpgvH96do+hz9qb0FEEiSeorDazCYAbma5ZvY9gkNJklwd8rL5j2MG8cKijbxXvjXsOCLSBsVTFC4DLic6Elo5MDp4LiG48Kj+dMrP5qYXFocdRUTaoHj6PtoETEtCFolDQX4OFx89kN//ezEL1m5jWN9OYUcSkTakwaJgZjc19sJU6Do7U311wkDufHU5N7+4mFumHR52HBFpQxrbU7gMeJ/o4DdrqX8cZQlB5/Y5XDRxAH98YQkfrN/O0N4FYUcSkTaisXMKfYDbiV6OegHR0ddmuPtf1Z12+C6eOJAOuVnc/KKuRBKR1tNgUXD3ze5+m7sfB1wEdAHmm9kFyQonDevaIZcLjhrAE/PWsmTjjrDjiEgbEc8dzWOJ9n90PvA0GnUtZfzHMQPJz87if7W3ICKtpMGiYGY/N7NZwHeAl4FSd7/E3RckLZ00qkfHPM4f34/H5qxhxaZPwo4jIm1AY3sKPwY6A6OA/wfMNrN5Zvaemc1LSjpp0tcmDSInK6K9BRFpFY1dfTQwaSmkxXoW5HPekf24582VXHn8EEq6tQ87koikscZONK9sbGpqw2Z2l5ltNLP361n2PTNzM+sRPDczu8nMlgR7I2MP7GNllq9PGkyWGbe8pL0FETkwcXWI10J3A1PqNppZCfAFYFVM84nAkGC6lOiY0BKn3p3zOfuIEv45q5zyj3eGHUdE0ljCioK7vwLUNyLM74DvA7EDDk8F7vGot4AuZtYnUdnaoss+NxiA215eGnISEUln8VySelU8bfEws1OBNcGYDLGKgNjuucuDtvq2camZlZlZWUVFRUtitElFXdpxZmkJ97+zmuW6EklEWiiePYX6xmO+qLlvZGbtgR8CP6lvcT1tXk8b7n67u5e6e2lhYWFzY7Rp3/r8EHKzItzwlHo2F5GWaaxDvHOB84CBZjYjZlEBwShszTSY6BVNc80MoJjoZa7jiO4ZlMSsW0y0vyVphp4F+fzncQfxq2c+4I0lm5hwUI+wI4lImmnsktQ3gHVAD+A3Me3bgWbfp+Du7wE9a56b2QqiN8RtCorOFWZ2H3AksNXd1zX3PQQuOXogf5+5iuufWMCTVx5DVkT9GIpI/Jq6JPUldz8KWER0D6EAKHf3yqY2bGb/AN4EhppZuZld0sjqTwHLgCXAHcB/NuMzSIz8nCyuPfEQFq3fzoNldUdRFRFpXDwnms8E3gbOBM4CZprZV5p6nbuf6+593D3H3Yvd/c46ywcEA/gQXHV0ubsPdvfD3L2sZR9HAE4e2YfD+3fl189+wPbd+8KOIyJpJJ4TzT8CjnD36e5+ITCOaBcYkqLMjJ+cPIxNO/Zyy0u6RFVE4hdPUYi4+8aY55vjfJ2EaFRJF748pog7X1vO6o90Q5uIxCeeP+7/MrNnzOwiM7sIeJLoOQBJcd+fMpSIwY1PLwo7ioikiSaLgrtfDfwJGEm0x9Tb3f2aRAeTA9enczu+PmkwT763jndW1HdzuYjIp8V7GOh14EXg+WBe0sTXjx1E7075/NcTC6iurvd+QBGRWvFcfXQW0auPvkIzrj6S1NA+N5vvTxnKvPKtPPLumrDjiEiKi2dP4Yfo6qO0dtroIkYWd+aXzyxi594mbzERkQymq48yQCQSvUR1w7Y9/OnlZWHHEZEU1tKrj55ObCxpbaUDuvGlkX340ytLWbd1V9hxRCRFtfTqo+8nOpi0vmunHEK16xJVEWlYXIeB3P1hd/+Ou38bmGFm0xKcSxKgpFt7Ljt2MI/NWcuz89eHHUdEUlCDRcHMOpnZdWZ2s5l9MRhH+QqiHdedlbyI0pquOO4ghvftxHUPv8emHXvCjiMiKaaxPYW/AUOB94D/AJ4l2ineVHefmoRskgC52RF+d/Zotu+p5NqH3sNd9y6IyH6NFYVB7n6Ru/8JOBcoBU529znJiSaJcnCvAr5/wlD+vXADD5aVhx1HRFJIY0Whts9ld68Clrv79sRHkmS4eOJAxg/qxs8fn68O80SkVmNFYZSZbQum7cDImnkz25asgJIYkYjx6zNHETHjuw/MpUpdYIgIjY+8luXunYKpwN2zY+Y7JTOkJEZx1/b89NThvL3iI+58TTe1iYjuTM54Z4wt4oThvfj1Mx+yaL12AEUynYpChjMzbvjyYXRql8O375/LnsqqsCOJSIhUFITuHfO48fTDWLhuG7//9+Kw44hIiFQUBIDPD+vF2aUl/OnlpZRpQB6RjJWwomBmd5nZRjN7P6btV2a2yMzmmdkjZtYlZtl1ZrbEzD4wsxMSlUsa9uNThlHUtR3feWAuO/aoi22RTJTIPYW7gSl12p4DRrj7SOBD4DoAMxsGnAMMD15zi5llJTCb1KNjXja/OXM0qz/eyU8fm6+7nUUyUMKKgru/AnxUp+1Zd6/5CfoWUBzMTwXuc/c97r4cWEJ0MB9JsnEDu/HNyUN4aHY5f3xhSdhxRCTJskN874uB+4P5IqJFokZ50PYZZnYpcClAv379EpkvY33780Mo/3gnv33uQ3oW5HHOOH3PIpkilBPNZvZDoBK4t6apntXqPXbh7re7e6m7lxYWFiYqYkYzM35xxkiOPbiQHz76Ps8v3BB2JBFJkqQXBTObDpwMTPP9B63LgZKY1YqBtcnOJvvlZEW4ZdpYhvftxOV/n83sVR+HHUlEkiCpRcHMpgDXAKe6e2wvbDOAc8wsz8wGAkOAt5OZTT6rQ142d110BL065XPJ3e+wtGJH2JFEJMESeUnqP4A3gaFmVm5mlwA3AwXAc2Y2x8xuA3D3+cADwALgX8DlQc+sErIeHfO45+JxZEWMC+98mw3bdocdSUQSyNL5ssPS0lIvKysLO0ZGmFe+hXNuf4v+3Ttw/9fH0yk/J+xIItJCZjbL3UvrW6Y7miUuI4u7cOv5h7N4w3Yu+9ss9ZEk0kapKEjcjj24kF9+ZSRvLN3Mdx+YS7XGYBBpc8K8T0HS0Olji9m4fQ83Pr2Igvwcrp86nJws/bYQaStUFKTZvj5pENt27eOWl5aydOMObp42hp4F+WHHEpFWoJ940mxmxvenHMLvzx7NvDVbOOWPrzFrpe5jEGkLVBSkxU4bU8TD35hIbnaEc25/k3tnrlQneiJpTkVBDsiwvp14/IqjmTC4Bz985H2ueWgeu/fpyiSRdKWiIAesS/tc7rroCL45+SAeKCvnzNveZM2WXWHHEpEWUFGQVpEVMb77xaHcfsHhLN/0Caf88TVeX7Ip7Fgi0kwqCtKqvji8N49dMZFuHXK54M6Z/PJfi9iyc2/YsUQkTioK0uoGF3bk0csnMnV0Ebe8tJSJN77AjU8vYtOOPWFHE5EmqO8jSahF67dx8wtLePK9deRlR5h2ZH++PmkQPTvpvgaRsDTW95GKgiTF0ood/O+LS3hszlqyIsbZpSVc9rnBFHVpF3Y0kYyjoiApY+XmT7j1paU8NLscgDPGFnP++P4M69OJSKS+AfhEpLWpKEjKWbNlF7e9tKAUCd4AAAocSURBVJT731nN3qpqenTM5eiDejDp4EKOHtJD3WaIJJCKgqSsTTv28PIHFbyyuILXFm9i8yfRK5UO6V3AsQcXcsyQQkoHdCU/JyvkpCJth4qCpIXqamfBum28sriCVz/cRNnKj9hX5eTnRBjWpxODCzsyuGfH6GNhB/p1a0+2emgVaTYVBUlLn+ypZObyzby6eBOL1m1nacUONm7ff1lrTpbRv3sHBhd2YHBhR/p2aUdhQR49C/IoLMijR8c87WGI1KOxoqCusyVldcjLZvIhvZh8SK/atq279rGsYgdLKz5hacUOlm7cwZKNO3h+4UYq6xn0p1N+Nj075VPYMVoourTPoVN+DgX52XRqFzzm59TOF+Rn0yE3m3Y5WTrxLRlJRUHSSud2OYzp15Ux/bp+qr2yqprNn+ylYvseNm7fTcX2PbXTxuBxbvkWtu7ax7Zd+4hn0Li87AjtcrNol5O1/zGYz8uOkJsdITcr+pgTPOZmR8gL5rOzImRHjJysCNlZRk4k+pidFSEnYrXLsyJGdsSIBI/R5xGygvmsCGRFImSZEYmwv92ij5GYeTNi5lXUpPlUFKRNyM6K0KtTPr065QOdG13X3dm5t4ptu/exfXcl23YFj7v3sW13Jbv2VrJzbxW79lWxa28w7dv/uH13JZsrq9lbVc3eymCqM58qaoqHWXQ+YkbEqC0kZtGiE22PFp3aeds/X/f1FrTFPq+7fuw2PrU8UrN+7HIw6ltnf3t971nz3NjfTux2iX5W+9R2osuAz2TlM++5P2t0W9FlZo3nq7tupM4yq/3MQT7q/zxWk+Ez24l2RNmjY16r/5tJWFEws7uAk4GN7j4iaOsG3A8MAFYAZ7n7xxb9SfMH4CRgJ3CRu89OVDbJbGZGh7xsOuRl06fx+tEi7s6+Kqeyujr6WFVNZbWzr6qayk+1O1XuVFVH26uqo88rq52qquAxaKuujj6vrq55zf6p2mseiZnfv261R0/iR5dF16mZqqqjeWveh5ptBI/uTnU1VAXzVdWOA9UefV10nZptQlV1db3rVFfzqXVj36va9y/zmOex6zvRzxC8LOYzULudmvY0Pk3aLJcdO5hrTzyk1bebyD2Fu4GbgXti2q4Fnnf3G83s2uD5NcCJwJBgOhK4NXgUSTtmRm62kauuxUJTX3GJFhSPKVbEFKb9RSXaVmfd6ujz2CLltUUruqyqen9BqlvQavNU1zwPXh/Mf7qo7c9b971iC9/gwo4J+e4SVhTc/RUzG1CneSrwuWD+r8BLRIvCVOAej14K9ZaZdTGzPu6+LlH5RKTtMjOyDLLQeZXmSvZPmV41f+iDx55BexGwOma98qDtM8zsUjMrM7OyioqKhIYVEck0qbJ/W185r/fIoLvf7u6l7l5aWFiY4FgiIpkl2UVhg5n1AQgeNwbt5UBJzHrFwNokZxMRyXjJLgozgOnB/HTgsZj2Cy1qPLBV5xNERJIvkZek/oPoSeUeZlYO/BS4EXjAzC4BVgFnBqs/RfRy1CVEL0n9aqJyiYhIwxJ59dG5DSw6vp51Hbg8UVlERCQ+qXKiWUREUkBa95JqZhXAyha+vAewqRXjJIMyJ0e6ZU63vKDMydJQ5v7uXu/lm2ldFA6EmZU11HVsqlLm5Ei3zOmWF5Q5WVqSWYePRESkloqCiIjUyuSicHvYAVpAmZMj3TKnW15Q5mRpduaMPacgIiKflcl7CiIiUoeKgoiI1MrIomBmU8zsAzNbEgz2k/LMbIWZvWdmc8ysLOw89TGzu8xso5m9H9PWzcyeM7PFwWPXxraRTA3k/ZmZrQm+5zlmdlKYGesysxIze9HMFprZfDO7KmhP5e+5ocwp+V2bWb6ZvW1mc4O8Pw/aB5rZzOA7vt/McsPOWqORzHeb2fKY73h0k9vKtHMKZpYFfAh8gWjvrO8A57r7glCDNcHMVgCl7p6yN8+Y2SRgB9EBk2qGYP0l8FHMaHtd3f2aMHPWaCDvz4Ad7v7rMLM1JOhduI+7zzazAmAWcBpwEan7PTeU+SxS8LsOhgfu4O47zCwHeA24CvgO8LC732dmtwFz3f3WMLPWaCTzZcAT7v7PeLeViXsK44Al7r7M3fcC9xEd+U0OkLu/AnxUp3kq0VH2CB5PS2qoRjSQN6W5+7qa8cvdfTuwkOiAVKn8PTeUOSV51I7gaU4wOTAZqPnjmmrfcUOZmy0Ti0Lco7ylGAeeNbNZZnZp2GGaoaHR9lLZFWY2Lzi8lDKHYeoKhrsdA8wkTb7nOpkhRb9rM8syszlEx3x5DlgKbHH3ymCVlPu7UTezu9d8x/8TfMe/M7O8praTiUUh7lHeUsxEdx8LnAhcHhz6kNZ3KzAYGA2sA34Tbpz6mVlH4CHgW+6+Lew88agnc8p+1+5e5e6jiQ74NQ44tL7VkpuqcXUzm9kI4DrgEOAIoBvQ5CHFTCwKaTnKm7uvDR43Ao8Q/YeaDhoabS8lufuG4D9XNXAHKfg9B8eMHwLudfeHg+aU/p7ry5wO37W7bwFeAsYDXcysZriBlP27EZN5SnDozt19D/AX4viOM7EovAMMCa4kyAXOITryW8oysw7BCTrMrAPwReD9xl+VMhoabS8l1fxhDXyZFPuegxOKdwIL3f23MYtS9ntuKHOqftdmVmhmXYL5dsDniZ4HeRH4SrBaqn3H9WVeFPNDwYieA2nyO864q48Agkvffg9kAXe5+/+EHKlRZjaI6N4BRAdG+nsqZraY0faADURH23sUeADoRzDanrunxMndBvJ+jujhDAdWAF9PpaFhzexo4FXgPaA6aP4B0WP0qfo9N5T5XFLwuzazkURPJGcR/eH8gLtfH/w/vI/oYZh3gfODX+ChayTzC0Ah0cPmc4DLYk5I17+tTCwKIiJSv0w8fCQiIg1QURARkVoqCiIiUktFQUREaqkoiIhILRUFkUaYWVVMD5NzrBV71TWzARbTQ6tIKshuehWRjLYr6DpAJCNoT0GkBSw6vsUvgj7s3zazg4L2/mb2fNAB2fNm1i9o72VmjwT93c81swnBprLM7I6gD/xng7tRRUKjoiDSuHZ1Dh+dHbNsm7uPA24meoc8wfw97j4SuBe4KWi/CXjZ3UcBY4H5QfsQ4H/dfTiwBTgjwZ9HpFG6o1mkEWa2w9071tO+Apjs7suCzt7Wu3t3M9tEdECZfUH7OnfvYWYVQHFstwhBN9LPufuQ4Pk1QI67/3fiP5lI/bSnINJy3sB8Q+vUJ7bvnCp0nk9CpqIg0nJnxzy+Gcy/QbTnXYBpRIdFBHge+AbUDobSKVkhRZpDv0pEGtcuGM2qxr/cveay1Dwzm0n0x9W5QduVwF1mdjVQAXw1aL8KuN3MLiG6R/ANogPLiKQUnVMQaYHgnEKpu28KO4tIa9LhIxERqaU9BRERqaU9BRERqaWiICIitVQURESkloqCiIjUUlEQEZFa/x+OOaKf7LQkpQAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + ": Evaluate the new model against the test set:\n", + "3000/1 [================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 77us/sample - loss: 11427.7853 - root_mean_squared_error: 100.9818\n" + ] + }, + { + "data": { + "text/plain": [ + "[10197.325406901042, 100.98182]" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# The following variables are the hyperparameters.\n", + "learning_rate = 0.04\n", + "epochs = 35\n", + "\n", + "# Build the model, this time passing in the buckets_feature_layer.\n", + "my_model = create_model(learning_rate, buckets_feature_layer)\n", + "\n", + "# Train the model on the training set.\n", + "epochs, rmse = train_model(my_model, train_df, epochs, batch_size, label_name)\n", + "\n", + "plot_the_loss_curve(epochs, rmse)\n", + "\n", + "print(\"\\n: Evaluate the new model against the test set:\")\n", + "my_model.evaluate(x=test_features, y=test_label, batch_size=batch_size)" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "resolution_in_degrees = 1.0 \n", + "\n", + "# Create a new empty list that will eventually hold the generated feature column.\n", + "feature_columns = []\n", + "\n", + "# Create a bucket feature column for latitude.\n", + "latitude_as_a_numeric_column = tf.feature_column.numeric_column(\"latitude\")\n", + "latitude_boundaries = list(np.arange(int(min(train_df['latitude'])), int(max(train_df['latitude'])), resolution_in_degrees))\n", + "latitude = tf.feature_column.bucketized_column(latitude_as_a_numeric_column, latitude_boundaries)\n", + "\n", + "# Create a bucket feature column for longitude.\n", + "longitude_as_a_numeric_column = tf.feature_column.numeric_column(\"longitude\")\n", + "longitude_boundaries = list(np.arange(int(min(train_df['longitude'])), int(max(train_df['longitude'])), resolution_in_degrees))\n", + "longitude = tf.feature_column.bucketized_column(longitude_as_a_numeric_column, longitude_boundaries)\n", + "\n", + "# Create a feature cross of latitude and longitude.\n", + "latitude_x_longitude = tf.feature_column.crossed_column([latitude, longitude], hash_bucket_size=100)\n", + "crossed_feature = tf.feature_column.indicator_column(latitude_x_longitude)\n", + "feature_columns.append(crossed_feature)\n", + "\n", + "# Convert the list of feature columns into a layer that will later be fed into\n", + "# the model. \n", + "feature_cross_feature_layer = layers.DenseFeatures(feature_columns)" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "WARNING:tensorflow:From /home/saurabh/anaconda3/envs/ultron/lib/python3.7/site-packages/tensorflow_core/python/feature_column/feature_column_v2.py:4276: IndicatorColumn._variable_shape (from tensorflow.python.feature_column.feature_column_v2) is deprecated and will be removed in a future version.\n", + "Instructions for updating:\n", + "The old _FeatureColumn APIs are being deprecated. Please use the new FeatureColumn APIs instead.\n", + "WARNING:tensorflow:From /home/saurabh/anaconda3/envs/ultron/lib/python3.7/site-packages/tensorflow_core/python/feature_column/feature_column_v2.py:4331: CrossedColumn._num_buckets (from tensorflow.python.feature_column.feature_column_v2) is deprecated and will be removed in a future version.\n", + "Instructions for updating:\n", + "The old _FeatureColumn APIs are being deprecated. Please use the new FeatureColumn APIs instead.\n", + "Train on 17000 samples\n", + "Epoch 1/35\n", + "17000/17000 [==============================] - 1s 45us/sample - loss: 53546.8460 - root_mean_squared_error: 231.4019\n", + "Epoch 2/35\n", + "17000/17000 [==============================] - 1s 38us/sample - loss: 48502.8735 - root_mean_squared_error: 220.2337\n", + "Epoch 3/35\n", + "17000/17000 [==============================] - 0s 27us/sample - loss: 43826.0806 - root_mean_squared_error: 209.3468\n", + "Epoch 4/35\n", + "17000/17000 [==============================] - 0s 19us/sample - loss: 39458.4162 - root_mean_squared_error: 198.6414\n", + "Epoch 5/35\n", + "17000/17000 [==============================] - 0s 20us/sample - loss: 35430.6029 - root_mean_squared_error: 188.2302\n", + "Epoch 6/35\n", + "17000/17000 [==============================] - 0s 20us/sample - loss: 31757.4062 - root_mean_squared_error: 178.2061\n", + "Epoch 7/35\n", + "17000/17000 [==============================] - 0s 19us/sample - loss: 28399.7324 - root_mean_squared_error: 168.5222\n", + "Epoch 8/35\n", + "17000/17000 [==============================] - 0s 18us/sample - loss: 25374.4155 - root_mean_squared_error: 159.2935\n", + "Epoch 9/35\n", + "17000/17000 [==============================] - 0s 20us/sample - loss: 22669.0720 - root_mean_squared_error: 150.5625\n", + "Epoch 10/35\n", + "17000/17000 [==============================] - 0s 19us/sample - loss: 20262.4072 - root_mean_squared_error: 142.3461\n", + "Epoch 11/35\n", + "17000/17000 [==============================] - 0s 18us/sample - loss: 18148.7474 - root_mean_squared_error: 134.7173\n", + "Epoch 12/35\n", + "17000/17000 [==============================] - 0s 17us/sample - loss: 16338.6607 - root_mean_squared_error: 127.8228\n", + "Epoch 13/35\n", + "17000/17000 [==============================] - 0s 16us/sample - loss: 14797.5092 - root_mean_squared_error: 121.6451\n", + "Epoch 14/35\n", + "17000/17000 [==============================] - 0s 17us/sample - loss: 13513.8720 - root_mean_squared_error: 116.2492\n", + "Epoch 15/35\n", + "17000/17000 [==============================] - 0s 29us/sample - loss: 12458.3139 - root_mean_squared_error: 111.6168\n", + "Epoch 16/35\n", + "17000/17000 [==============================] - 1s 36us/sample - loss: 11637.7391 - root_mean_squared_error: 107.8783\n", + "Epoch 17/35\n", + "17000/17000 [==============================] - 1s 31us/sample - loss: 11028.8733 - root_mean_squared_error: 105.0185\n", + "Epoch 18/35\n", + "17000/17000 [==============================] - 0s 20us/sample - loss: 10607.6470 - root_mean_squared_error: 102.9934\n", + "Epoch 19/35\n", + "17000/17000 [==============================] - 0s 20us/sample - loss: 10325.9042 - root_mean_squared_error: 101.6165\n", + "Epoch 20/35\n", + "17000/17000 [==============================] - 0s 20us/sample - loss: 10136.3310 - root_mean_squared_error: 100.6793\n", + "Epoch 21/35\n", + "17000/17000 [==============================] - 0s 18us/sample - loss: 10004.0588 - root_mean_squared_error: 100.0203\n", + "Epoch 22/35\n", + "17000/17000 [==============================] - 0s 20us/sample - loss: 9906.8830 - root_mean_squared_error: 99.5333\n", + "Epoch 23/35\n", + "17000/17000 [==============================] - 0s 19us/sample - loss: 9830.7450 - root_mean_squared_error: 99.1501\n", + "Epoch 24/35\n", + "17000/17000 [==============================] - 0s 20us/sample - loss: 9773.0076 - root_mean_squared_error: 98.8585\n", + "Epoch 25/35\n", + "17000/17000 [==============================] - 0s 20us/sample - loss: 9731.9590 - root_mean_squared_error: 98.6507\n", + "Epoch 26/35\n", + "17000/17000 [==============================] - 0s 18us/sample - loss: 9701.7866 - root_mean_squared_error: 98.4976\n", + "Epoch 27/35\n", + "17000/17000 [==============================] - 0s 17us/sample - loss: 9679.7487 - root_mean_squared_error: 98.3857\n", + "Epoch 28/35\n", + "17000/17000 [==============================] - 0s 18us/sample - loss: 9662.7869 - root_mean_squared_error: 98.2995\n", + "Epoch 29/35\n", + "17000/17000 [==============================] - 0s 20us/sample - loss: 9650.5098 - root_mean_squared_error: 98.2370\n", + "Epoch 30/35\n", + "17000/17000 [==============================] - 0s 18us/sample - loss: 9641.1149 - root_mean_squared_error: 98.1892\n", + "Epoch 31/35\n", + "17000/17000 [==============================] - 0s 25us/sample - loss: 9633.8857 - root_mean_squared_error: 98.1524\n", + "Epoch 32/35\n", + "17000/17000 [==============================] - 0s 22us/sample - loss: 9628.3779 - root_mean_squared_error: 98.1243\n", + "Epoch 33/35\n", + "17000/17000 [==============================] - 0s 24us/sample - loss: 9624.0378 - root_mean_squared_error: 98.1022\n", + "Epoch 34/35\n", + "17000/17000 [==============================] - 1s 50us/sample - loss: 9620.7955 - root_mean_squared_error: 98.0856\n", + "Epoch 35/35\n", + "17000/17000 [==============================] - 1s 48us/sample - loss: 9618.2665 - root_mean_squared_error: 98.0728\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd5hU5fn/8fe9jYWls0uRtjSRIiAiCopGsKBRSYwaO8ZCTDDWGDUmtqhfjd34U4IJKsYaKyr2aIwFFJAOAlJXyoKUpbjA7t6/P+bsOuKWYWH2zO58Xtd1rjnznHNmPjPi3Hva85i7IyIiApASdgAREUkcKgoiIlJGRUFERMqoKIiISBkVBRERKZMWdoA9kZ2d7bm5uWHHEBGpVaZOnbrO3XPKWxa3omBm7YHxQGugBBjr7g9ELf89cBeQ4+7rzMyAB4DjgW3Aee4+rbL3yM3NZcqUKfH6CCIidZKZLatoWTz3FIqAq9x9mpk1Aqaa2bvuPjcoGEcDy6PWPw7oFkwHA48EjyIiUkPidk7B3VeV/qXv7puBeUDbYPF9wB+A6DvnRgDjPWIS0NTM2sQrn4iI/FiNnGg2s1zgAGCymZ0EfOPuM3ZZrS2wIup5Ht8XERERqQFxP9FsZg2BF4HLiRxSuh44prxVy2n7UR8cZjYKGAXQoUOHvRdURJLKzp07ycvLo7CwMOwocZOZmUm7du1IT0+PeZu4FgUzSydSEJ5y95fMbH+gEzAjcl6ZdsA0MxtIZM+gfdTm7YCVu76mu48FxgIMGDBAHTeJSLXk5eXRqFEjcnNzCX6P6hR359tvvyUvL49OnTrFvF3cDh8FVxP9E5jn7vcCuPssd2/p7rnunkukEPR399XABOBcizgE2OTuq+KVT0SSW2FhIS1atKiTBQHAzGjRosVu7wnFc0/hUOAcYJaZTQ/a/ujuEytYfyKRy1EXEbkk9VdxzCYiUmcLQqnqfL64FQV3/5jyzxNEr5MbNe/A6HjlERGRqqmbCxGRkDRs2DDsCD+ioiAiImVUFEREEsiyZcsYNmwYffr0YdiwYSxfHun44d///je9e/emb9++HH744QDMmTOHgQMH0q9fP/r06cPChQv3+P1rdYd4IiJ7w82vzWHuyoK9+po992nMjSf22u3tLrnkEs4991xGjhzJuHHjuPTSS3nllVe45ZZbePvtt2nbti0bN24EYMyYMVx22WWcddZZ7Nixg+Li4j3OrT0FEZEE8tlnn3HmmWcCcM455/Dxxx8DcOihh3Leeefx6KOPlv34Dxo0iNtvv50777yTZcuWUb9+/T1+f+0piEjSq85f9DWl9LLSMWPGMHnyZN544w369evH9OnTOfPMMzn44IN54403OPbYY/nHP/7B0KFD9+j9knJP4Yul6zl1zKcUFO4MO4qIyA8MHjyYZ599FoCnnnqKww47DICvv/6agw8+mFtuuYXs7GxWrFjB4sWL6dy5M5deeiknnXQSM2fO3OP3T8qiUD89lS+WbuDhD74OO4qIJLFt27bRrl27sunee+/lwQcf5LHHHqNPnz48+eSTPPBAZBiaq6++mv3335/evXtz+OGH07dvX5577jl69+5Nv379mD9/Pueee+4eZ7LIPWO104ABA7y6g+xc+dx0Xp+1iv9cdQTtmjXYy8lEJNHNmzePHj16hB0j7sr7nGY21d0HlLd+Uu4pAFx1bHcMuPvtr8KOIiKSMJK2KLRtWp8LDuvEK9NXMjNvY9hxREQSQtIWBYDf/KQLLbIyuPWNedTmw2giUj11/f/76ny+pC4KjTLTufzoffl8yXrenbsm7DgiUoMyMzP59ttv62xhKB1PITMzc7e2S/r7FE4/qD2Pf7KEO96cz5H7tSQ9NanrpEjSaNeuHXl5eaxduzbsKHFTOvLa7kj6opCemsJ1x/XgwvFTeHryckYOzg07kojUgPT09N0akSxZ6M9iYFiPlgzq3IL731ugG9pEJKmpKBC5jfz6n/Zgw7aduqFNRJKaikKgd9smnHxAW8Z9soS8DdvCjiMiEgoVhSi/D25ou0s3tIlIklJRiLJP0/pcOKQTr05fyYwVuqFNRJJP3IqCmbU3sw/MbJ6ZzTGzy4L2u8xsvpnNNLOXzaxp1DbXmdkiM/vKzI6NV7bKXHxE5Ia22ybqhjYRST7x3FMoAq5y9x7AIcBoM+sJvAv0dvc+wALgOoBg2elAL2A48LCZpcYxX7l0Q5uIJLO4FQV3X+Xu04L5zcA8oK27v+PuRcFqk4DSOytGAM+6+3Z3XwIsAgbGK19lzjioPV1ysrjjzfnsLC4JI4KISChq5JyCmeUCBwCTd1l0PvBmMN8WWBG1LC9o2/W1RpnZFDObEq87EdNSU/jj8T1YvG4r/5q0LC7vISKSiOJeFMysIfAicLm7F0S1X0/kENNTpU3lbP6jg/ruPtbdB7j7gJycnHhEBmDofi0Z0i2b+99byIatO+L2PiIiiSSuRcHM0okUhKfc/aWo9pHACcBZ/v3Z3DygfdTm7YCV8cxXGTPjTz/tyebCnTzw/sKwYoiI1Kh4Xn1kwD+Bee5+b1T7cOAa4CR3j75LbAJwupnVM7NOQDfg83jli0X31o048+AOPDlpGYvyN4cZRUSkRsRzT+FQ4BxgqJlND6bjgYeARsC7QdsYAHefAzwPzAXeAka7e3Ec88XkiqP2pUFGKre+MS/sKCIicRe3XlLd/WPKP08wsZJtbgNui1em6mjRsB6XDevGrW/M44Ov8jmye8uwI4mIxI3uaI7BuYNyyW3RgNvemKdLVEWkTlNRiEFGWgrX/7Qni/K38PTk5WHHERGJGxWFGB3VoyWHdm3Bfe8tYOM2XaIqInWTikKMSi9RLfhOl6iKSN2lorAberRpzOkDO/DkZ8tYlL8l7DgiInudisJuuvLofamfnsrtE3WJqojUPSoKuym7YT1+N6wr/5mfz38XxKfvJRGRsKgoVMPIwbl0bNGAW1+fS5EuURWROkRFoRrqpaXyx+N7sDB/C898rktURaTuqLQomFmqmf2rpsLUJsf0bMWgzi24990FbNq2M+w4IiJ7RaVFIeh7KMfMMmooT61hZvz5hJ5s+m4n9723IOw4IiJ7RSx9Hy0FPjGzCcDW0sbonk+TVc99GnPGwEgvqmcd3IFurRqFHUlEZI/Eck5hJfB6sG6jqEmAq47pTlZGKre8Ppfvh4YQEamdqtxTcPebAcysUeSp666tKM2zMrji6H25+bW5vDcvn6N7tgo7kohItVW5p2Bmvc3sS2A2MMfMpppZr/hHqz3OPqQj3Vo25NY35rK9KPQhIEREqi2Ww0djgSvdvaO7dwSuAh6Nb6zaJT01hRtO7Mmyb7cx7uOlYccREam2WIpClrt/UPrE3T8EsuKWqJYa0i2Ho3q04qH/LCS/oDDsOCIi1RJLUVhsZn82s9xg+hOwJN7BaqM//bQHO4udO9/6KuwoIiLVEktROB/IAV4KpmzgV/EMVVvlZmdx/mGdeHFaHl8u3xB2HBGR3VblHc3AH939UnfvH0yXu7t+8SpwydCu5DSqx82vzaWkRJeoikjtEssdzQdW54XNrL2ZfWBm88xsjpldFrQ3N7N3zWxh8NgsaDcze9DMFpnZTDPrX533DVvDemlcM3w/pq/YyMtffhN2HBGR3RLL4aMvzWyCmZ1jZieXTjFsVwRc5e49gEOA0WbWE7gWeN/duwHvB88BjgO6BdMo4JHd/TCJ4uQD2tK3fVPufGs+W7YXhR1HRCRmsRSF5sC3wFDgxGA6oaqN3H2Vu08L5jcD84C2wAjgiWC1J4CfBfMjgPEeMQloamZtduOzJIyUFOPGE3uSv3k7D3+wKOw4IiIxq/SO5uCcwkx3v29P3sTMcoEDgMlAK3dfBZHCYWYtg9XaAiuiNssL2lbt8lqjiOxJ0KFDhz2JFVf9OzTj5APa8o//LeGXB7WnYwtdxSsiiS+Wcwon7ckbmFlD4EXgcncvqGzV8iKUk2msuw9w9wE5OTl7Ei3urjluP9JSjVvf0NCdIlI7xHL46FMze8jMhphZ/9Iplhc3s3QiBeEpd38paF5TelgoeMwP2vOA9lGbtyPSGV+t1apxJqOP7Mq7c9do6E4RqRViKQqDgV7ALcA9wXR3VRuZmQH/BObt0s32BGBkMD8SeDWq/dzgKqRDgE2lh5lqswuHdKJTdhY3T5ijfpFEJOHF0kvqkdV87UOBc4BZZjY9aPsjcAfwvJldACwHTg2WTQSOBxYB26gjN8jVS0vlxhN7ct5jX/DPj5fw2590DTuSiEiFKiwKZna/u18ezF/m7g9ELXvc3c+r7IXd/WPKP08AMKyc9R0YHUvo2uYn3VtydM9W/O39Rfz8gLa0aVI/7EgiIuWq7PDR4VHzI3dZ1icOWeq0G07oSYm7TjqLSEKrrChYBfNSDe2bN+C3P+nKGzNX8cmidWHHEREpV2VFIcXMmplZi6j55mbWHEitoXx1yq+P6Ez75vW5ccIcdhaXhB1HRORHKisKTYCpwBSgMTAteD4VjdFcLZnpqdx4Qi8W5W/h8U+Whh1HRORHKjzR7O65NZgjaRzVsxVD92vJ/e8tYES/fWjZODPsSCIiZWK5T0H2shtO6MnOYuf2iTrpLCKJRUUhBLnZWfz6iM68Mn0lkxd/G3YcEZEyKgoh+e1PutK2aeSkc5FOOotIgqiwKJReaVTRVJMh66L6Gan8+YQezF+9mScnLQs7jogIUPmeQumVR1OBtcACYGEwPzX+0eq+Y3u1Zki3bO59ZwFrN28PO46ISMVFwd07uXtn4G3gRHfPdvcWRAbYeami7SR2ZsZNJ/WisKiYO96cH3YcEZGYzikc5O4TS5+4+5vAEfGLlFy65DTkgsM68+K0PD5fsj7sOCKS5GIpCuvM7E9mlmtmHc3seiLDc8pecumwyEnn61+exY4inXQWkfDEUhTOAHKAl4MpJ2iTvaRBRhq3jOjFwvwt/OPjxWHHEZEkFst4CuuBy8ysobtvqYFMSWlYj1Yc26sVD76/kBP77EP75g3CjiQiSajKPQUzG2xmc4G5wfO+ZvZw3JMloZtO6kWqGX9+dTaR4SVERGpWLIeP7gOOJTiP4O4z+OFYC7KXtGlSnyuP6c6HX63lzdmrw44jIkkopjua3X3FLk0abDhORg7qSM82jbn5tTlsLtwZdhwRSTKxFIUVZjYYcDPLMLPfA+rJLU7SUlO4/eT9yd+8nXveWRB2HBFJMrEUhYuJjJ3cFsgD+lFHx1JOFP3aN+Xsgzsy/rOlzMrbFHYcEUkilRYFM0sFznH3s9y9lbu3dPez3b3K+xTMbJyZ5ZvZ7Ki2fmY2ycymm9kUMxsYtJuZPWhmi8xsppn13+NPVstdPbw7LRrW448vz6K4RCedRaRmVFoU3L0YGFHN134cGL5L21+Bm929H3BD8BzgOKBbMI0CHqnme9YZjTPT+fMJPZn1zSae/Gxp2HFEJEnEcvjoEzN7yMyGmFn/0qmqjdz9I2DXfhucyNCeEBnuc2UwPwIY7xGTgKZm1ibGz1BnndinDUO6ZXP3OwtYU1AYdhwRSQKxFIXBQC/gFuCeYLq7mu93OXCXma0IXuO6oL0tEH2FU17Q9iNmNio49DRl7dq11YxRO5gZfxnRmx3FJdzy2tyw44hIEqiyKLj7keVMQ6v5fr8BrnD39sAVwD+DdivvrSvIM9bdB7j7gJycnGrGqD1ys7P43ZFdeWPWKj74Kj/sOCJSx8V0n4KZ/dTM/mBmN5RO1Xy/kXzf7fa/gYHBfB7QPmq9dnx/aCnpjTqiM51zsrjh1dls21EUdhwRqcNi6eZiDPBL4HdE/qI/FehYzfdbyffdbg8lMmgPwATg3OAqpEOATe6+qprvUefUS0vl9p/vz4r133H/ewur3kBEpJpiOqfg7ucCG9z9ZmAQP/yrvlxm9gzwGdDdzPLM7ALgIuAeM5sB3E7kSiOAicBiYBHwKPDb3f4kddwhnVtwxsAO/ON/i5mZtzHsOCJSR1XZSyrwXfC4zcz2IdIHUqeqNnL3irrXPrCcdR3dEFela4/bj/fnreGaF2cx4ZJDSU+N6eifiEjMYvlVed3MmgJ3AdOApcCz8Qwl5WtSP51bRvRm3qoCHv2fxl0Qkb0vlquP/uLuG939RSLnEvZz9z/HP5qUZ3jv1gzv1Zr731vIknVbw44jInVMLCeazy2diJxwHhHMS0huGdGLemkpXPviTErUBYaI7EWxHD46KGoaAtwEnBTHTFKFlo0zuf74Hkxesp7np+zaq7mISPXFMhzn76Kfm1kT4Mm4JZKY/PKg9rwy/RtumziPofu1pGXjzLAjiUgdUJ3LV7YR6bhOQmRm/N/JfdheVMKNE+aEHUdE6ohYzim8ZmYTgul14Cvg1fhHk6p0ys7i8qO68ebs1byl4TtFZC+I5T6F6M7vioBl7p4Xpzyymy4a0pnXZqzihldnM6hLC5rUTw87kojUYrFckvrfqOkTFYTEkp6awp2/2J91W7Zzx5vzw44jIrVcLIePNptZQTnTZjMrqImQUrk+7ZpywWGdeObz5UxaXOWgeCIiFYrlRPN9wLVExjdoB1wD3Orujdy9caVbSo258ujudGjegOtemsV3O4rDjiMitVQsReFYd3/Y3Te7e4G7PwL8It7BZPfUz0jljpP3Z8m6rdz19ldhxxGRWiqWolBsZmeZWaqZpZjZWYD+FE1Ag7tmc+6gjjz26RIm6zCSiFRDLEXhTOA0YA2QT2Q8hTPjGUqq75rh+9G+WQOufmGmBuQRkd0Wy9VHS919hLtnB9PP3H1pDWSTasiql8Zdp/Rh+fpt3KmrkURkN1VYFMzsIjPrFsybmY0zs01mNtPM+tdcRNldB3duwa8OzeWJz5bx6aJ1YccRkVqksj2Fy4iMnQBwBtAX6AxcCTwQ31iyp/5w7H50ys7i6hdmsmW7DiOJSGwqKwpF7r4zmD8BGO/u37r7e0BW/KPJnqifkcrdp/Zh5abvuH3ivLDjiEgtUVlRKDGzNmaWCQwD3otaVj++sWRvOLBjcy4a0pmnJy/nowVrw44jIrVAZUXhBmAKkUNIE9x9DoCZHQFoLMha4sqj96VLThbXvjiTgsKdVW8gIkmtwqLg7q8TGX6zh7tfFLVoCpER2CoVnJjON7PZu7T/zsy+MrM5ZvbXqPbrzGxRsOzY3f8oUp7M9FTuPrUvqwsKufX1uWHHEZEEV+klqe5e5O4bdmnb6u5bYnjtx4Hh0Q1mdiQwAujj7r0IemA1s57A6UCvYJuHzSw11g8hlTugQzN+fUQXnp+Sxwfz88OOIyIJrDqD7MTE3T8C1u/S/BvgDnffHqxT+gs1AnjW3be7+xJgETAwXtmS0eVHdWPfVg259qWZbNqmw0giUr64FYUK7AsMMbPJZvZfMzsoaG8LRA82nBe0/YiZjTKzKWY2Ze1anTyNVb20VO45tR/rtuzgptc0UpuIlC+momBmbc1ssJkdXjpV8/3SgGbAIcDVwPNmZoCVs66X9wLuPtbdB7j7gJycnGrGSE77t2vC6CO78vKX3/DajJVhxxGRBFTlyGtmdieRE8tz+b4jPAc+qsb75QEvubsDn5tZCZAdtLePWq8doF+tOPjd0K58tGAt1788i/4dm9G2qa4uFpHvxbKn8DOgu7sf7+4nBtNJ1Xy/V4ChAGa2L5ABrAMmAKebWT0z6wR0Az6v5ntIJdJTU3jg9H4UlzhXPjed4pJyd8hEJEnFUhQWA7s98K+ZPQN8BnQ3szwzuwAYB3QOLlN9FhjpEXOA54nsjbwFjHZ3dc8dJx1bZHHTSb2YvGQ9f//o67DjiEgCqfLwEbANmG5m7wPbSxvd/dLKNnL3MypYdHYF698G3BZDHtkLTjmwHR8uWMu97yzgsK7Z9GnXNOxIIpIAYtlTmAD8BfgUmBo1SS1mZtz+s/3JaVSPy56drrEXRASIYU/B3Z+oiSBS85o0SOfe0/px5j8m8ZfX5/J/J/cJO5KIhKzKPQUz62ZmL5jZXDNbXDrVRDiJv0FdWnDxEV145vMVvDV7ddhxRCRksRw+egx4BCgCjgTGA0/GM5TUrCuO2pf92zbh2pdmsqagMOw4IhKiWIpCfXd/HzB3X+buNxFcVip1Q0ZaCvef3o/tO0u46vkZlOgyVZGkFUtRKDSzFGChmV1iZj8HWsY5l9SwLjkNueHEnny8aB3jPlkSdhwRCUksReFyoAFwKXAgkUtKR8YzlITj9IPac0zPVvz1ra+Ys3JT2HFEJARVFgV3/yLoKnuDu//K3X/h7pNqIJvUMDPjjl/0oWmDdC595ku2amxnkaQTy9VHg8xsLjAveN7XzB6OezIJRfOsDO4/vR9L1m3lT6/MJtJNlYgki1gOH90PHAt8C+DuM4Dq9pIqtcDgLtlcftS+vPzlNzz3xYqqNxCROiOmrrPdfddfBvVLVMeNPrIrQ7plc8OEOcxdWRB2HBGpIbEUhRVmNhhwM8sws98THEqSuis1xbjvl/1o1iCd0U9PY3OhRmsTSQaxFIWLgdFERkLLA/oFz6WOy25Yj7+d0Z/l67dx3UuzdH5BJAnEcvXROnc/y91buXtLdz/b3b+tiXASvoGdmnPVMfvy+sxV/GvSsrDjiEicVdghnpk9WNmGVXWdLXXHxYd34Ysl6/nL6/Po174Z+7drEnYkEYmTyvYULgYOIzIs5hR+2G22us5OIikpxr2n9SO7YQajn57Gpu90fkGkrqqsKLQBxhK5HPUcIqOvTXD3J9SddvJplpXB387sz8qN3/GHF2bo/IJIHVVhUXD3b919jLsfCZwHNAXmmNk5NRVOEsuBHZtx7XH78facNTz2ydKw44hIHMRyR3N/Iv0fnQ28iQ4dJbULDuvE0T1bcfvEeXy5fEPYcURkL6uwKJjZzWY2FbgS+C8wwN0vcPe5NZZOEo6ZcfcpfWndJJPfPjWNtZu3V72RiNQale0p/BloAvQF/g+YZmYzzWyWmc2s6oXNbJyZ5ZvZ7HKW/d7M3Myyg+dmZg+a2aLgPfpX8/NIDWjSIJ2/n3MgG7bt4Df/msr2It3gLlJXVFYUOgHDgBOC6cRgKp2vyuPA8F0bzaw9cDSwPKr5OKBbMI0iMtKbJLBe+zTh7lP7MmXZBm58dY5OPIvUERXep+Due3Snkrt/ZGa55Sy6D/gD8GpU2whgvEd+WSaZWVMza+Puq/Ykg8TXCX32Yd6qAv7fB1/Ta5/GnDMoN+xIIrKHYuoQb28xs5OAb4KeVqO1BaI73csL2sp7jVFmNsXMpqxduzZOSSVWVx3dnWH7teTm1+by2de60V2ktquxomBmDYDrgRvKW1xOW7nHI9x9rLsPcPcBOTk5ezOiVENKinH/6f3Izc7it09NZcX6bWFHEpE9EMslqZfF0haDLkTOU8wws6VAOyInr1sT2TNoH7VuOyJ3Ukst0CgznUfPHUBxiXPR+Cls26ER20Rqq1j2FMobj/m83X0jd58VdKiX6+65RApBf3dfDUwAzg2uQjoE2KTzCbVLp+ws/nZmfxas2czv/607nkVqq8ruUzjDzF4DOpnZhKjpA4JR2CpjZs8AnwHdzSzPzC6oZPWJwGJgEfAo8Nvd+hSSEI7YN4frjuvBxFmreeg/i8KOIyLVUOHVR8CnwCogG7gnqn0zUOV9Cu5+RhXLc6PmHY3RUCdcOKQTc1cVcM+7C+jeuhHH9GoddiQR2Q2V9X20zN0/dPdBwHygUTDlubsOGku5zIz/O3l/+rZrwhXPTWfBms1hRxKR3RDLieZTgc+BU4HTgMlmdkq8g0ntlZmeyt/PGUCDemn86rEvWFNQGHYkEYlRLCea/wQc5O4j3f1cYCCRLjBEKtS6SSbjRh7Exm07GDnucwo0xrNIrRBLUUhx9/yo59/GuJ0kuf3bNWHMOQeyKH8Lo8ZPoXCn+kgSSXSx/Li/ZWZvm9l5ZnYe8AaRq4VEqjSkWw53n9qXSYvXc+Xz0yku0aWqIomssquPAHD3q83sZCJDcxow1t1fjnsyqTN+dkBb1m3Zzq1vzCOn4RxuOqkXZuXdxC4iYauyKAQ+AXYS6Xri8/jFkbrqwiGdWVNQyKP/W0LLxpmMPrJr2JFEpByxXH10GpFCcAq6+kj2wHXH9eBn/fbhrre/4vkpK6reQERqXCx7CtcTufooH8DMcoD3gBfiGUzqnpQU46+n9OXbrTu47qVZtMjKYFiPVmHHEpEouvpIalRGWgqPnH0gPds0ZvTT05imcZ5FEkp1rz56M76xpC5rWC+NcecdRKvGmZz/+Bcsyt8SdiQRCVRZFNz9auDvQB8i4zWPdfc/xDuY1G05jeox/vyBpKUYZz46icVrVRhEEkFMh4Hc/SV3v9LdrwAmmNlZcc4lSaBjiyyeuvAQikuc08dO4msVBpHQVdZ1dmMzu87MHjKzY4KxDi4h0sX1aTUXUeqy7q0b8fRFkcJwhgqDSOgq21N4EugOzAIuBN4h0ineCHcfUQPZJEl0b92IZ0YdQolH9hh0jkEkPJUVhc7ufp67/x04AxgAnODu02smmiSTfVs14pmLDsEdznhUhUEkLJUVhbJuLd29GFji7uocX+KmW6tGPDvqYNwJ9hj0z02kplVWFPqaWUEwbQb6lM6bWUFNBZTk0rVlpDAAnD52Mgs1SI9Ijaps5LVUd28cTI3cPS1qvnFNhpTkEikMh2AWOZSk0dtEao7uTJaE1LVlQ5656BDMjDPGTuKr1SoMIjUhbkXBzMaZWb6ZzY5qu8vM5pvZTDN72cyaRi27zswWmdlXZnZsvHJJ7dG1ZUOeHXUIqSnGKWM+5dOv14UdSaTOi+eewuPA8F3a3gV6u3sfYAFwHYCZ9QROB3oF2zxsZqlxzCa1RJechrz028G0bpzJyHGf89K0vLAjidRpcSsK7v4RsH6XtnfcvSh4OgloF8yPAJ519+3uvgRYRGQsaBHaNWvAC78ZzICOzbny+Rk88N5C3DWCm0g8hHlO4Xy+71ivLRDdwX5e0PYjZjbKzKaY2ZS1a9fGOaIkiib103ni/IGc3L8t9723gKtfmMmOopKwY4nUOaEUBTO7HigCniptKme1cv8UdPex7j7A3Qfk5N/bTOcAAA3MSURBVOTEK6IkoIy0FO45tS+XDevGC1PzOP/xLygo3Fn1hiISsxovCmY2EjgBOMu/PwaQB7SPWq0dsLKms0niMzOuOHpf7j61L5MWf8spj3zKNxu/CzuWSJ1Ro0XBzIYD1wAnufu2qEUTgNPNrJ6ZdQK6obGgpRKnHNiOJ84fyKpNhfz8/33C7G82hR1JpE6I5yWpzwCfAd3NLM/MLgAeAhoB75rZdDMbA+Duc4DngbnAW8DooGsNkQod2jWbF38zmPTUFE77+2e8PWd12JFEaj2rzVdxDBgwwKdMmRJ2DAlZfkEhF46fwsy8TZx/aCeuPW4/MtJ0X6ZIRcxsqrsPKG+Z/s+RWq9l40z+ffEgzhucy7hPlnDqmE9ZsX5b1RuKyI+oKEidUC8tlZtO6sWYs/uzeN1Wjn/wf7w1e1XYsURqHRUFqVOG927DxEuH0Dk7i4v/NY0bX53N9iKdnhKJlYqC1Dntmzfg3xcP5sLDOvHEZ8v4xSOfsnTd1rBjidQKKgpSJ2WkpfCnE3ryj3MHsGL9d5zwt495faZufRGpioqC1GlH9WzFxMuGsG+rhlzy9Jdc/uyX5G8uDDuWSMJSUZA6r23T+jz360FcOqwbE2etZtjd/2Xcx0soKlbfSSK7UlGQpJCemsKVR+/L21cczgEdm3HL63M54W8f88XS9VVvLJJEVBQkqXTKzuKJXx3EmLMPZHNhEaeO+Yyrnp/B2s3bw44mkhBUFCTpmBnDe7fm3SsPZ/SRXZgw4xuG3vMhT3y6VIeUJOmpKEjSapCRxtXH7sdblx9Ov/ZNuXHCHE566BM+/Cpfg/hI0lJRkKTXJach488fyMNn9Wfjth2c99gXHPfA/3j5yzx2as9Bkow6xBOJsqOohAkzVjL2o69ZsGYL+zTJ5IIhnTn9oPZk1UsLO57IXlFZh3gqCiLlKClxPlyQz9//u5jJS9bTODONcwZ15LzBnchpVC/seCJ7REVBZA98uXwDYz9azFtzVpOemsIv+rfltAHt6de+KWbljSQrkthUFET2giXrtvLo/xbzwtQ8dhSV0KZJJsN7t+a43m04sGMzUlNUIKR2UFEQ2Ys2fbeT9+etYeKs1Xy0cC07ikrIaVSPY3u14vjebRjYqTlpqbqGQxKXioJInGzZXsR/5ufz1uxVfDB/Ld/tLKZ5VgZH92jFoC4t6N+hGe2b19dhJkkoKgoiNeC7HcX8d0E+E2et5j/z89myvQiA7IYZHNChGQd0aEr/Ds3o064JDTJ0JZOEp7KiELd/mWY2DjgByHf33kFbc+A5IBdYCpzm7hss8mfUA8DxwDbgPHefFq9sIvFQPyOV4b3bMLx3G4qKS1iwZgvTlm9g2vINfLl8I+/OXQNAaorRo00jDmjfjH1bNaRTdkM652TRunEmKTovISGL256CmR0ObAHGRxWFvwLr3f0OM7sWaObu15jZ8cDviBSFg4EH3P3gqt5DewpSm6zfuoPpKzYwbdlGpi3fwMy8TWV7EwD101PplJ1Fp5wsumRn0TmnIbnZWezTNJPsrHoqGLLXhLKn4O4fmVnuLs0jgJ8E808AHwLXBO3jPVKhJplZUzNr4+4aZFfqjOZZGQzdrxVD92sFgLuzpmA7i9duYfG6rSxeu5XF67Yw+5tNvDlrFSVRf6+lpRgtG9WjVZNMWjfOpFXjTFpHzbdomEGzBhk0a5Cuk9yyR2r6wGar0h96d19lZi2D9rbAiqj18oI2FQWps8ws8sPeJJPBXbN/sGx7UTHLv93GknVbWV1QyOpNhawuKGRNQSEL1mzmowVr2bqj/LGnm9RPp3lWpEBEHjNolpVBk/rpNM5Mo1FmOo3rp9E4M53G9dNplBmZb5CRqhPiUuNFoSLl/Uss97iWmY0CRgF06NAhnplEQlMvLZVurRrRrVWjCtfZXLiTNQWFrCnYzvqtO9iwbUfkcesO1m/byYatO1i5sZA5KwtYv3UH24sq78cpNcVokJFKVkYaDeoFjxmpZNVLK2uvn5FKZnoqmekp1Eur+DE91UhPSyEjNYWM4DE9LYX0VKNeaippqUZaqpGekqLDYgmmpovCmtLDQmbWBsgP2vOA9lHrtQPKHVDX3ccCYyFyTiGeYUUSWaPMdBplptO1ZcWFI9r2omI2FxZR8N1OCoLHzYVFFBTuLJvfuqOIbduLI487itm6vYi1m7f/oH17UQk7qigwu8MM0lNSSE0JCkVqMJ9iZY8ppY8WWSfVIstSg7aU4LlZpLilmmFmpKZQttysdJ7g+ffzKSkAkeel6xmRvbkfPg/aIqtjRJYblL2HBR+qbP2odUq3j/7skZeyqPkftkev9/12Rp92TTgot/le++9QqqaLwgRgJHBH8PhqVPslZvYskRPNm3Q+QWTvqpeWSr2GqWQ33PO+m0pKnB3FJRTuLKZwZwnbi374uLO4hB3FJewsCh6LI4VkR7GXtRWXOEXFTlFJCUUlTlFxCTuLPdJeEpkvKXGK3SkqicxHPxYHU4lH1t9RTNnzEneKSyLnbYpLHAdK3HGnbHlJsLwkaHMizz3qeUnJDx/dwfHgEYh6/v1r7PHXG5OLj+hSu4qCmT1D5KRytpnlATcSKQbPm9kFwHLg1GD1iUSuPFpE5JLUX8Url4jsuZQUIzMlcihJyldaYMqKDd8XjNJC8v26P273qNf54fPIY720+FxQEM+rj86oYNGwctZ1YHS8soiI1LTSQ0/BszCj7BZduyYiImVUFEREpIyKgoiIlFFREBGRMioKIiJSRkVBRETK1OrxFMxsLbCsmptnA+v2YpyaoMw1o7Zlrm15QZlrSkWZO7p7Tnkb1OqisCfMbEpFXccmKmWuGbUtc23LC8pcU6qTWYePRESkjIqCiIiUSeaiMDbsANWgzDWjtmWubXlBmWvKbmdO2nMKIiLyY8m8pyAiIrtQURARkTJJWRTMbLiZfWVmi8zs2rDzxMLMlprZLDObbmZTws5THjMbZ2b5ZjY7qq25mb1rZguDx2ZhZoxWQd6bzOyb4HuebmbHh5lxV2bW3sw+MLN5ZjbHzC4L2hP5e64oc0J+12aWaWafm9mMIO/NQXsnM5scfMfPmVlG2FlLVZL5cTNbEvUd96vytZLtnIKZpQILgKOJDAP6BXCGu88NNVgVzGwpMMDdE/bmGTM7HNgCjHf33kHbX4H17n5HUICbufs1YeYsVUHem4At7n53mNkqEgxj28bdp5lZI2Aq8DPgPBL3e64o82kk4HdtkfEys9x9i5mlAx8DlwFXAi+5+7NmNgaY4e6PhJm1VCWZLwZed/cXYn2tZNxTGAgscvfF7r4DeBYYEXKmOsHdPwLW79I8AngimH+CyI9BQqggb0Jz91XuPi2Y3wzMA9qS2N9zRZkTkkdsCZ6mB5MDQ4HSH9dE+44ryrzbkrEotAVWRD3PI4H/gUZx4B0zm2pmo8IOsxtalY63HTy2DDlPLC4xs5nB4aWEOQyzKzPLBQ4AJlNLvuddMkOCftdmlmpm04F84F3ga2CjuxcFqyTc78aumd299Du+LfiO7zOzKgfoTsaiUN64eLXhGNqh7t4fOA4YHRz6kL3vEaAL0A9YBdwTbpzymVlD4EXgcncvCDtPLMrJnLDftbsXu3s/oB2Rows9ylutZlNVbtfMZtYbuA7YDzgIaA5UeUgxGYtCHtA+6nk7YGVIWWLm7iuDx3zgZSL/UGuDNcEx5dJjy/kh56mUu68J/ucqAR4lAb/n4Jjxi8BT7v5S0JzQ33N5mWvDd+3uG4EPgUOApmZWOq59wv5uRGUeHhy6c3ffDjxGDN9xMhaFL4BuwZUEGcDpwISQM1XKzLKCE3SYWRZwDDC78q0SxgRgZDA/Eng1xCxVKv1hDfycBPuegxOK/wTmufu9UYsS9nuuKHOiftdmlmNmTYP5+sBRRM6DfACcEqyWaN9xeZnnR/2hYETOgVT5HSfd1UcAwaVv9wOpwDh3vy3kSJUys85E9g4A0oCnEzGzmT0D/IRId71rgBuBV4DngQ7AcuBUd0+Ik7sV5P0JkcMZDiwFfl16rD4RmNlhwP+AWUBJ0PxHIsfoE/V7rijzGSTgd21mfYicSE4l8ofz8+5+S/D/4bNEDsN8CZwd/AUeukoy/wfIIXLYfDpwcdQJ6fJfKxmLgoiIlC8ZDx+JiEgFVBRERKSMioKIiJRRURARkTIqCiIiUkZFQaQSZlYc1cPkdNuLveqaWa5F9dAqkgjSql5FJKl9F3QdIJIUtKcgUg0WGd/izqAP+8/NrGvQ3tHM3g86IHvfzDoE7a3M7OWgv/sZZjY4eKlUM3s06AP/neBuVJHQqCiIVK7+LoePfhm1rMDdBwIPEblDnmB+vLv3AZ4CHgzaHwT+6+59gf7AnKC9G/D/3L0XsBH4RZw/j0ildEezSCXMbIu7NyynfSkw1N0XB529rXb3Fma2jsiAMjuD9lXunm1ma4F20d0iBN1Iv+vu3YLn1wDp7n5r/D+ZSPm0pyBSfV7BfEXrlCe675xidJ5PQqaiIFJ9v4x6/CyY/5RIz7sAZxEZFhHgfeA3UDYYSuOaCimyO/RXiUjl6gejWZV6y91LL0utZ2aTifxxdUbQdikwzsyuBtYCvwraLwPGmtkFRPYIfkNkYBmRhKJzCiLVEJxTGODu68LOIrI36fCRiIiU0Z6CiIiU0Z6CiIiUUVEQEZEyKgoiIlJGRUFERMqoKIiISJn/D4fbcW7Tr2TTAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + ": Evaluate the new model against the test set:\n", + "3000/1 [================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 114us/sample - loss: 10760.6980 - root_mean_squared_error: 97.5697\n" + ] + }, + { + "data": { + "text/plain": [ + "[9519.838395182293, 97.56966]" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# The following variables are the hyperparameters.\n", + "learning_rate = 0.04\n", + "epochs = 35\n", + "\n", + "# Build the model, this time passing in the feature_cross_feature_layer: \n", + "my_model = create_model(learning_rate, feature_cross_feature_layer)\n", + "\n", + "# Train the model on the training set.\n", + "epochs, rmse = train_model(my_model, train_df, epochs, batch_size, label_name)\n", + "\n", + "plot_the_loss_curve(epochs, rmse)\n", + "\n", + "print(\"\\n: Evaluate the new model against the test set:\")\n", + "my_model.evaluate(x=test_features, y=test_label, batch_size=batch_size)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Regularization\n", + "Instead of simply aiming to minimize loss (empirical risk minimization):\n", + "\n", + " minimize(Loss(Data|Model))\n", + "\n", + "we'll now minimize loss+complexity, which is called structural risk minimization:\n", + "\n", + " minimize(Loss(Data|Model) + λ complexity(Model)) \n", + " \n", + "Our training optimization algorithm is now a function of two terms: the loss term, which measures how well the model fits the data, and the regularization term, which measures model complexity.\n", + "\n", + "Machine Learning Crash Course focuses on two common (and somewhat related) ways to think of model complexity:\n", + "\n", + "1. Model complexity as a function of the weights of all the features in the model.\n", + "2. Model complexity as a function of the total number of features with nonzero weights. (A later module covers this approach.)\n", + "\n", + "We can quantify complexity using the L2 regularization formula, which defines the regularization term as the sum of the squares of all the feature weights:\n", + "\n", + "$L_2 regularization term = ||w||_2^2 = \\sum_{i=1}^n{w}_i^2$\n", + "\n", + "Performing L2 regularization has the following effect on a model\n", + "\n", + "1. Encourages weight values toward 0 (but not exactly 0)\n", + "2. Encourages the mean of the weights toward 0, with a normal (bell-shaped or Gaussian) distribution.\n", + "\n", + "Increasing the lambda value strengthens the regularization effect. For example, the histogram of weights for a high value of lambda might look as shown in Figure 2.\n", + "\n", + "![](https://developers.google.com/machine-learning/crash-course/images/HighLambda.svg)\n", + "\n", + "When choosing a lambda value, the goal is to strike the right balance between simplicity and training-data fit:\n", + "- If your lambda value is too high, your model will be simple, but you run the risk of underfitting your data. Your model won't learn enough about the training data to make useful predictions.\n", + "- If your lambda value is too low, your model will be more complex, and you run the risk of overfitting your data. Your model will learn too much about the particularities of the training data, and won't be able to generalize to new data." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Logistic Regression\n", + "\n", + "In many cases, you'll map the logistic regression output into the solution to a binary classification problem, in which the goal is to correctly predict one of two possible labels (e.g., \"spam\" or \"not spam\"). A later module focuses on that.\n", + "\n", + "You might be wondering how a logistic regression model can ensure output that always falls between 0 and 1. As it happens, a sigmoid function, defined as follows, produces output having those same characteristics:\n", + "\n", + "$y = \\frac{1}{1+e^{-z}}$\n", + "\n", + "$z = b + \\sum_{i=1}^N {w_i}{x_i}$\n", + "\n", + "The sigmoid function yields the following plot:\n", + "![](https://developers.google.com/machine-learning/crash-course/images/SigmoidFunction.png)\n", + "\n", + "### Loss and Regression\n", + "The loss function for linear regression is squared loss. The loss function for logistic regression is Log Loss, which is defined as follows:\n", + "\n", + "$Loss = \\sum_{(x,y)\\in{D}}{-ylog(y') - (1-y)log(1-y')}$\n", + "\n", + "Regularization is extremely important in logistic regression modeling. Without regularization, the asymptotic nature of logistic regression would keep driving loss towards 0 in high dimensions. Consequently, most logistic regression models use one of the following two strategies to dampen model complexity:\n", + "\n", + "- L2 regularization.\n", + "- Early stopping, that is, limiting the number of training steps or the learning rate." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.4" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +}