Tensorflow.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "import shutil\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import tensorflow as tf\n",
    "\n",
    "print(tf.__version__)\n",
    "tf.logging.set_verbosity(tf.logging.INFO)\n",
    "pd.options.display.max_rows = 10\n",
    "pd.options.display.float_format = '{:.1f}'.format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"https://storage.googleapis.com/ml_universities/california_housing_train.csv\", sep=\",\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['num_rooms'] = df['total_rooms'] / df['households']\n",
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this exercise, we'll be trying to predict median_house_value. It will be our label (sometimes also called a target). We'll use num_rooms as our input feature.\n",
    "\n",
    "To train our model, we'll use the LinearRegressor estimator. The Estimator takes care of a lot of the plumbing, and exposes a convenient way to interact with data, training, and evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_and_evaluate(output_dir, num_train_steps):\n",
    "  estimator = tf.estimator\n",
    "  \n",
    "  #Add rmse evaluation metric\n",
    "  def rmse(labels, predictions):\n",
    "    pred_values = tf.cast(predictions['predictions'],tf.float64)\n",
    "    return {'rmse': tf.metrics.root_mean_squared_error(labels, pred_values)}\n",
    "  estimator = tf.contrib.estimator.add_metrics(estimator,rmse)\n",
    "  \n",
    "  train_spec=tf.estimator.TrainSpec(\n",
    "                       input_fn = tf.estimator.inputs.pandas_input_fn \n",
    "                       max_steps = num_train_steps)\n",
    "  eval_spec=tf.estimator.EvalSpec(\n",
    "                       input_fn = tf.estimator.inputs.pandas_input_fn\n",
    "                       steps = None,\n",
    "                       start_delay_secs = 1, # start evaluating after N seconds\n",
    "                       throttle_secs = 10,  # evaluate every N seconds\n",
    "                       )\n",
    "  tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)\n",
    "  \n",
    "# Run training    \n",
    "shutil.rmtree(OUTDIR, ignore_errors = True) # start fresh each time\n",
    "train_and_evaluate(OUTDIR, num_train_steps = 100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Scale the output\n",
    "Let's scale the target values so that the default parameters are more appropriate. Note that the RMSE here is now in 100000s so if you get RMSE=0.9, it really means RMSE=90000."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SCALE = 100000\n",
    "OUTDIR = './housing_trained'\n",
    "def train_and_evaluate(output_dir, num_train_steps):\n",
    "  estimator = #TODO\n",
    "  \n",
    "  #Add rmse evaluation metric\n",
    "  def rmse(labels, predictions):\n",
    "    pred_values = tf.cast(predictions['predictions'],tf.float64)\n",
    "    return {'rmse': tf.metrics.root_mean_squared_error(labels*SCALE, pred_values*SCALE)}\n",
    "  estimator = tf.contrib.estimator.add_metrics(estimator,rmse)\n",
    "  \n",
    "  train_spec=tf.estimator.TrainSpec(\n",
    "                       input_fn = ,#TODO\n",
    "                       max_steps = num_train_steps)\n",
    "  eval_spec=tf.estimator.EvalSpec(\n",
    "                       input_fn = ,#TODO\n",
    "                       steps = None,\n",
    "                       start_delay_secs = 1, # start evaluating after N seconds\n",
    "                       throttle_secs = 10,  # evaluate every N seconds\n",
    "                       )\n",
    "  tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)\n",
    "\n",
    "# Run training    \n",
    "shutil.rmtree(OUTDIR, ignore_errors = True) # start fresh each time\n",
    "train_and_evaluate(OUTDIR, num_train_steps = 100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Change learning rate and batch size\n",
    "Can you come up with better parameters? Note the default learning_rate is smaller of 0.2 or 1/sqrt(num_features), and default batch_size is 128. You can also change num_train_steps to train longer if neccessary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SCALE = 100000\n",
    "OUTDIR = './housing_trained'\n",
    "def train_and_evaluate(output_dir, num_train_steps):\n",
    "  myopt = #TODO: use tf.train.FtrlOptimizer and set learning rate\n",
    "  estimator = tf.estimator.LinearRegressor(\n",
    "                       model_dir = output_dir, \n",
    "                       feature_columns = [tf.feature_column.numeric_column('num_rooms')],\n",
    "                       optimizer = myopt)\n",
    "  \n",
    "  #Add rmse evaluation metric\n",
    "  def rmse(labels, predictions):\n",
    "    pred_values = tf.cast(predictions['predictions'],tf.float64)\n",
    "    return {'rmse': tf.metrics.root_mean_squared_error(labels*SCALE, pred_values*SCALE)}\n",
    "  estimator = tf.contrib.estimator.add_metrics(estimator,rmse)\n",
    "  \n",
    "  train_spec=tf.estimator.TrainSpec(\n",
    "                       input_fn = ,#TODO: make sure to specify batch_size\n",
    "                       max_steps = num_train_steps)\n",
    "  eval_spec=tf.estimator.EvalSpec(\n",
    "                       input_fn = ,#TODO\n",
    "                       steps = None,\n",
    "                       start_delay_secs = 1, # start evaluating after N seconds\n",
    "                       throttle_secs = 10,  # evaluate every N seconds\n",
    "                       )\n",
    "  tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)\n",
    "\n",
    "# Run training    \n",
    "shutil.rmtree(OUTDIR, ignore_errors = True) # start fresh each time\n",
    "train_and_evaluate(OUTDIR, num_train_steps = 100) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Is there a standard method for tuning the model?\n",
    "This is a commonly asked question. The short answer is that the effects of different hyperparameters is data dependent. So there are no hard and fast rules; you'll need to run tests on your data.\n",
    "\n",
    "Here are a few rules of thumb that may help guide you:\n",
    "\n",
    "Training error should steadily decrease, steeply at first, and should eventually plateau as training converges.\n",
    "If the training has not converged, try running it for longer.\n",
    "If the training error decreases too slowly, increasing the learning rate may help it decrease faster.\n",
    "But sometimes the exact opposite may happen if the learning rate is too high.\n",
    "If the training error varies wildly, try decreasing the learning rate.\n",
    "Lower learning rate plus larger number of steps or larger batch size is often a good combination.\n",
    "Very small batch sizes can also cause instability. First try larger values like 100 or 1000, and decrease until you see degradation.\n",
    "Again, never go strictly by these rules of thumb, because the effects are data dependent. Always experiment and verify.\n",
    "\n",
    "3: Try adding more features\n",
    "See if you can do any better by adding more features.\n",
    "\n",
    "Don't take more than 5 minutes on this portion."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}