Chapter 17
Chapter 14: Going Deeper -- the Mechanics of TensorFlow (Part 2/3)
NotebookPython 329 cells
Python Machine Learning 3rd Edition by Sebastian Raschka & Vahid Mirjalili, Packt Publishing Ltd. 2019
Code Repository: https://github.com/rasbt/python-machine-learning-book-3rd-edition
Code License: MIT License
Chapter 14: Going Deeper -- the Mechanics of TensorFlow (Part 2/3)
Note that the optional watermark extension is a small IPython notebook plugin that I developed to make the code reproducible. You can just skip the following line(s).
In [1]python · cell 4
python
%load_ext watermark
%watermark -a "Sebastian Raschka & Vahid Mirjalili" -u -d -p numpy,scipy,matplotlib,tensorflowOutput
Sebastian Raschka & Vahid Mirjalili last updated: 2019-11-03 numpy 1.17.3 scipy 1.3.1 matplotlib 3.1.1 tensorflow 2.0.0
In [2]python · cell 5
python
import numpy as np
import tensorflow as tf
import pandas as pd
from IPython.display import ImageTensorFlow Estimators
Steps for using pre-made estimators
- Step 1: Define the input function for importing the data
- Step 2: Define the feature columns to bridge between the estimator and the data
- Step 3: Instantiate an estimator or convert a Keras model to an estimator
- Step 4: Use the estimator: train() evaluate() predict()
In [3]python · cell 7
python
tf.random.set_seed(1)
np.random.seed(1)Working with feature columns
In [4]python · cell 9
python
Image(filename='images/02.png', width=700)Output
<IPython.core.display.Image object>
[省略较大 image/png 输出]
In [5]python · cell 10
python
dataset_path = tf.keras.utils.get_file("auto-mpg.data",
("http://archive.ics.uci.edu/ml/machine-learning-databases"
"/auto-mpg/auto-mpg.data"))
column_names = ['MPG', 'Cylinders', 'Displacement', 'Horsepower',
'Weight', 'Acceleration', 'ModelYear', 'Origin']
df = pd.read_csv(dataset_path, names=column_names,
na_values = "?", comment='\t',
sep=" ", skipinitialspace=True)
df.tail()Output
MPG Cylinders Displacement Horsepower Weight Acceleration \
393 27.0 4 140.0 86.0 2790.0 15.6
394 44.0 4 97.0 52.0 2130.0 24.6
395 32.0 4 135.0 84.0 2295.0 11.6
396 28.0 4 120.0 79.0 2625.0 18.6
397 31.0 4 119.0 82.0 2720.0 19.4
ModelYear Origin
393 82 1
394 82 2
395 82 1
396 82 1
397 82 1
.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}
.dataframe tbody tr th {
vertical-align: top;
}
.dataframe thead th {
text-align: right;
}
| MPG | Cylinders | Displacement | Horsepower | Weight | Acceleration | ModelYear | Origin | |
|---|---|---|---|---|---|---|---|---|
| 393 | 27.0 | 4 | 140.0 | 86.0 | 2790.0 | 15.6 | 82 | 1 |
| 394 | 44.0 | 4 | 97.0 | 52.0 | 2130.0 | 24.6 | 82 | 2 |
| 395 | 32.0 | 4 | 135.0 | 84.0 | 2295.0 | 11.6 | 82 | 1 |
| 396 | 28.0 | 4 | 120.0 | 79.0 | 2625.0 | 18.6 | 82 | 1 |
| 397 | 31.0 | 4 | 119.0 | 82.0 | 2720.0 | 19.4 | 82 | 1 |
In [6]python · cell 11
python
print(df.isna().sum())
df = df.dropna()
df = df.reset_index(drop=True)
df.tail()Output
MPG 0 Cylinders 0 Displacement 0 Horsepower 6 Weight 0 Acceleration 0 ModelYear 0 Origin 0 dtype: int64
MPG Cylinders Displacement Horsepower Weight Acceleration \
387 27.0 4 140.0 86.0 2790.0 15.6
388 44.0 4 97.0 52.0 2130.0 24.6
389 32.0 4 135.0 84.0 2295.0 11.6
390 28.0 4 120.0 79.0 2625.0 18.6
391 31.0 4 119.0 82.0 2720.0 19.4
ModelYear Origin
387 82 1
388 82 2
389 82 1
390 82 1
391 82 1
.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}
.dataframe tbody tr th {
vertical-align: top;
}
.dataframe thead th {
text-align: right;
}
| MPG | Cylinders | Displacement | Horsepower | Weight | Acceleration | ModelYear | Origin | |
|---|---|---|---|---|---|---|---|---|
| 387 | 27.0 | 4 | 140.0 | 86.0 | 2790.0 | 15.6 | 82 | 1 |
| 388 | 44.0 | 4 | 97.0 | 52.0 | 2130.0 | 24.6 | 82 | 2 |
| 389 | 32.0 | 4 | 135.0 | 84.0 | 2295.0 | 11.6 | 82 | 1 |
| 390 | 28.0 | 4 | 120.0 | 79.0 | 2625.0 | 18.6 | 82 | 1 |
| 391 | 31.0 | 4 | 119.0 | 82.0 | 2720.0 | 19.4 | 82 | 1 |
In [7]python · cell 12
python
import sklearn
import sklearn.model_selection
df_train, df_test = sklearn.model_selection.train_test_split(df, train_size=0.8)
train_stats = df_train.describe().transpose()
train_statsOutput
count mean std min 25% 50% 75% \
MPG 313.0 23.404153 7.666909 9.0 17.5 23.0 29.0
Cylinders 313.0 5.402556 1.701506 3.0 4.0 4.0 8.0
Displacement 313.0 189.512780 102.675646 68.0 104.0 140.0 260.0
Horsepower 313.0 102.929712 37.919046 46.0 75.0 92.0 120.0
Weight 313.0 2961.198083 848.602146 1613.0 2219.0 2755.0 3574.0
Acceleration 313.0 15.704473 2.725399 8.5 14.0 15.5 17.3
ModelYear 313.0 75.929712 3.675305 70.0 73.0 76.0 79.0
Origin 313.0 1.591054 0.807923 1.0 1.0 1.0 2.0
max
MPG 46.6
Cylinders 8.0
Displacement 455.0
Horsepower 230.0
Weight 5140.0
Acceleration 24.8
ModelYear 82.0
Origin 3.0
.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}
.dataframe tbody tr th {
vertical-align: top;
}
.dataframe thead th {
text-align: right;
}
| count | mean | std | min | 25% | 50% | 75% | max | |
|---|---|---|---|---|---|---|---|---|
| MPG | 313.0 | 23.404153 | 7.666909 | 9.0 | 17.5 | 23.0 | 29.0 | 46.6 |
| Cylinders | 313.0 | 5.402556 | 1.701506 | 3.0 | 4.0 | 4.0 | 8.0 | 8.0 |
| Displacement | 313.0 | 189.512780 | 102.675646 | 68.0 | 104.0 | 140.0 | 260.0 | 455.0 |
| Horsepower | 313.0 | 102.929712 | 37.919046 | 46.0 | 75.0 | 92.0 | 120.0 | 230.0 |
| Weight | 313.0 | 2961.198083 | 848.602146 | 1613.0 | 2219.0 | 2755.0 | 3574.0 | 5140.0 |
| Acceleration | 313.0 | 15.704473 | 2.725399 | 8.5 | 14.0 | 15.5 | 17.3 | 24.8 |
| ModelYear | 313.0 | 75.929712 | 3.675305 | 70.0 | 73.0 | 76.0 | 79.0 | 82.0 |
| Origin | 313.0 | 1.591054 | 0.807923 | 1.0 | 1.0 | 1.0 | 2.0 | 3.0 |
In [8]python · cell 13
python
numeric_column_names = ['Cylinders', 'Displacement', 'Horsepower', 'Weight', 'Acceleration']
df_train_norm, df_test_norm = df_train.copy(), df_test.copy()
for col_name in numeric_column_names:
mean = train_stats.loc[col_name, 'mean']
std = train_stats.loc[col_name, 'std']
df_train_norm.loc[:, col_name] = (df_train_norm.loc[:, col_name] - mean)/std
df_test_norm.loc[:, col_name] = (df_test_norm.loc[:, col_name] - mean)/std
df_train_norm.tail()Output
MPG Cylinders Displacement Horsepower Weight Acceleration \
203 28.0 -0.824303 -0.901020 -0.736562 -0.950031 0.255202
255 19.4 0.351127 0.413800 -0.340982 0.293190 0.548737
72 13.0 1.526556 1.144256 0.713897 1.339617 -0.625403
235 30.5 -0.824303 -0.891280 -1.053025 -1.072585 0.475353
37 14.0 1.526556 1.563051 1.636916 1.470420 -1.359240
ModelYear Origin
203 76 3
255 78 1
72 72 1
235 77 1
37 71 1
.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}
.dataframe tbody tr th {
vertical-align: top;
}
.dataframe thead th {
text-align: right;
}
| MPG | Cylinders | Displacement | Horsepower | Weight | Acceleration | ModelYear | Origin | |
|---|---|---|---|---|---|---|---|---|
| 203 | 28.0 | -0.824303 | -0.901020 | -0.736562 | -0.950031 | 0.255202 | 76 | 3 |
| 255 | 19.4 | 0.351127 | 0.413800 | -0.340982 | 0.293190 | 0.548737 | 78 | 1 |
| 72 | 13.0 | 1.526556 | 1.144256 | 0.713897 | 1.339617 | -0.625403 | 72 | 1 |
| 235 | 30.5 | -0.824303 | -0.891280 | -1.053025 | -1.072585 | 0.475353 | 77 | 1 |
| 37 | 14.0 | 1.526556 | 1.563051 | 1.636916 | 1.470420 | -1.359240 | 71 | 1 |
Numeric Columns
In [9]python · cell 15
python
numeric_features = []
for col_name in numeric_column_names:
numeric_features.append(tf.feature_column.numeric_column(key=col_name))
numeric_featuresOutput
[NumericColumn(key='Cylinders', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), NumericColumn(key='Displacement', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), NumericColumn(key='Horsepower', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), NumericColumn(key='Weight', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), NumericColumn(key='Acceleration', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None)]
In [10]python · cell 16
python
feature_year = tf.feature_column.numeric_column(key="ModelYear")
bucketized_features = []
bucketized_features.append(tf.feature_column.bucketized_column(
source_column=feature_year,
boundaries=[73, 76, 79]))
print(bucketized_features)Output
[BucketizedColumn(source_column=NumericColumn(key='ModelYear', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), boundaries=(73, 76, 79))]
In [11]python · cell 17
python
feature_origin = tf.feature_column.categorical_column_with_vocabulary_list(
key='Origin',
vocabulary_list=[1, 2, 3])
categorical_indicator_features = []
categorical_indicator_features.append(tf.feature_column.indicator_column(feature_origin))
print(categorical_indicator_features)Output
[IndicatorColumn(categorical_column=VocabularyListCategoricalColumn(key='Origin', vocabulary_list=(1, 2, 3), dtype=tf.int64, default_value=-1, num_oov_buckets=0))]
Machine learning with pre-made Estimators
In [12]python · cell 19
python
def train_input_fn(df_train, batch_size=8):
df = df_train.copy()
train_x, train_y = df, df.pop('MPG')
dataset = tf.data.Dataset.from_tensor_slices((dict(train_x), train_y))
# shuffle, repeat, and batch the examples
return dataset.shuffle(1000).repeat().batch(batch_size)
## inspection
ds = train_input_fn(df_train_norm)
batch = next(iter(ds))
print('Keys:', batch[0].keys())
print('Batch Model Years:', batch[0]['ModelYear'])Output
Keys: dict_keys(['Cylinders', 'Displacement', 'Horsepower', 'Weight', 'Acceleration', 'ModelYear', 'Origin']) Batch Model Years: tf.Tensor([82 78 76 72 78 73 70 78], shape=(8,), dtype=int32)
In [13]python · cell 20
python
all_feature_columns = (numeric_features +
bucketized_features +
categorical_indicator_features)
print(all_feature_columns)Output
[NumericColumn(key='Cylinders', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), NumericColumn(key='Displacement', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), NumericColumn(key='Horsepower', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), NumericColumn(key='Weight', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), NumericColumn(key='Acceleration', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), BucketizedColumn(source_column=NumericColumn(key='ModelYear', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), boundaries=(73, 76, 79)), IndicatorColumn(categorical_column=VocabularyListCategoricalColumn(key='Origin', vocabulary_list=(1, 2, 3), dtype=tf.int64, default_value=-1, num_oov_buckets=0))]
In [14]python · cell 21
python
regressor = tf.estimator.DNNRegressor(
feature_columns=all_feature_columns,
hidden_units=[32, 10],
model_dir='models/autompg-dnnregressor/')Output
INFO:tensorflow:Using default config.
INFO:tensorflow:Using config: {'_model_dir': 'models/autompg-dnnregressor/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
rewrite_options {
meta_optimizer_iterations: ONE
}
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f47c054d650>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
In [15]python · cell 22
python
EPOCHS = 1000
BATCH_SIZE = 8
total_steps = EPOCHS * int(np.ceil(len(df_train) / BATCH_SIZE))
print('Training Steps:', total_steps)
regressor.train(
input_fn=lambda:train_input_fn(df_train_norm, batch_size=BATCH_SIZE),
steps=total_steps)Output
Training Steps: 40000
WARNING:tensorflow:From /home/vahid/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From /home/vahid/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow_core/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
INFO:tensorflow:Calling model_fn.
WARNING:tensorflow:Layer dnn is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2. The layer has dtype float32 because it's dtype defaults to floatx.
If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.
To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.
WARNING:tensorflow:From /home/vahid/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow_core/python/feature_column/feature_column_v2.py:4276: IndicatorColumn._variable_shape (from tensorflow.python.feature_column.feature_column_v2) is deprecated and will be removed in a future version.
Instructions for updating:
The old _FeatureColumn APIs are being deprecated. Please use the new FeatureColumn APIs instead.
WARNING:tensorflow:From /home/vahid/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow_core/python/feature_column/feature_column_v2.py:4331: VocabularyListCategoricalColumn._num_buckets (from tensorflow.python.feature_column.feature_column_v2) is deprecated and will be removed in a future version.
Instructions for updating:
The old _FeatureColumn APIs are being deprecated. Please use the new FeatureColumn APIs instead.
WARNING:tensorflow:From /home/vahid/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/head/regression_head.py:156: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
WARNING:tensorflow:From /home/vahid/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow_core/python/keras/optimizer_v2/adagrad.py:108: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 0 into models/autompg-dnnregressor/model.ckpt.
INFO:tensorflow:loss = 610.1284, step = 0
INFO:tensorflow:global_step/sec: 448.416
INFO:tensorflow:loss = 712.06726, step = 100 (0.225 sec)
INFO:tensorflow:global_step/sec: 585.712
INFO:tensorflow:loss = 822.8959, step = 200 (0.170 sec)
INFO:tensorflow:global_step/sec: 648.546
INFO:tensorflow:loss = 710.2063, step = 300 (0.154 sec)
INFO:tensorflow:global_step/sec: 648.579
INFO:tensorflow:loss = 583.29407, step = 400 (0.154 sec)
INFO:tensorflow:global_step/sec: 643.959
INFO:tensorflow:loss = 582.77167, step = 500 (0.155 sec)
INFO:tensorflow:global_step/sec: 645.776
INFO:tensorflow:loss = 649.7673, step = 600 (0.155 sec)
INFO:tensorflow:global_step/sec: 572.456
INFO:tensorflow:loss = 396.41064, step = 700 (0.175 sec)
INFO:tensorflow:global_step/sec: 566.38
INFO:tensorflow:loss = 669.36633, step = 800 (0.177 sec)
INFO:tensorflow:global_step/sec: 566.658
INFO:tensorflow:loss = 632.7246, step = 900 (0.177 sec)
INFO:tensorflow:global_step/sec: 617.634
INFO:tensorflow:loss = 758.34875, step = 1000 (0.162 sec)
INFO:tensorflow:global_step/sec: 620.319
INFO:tensorflow:loss = 392.08755, step = 1100 (0.161 sec)
INFO:tensorflow:global_step/sec: 564.039
INFO:tensorflow:loss = 476.3385, step = 1200 (0.177 sec)
INFO:tensorflow:global_step/sec: 572.083
INFO:tensorflow:loss = 506.0124, step = 1300 (0.175 sec)
INFO:tensorflow:global_step/sec: 567.277
INFO:tensorflow:loss = 577.36475, step = 1400 (0.176 sec)
INFO:tensorflow:global_step/sec: 565.847
INFO:tensorflow:loss = 393.88156, step = 1500 (0.177 sec)
INFO:tensorflow:global_step/sec: 566.604
INFO:tensorflow:loss = 635.9873, step = 1600 (0.177 sec)
INFO:tensorflow:global_step/sec: 586.154
INFO:tensorflow:loss = 423.8427, step = 1700 (0.171 sec)
INFO:tensorflow:global_step/sec: 647.064
INFO:tensorflow:loss = 498.0647, step = 1800 (0.155 sec)
INFO:tensorflow:global_step/sec: 648.755
INFO:tensorflow:loss = 283.3813, step = 1900 (0.154 sec)
INFO:tensorflow:global_step/sec: 666.887
INFO:tensorflow:loss = 405.86353, step = 2000 (0.150 sec)
INFO:tensorflow:global_step/sec: 652.271
INFO:tensorflow:loss = 501.8705, step = 2100 (0.153 sec)
INFO:tensorflow:global_step/sec: 644.585
INFO:tensorflow:loss = 304.44452, step = 2200 (0.155 sec)
INFO:tensorflow:global_step/sec: 649.773
INFO:tensorflow:loss = 422.43158, step = 2300 (0.154 sec)
INFO:tensorflow:global_step/sec: 653.582
INFO:tensorflow:loss = 365.81732, step = 2400 (0.153 sec)
INFO:tensorflow:global_step/sec: 655.929
INFO:tensorflow:loss = 402.35364, step = 2500 (0.152 sec)
INFO:tensorflow:global_step/sec: 657.293
INFO:tensorflow:loss = 479.02838, step = 2600 (0.152 sec)
INFO:tensorflow:global_step/sec: 612.069
INFO:tensorflow:loss = 361.15582, step = 2700 (0.163 sec)
INFO:tensorflow:global_step/sec: 645.744
INFO:tensorflow:loss = 379.43427, step = 2800 (0.155 sec)
INFO:tensorflow:global_step/sec: 568.614
INFO:tensorflow:loss = 320.28763, step = 2900 (0.176 sec)
INFO:tensorflow:global_step/sec: 638.986
INFO:tensorflow:loss = 385.37427, step = 3000 (0.156 sec)
INFO:tensorflow:global_step/sec: 656.466
INFO:tensorflow:loss = 512.6961, step = 3100 (0.152 sec)
INFO:tensorflow:global_step/sec: 568.927
INFO:tensorflow:loss = 316.60834, step = 3200 (0.176 sec)
INFO:tensorflow:global_step/sec: 564.392
INFO:tensorflow:loss = 273.2796, step = 3300 (0.177 sec)
INFO:tensorflow:global_step/sec: 646.287
INFO:tensorflow:loss = 502.52917, step = 3400 (0.155 sec)
INFO:tensorflow:global_step/sec: 651.162
INFO:tensorflow:loss = 349.2741, step = 3500 (0.154 sec)
INFO:tensorflow:global_step/sec: 649.318
INFO:tensorflow:loss = 139.09305, step = 3600 (0.154 sec)
INFO:tensorflow:global_step/sec: 653.045
INFO:tensorflow:loss = 198.39453, step = 3700 (0.153 sec)
INFO:tensorflow:global_step/sec: 648.472
INFO:tensorflow:loss = 331.9128, step = 3800 (0.154 sec)
INFO:tensorflow:global_step/sec: 643.694
INFO:tensorflow:loss = 345.54785, step = 3900 (0.155 sec)
INFO:tensorflow:global_step/sec: 639.938
INFO:tensorflow:loss = 434.36136, step = 4000 (0.156 sec)
INFO:tensorflow:global_step/sec: 654.364
INFO:tensorflow:loss = 271.73187, step = 4100 (0.153 sec)
INFO:tensorflow:global_step/sec: 659.689
INFO:tensorflow:loss = 509.27518, step = 4200 (0.152 sec)
INFO:tensorflow:global_step/sec: 657.273
INFO:tensorflow:loss = 317.50986, step = 4300 (0.152 sec)
INFO:tensorflow:global_step/sec: 553.198
INFO:tensorflow:loss = 186.29042, step = 4400 (0.181 sec)
INFO:tensorflow:global_step/sec: 557.877
INFO:tensorflow:loss = 368.4395, step = 4500 (0.179 sec)
INFO:tensorflow:global_step/sec: 556.365
INFO:tensorflow:loss = 193.78143, step = 4600 (0.180 sec)
INFO:tensorflow:global_step/sec: 570.239
INFO:tensorflow:loss = 441.1708, step = 4700 (0.175 sec)
INFO:tensorflow:global_step/sec: 567.788
INFO:tensorflow:loss = 240.40808, step = 4800 (0.176 sec)
INFO:tensorflow:global_step/sec: 587.168
INFO:tensorflow:loss = 319.34958, step = 4900 (0.170 sec)
INFO:tensorflow:global_step/sec: 613.08
INFO:tensorflow:loss = 270.3896, step = 5000 (0.163 sec)
INFO:tensorflow:global_step/sec: 615.585
INFO:tensorflow:loss = 154.97989, step = 5100 (0.162 sec)
INFO:tensorflow:global_step/sec: 618.903
INFO:tensorflow:loss = 253.60646, step = 5200 (0.162 sec)
INFO:tensorflow:global_step/sec: 613.373
INFO:tensorflow:loss = 250.14478, step = 5300 (0.163 sec)
INFO:tensorflow:global_step/sec: 618.895
INFO:tensorflow:loss = 314.1364, step = 5400 (0.162 sec)
INFO:tensorflow:global_step/sec: 615.247
INFO:tensorflow:loss = 287.53287, step = 5500 (0.163 sec)
INFO:tensorflow:global_step/sec: 579.559
INFO:tensorflow:loss = 292.2387, step = 5600 (0.173 sec)
INFO:tensorflow:global_step/sec: 564.426
INFO:tensorflow:loss = 224.04553, step = 5700 (0.177 sec)
INFO:tensorflow:global_step/sec: 568.456
INFO:tensorflow:loss = 353.1428, step = 5800 (0.176 sec)
INFO:tensorflow:global_step/sec: 563.157
INFO:tensorflow:loss = 335.3949, step = 5900 (0.177 sec)
INFO:tensorflow:global_step/sec: 569.556
INFO:tensorflow:loss = 187.25581, step = 6000 (0.176 sec)
INFO:tensorflow:global_step/sec: 560.625
INFO:tensorflow:loss = 207.89288, step = 6100 (0.178 sec)
INFO:tensorflow:global_step/sec: 564.964
INFO:tensorflow:loss = 352.81, step = 6200 (0.177 sec)
INFO:tensorflow:global_step/sec: 601.182
INFO:tensorflow:loss = 200.35771, step = 6300 (0.166 sec)
INFO:tensorflow:global_step/sec: 624.884
INFO:tensorflow:loss = 148.6445, step = 6400 (0.160 sec)
INFO:tensorflow:global_step/sec: 617.641
INFO:tensorflow:loss = 98.60468, step = 6500 (0.162 sec)
INFO:tensorflow:global_step/sec: 565.054
INFO:tensorflow:loss = 272.97253, step = 6600 (0.177 sec)
INFO:tensorflow:global_step/sec: 557.052
INFO:tensorflow:loss = 189.83186, step = 6700 (0.180 sec)
INFO:tensorflow:global_step/sec: 559.404
INFO:tensorflow:loss = 177.83542, step = 6800 (0.179 sec)
INFO:tensorflow:global_step/sec: 616.915
INFO:tensorflow:loss = 154.5396, step = 6900 (0.162 sec)
INFO:tensorflow:global_step/sec: 586.968
INFO:tensorflow:loss = 312.20593, step = 7000 (0.170 sec)
INFO:tensorflow:global_step/sec: 584.733
INFO:tensorflow:loss = 146.29216, step = 7100 (0.171 sec)
INFO:tensorflow:global_step/sec: 645.786
INFO:tensorflow:loss = 175.46495, step = 7200 (0.155 sec)
INFO:tensorflow:global_step/sec: 652.056
INFO:tensorflow:loss = 225.34903, step = 7300 (0.153 sec)
INFO:tensorflow:global_step/sec: 642.788
INFO:tensorflow:loss = 203.46408, step = 7400 (0.155 sec)
INFO:tensorflow:global_step/sec: 657.435
INFO:tensorflow:loss = 115.84112, step = 7500 (0.152 sec)
INFO:tensorflow:global_step/sec: 650.413
INFO:tensorflow:loss = 222.01236, step = 7600 (0.154 sec)
INFO:tensorflow:global_step/sec: 566.584
INFO:tensorflow:loss = 195.82758, step = 7700 (0.176 sec)
INFO:tensorflow:global_step/sec: 635.592
INFO:tensorflow:loss = 135.50404, step = 7800 (0.157 sec)
INFO:tensorflow:global_step/sec: 581.135
INFO:tensorflow:loss = 128.616, step = 7900 (0.172 sec)
INFO:tensorflow:global_step/sec: 565.747
INFO:tensorflow:loss = 142.26901, step = 8000 (0.177 sec)
INFO:tensorflow:global_step/sec: 564.354
INFO:tensorflow:loss = 186.27264, step = 8100 (0.177 sec)
INFO:tensorflow:global_step/sec: 579.482
INFO:tensorflow:loss = 285.70624, step = 8200 (0.172 sec)
INFO:tensorflow:global_step/sec: 565.625
INFO:tensorflow:loss = 119.08911, step = 8300 (0.177 sec)
INFO:tensorflow:global_step/sec: 556.787
INFO:tensorflow:loss = 122.49934, step = 8400 (0.180 sec)
INFO:tensorflow:global_step/sec: 615.835
INFO:tensorflow:loss = 179.74419, step = 8500 (0.162 sec)
INFO:tensorflow:global_step/sec: 618.261
INFO:tensorflow:loss = 101.167206, step = 8600 (0.162 sec)
INFO:tensorflow:global_step/sec: 616.308
INFO:tensorflow:loss = 233.63202, step = 8700 (0.162 sec)
INFO:tensorflow:global_step/sec: 624.669
INFO:tensorflow:loss = 69.64491, step = 8800 (0.160 sec)
INFO:tensorflow:global_step/sec: 618.563
INFO:tensorflow:loss = 127.40744, step = 8900 (0.162 sec)
INFO:tensorflow:global_step/sec: 620.277
INFO:tensorflow:loss = 145.69806, step = 9000 (0.161 sec)
INFO:tensorflow:global_step/sec: 579.132
INFO:tensorflow:loss = 172.43636, step = 9100 (0.173 sec)
INFO:tensorflow:global_step/sec: 575.305
INFO:tensorflow:loss = 207.42215, step = 9200 (0.174 sec)
INFO:tensorflow:global_step/sec: 613.089
INFO:tensorflow:loss = 132.4537, step = 9300 (0.163 sec)
INFO:tensorflow:global_step/sec: 619.394
INFO:tensorflow:loss = 70.08789, step = 9400 (0.161 sec)
INFO:tensorflow:global_step/sec: 613.773
INFO:tensorflow:loss = 62.70175, step = 9500 (0.163 sec)
INFO:tensorflow:global_step/sec: 612.9
INFO:tensorflow:loss = 132.35649, step = 9600 (0.163 sec)
INFO:tensorflow:global_step/sec: 615.735
INFO:tensorflow:loss = 109.410255, step = 9700 (0.163 sec)
INFO:tensorflow:global_step/sec: 612.842
INFO:tensorflow:loss = 166.54138, step = 9800 (0.163 sec)
INFO:tensorflow:global_step/sec: 589.04
INFO:tensorflow:loss = 97.47385, step = 9900 (0.170 sec)
INFO:tensorflow:global_step/sec: 559.295
INFO:tensorflow:loss = 103.19119, step = 10000 (0.179 sec)
INFO:tensorflow:global_step/sec: 560.623
INFO:tensorflow:loss = 85.13874, step = 10100 (0.178 sec)
INFO:tensorflow:global_step/sec: 569.201
INFO:tensorflow:loss = 106.877075, step = 10200 (0.176 sec)
INFO:tensorflow:global_step/sec: 581.087
INFO:tensorflow:loss = 176.70938, step = 10300 (0.172 sec)
INFO:tensorflow:global_step/sec: 590.159
INFO:tensorflow:loss = 87.25897, step = 10400 (0.169 sec)
INFO:tensorflow:global_step/sec: 640.17
INFO:tensorflow:loss = 92.061295, step = 10500 (0.156 sec)
INFO:tensorflow:global_step/sec: 635.601
INFO:tensorflow:loss = 122.7544, step = 10600 (0.157 sec)
INFO:tensorflow:global_step/sec: 642.358
INFO:tensorflow:loss = 58.609745, step = 10700 (0.156 sec)
INFO:tensorflow:global_step/sec: 593.496
INFO:tensorflow:loss = 130.04077, step = 10800 (0.168 sec)
INFO:tensorflow:global_step/sec: 591.825
INFO:tensorflow:loss = 139.86115, step = 10900 (0.169 sec)
INFO:tensorflow:global_step/sec: 576.898
INFO:tensorflow:loss = 128.72388, step = 11000 (0.173 sec)
INFO:tensorflow:global_step/sec: 567.129
INFO:tensorflow:loss = 82.31069, step = 11100 (0.176 sec)
INFO:tensorflow:global_step/sec: 555.931
INFO:tensorflow:loss = 77.19307, step = 11200 (0.180 sec)
INFO:tensorflow:global_step/sec: 569.253
INFO:tensorflow:loss = 84.63773, step = 11300 (0.176 sec)
INFO:tensorflow:global_step/sec: 572.477
INFO:tensorflow:loss = 91.148834, step = 11400 (0.175 sec)
INFO:tensorflow:global_step/sec: 571.016
INFO:tensorflow:loss = 41.38865, step = 11500 (0.175 sec)
INFO:tensorflow:global_step/sec: 573.122
INFO:tensorflow:loss = 99.62012, step = 11600 (0.174 sec)
INFO:tensorflow:global_step/sec: 650.055
INFO:tensorflow:loss = 55.67815, step = 11700 (0.154 sec)
INFO:tensorflow:global_step/sec: 654.057
INFO:tensorflow:loss = 59.75948, step = 11800 (0.153 sec)
INFO:tensorflow:global_step/sec: 654.278
INFO:tensorflow:loss = 60.011417, step = 11900 (0.153 sec)
INFO:tensorflow:global_step/sec: 646.568
INFO:tensorflow:loss = 115.91528, step = 12000 (0.155 sec)
INFO:tensorflow:global_step/sec: 568.138
INFO:tensorflow:loss = 73.505295, step = 12100 (0.176 sec)
INFO:tensorflow:global_step/sec: 557.474
INFO:tensorflow:loss = 88.06746, step = 12200 (0.179 sec)
INFO:tensorflow:global_step/sec: 560.304
INFO:tensorflow:loss = 85.41072, step = 12300 (0.179 sec)
INFO:tensorflow:global_step/sec: 587.284
INFO:tensorflow:loss = 56.19171, step = 12400 (0.170 sec)
INFO:tensorflow:global_step/sec: 637.918
INFO:tensorflow:loss = 64.24606, step = 12500 (0.157 sec)
INFO:tensorflow:global_step/sec: 601.848
INFO:tensorflow:loss = 52.26082, step = 12600 (0.166 sec)
INFO:tensorflow:global_step/sec: 664.047
INFO:tensorflow:loss = 52.31598, step = 12700 (0.151 sec)
INFO:tensorflow:global_step/sec: 647.043
INFO:tensorflow:loss = 99.53073, step = 12800 (0.155 sec)
INFO:tensorflow:global_step/sec: 649.28
INFO:tensorflow:loss = 85.73153, step = 12900 (0.154 sec)
INFO:tensorflow:global_step/sec: 660.834
INFO:tensorflow:loss = 59.555412, step = 13000 (0.151 sec)
INFO:tensorflow:global_step/sec: 653.213
INFO:tensorflow:loss = 69.10744, step = 13100 (0.153 sec)
INFO:tensorflow:global_step/sec: 661.769
INFO:tensorflow:loss = 66.7493, step = 13200 (0.151 sec)
INFO:tensorflow:global_step/sec: 653.19
INFO:tensorflow:loss = 50.434845, step = 13300 (0.153 sec)
INFO:tensorflow:global_step/sec: 662.245
INFO:tensorflow:loss = 119.52582, step = 13400 (0.151 sec)
INFO:tensorflow:global_step/sec: 651.11
INFO:tensorflow:loss = 39.486324, step = 13500 (0.154 sec)
INFO:tensorflow:global_step/sec: 572.621
INFO:tensorflow:loss = 36.93599, step = 13600 (0.175 sec)
INFO:tensorflow:global_step/sec: 556.868
INFO:tensorflow:loss = 35.120228, step = 13700 (0.180 sec)
INFO:tensorflow:global_step/sec: 566.421
INFO:tensorflow:loss = 28.110342, step = 13800 (0.177 sec)
INFO:tensorflow:global_step/sec: 570.946
INFO:tensorflow:loss = 50.28643, step = 13900 (0.175 sec)
INFO:tensorflow:global_step/sec: 648.298
INFO:tensorflow:loss = 34.63925, step = 14000 (0.154 sec)
INFO:tensorflow:global_step/sec: 661.018
INFO:tensorflow:loss = 75.51688, step = 14100 (0.151 sec)
INFO:tensorflow:global_step/sec: 652.163
INFO:tensorflow:loss = 58.08398, step = 14200 (0.153 sec)
INFO:tensorflow:global_step/sec: 650.281
INFO:tensorflow:loss = 43.38926, step = 14300 (0.154 sec)
INFO:tensorflow:global_step/sec: 659.09
INFO:tensorflow:loss = 31.798458, step = 14400 (0.152 sec)
INFO:tensorflow:global_step/sec: 642.84
INFO:tensorflow:loss = 50.548183, step = 14500 (0.155 sec)
INFO:tensorflow:global_step/sec: 632.044
INFO:tensorflow:loss = 52.485527, step = 14600 (0.158 sec)
INFO:tensorflow:global_step/sec: 615.95
INFO:tensorflow:loss = 46.583717, step = 14700 (0.162 sec)
INFO:tensorflow:global_step/sec: 584.977
INFO:tensorflow:loss = 57.482048, step = 14800 (0.171 sec)
INFO:tensorflow:global_step/sec: 557.703
INFO:tensorflow:loss = 28.527182, step = 14900 (0.179 sec)
INFO:tensorflow:global_step/sec: 559.545
INFO:tensorflow:loss = 24.597378, step = 15000 (0.179 sec)
INFO:tensorflow:global_step/sec: 611.333
INFO:tensorflow:loss = 67.47382, step = 15100 (0.164 sec)
INFO:tensorflow:global_step/sec: 650.873
INFO:tensorflow:loss = 9.842392, step = 15200 (0.154 sec)
INFO:tensorflow:global_step/sec: 645.772
INFO:tensorflow:loss = 38.127098, step = 15300 (0.155 sec)
INFO:tensorflow:global_step/sec: 650.294
INFO:tensorflow:loss = 32.482162, step = 15400 (0.154 sec)
INFO:tensorflow:global_step/sec: 647.809
INFO:tensorflow:loss = 21.738888, step = 15500 (0.154 sec)
INFO:tensorflow:global_step/sec: 645.159
INFO:tensorflow:loss = 47.9902, step = 15600 (0.155 sec)
INFO:tensorflow:global_step/sec: 640.936
INFO:tensorflow:loss = 71.79877, step = 15700 (0.156 sec)
INFO:tensorflow:global_step/sec: 645.737
INFO:tensorflow:loss = 53.093693, step = 15800 (0.155 sec)
INFO:tensorflow:global_step/sec: 566.742
INFO:tensorflow:loss = 35.7934, step = 15900 (0.176 sec)
INFO:tensorflow:global_step/sec: 626.839
INFO:tensorflow:loss = 48.071213, step = 16000 (0.159 sec)
INFO:tensorflow:global_step/sec: 654.178
INFO:tensorflow:loss = 23.8972, step = 16100 (0.153 sec)
INFO:tensorflow:global_step/sec: 562.791
INFO:tensorflow:loss = 47.391075, step = 16200 (0.178 sec)
INFO:tensorflow:global_step/sec: 653.972
INFO:tensorflow:loss = 33.310158, step = 16300 (0.153 sec)
INFO:tensorflow:global_step/sec: 656.621
INFO:tensorflow:loss = 36.17897, step = 16400 (0.152 sec)
INFO:tensorflow:global_step/sec: 639.331
INFO:tensorflow:loss = 29.826387, step = 16500 (0.156 sec)
INFO:tensorflow:global_step/sec: 644.783
INFO:tensorflow:loss = 39.875584, step = 16600 (0.155 sec)
INFO:tensorflow:global_step/sec: 641.618
INFO:tensorflow:loss = 62.336967, step = 16700 (0.156 sec)
INFO:tensorflow:global_step/sec: 633.634
INFO:tensorflow:loss = 25.50624, step = 16800 (0.158 sec)
INFO:tensorflow:global_step/sec: 613.129
INFO:tensorflow:loss = 26.36065, step = 16900 (0.163 sec)
INFO:tensorflow:global_step/sec: 617.887
INFO:tensorflow:loss = 22.199059, step = 17000 (0.162 sec)
INFO:tensorflow:global_step/sec: 636.057
INFO:tensorflow:loss = 35.419243, step = 17100 (0.157 sec)
INFO:tensorflow:global_step/sec: 567.684
INFO:tensorflow:loss = 16.926636, step = 17200 (0.176 sec)
INFO:tensorflow:global_step/sec: 552.403
INFO:tensorflow:loss = 41.73528, step = 17300 (0.181 sec)
INFO:tensorflow:global_step/sec: 554.903
INFO:tensorflow:loss = 60.46621, step = 17400 (0.180 sec)
INFO:tensorflow:global_step/sec: 606.698
INFO:tensorflow:loss = 9.7304735, step = 17500 (0.165 sec)
INFO:tensorflow:global_step/sec: 629.775
INFO:tensorflow:loss = 23.861652, step = 17600 (0.159 sec)
INFO:tensorflow:global_step/sec: 580.814
INFO:tensorflow:loss = 44.88912, step = 17700 (0.172 sec)
INFO:tensorflow:global_step/sec: 557.919
INFO:tensorflow:loss = 21.361008, step = 17800 (0.179 sec)
INFO:tensorflow:global_step/sec: 553.493
INFO:tensorflow:loss = 33.784424, step = 17900 (0.181 sec)
INFO:tensorflow:global_step/sec: 560.444
INFO:tensorflow:loss = 20.45843, step = 18000 (0.178 sec)
INFO:tensorflow:global_step/sec: 565.069
INFO:tensorflow:loss = 31.591064, step = 18100 (0.177 sec)
INFO:tensorflow:global_step/sec: 562.811
INFO:tensorflow:loss = 13.793056, step = 18200 (0.178 sec)
INFO:tensorflow:global_step/sec: 556.338
INFO:tensorflow:loss = 45.03756, step = 18300 (0.180 sec)
INFO:tensorflow:global_step/sec: 556.655
INFO:tensorflow:loss = 11.519857, step = 18400 (0.180 sec)
INFO:tensorflow:global_step/sec: 643.54
INFO:tensorflow:loss = 34.568317, step = 18500 (0.155 sec)
INFO:tensorflow:global_step/sec: 649.191
INFO:tensorflow:loss = 44.755013, step = 18600 (0.154 sec)
INFO:tensorflow:global_step/sec: 576.328
INFO:tensorflow:loss = 14.43598, step = 18700 (0.174 sec)
INFO:tensorflow:global_step/sec: 564.586
INFO:tensorflow:loss = 10.982545, step = 18800 (0.177 sec)
INFO:tensorflow:global_step/sec: 644.083
INFO:tensorflow:loss = 30.731411, step = 18900 (0.155 sec)
INFO:tensorflow:global_step/sec: 643.407
INFO:tensorflow:loss = 19.440567, step = 19000 (0.155 sec)
INFO:tensorflow:global_step/sec: 607.951
INFO:tensorflow:loss = 63.62726, step = 19100 (0.164 sec)
INFO:tensorflow:global_step/sec: 573.549
INFO:tensorflow:loss = 13.900019, step = 19200 (0.175 sec)
INFO:tensorflow:global_step/sec: 580.514
INFO:tensorflow:loss = 47.006104, step = 19300 (0.172 sec)
INFO:tensorflow:global_step/sec: 642.58
INFO:tensorflow:loss = 12.699298, step = 19400 (0.155 sec)
INFO:tensorflow:global_step/sec: 641.932
INFO:tensorflow:loss = 20.420689, step = 19500 (0.156 sec)
INFO:tensorflow:global_step/sec: 641.287
INFO:tensorflow:loss = 11.957592, step = 19600 (0.156 sec)
INFO:tensorflow:global_step/sec: 635.855
INFO:tensorflow:loss = 20.387909, step = 19700 (0.157 sec)
INFO:tensorflow:global_step/sec: 641.552
INFO:tensorflow:loss = 63.668373, step = 19800 (0.156 sec)
INFO:tensorflow:global_step/sec: 564.799
INFO:tensorflow:loss = 29.952385, step = 19900 (0.177 sec)
INFO:tensorflow:global_step/sec: 561.545
INFO:tensorflow:loss = 3.468977, step = 20000 (0.178 sec)
INFO:tensorflow:global_step/sec: 564.833
INFO:tensorflow:loss = 14.017977, step = 20100 (0.177 sec)
INFO:tensorflow:global_step/sec: 559.31
INFO:tensorflow:loss = 23.940512, step = 20200 (0.179 sec)
INFO:tensorflow:global_step/sec: 561.595
INFO:tensorflow:loss = 15.872161, step = 20300 (0.178 sec)
INFO:tensorflow:global_step/sec: 557.326
INFO:tensorflow:loss = 41.10446, step = 20400 (0.179 sec)
INFO:tensorflow:global_step/sec: 597.31
INFO:tensorflow:loss = 40.06671, step = 20500 (0.167 sec)
INFO:tensorflow:global_step/sec: 615.561
INFO:tensorflow:loss = 40.576504, step = 20600 (0.162 sec)
INFO:tensorflow:global_step/sec: 599.323
INFO:tensorflow:loss = 9.951186, step = 20700 (0.167 sec)
INFO:tensorflow:global_step/sec: 564.476
INFO:tensorflow:loss = 26.887535, step = 20800 (0.177 sec)
INFO:tensorflow:global_step/sec: 558.206
INFO:tensorflow:loss = 22.223, step = 20900 (0.179 sec)
INFO:tensorflow:global_step/sec: 558.813
INFO:tensorflow:loss = 26.191727, step = 21000 (0.179 sec)
INFO:tensorflow:global_step/sec: 556.229
INFO:tensorflow:loss = 18.055063, step = 21100 (0.180 sec)
INFO:tensorflow:global_step/sec: 554.705
INFO:tensorflow:loss = 17.920183, step = 21200 (0.180 sec)
INFO:tensorflow:global_step/sec: 562.668
INFO:tensorflow:loss = 26.513557, step = 21300 (0.178 sec)
INFO:tensorflow:global_step/sec: 640.564
INFO:tensorflow:loss = 21.903809, step = 21400 (0.156 sec)
INFO:tensorflow:global_step/sec: 647.2
INFO:tensorflow:loss = 30.5953, step = 21500 (0.154 sec)
INFO:tensorflow:global_step/sec: 638.928
INFO:tensorflow:loss = 27.62001, step = 21600 (0.157 sec)
INFO:tensorflow:global_step/sec: 646.709
INFO:tensorflow:loss = 20.175642, step = 21700 (0.155 sec)
INFO:tensorflow:global_step/sec: 648.56
INFO:tensorflow:loss = 4.968134, step = 21800 (0.154 sec)
INFO:tensorflow:global_step/sec: 639.462
INFO:tensorflow:loss = 51.621918, step = 21900 (0.156 sec)
INFO:tensorflow:global_step/sec: 640.555
INFO:tensorflow:loss = 32.733124, step = 22000 (0.156 sec)
INFO:tensorflow:global_step/sec: 633.889
INFO:tensorflow:loss = 11.014718, step = 22100 (0.158 sec)
INFO:tensorflow:global_step/sec: 630.786
INFO:tensorflow:loss = 14.58586, step = 22200 (0.159 sec)
INFO:tensorflow:global_step/sec: 563.225
INFO:tensorflow:loss = 8.891371, step = 22300 (0.177 sec)
INFO:tensorflow:global_step/sec: 560.618
INFO:tensorflow:loss = 38.58745, step = 22400 (0.178 sec)
INFO:tensorflow:global_step/sec: 556.76
INFO:tensorflow:loss = 22.485962, step = 22500 (0.180 sec)
INFO:tensorflow:global_step/sec: 563.099
INFO:tensorflow:loss = 15.741159, step = 22600 (0.178 sec)
INFO:tensorflow:global_step/sec: 557.673
INFO:tensorflow:loss = 15.188747, step = 22700 (0.179 sec)
INFO:tensorflow:global_step/sec: 589.91
INFO:tensorflow:loss = 22.645355, step = 22800 (0.169 sec)
INFO:tensorflow:global_step/sec: 556.956
INFO:tensorflow:loss = 12.435162, step = 22900 (0.180 sec)
INFO:tensorflow:global_step/sec: 558.367
INFO:tensorflow:loss = 28.399435, step = 23000 (0.179 sec)
INFO:tensorflow:global_step/sec: 554.499
INFO:tensorflow:loss = 29.847202, step = 23100 (0.180 sec)
INFO:tensorflow:global_step/sec: 557.706
INFO:tensorflow:loss = 24.096184, step = 23200 (0.179 sec)
INFO:tensorflow:global_step/sec: 633.393
INFO:tensorflow:loss = 29.605507, step = 23300 (0.158 sec)
INFO:tensorflow:global_step/sec: 638.07
INFO:tensorflow:loss = 6.675656, step = 23400 (0.157 sec)
INFO:tensorflow:global_step/sec: 645.546
INFO:tensorflow:loss = 8.849547, step = 23500 (0.155 sec)
INFO:tensorflow:global_step/sec: 641.03
INFO:tensorflow:loss = 22.545252, step = 23600 (0.156 sec)
INFO:tensorflow:global_step/sec: 648.982
INFO:tensorflow:loss = 32.837273, step = 23700 (0.154 sec)
INFO:tensorflow:global_step/sec: 652.554
INFO:tensorflow:loss = 23.164417, step = 23800 (0.153 sec)
INFO:tensorflow:global_step/sec: 645.183
INFO:tensorflow:loss = 20.617609, step = 23900 (0.155 sec)
INFO:tensorflow:global_step/sec: 650.653
INFO:tensorflow:loss = 14.767559, step = 24000 (0.154 sec)
INFO:tensorflow:global_step/sec: 635.294
INFO:tensorflow:loss = 38.283543, step = 24100 (0.157 sec)
INFO:tensorflow:global_step/sec: 644.853
INFO:tensorflow:loss = 44.3748, step = 24200 (0.155 sec)
INFO:tensorflow:global_step/sec: 637.927
INFO:tensorflow:loss = 40.130806, step = 24300 (0.157 sec)
INFO:tensorflow:global_step/sec: 636.607
INFO:tensorflow:loss = 13.896637, step = 24400 (0.157 sec)
INFO:tensorflow:global_step/sec: 640.791
INFO:tensorflow:loss = 15.867412, step = 24500 (0.156 sec)
INFO:tensorflow:global_step/sec: 643.12
INFO:tensorflow:loss = 27.252453, step = 24600 (0.155 sec)
INFO:tensorflow:global_step/sec: 637.837
INFO:tensorflow:loss = 10.193189, step = 24700 (0.157 sec)
INFO:tensorflow:global_step/sec: 558.504
INFO:tensorflow:loss = 7.39489, step = 24800 (0.179 sec)
INFO:tensorflow:global_step/sec: 552.473
INFO:tensorflow:loss = 43.868538, step = 24900 (0.181 sec)
INFO:tensorflow:global_step/sec: 557.42
INFO:tensorflow:loss = 39.280296, step = 25000 (0.179 sec)
INFO:tensorflow:global_step/sec: 557.644
INFO:tensorflow:loss = 32.257843, step = 25100 (0.179 sec)
INFO:tensorflow:global_step/sec: 552.62
INFO:tensorflow:loss = 15.869023, step = 25200 (0.181 sec)
INFO:tensorflow:global_step/sec: 596.984
INFO:tensorflow:loss = 14.115913, step = 25300 (0.167 sec)
INFO:tensorflow:global_step/sec: 562.627
INFO:tensorflow:loss = 10.347841, step = 25400 (0.178 sec)
INFO:tensorflow:global_step/sec: 555.116
INFO:tensorflow:loss = 28.828302, step = 25500 (0.180 sec)
INFO:tensorflow:global_step/sec: 562.795
INFO:tensorflow:loss = 21.246841, step = 25600 (0.178 sec)
INFO:tensorflow:global_step/sec: 626.387
INFO:tensorflow:loss = 9.311673, step = 25700 (0.160 sec)
INFO:tensorflow:global_step/sec: 645.753
INFO:tensorflow:loss = 24.5037, step = 25800 (0.155 sec)
INFO:tensorflow:global_step/sec: 646.638
INFO:tensorflow:loss = 16.885513, step = 25900 (0.155 sec)
INFO:tensorflow:global_step/sec: 647.599
INFO:tensorflow:loss = 17.163336, step = 26000 (0.154 sec)
INFO:tensorflow:global_step/sec: 637.261
INFO:tensorflow:loss = 19.075056, step = 26100 (0.157 sec)
INFO:tensorflow:global_step/sec: 630.96
INFO:tensorflow:loss = 15.663348, step = 26200 (0.159 sec)
INFO:tensorflow:global_step/sec: 648.268
INFO:tensorflow:loss = 18.120064, step = 26300 (0.154 sec)
INFO:tensorflow:global_step/sec: 647.007
INFO:tensorflow:loss = 41.735718, step = 26400 (0.155 sec)
INFO:tensorflow:global_step/sec: 646.607
INFO:tensorflow:loss = 17.466406, step = 26500 (0.155 sec)
INFO:tensorflow:global_step/sec: 645.538
INFO:tensorflow:loss = 27.561075, step = 26600 (0.155 sec)
INFO:tensorflow:global_step/sec: 641.795
INFO:tensorflow:loss = 9.69916, step = 26700 (0.156 sec)
INFO:tensorflow:global_step/sec: 631.119
INFO:tensorflow:loss = 83.92375, step = 26800 (0.158 sec)
INFO:tensorflow:global_step/sec: 639.714
INFO:tensorflow:loss = 16.178406, step = 26900 (0.156 sec)
INFO:tensorflow:global_step/sec: 637.172
INFO:tensorflow:loss = 14.685428, step = 27000 (0.157 sec)
INFO:tensorflow:global_step/sec: 637.822
INFO:tensorflow:loss = 16.487114, step = 27100 (0.157 sec)
INFO:tensorflow:global_step/sec: 626.523
INFO:tensorflow:loss = 6.233879, step = 27200 (0.160 sec)
INFO:tensorflow:global_step/sec: 562.631
INFO:tensorflow:loss = 25.59382, step = 27300 (0.178 sec)
INFO:tensorflow:global_step/sec: 588.092
INFO:tensorflow:loss = 7.877035, step = 27400 (0.170 sec)
INFO:tensorflow:global_step/sec: 625.747
INFO:tensorflow:loss = 26.259216, step = 27500 (0.160 sec)
INFO:tensorflow:global_step/sec: 607.397
INFO:tensorflow:loss = 18.831963, step = 27600 (0.165 sec)
INFO:tensorflow:global_step/sec: 614.569
INFO:tensorflow:loss = 10.519957, step = 27700 (0.163 sec)
INFO:tensorflow:global_step/sec: 610.519
INFO:tensorflow:loss = 29.63565, step = 27800 (0.164 sec)
INFO:tensorflow:global_step/sec: 617.845
INFO:tensorflow:loss = 15.680868, step = 27900 (0.162 sec)
INFO:tensorflow:global_step/sec: 610.635
INFO:tensorflow:loss = 31.339779, step = 28000 (0.164 sec)
INFO:tensorflow:global_step/sec: 610.08
INFO:tensorflow:loss = 16.819326, step = 28100 (0.164 sec)
INFO:tensorflow:global_step/sec: 616.965
INFO:tensorflow:loss = 1.2390225, step = 28200 (0.162 sec)
INFO:tensorflow:global_step/sec: 614.248
INFO:tensorflow:loss = 18.569372, step = 28300 (0.163 sec)
INFO:tensorflow:global_step/sec: 631.205
INFO:tensorflow:loss = 69.872086, step = 28400 (0.159 sec)
INFO:tensorflow:global_step/sec: 647.207
INFO:tensorflow:loss = 7.713742, step = 28500 (0.154 sec)
INFO:tensorflow:global_step/sec: 650.569
INFO:tensorflow:loss = 8.96863, step = 28600 (0.154 sec)
INFO:tensorflow:global_step/sec: 635.145
INFO:tensorflow:loss = 8.195501, step = 28700 (0.157 sec)
INFO:tensorflow:global_step/sec: 639.996
INFO:tensorflow:loss = 7.895544, step = 28800 (0.156 sec)
INFO:tensorflow:global_step/sec: 634.209
INFO:tensorflow:loss = 9.781753, step = 28900 (0.158 sec)
INFO:tensorflow:global_step/sec: 642.806
INFO:tensorflow:loss = 23.838917, step = 29000 (0.156 sec)
INFO:tensorflow:global_step/sec: 639.762
INFO:tensorflow:loss = 13.926859, step = 29100 (0.156 sec)
INFO:tensorflow:global_step/sec: 635.524
INFO:tensorflow:loss = 20.532545, step = 29200 (0.157 sec)
INFO:tensorflow:global_step/sec: 641.009
INFO:tensorflow:loss = 15.29386, step = 29300 (0.156 sec)
INFO:tensorflow:global_step/sec: 636.193
INFO:tensorflow:loss = 9.65447, step = 29400 (0.157 sec)
INFO:tensorflow:global_step/sec: 636.497
INFO:tensorflow:loss = 17.778759, step = 29500 (0.157 sec)
INFO:tensorflow:global_step/sec: 638.647
INFO:tensorflow:loss = 53.59935, step = 29600 (0.157 sec)
INFO:tensorflow:global_step/sec: 641.218
INFO:tensorflow:loss = 1.7887146, step = 29700 (0.156 sec)
INFO:tensorflow:global_step/sec: 632.812
INFO:tensorflow:loss = 9.185707, step = 29800 (0.158 sec)
INFO:tensorflow:global_step/sec: 636.626
INFO:tensorflow:loss = 2.0348744, step = 29900 (0.157 sec)
INFO:tensorflow:global_step/sec: 641.384
INFO:tensorflow:loss = 39.938457, step = 30000 (0.156 sec)
INFO:tensorflow:global_step/sec: 636.928
INFO:tensorflow:loss = 35.079205, step = 30100 (0.157 sec)
INFO:tensorflow:global_step/sec: 631.889
INFO:tensorflow:loss = 11.830448, step = 30200 (0.158 sec)
INFO:tensorflow:global_step/sec: 629.569
INFO:tensorflow:loss = 7.7222857, step = 30300 (0.159 sec)
INFO:tensorflow:global_step/sec: 629.26
INFO:tensorflow:loss = 5.0043783, step = 30400 (0.159 sec)
INFO:tensorflow:global_step/sec: 633.129
INFO:tensorflow:loss = 28.834631, step = 30500 (0.158 sec)
INFO:tensorflow:global_step/sec: 560.8
INFO:tensorflow:loss = 49.048584, step = 30600 (0.178 sec)
INFO:tensorflow:global_step/sec: 630.476
INFO:tensorflow:loss = 12.857588, step = 30700 (0.159 sec)
INFO:tensorflow:global_step/sec: 576.142
INFO:tensorflow:loss = 18.584694, step = 30800 (0.174 sec)
INFO:tensorflow:global_step/sec: 554.342
INFO:tensorflow:loss = 30.525942, step = 30900 (0.180 sec)
INFO:tensorflow:global_step/sec: 567.578
INFO:tensorflow:loss = 33.63379, step = 31000 (0.176 sec)
INFO:tensorflow:global_step/sec: 560.022
INFO:tensorflow:loss = 14.771801, step = 31100 (0.178 sec)
INFO:tensorflow:global_step/sec: 558.288
INFO:tensorflow:loss = 3.7000563, step = 31200 (0.179 sec)
INFO:tensorflow:global_step/sec: 625.088
INFO:tensorflow:loss = 3.9122221, step = 31300 (0.160 sec)
INFO:tensorflow:global_step/sec: 619.433
INFO:tensorflow:loss = 13.096632, step = 31400 (0.161 sec)
INFO:tensorflow:global_step/sec: 630.211
INFO:tensorflow:loss = 35.163918, step = 31500 (0.159 sec)
INFO:tensorflow:global_step/sec: 641.753
INFO:tensorflow:loss = 18.955883, step = 31600 (0.156 sec)
INFO:tensorflow:global_step/sec: 642.018
INFO:tensorflow:loss = 24.371445, step = 31700 (0.156 sec)
INFO:tensorflow:global_step/sec: 654.459
INFO:tensorflow:loss = 10.686918, step = 31800 (0.153 sec)
INFO:tensorflow:global_step/sec: 648.877
INFO:tensorflow:loss = 10.00495, step = 31900 (0.154 sec)
INFO:tensorflow:global_step/sec: 636.379
INFO:tensorflow:loss = 34.196156, step = 32000 (0.157 sec)
INFO:tensorflow:global_step/sec: 639.306
INFO:tensorflow:loss = 17.267265, step = 32100 (0.156 sec)
INFO:tensorflow:global_step/sec: 628.14
INFO:tensorflow:loss = 3.5982966, step = 32200 (0.159 sec)
INFO:tensorflow:global_step/sec: 654.144
INFO:tensorflow:loss = 22.826725, step = 32300 (0.153 sec)
INFO:tensorflow:global_step/sec: 656.323
INFO:tensorflow:loss = 18.253302, step = 32400 (0.152 sec)
INFO:tensorflow:global_step/sec: 652.179
INFO:tensorflow:loss = 19.89624, step = 32500 (0.153 sec)
INFO:tensorflow:global_step/sec: 648.347
INFO:tensorflow:loss = 26.507126, step = 32600 (0.154 sec)
INFO:tensorflow:global_step/sec: 639.534
INFO:tensorflow:loss = 9.670689, step = 32700 (0.156 sec)
INFO:tensorflow:global_step/sec: 628.019
INFO:tensorflow:loss = 19.151632, step = 32800 (0.159 sec)
INFO:tensorflow:global_step/sec: 645.71
INFO:tensorflow:loss = 11.301746, step = 32900 (0.155 sec)
INFO:tensorflow:global_step/sec: 632.64
INFO:tensorflow:loss = 39.349358, step = 33000 (0.158 sec)
INFO:tensorflow:global_step/sec: 644.028
INFO:tensorflow:loss = 32.2932, step = 33100 (0.155 sec)
INFO:tensorflow:global_step/sec: 627.79
INFO:tensorflow:loss = 14.889015, step = 33200 (0.159 sec)
INFO:tensorflow:global_step/sec: 646.24
INFO:tensorflow:loss = 41.704803, step = 33300 (0.155 sec)
INFO:tensorflow:global_step/sec: 635.648
INFO:tensorflow:loss = 15.027176, step = 33400 (0.157 sec)
INFO:tensorflow:global_step/sec: 646.35
INFO:tensorflow:loss = 12.243839, step = 33500 (0.155 sec)
INFO:tensorflow:global_step/sec: 651.522
INFO:tensorflow:loss = 5.87178, step = 33600 (0.153 sec)
INFO:tensorflow:global_step/sec: 650.649
INFO:tensorflow:loss = 16.51314, step = 33700 (0.154 sec)
INFO:tensorflow:global_step/sec: 648.37
INFO:tensorflow:loss = 15.912634, step = 33800 (0.154 sec)
INFO:tensorflow:global_step/sec: 645.818
INFO:tensorflow:loss = 6.8702297, step = 33900 (0.155 sec)
INFO:tensorflow:global_step/sec: 637.876
INFO:tensorflow:loss = 9.3833275, step = 34000 (0.157 sec)
INFO:tensorflow:global_step/sec: 635.339
INFO:tensorflow:loss = 6.315841, step = 34100 (0.157 sec)
INFO:tensorflow:global_step/sec: 648.389
INFO:tensorflow:loss = 23.294544, step = 34200 (0.154 sec)
INFO:tensorflow:global_step/sec: 626.389
INFO:tensorflow:loss = 14.866375, step = 34300 (0.160 sec)
INFO:tensorflow:global_step/sec: 636.443
INFO:tensorflow:loss = 23.482327, step = 34400 (0.157 sec)
INFO:tensorflow:global_step/sec: 646.709
INFO:tensorflow:loss = 5.9481287, step = 34500 (0.155 sec)
INFO:tensorflow:global_step/sec: 648.761
INFO:tensorflow:loss = 10.6761265, step = 34600 (0.154 sec)
INFO:tensorflow:global_step/sec: 638.862
INFO:tensorflow:loss = 4.912558, step = 34700 (0.157 sec)
INFO:tensorflow:global_step/sec: 642.065
INFO:tensorflow:loss = 25.11901, step = 34800 (0.156 sec)
INFO:tensorflow:global_step/sec: 612.919
INFO:tensorflow:loss = 8.14024, step = 34900 (0.163 sec)
INFO:tensorflow:global_step/sec: 556.115
INFO:tensorflow:loss = 17.55474, step = 35000 (0.180 sec)
INFO:tensorflow:global_step/sec: 553.651
INFO:tensorflow:loss = 5.6831026, step = 35100 (0.181 sec)
INFO:tensorflow:global_step/sec: 563.169
INFO:tensorflow:loss = 12.501162, step = 35200 (0.177 sec)
INFO:tensorflow:global_step/sec: 643.02
INFO:tensorflow:loss = 25.764713, step = 35300 (0.156 sec)
INFO:tensorflow:global_step/sec: 644.011
INFO:tensorflow:loss = 10.363308, step = 35400 (0.155 sec)
INFO:tensorflow:global_step/sec: 638.251
INFO:tensorflow:loss = 23.721325, step = 35500 (0.156 sec)
INFO:tensorflow:global_step/sec: 640.312
INFO:tensorflow:loss = 13.199163, step = 35600 (0.156 sec)
INFO:tensorflow:global_step/sec: 651.36
INFO:tensorflow:loss = 31.616188, step = 35700 (0.154 sec)
INFO:tensorflow:global_step/sec: 613.686
INFO:tensorflow:loss = 5.4822397, step = 35800 (0.163 sec)
INFO:tensorflow:global_step/sec: 618.484
INFO:tensorflow:loss = 9.069538, step = 35900 (0.162 sec)
INFO:tensorflow:global_step/sec: 617.06
INFO:tensorflow:loss = 8.100636, step = 36000 (0.162 sec)
INFO:tensorflow:global_step/sec: 615.627
INFO:tensorflow:loss = 16.85956, step = 36100 (0.162 sec)
INFO:tensorflow:global_step/sec: 612.812
INFO:tensorflow:loss = 18.85649, step = 36200 (0.163 sec)
INFO:tensorflow:global_step/sec: 616.103
INFO:tensorflow:loss = 13.850594, step = 36300 (0.162 sec)
INFO:tensorflow:global_step/sec: 625.165
INFO:tensorflow:loss = 51.77982, step = 36400 (0.160 sec)
INFO:tensorflow:global_step/sec: 641.928
INFO:tensorflow:loss = 21.499207, step = 36500 (0.156 sec)
INFO:tensorflow:global_step/sec: 641.116
INFO:tensorflow:loss = 9.992126, step = 36600 (0.156 sec)
INFO:tensorflow:global_step/sec: 646.039
INFO:tensorflow:loss = 11.250916, step = 36700 (0.155 sec)
INFO:tensorflow:global_step/sec: 647.816
INFO:tensorflow:loss = 9.30033, step = 36800 (0.154 sec)
INFO:tensorflow:global_step/sec: 648.269
INFO:tensorflow:loss = 4.329362, step = 36900 (0.154 sec)
INFO:tensorflow:global_step/sec: 640.78
INFO:tensorflow:loss = 10.897452, step = 37000 (0.156 sec)
INFO:tensorflow:global_step/sec: 645.487
INFO:tensorflow:loss = 9.148363, step = 37100 (0.155 sec)
INFO:tensorflow:global_step/sec: 651.223
INFO:tensorflow:loss = 30.404154, step = 37200 (0.154 sec)
INFO:tensorflow:global_step/sec: 641.637
INFO:tensorflow:loss = 14.910101, step = 37300 (0.156 sec)
INFO:tensorflow:global_step/sec: 650.827
INFO:tensorflow:loss = 15.732323, step = 37400 (0.154 sec)
INFO:tensorflow:global_step/sec: 652.423
INFO:tensorflow:loss = 12.93858, step = 37500 (0.153 sec)
INFO:tensorflow:global_step/sec: 644.518
INFO:tensorflow:loss = 31.476284, step = 37600 (0.155 sec)
INFO:tensorflow:global_step/sec: 644.811
INFO:tensorflow:loss = 12.837719, step = 37700 (0.155 sec)
INFO:tensorflow:global_step/sec: 635.591
INFO:tensorflow:loss = 6.9836364, step = 37800 (0.157 sec)
INFO:tensorflow:global_step/sec: 640.438
INFO:tensorflow:loss = 3.7009566, step = 37900 (0.156 sec)
INFO:tensorflow:global_step/sec: 652.57
INFO:tensorflow:loss = 15.457409, step = 38000 (0.153 sec)
INFO:tensorflow:global_step/sec: 641.224
INFO:tensorflow:loss = 15.640904, step = 38100 (0.156 sec)
INFO:tensorflow:global_step/sec: 652.626
INFO:tensorflow:loss = 3.6458635, step = 38200 (0.153 sec)
INFO:tensorflow:global_step/sec: 654.56
INFO:tensorflow:loss = 7.718712, step = 38300 (0.153 sec)
INFO:tensorflow:global_step/sec: 645.262
INFO:tensorflow:loss = 17.353992, step = 38400 (0.155 sec)
INFO:tensorflow:global_step/sec: 644.266
INFO:tensorflow:loss = 10.383542, step = 38500 (0.155 sec)
INFO:tensorflow:global_step/sec: 646.131
INFO:tensorflow:loss = 19.498116, step = 38600 (0.155 sec)
INFO:tensorflow:global_step/sec: 650.171
INFO:tensorflow:loss = 50.384224, step = 38700 (0.154 sec)
INFO:tensorflow:global_step/sec: 649.759
INFO:tensorflow:loss = 5.805867, step = 38800 (0.154 sec)
INFO:tensorflow:global_step/sec: 634.323
INFO:tensorflow:loss = 7.7724056, step = 38900 (0.158 sec)
INFO:tensorflow:global_step/sec: 633.254
INFO:tensorflow:loss = 7.2014084, step = 39000 (0.158 sec)
INFO:tensorflow:global_step/sec: 648.197
INFO:tensorflow:loss = 17.135563, step = 39100 (0.154 sec)
INFO:tensorflow:global_step/sec: 651.45
INFO:tensorflow:loss = 26.166622, step = 39200 (0.153 sec)
INFO:tensorflow:global_step/sec: 644.187
INFO:tensorflow:loss = 16.454807, step = 39300 (0.155 sec)
INFO:tensorflow:global_step/sec: 645.437
INFO:tensorflow:loss = 25.390144, step = 39400 (0.155 sec)
INFO:tensorflow:global_step/sec: 637.175
INFO:tensorflow:loss = 17.874966, step = 39500 (0.157 sec)
INFO:tensorflow:global_step/sec: 632.669
INFO:tensorflow:loss = 11.231724, step = 39600 (0.158 sec)
INFO:tensorflow:global_step/sec: 617.176
INFO:tensorflow:loss = 16.137482, step = 39700 (0.162 sec)
INFO:tensorflow:global_step/sec: 609.228
INFO:tensorflow:loss = 20.8172, step = 39800 (0.164 sec)
INFO:tensorflow:global_step/sec: 587.507
INFO:tensorflow:loss = 24.547935, step = 39900 (0.170 sec)
INFO:tensorflow:Saving checkpoints for 40000 into models/autompg-dnnregressor/model.ckpt.
INFO:tensorflow:Loss for final step: 43.661736.
<tensorflow_estimator.python.estimator.canned.dnn.DNNRegressorV2 at 0x7f47cde76a10>
In [16]python · cell 23
python
reloaded_regressor = tf.estimator.DNNRegressor(
feature_columns=all_feature_columns,
hidden_units=[32, 10],
warm_start_from='models/autompg-dnnregressor/',
model_dir='models/autompg-dnnregressor/')Output
INFO:tensorflow:Using default config.
INFO:tensorflow:Using config: {'_model_dir': 'models/autompg-dnnregressor/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
rewrite_options {
meta_optimizer_iterations: ONE
}
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f47c054ddd0>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
In [17]python · cell 24
python
def eval_input_fn(df_test, batch_size=8):
df = df_test.copy()
test_x, test_y = df, df.pop('MPG')
dataset = tf.data.Dataset.from_tensor_slices((dict(test_x), test_y))
return dataset.batch(batch_size)
eval_results = reloaded_regressor.evaluate(
input_fn=lambda:eval_input_fn(df_test_norm, batch_size=8))
for key in eval_results:
print('{:15s} {}'.format(key, eval_results[key]))
print('Average-Loss {:.4f}'.format(eval_results['average_loss']))Output
INFO:tensorflow:Calling model_fn.
WARNING:tensorflow:Layer dnn is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2. The layer has dtype float32 because it's dtype defaults to floatx.
If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.
To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2019-11-03T11:17:46Z
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from models/autompg-dnnregressor/model.ckpt-40000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Finished evaluation at 2019-11-03-11:17:47
INFO:tensorflow:Saving dict for global step 40000: average_loss = 19.54804, global_step = 40000, label/mean = 23.611393, loss = 19.48917, prediction/mean = 21.74643
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 40000: models/autompg-dnnregressor/model.ckpt-40000
average_loss 19.54804039001465
label/mean 23.611392974853516
loss 19.48917007446289
prediction/mean 21.746429443359375
global_step 40000
Average-Loss 19.5480
In [18]python · cell 25
python
pred_res = regressor.predict(input_fn=lambda: eval_input_fn(df_test_norm, batch_size=8))
print(next(iter(pred_res)))Output
INFO:tensorflow:Calling model_fn.
WARNING:tensorflow:Layer dnn is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2. The layer has dtype float32 because it's dtype defaults to floatx.
If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.
To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from models/autompg-dnnregressor/model.ckpt-40000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
{'predictions': array([23.719353], dtype=float32)}
Boosted Tree Regressor
In [19]python · cell 27
python
boosted_tree = tf.estimator.BoostedTreesRegressor(
feature_columns=all_feature_columns,
n_batches_per_layer=20,
n_trees=200)
boosted_tree.train(
input_fn=lambda:train_input_fn(df_train_norm, batch_size=BATCH_SIZE))
eval_results = boosted_tree.evaluate(
input_fn=lambda:eval_input_fn(df_test_norm, batch_size=8))
print(eval_results)
print('Average-Loss {:.4f}'.format(eval_results['average_loss']))Output
INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpbzo1p2wi
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpbzo1p2wi', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
rewrite_options {
meta_optimizer_iterations: ONE
}
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f47bc30b7d0>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Calling model_fn.
WARNING:tensorflow:From /home/vahid/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/canned/boosted_trees.py:214: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
WARNING:tensorflow:Issue encountered when serializing resources.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
'_Resource' object has no attribute 'name'
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
WARNING:tensorflow:Issue encountered when serializing resources.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
'_Resource' object has no attribute 'name'
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpbzo1p2wi/model.ckpt.
WARNING:tensorflow:Issue encountered when serializing resources.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
'_Resource' object has no attribute 'name'
INFO:tensorflow:loss = 402.19623, step = 0
WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 0 vs previous value: 0. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.
WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 0 vs previous value: 0. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.
WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 0 vs previous value: 0. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.
WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 0 vs previous value: 0. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.
WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 0 vs previous value: 0. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.
INFO:tensorflow:loss = 289.26328, step = 80 (0.462 sec)
INFO:tensorflow:global_step/sec: 157.704
INFO:tensorflow:loss = 93.58242, step = 180 (0.363 sec)
INFO:tensorflow:global_step/sec: 422.808
INFO:tensorflow:loss = 45.606873, step = 280 (0.243 sec)
INFO:tensorflow:global_step/sec: 416.715
INFO:tensorflow:loss = 19.545433, step = 380 (0.240 sec)
INFO:tensorflow:global_step/sec: 416.626
INFO:tensorflow:loss = 6.4179554, step = 480 (0.245 sec)
INFO:tensorflow:global_step/sec: 407.822
INFO:tensorflow:loss = 4.7701707, step = 580 (0.231 sec)
INFO:tensorflow:global_step/sec: 408.05
INFO:tensorflow:loss = 4.569898, step = 680 (0.244 sec)
INFO:tensorflow:global_step/sec: 420.57
INFO:tensorflow:loss = 2.5075686, step = 780 (0.249 sec)
INFO:tensorflow:global_step/sec: 410.68
INFO:tensorflow:loss = 2.6939745, step = 880 (0.244 sec)
INFO:tensorflow:global_step/sec: 411.964
INFO:tensorflow:loss = 1.5966964, step = 980 (0.248 sec)
INFO:tensorflow:global_step/sec: 403.965
INFO:tensorflow:loss = 3.3678646, step = 1080 (0.250 sec)
INFO:tensorflow:global_step/sec: 398.728
INFO:tensorflow:loss = 2.3181179, step = 1180 (0.238 sec)
INFO:tensorflow:global_step/sec: 396.897
INFO:tensorflow:loss = 1.8086417, step = 1280 (0.250 sec)
INFO:tensorflow:global_step/sec: 414.237
INFO:tensorflow:loss = 0.6904925, step = 1380 (0.246 sec)
INFO:tensorflow:global_step/sec: 411.693
INFO:tensorflow:loss = 1.8734654, step = 1480 (0.250 sec)
INFO:tensorflow:global_step/sec: 401.569
INFO:tensorflow:loss = 2.5979433, step = 1580 (0.254 sec)
INFO:tensorflow:global_step/sec: 395.667
INFO:tensorflow:loss = 2.0128171, step = 1680 (0.256 sec)
INFO:tensorflow:global_step/sec: 392.234
INFO:tensorflow:loss = 2.469627, step = 1780 (0.244 sec)
INFO:tensorflow:global_step/sec: 386.751
INFO:tensorflow:loss = 0.87159, step = 1880 (0.253 sec)
INFO:tensorflow:global_step/sec: 404.765
INFO:tensorflow:loss = 0.80283445, step = 1980 (0.254 sec)
INFO:tensorflow:global_step/sec: 401.5
INFO:tensorflow:loss = 1.524719, step = 2080 (0.261 sec)
INFO:tensorflow:global_step/sec: 385.878
INFO:tensorflow:loss = 1.0228136, step = 2180 (0.261 sec)
INFO:tensorflow:global_step/sec: 382.386
INFO:tensorflow:loss = 1.0036705, step = 2280 (0.263 sec)
INFO:tensorflow:global_step/sec: 382.23
INFO:tensorflow:loss = 1.0771171, step = 2380 (0.245 sec)
INFO:tensorflow:global_step/sec: 388.433
INFO:tensorflow:loss = 0.9643565, step = 2480 (0.251 sec)
INFO:tensorflow:global_step/sec: 409.442
INFO:tensorflow:loss = 1.4598124, step = 2580 (0.264 sec)
INFO:tensorflow:global_step/sec: 382.398
INFO:tensorflow:loss = 0.7518444, step = 2680 (0.260 sec)
INFO:tensorflow:global_step/sec: 387.657
INFO:tensorflow:loss = 0.71297884, step = 2780 (0.260 sec)
INFO:tensorflow:global_step/sec: 387.516
INFO:tensorflow:loss = 0.21006158, step = 2880 (0.261 sec)
INFO:tensorflow:global_step/sec: 380.228
INFO:tensorflow:loss = 0.64975756, step = 2980 (0.252 sec)
INFO:tensorflow:global_step/sec: 375.953
INFO:tensorflow:loss = 0.3568688, step = 3080 (0.262 sec)
INFO:tensorflow:global_step/sec: 394.311
INFO:tensorflow:loss = 1.0947809, step = 3180 (0.260 sec)
INFO:tensorflow:global_step/sec: 389.576
INFO:tensorflow:loss = 0.38473517, step = 3280 (0.262 sec)
INFO:tensorflow:global_step/sec: 383.038
INFO:tensorflow:loss = 0.37087482, step = 3380 (0.266 sec)
INFO:tensorflow:global_step/sec: 377.258
INFO:tensorflow:loss = 0.37313935, step = 3480 (0.268 sec)
INFO:tensorflow:global_step/sec: 375.779
INFO:tensorflow:loss = 0.6371509, step = 3580 (0.253 sec)
INFO:tensorflow:global_step/sec: 376.039
INFO:tensorflow:loss = 0.6737277, step = 3680 (0.258 sec)
INFO:tensorflow:global_step/sec: 397.449
INFO:tensorflow:loss = 0.22763562, step = 3780 (0.264 sec)
INFO:tensorflow:global_step/sec: 379.907
INFO:tensorflow:loss = 0.70576984, step = 3880 (0.270 sec)
INFO:tensorflow:global_step/sec: 375.692
INFO:tensorflow:loss = 0.32033288, step = 3980 (0.266 sec)
INFO:tensorflow:global_step/sec: 376.935
INFO:tensorflow:loss = 0.5732076, step = 4080 (0.271 sec)
INFO:tensorflow:global_step/sec: 369.125
INFO:tensorflow:loss = 0.22866802, step = 4180 (0.257 sec)
INFO:tensorflow:global_step/sec: 370.509
INFO:tensorflow:loss = 0.27701426, step = 4280 (0.262 sec)
INFO:tensorflow:global_step/sec: 388.812
INFO:tensorflow:loss = 0.2290253, step = 4380 (0.273 sec)
INFO:tensorflow:global_step/sec: 373.834
INFO:tensorflow:loss = 0.24748756, step = 4480 (0.270 sec)
INFO:tensorflow:global_step/sec: 373.023
INFO:tensorflow:loss = 0.2879139, step = 4580 (0.275 sec)
INFO:tensorflow:global_step/sec: 364.77
INFO:tensorflow:loss = 0.28078204, step = 4680 (0.272 sec)
INFO:tensorflow:global_step/sec: 368.265
INFO:tensorflow:loss = 0.1984863, step = 4780 (0.261 sec)
INFO:tensorflow:global_step/sec: 363.698
INFO:tensorflow:loss = 0.31559613, step = 4880 (0.271 sec)
INFO:tensorflow:global_step/sec: 377.83
INFO:tensorflow:loss = 0.2904449, step = 4980 (0.277 sec)
INFO:tensorflow:global_step/sec: 363.8
INFO:tensorflow:loss = 0.28680754, step = 5080 (0.275 sec)
INFO:tensorflow:global_step/sec: 367.857
INFO:tensorflow:loss = 0.374867, step = 5180 (0.274 sec)
INFO:tensorflow:global_step/sec: 366.626
INFO:tensorflow:loss = 0.3683201, step = 5280 (0.280 sec)
INFO:tensorflow:global_step/sec: 357.256
INFO:tensorflow:loss = 0.2899915, step = 5380 (0.265 sec)
INFO:tensorflow:global_step/sec: 359.244
INFO:tensorflow:loss = 0.1280297, step = 5480 (0.268 sec)
INFO:tensorflow:global_step/sec: 381.8
INFO:tensorflow:loss = 0.7579371, step = 5580 (0.274 sec)
INFO:tensorflow:global_step/sec: 366.796
INFO:tensorflow:loss = 0.20086025, step = 5680 (0.278 sec)
INFO:tensorflow:global_step/sec: 363.209
INFO:tensorflow:loss = 0.20468965, step = 5780 (0.282 sec)
INFO:tensorflow:global_step/sec: 355.186
INFO:tensorflow:loss = 0.084839374, step = 5880 (0.284 sec)
INFO:tensorflow:global_step/sec: 353.547
INFO:tensorflow:loss = 0.7841339, step = 5980 (0.268 sec)
INFO:tensorflow:global_step/sec: 355.69
INFO:tensorflow:loss = 0.4825125, step = 6080 (0.270 sec)
INFO:tensorflow:global_step/sec: 378.825
INFO:tensorflow:loss = 0.15031722, step = 6180 (0.278 sec)
INFO:tensorflow:global_step/sec: 363.354
INFO:tensorflow:loss = 0.09604564, step = 6280 (0.281 sec)
INFO:tensorflow:global_step/sec: 359.431
INFO:tensorflow:loss = 0.22453651, step = 6380 (0.281 sec)
INFO:tensorflow:global_step/sec: 355.953
INFO:tensorflow:loss = 0.066752866, step = 6480 (0.287 sec)
INFO:tensorflow:global_step/sec: 348.331
INFO:tensorflow:loss = 0.13314456, step = 6580 (0.275 sec)
INFO:tensorflow:global_step/sec: 346.788
INFO:tensorflow:loss = 0.11664696, step = 6680 (0.281 sec)
INFO:tensorflow:global_step/sec: 365.916
INFO:tensorflow:loss = 0.24780986, step = 6780 (0.280 sec)
INFO:tensorflow:global_step/sec: 360.304
INFO:tensorflow:loss = 0.16076241, step = 6880 (0.289 sec)
INFO:tensorflow:global_step/sec: 347.319
INFO:tensorflow:loss = 0.15068403, step = 6980 (0.290 sec)
INFO:tensorflow:global_step/sec: 347.161
INFO:tensorflow:loss = 0.063347995, step = 7080 (0.288 sec)
INFO:tensorflow:global_step/sec: 346.469
INFO:tensorflow:loss = 0.17705172, step = 7180 (0.279 sec)
INFO:tensorflow:global_step/sec: 342.538
INFO:tensorflow:loss = 0.1235522, step = 7280 (0.283 sec)
INFO:tensorflow:global_step/sec: 362.863
INFO:tensorflow:loss = 0.19375022, step = 7380 (0.283 sec)
INFO:tensorflow:global_step/sec: 356.934
INFO:tensorflow:loss = 0.09878422, step = 7480 (0.285 sec)
INFO:tensorflow:global_step/sec: 350.67
INFO:tensorflow:loss = 0.044014893, step = 7580 (0.288 sec)
INFO:tensorflow:global_step/sec: 348.951
INFO:tensorflow:loss = 0.090123355, step = 7680 (0.292 sec)
INFO:tensorflow:global_step/sec: 343.379
INFO:tensorflow:loss = 0.1411789, step = 7780 (0.277 sec)
INFO:tensorflow:global_step/sec: 343.451
INFO:tensorflow:loss = 0.0606481, step = 7880 (0.286 sec)
INFO:tensorflow:global_step/sec: 358.731
INFO:tensorflow:loss = 0.11701955, step = 7980 (0.293 sec)
INFO:tensorflow:global_step/sec: 342.975
INFO:tensorflow:loss = 0.2144481, step = 8080 (0.299 sec)
INFO:tensorflow:global_step/sec: 337.35
INFO:tensorflow:loss = 0.13061918, step = 8180 (0.301 sec)
INFO:tensorflow:global_step/sec: 331.849
INFO:tensorflow:loss = 0.013081398, step = 8280 (0.298 sec)
INFO:tensorflow:global_step/sec: 336.503
INFO:tensorflow:loss = 0.027076408, step = 8380 (0.286 sec)
INFO:tensorflow:global_step/sec: 332.512
INFO:tensorflow:loss = 0.010121934, step = 8480 (0.293 sec)
INFO:tensorflow:global_step/sec: 352.257
INFO:tensorflow:loss = 0.023727953, step = 8580 (0.294 sec)
INFO:tensorflow:global_step/sec: 343.327
INFO:tensorflow:loss = 0.13345344, step = 8680 (0.296 sec)
INFO:tensorflow:global_step/sec: 339.463
INFO:tensorflow:loss = 0.06767905, step = 8780 (0.298 sec)
INFO:tensorflow:global_step/sec: 336.43
INFO:tensorflow:loss = 0.03239054, step = 8880 (0.299 sec)
INFO:tensorflow:global_step/sec: 337.212
INFO:tensorflow:loss = 0.03417517, step = 8980 (0.288 sec)
INFO:tensorflow:global_step/sec: 329.745
INFO:tensorflow:loss = 0.04349177, step = 9080 (0.295 sec)
INFO:tensorflow:global_step/sec: 346.215
INFO:tensorflow:loss = 0.10747677, step = 9180 (0.297 sec)
INFO:tensorflow:global_step/sec: 341.222
INFO:tensorflow:loss = 0.08463769, step = 9280 (0.302 sec)
INFO:tensorflow:global_step/sec: 333.558
INFO:tensorflow:loss = 0.022979608, step = 9380 (0.303 sec)
INFO:tensorflow:global_step/sec: 329.165
INFO:tensorflow:loss = 0.07760788, step = 9480 (0.310 sec)
INFO:tensorflow:global_step/sec: 322.089
INFO:tensorflow:loss = 0.038779423, step = 9580 (0.292 sec)
INFO:tensorflow:global_step/sec: 329.556
INFO:tensorflow:loss = 0.014404967, step = 9680 (0.297 sec)
INFO:tensorflow:global_step/sec: 343.326
INFO:tensorflow:loss = 0.06990504, step = 9780 (0.305 sec)
INFO:tensorflow:global_step/sec: 333.686
INFO:tensorflow:loss = 0.036858298, step = 9880 (0.305 sec)
INFO:tensorflow:global_step/sec: 326.461
INFO:tensorflow:loss = 0.047570646, step = 9980 (0.312 sec)
INFO:tensorflow:global_step/sec: 321.895
INFO:tensorflow:loss = 0.059428196, step = 10080 (0.309 sec)
INFO:tensorflow:global_step/sec: 325.738
INFO:tensorflow:loss = 0.05054853, step = 10180 (0.293 sec)
INFO:tensorflow:global_step/sec: 327.29
INFO:tensorflow:loss = 0.04085783, step = 10280 (0.300 sec)
INFO:tensorflow:global_step/sec: 337.825
INFO:tensorflow:loss = 0.06833278, step = 10380 (0.309 sec)
INFO:tensorflow:global_step/sec: 328.799
INFO:tensorflow:loss = 0.03984513, step = 10480 (0.309 sec)
INFO:tensorflow:global_step/sec: 325.714
INFO:tensorflow:loss = 0.029430978, step = 10580 (0.313 sec)
INFO:tensorflow:global_step/sec: 320.448
INFO:tensorflow:loss = 0.015103683, step = 10680 (0.310 sec)
INFO:tensorflow:global_step/sec: 321.814
INFO:tensorflow:loss = 0.055365227, step = 10780 (0.303 sec)
INFO:tensorflow:global_step/sec: 315.217
INFO:tensorflow:loss = 0.016110064, step = 10880 (0.316 sec)
INFO:tensorflow:global_step/sec: 323.304
INFO:tensorflow:loss = 0.006240257, step = 10980 (0.315 sec)
INFO:tensorflow:global_step/sec: 321.096
INFO:tensorflow:loss = 0.007149349, step = 11080 (0.321 sec)
INFO:tensorflow:global_step/sec: 314.465
INFO:tensorflow:loss = 0.0066786045, step = 11180 (0.312 sec)
INFO:tensorflow:global_step/sec: 320.341
INFO:tensorflow:loss = 0.025937172, step = 11280 (0.312 sec)
INFO:tensorflow:global_step/sec: 321.417
INFO:tensorflow:loss = 0.016570274, step = 11380 (0.303 sec)
INFO:tensorflow:global_step/sec: 317.392
INFO:tensorflow:loss = 0.0033354259, step = 11480 (0.308 sec)
INFO:tensorflow:global_step/sec: 330.218
INFO:tensorflow:loss = 0.017488046, step = 11580 (0.314 sec)
INFO:tensorflow:global_step/sec: 320.864
INFO:tensorflow:loss = 0.02159322, step = 11680 (0.322 sec)
INFO:tensorflow:global_step/sec: 310.693
INFO:tensorflow:loss = 0.020893702, step = 11780 (0.323 sec)
INFO:tensorflow:global_step/sec: 310.939
INFO:tensorflow:loss = 0.017859623, step = 11880 (0.326 sec)
INFO:tensorflow:global_step/sec: 304.814
INFO:tensorflow:loss = 0.014102906, step = 11980 (0.310 sec)
INFO:tensorflow:global_step/sec: 311.383
INFO:tensorflow:loss = 0.014420295, step = 12080 (0.316 sec)
INFO:tensorflow:global_step/sec: 323.922
INFO:tensorflow:loss = 0.012980898, step = 12180 (0.323 sec)
INFO:tensorflow:global_step/sec: 312.002
INFO:tensorflow:loss = 0.008047884, step = 12280 (0.324 sec)
INFO:tensorflow:global_step/sec: 309.195
INFO:tensorflow:loss = 0.005332183, step = 12380 (0.328 sec)
INFO:tensorflow:global_step/sec: 307.363
INFO:tensorflow:loss = 0.009909308, step = 12480 (0.331 sec)
INFO:tensorflow:global_step/sec: 303.166
INFO:tensorflow:loss = 0.018593434, step = 12580 (0.310 sec)
INFO:tensorflow:global_step/sec: 307.677
INFO:tensorflow:loss = 0.009453268, step = 12680 (0.318 sec)
INFO:tensorflow:global_step/sec: 323.497
INFO:tensorflow:loss = 0.0074377223, step = 12780 (0.317 sec)
INFO:tensorflow:global_step/sec: 318.278
INFO:tensorflow:loss = 0.0067944657, step = 12880 (0.326 sec)
INFO:tensorflow:global_step/sec: 307.95
INFO:tensorflow:loss = 0.009621896, step = 12980 (0.332 sec)
INFO:tensorflow:global_step/sec: 303.108
INFO:tensorflow:loss = 0.007392729, step = 13080 (0.329 sec)
INFO:tensorflow:global_step/sec: 303.111
INFO:tensorflow:loss = 0.0070271464, step = 13180 (0.317 sec)
INFO:tensorflow:global_step/sec: 302.852
INFO:tensorflow:loss = 0.01419846, step = 13280 (0.325 sec)
INFO:tensorflow:global_step/sec: 311.988
INFO:tensorflow:loss = 0.00879844, step = 13380 (0.330 sec)
INFO:tensorflow:global_step/sec: 307.168
INFO:tensorflow:loss = 0.0035331238, step = 13480 (0.333 sec)
INFO:tensorflow:global_step/sec: 301.33
INFO:tensorflow:loss = 0.004036055, step = 13580 (0.334 sec)
INFO:tensorflow:global_step/sec: 300.952
INFO:tensorflow:loss = 0.0021674812, step = 13680 (0.335 sec)
INFO:tensorflow:global_step/sec: 298.644
INFO:tensorflow:loss = 0.0044945157, step = 13780 (0.318 sec)
INFO:tensorflow:global_step/sec: 302.261
INFO:tensorflow:loss = 0.008261169, step = 13880 (0.328 sec)
INFO:tensorflow:global_step/sec: 310.556
INFO:tensorflow:loss = 0.007413184, step = 13980 (0.337 sec)
INFO:tensorflow:global_step/sec: 300.33
INFO:tensorflow:loss = 0.01038721, step = 14080 (0.331 sec)
INFO:tensorflow:global_step/sec: 304.304
INFO:tensorflow:loss = 0.0020925598, step = 14180 (0.329 sec)
INFO:tensorflow:global_step/sec: 303.857
INFO:tensorflow:loss = 0.0072769765, step = 14280 (0.337 sec)
INFO:tensorflow:global_step/sec: 297.895
INFO:tensorflow:loss = 0.0018916101, step = 14380 (0.326 sec)
INFO:tensorflow:global_step/sec: 294.092
INFO:tensorflow:loss = 0.0027799625, step = 14480 (0.327 sec)
INFO:tensorflow:global_step/sec: 312.093
INFO:tensorflow:loss = 0.0037557913, step = 14580 (0.334 sec)
INFO:tensorflow:global_step/sec: 301.829
INFO:tensorflow:loss = 0.0015468008, step = 14680 (0.334 sec)
INFO:tensorflow:global_step/sec: 302.182
INFO:tensorflow:loss = 0.0018402252, step = 14780 (0.332 sec)
INFO:tensorflow:global_step/sec: 301.447
INFO:tensorflow:loss = 0.0063510793, step = 14880 (0.339 sec)
INFO:tensorflow:global_step/sec: 294.111
INFO:tensorflow:loss = 0.003960237, step = 14980 (0.327 sec)
INFO:tensorflow:global_step/sec: 295.082
INFO:tensorflow:loss = 0.0021010689, step = 15080 (0.333 sec)
INFO:tensorflow:global_step/sec: 306.512
INFO:tensorflow:loss = 0.0011556938, step = 15180 (0.338 sec)
INFO:tensorflow:global_step/sec: 298.883
INFO:tensorflow:loss = 0.0009854774, step = 15280 (0.337 sec)
INFO:tensorflow:global_step/sec: 299.258
INFO:tensorflow:loss = 0.0059409747, step = 15380 (0.333 sec)
INFO:tensorflow:global_step/sec: 299.457
INFO:tensorflow:loss = 0.0022082897, step = 15480 (0.338 sec)
INFO:tensorflow:global_step/sec: 298.035
INFO:tensorflow:loss = 0.0036195924, step = 15580 (0.323 sec)
INFO:tensorflow:global_step/sec: 297.116
INFO:tensorflow:loss = 0.005268056, step = 15680 (0.332 sec)
INFO:tensorflow:global_step/sec: 304.785
INFO:tensorflow:loss = 0.0021239321, step = 15780 (0.342 sec)
INFO:tensorflow:global_step/sec: 298.12
INFO:tensorflow:loss = 0.0127066765, step = 15880 (0.339 sec)
INFO:tensorflow:global_step/sec: 295.993
INFO:tensorflow:loss = 0.0021492667, step = 15980 (0.341 sec)
INFO:tensorflow:global_step/sec: 293.45
INFO:tensorflow:loss = 0.003911408, step = 16080 (0.343 sec)
INFO:tensorflow:global_step/sec: 291.821
INFO:tensorflow:loss = 0.004051245, step = 16180 (0.334 sec)
INFO:tensorflow:global_step/sec: 287.44
INFO:tensorflow:loss = 0.0049018306, step = 16280 (0.342 sec)
INFO:tensorflow:global_step/sec: 297.459
INFO:tensorflow:loss = 0.0026472202, step = 16380 (0.345 sec)
INFO:tensorflow:global_step/sec: 293.164
INFO:tensorflow:loss = 0.0038542324, step = 16480 (0.348 sec)
INFO:tensorflow:global_step/sec: 288.779
INFO:tensorflow:loss = 0.003773787, step = 16580 (0.346 sec)
INFO:tensorflow:global_step/sec: 289.185
INFO:tensorflow:loss = 0.0026647656, step = 16680 (0.343 sec)
INFO:tensorflow:global_step/sec: 291.876
INFO:tensorflow:loss = 0.0024704284, step = 16780 (0.334 sec)
INFO:tensorflow:global_step/sec: 288.324
INFO:tensorflow:loss = 0.0034512142, step = 16880 (0.347 sec)
INFO:tensorflow:global_step/sec: 292.507
INFO:tensorflow:loss = 0.0062024607, step = 16980 (0.346 sec)
INFO:tensorflow:global_step/sec: 291.147
INFO:tensorflow:loss = 0.0022722099, step = 17080 (0.351 sec)
INFO:tensorflow:global_step/sec: 287.208
INFO:tensorflow:loss = 0.0014444834, step = 17180 (0.352 sec)
INFO:tensorflow:global_step/sec: 283.574
INFO:tensorflow:loss = 0.0074605285, step = 17280 (0.357 sec)
INFO:tensorflow:global_step/sec: 281.604
INFO:tensorflow:loss = 0.003752734, step = 17380 (0.339 sec)
INFO:tensorflow:global_step/sec: 283.366
INFO:tensorflow:loss = 0.0012563546, step = 17480 (0.342 sec)
INFO:tensorflow:global_step/sec: 297.925
INFO:tensorflow:loss = 0.003298856, step = 17580 (0.347 sec)
INFO:tensorflow:global_step/sec: 292.149
INFO:tensorflow:loss = 0.0021164892, step = 17680 (0.346 sec)
INFO:tensorflow:global_step/sec: 289.272
INFO:tensorflow:loss = 0.0027668625, step = 17780 (0.350 sec)
INFO:tensorflow:global_step/sec: 286.518
INFO:tensorflow:loss = 0.0038928108, step = 17880 (0.356 sec)
INFO:tensorflow:global_step/sec: 280.948
INFO:tensorflow:loss = 0.00068626396, step = 17980 (0.340 sec)
INFO:tensorflow:global_step/sec: 281.988
INFO:tensorflow:loss = 0.0011843208, step = 18080 (0.349 sec)
INFO:tensorflow:global_step/sec: 292.284
INFO:tensorflow:loss = 0.0018866074, step = 18180 (0.351 sec)
INFO:tensorflow:global_step/sec: 288.176
INFO:tensorflow:loss = 0.0005333081, step = 18280 (0.352 sec)
INFO:tensorflow:global_step/sec: 285.166
INFO:tensorflow:loss = 0.0005375584, step = 18380 (0.360 sec)
INFO:tensorflow:global_step/sec: 279.2
INFO:tensorflow:loss = 0.0067465273, step = 18480 (0.355 sec)
INFO:tensorflow:global_step/sec: 280.193
INFO:tensorflow:loss = 0.0013988668, step = 18580 (0.344 sec)
INFO:tensorflow:global_step/sec: 281.697
INFO:tensorflow:loss = 0.0014645823, step = 18680 (0.351 sec)
INFO:tensorflow:global_step/sec: 288.122
INFO:tensorflow:loss = 0.0014383025, step = 18780 (0.360 sec)
INFO:tensorflow:global_step/sec: 282.176
INFO:tensorflow:loss = 0.0014143193, step = 18880 (0.361 sec)
INFO:tensorflow:global_step/sec: 277.737
INFO:tensorflow:loss = 0.0013943117, step = 18980 (0.357 sec)
INFO:tensorflow:global_step/sec: 281.123
INFO:tensorflow:loss = 0.0006448065, step = 19080 (0.357 sec)
INFO:tensorflow:global_step/sec: 281.612
INFO:tensorflow:loss = 0.0014809513, step = 19180 (0.348 sec)
INFO:tensorflow:global_step/sec: 274.105
INFO:tensorflow:loss = 0.0008602524, step = 19280 (0.358 sec)
INFO:tensorflow:global_step/sec: 285.319
INFO:tensorflow:loss = 0.0006964795, step = 19380 (0.363 sec)
INFO:tensorflow:global_step/sec: 277.315
INFO:tensorflow:loss = 0.00035264163, step = 19480 (0.366 sec)
INFO:tensorflow:global_step/sec: 274.599
INFO:tensorflow:loss = 0.0010025422, step = 19580 (0.367 sec)
INFO:tensorflow:global_step/sec: 273.115
INFO:tensorflow:loss = 0.0007096651, step = 19680 (0.362 sec)
INFO:tensorflow:global_step/sec: 276.323
INFO:tensorflow:loss = 0.0013329595, step = 19780 (0.351 sec)
INFO:tensorflow:global_step/sec: 274.789
INFO:tensorflow:loss = 0.0008460893, step = 19880 (0.357 sec)
INFO:tensorflow:global_step/sec: 283.638
INFO:tensorflow:loss = 0.0011283578, step = 19980 (0.368 sec)
INFO:tensorflow:global_step/sec: 275.094
INFO:tensorflow:loss = 0.00089822686, step = 20080 (0.365 sec)
INFO:tensorflow:global_step/sec: 275.392
INFO:tensorflow:loss = 0.0014473142, step = 20180 (0.364 sec)
INFO:tensorflow:global_step/sec: 276.458
INFO:tensorflow:loss = 0.0008915104, step = 20280 (0.373 sec)
INFO:tensorflow:global_step/sec: 268.018
INFO:tensorflow:loss = 0.0004781757, step = 20380 (0.353 sec)
INFO:tensorflow:global_step/sec: 272.515
INFO:tensorflow:loss = 0.0004186085, step = 20480 (0.363 sec)
INFO:tensorflow:global_step/sec: 280.449
INFO:tensorflow:loss = 0.0008953349, step = 20580 (0.364 sec)
INFO:tensorflow:global_step/sec: 278.265
INFO:tensorflow:loss = 0.0015090622, step = 20680 (0.371 sec)
INFO:tensorflow:global_step/sec: 270.082
INFO:tensorflow:loss = 0.0010438098, step = 20780 (0.374 sec)
INFO:tensorflow:global_step/sec: 267.97
INFO:tensorflow:loss = 0.00050447625, step = 20880 (0.376 sec)
INFO:tensorflow:global_step/sec: 267.517
INFO:tensorflow:loss = 0.00037436924, step = 20980 (0.364 sec)
INFO:tensorflow:global_step/sec: 262.304
INFO:tensorflow:loss = 0.0005487846, step = 21080 (0.371 sec)
INFO:tensorflow:global_step/sec: 276.63
INFO:tensorflow:loss = 0.0012135495, step = 21180 (0.372 sec)
INFO:tensorflow:global_step/sec: 271.146
INFO:tensorflow:loss = 0.00050225714, step = 21280 (0.374 sec)
INFO:tensorflow:global_step/sec: 266.848
INFO:tensorflow:loss = 0.0005835245, step = 21380 (0.380 sec)
INFO:tensorflow:global_step/sec: 266.179
INFO:tensorflow:loss = 0.0004619556, step = 21480 (0.375 sec)
INFO:tensorflow:global_step/sec: 266.419
INFO:tensorflow:loss = 0.00033856914, step = 21580 (0.363 sec)
INFO:tensorflow:global_step/sec: 265.468
INFO:tensorflow:loss = 0.0008394742, step = 21680 (0.373 sec)
INFO:tensorflow:global_step/sec: 272.316
INFO:tensorflow:loss = 0.00030781276, step = 21780 (0.374 sec)
INFO:tensorflow:global_step/sec: 270.614
INFO:tensorflow:loss = 0.00032267775, step = 21880 (0.375 sec)
INFO:tensorflow:global_step/sec: 266.912
INFO:tensorflow:loss = 0.00024132222, step = 21980 (0.378 sec)
INFO:tensorflow:global_step/sec: 265.246
INFO:tensorflow:loss = 0.00028675678, step = 22080 (0.376 sec)
INFO:tensorflow:global_step/sec: 266.509
INFO:tensorflow:loss = 0.0009781871, step = 22180 (0.365 sec)
INFO:tensorflow:global_step/sec: 263.828
INFO:tensorflow:loss = 0.0010109144, step = 22280 (0.370 sec)
INFO:tensorflow:global_step/sec: 274.649
INFO:tensorflow:loss = 0.00025149249, step = 22380 (0.378 sec)
INFO:tensorflow:global_step/sec: 267.999
INFO:tensorflow:loss = 0.00020908765, step = 22480 (0.378 sec)
INFO:tensorflow:global_step/sec: 265.322
INFO:tensorflow:loss = 0.0004320807, step = 22580 (0.384 sec)
INFO:tensorflow:global_step/sec: 260.926
INFO:tensorflow:loss = 0.0002488165, step = 22680 (0.385 sec)
INFO:tensorflow:global_step/sec: 259.177
INFO:tensorflow:loss = 0.0004015111, step = 22780 (0.376 sec)
INFO:tensorflow:global_step/sec: 256.529
INFO:tensorflow:loss = 0.00037404272, step = 22880 (0.385 sec)
INFO:tensorflow:global_step/sec: 264.999
INFO:tensorflow:loss = 0.00039812157, step = 22980 (0.387 sec)
INFO:tensorflow:global_step/sec: 260.906
INFO:tensorflow:loss = 0.0005162174, step = 23080 (0.386 sec)
INFO:tensorflow:global_step/sec: 259.663
INFO:tensorflow:loss = 0.00032000744, step = 23180 (0.387 sec)
INFO:tensorflow:global_step/sec: 258.701
INFO:tensorflow:loss = 0.00025557584, step = 23280 (0.391 sec)
INFO:tensorflow:global_step/sec: 255.811
INFO:tensorflow:loss = 0.00018507428, step = 23380 (0.372 sec)
INFO:tensorflow:global_step/sec: 260.467
INFO:tensorflow:loss = 0.00010121861, step = 23480 (0.376 sec)
INFO:tensorflow:global_step/sec: 271.391
INFO:tensorflow:loss = 0.00043678225, step = 23580 (0.381 sec)
INFO:tensorflow:global_step/sec: 264.417
INFO:tensorflow:loss = 0.0002813889, step = 23680 (0.391 sec)
INFO:tensorflow:global_step/sec: 257.244
INFO:tensorflow:loss = 9.453914e-05, step = 23780 (0.393 sec)
INFO:tensorflow:global_step/sec: 254.353
INFO:tensorflow:loss = 0.0002390909, step = 23880 (0.390 sec)
INFO:tensorflow:global_step/sec: 240.95
INFO:tensorflow:loss = 0.0008116873, step = 23980 (0.442 sec)
INFO:tensorflow:global_step/sec: 229.283
INFO:tensorflow:Saving checkpoints for 24000 into /tmp/tmpbzo1p2wi/model.ckpt.
WARNING:tensorflow:Issue encountered when serializing resources.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
'_Resource' object has no attribute 'name'
INFO:tensorflow:Loss for final step: 0.00040755837.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2019-11-03T11:19:05Z
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmpbzo1p2wi/model.ckpt-24000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Finished evaluation at 2019-11-03-11:19:05
INFO:tensorflow:Saving dict for global step 24000: average_loss = 12.3817, global_step = 24000, label/mean = 23.611393, loss = 12.283247, prediction/mean = 22.392288
WARNING:tensorflow:Issue encountered when serializing resources.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
'_Resource' object has no attribute 'name'
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 24000: /tmp/tmpbzo1p2wi/model.ckpt-24000
{'average_loss': 12.3817, 'label/mean': 23.611393, 'loss': 12.283247, 'prediction/mean': 22.392288, 'global_step': 24000}
Average-Loss 12.3817
Readers may ignore the next cell.
In [20]python · cell 29
python
! python ../.convert_notebook_to_script.py --input ch14_part2.ipynb --output ch14_part2.pyOutput
[NbConvertApp] Converting notebook ch14_part2.ipynb to script [NbConvertApp] Writing 6364 bytes to ch14_part2.py