Deep Learning of Tensorflow Production Environment Deployment (Part 2: Model Deployment)

The previous one talked about the deployment of environment. This one talked about how to derive PB model from the code point of view and how to make service invocation.

1 Hello World

After the docker is deployed, tensorflow / serving can be pulled out directly if it is a CPU environment, and it is a trouble if it is a GPU environment. Refer to the previous article, we will not go into details here.

The CPU version can pull tensorflow/serving directly, and docker will automatically pull latest version:

Docker pull tensorflow/serving

If you want to specify a version of tensorflow, check here: https://hub.docker.com/r/tensorflow/serving/tags/

For example, if I need the 1.12.0 version of tf, then I can also pull the specified version:

Docker pull tensorflow/serving: 1.12.0

After pulling out the image, you need to download a program code for Hello world.

Mkdir-p/tmp/tfserving

Cd/tmp/tfserving

Git clone https://github.com/tensorflow/serving

There is a corresponding test model in tensorflow/service github, which is y = 0.5 * x + 2. That is to say, input a number and output a corresponding y.

Run the following command to deploy the service in docker:

Docker run-p 8501:8501 -- mount type = bind, source=/tmp/serving/tensorflow_serving/servables/tensorflow/testdata/saved_model_half_plus_two_cpu, target=/models/half_plus_two-e MODEL_NAME=half_plus_two-t tensorflow/serving&

In the above command, mount the / TMP / serving / tensorflow_serving / servables / tensorflow / testdata / saved_model_half_plus_two_cpu path to / Models / half_plus_two, so tensorflow_serving can load the model under models, and then open the HTTP interface of 8501.

Execute docker PS to view the list of services:

_docker PS

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

7decb4286057 tensorflow/serving "/ usr/bin/tf_serving..." "7 seconds ago Up 6 seconds 8500/tcp, 0.0.0:8501->8501/tcp eager_dewdney

Send an HTTP request to test:

Curl-d'{"instances": [1.0, 2.0, 5.0]}'-X POST http://localhost:8501/v1/models/half_plus_two:predict

{

"Predictions": [2.5, 3.0, 4.5]

]

%%

2 MNIST

Because of the previous example, there is only PB model in service engineering, and there is no training and exporting of model, so the doorway can not be seen. This part is based on the example of handwriting recognition directly, showing how to export the model from tensorflow training code, and how to invoke the model through grpc service.

Training and Exporting:

#!/usr/bin/env Python

""

Softmax regression model is trained and derived. SaveModel is used to export training model and add signature.

""

From future import print_function

Import OS

Import sys

# This is a placeholder for a Google-internal import.

Import tensorflow as TF

Import SSL

Ssl. _create_default_https_context = ssl. _create_unverified_context

Import basic.mnist_input_data as mnist_input_data

# Define model parameters

Tf.app.flags.DEFINE_integer('training_iteration', 10,'number of training iterations')

Tf. app. flags. DEFINE_integer ('model_version', 2,'version number of the model.')

Tf. app. flags. DEFINE_string ('work_dir','. / tmp','Working Directory')

FLAGS = tf. app. flags. FLAGS

Def main ():

# Calibration of parameters

# if len (sys. argv) < 2 or sys. argv [-1]. start with ('-'):

# print ('Usage: mnist_saved_model.py [- training_iteration=x]'

#'[- model_version=y] export_dir')

# sys. exit (- 1)

# If FLAGS.training_iteration <= 0:

# print ('Please specify a positive value for training iteration.')

# sys. exit (- 1)

# If FLAGS.model_version<== 0:

# print ('Please specify a positive value for version number.')

# sys. exit (- 1)


# Train model

Print ('Training model...')

MNIST = mnist_input_data. read_data_sets (FLAGS. work_dir, one_hot = True)

Sess = tf. InteractiveSession ()

Serialized_tf_example = tf. placeholder (tf. string, name ='tf_example')

Feature_configs = {'x': tf.FixedLenFeature (shape=[784], dtype=tf.float32),}

Tf_example = tf.parse_example (serialized_tf_example, feature_configs)

X = tf. identity (tf_example ['x'], name ='x') # use tf. identity () to assign name

Y_ = tf. placeholder ('float', shape= [None, 10])

W = tf. Variable (tf. zeros ([784, 10]))

B = tf. Variable (tf. zeros ([10]))

Sess. run (tf. global_variables_initializer())

Y = tf. nn. softmax (tf. matmul (x, w) + b, name ='y')

Cross_entropy = tf. reduce_sum (y_* tf. log (y))

Train_step = tf. train. Gradient DescentOptimizer (0.01). minimize (cross_entropy)

Values, indices = tf. nn. top_k (y, 10)

Table = tf. contrib. lookup. index_to_string_table_from_tensor（

Tf.constant ([str (i) for I in range (10)])

Prediction_classes = table. lookup (tf. to_int64 (indices))

For in range (FLAGS. training_iteration):

Batch = mnist. train. next_batch (50)

Train_step.run (feed_dict={x: batch [0], y_: batch [1]})

Correct_prediction = tf. equal (tf. argmax (y, 1), tf. argmax (y, 1))

Accuracy = tf. reduce_mean (tf. cast (correct_prediction,'float'))

Print ('training accuracy% g'%) sess. run（

Accuracy, feed_dict={

X: mnist. test. images,

Y_: mnis

Please read the Chinese version for details.