99、Tensorflow Serving 實現模型的部署
昨晚終於實現了Tensorflow模型的部署 使用TensorFlow Serving
1、使用Docker 獲取Tensorflow Serving的鏡像，Docker在國內的需要將鏡像的Repository地址設置爲阿里雲的加速地址，這個大家可以自己去CSDN上面找
然後啓動docker
2、使用Tensorflow 的 SaveModelBuilder保存Tensorflow的計算圖模型，並且設置Signature,
Signature主要用來標識模型的輸入值的名稱和類型
builder = saved_model_builder.SavedModelBuilder(export_path)
classification_inputs = utils.build_tensor_info(cnn.input_x)
classification_dropout_keep_prob = utils.build_tensor_info(cnn.dropout_keep_prob)
classification_outputs_classes = utils.build_tensor_info(prediction_classes)
classification_outputs_scores = utils.build_tensor_info(cnn.scores)
classification_signature = signature_def_utils.build_signature_def(
inputs={signature_constants.CLASSIFY_INPUTS: classification_inputs,
signature_constants.CLASSIFY_INPUTS:classification_dropout_keep_prob
},
outputs={
signature_constants.CLASSIFY_OUTPUT_CLASSES:
classification_outputs_classes,
signature_constants.CLASSIFY_OUTPUT_SCORES:
classification_outputs_scores
},
method_name=signature_constants.CLASSIFY_METHOD_NAME)
tensor_info_x = utils.build_tensor_info(cnn.input_x)
tensor_info_y = utils.build_tensor_info(cnn.predictions)
tensor_info_dropout_keep_prob = utils.build_tensor_info(cnn.dropout_keep_prob)
prediction_signature = signature_def_utils.build_signature_def(
inputs={'inputX': tensor_info_x,
'input_dropout_keep_prob':tensor_info_dropout_keep_prob},
outputs={'predictClass': tensor_info_y},
method_name=signature_constants.PREDICT_METHOD_NAME)
legacy_init_op = tf.group(tf.tables_initializer(), name='legacy_init_op')
#add the sigs to the servable
builder.add_meta_graph_and_variables(
sess, [tag_constants.SERVING],
signature_def_map={
'textclassified':
prediction_signature,
signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
classification_signature,
},
legacy_init_op=legacy_init_op)
#save it!
builder.save(True)
保存之後的計算圖的結構可以從下面這裏看見，下面這裏只給出模型的signature部分，因爲signature裏面定義了你到時候call restful接口的參數名稱和類型
signature_def {
key: "serving_default"
value {
inputs {
key: "inputs"
value {
name: "dropout_keep_prob:0"
dtype: DT_FLOAT
tensor_shape {
unknown_rank: true
}
}
}
outputs {
key: "classes"
value {
name: "index_to_string_Lookup:0"
dtype: DT_STRING
tensor_shape {
dim {
size: 1
}
}
}
}
outputs {
key: "scores"
value {
name: "output/scores:0"
dtype: DT_FLOAT
tensor_shape {
dim {
size: -1
}
dim {
size: 2
}
}
}
}
method_name: "tensorflow/serving/classify"
}
}
signature_def {
key: "textclassified"
value {
inputs {
key: "inputX"
value {
name: "input_x:0"
dtype: DT_INT32
tensor_shape {
dim {
size: -1
}
dim {
size: 40
}
}
}
}
inputs {
key: "input_dropout_keep_prob"
value {
name: "dropout_keep_prob:0"
dtype: DT_FLOAT
tensor_shape {
unknown_rank: true
}
}
}
outputs {
key: "predictClass"
value {
name: "output/predictions:0"
dtype: DT_INT64
tensor_shape {
dim {
size: -1
}
}
}
}
method_name: "tensorflow/serving/predict"
}
}
}
從上面的Signature定義可以看出 到時候call restfull 接口需要傳兩個參數，
int32類型的名稱爲inputX參數
float類型名稱爲input_drop_out_keep_prob的參數
上面的protocol buffer 中
textclassified表示使用TextCnn卷積神經網絡來進行預測，然後預測功能的名稱叫做textclassified
3、將模型部署到Tensorflow Serving 上面
首先把模型通過工具傳輸到docker上面
模型的結構如下
傳到docker上面，然後在外邊套一個文件夾名字起位模型的名字，叫做
text_classified_model 然後執行下面這條命令運行tensorflow/serving
docker run -p 8500:8500 --mount type=bind,source=/home/docker/model/text_classified_model,target=/mo dels/text_classified_model -e MODEL_NAME=text_classified_model -t tensorflow/serving
source表示模型在docker上面的路徑
target表示模型在docker中TensorFlow/serving container上面的路徑
然後輸入如下所示
另一個是gRPC接口端口是8500
gRPC是HTTP/2協議，REST API 是HTTP/1協議
區別是gRPC只有POST/GET兩種請求方式
REST API還有其餘很多種 列如 PUT/DELETE 等
4、客戶端調用gPRC接口
需要傳兩個參數，
一個是
inputX
另一個是
input_dropout_keep_prob
'''
Created on 2018年10月17日
@author: 95890
'''
"""Send text to tensorflow serving and gets result
"""
# This is a placeholder for a Google-internal import.
from grpc.beta import implementations
import tensorflow as tf
import data_helpers
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2
from tensorflow.contrib import learn
import numpy as np
tf.flags.DEFINE_string("positive_data_file", "./data/rt-polaritydata/rt-polarity.pos", "Data source for the positive data.")
tf.flags.DEFINE_string("negative_data_file", "./data/rt-polaritydata/rt-polarity.neg", "Data source for the negative data.")
tf.flags.DEFINE_string('server', '192.168.99.100:8500',
'PredictionService host:port')
FLAGS = tf.flags.FLAGS
x_text=[]
y=[]
max_document_length=40
def main(_):
testStr =["wisegirls is its low-key quality and genuine"]
if x_text.__len__()==0:
x_text, y = data_helpers.load_data_and_labels(FLAGS.positive_data_file, FLAGS.negative_data_file)
max_document_length = max([len(x.split(" ")) for x in x_text])
vocab_processor = learn.preprocessing.VocabularyProcessor(max_document_length)
vocab_processor.fit(x_text)
x = np.array(list(vocab_processor.fit_transform(testStr)))
host, port = FLAGS.server.split(':')
channel = implementations.insecure_channel(host, int(port))
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
request = predict_pb2.PredictRequest()
request.model_spec.name = "text_classified_model"
request.model_spec.signature_name = 'textclassified'
dropout_keep_prob = np.float(1.0)
request.inputs['inputX'].CopyFrom(
tf.contrib.util.make_tensor_proto(x, shape=[1,40],dtype=np.int32))
request.inputs['input_dropout_keep_prob'].CopyFrom(
tf.contrib.util.make_tensor_proto(dropout_keep_prob, shape=[1],dtype=np.float))
result = stub.Predict(request, 10.0) # 10 secs timeout
print(result)
if __name__ == '__main__':
tf.app.run()
outputs {
key: "predictClass"
value {
dtype: DT_INT64
tensor_shape {
dim {
size: 1
}
}
int64_val: 1
}
}
model_spec {
name: "text_classified_model"
version {
value: 1
}
signature_name: "textclassified"
}
從上面的結果可以看出，我們傳入了一句話
wisegirls is its low-key quality and genuine
分類的結果
predictClass
int64_val: 1
分成第一類
這個真的是神經網絡的部署呀。
啦啦啦 ， Tensorflow真的很牛，上至瀏覽器，下到手機，一次訓練，一次導出。處處運行。
沒有不敢想，只有不敢做
The Full version can be find here
https://github.com/weizhenzhao/TextCNN_Tensorflow_Serving/tree/master
Thanks
WeiZhen