xgboost模型序列化存储并推理
xgboost模型序列化存储并推理
参考了博客 https://github.com/apachecn/ml-mastery-zh/blob/master/docs/xgboost/save-gradient-boosting-models-xgboost-python.md ，但是修改了一些过时的部分。
我们在 Pima 印第安人糖尿病数据集 上训练xgboost模型，训练数据集在GitHub 下载 1
wget https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv
Pickle
Pickle是一个python序列化的标准方法。
先训练一个模型,然后将模型按照Pickle的形式存储，接下来读取模型并进行推理 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24import random
from numpy import loadtxt
import xgboost
import pickle
from sklearn import model_selection
from sklearn.metrics import accuracy_score
from sklearn import model_selection as cross_validation
# load data
dataset = loadtxt('pima-indians-diabetes.data.csv', delimiter=",")
# split data into X and y
X = dataset[:,0:8]
Y = dataset[:,8]
# split data into train and test sets
seed = random.randint(1, 100)
test_size = 0.33
X_train, X_test, y_train, y_test = cross_validation.train_test_split(X, Y, test_size=test_size, random_state=seed)
# fit model no training data
model = xgboost.XGBClassifier()
model.fit(X_train, y_train)
# save model to file
pickle.dump(model, open("pima.pickle.dat", "wb"))
读取模型并推理 1
2
3
4
5
6
7
8
9
10
11# load model from file
loaded_model = pickle.load(open("pima.pickle.dat", "rb"))
# train model again
loaded_model.fit(X_train, y_train)
# make predictions for test data
y_pred = loaded_model.predict(X_test)
predictions = [round(value) for value in y_pred]
# evaluate predictions
accuracy = accuracy_score(y_test, predictions)
print("Accuracy: %.2f%%" % (accuracy * 100.0))
joblib
Joblib 是一组在 Python 中提供轻量级流水线的工具，joblib 在大型 numpy 数组上通常要快得多
用法实际上和pickle基本相同。 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23# Train XGBoost model, save to file using joblib, load and make predictions
import random
from numpy import loadtxt
import xgboost
import joblib
from sklearn import model_selection
from sklearn.metrics import accuracy_score
from sklearn import model_selection as cross_validation
# load data
dataset = loadtxt('pima-indians-diabetes.data.csv', delimiter=",")
# split data into X and y
X = dataset[:,0:8]
Y = dataset[:,8]
# split data into train and test sets
seed = random.randint(1, 100)
test_size = 0.33
X_train, X_test, y_train, y_test = cross_validation.train_test_split(X, Y, test_size=test_size, random_state=seed)
# fit model no training data
model = xgboost.XGBClassifier()
model.fit(X_train, y_train)
# save model to file
joblib.dump(model, "pima.joblib.dat")
读取模型并推理
1 | |