Use LibTorch to call the pytorch model in C++

Updated: December 17, 2022 16:49:00 by sq_damowang

This article mainly introduces the use of LibTorch for C++ to call pytorch model, has a good reference value, I hope to help you. If there are mistakes or incomplete areas, please feel free to comment

catalogue

Sum up

The day before yesterday for some reason need to use C++ to call PyTorch, so I came into contact with LibTorch, with two days and finally have a certain effect, so record it.

environment

PyTorch1.6.0
cuda10.2
opencv4.4.0
VS2017

Concrete process

Download LibTorch

Go to PyTorch official website to download LibTorch package, select the corresponding version, here I select Stable(1.6.0), Windows, LibTorch, C++/JAVA, 10.2, and then I select release version to download, as shown below

Don't worry about it after you download it. You can use it later

Generate the model file with pytorch

I first created a python file to load the resnet50 pre-trained model, used to generate the model file, the code is as follows

import torch import torchvision.models as models from PIL import Image import numpy as np from torchvision import transforms model_resnet = models.resnet50(pretrained=True).cuda() # model_resnet.load_state_dict(torch.load("resnet_Epoch_4_Top1_99.75845336914062.pkl")) model_resnet.eval() # Select any picture, And write its path in the open method, which reads the image, Image = image.open ("111.jpg").convert('RGB') tf = transforms.Compose([transforms.Resize((224, 224))), transforms.ToTensor(), # transforms.Normalize(mean=[0.5]*3, std=[0.5]*3)]) img = tf(image) img = img.unsqueeze(dim=0) print(img.shape) input = torch.rand(1, 3, 224, 224).cuda() traced_script_module_resnet = torch.jit.trace(model_resnet, input) output = traced_script_module_resnet(img.cuda()) print(output.shape) pred = torch.argmax(output, dim=1) print(pred) traced_script_module_resnet.save("model_resnet_jit_cuda.pt")

Finally, a model_resnet_jit_cuda.pt file can be generated, resulting in the output shown below

The first line is the shape of the image we read. After reading the image, we went through various resizes, increased dimensions, and modified the shape of the image data into the format accepted by the model. We can see that the predicted result is 921. Then we will use the generated model_resnet_jit_cuda.pt file.

VS Create the project and configure the environment

I created the vs project Project1 under this python file path

Once created, we open the Project1 folder, which contains the following contents

Now that the VS project has been created, configure the project environment. Unzip the LibTorch you downloaded to the current directory. After unzip, a libtorch folder will appear. The contents of the folder directory are

Here, configure the folder path I have selected into the project properties, open the newly created VS project and select the x 64 version of the relaese project

Then click Project ->Project1 Properties to bring up the properties page

In the properties page, also note the x 64 platform of release, click the VC++ directory, load the include folder path that I have framed before in the include directory, load the lib folder path that I have framed in the library directory, and at the same time, we also need to use opencv. Therefore, it is also necessary to load the include folder and opencv2 folder of opencv under the include directory, and load opencv\build\x64\vc14\lib under the library directory, as shown in the following figure

Then, in the linker -> Input on the properties page, add additional dependencies, first adding the opencv dependencies

opencv_world440.lib, (if you've been using Debug mode, add opencv_world440d.lib), then add all the.lib files in libtorch/lib, and open the folder

Write it all in and click OK, as shown in the image below

Then click the linker -> Command line and add /INCLUDE:? warp_size@cuda@at@@YAHXZ this sentence, added this sentence because we want to use cuda version, if the cpu version can not be added.

Finally click C/C++ -> General SDL check and set to No

Click C/C++ -> Language Match Mode and set to No

At this point our configuration is all over! At last! Copy all the files in the libtorch/lib folder and paste them into the project folder Project1/ x 64/release (click on Project1 here and you can see that there is also a x 64/release in there, I was confused about where to put it, then I tried them all, Turns out you can leave it in there)

Run the VS2017 project file

Then I ran the VS project next to an empty main file with no errors, the configuration was mostly fine, and finally added the full code as follows

#include <torch/script.h> // One-stop header. #include <opencv2/opencv.hpp> #include <iostream> #include <memory> //https://pytorch.org/tutorials/advanced/cpp_export.html std::string image_path = ".. /.. /111.jpg"; int main(int argc, const char* argv[]) { // Deserialize the ScriptModule from a file using torch::jit::load(). //std::shared_ptr<torch::jit::script::Module> module = torch::jit::load(".. /.. /model_resnet_jit.pt"); using torch::jit::script::Module; Module module = torch::jit::load(".. /.. /model_resnet_jit_cuda.pt"); module.to(at::kCUDA); //assert(module ! = nullptr); //std::cout << "ok\n"; auto image = cv::imread(image_path, cv::ImreadModes::IMREAD_COLOR); cv::cvtColor(image, image, cv::COLOR_BGR2RGB); cv::Mat image_transfomed; cv::resize(image, image_transfomed, cv::Size(224, 224)); // Convert to Tensor torch::Tensor tensor_image = torch::from_blob(image_transfomed.data, {image_transfomed.rows, image_transfomed.cols,3 }, torch::kByte); tensor_image = tensor_image.permute({2,0,1}); tensor_image = tensor_image.toType(torch::kFloat); tensor_image = tensor_image.div(255); tensor_image = tensor_image.unsqueeze(0); tensor_image = tensor_image.to(at::kCUDA); // Network forward calculation at::Tensor output = module.forward({tensor_image}).toTensor(); //std::cout << "output:" << output << std::endl; auto prediction = output.argmax(1); std::cout << "prediction:" << prediction << std::endl; int maxk = 3; auto top3 = std::get<1>(output.topk(maxk, 1, true, true)); std::cout << "top3: " << top3 << '\n'; std::vector<int> res; for (auto i = 0; i < maxk; i++) { res.push_back(top3[0][i].item().toInt()); } for (auto i : res) { std::cout << i << " "; } std::cout << "\n"; system("pause"); }

Get the final output of 921, you can see the same as the previous python file output, here also output its top three, respectively, are 921,787,490.

Notice that my two outputs have the same premise:

1, make sure to load the model generated by the corresponding python file!

2, the input picture is the same! And the same transformation is performed under python and C++. Here, I convert it to RGB model under python, resize (224, 224), divide each element value by 255.0, convert it to between 0 and 1 (ToTensor () method), and finally convert the dimension to 1. 3, 224, 224, in C++, it is also necessary to convert the BGR model to RGB model, scale the image to 224,224, divide the pixel value by 255, convert the type to float, and finally convert the dimension to 1,3,224,224, and then carry out network forward calculation.

Sum up

The above is personal experience, I hope to give you a reference, but also hope that you support the script home.

Articles you may be interested in:

An introduction to the basic syntax of Python learning

Hello, everyone, this article is mainly about the basic grammar of Python learning, interested students come and have a look, if it is helpful to you remember to collect it, convenient for next browsing
2021-12-12
Python gevent coroutine switch implementation details

This article mainly introduces the Python gevent coroutine switch implementation details, the article through the example code is very detailed, for everyone's study or work has a certain reference learning value, the need of friends can refer to it
2020-09-09
Object Orientation for python Learning

Python has been an object-oriented language since day one. Because of this, it is very easy to create and use classes and objects. This article mainly introduces the object-oriented information in python, belongs to the primary article of python object-oriented learning, this chapter will help you in the use of Python object-oriented programming technology to improve all aspects of the need for reference.
2017-01-01
Python automatically input ERP system data

This article mainly introduces how Python automatically input ERP system data, the best way to solve Excel problems with Python, the article has detailed code examples, need friends can refer to read
2023-04-04
Python implement bucket sort and quick sort algorithm combined application example

This article mainly introduces the combination of Python bucket sort and quick sort algorithm application, combined with example form analysis of Python quick sort and bucket sort application related implementation skills, need friends can refer to the next
2017-11-11
Discussion on pytorch torch.backends.cudnn setting function

Today, Xiaobian will share a brief discussion on the role of pytorch torch.backends.cudnn setting, which has a good reference value, and I hope to help you. Let's take a look
2020-02-02
Resolve display problem of map function in Python3

Today, Xiaobian will share a solution to the Python3 map function display problem, with a good reference value, I hope to help you. Let's take a look
2019-12-12
Example of map(), a function commonly used in Python

This article mainly introduces the relevant information about the use of map() function, which is very commonly used in Python. map() function is a built-in function of Python. It maps the incoming sequence data according to the function parameters provided
2023-10-10
python regular expressions match instances with arbitrary characters in the middle of []

Today, Xiaobian will share an example of python regular expression matching [] for any character in the middle, which has a good reference value, and I hope it will be helpful to you. Let's take a look
2018-12-12
python automatically send test report email function

This article mainly introduces the realization of python automatic test report email function, the article introduces very detailed through the example code, for everyone's study or work has a certain reference learning value, the need for friends to learn together with the small series
2019-01-01