1We use alexnet as an example
2
3### prepare caffe prototxt and model
4
5These files will usually generated when trained with caffe
6```
7train.prototxt
8deploy.prototxt
9snapshot_10000.caffemodel
10```
11deploy.prototxt and caffemodel file are enough for TEST phase
12
13alexnet deploy.prototxt can be downloaded here
14
15https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet
16
17alexnet caffemodel can be downloaded here
18
19http://dl.caffe.berkeleyvision.org/bvlc_alexnet.caffemodel
20
21### convert to ncnn model
22
23Convert old caffe prototxt and caffemodel to new ones using tools in caffe
24
25because the ncnn convert tool needs the new format
26```
27upgrade_net_proto_text [old prototxt] [new prototxt]
28upgrade_net_proto_binary [old caffemodel] [new caffemodel]
29```
30
31Use Input layer as input, set N dim as 1 since only one image can be processed each time
32```
33layer {
34  name: "data"
35  type: "Input"
36  top: "data"
37  input_param { shape: { dim: 1 dim: 3 dim: 227 dim: 227 } }
38}
39```
40Use caffe2ncnn tool to convert caffe model to ncnn model
41```
42caffe2ncnn deploy.prototxt bvlc_alexnet.caffemodel alexnet.param alexnet.bin
43```
44
45### strip visible string
46
47It is already enough for deploying with param and bin file only, but there are visible strings in param file, it may not be suitable to distrubute plain neural network information in your APP.
48
49You can use ncnn2mem tool to convert plain model file to binary representation. It will generate alexnet.param.bin and two static array code files.
50```
51ncnn2mem alexnet.param alexnet.bin alexnet.id.h alexnet.mem.h
52```
53
54### load model
55
56Load param and bin file, the easy way
57```cpp
58ncnn::Net net;
59net.load_param("alexnet.param");
60net.load_model("alexnet.bin");
61```
62Load binary param.bin and bin file, no visible strings included, suitable for bundled as APP resource
63```cpp
64ncnn::Net net;
65net.load_param_bin("alexnet.param.bin");
66net.load_model("alexnet.bin");
67```
68Load network and model from external memory, no visible strings included, no external resource files bundled, the whole model is hardcoded in your program
69
70You may use this way to load from android asset resource
71```cpp
72#include "alexnet.mem.h"
73ncnn::Net net;
74net.load_param(alexnet_param_bin);
75net.load_model(alexnet_bin);
76```
77You can choose either way to load model. Loading from external memory is zero-copy, which means you must keep your memory buffer during processing
78
79### unload model
80```cpp
81net.clear();
82```
83
84### input and output
85
86ncnn Mat is the data structure for input and output data
87
88Input image should be converted to Mat, and substracted mean values and normalized when needed
89
90```cpp
91#include "mat.h"
92unsigned char* rgbdata;// data pointer to RGB image pixels
93int w;// image width
94int h;// image height
95ncnn::Mat in = ncnn::Mat::from_pixels(rgbdata, ncnn::Mat::PIXEL_RGB, w, h);
96
97const float mean_vals[3] = {104.f, 117.f, 123.f};
98in.substract_mean_normalize(mean_vals, 0);
99```
100Execute the network inference and retrieve the result
101```cpp
102#include "net.h"
103ncnn::Mat in;// input blob as above
104ncnn::Mat out;
105ncnn::Extractor ex = net.create_extractor();
106ex.set_light_mode(true);
107ex.input("data", in);
108ex.extract("prob", out);
109```
110If you load model with binary param.bin file, you should use the enum value in alexnet.id.h file instead of the blob name
111```cpp
112#include "net.h"
113#include "alexnet.id.h"
114ncnn::Mat in;// input blob as above
115ncnn::Mat out;
116ncnn::Extractor ex = net.create_extractor();
117ex.set_light_mode(true);
118ex.input(alexnet_param_id::BLOB_data, in);
119ex.extract(alexnet_param_id::BLOB_prob, out);
120```
121Read the data in the output Mat. Iterate data to get all classification scores.
122```cpp
123ncnn::Mat out_flatterned = out.reshape(out.w * out.h * out.c);
124std::vector<float> scores;
125scores.resize(out_flatterned.w);
126for (int j=0; j<out_flatterned.w; j++)
127{
128    scores[j] = out_flatterned[j];
129}
130```
131
132### some tricks
133
134Set multithreading thread number with Extractor
135```cpp
136ex.set_num_threads(4);
137```
138Convert image colorspace and resize image with Mat convenient function, these functions are well optimized
139
140Support RGB2GRAY GRAY2RGB RGB2BGR etc, support scale up and scale down
141```cpp
142#include "mat.h"
143unsigned char* rgbdata;// data pointer to RGB image pixels
144int w;// image width
145int h;// image height
146int target_width = 227;// target resized width
147int target_height = 227;// target resized height
148ncnn::Mat in = ncnn::Mat::from_pixels_resize(rgbdata, ncnn::Mat::PIXEL_RGB2GRAY, w, h, target_width, target_height);
149```
150You can concat multiple model files into one, and load this single file from FILE* interface.
151
152It should ease the distribution of param and model files.
153
154> $ cat alexnet.param.bin alexnet.bin > alexnet-all.bin
155
156```cpp
157#include "net.h"
158FILE* fp = fopen("alexnet-all.bin", "rb");
159net.load_param_bin(fp);
160net.load_model(fp);
161fclose(fp);
162```
163