1We use alexnet as an example 2 3### prepare caffe prototxt and model 4 5These files will usually generated when trained with caffe 6``` 7train.prototxt 8deploy.prototxt 9snapshot_10000.caffemodel 10``` 11deploy.prototxt and caffemodel file are enough for TEST phase 12 13alexnet deploy.prototxt can be downloaded here 14 15https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet 16 17alexnet caffemodel can be downloaded here 18 19http://dl.caffe.berkeleyvision.org/bvlc_alexnet.caffemodel 20 21### convert to ncnn model 22 23Convert old caffe prototxt and caffemodel to new ones using tools in caffe 24 25because the ncnn convert tool needs the new format 26``` 27upgrade_net_proto_text [old prototxt] [new prototxt] 28upgrade_net_proto_binary [old caffemodel] [new caffemodel] 29``` 30 31Use Input layer as input, set N dim as 1 since only one image can be processed each time 32``` 33layer { 34 name: "data" 35 type: "Input" 36 top: "data" 37 input_param { shape: { dim: 1 dim: 3 dim: 227 dim: 227 } } 38} 39``` 40Use caffe2ncnn tool to convert caffe model to ncnn model 41``` 42caffe2ncnn deploy.prototxt bvlc_alexnet.caffemodel alexnet.param alexnet.bin 43``` 44 45### strip visible string 46 47It is already enough for deploying with param and bin file only, but there are visible strings in param file, it may not be suitable to distrubute plain neural network information in your APP. 48 49You can use ncnn2mem tool to convert plain model file to binary representation. It will generate alexnet.param.bin and two static array code files. 50``` 51ncnn2mem alexnet.param alexnet.bin alexnet.id.h alexnet.mem.h 52``` 53 54### load model 55 56Load param and bin file, the easy way 57```cpp 58ncnn::Net net; 59net.load_param("alexnet.param"); 60net.load_model("alexnet.bin"); 61``` 62Load binary param.bin and bin file, no visible strings included, suitable for bundled as APP resource 63```cpp 64ncnn::Net net; 65net.load_param_bin("alexnet.param.bin"); 66net.load_model("alexnet.bin"); 67``` 68Load network and model from external memory, no visible strings included, no external resource files bundled, the whole model is hardcoded in your program 69 70You may use this way to load from android asset resource 71```cpp 72#include "alexnet.mem.h" 73ncnn::Net net; 74net.load_param(alexnet_param_bin); 75net.load_model(alexnet_bin); 76``` 77You can choose either way to load model. Loading from external memory is zero-copy, which means you must keep your memory buffer during processing 78 79### unload model 80```cpp 81net.clear(); 82``` 83 84### input and output 85 86ncnn Mat is the data structure for input and output data 87 88Input image should be converted to Mat, and substracted mean values and normalized when needed 89 90```cpp 91#include "mat.h" 92unsigned char* rgbdata;// data pointer to RGB image pixels 93int w;// image width 94int h;// image height 95ncnn::Mat in = ncnn::Mat::from_pixels(rgbdata, ncnn::Mat::PIXEL_RGB, w, h); 96 97const float mean_vals[3] = {104.f, 117.f, 123.f}; 98in.substract_mean_normalize(mean_vals, 0); 99``` 100Execute the network inference and retrieve the result 101```cpp 102#include "net.h" 103ncnn::Mat in;// input blob as above 104ncnn::Mat out; 105ncnn::Extractor ex = net.create_extractor(); 106ex.set_light_mode(true); 107ex.input("data", in); 108ex.extract("prob", out); 109``` 110If you load model with binary param.bin file, you should use the enum value in alexnet.id.h file instead of the blob name 111```cpp 112#include "net.h" 113#include "alexnet.id.h" 114ncnn::Mat in;// input blob as above 115ncnn::Mat out; 116ncnn::Extractor ex = net.create_extractor(); 117ex.set_light_mode(true); 118ex.input(alexnet_param_id::BLOB_data, in); 119ex.extract(alexnet_param_id::BLOB_prob, out); 120``` 121Read the data in the output Mat. Iterate data to get all classification scores. 122```cpp 123ncnn::Mat out_flatterned = out.reshape(out.w * out.h * out.c); 124std::vector<float> scores; 125scores.resize(out_flatterned.w); 126for (int j=0; j<out_flatterned.w; j++) 127{ 128 scores[j] = out_flatterned[j]; 129} 130``` 131 132### some tricks 133 134Set multithreading thread number with Extractor 135```cpp 136ex.set_num_threads(4); 137``` 138Convert image colorspace and resize image with Mat convenient function, these functions are well optimized 139 140Support RGB2GRAY GRAY2RGB RGB2BGR etc, support scale up and scale down 141```cpp 142#include "mat.h" 143unsigned char* rgbdata;// data pointer to RGB image pixels 144int w;// image width 145int h;// image height 146int target_width = 227;// target resized width 147int target_height = 227;// target resized height 148ncnn::Mat in = ncnn::Mat::from_pixels_resize(rgbdata, ncnn::Mat::PIXEL_RGB2GRAY, w, h, target_width, target_height); 149``` 150You can concat multiple model files into one, and load this single file from FILE* interface. 151 152It should ease the distribution of param and model files. 153 154> $ cat alexnet.param.bin alexnet.bin > alexnet-all.bin 155 156```cpp 157#include "net.h" 158FILE* fp = fopen("alexnet-all.bin", "rb"); 159net.load_param_bin(fp); 160net.load_model(fp); 161fclose(fp); 162``` 163