1# PyTorch README
2
3Currently, we have already implemented both the the PyTorch -> IR part and the IR -> PyTorch part.
4
5Models                   | Caffe | CoreML | CNTK | Keras | MXNet | PyTorch | TensorFlow| Onnx
6:-----------------------:|:-----:|:------:|:----:|:-----:|:-----:|:-------:|:------:|:------:|
7Vgg16                    |   √   |   √    |      |   √   |   √   |    √    | √       | √
8Inception_v3             |   √   |   √    |      |   √   |   √   |    √    | √       | √
9ResNet 50                |   √   |   √    |      |   √   |   √   |    √    | √       | √
10MobileNet V1             |   √   |   √    |      |   √   |   √   |    √    | √       | √
11Tiny-yolo                |       |   √    |      |   √   |   √   |    √    | √       | √
12
13**√** - Correctness tested
14
15**o** - Some difference after conversion
16
17**space** - not tested
18
19
20The PyTorch parser is modified from branch [pytorch](https://github.com/Microsoft/MMdnn/tree/pytorch) , using jit CppOP to build the graph.
21
22Any contribution is welcome.
23
24## Extract PyTorch pre-trained models
25
26You can refer [PyTorch model extractor](https://github.com/Microsoft/MMdnn/blob/master/mmdnn/conversion/examples/pytorch/extractor.py) to extract your pytorch models.
27
28```bash
29$ mmdownload -f pytorch -h
30Support frameworks: ['alexnet', 'densenet121', 'densenet161', 'densenet169', 'densenet201', 'inception_v3', 'resnet101', 'resnet152', 'resnet18', 'resnet34', 'resnet50', 'vgg11', 'vgg11_bn', 'vgg13', 'vgg13_bn', 'vgg16', 'vgg16_bn', 'vgg19', 'vgg19_bn']
31
32$ mmdownload -f pytorch -n resnet101 -o ./
33Downloading: "https://download.pytorch.org/models/resnet101-5d3b4d8f.pth" to /my/home/.torch/models/resnet101-5d3b4d8f.pth
34███████████████████| 102502400/102502400 [00:06<00:00, 15858546.50it/s]
35PyTorch pretrained model is saved as [./imagenet_resnet101.pth].
36
37```
38
39### Convert Pytorch pre-trained models to IR
40You can convert the whole pytorch model to IR structure. Please remember for the generality, we now only take the whole model `pth`, not just the state dict. To be more specific, it is save using `torch.save()` and `torch.load()` can load the whole model.
41
42```bash
43$ mmtoir -f pytorch -d resnet101 --inputShape 3,224,224 -n imagenet_resnet101.pth
44```
45
46Please bear in mind that always add `--inputShape` argparse. This thing is different from other framework because pytorch is a dynamic framework.
47
48Then you will get
49```
50IR network structure is saved as [resnet101.json].
51IR network structure is saved as [resnet101.pb].
52IR weights are saved as [resnet101.npy].
53```
54
55### Convert models from IR to PyTorch code snippet and weights
56
57You can use following bash command to convert the IR architecture file [*inception_v3.pb*] and weights file [*inception_v3.npy*] to Caffe Python code file[*pytorch_inception_v3.py*] and IR weights file suit for caffe model[*pytorch_inception_v3.npy*]
58
59> Note: We need to transform the IR weights to PyTorch suitable weights. Use argument *-dw* to specify the output weight file name.
60
61```bash
62$ mmtocode -f pytorch -n inception_v3.pb --IRWeightPath inception_v3.npy --dstModelPath pytorch_inception_v3.py -dw pytorch_inception_v3.npy
63
64Parse file [inception_v3.pb] with binary format successfully.
65Target network code snippet is saved as [pytorch_inception_v3.py].
66Target weights are saved as [pytorch_inception_v3.npy].
67```
68
69### Generate PyTorch model from code snippet file and weight file
70
71You can use following bash command to generate PyTorch model file [*pytorch_inception_v3.pth*] from python code [*pytorch_inception_v3.py*] and weights file [*pytorch_inception_v3.npy*] for further usage.
72
73```bash
74$ mmtomodel -f pytorch -in pytorch_inception_v3.py -iw pytorch_inception_v3.npy -o pytorch_inception_v3.pth
75
76PyTorch model file is saved as [pytorch_inception_v3.pth], generated by [pytorch_inception_v3.py] and [pytorch_inception_v3.npy]. Notice that you may need [pytorch_inception_v3.py] to load the model back.
77
78```
79
80## Example
81
82Detail scripts of *Tensorflow slim resnet_v1_101 model* to *PyTorch* conversion are in [issue 22](https://github.com/Microsoft/MMdnn/issues/22). You can refer it to implement your conversion.
83
84## Develop version
85
86Ubuntu 16.04 with
87
88- PyTorch 0.4.0
89
90@ 2018/04/25
91
92## Links
93
94- [pytorch to keras converter](https://github.com/nerox8664/pytorch2keras)
95
96## Limitation
97
98- The main dataflow in a pytorch network is converted from NHWC(channel last) to NCHW(channel first) format, but some operators (like Concat) with axis may not transform correctly. You may need to correct it manually.
99
100- Currently, no RNN-related operations supported
101
102## FAQ
103
104- There are two types models saved in PyTorch. One is including architecture and weights, which is supported in the MMdnn now. The other  one is only including the weights, which is not supported now.
105
106```python
107only_weight_file = "./alexnet-owt-4df8aa71.pth"      # Download from the model zoo
108architecture_weight_file = "imagenet_alexnet.pth"    # Download using mmdownload()
109
110m = torch.load(only_weight_file)                    # <class 'collections.OrderedDict'>
111m_1 = torch.load(architecture_weight_file)          # <class 'torchvision.models.alexnet.AlexNet'> supported!
112
113```
114- When you get the error "AttributeError: 'collections.OrderedDict' object has no attribute 'state_dict'" , it's because you use the model only include weights part. You need to save a new model with archietecture
115
116```python
117torch.save(model, filename)
118```
119
120- How to load the converted PyTorch model ?
121
122```python
123
124import torch
125import imp
126import numpy as np
127MainModel = imp.load_source('MainModel', "tf_pytorch_vgg19.py")
128
129the_model = torch.load("tf_pytorch_vgg19.pth")
130the_model.eval()
131
132x = np.random.random([224,224,3])
133x = np.transpose(x, (2, 0, 1))
134x = np.expand_dims(x, 0).copy()
135data = torch.from_numpy(x)
136data = torch.autograd.Variable(data, requires_grad = False).float()
137
138predict = the_model(data)
139
140
141```
142
143
144