Source: GiantPandaCV

Editing market platform

Torch.fx is really a good job for PyTorch, because it eliminates the gap between some dynamic diagrams and static diagrams. For example, in the aspect of graph rewriting, torch.fx makes it very easy for PyTorch to do some operator fusion optimization of other static graph frameworks. Torch.fx greatly reduces the difficulty of post-training quantization, perceptual training quantization and AMP, thanks to the fact that we can directly operate this IR on Python layer, so I think this is a good job. Especially algorithm engineer developed by PyTorch, we can now open our minds based on this feature. The selling point of torch.fx is that it uses pure Python language to realize a library, which can capture the calculation diagram of PyTorch program and convert it into an IR, and it is very convenient to transfer this IR. At the same time, it provides Python code functions, which will make the converted IR Codegen legal. I think writing Pass in Eager is as smooth as inserting or deleting a topic in a linked list, and it has been achieved.

The link of PyTorch FX paper is behind the performance mender model on Intel Xeon gold 6138 CPU @ 2.00 GHz. The reasoning speed of post-quantization model based on torch.fx is 3.3 times faster than that of float model. Moreover, the quantization operation based on torch.fx is much simpler than that based on TorchScript IR.

Torch.fx can also be used for Op fusion, and Figure 7 shows the application of Conv+BN fusion based on Torch.fx on ResNet50. On the performance of NVIDIA Tesla V100-SX216GB with CUDA version1.0 and Intel Xeon Gold 6138 CPU @ 2.00 GHz, we can see that the delay on GPU is reduced by about 6%, and the delay on CPU is reduced.

In addition, torch.fx can also be used for FLOPs calculation, memory bandwidth usage analysis, data value size estimation of workload, etc. , analyze the memory and speed of the program when it runs. Torch.fx can also be used for shape inference, DAG visual drawing corresponding to the model, and so on.

Finally, torch.fx also supports ASIC acceleration at runtime (that is, reducing the operators in torch.fx to the corresponding ASIC). Figure 8 below shows the acceleration after reasoning ResNet50 and LearningToPaint based on torch.fx and lowering the operator to TensorRT:

Torch.fx is really a good job for PyTorch, because it eliminates the gap between some dynamic diagrams and static diagrams. For example, in the aspect of graph rewriting, torch.fx makes it very easy for PyTorch to do some operator fusion optimization of other static graph frameworks. Torch.fx greatly reduces the difficulty of post-training quantization, perceptual training quantization and AMP, thanks to the fact that we can directly operate this IR on Python layer, so I think this is a good job. Especially algorithm engineer developed by PyTorch, we can now open our minds based on this feature. I have done QAT work around FX before. If you are interested, please read: Quantitative Perception Training Based on OneFlow: /p/3976505 14.

Finally, to sum up, the selling point of torch.fx is that it implements a library in pure Python language, which can capture the calculation diagram of PyTorch program and convert it into an IR, and it is very convenient to transfer this IR. At the same time, it provides Python code functions, which will make the converted IR Codegen legal. I think writing Pass in Eager is as smooth as inserting or deleting a topic in a linked list, and it has been achieved.

The reading method of Shen Mu's thesis is really scientific, so I like it again at the end of the article.