Research › Analysis

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Paszke, A. et al. · NeurIPS (2019) · Paper

Q2 Computing analysis. All mathematical results and empirical findings are attributed to the original authors. We present our reading of the work as it relates to robotic automation.

What this paper solves

Deep learning frameworks prior to PyTorch required defining the computation graph statically before execution (define-and-run). This made debugging difficult, dynamic architectures cumbersome, and rapid research iteration slow. PyTorch introduced define-by-run execution: the computation graph is constructed dynamically as operations execute, making the framework behave like standard imperative Python and enabling automatic differentiation through arbitrary control flow.

PyTorch is the implementation substrate for every neural network in this archive. The differentiable physics training in Back to Newton's Laws (#1), the PPO training in Champion-Level Drone Racing (#11), and the gradient-based trajectory optimization in EGO-Planner (#3) and EGO-Swarm (#4) all depend on PyTorch's autograd system to propagate gradients through computation graphs that could not be expressed in a static graph framework.

Key mathematical framework

Reverse-mode automatic differentiation

PyTorch implements reverse-mode automatic differentiation (backpropagation). Every tensor operation records itself in a directed acyclic graph of $Function$ nodes. Given a scalar loss $L$ , the gradient with respect to any parameter $θ$ is computed by the chain rule traversed in reverse topological order through the graph:

\frac{\partial L}{\partial θ} = \frac{\partial L}{\partial y _{n}} \cdot \frac{\partial y _{n}}{\partial y _{n - 1}} \dots \frac{\partial y _{1}}{\partial θ}

Each node stores only its local Jacobian-vector product function (the vector-Jacobian product, or VJP). The backward pass multiplies these products sequentially, accumulating gradients without materializing full Jacobian matrices. This is what makes backpropagation through millions of parameters computationally feasible.

Gradient accumulation and parameter update

Gradients are accumulated on the $.grad$ attribute of each leaf tensor (parameter). An optimizer then applies the update rule. For stochastic gradient descent:

θ \leftarrow θ - η \cdot \nabla_{θ} L

For Adam, which maintains running estimates of first and second gradient moments:

m_{t} = β_{1} m_{t - 1} + (1 - β_{1}) \nabla_{θ} L

v_{t} = β_{2} v_{t - 1} + (1 - β_{2}) (\nabla_{θ} L)^{2}

θ \leftarrow θ - η \cdot \frac{m ^ _{t}}{v ^ _{t} + ϵ}

where $\overset{m}{^}_{t} = m_{t} / (1 - β_{1}^{t})$ and $\overset{v}{^}_{t} = v_{t} / (1 - β_{2}^{t})$ are bias-corrected estimates. PyTorch's autograd computes $\nabla_{θ} L$ for any differentiable computation graph with a single call to $L .backward()$ .

Dynamic graph and differentiable physics

The define-by-run model is what makes differentiable physics simulation possible in PyTorch. In Back to Newton's Laws (#1), the training loop unrolls the physical simulation for $N$ timesteps, with each state transition implemented as a PyTorch operation:

x_{k + 1} = f_{physics} (x_{k}, π_{θ} (o_{k}))

Because $f_{physics}$ is composed of differentiable PyTorch operations (matrix multiplies, elementwise arithmetic, exponentials), the gradient $\partial L / \partial θ$ flows directly through the physics without any additional implementation. A static graph framework would require declaring the entire unrolled trajectory before knowing how many timesteps to simulate.

Empirical results

Performance: Competitive with or exceeding TensorFlow on standard benchmarks while maintaining imperative execution semantics
Adoption: Became the dominant framework in academic robotics and reinforcement learning research within two years of the NeurIPS 2019 publication
Ecosystem: TorchScript and TorchServe enable deployment of PyTorch models to production systems and embedded hardware without the Python runtime

What this means for robotic automation

PyTorch's contribution to this archive is as infrastructure rather than algorithm. Every training loop, every gradient computation, and every model deployment in the research cited here depends on the autograd engine described in this paper. The reason differentiable physics training is feasible at all is that PyTorch allows physics simulation to be expressed as ordinary tensor operations with automatic gradient tracking.

For edge deployment, TorchScript's ability to compile a trained model to a static computation graph that runs without Python is what allows policies trained with dynamic graphs in simulation to execute on embedded hardware like the Jetson TX2 in Swift or the Mango Pi in Back to Newton's Laws. The training and deployment environments use the same model weights but different execution backends.