Build a Neural Network From Scratch in PyTorch Without nn.Module

Sat, 08 Aug 2026 12:59:00 +0100

PyTorch: Zero to Hero — Step 03

Most PyTorch tutorials begin with something like this:

import torch.nn as nn

model = nn.Sequential(
    nn.Linear(2, 16),
    nn.ReLU(),
    nn.Linear(16, 2),
)

That is useful code.

It is also hiding almost everything interesting.

In this post we are going to build the same kind of neural network using ordinary PyTorch tensors.

No nn.Module.

No nn.Linear.

No torch.optim.Adam.

No optimizer.step().

We will manually create the parameters, write the forward pass, calculate the loss, call autograd, update the weights, zero the gradients, batch the data, evaluate the model and then compare the result with the idiomatic PyTorch version.

PyTorch Autograd Debugging: requires_grad, detach, backward() and NaN Gradients

Sat, 08 Aug 2026 12:55:00 +0100

PyTorch: Zero to Hero — Step 02

In the previous post we treated tensor shapes as a debugging problem rather than a mathematical vocabulary exercise.

We are going to do the same thing with autograd.

If you use PyTorch for any serious amount of time, you eventually see errors like:

RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

or:

RuntimeError: Trying to backward through the graph a second time...

or worse:

Backpropagation on Programmer.ie: Modern AI programming

Build a Neural Network From Scratch in PyTorch Without nn.Module

PyTorch: Zero to Hero — Step 03

PyTorch Autograd Debugging: requires_grad, detach, backward() and NaN Gradients

PyTorch: Zero to Hero — Step 02