PyTorch Tutorial for Beginners: Training Neural Networks Step by Step

A PyTorch tutorial for beginners usually means one thing: you want to train a neural network yourself, not just stare at a diagram of one. PyTorch is a good first framework because its workflow is explicit. You see tensors, batches, model layers, gradients, loss values, and parameter updates in code. That visibility matters when you are trying to understand what actually happens during training.

The official PyTorch beginner tutorials, including the neural network and model-building guides in the PyTorch documentation, teach the same pattern used in real projects: prepare data, define an nn.Module, choose a loss function, run a training loop, evaluate, then save the model. This guide follows that path with practical notes that save beginners hours of debugging.

What You Need Before Starting

You can run PyTorch locally with Python 3.10 or newer, but beginners often move faster in Google Colab. Colab usually ships with PyTorch already installed, and you can switch to a GPU runtime under Runtime then Change runtime type. For small datasets like MNIST or Fashion-MNIST, CPU is fine. For convolutional networks or transformers, use a GPU.

The basic imports are short:

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader
from torchvision import datasets, transforms

Those five lines cover tensors, neural network layers, optimizers, batch loading, and common vision datasets.

The PyTorch Training Workflow

A clean PyTorch training loop has six steps. Learn them in this order. Do not jump to complex models until this feels routine.

1. Load and transform the dataset.
2. Create DataLoader objects for mini-batches.
3. Define the model by subclassing nn.Module.
4. Select a loss function and optimizer.
5. Train with a forward pass, backward pass, and update.
6. Evaluate on data the model did not train on.

That is the core of training neural networks in PyTorch. Transformers, CNNs, and tabular classifiers all follow the same structure.

Step 1: Prepare the Dataset

For a beginner-friendly image classification task, Fashion-MNIST beats plain MNIST. Shirts, sneakers, and coats are slightly less toy-like than handwritten digits, and the dataset still downloads quickly through torchvision.

transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.5,), (0.5,))
])

train_data = datasets.FashionMNIST(
    root='data', train=True, download=True, transform=transform
)

test_data = datasets.FashionMNIST(
    root='data', train=False, download=True, transform=transform
)

train_loader = DataLoader(train_data, batch_size=64, shuffle=True)
test_loader = DataLoader(test_data, batch_size=64, shuffle=False)

shuffle=True matters for training because the model should not see examples in a fixed order every epoch. For test data, keep it false unless you have a reason.

Step 2: Build a Neural Network with nn.Module

The standard PyTorch pattern is a class that inherits from nn.Module. Define layers in __init__. Define the data flow in forward.

class NeuralNetwork(nn.Module):
    def __init__(self):
        super().__init__()
        self.flatten = nn.Flatten()
        self.network = nn.Sequential(
            nn.Linear(28 * 28, 128),
            nn.ReLU(),
            nn.Linear(128, 64),
            nn.ReLU(),
            nn.Linear(64, 10)
        )

    def forward(self, x):
        x = self.flatten(x)
        return self.network(x)

model = NeuralNetwork()

Notice the final layer returns 10 raw scores, one per class. Do not add Softmax before nn.CrossEntropyLoss. This is a common beginner mistake. CrossEntropyLoss expects raw logits and applies the right math internally.

Here is a bug you will probably hit at least once:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (64x784 and 28x128)

That usually means your first Linear layer has the wrong input size, or you forgot to flatten the image. Fashion-MNIST images are 28 by 28 pixels, so the flattened input size is 784.

Step 3: Choose Loss Function and Optimizer

For multi-class classification, use nn.CrossEntropyLoss. For regression, use nn.MSELoss. For binary classification, nn.BCEWithLogitsLoss is usually safer than applying sigmoid manually and then using BCELoss.

criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

Adam with a learning rate of 0.001 is a sensible default for this beginner model. To be blunt, lr=0.1 with Adam is usually asking for trouble. You may see loss bounce around or turn into nan. With plain SGD, larger learning rates can work, but you need to test carefully.

Step 4: Write the PyTorch Training Loop

The training loop is where PyTorch feels different from higher-level tools. You explicitly control the forward pass, gradient calculation, and parameter update.

epochs = 5

for epoch in range(epochs):
    model.train()
    running_loss = 0.0

    for images, labels in train_loader:
        optimizer.zero_grad()
        outputs = model(images)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        running_loss += loss.item()

    avg_loss = running_loss / len(train_loader)
    print(f'Epoch {epoch + 1}, loss: {avg_loss:.4f}')

Three lines deserve special attention:

• optimizer.zero_grad(): PyTorch accumulates gradients by default. Forget this and each batch adds its gradients on top of the previous batch.
• loss.backward(): computes gradients for model parameters.
• optimizer.step(): updates weights using those gradients.

Recent PyTorch code often uses optimizer.zero_grad(set_to_none=True) for memory behavior, but plain zero_grad() is clearer when you are learning. Use the simpler version first.

Step 5: Evaluate the Model Correctly

Training accuracy is not enough. You need a separate test set. Switch the model to evaluation mode with model.eval(). This changes behavior for layers such as dropout and batch normalization.

model.eval()
correct = 0
total = 0

with torch.no_grad():
    for images, labels in test_loader:
        outputs = model(images)
        _, predicted = torch.max(outputs, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

accuracy = 100 * correct / total
print(f'Test accuracy: {accuracy:.2f}%')

torch.no_grad() tells PyTorch not to build a computation graph during evaluation. That saves memory and speeds up inference. Forgetting it will not usually break your code, but it is wasteful.

Step 6: Save and Load the Model

Save the model weights, not the entire Python object, unless you have a specific reason. The state_dict approach is the common production-friendly habit.

torch.save(model.state_dict(), 'fashion_mnist_model.pth')

loaded_model = NeuralNetwork()
loaded_model.load_state_dict(torch.load('fashion_mnist_model.pth'))
loaded_model.eval()

If you trained on GPU and load on CPU, pass map_location='cpu' inside torch.load. That small detail prevents annoying device errors during demos and deployment tests.

Common Beginner Mistakes in PyTorch

Forgetting model modes

Use model.train() during training and model.eval() during validation or testing. This is not optional when your model uses dropout or batch normalization.

Using the wrong label shape

CrossEntropyLoss expects class indices such as 0, 1, or 9, not one-hot vectors. If your labels are one-hot encoded, convert them before training.

Adding Softmax too early

For multi-class classification with CrossEntropyLoss, pass raw logits. Add softmax only when you need probabilities for display or downstream decision logic.

Ignoring device placement

If you use a GPU, move both the model and each batch to the same device. A typical failure reads Expected all tensors to be on the same device. Put device handling in one place and keep it consistent.

Where PyTorch Fits in a Machine Learning Learning Path

A PyTorch tutorial for beginners is a strong next step once you understand Python, NumPy, basic statistics, and supervised learning. If your goal is enterprise AI work, do not stop at notebook examples. Learn experiment tracking, data versioning, model evaluation, and deployment basics.

For structured study, this material connects naturally to Global Tech Council resources in machine learning, artificial intelligence, data science, and programming. If you are preparing for certification, practice writing the training loop from memory. Candidates often understand the concept but mix up the order of zero_grad, backward, and step under exam pressure.

Practical Next Step

Run the Fashion-MNIST example, then make two changes: replace Adam with SGD, and test the learning rate across 0.1, 0.01, and 0.001. Watch the loss curve. That single experiment teaches more about training neural networks than another hour of passive reading. After that, move to a small convolutional neural network and tie your work to a structured Global Tech Council machine learning or AI certification path.