A Beginner’s Guide to PyTorch

Image by Editor | Midjourney & Canva

Deep Learning is widely used in many areas of Artificial Intelligence research and has contributed to technological advancements. For example, text generation, facial recognition, and voice synthesis applications are based on deep learning research.

One of the most used Deep Learning packages is PyTorch. It is an open-source package created by Meta AI in 2016 and has since been used by many.
There are a lot of PyTorch advantages, including:

• Flexible model architecture
• Native Support for CUDA (Can use GPU)
• Python-based
• Providing lower-level controls, which are useful for research and many use cases
• Active development by the developer and community

Let’s explore PyTorch with this article to help you get started.

Preparation

 
You should visit their installation webpage and select the one that suits your environment’s requirements. The below code is the installation example.

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu

With the PyTorch ready, let’s get into the central part.
 

PyTorch Tensor

 
Tensor is the building block in PyTorch. It is similar to the NumPy array but has access to a GPU. We can try to create a PyTorch Tensor using the following code:

a = torch.tensor([2, 4, 5])
print(a)

Output>> 
tensor([2, 4, 5])

Like the NumPy array Tensor, it allows matrix operations.

e = torch.tensor([[1, 2, 3],
                [4, 5, 6]])
f = torch.tensor([7, 8, 9])

print(e * f)

Output>>
tensor([[ 7, 16, 27],
        [28, 40, 54]])

It’s also possible to perform the matrix multiplication.

g = torch.randn(2, 3)
h = torch.randn(3, 2)
print( g @ h)

Output>> 
tensor([[-0.8357,  0.0583],
        [-2.7121,  2.1980]])

We can access the Tensor information by using the code below.

x = torch.rand(3,4)

print("Shape:", x.shape)
print("Data type:", x.dtype)
print("Device:", x.device)

Output>>
Shape: torch.Size([3, 4])
Data type: torch.float32
Device: cpu

Neural Network Training with PyTorch

By defining the NN using the nn.Module class, we can develop a simple model. Let’s try it out with the code below.

import torch

class SimpleNet(nn.Module):
    def __init__(self, input, hidden, output):
        super(SimpleNet, self).__init__()
        self.fc1 = torch.nn.Linear(input, hidden)
        self.fc2 = torch.nn.Linear(hidden, output)
    
    def forward(self, x):
        x = torch.nn.functional.relu(self.fc1(x))
        x = self.fc2(x)
        return x

inp = 10
hid = 10
outp = 2
model = SimpleNet(inp, hid, out)

print(model)

Output>>
SimpleNet(
  (fc1): Linear(in_features=10, out_features=10, bias=True)
  (fc2): Linear(in_features=10, out_features=2, bias=True)
)

The above code defines a SimpleNet class that inherits from nn.Module, which sets up the layers. We use nn.Linear is for the layers, and relu as the activation function.

We can add more layers or use different layers like Conv2D or CNN. But we would not use that.

Next, we would train the SimpleNet we developed with sample Tensor data.

import torch

inp = torch.randn(100, 10) 
tar = torch.randint(0, 2, (100,)) 
criterion = torch.nn.CrossEntropyLoss()
optimizr = torch.optim.SGD(model.parameters(), lr=0.01)

epochs = 100
batchsize = 10

for epoch in range(numepochs):
    model.train()
    
    for i in range(0, inp.size(0), batchsize):
        batch_inp = inputs[i:i+batch_size]
        batch_tar = targets[i:i+batch_size]
        
        out = model(batch_inp)
        loss = criterion(out, batch_tar)
     
        optimizer.zero_grad()
        loss.backward()
        optimizr.step()
    
    if (epoch + 1) % 10 == 0:
        print(f'Epoch [epoch+1/num_epochs], Loss: round(loss.item(),4)')

During the training above, we use random Tensor data and initiate the loss function called CrossEntropyLoss. Also, we initiate the SGD optimizer to manage the model parameters to minimize the loss.

The training process runs multiple times according to the epoch numbers and then performs the optimization process. This is the usual deep-learning process.

We can add several steps to more complex training to improve training, like early stopping, learning rate, and other techniques.

Lastly, we can evaluate the model we have trained with the unseen data. The following code allows us to do that.

from sklearn.metrics import classification_report

model.eval()
test_inputs = torch.randn(20, 10)
test_targets = torch.randint(0, 2, (20,))

with torch.no_grad():
    test_outputs = model(test_inputs)
    _, predicted = torch.max(test_outputs, 1)

print(classification_report(test_targets, predicted))

What happened above is that we switched the model into the evaluation mode, which turned off dropout and batch normalization updates. Additionally, we disable the gradient computation process to speed up the process.

You can visit the PyTorch documentation to learn further about what you can do.
 

Conclusion

 
In this article, we will go through the basics of PyTorch. From tensor creation to tensor operations and developing a simple NN model. The article is an introductory level that every beginner should be able to follow quickly.

Cornellius Yudha Wijaya is a data science assistant manager and data writer. While working full-time at Allianz Indonesia, he loves to share Python and data tips via social media and writing media. Cornellius writes on a variety of AI and machine learning topics.