Chapter 28
04. PyTorch Custom Datasets Exercises Template
NotebookPython 346 cells
04. PyTorch Custom Datasets Exercises Template
Welcome to the 04. PyTorch Custom Datasets exercise template.
The best way to practice PyTorch code is to write more PyTorch code.
So read the original notebook and try to complete the exercises by writing code where it's required.
Feel free to reference the original resources whenever you need but should practice writing all of the code yourself.
Resources
- These exercises/solutions are based on notebook 04 of the Learn PyTorch for Deep Learning course.
- See a live walkthrough of the solutions (errors and all) on YouTube.
- See other solutions on the course GitHub.
In [1]python · cell 3
python
# Check for GPU
!nvidia-smiOutput
Mon Apr 18 22:14:23 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.32.03 Driver Version: 460.32.03 CUDA Version: 11.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla P100-PCIE... Off | 00000000:00:04.0 Off | 0 |
| N/A 37C P0 28W / 250W | 0MiB / 16280MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
In [2]python · cell 4
python
# Import torch
import torch
from torch import nn
# Exercises require PyTorch > 1.10.0
print(torch.__version__)
# Setup device agnostic code
device = "cuda" if torch.cuda.is_available() else "cpu"
deviceOutput
1.10.0+cu111
'cuda'
1. Our models are underperforming (not fitting the data well). What are 3 methods for preventing underfitting? Write them down and explain each with a sentence.
2. Recreate the data loading functions we built in sections 1, 2, 3 and 4 of notebook 04. You should have train and test DataLoader's ready to use.
In [3]python · cell 8
python
# 1. Get dataIn [4]python · cell 9
python
# 2. Become one with the data
import os
def walk_through_dir(dir_path):
"""Walks through dir_path returning file counts of its contents."""
for dirpath, dirnames, filenames in os.walk(dir_path):
print(f"There are {len(dirnames)} directories and {len(filenames)} images in '{dirpath}'.")In [5]python · cell 10
python
# Setup train and testing pathsIn [6]python · cell 11
python
# Visualize an imageIn [7]python · cell 12
python
# Do the image visualization with matplotlibWe've got some images in our folders.
Now we need to make them compatible with PyTorch by:
- Transform the data into tensors.
- Turn the tensor data into a
torch.utils.data.Datasetand later atorch.utils.data.DataLoader.
In [8]python · cell 14
python
# 3.1 Transforming data with torchvision.transformsIn [9]python · cell 15
python
# Write transform for turning images into tensorsIn [10]python · cell 16
python
# Write a function to plot transformed imagesLoad image data using ImageFolder
In [11]python · cell 18
python
# Use ImageFolder to create dataset(s)In [ ]python · cell 19
python
# Get class names as a list
class_names = train_data.classes
class_namesOutput
['pizza', 'steak', 'sushi']
In [ ]python · cell 20
python
# Can also get class names as a dict
class_dict = train_data.class_to_idx
class_dictOutput
{'pizza': 0, 'steak': 1, 'sushi': 2}In [ ]python · cell 21
python
# Check the lengths of each dataset
len(train_data), len(test_data)Output
(225, 75)
In [ ]python · cell 22
python
# Turn train and test Datasets into DataLoadersOutput
(<torch.utils.data.dataloader.DataLoader at 0x7fce57ec08d0>, <torch.utils.data.dataloader.DataLoader at 0x7fce57ec0dd0>)
In [ ]python · cell 23
python
# How many batches of images are in our data loaders?Output
(225, 75)
3. Recreate model_0 we built in section 7 of notebook 04.
In [ ]python · cell 25
python
4. Create training and testing functions for model_0.
In [ ]python · cell 27
python
def train_step(model: torch.nn.Module,
dataloader: torch.utils.data.DataLoader,
loss_fn: torch.nn.Module,
optimizer: torch.optim.Optimizer):
# Put the model in train mode
model.train()
# Setup train loss and train accuracy values
train_loss, train_acc = 0, 0
# Loop through data loader and data batches
# Send data to target device
# 1. Forward pass
# 2. Calculate and accumulate loss
# 3. Optimizer zero grad
# 4. Loss backward
# 5. Optimizer step
# Calculate and accumualte accuracy metric across all batches
# Adjust metrics to get average loss and average accuracy per batch
In [ ]python · cell 28
python
def test_step(model: torch.nn.Module,
dataloader: torch.utils.data.DataLoader,
loss_fn: torch.nn.Module):
# Put model in eval mode
model.eval()
# Setup the test loss and test accuracy values
test_loss, test_acc = 0, 0
# Turn on inference context manager
# Loop through DataLoader batches
# Send data to target device
# 1. Forward pass
# 2. Calculuate and accumulate loss
# Calculate and accumulate accuracy
# Adjust metrics to get average loss and accuracy per batchIn [ ]python · cell 29
python
from tqdm.auto import tqdm
def train(model: torch.nn.Module,
train_dataloader: torch.utils.data.DataLoader,
test_dataloader: torch.utils.data.DataLoader,
optimizer: torch.optim.Optimizer,
loss_fn: torch.nn.Module = nn.CrossEntropyLoss(),
epochs: int = 5):
# Create results dictionary
results = {"train_loss": [],
"train_acc": [],
"test_loss": [],
"test_acc": []}
# Loop through the training and testing steps for a number of epochs
for epoch in tqdm(range(epochs)):
# Train step
train_loss, train_acc = train_step(model=model,
dataloader=train_dataloader,
loss_fn=loss_fn,
optimizer=optimizer)
# Test step
test_loss, test_acc = test_step(model=model,
dataloader=test_dataloader,
loss_fn=loss_fn)
# Print out what's happening
print(f"Epoch: {epoch+1} | "
f"train_loss: {train_loss:.4f} | "
f"train_acc: {train_acc:.4f} | "
f"test_loss: {test_loss:.4f} | "
f"test_acc: {test_acc:.4f}"
)
# Update the results dictionary
results["train_loss"].append(train_loss)
results["train_acc"].append(train_acc)
results["test_loss"].append(test_loss)
results["test_acc"].append(test_acc)
# Return the results dictionary
return results5. Try training the model you made in exercise 3 for 5, 20 and 50 epochs, what happens to the results?
- Use
torch.optim.Adam()with a learning rate of 0.001 as the optimizer.
In [ ]python · cell 31
python
# Train for 5 epochs
torch.manual_seed(42)
torch.cuda.manual_seed(42)
loss_fn = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(#TODO,
lr=0.001)In [ ]python · cell 32
python
# Train for 20 epochs
torch.manual_seed(42)
torch.cuda.manual_seed(42)
loss_fn = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(#TODO,
lr=0.001)In [ ]python · cell 33
python
# Train for 50 epochs
torch.manual_seed(42)
torch.cuda.manual_seed(42)
loss_fn = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(#TODO,
lr=0.001)It looks like our model is starting to overfit towards the end (performing far better on the training data than on the testing data).
In order to fix this, we'd have to introduce ways of preventing overfitting.
6. Double the number of hidden units in your model and train it for 20 epochs, what happens to the results?
In [ ]python · cell 36
python
# Double the number of hidden units and train for 20 epochs
torch.manual_seed(42)
torch.cuda.manual_seed(42)It looks like the model is still overfitting, even when changing the number of hidden units.
To fix this, we'd have to look at ways to prevent overfitting with our model.
7. Double the data you're using with your model from step 6 and train it for 20 epochs, what happens to the results?
- Note: You can use the custom data creation notebook to scale up your Food101 dataset.
- You can also find the already formatted double data (20% instead of 10% subset) dataset on GitHub, you will need to write download code like in exercise 2 to get it into this notebook.
In [ ]python · cell 39
python
# Download 20% data for Pizza/Steak/Sushi from GitHub
import requests
import zipfile
from pathlib import Path
# Setup path to data folder
data_path = Path("data/")
image_path = data_path / "pizza_steak_sushi_20_percent"
# If the image folder doesn't exist, download it and prepare it...
if image_path.is_dir():
print(f"{image_path} directory exists.")
else:
print(f"Did not find {image_path} directory, creating one...")
image_path.mkdir(parents=True, exist_ok=True)
# Download pizza, steak, sushi data
with open(data_path / "pizza_steak_sushi_20_percent.zip", "wb") as f:
request = requests.get("https://github.com/mrdbourke/pytorch-deep-learning/raw/main/data/pizza_steak_sushi_20_percent.zip")
print("Downloading pizza, steak, sushi 20% data...")
f.write(request.content)
# Unzip pizza, steak, sushi data
with zipfile.ZipFile(data_path / "pizza_steak_sushi_20_percent.zip", "r") as zip_ref:
print("Unzipping pizza, steak, sushi 20% data...")
zip_ref.extractall(image_path)In [ ]python · cell 40
python
# See how many images we have
walk_through_dir(image_path)Excellent, we now have double the training and testing images...
In [ ]python · cell 42
python
# Create the train and test paths
train_data_20_percent_path = image_path / "train"
test_data_20_percent_path = image_path / "test"
train_data_20_percent_path, test_data_20_percent_pathIn [ ]python · cell 43
python
# Turn the 20 percent datapaths into Datasets and DataLoaders
from torchvision.datasets import ImageFolder
from torchvision import transforms
from torch.utils.data import DataLoader
simple_transform = transforms.Compose([
transforms.Resize((64, 64)),
transforms.ToTensor()
])
# Create datasets
# Create dataloadersIn [ ]python · cell 44
python
# Train a model with increased amount of data
torch.manual_seed(42)
torch.cuda.manual_seed(42)8. Make a prediction on your own custom image of pizza/steak/sushi (you could even download one from the internet) with your trained model from exercise 7 and share your prediction.
- Does the model you trained in exercise 7 get it right?
- If not, what do you think you could do to improve it?
In [ ]python · cell 46
python