Chapter 30
06. PyTorch Transfer Learning Exercises
NotebookPython 344 cells
06. PyTorch Transfer Learning Exercises
Welcome to the 06. PyTorch Transfer Learning exercise template notebook.
There are several questions in this notebook and it's your goal to answer them by writing Python and PyTorch code.
Note: There may be more than one solution to each of the exercises, don't worry too much about the exact right answer. Try to write some code that works first and then improve it if you can.
Resources and solutions
- These exercises/solutions are based on section 06. PyTorch Transfer Learning of the Learn PyTorch for Deep Learning course by Zero to Mastery.
Solutions:
Try to complete the code below before looking at these.
- See a live walkthrough of the solutions (errors and all) on YouTube.
- See an example solutions notebook for these exercises on GitHub.
1. Make predictions on the entire test dataset and plot a confusion matrix for the results of our model compared to the truth labels.
- Note: You will need to get the dataset and the trained model/retrain the model from notebook 06 to perform predictions.
- Check out 03. PyTorch Computer Vision section 10 for ideas.
In [ ]python · cell 4
python
# Import required libraries/code
import torch
import torchvision
import numpy as np
import matplotlib.pyplot as plt
from torch import nn
from torchvision import transforms, datasets
# Try to get torchinfo, install it if it doesn't work
try:
from torchinfo import summary
except:
print("[INFO] Couldn't find torchinfo... installing it.")
!pip install -q torchinfo
from torchinfo import summary
# Try to import the going_modular directory, download it from GitHub if it doesn't work
try:
from going_modular.going_modular import data_setup, engine
except:
# Get the going_modular scripts
print("[INFO] Couldn't find going_modular scripts... downloading them from GitHub.")
!git clone https://github.com/mrdbourke/pytorch-deep-learning
!mv pytorch-deep-learning/going_modular .
!rm -rf pytorch-deep-learning
from going_modular.going_modular import data_setup, engineOutput
[INFO] Couldn't find torchinfo... installing it. [INFO] Couldn't find going_modular scripts... downloading them from GitHub. Cloning into 'pytorch-deep-learning'... remote: Enumerating objects: 1708, done.[K remote: Counting objects: 100% (160/160), done.[K remote: Compressing objects: 100% (88/88), done.[K remote: Total 1708 (delta 67), reused 151 (delta 60), pack-reused 1548[K Receiving objects: 100% (1708/1708), 230.85 MiB | 14.36 MiB/s, done. Resolving deltas: 100% (927/927), done. Checking out files: 100% (124/124), done.
In [ ]python · cell 5
python
# Setup device agnostic code
device = "cuda" if torch.cuda.is_available() else "cpu"
deviceOutput
'cuda'
Get data
In [ ]python · cell 7
python
import os
import requests
import zipfile
from pathlib import Path
# Setup path to data folder
data_path = Path("data/")
image_path = data_path / "pizza_steak_sushi"
# If the image folder doesn't exist, download it and prepare it...
if image_path.is_dir():
print(f"{image_path} directory exists.")
else:
print(f"Did not find {image_path} directory, creating one...")
image_path.mkdir(parents=True, exist_ok=True)
# Download pizza, steak, sushi data
with open(data_path / "pizza_steak_sushi.zip", "wb") as f:
request = requests.get("https://github.com/mrdbourke/pytorch-deep-learning/raw/main/data/pizza_steak_sushi.zip")
print("Downloading pizza, steak, sushi data...")
f.write(request.content)
# Unzip pizza, steak, sushi data
with zipfile.ZipFile(data_path / "pizza_steak_sushi.zip", "r") as zip_ref:
print("Unzipping pizza, steak, sushi data...")
zip_ref.extractall(image_path)
# Remove .zip file
os.remove(data_path / "pizza_steak_sushi.zip")
# Setup Dirs
train_dir = image_path / "train"
test_dir = image_path / "test"Output
Did not find data/pizza_steak_sushi directory, creating one... Downloading pizza, steak, sushi data... Unzipping pizza, steak, sushi data...
Prepare data
In [ ]python · cell 9
python
# Create a transforms pipeline
simple_transform = transforms.Compose([
transforms.Resize((224, 224)), # 1. Reshape all images to 224x224 (though some models may require different sizes)
transforms.ToTensor(), # 2. Turn image values to between 0 & 1
transforms.Normalize(mean=[0.485, 0.456, 0.406], # 3. A mean of [0.485, 0.456, 0.406] (across each colour channel)
std=[0.229, 0.224, 0.225]) # 4. A standard deviation of [0.229, 0.224, 0.225] (across each colour channel),
])In [ ]python · cell 10
python
# Create training and testing DataLoader's as well as get a list of class names
train_dataloader, test_dataloader, class_names = data_setup.create_dataloaders(train_dir=train_dir,
test_dir=test_dir,
transform=simple_transform, # resize, convert images to between 0 & 1 and normalize them
batch_size=32) # set mini-batch size to 32
train_dataloader, test_dataloader, class_namesOutput
(<torch.utils.data.dataloader.DataLoader at 0x7f5f520c9bd0>, <torch.utils.data.dataloader.DataLoader at 0x7f5f520c9c90>, ['pizza', 'steak', 'sushi'])
Get and prepare a pretrained model
In [ ]python · cell 12
python
# Setup the model with pretrained weights and send it to the target device
model_0 = torchvision.models.efficientnet_b0(pretrained=True).to(device)
#model_0 # uncomment to output (it's very long)Output
Downloading: "https://download.pytorch.org/models/efficientnet_b0_rwightman-3dd342df.pth" to /root/.cache/torch/hub/checkpoints/efficientnet_b0_rwightman-3dd342df.pth
0%| | 0.00/20.5M [00:00<?, ?B/s]
In [ ]python · cell 13
python
# Freeze all base layers in the "features" section of the model (the feature extractor) by setting requires_grad=False
for param in model_0.features.parameters():
param.requires_grad = FalseIn [ ]python · cell 14
python
# Set the manual seeds
torch.manual_seed(42)
torch.cuda.manual_seed(42)
# Get the length of class_names (one output unit for each class)
output_shape = len(class_names)
# Recreate the classifier layer and seed it to the target device
model_0.classifier = torch.nn.Sequential(
torch.nn.Dropout(p=0.2, inplace=True),
torch.nn.Linear(in_features=1280,
out_features=output_shape, # same number of output units as our number of classes
bias=True)).to(device)Train model
In [ ]python · cell 16
python
# Define loss and optimizer
loss_fn = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model_0.parameters(), lr=0.001)In [ ]python · cell 17
python
# Set the random seeds
torch.manual_seed(42)
torch.cuda.manual_seed(42)
# Start the timer
from timeit import default_timer as timer
start_time = timer()
# Setup training and save the results
model_0_results = engine.train(model=model_0,
train_dataloader=train_dataloader,
test_dataloader=test_dataloader,
optimizer=optimizer,
loss_fn=loss_fn,
epochs=5,
device=device)
# End the timer and print out how long it took
end_time = timer()
print(f"[INFO] Total training time: {end_time-start_time:.3f} seconds")Output
0%| | 0/5 [00:00<?, ?it/s]
Epoch: 1 | train_loss: 1.0894 | train_acc: 0.4492 | test_loss: 0.9214 | test_acc: 0.5085 Epoch: 2 | train_loss: 0.8697 | train_acc: 0.7734 | test_loss: 0.8036 | test_acc: 0.7434 Epoch: 3 | train_loss: 0.7769 | train_acc: 0.7734 | test_loss: 0.7404 | test_acc: 0.7737 Epoch: 4 | train_loss: 0.7244 | train_acc: 0.7422 | test_loss: 0.6488 | test_acc: 0.8864 Epoch: 5 | train_loss: 0.6426 | train_acc: 0.7812 | test_loss: 0.6254 | test_acc: 0.8968 [INFO] Total training time: 31.032 seconds
Make predictions on the entire test dataset with the model
In [ ]python · cell 19
python
# TODOMake a confusion matrix with the test preds and the truth labels
Need the following libraries to make a confusion matrix:
- torchmetrics - https://torchmetrics.readthedocs.io/en/stable/
- mlxtend - http://rasbt.github.io/mlxtend/
In [ ]python · cell 22
python
# See if torchmetrics exists, if not, install it
try:
import torchmetrics, mlxtend
print(f"mlxtend version: {mlxtend.__version__}")
assert int(mlxtend.__version__.split(".")[1]) >= 19, "mlxtend verison should be 0.19.0 or higher"
except:
!pip install -q torchmetrics -U mlxtend # <- Note: If you're using Google Colab, this may require restarting the runtime
import torchmetrics, mlxtend
print(f"mlxtend version: {mlxtend.__version__}")Output
[K |████████████████████████████████| 409 kB 7.5 MB/s [K |████████████████████████████████| 1.3 MB 45.7 MB/s [?25hmlxtend version: 0.19.0
In [ ]python · cell 23
python
# Import mlxtend upgraded version
import mlxtend
print(mlxtend.__version__)
assert int(mlxtend.__version__.split(".")[1]) >= 19 # should be version 0.19.0 or higherOutput
0.19.0
In [ ]python · cell 24
python
# TODO2. Get the "most wrong" of the predictions on the test dataset and plot the 5 "most wrong" images. You can do this by:
- Predicting across all of the test dataset, storing the labels and predicted probabilities.
- Sort the predictions by wrong prediction and then descending predicted probabilities, this will give you the wrong predictions with the highest prediction probabilities, in other words, the "most wrong".
- Plot the top 5 "most wrong" images, why do you think the model got these wrong?
You'll want to:
- Create a DataFrame with sample, label, prediction, pred prob
- Sort DataFrame by correct (does label == prediction)
- Sort DataFrame by pred prob (descending)
- Plot the top 5 "most wrong" image predictions
In [ ]python · cell 26
python
# TODO3. Predict on your own image of pizza/steak/sushi - how does the model go? What happens if you predict on an image that isn't pizza/steak/sushi?
- Here you can get an image from a website like http://www.unsplash.com to try it out or you can upload your own.
In [ ]python · cell 28
python
# TODO: Get an image of pizza/steak/sushiIn [ ]python · cell 29
python
# TODO: Get an image of not pizza/steak/sushi4. Train the model from section 4 in notebook 06 part 3 for longer (10 epochs should do), what happens to the performance?
- See the model in notebook 06 part 3 for reference: https://www.learnpytorch.io/06_pytorch_transfer_learning/#3-getting-a-pretrained-model
In [ ]python · cell 31
python
# TODO: Recreate a new model In [ ]python · cell 32
python
# TODO: Train the model for 10 epochs5. Train the model from section 4 above with more data, say 20% of the images from Food101 of Pizza, Steak and Sushi images.
- You can find the 20% Pizza, Steak, Sushi dataset on the course GitHub. It was created with the notebook
extras/04_custom_data_creation.ipynb.
Get 20% data
In [ ]python · cell 35
python
import os
import requests
import zipfile
from pathlib import Path
# Setup path to data folder
data_path = Path("data/")
image_path = data_path / "pizza_steak_sushi_20_percent"
image_data_zip_path = "pizza_steak_sushi_20_percent.zip"
# If the image folder doesn't exist, download it and prepare it...
if image_path.is_dir():
print(f"{image_path} directory exists.")
else:
print(f"Did not find {image_path} directory, creating one...")
image_path.mkdir(parents=True, exist_ok=True)
# Download pizza, steak, sushi data
with open(data_path / image_data_zip_path, "wb") as f:
request = requests.get("https://github.com/mrdbourke/pytorch-deep-learning/raw/main/data/pizza_steak_sushi_20_percent.zip")
print("Downloading pizza, steak, sushi data...")
f.write(request.content)
# Unzip pizza, steak, sushi data
with zipfile.ZipFile(data_path / image_data_zip_path, "r") as zip_ref:
print("Unzipping pizza, steak, sushi 20% data...")
zip_ref.extractall(image_path)
# Remove .zip file
os.remove(data_path / image_data_zip_path)
# Setup Dirs
train_dir_20_percent = image_path / "train"
test_dir_20_percent = image_path / "test"
train_dir_20_percent, test_dir_20_percentOutput
Did not find data/pizza_steak_sushi_20_percent directory, creating one... Downloading pizza, steak, sushi data... Unzipping pizza, steak, sushi 20% data...
(PosixPath('data/pizza_steak_sushi_20_percent/train'),
PosixPath('data/pizza_steak_sushi_20_percent/test'))Create DataLoaders
In [ ]python · cell 37
python
# Create a transforms pipeline
simple_transform = transforms.Compose([
transforms.Resize((224, 224)), # 1. Reshape all images to 224x224 (though some models may require different sizes)
transforms.ToTensor(), # 2. Turn image values to between 0 & 1
transforms.Normalize(mean=[0.485, 0.456, 0.406], # 3. A mean of [0.485, 0.456, 0.406] (across each colour channel)
std=[0.229, 0.224, 0.225]) # 4. A standard deviation of [0.229, 0.224, 0.225] (across each colour channel),
])In [ ]python · cell 38
python
# Create training and testing DataLoader's as well as get a list of class names
train_dataloader_20_percent, test_dataloader_20_percent, class_names = data_setup.create_dataloaders(train_dir=train_dir_20_percent,
test_dir=test_dir_20_percent,
transform=simple_transform, # resize, convert images to between 0 & 1 and normalize them
batch_size=32) # set mini-batch size to 32
train_dataloader_20_percent, test_dataloader_20_percent, class_namesOutput
(<torch.utils.data.dataloader.DataLoader at 0x7f5ede28e390>, <torch.utils.data.dataloader.DataLoader at 0x7f5ede28e210>, ['pizza', 'steak', 'sushi'])
Get a pretrained model
In [ ]python · cell 40
python
# TODOTrain a model with 20% of the data
In [ ]python · cell 42
python
# TODO6. Try a different model from torchvision.models on the Pizza, Steak, Sushi data, how does this model perform?
- You'll have to change the size of the classifier layer to suit our problem.
- You may want to try an EfficientNet with a higher number than our B0, perhaps
torchvision.models.efficientnet_b2()?- Note: Depending on the model you use you will have to prepare/transform the data in a certain way.
In [ ]python · cell 44
python
# TODO