Chapter 46
06. PyTorch Transfer Learning
NotebookPython 347 cells
06. PyTorch Transfer Learning
What is transfer learning?
Transfer learning involves taking the parameters of what one model has learned on another dataset and applying to our own problem.
- Pretrained model = foundation models
In [ ]python · cell 3
python
import torch
import torchvision
print(torch.__version__) # want 1.12+
print(torchvision.__version__) # want 0.13+Output
1.13.0.dev20220624+cu113 0.14.0.dev20220624+cu113
In [ ]python · cell 4
python
# For this notebook to run with updated APIs, we need torch 1.12+ and torchvision 0.13+
try:
import torch
import torchvision
assert int(torch.__version__.split(".")[1]) >= 12, "torch version should be 1.12+"
assert int(torchvision.__version__.split(".")[1]) >= 13, "torchvision version should be 0.13+"
print(f"torch version: {torch.__version__}")
print(f"torchvision version: {torchvision.__version__}")
except:
print(f"[INFO] torch/torchvision versions not as required, installing nightly versions.")
!pip3 install -U --pre torch torchvision --extra-index-url https://download.pytorch.org/whl/nightly/cu113
import torch
import torchvision
print(f"torch version: {torch.__version__}")
print(f"torchvision version: {torchvision.__version__}")Output
torch version: 1.13.0.dev20220624+cu113 torchvision version: 0.14.0.dev20220624+cu113
Now we've got the versions of torch and torchvision, we're after, let's import the code we've written in previous sections so that we don't have to write it all again.
In [4]python · cell 6
python
# Continue with regular imports
import matplotlib.pyplot as plt
import torch
import torchvision
from torch import nn
from torchvision import transforms
# Try to get torchinfo, install it if it doesn't work
try:
from torchinfo import summary
except:
print("[INFO] Couldn't find torchinfo... installing it.")
!pip install -q torchinfo
from torchinfo import summary
# Try to import the going_modular directory, download it from GitHub if it doesn't work
try:
from going_modular.going_modular import data_setup, engine
except:
# Get the going_modular scripts
print("[INFO] Couldn't find going_modular scripts... downloading them from GitHub.")
!git clone https://github.com/mrdbourke/pytorch-deep-learning
!mv pytorch-deep-learning/going_modular .
!rm -rf pytorch-deep-learning
from going_modular.going_modular import data_setup, engineOutput
[INFO] Couldn't find going_modular scripts... downloading them from GitHub. Cloning into 'pytorch-deep-learning'... remote: Enumerating objects: 1964, done.[K remote: Counting objects: 100% (217/217), done.[K remote: Compressing objects: 100% (111/111), done.[K remote: Total 1964 (delta 96), reused 207 (delta 94), pack-reused 1747[K Receiving objects: 100% (1964/1964), 284.02 MiB | 13.81 MiB/s, done. Resolving deltas: 100% (1084/1084), done. Checking out files: 100% (143/143), done.
In [5]python · cell 7
python
# Setup device agnostic code
device = "cuda" if torch.cuda.is_available() else "cpu"
deviceOutput
'cuda'
1. Get data
We need our pizza, steak, sushi data to build a transfer learning model on.
In [6]python · cell 9
python
import os
import zipfile
from pathlib import Path
import requests
# Setup data path
data_path = Path("data/")
image_path = data_path / "pizza_steak_sushi" # images from a subset of classes from the Food101 dataset
# If the image folder doesn't exist, download it and prepare it...
if image_path.is_dir():
print(f"{image_path} directory exists, skipping re-download.")
else:
print(f"Did not find {image_path}, downloading it...")
image_path.mkdir(parents=True, exist_ok=True)
# Download pizza, steak, sushi data
with open(data_path / "pizza_steak_sushi.zip", "wb") as f:
request = requests.get("https://github.com/mrdbourke/pytorch-deep-learning/raw/main/data/pizza_steak_sushi.zip")
print("Downloading pizza, steak, sushi data...")
f.write(request.content)
# unzip pizza, steak, sushi data
with zipfile.ZipFile(data_path / "pizza_steak_sushi.zip", "r") as zip_ref:
print("Unzipping pizza, steak, sushi data...")
zip_ref.extractall(image_path)
# Remove .zip file
os.remove(data_path / "pizza_steak_sushi.zip")Output
Did not find data/pizza_steak_sushi, downloading it... Downloading pizza, steak, sushi data... Unzipping pizza, steak, sushi data...
In [7]python · cell 10
python
# Setup directory path
train_dir = image_path / "train"
test_dir = image_path / "test"
train_dir, test_dirOutput
(PosixPath('data/pizza_steak_sushi/train'),
PosixPath('data/pizza_steak_sushi/test'))2. Create Datasets and DataLoaders
Now we've got some data, want to turn it into PyTorch DataLoaders.
To do so, we can use data_setup.py and the create_dataloaders() function we made in 05. PyTorch Going Modular.
There's one thing we have to think about when loading: how to transform it?
And with torchvision 0.13+ there's two ways to do this:
- Manually created transforms - you define what transforms you want your data to go through.
- Automatically created transforms - the transforms for your data are defined by the model you'd like to use.
Important point: when using a pretrained model, it's important that the data (including your custom data) that you pass through it is transformed in the same way that the data the model was trained on.
2.1 Creating a transform for torchvision.models (manual creation)
torchvision.models contains pretrained models (models ready for transfer learning) right within torchvision.
All pre-trained models expect input images normalized in the same way, i.e. mini-batches of 3-channel RGB images of shape (3 x H x W), where H and W are expected to be at least 224. The images have to be loaded in to a range of [0, 1] and then normalized using mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225]. You can use the following transform to normalize.
In [8]python · cell 13
python
from torchvision import transforms
normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
manual_transforms = transforms.Compose([
transforms.Resize((224, 224)), # resize image to 224, 224 (height x width)
transforms.ToTensor(), # get images into range [0, 1]
normalize]) # make sure images have the same distribution as ImageNet (where our pretrained models have been trained)In [9]python · cell 14
python
from going_modular.going_modular import data_setup
train_dataloader, test_dataloader, class_names = data_setup.create_dataloaders(train_dir=train_dir,
test_dir=test_dir,
transform=manual_transforms,
batch_size=32)
train_dataloader, test_dataloader, class_namesOutput
(<torch.utils.data.dataloader.DataLoader at 0x7f795b3aa850>, <torch.utils.data.dataloader.DataLoader at 0x7f795b354910>, ['pizza', 'steak', 'sushi'])
2.2 Creating a transform for torchvision.models (auto creation)
As of torchvision v0.13+ there is now support for automatic data transform creation based on the pretrained model weights you're using.
In [10]python · cell 16
python
import torchvision
torchvision.__version__Output
'0.14.0.dev20220624+cu113'
In [11]python · cell 17
python
# Get a set of pretrained model weights
weights = torchvision.models.EfficientNet_B0_Weights.DEFAULT # "DEFAULT" = best available weights
weightsOutput
EfficientNet_B0_Weights.IMAGENET1K_V1
In [12]python · cell 18
python
# Get the transforms used to create our pretrained weights
auto_transforms = weights.transforms()
auto_transformsOutput
ImageClassification(
crop_size=[224]
resize_size=[256]
mean=[0.485, 0.456, 0.406]
std=[0.229, 0.224, 0.225]
interpolation=InterpolationMode.BICUBIC
)In [13]python · cell 19
python
# Create DataLoaders using automatic transforms
train_dataloader, test_dataloader, class_names = data_setup.create_dataloaders(train_dir=train_dir,
test_dir=test_dir,
transform=auto_transforms,
batch_size=32)
train_dataloader, test_dataloader, class_namesOutput
(<torch.utils.data.dataloader.DataLoader at 0x7f795af1ee50>, <torch.utils.data.dataloader.DataLoader at 0x7f795af1ef90>, ['pizza', 'steak', 'sushi'])
3. Getting a pretrained model
There are various places to get a pretrained model, such as:
- PyTorch domain libraries
- Libraries like
timm(torch image models) - HuggingFace Hub (for plenty of different models)
- Paperswithcode (for models across different problem spaces/domains)
3.1 Which pretrained model should you use?
Experiment, experiment, experiment!
The whole idea of transfer learning: take an already well-performing model from a problem space similar to your own and then customize to your own problem.
Three things to consider:
- Speed - how fast does it need to run?
- Size - how big is the model?
- Performance - how well does it go on your chosen problem (e.g. how well does it classify food images? for FoodVision Mini)?
Where does the model live?
Is it on device? (like a self-driving car)
Or does it live on a server?
Looking at https://pytorch.org/vision/main/models.html#table-of-all-available-classification-weights
Which model should we chose?
For our case (deploying FoodVision Mini on a mobile device), it looks like EffNetB0 is one of our best options in terms performance vs size.
However, in light of The Bitter Lesson, if we had infinite compute, we'd likely pick the biggest model + most parameters + most general we could - http://www.incompleteideas.net/IncIdeas/BitterLesson.html
3.2 Setting up a pretrained model
Want to create an instance of a pretrained EffNetB0 - https://pytorch.org/vision/main/models/generated/torchvision.models.efficientnet_b0.html#torchvision.models.EfficientNet_B0_Weights
In [14]python · cell 23
python
# OLD method of creating a pretrained model (prior to torchvision v0.13)
# model = torchvision.models.efficientnet_b0(pretrained=True)
# New method of creating a pretrained model (torchvision v0.13+)
weights = torchvision.models.EfficientNet_B0_Weights.DEFAULT # ".DEFAULT" = best available weights
model = torchvision.models.efficientnet_b0(weights=weights).to(device)
model Output
Downloading: "https://download.pytorch.org/models/efficientnet_b0_rwightman-3dd342df.pth" to /root/.cache/torch/hub/checkpoints/efficientnet_b0_rwightman-3dd342df.pth
0%| | 0.00/20.5M [00:00<?, ?B/s]
EfficientNet(
(features): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(3, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): Sequential(
(0): MBConv(
(block): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=32, bias=False)
(1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): SqueezeExcitation(
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc1): Conv2d(32, 8, kernel_size=(1, 1), stride=(1, 1))
(fc2): Conv2d(8, 32, kernel_size=(1, 1), stride=(1, 1))
(activation): SiLU(inplace=True)
(scale_activation): Sigmoid()
)
(2): Conv2dNormActivation(
(0): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(stochastic_depth): StochasticDepth(p=0.0, mode=row)
)
)
(2): Sequential(
(0): MBConv(
(block): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(16, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): Conv2dNormActivation(
(0): Conv2d(96, 96, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=96, bias=False)
(1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(2): SqueezeExcitation(
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc1): Conv2d(96, 4, kernel_size=(1, 1), stride=(1, 1))
(fc2): Conv2d(4, 96, kernel_size=(1, 1), stride=(1, 1))
(activation): SiLU(inplace=True)
(scale_activation): Sigmoid()
)
(3): Conv2dNormActivation(
(0): Conv2d(96, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(stochastic_depth): StochasticDepth(p=0.0125, mode=row)
)
(1): MBConv(
(block): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(24, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): Conv2dNormActivation(
(0): Conv2d(144, 144, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=144, bias=False)
(1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(2): SqueezeExcitation(
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc1): Conv2d(144, 6, kernel_size=(1, 1), stride=(1, 1))
(fc2): Conv2d(6, 144, kernel_size=(1, 1), stride=(1, 1))
(activation): SiLU(inplace=True)
(scale_activation): Sigmoid()
)
(3): Conv2dNormActivation(
(0): Conv2d(144, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(stochastic_depth): StochasticDepth(p=0.025, mode=row)
)
)
(3): Sequential(
(0): MBConv(
(block): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(24, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): Conv2dNormActivation(
(0): Conv2d(144, 144, kernel_size=(5, 5), stride=(2, 2), padding=(2, 2), groups=144, bias=False)
(1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(2): SqueezeExcitation(
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc1): Conv2d(144, 6, kernel_size=(1, 1), stride=(1, 1))
(fc2): Conv2d(6, 144, kernel_size=(1, 1), stride=(1, 1))
(activation): SiLU(inplace=True)
(scale_activation): Sigmoid()
)
(3): Conv2dNormActivation(
(0): Conv2d(144, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(40, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(stochastic_depth): StochasticDepth(p=0.037500000000000006, mode=row)
)
(1): MBConv(
(block): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(240, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): Conv2dNormActivation(
(0): Conv2d(240, 240, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=240, bias=False)
(1): BatchNorm2d(240, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(2): SqueezeExcitation(
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc1): Conv2d(240, 10, kernel_size=(1, 1), stride=(1, 1))
(fc2): Conv2d(10, 240, kernel_size=(1, 1), stride=(1, 1))
(activation): SiLU(inplace=True)
(scale_activation): Sigmoid()
)
(3): Conv2dNormActivation(
(0): Conv2d(240, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(40, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(stochastic_depth): StochasticDepth(p=0.05, mode=row)
)
)
(4): Sequential(
(0): MBConv(
(block): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(240, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): Conv2dNormActivation(
(0): Conv2d(240, 240, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=240, bias=False)
(1): BatchNorm2d(240, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(2): SqueezeExcitation(
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc1): Conv2d(240, 10, kernel_size=(1, 1), stride=(1, 1))
(fc2): Conv2d(10, 240, kernel_size=(1, 1), stride=(1, 1))
(activation): SiLU(inplace=True)
(scale_activation): Sigmoid()
)
(3): Conv2dNormActivation(
(0): Conv2d(240, 80, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(80, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(stochastic_depth): StochasticDepth(p=0.0625, mode=row)
)
(1): MBConv(
(block): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(80, 480, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): Conv2dNormActivation(
(0): Conv2d(480, 480, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=480, bias=False)
(1): BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(2): SqueezeExcitation(
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc1): Conv2d(480, 20, kernel_size=(1, 1), stride=(1, 1))
(fc2): Conv2d(20, 480, kernel_size=(1, 1), stride=(1, 1))
(activation): SiLU(inplace=True)
(scale_activation): Sigmoid()
)
(3): Conv2dNormActivation(
(0): Conv2d(480, 80, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(80, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(stochastic_depth): StochasticDepth(p=0.07500000000000001, mode=row)
)
(2): MBConv(
(block): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(80, 480, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): Conv2dNormActivation(
(0): Conv2d(480, 480, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=480, bias=False)
(1): BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(2): SqueezeExcitation(
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc1): Conv2d(480, 20, kernel_size=(1, 1), stride=(1, 1))
(fc2): Conv2d(20, 480, kernel_size=(1, 1), stride=(1, 1))
(activation): SiLU(inplace=True)
(scale_activation): Sigmoid()
)
(3): Conv2dNormActivation(
(0): Conv2d(480, 80, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(80, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(stochastic_depth): StochasticDepth(p=0.08750000000000001, mode=row)
)
)
(5): Sequential(
(0): MBConv(
(block): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(80, 480, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): Conv2dNormActivation(
(0): Conv2d(480, 480, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=480, bias=False)
(1): BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(2): SqueezeExcitation(
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc1): Conv2d(480, 20, kernel_size=(1, 1), stride=(1, 1))
(fc2): Conv2d(20, 480, kernel_size=(1, 1), stride=(1, 1))
(activation): SiLU(inplace=True)
(scale_activation): Sigmoid()
)
(3): Conv2dNormActivation(
(0): Conv2d(480, 112, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(stochastic_depth): StochasticDepth(p=0.1, mode=row)
)
(1): MBConv(
(block): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(112, 672, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): Conv2dNormActivation(
(0): Conv2d(672, 672, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=672, bias=False)
(1): BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(2): SqueezeExcitation(
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc1): Conv2d(672, 28, kernel_size=(1, 1), stride=(1, 1))
(fc2): Conv2d(28, 672, kernel_size=(1, 1), stride=(1, 1))
(activation): SiLU(inplace=True)
(scale_activation): Sigmoid()
)
(3): Conv2dNormActivation(
(0): Conv2d(672, 112, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(stochastic_depth): StochasticDepth(p=0.1125, mode=row)
)
(2): MBConv(
(block): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(112, 672, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): Conv2dNormActivation(
(0): Conv2d(672, 672, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=672, bias=False)
(1): BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(2): SqueezeExcitation(
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc1): Conv2d(672, 28, kernel_size=(1, 1), stride=(1, 1))
(fc2): Conv2d(28, 672, kernel_size=(1, 1), stride=(1, 1))
(activation): SiLU(inplace=True)
(scale_activation): Sigmoid()
)
(3): Conv2dNormActivation(
(0): Conv2d(672, 112, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(stochastic_depth): StochasticDepth(p=0.125, mode=row)
)
)
(6): Sequential(
(0): MBConv(
(block): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(112, 672, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): Conv2dNormActivation(
(0): Conv2d(672, 672, kernel_size=(5, 5), stride=(2, 2), padding=(2, 2), groups=672, bias=False)
(1): BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(2): SqueezeExcitation(
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc1): Conv2d(672, 28, kernel_size=(1, 1), stride=(1, 1))
(fc2): Conv2d(28, 672, kernel_size=(1, 1), stride=(1, 1))
(activation): SiLU(inplace=True)
(scale_activation): Sigmoid()
)
(3): Conv2dNormActivation(
(0): Conv2d(672, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(stochastic_depth): StochasticDepth(p=0.1375, mode=row)
)
(1): MBConv(
(block): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(192, 1152, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): Conv2dNormActivation(
(0): Conv2d(1152, 1152, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1152, bias=False)
(1): BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(2): SqueezeExcitation(
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc1): Conv2d(1152, 48, kernel_size=(1, 1), stride=(1, 1))
(fc2): Conv2d(48, 1152, kernel_size=(1, 1), stride=(1, 1))
(activation): SiLU(inplace=True)
(scale_activation): Sigmoid()
)
(3): Conv2dNormActivation(
(0): Conv2d(1152, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(stochastic_depth): StochasticDepth(p=0.15000000000000002, mode=row)
)
(2): MBConv(
(block): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(192, 1152, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): Conv2dNormActivation(
(0): Conv2d(1152, 1152, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1152, bias=False)
(1): BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(2): SqueezeExcitation(
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc1): Conv2d(1152, 48, kernel_size=(1, 1), stride=(1, 1))
(fc2): Conv2d(48, 1152, kernel_size=(1, 1), stride=(1, 1))
(activation): SiLU(inplace=True)
(scale_activation): Sigmoid()
)
(3): Conv2dNormActivation(
(0): Conv2d(1152, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(stochastic_depth): StochasticDepth(p=0.1625, mode=row)
)
(3): MBConv(
(block): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(192, 1152, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): Conv2dNormActivation(
(0): Conv2d(1152, 1152, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1152, bias=False)
(1): BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(2): SqueezeExcitation(
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc1): Conv2d(1152, 48, kernel_size=(1, 1), stride=(1, 1))
(fc2): Conv2d(48, 1152, kernel_size=(1, 1), stride=(1, 1))
(activation): SiLU(inplace=True)
(scale_activation): Sigmoid()
)
(3): Conv2dNormActivation(
(0): Conv2d(1152, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(stochastic_depth): StochasticDepth(p=0.17500000000000002, mode=row)
)
)
(7): Sequential(
(0): MBConv(
(block): Sequential(
(0): Conv2dNormActivation(
(0): Conv2d(192, 1152, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(1): Conv2dNormActivation(
(0): Conv2d(1152, 1152, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=1152, bias=False)
(1): BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
(2): SqueezeExcitation(
(avgpool): AdaptiveAvgPool2d(output_size=1)
(fc1): Conv2d(1152, 48, kernel_size=(1, 1), stride=(1, 1))
(fc2): Conv2d(48, 1152, kernel_size=(1, 1), stride=(1, 1))
(activation): SiLU(inplace=True)
(scale_activation): Sigmoid()
)
(3): Conv2dNormActivation(
(0): Conv2d(1152, 320, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(stochastic_depth): StochasticDepth(p=0.1875, mode=row)
)
)
(8): Conv2dNormActivation(
(0): Conv2d(320, 1280, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(1280, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): SiLU(inplace=True)
)
)
(avgpool): AdaptiveAvgPool2d(output_size=1)
(classifier): Sequential(
(0): Dropout(p=0.2, inplace=True)
(1): Linear(in_features=1280, out_features=1000, bias=True)
)
)In [15]python · cell 24
python
model.classifierOutput
Sequential( (0): Dropout(p=0.2, inplace=True) (1): Linear(in_features=1280, out_features=1000, bias=True) )
3.3 Getting a summary of our model with torchinfo.summary()
In [16]python · cell 26
python
# Print with torchinfo
from torchinfo import summary
summary(model=model,
input_size=(1, 3, 224, 224), # example of [batch_size, color_channels, height, width]
col_names=["input_size", "output_size", "num_params", "trainable"],
col_width=20,
row_settings=["var_names"])Output
============================================================================================================================================ Layer (type (var_name)) Input Shape Output Shape Param # Trainable ============================================================================================================================================ EfficientNet (EfficientNet) [1, 3, 224, 224] [1, 1000] -- True ├─Sequential (features) [1, 3, 224, 224] [1, 1280, 7, 7] -- True │ └─Conv2dNormActivation (0) [1, 3, 224, 224] [1, 32, 112, 112] -- True │ │ └─Conv2d (0) [1, 3, 224, 224] [1, 32, 112, 112] 864 True │ │ └─BatchNorm2d (1) [1, 32, 112, 112] [1, 32, 112, 112] 64 True │ │ └─SiLU (2) [1, 32, 112, 112] [1, 32, 112, 112] -- -- │ └─Sequential (1) [1, 32, 112, 112] [1, 16, 112, 112] -- True │ │ └─MBConv (0) [1, 32, 112, 112] [1, 16, 112, 112] 1,448 True │ └─Sequential (2) [1, 16, 112, 112] [1, 24, 56, 56] -- True │ │ └─MBConv (0) [1, 16, 112, 112] [1, 24, 56, 56] 6,004 True │ │ └─MBConv (1) [1, 24, 56, 56] [1, 24, 56, 56] 10,710 True │ └─Sequential (3) [1, 24, 56, 56] [1, 40, 28, 28] -- True │ │ └─MBConv (0) [1, 24, 56, 56] [1, 40, 28, 28] 15,350 True │ │ └─MBConv (1) [1, 40, 28, 28] [1, 40, 28, 28] 31,290 True │ └─Sequential (4) [1, 40, 28, 28] [1, 80, 14, 14] -- True │ │ └─MBConv (0) [1, 40, 28, 28] [1, 80, 14, 14] 37,130 True │ │ └─MBConv (1) [1, 80, 14, 14] [1, 80, 14, 14] 102,900 True │ │ └─MBConv (2) [1, 80, 14, 14] [1, 80, 14, 14] 102,900 True │ └─Sequential (5) [1, 80, 14, 14] [1, 112, 14, 14] -- True │ │ └─MBConv (0) [1, 80, 14, 14] [1, 112, 14, 14] 126,004 True │ │ └─MBConv (1) [1, 112, 14, 14] [1, 112, 14, 14] 208,572 True │ │ └─MBConv (2) [1, 112, 14, 14] [1, 112, 14, 14] 208,572 True │ └─Sequential (6) [1, 112, 14, 14] [1, 192, 7, 7] -- True │ │ └─MBConv (0) [1, 112, 14, 14] [1, 192, 7, 7] 262,492 True │ │ └─MBConv (1) [1, 192, 7, 7] [1, 192, 7, 7] 587,952 True │ │ └─MBConv (2) [1, 192, 7, 7] [1, 192, 7, 7] 587,952 True │ │ └─MBConv (3) [1, 192, 7, 7] [1, 192, 7, 7] 587,952 True │ └─Sequential (7) [1, 192, 7, 7] [1, 320, 7, 7] -- True │ │ └─MBConv (0) [1, 192, 7, 7] [1, 320, 7, 7] 717,232 True │ └─Conv2dNormActivation (8) [1, 320, 7, 7] [1, 1280, 7, 7] -- True │ │ └─Conv2d (0) [1, 320, 7, 7] [1, 1280, 7, 7] 409,600 True │ │ └─BatchNorm2d (1) [1, 1280, 7, 7] [1, 1280, 7, 7] 2,560 True │ │ └─SiLU (2) [1, 1280, 7, 7] [1, 1280, 7, 7] -- -- ├─AdaptiveAvgPool2d (avgpool) [1, 1280, 7, 7] [1, 1280, 1, 1] -- -- ├─Sequential (classifier) [1, 1280] [1, 1000] -- True │ └─Dropout (0) [1, 1280] [1, 1280] -- -- │ └─Linear (1) [1, 1280] [1, 1000] 1,281,000 True ============================================================================================================================================ Total params: 5,288,548 Trainable params: 5,288,548 Non-trainable params: 0 Total mult-adds (M): 385.87 ============================================================================================================================================ Input size (MB): 0.60 Forward/backward pass size (MB): 107.89 Params size (MB): 21.15 Estimated Total Size (MB): 129.64 ============================================================================================================================================
3.4 Freezing the base model and changing the output layer to suit our needs
With a feature extractor model, typically you will "freeze" the base layers of a pretrained/foundation model and update the output layers to suit your own problem.
In [17]python · cell 28
python
# Freeze all of the base layers in EffNetB0
for param in model.features.parameters():
# print(param)
param.requires_grad = FalseIn [18]python · cell 29
python
len(class_names)Output
3
In [19]python · cell 30
python
# Update the classifier head of our model to suit our problem
from torch import nn
torch.manual_seed(42)
torch.cuda.manual_seed(42)
model.classifier = nn.Sequential(
nn.Dropout(p=0.2, inplace=True),
nn.Linear(in_features=1280, # feature vector coming in
out_features=len(class_names))).to(device) # how many classes do we have?
model.classifierOutput
Sequential( (0): Dropout(p=0.2, inplace=True) (1): Linear(in_features=1280, out_features=3, bias=True) )
In [20]python · cell 31
python
summary(model=model,
input_size=(1, 3, 224, 224), # example of [batch_size, color_channels, height, width]
col_names=["input_size", "output_size", "num_params", "trainable"],
col_width=20,
row_settings=["var_names"])Output
============================================================================================================================================ Layer (type (var_name)) Input Shape Output Shape Param # Trainable ============================================================================================================================================ EfficientNet (EfficientNet) [1, 3, 224, 224] [1, 3] -- Partial ├─Sequential (features) [1, 3, 224, 224] [1, 1280, 7, 7] -- False │ └─Conv2dNormActivation (0) [1, 3, 224, 224] [1, 32, 112, 112] -- False │ │ └─Conv2d (0) [1, 3, 224, 224] [1, 32, 112, 112] (864) False │ │ └─BatchNorm2d (1) [1, 32, 112, 112] [1, 32, 112, 112] (64) False │ │ └─SiLU (2) [1, 32, 112, 112] [1, 32, 112, 112] -- -- │ └─Sequential (1) [1, 32, 112, 112] [1, 16, 112, 112] -- False │ │ └─MBConv (0) [1, 32, 112, 112] [1, 16, 112, 112] (1,448) False │ └─Sequential (2) [1, 16, 112, 112] [1, 24, 56, 56] -- False │ │ └─MBConv (0) [1, 16, 112, 112] [1, 24, 56, 56] (6,004) False │ │ └─MBConv (1) [1, 24, 56, 56] [1, 24, 56, 56] (10,710) False │ └─Sequential (3) [1, 24, 56, 56] [1, 40, 28, 28] -- False │ │ └─MBConv (0) [1, 24, 56, 56] [1, 40, 28, 28] (15,350) False │ │ └─MBConv (1) [1, 40, 28, 28] [1, 40, 28, 28] (31,290) False │ └─Sequential (4) [1, 40, 28, 28] [1, 80, 14, 14] -- False │ │ └─MBConv (0) [1, 40, 28, 28] [1, 80, 14, 14] (37,130) False │ │ └─MBConv (1) [1, 80, 14, 14] [1, 80, 14, 14] (102,900) False │ │ └─MBConv (2) [1, 80, 14, 14] [1, 80, 14, 14] (102,900) False │ └─Sequential (5) [1, 80, 14, 14] [1, 112, 14, 14] -- False │ │ └─MBConv (0) [1, 80, 14, 14] [1, 112, 14, 14] (126,004) False │ │ └─MBConv (1) [1, 112, 14, 14] [1, 112, 14, 14] (208,572) False │ │ └─MBConv (2) [1, 112, 14, 14] [1, 112, 14, 14] (208,572) False │ └─Sequential (6) [1, 112, 14, 14] [1, 192, 7, 7] -- False │ │ └─MBConv (0) [1, 112, 14, 14] [1, 192, 7, 7] (262,492) False │ │ └─MBConv (1) [1, 192, 7, 7] [1, 192, 7, 7] (587,952) False │ │ └─MBConv (2) [1, 192, 7, 7] [1, 192, 7, 7] (587,952) False │ │ └─MBConv (3) [1, 192, 7, 7] [1, 192, 7, 7] (587,952) False │ └─Sequential (7) [1, 192, 7, 7] [1, 320, 7, 7] -- False │ │ └─MBConv (0) [1, 192, 7, 7] [1, 320, 7, 7] (717,232) False │ └─Conv2dNormActivation (8) [1, 320, 7, 7] [1, 1280, 7, 7] -- False │ │ └─Conv2d (0) [1, 320, 7, 7] [1, 1280, 7, 7] (409,600) False │ │ └─BatchNorm2d (1) [1, 1280, 7, 7] [1, 1280, 7, 7] (2,560) False │ │ └─SiLU (2) [1, 1280, 7, 7] [1, 1280, 7, 7] -- -- ├─AdaptiveAvgPool2d (avgpool) [1, 1280, 7, 7] [1, 1280, 1, 1] -- -- ├─Sequential (classifier) [1, 1280] [1, 3] -- True │ └─Dropout (0) [1, 1280] [1, 1280] -- -- │ └─Linear (1) [1, 1280] [1, 3] 3,843 True ============================================================================================================================================ Total params: 4,011,391 Trainable params: 3,843 Non-trainable params: 4,007,548 Total mult-adds (M): 384.59 ============================================================================================================================================ Input size (MB): 0.60 Forward/backward pass size (MB): 107.88 Params size (MB): 16.05 Estimated Total Size (MB): 124.53 ============================================================================================================================================
4. Train model
In [22]python · cell 33
python
# Define loss and optimizer
loss_fn = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)In [24]python · cell 34
python
# Import train function
from going_modular.going_modular import engine
# Set the manual seeds
torch.manual_seed(42)
torch.cuda.manual_seed(42)
# Start the timer
from timeit import default_timer as timer
start_time = timer()
# Setup training and save the results
results = engine.train(model=model,
train_dataloader=train_dataloader,
test_dataloader=test_dataloader,
optimizer=optimizer,
loss_fn=loss_fn,
epochs=5,
device=device)
# End the timer and print out how long it took
end_time = timer()
print(f"[INFO] Total training time: {end_time-start_time:.3f} seconds")Output
0%| | 0/5 [00:00<?, ?it/s]
Epoch: 1 | train_loss: 1.0929 | train_acc: 0.4023 | test_loss: 0.9125 | test_acc: 0.5502 Epoch: 2 | train_loss: 0.8703 | train_acc: 0.7773 | test_loss: 0.7900 | test_acc: 0.8153 Epoch: 3 | train_loss: 0.7648 | train_acc: 0.8008 | test_loss: 0.7433 | test_acc: 0.8561 Epoch: 4 | train_loss: 0.7114 | train_acc: 0.7578 | test_loss: 0.6344 | test_acc: 0.8655 Epoch: 5 | train_loss: 0.6252 | train_acc: 0.7930 | test_loss: 0.6238 | test_acc: 0.8864 [INFO] Total training time: 13.046 seconds
5. Evalaute model by plotting loss curves
In [28]python · cell 36
python
try:
from helper_functions import plot_loss_curves
except:
print(f"[INFO] Couldn't find helper_functions.py, downloading...")
with open("helper_functions.py", "wb") as f:
import requests
request = requests.get("https://github.com/mrdbourke/pytorch-deep-learning/raw/main/helper_functions.py")
f.write(request.content)
from helper_functions import plot_loss_curves
# Plot the loss curves of our model
plot_loss_curves(results)Output
[INFO] Couldn't find helper_functions.py, downloading...
<Figure size 1080x504 with 2 Axes>
What do our loss curves look like in terms of the ideal loss curve?
See here for more: https://www.learnpytorch.io/04_pytorch_custom_datasets/#8-what-should-an-ideal-loss-curve-look-like
6. Make predictions on images from the test set
Let's adhere to the data explorer's motto of visualize, visualize, visualize!
And make some qualitiative predictions on our test set.
Some things to keep in mind when making predictions/inference on test data/custom data.
We have to make sure that our test/custom data is:
- Same shape - images need to be same shape as model was trained on
- Same datatype - custom data should be in the same data type
- Same device - custom data/test data should be on the same device as the model
- Same transform - if you've transformed your custom data, ideally you will transform the test data and custom data the same
To do all of this automagically, let's create a function called pred_and_plot_image():
The function will be similar to the one here: https://www.learnpytorch.io/04_pytorch_custom_datasets/#113-putting-custom-image-prediction-together-building-a-function
- Take in a trained model, a list of class names, a filepath to a target image, an image size, a transform and a target device
- Open the image with
PIL.Image.Open() - Create a transform if one doesn't exist
- Make sure the model is on the target device
- Turn the model to
model.eval()mode to make sure it's ready for inference (this will turn off things likenn.Dropout()) - Transform the target image and make sure its dimensionality is suited for the model (this mainly relates to batch size)
- Make a prediction on the image by passing to the model
- Convert the model's output logits to prediction probabilities using
torch.softmax() - Convert model's prediction probabilities to prediction labels using
torch.argmax() - Plot the image with
matplotliband set the title to the prediction label from step 9 and prediction probability from step 8
In [48]python · cell 39
python
from typing import List, Tuple
from PIL import Image
from torchvision import transforms
# 1. Take in a trained model...
def pred_and_plot_image(model: torch.nn.Module,
image_path: str,
class_names: List[str],
image_size: Tuple[int, int] = (224, 224),
transform: torchvision.transforms = None,
device: torch.device=device):
# 2. Open the image with PIL
img = Image.open(image_path)
# 3. Create a transform if one doesn't exist
if transform is not None:
image_transform = transform
else:
image_transform = transforms.Compose([
transforms.Resize(image_size),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
])
### Predict on image ###
# 4. Make sure the model is on the target device
model.to(device)
# 5. Turn on inference mode and eval mode
model.eval()
with torch.inference_mode():
# 6. Transform the image and add an extra batch dimension
transformed_image = image_transform(img).unsqueeze(dim=0) # [batch_size, color_channels, height, width]
# 7. Make a prediction on the transformed image by passing it to the model (also ensure it's on the target device)
target_image_pred = model(transformed_image.to(device))
# 8. Convert the model's output logits to pred probs
target_image_pred_probs = torch.softmax(target_image_pred, dim=1)
# print(target_image_pred_probs.max())
# 9. Convert the model's pred probs to pred labels
target_image_pred_label = torch.argmax(target_image_pred_probs, dim=1)
# 10. Plot image with predicted label and probability
plt.figure()
plt.imshow(img)
plt.title(f"Pred: {class_names[target_image_pred_label]} | Prob: {target_image_pred_probs.max():.3f}")
plt.axis(False);In [49]python · cell 40
python
test_dirOutput
PosixPath('data/pizza_steak_sushi/test')In [50]python · cell 41
python
class_namesOutput
['pizza', 'steak', 'sushi']
In [54]python · cell 42
python
# Get a random list of image paths from the test set
import random
num_images_to_plot = 3
test_image_path_list = list(Path(test_dir).glob("*/*.jpg"))
test_image_path_sample = random.sample(population=test_image_path_list,
k=num_images_to_plot)
# Make predictions on and plot the images
for image_path in test_image_path_sample:
pred_and_plot_image(model=model,
image_path=image_path,
class_names=class_names,
image_size=(224, 224))Output
<Figure size 432x288 with 1 Axes>
[省略较大 image/png 输出]
<Figure size 432x288 with 1 Axes>
[省略较大 image/png 输出]
<Figure size 432x288 with 1 Axes>
[省略较大 image/png 输出]
6.1 Making predictions on a custom image
Let's make a prediction on the pizza dad image - https://github.com/mrdbourke/pytorch-deep-learning/blob/main/images/04-pizza-dad.jpeg
In [56]python · cell 44
python
# Download the image
import requests
# Setup custom image path
custom_image_path = data_path / "04-pizza-dad.jpeg"
# Download the image if it doesn't exist
if not custom_image_path.is_file():
with open(custom_image_path, "wb") as f:
# Download image from GitHub with "raw" link
request = requests.get("https://github.com/mrdbourke/pytorch-deep-learning/raw/main/images/04-pizza-dad.jpeg")
print(f"Download {custom_image_path}...")
f.write(request.content)
else:
print(f"{custom_image_path} already exists, skipping download...")Output
Download data/04-pizza-dad.jpeg...
In [57]python · cell 45
python
# Predict on custom image
pred_and_plot_image(model=model,
image_path=custom_image_path,
class_names=class_names)Output
<Figure size 432x288 with 1 Axes>
[省略较大 image/png 输出]
Exercises
See exercises and extra-curriculum to practice what you've learned here: https://www.learnpytorch.io/06_pytorch_transfer_learning/#exercises
In [ ]python · cell 47
python