Image Classification

图像分类

Now that you understand what deep learning is, what it's for, and how to create and deploy a model, it's time for us to go deeper! In an ideal world deep learning practitioners wouldn't have to know every detail of how things work under the hood… But as yet, we don't live in an ideal world. The truth is, to make your model really work, and work reliably, there are a lot of details you have to get right, and a lot of details that you have to check. This process requires being able to look inside your neural network as it trains, and as it makes predictions, find possible problems, and know how to fix them.

So, from here on in the book we are going to do a deep dive into the mechanics of deep learning. What is the architecture of a computer vision model, an NLP model, a tabular model, and so on? How do you create an architecture that matches the needs of your particular domain? How do you get the best possible results from the training process? How do you make things faster? What do you have to change as your datasets change?

We will start by repeating the same basic applications that we looked at in the first chapter, but we are going to do two things:

• Make them better.
• Apply them to a wider variety of types of data.

In order to do these two things, we will have to learn all of the pieces of the deep learning puzzle. This includes different types of layers, regularization methods, optimizers, how to put layers together into architectures, labeling techniques, and much more. We are not just going to dump all of these things on you, though; we will introduce them progressively as needed, to solve actual problems related to the projects we are working on.

现在您已经了解了深度学习是什么、它的用途以及如何创建和部署模型，是时候让我们更深入地学习了！在一个理想的世界里，深度学习从业者不必知道事情如何在幕后工作的每一个细节……但到目前为止，我们还没有生活在一个理想的世界里。事实是，要使您的模型真正工作并可靠地工作，您必须正确处理很多细节，并且必须检查很多细节。这个过程需要能够在你的神经网络训练、预测、发现可能的问题并知道如何解决它们时查看它的内部。

因此，从本书的这里开始，我们将深入探讨深度学习的机制。计算机视觉模型、NLP 模型、表格模型等的架构是什么？您如何创建一个符合您特定领域需求的架构？您如何从培训过程中获得最佳结果？你如何让事情变得更快？随着数据集的变化，你必须改变什么？

我们将首先重复我们在第一章中看到的相同的基本应用程序，但我们要做两件事：

• 让他们变得更好。
• 将它们应用于更广泛的数据类型。

为了做这两件事，我们必须学习深度学习难题的所有部分。这包括不同类型的层、正则化方法、优化器、如何将层组合到架构中、标记技术等等。不过，我们不只是把所有这些东西都扔给你；我们将根据需要逐步引入它们，以解决与我们正在进行的项目相关的实际问题。

From Dogs and Cats to Pet Breeds

从狗和猫到宠物品种

In our very first model we learned how to classify dogs versus cats. Just a few years ago this was considered a very challenging task—but today, it's far too easy! We will not be able to show you the nuances of training models with this problem, because we get a nearly perfect result without worrying about any of the details. But it turns out that the same dataset also allows us to work on a much more challenging problem: figuring out what breed of pet is shown in each image.

In <<chapter_intro>> we presented the applications as already-solved problems. But this is not how things work in real life. We start with some dataset that we know nothing about. We then have to figure out how it is put together, how to extract the data we need from it, and what that data looks like. For the rest of this book we will be showing you how to solve these problems in practice, including all of the intermediate steps necessary to understand the data that you are working with and test your modeling as you go.

We already downloaded the Pet dataset, and we can get a path to this dataset using the same code as in <<chapter_intro>>:

在我们的第一个模型中，我们学习了如何对狗和猫进行分类。就在几年前，这被认为是一项非常具有挑战性的任务——但今天，这太容易了！我们将无法向您展示具有此问题的训练模型的细微差别，因为我们无需担心任何细节即可获得近乎完美的结果。但事实证明，相同的数据集还允许我们解决一个更具挑战性的问题：找出每张图像中显示的宠物品种。

在 《chapter_intro》 中，我们将应用程序表示为已解决的问题。但这不是现实生活中的事情。我们从一些我们一无所知的数据集开始。然后我们必须弄清楚它是如何组合在一起的，如何从中提取我们需要的数据，以及这些数据是什么样的。在本书的其余部分，我们将向您展示如何在实践中解决这些问题，包括理解您正在使用的数据和测试您的建模所需的所有中间步骤。

我们已经下载了 Pet 数据集，我们可以使用与 《chapter_intro》 中相同的代码获取该数据集的路径：

Now if we are going to understand how to extract the breed of each pet from each image we're going to need to understand how this data is laid out. Such details of data layout are a vital piece of the deep learning puzzle. Data is usually provided in one of these two ways:

• Individual files representing items of data, such as text documents or images, possibly organized into folders or with filenames representing information about those items
• A table of data, such as in CSV format, where each row is an item which may include filenames providing a connection between the data in the table and data in other formats, such as text documents and images

There are exceptions to these rules—particularly in domains such as genomics, where there can be binary database formats or even network streams—but overall the vast majority of the datasets you'll work with will use some combination of these two formats.

To see what is in our dataset we can use the ls method:

现在，如果我们要了解如何从每张图像中提取每只宠物的品种，我们将需要了解这些数据的布局方式。数据布局的这些细节是深度学习难题的重要组成部分。数据通常以以下两种方式之一提供：

代表数据项的单个文件，例如文本文档或图像，可能组织到文件夹中或文件名代表有关这些项的信息 数据表，例如 CSV 格式，其中每一行是一个项目，其中可能包含文件名，提供表中数据与其他格式数据（例如文本文档和图像）之间的连接 这些规则也有例外——特别是在基因组学等领域，其中可能存在二进制数据库格式甚至网络流——但总体而言，您将使用的绝大多数数据集将使用这两种格式的某种组合。

要查看数据集中的内容，我们可以使用 ls 方法：

We can see that this dataset provides us with images and annotations directories. The website for the dataset tells us that the annotations directory contains information about where the pets are rather than what they are. In this chapter, we will be doing classification, not localization, which is to say that we care about what the pets are, not where they are. Therefore, we will ignore the annotations directory for now. So, let's have a look inside the images directory:

我们可以看到这个数据集为我们提供了图像和注释目录。数据集的网站告诉我们，注释目录包含有关宠物在哪里而不是它们是什么的信息。在本章中，我们将进行分类，而不是本地化，也就是说，我们关心宠物是什么，而不是它们在哪里。因此，我们现在将忽略注释目录。所以，让我们看一下 images 目录：

Most functions and methods in fastai that return a collection use a class called L. L can be thought of as an enhanced version of the ordinary Python list type, with added conveniences for common operations. For instance, when we display an object of this class in a notebook it appears in the format shown there. The first thing that is shown is the number of items in the collection, prefixed with a #. You'll also see in the preceding output that the list is suffixed with an ellipsis. This means that only the first few items are displayed—which is a good thing, because we would not want more than 7,000 filenames on our screen!

By examining these filenames, we can see how they appear to be structured. Each filename contains the pet breed, and then an underscore (_), a number, and finally the file extension. We need to create a piece of code that extracts the breed from a single Path. Jupyter notebooks make this easy, because we can gradually build up something that works, and then use it for the entire dataset. We do have to be careful to not make too many assumptions at this point. For instance, if you look carefully you may notice that some of the pet breeds contain multiple words, so we cannot simply break at the first _ character that we find. To allow us to test our code, let's pick out one of these filenames:

fastai 中大多数返回集合的函数和方法都使用一个名为 L 的类。L 可以被认为是普通 Python list类型的增强版本，为常用操作增加了便利。例如，当我们在笔记本中显示此类的对象时，它会以此处显示的格式显示。显示的第一件事是集合中的项目数，以 # 为前缀。您还将在前面的输出中看到该列表以省略号为后缀。这意味着只显示前几个项目——这是一件好事，因为我们不希望屏幕上显示超过 7,000 个文件名！

通过检查这些文件名，我们可以看到它们的结构。每个文件名都包含宠物品种，然后是下划线 (_)、数字，最后是文件扩展名。我们需要创建一段代码，从单个Path中提取品种。 Jupyter notebook 让这一切变得简单，因为我们可以逐渐建立一些可行的东西，然后将其用于整个数据集。在这一点上，我们必须小心不要做出太多假设。例如，如果您仔细观察，您可能会注意到某些宠物品种包含多个单词，因此我们不能简单地在找到的第一个 _ 字符处中断。为了让我们能够测试我们的代码，让我们选择以下文件名之一：

The most powerful and flexible way to extract information from strings like this is to use a regular expression, also known as a regex. A regular expression is a special string, written in the regular expression language, which specifies a general rule for deciding if another string passes a test (i.e., "matches" the regular expression), and also possibly for plucking a particular part or parts out of that other string.

In this case, we need a regular expression that extracts the pet breed from the filename.

We do not have the space to give you a complete regular expression tutorial here, but there are many excellent ones online and we know that many of you will already be familiar with this wonderful tool. If you're not, that is totally fine—this is a great opportunity for you to rectify that! We find that regular expressions are one of the most useful tools in our programming toolkit, and many of our students tell us that this is one of the things they are most excited to learn about. So head over to Google and search for "regular expressions tutorial" now, and then come back here after you've had a good look around. The book's website also provides a list of our favorites.

a: Not only are regular expressions dead handy, but they also have interesting roots. They are "regular" because they were originally examples of a "regular" language, the lowest rung within the Chomsky hierarchy, a grammar classification developed by linguist Noam Chomsky, who also wrote Syntactic Structures, the pioneering work searching for the formal grammar underlying human language. This is one of the charms of computing: it may be that the hammer you reach for every day in fact came from a spaceship.

When you are writing a regular expression, the best way to start is just to try it against one example at first. Let's use the findall method to try a regular expression against the filename of the fname object:

从这样的字符串中提取信息的最强大和最灵活的方法是使用正则表达式，也称为正则表达式。正则表达式是用正则表达式语言编写的特殊字符串，它指定了一个通用规则，用于确定另一个字符串是否通过测试（即“匹配”正则表达式），也可能用于提取特定部分或部分的其他字符串。

在这种情况下，我们需要一个从文件名中提取宠物品种的正则表达式。

我们没有篇幅在这里给你一个完整的正则表达式教程，但是网上有很多优秀的，我们知道你们中的许多人已经熟悉了这个很棒的工具。如果你不是，那很好——这是你纠正它的好机会！我们发现正则表达式是我们编程工具包中最有用的工具之一，我们的许多学生告诉我们，这是他们最感兴趣的学习内容之一。所以现在去谷歌搜索“正则表达式教程”，然后在你浏览完之后再回到这里。这本书的网站还提供了我们最喜欢的列表。

a：正则表达式不仅非常方便，而且它们也有有趣的根源。它们是“常规的”，因为它们最初是“常规”语言的示例，乔姆斯基层次结构中的最低级别，由语言学家诺姆乔姆斯基开发的语法分类，他还写了句法结构，这是寻找人类基础的正式语法的开创性工作语。这就是计算的魅力之一：可能你每天拿的锤子实际上来自宇宙飞船。

当你在写一个正则表达式时，最好的开始方法就是一开始就用一个例子来尝试。让我们使用 findall 方法尝试针对 fname 对象的文件名的正则表达式：

This regular expression plucks out all the characters leading up to the last underscore character, as long as the subsequence characters are numerical digits and then the JPEG file extension.

Now that we confirmed the regular expression works for the example, let's use it to label the whole dataset. fastai comes with many classes to help with labeling. For labeling with regular expressions, we can use the RegexLabeller class. In this example we use the data block API we saw in <<chapter_production>> (in fact, we nearly always use the data block API—it's so much more flexible than the simple factory methods we saw in <<chapter_intro>>):

这个正则表达式会提取出最后一个下划线字符之前的所有字符，只要子序列字符是数字，然后是 JPEG 文件扩展名。

现在我们确认正则表达式适用于示例，让我们用它来标记整个数据集。 fastai 提供了许多类来帮助标记。对于使用正则表达式进行标记，我们可以使用 RegexLabeller 类。在这个例子中，我们使用了我们在 《chapter_production》 中看到的数据块 API（事实上，我们几乎总是使用数据块 API——它比我们在 《chapter_intro》 中看到的简单工厂方法灵活得多）：

One important piece of this DataBlock call that we haven't seen before is in these two lines:

item_tfms=Resize(460),
batch_tfms=aug_transforms(size=224, min_scale=0.75)

These lines implement a fastai data augmentation strategy which we call presizing. Presizing is a particular way to do image augmentation that is designed to minimize data destruction while maintaining good performance.

我们以前从未见过的这个 DataBlock 调用的一个重要部分是以下两行：

这些行实现了我们称之为 presizing 的 fastai 数据增强策略。 Presizing 是一种特殊的图像增强方式，旨在最大限度地减少数据破坏，同时保持良好的性能。

Presizing

加压

We need our images to have the same dimensions, so that they can collate into tensors to be passed to the GPU. We also want to minimize the number of distinct augmentation computations we perform. The performance requirement suggests that we should, where possible, compose our augmentation transforms into fewer transforms (to reduce the number of computations and the number of lossy operations) and transform the images into uniform sizes (for more efficient processing on the GPU).

The challenge is that, if performed after resizing down to the augmented size, various common data augmentation transforms might introduce spurious empty zones, degrade data, or both. For instance, rotating an image by 45 degrees fills corner regions of the new bounds with emptiness, which will not teach the model anything. Many rotation and zooming operations will require interpolating to create pixels. These interpolated pixels are derived from the original image data but are still of lower quality.

To work around these challenges, presizing adopts two strategies that are shown in <>:

1. Resize images to relatively "large" dimensions—that is, dimensions significantly larger than the target training dimensions.
2. Compose all of the common augmentation operations (including a resize to the final target size) into one, and perform the combined operation on the GPU only once at the end of processing, rather than performing the operations individually and interpolating multiple times.

The first step, the resize, creates images large enough that they have spare margin to allow further augmentation transforms on their inner regions without creating empty zones. This transformation works by resizing to a square, using a large crop size. On the training set, the crop area is chosen randomly, and the size of the crop is selected to cover the entire width or height of the image, whichever is smaller.

In the second step, the GPU is used for all data augmentation, and all of the potentially destructive operations are done together, with a single interpolation at the end.

我们需要我们的图像具有相同的尺寸，以便它们可以整理成张量以传递给 GPU。我们还希望最小化我们执行的不同增强计算的数量。性能要求表明，我们应该在可能的情况下将我们的增强变换组合成更少的变换（以减少计算数量和有损操作的数量）并将图像变换成统一的尺寸（以便在 GPU 上进行更有效的处理）。

挑战在于，如果在调整到增强后的大小后执行，各种常见的数据增强转换可能会引入虚假的空白区域、降级数据或两者兼而有之。例如，将图像旋转 45 度会使新边界的角落区域充满空白，这不会教给模型任何东西。许多旋转和缩放操作需要插值来创建像素。这些插值像素源自原始图像数据，但质量仍然较低。

为了解决这些挑战，预选采用 《presizing》 中所示的两种策略：

• 将图像调整为相对“大”的尺寸——即，尺寸明显大于目标训练尺寸。
• 将所有常见的增强操作（包括调整到最终目标大小）组合为一个，并在处理结束时仅在 GPU 上执行一次组合操作，而不是单独执行操作并进行多次插值。

第一步，调整大小，创建足够大的图像，使它们有多余的余量，以允许在其内部区域进行进一步的增强变换，而不会创建空白区域。此转换通过使用较大的裁剪尺寸调整为正方形来实现。在训练集上随机选择裁剪区域，裁剪的大小选择覆盖图像的整个宽度或高度，取较小者。

第二步，GPU 用于所有数据增强，所有潜在的破坏性操作一起完成，最后只进行一次插值。

This picture shows the two steps:

1. Crop full width or height: This is in item_tfms, so it's applied to each individual image before it is copied to the GPU. It's used to ensure all images are the same size. On the training set, the crop area is chosen randomly. On the validation set, the center square of the image is always chosen.
2. Random crop and augment: This is in batch_tfms, so it's applied to a batch all at once on the GPU, which means it's fast. On the validation set, only the resize to the final size needed for the model is done here. On the training set, the random crop and any other augmentations are done first.

To implement this process in fastai you use Resize as an item transform with a large size, and RandomResizedCrop as a batch transform with a smaller size. RandomResizedCrop will be added for you if you include the min_scale parameter in your aug_transforms function, as was done in the DataBlock call in the previous section. Alternatively, you can use pad or squish instead of crop (the default) for the initial Resize.

<> shows the difference between an image that has been zoomed, interpolated, rotated, and then interpolated again (which is the approach used by all other deep learning libraries), shown here on the right, and an image that has been zoomed and rotated as one operation and then interpolated just once on the left (the fastai approach), shown here on the left.

这张图显示了两个步骤：

1. 裁剪全宽或全高：这在 item_tfms 中，因此在将其复制到 GPU 之前将其应用于每个单独的图像。它用于确保所有图像的大小相同。在训练集上，裁剪区域是随机选择的。在验证集上，始终选择图像的中心正方形。
2. 随机裁剪和扩充：这是在 batch_tfms 中，所以它在 GPU 上一次全部应用于批处理，这意味着它很快。在验证集上，此处仅将大小调整为模型所需的最终大小。在训练集上，首先进行随机裁剪和任何其他增强。

要在 fastai 中实现此过程，您将 Resize 用作具有大尺寸的项目变换，并将 RandomResizedCrop 用作具有较小尺寸的批量变换。如果您在 aug_transforms 函数中包含 min_scale参数，则会为您添加 RandomResizedCrop，就像在上一节中的 DataBlock 调用中所做的那样。或者，您可以使用 pad 或 squish 而不是crop（默认）进行初始调整大小。

《interpolations》 显示了经过缩放、插值、旋转然后再次插值的图像（这是所有其他深度学习库使用的方法）（如右侧所示）与已缩放和旋转的图像之间的差异作为一个操作，然后在左侧仅插值一次（fastai 方法），如左侧所示。

You can see that the image on the right is less well defined and has reflection padding artifacts in the bottom-left corner; also, the grass at the top left has disappeared entirely. We find that in practice using presizing significantly improves the accuracy of models, and often results in speedups too.

The fastai library also provides simple ways to check your data looks right before training a model, which is an extremely important step. We'll look at those next.

您可以看到右边的图像不太清晰，左下角有反射填充伪影；而且，左上角的草也完全消失了。我们发现，在实践中，使用 presizing 可以显着提高模型的准确性，并且通常也会导致加速。

fastai 库还提供了在训练模型之前检查数据外观的简单方法，这是非常重要的一步。我们接下来会看看那些。

Checking and Debugging a DataBlock

检查和调试数据块

We can never just assume that our code is working perfectly. Writing a DataBlock is just like writing a blueprint. You will get an error message if you have a syntax error somewhere in your code, but you have no guarantee that your template is going to work on your data source as you intend. So, before training a model you should always check your data. You can do this using the show_batch method:

我们永远不能假设我们的代码运行良好。编写 DataBlock 就像编写蓝图一样。如果您的代码中某处出现语法错误，您将收到一条错误消息，但您不能保证您的模板会按照您的预期在您的数据源上工作。因此，在训练模型之前，您应该始终检查您的数据。您可以使用 show_batch 方法执行此操作：

Take a look at each image, and check that each one seems to have the correct label for that breed of pet. Often, data scientists work with data with which they are not as familiar as domain experts may be: for instance, I actually don't know what a lot of these pet breeds are. Since I am not an expert on pet breeds, I would use Google images at this point to search for a few of these breeds, and make sure the images look similar to what I see in this output.

If you made a mistake while building your DataBlock, it is very likely you won't see it before this step. To debug this, we encourage you to use the summary method. It will attempt to create a batch from the source you give it, with a lot of details. Also, if it fails, you will see exactly at which point the error happens, and the library will try to give you some help. For instance, one common mistake is to forget to use a Resize transform, so you end up with pictures of different sizes and are not able to batch them. Here is what the summary would look like in that case (note that the exact text may have changed since the time of writing, but it will give you an idea):

查看每张图片，并检查每张图片是否都带有该宠物品种的正确标签。通常，数据科学家处理的数据不像领域专家那样熟悉：例如，我实际上不知道这些宠物品种有多少。由于我不是宠物品种方面的专家，所以此时我会使用 Google 图片来搜索其中一些品种，并确保这些图片看起来与我在此输出中看到的相似。

如果您在构建 DataBlock 时犯了错误，很可能在此步骤之前您不会看到它。要对此进行调试，我们鼓励您使用summary方法。它将尝试从您提供的源中创建一个批次，其中包含很多详细信息。此外，如果它失败了，您将确切地看到错误发生的时间点，并且库将尝试为您提供一些帮助。例如，一个常见的错误是忘记使用调整大小变换，因此您最终会得到不同大小的图片并且无法对它们进行批处理。以下是在这种情况下摘要的样子（请注意，自撰写本文以来，确切的文本可能已经改变，但它会给你一个想法）：

You can see exactly how we gathered the data and split it, how we went from a filename to a sample (the tuple (image, category)), then what item transforms were applied and how it failed to collate those samples in a batch (because of the different shapes).

Once you think your data looks right, we generally recommend the next step should be using it to train a simple model. We often see people put off the training of an actual model for far too long. As a result, they don't actually find out what their baseline results look like. Perhaps your problem doesn't need lots of fancy domain-specific engineering. Or perhaps the data doesn't seem to train the model at all. These are things that you want to know as soon as possible. For this initial test, we'll use the same simple model that we used in <<chapter_intro>>:

您可以准确地看到我们如何收集数据并对其进行拆分，我们如何从文件名变为样本（元组（图像，类别）），然后应用了哪些项目转换以及它如何未能批量整理这些样本（因为形状不同）。

一旦您认为您的数据看起来正确，我们通常建议下一步应该使用它来训练一个简单的模型。我们经常看到人们将实际模型的训练推迟太久。结果，他们实际上并没有发现他们的基线结果是什么样的。也许您的问题不需要大量花哨的特定领域工程。或者也许数据似乎根本没有训练模型。这些都是你想尽快知道的事情。对于这个初始测试，我们将使用与 《chapter_intro》 中相同的简单模型：

As we've briefly discussed before, the table shown when we fit a model shows us the results after each epoch of training. Remember, an epoch is one complete pass through all of the images in the data. The columns shown are the average loss over the items of the training set, the loss on the validation set, and any metrics that we requested—in this case, the error rate.

Remember that loss is whatever function we've decided to use to optimize the parameters of our model. But we haven't actually told fastai what loss function we want to use. So what is it doing? fastai will generally try to select an appropriate loss function based on what kind of data and model you are using. In this case we have image data and a categorical outcome, so fastai will default to using cross-entropy loss.

正如我们之前简要讨论过的，当我们拟合模型时显示的表格向我们展示了每个训练阶段后的结果。请记住，一个时期是对数据中所有图像的完整遍历。显示的列是训练集项目的平均损失、验证集的损失以及我们要求的任何指标——在本例中为错误率。

请记住，损失是我们决定用来优化模型参数的任何函数。但我们实际上并没有告诉 fastai 我们想要使用什么损失函数。那么它在做什么呢？ fastai 一般会根据你使用的数据类型和模型，尝试选择合适的损失函数。在这种情况下，我们有图像数据和分类结果，因此 fastai 将默认使用交叉熵损失。

Cross-Entropy Loss

交叉熵损失

Cross-entropy loss is a loss function that is similar to the one we used in the previous chapter, but (as we'll see) has two benefits:

• It works even when our dependent variable has more than two categories.
• It results in faster and more reliable training.

In order to understand how cross-entropy loss works for dependent variables with more than two categories, we first have to understand what the actual data and activations that are seen by the loss function look like.

交叉熵损失是一种类似于我们在上一章中使用的损失函数，但是（正如我们将看到的）有两个好处：

• 即使我们的因变量有两个以上的类别，它也有效。
• 它导致更快和更可靠的培训。

为了了解交叉熵损失如何对具有两个以上类别的因变量起作用，我们首先必须了解损失函数看到的实际数据和激活是什么样子的。

Viewing Activations and Labels

查看激活和标签

Let's take a look at the activations of our model. To actually get a batch of real data from our DataLoaders, we can use the one_batch method:

让我们看看我们模型的激活。要从我们的 DataLoader 中实际获取一批真实数据，我们可以使用 one_batch 方法：

As you see, this returns the dependent and independent variables, as a mini-batch. Let's see what is actually contained in our dependent variable:

如您所见，这会以小批量的形式返回因变量和自变量。让我们看看我们的因变量中实际包含的内容：

Our batch size is 64, so we have 64 rows in this tensor. Each row is a single integer between 0 and 36, representing our 37 possible pet breeds. We can view the predictions (that is, the activations of the final layer of our neural network) using Learner.get_preds. This function either takes a dataset index (0 for train and 1 for valid) or an iterator of batches. Thus, we can pass it a simple list with our batch to get our predictions. It returns predictions and targets by default, but since we already have the targets, we can effectively ignore them by assigning to the special variable _:

我们的批量大小是 64，所以我们在这个张量中有 64 行。每行都是 0 到 36 之间的单个整数，代表我们 37 种可能的宠物品种。我们可以使用 Learner.get_preds 查看预测（即神经网络最后一层的激活）。此函数采用数据集索引（0 表示训练，1 表示有效）或批次迭代器。因此，我们可以通过我们的批次向它传递一个简单的列表来获得我们的预测。它默认返回预测和目标，但由于我们已经有了目标，我们可以通过分配特殊变量 _ 来有效地忽略它们：

The actual predictions are 37 probabilities between 0 and 1, which add up to 1 in total:

实际的预测是 0 到 1 之间的 37 个概率，总共加起来为 1：

To transform the activations of our model into predictions like this, we used something called the softmax activation function.

为了将我们模型的激活转换成这样的预测，我们使用了一种叫做 softmax 激活函数的东西。

Softmax

Softmax

In our classification model, we use the softmax activation function in the final layer to ensure that the activations are all between 0 and 1, and that they sum to 1.

Softmax is similar to the sigmoid function, which we saw earlier. As a reminder sigmoid looks like this:

在我们的分类模型中，我们在最后一层使用 softmax 激活函数来确保激活都在 0 和 1 之间，并且它们的总和为 1。

Softmax 类似于我们之前看到的 sigmoid 函数。提醒一下，sigmoid 看起来像这样：

We can apply this function to a single column of activations from a neural network, and get back a column of numbers between 0 and 1, so it's a very useful activation function for our final layer.

Now think about what happens if we want to have more categories in our target (such as our 37 pet breeds). That means we'll need more activations than just a single column: we need an activation per category. We can create, for instance, a neural net that predicts 3s and 7s that returns two activations, one for each class—this will be a good first step toward creating the more general approach. Let's just use some random numbers with a standard deviation of 2 (so we multiply randn by 2) for this example, assuming we have 6 images and 2 possible categories (where the first column represents 3s and the second is 7s):

我们可以将此函数应用于神经网络的单列激活，并返回一列介于 0 和 1 之间的数字，因此对于我们的最后一层来说，它是一个非常有用的激活函数。

现在想想如果我们想在我们的目标中拥有更多类别（例如我们的 37 个宠物品种）会发生什么。这意味着我们需要更多的激活，而不仅仅是单个列：我们需要每个类别的激活。例如，我们可以创建一个预测 3s 和 7s 的神经网络，它返回两个激活，每个类一个——这将是创建更通用方法的良好第一步。假设我们有 6 张图像和 2 个可能的类别（其中第一列代表 3s，第二列是 7s），我们只使用一些标准差为 2 的随机数（因此我们将 randn 乘以 2）：

We can't just take the sigmoid of this directly, since we don't get rows that add to 1 (i.e., we want the probability of being a 3 plus the probability of being a 7 to add up to 1):

我们不能直接取这个的 sigmoid，因为我们没有得到加为 1 的行（即，我们希望为 3 的概率加上为 7 的概率加起来为 1）：

In <<chapter_mnist_basics>>, our neural net created a single activation per image, which we passed through the sigmoid function. That single activation represented the model's confidence that the input was a 3. Binary problems are a special case of classification problems, because the target can be treated as a single boolean value, as we did in mnist_loss. But binary problems can also be thought of in the context of the more general group of classifiers with any number of categories: in this case, we happen to have two categories. As we saw in the bear classifier, our neural net will return one activation per category.

So in the binary case, what do those activations really indicate? A single pair of activations simply indicates the relative confidence of the input being a 3 versus being a 7. The overall values, whether they are both high, or both low, don't matter—all that matters is which is higher, and by how much.

We would expect that since this is just another way of representing the same problem, that we would be able to use sigmoid directly on the two-activation version of our neural net. And indeed we can! We can just take the difference between the neural net activations, because that reflects how much more sure we are of the input being a 3 than a 7, and then take the sigmoid of that:

在 《chapter_mnist_basics》 中，我们的神经网络为每个图像创建了一个单独的激活，我们通过 sigmoid 函数传递了该激活。该单一激活表示模型对输入为 3 的置信度。二元问题是分类问题的一种特殊情况，因为可以将目标视为单个布尔值，就像我们在 mnist_loss 中所做的那样。但也可以在具有任意数量类别的更一般的分类器组的上下文中考虑二元问题：在这种情况下，我们碰巧有两个类别。正如我们在熊分类器中看到的那样，我们的神经网络将为每个类别返回一个激活。

那么在二进制情况下，这些激活真正表明了什么？一对激活只是表示输入是 3 与 7 的相对置信度。总体值，无论它们都是高还是低，都无关紧要——重要的是哪个更高，以及多少钱。

我们希望，由于这只是表示同一问题的另一种方式，我们将能够直接在我们的神经网络的两次激活版本上使用 sigmoid。我们确实可以！我们可以只取神经网络激活之间的差异，因为这反映了我们对输入是 3 比 7 的确定程度，然后取它的 sigmoid：

The second column (the probability of it being a 7) will then just be that value subtracted from 1. Now, we need a way to do all this that also works for more than two columns. It turns out that this function, called softmax, is exactly that:

def softmax(x): return exp(x) / exp(x).sum(dim=1, keepdim=True)

第二列（它是 7 的概率）将是从 1 中减去的值。现在，我们需要一种方法来完成所有这些操作，该方法也适用于两列以上。事实证明，这个名为 softmax 的函数正是：

def softmax(x): return exp(x) / exp(x).sum(dim=1, keepdim=True)

jargon: Exponential function (exp): Literally defined as e**x, where e is a special number approximately equal to 2.718. It is the inverse of the natural logarithm function. Note that exp is always positive, and it increases very rapidly!

行话：指数函数 (exp)：字面定义为 $e^x$，其中 e 是一个大约等于 2.718 的特殊数字。它是自然对数函数的反函数。请注意，exp 始终为正数，并且增加得非常快！

Let's check that softmax returns the same values as sigmoid for the first column, and those values subtracted from 1 for the second column:

让我们检查一下 softmax 是否返回与第一列的 sigmoid 相同的值，以及从 1 中减去第二列的那些值：

softmax is the multi-category equivalent of sigmoid—we have to use it any time we have more than two categories and the probabilities of the categories must add to 1, and we often use it even when there are just two categories, just to make things a bit more consistent. We could create other functions that have the properties that all activations are between 0 and 1, and sum to 1; however, no other function has the same relationship to the sigmoid function, which we've seen is smooth and symmetric. Also, we'll see shortly that the softmax function works well hand-in-hand with the loss function we will look at in the next section.

If we have three output activations, such as in our bear classifier, calculating softmax for a single bear image would then look like something like <<bear_softmax>>.

softmax 是 sigmoid 的多类别等价物——只要我们有两个以上的类别，并且类别的概率必须加到 1，我们就必须使用它，即使只有两个类别，我们也经常使用它，只是为了使事情更加一致。我们可以创建其他具有所有激活都在 0 和 1 之间并且总和为 1 的属性的函数；然而，没有其他函数与 sigmoid 函数具有相同的关系，我们已经看到它是平滑且对称的。此外，我们很快就会看到 softmax 函数与我们将在下一节中看到的损失函数一起工作得很好。

如果我们有三个输出激活，例如在我们的熊分类器中，计算单个熊图像的 softmax 将类似于 《bear_softmax》。

What does this function do in practice? Taking the exponential ensures all our numbers are positive, and then dividing by the sum ensures we are going to have a bunch of numbers that add up to 1. The exponential also has a nice property: if one of the numbers in our activations x is slightly bigger than the others, the exponential will amplify this (since it grows, well... exponentially), which means that in the softmax, that number will be closer to 1.

Intuitively, the softmax function really wants to pick one class among the others, so it's ideal for training a classifier when we know each picture has a definite label. (Note that it may be less ideal during inference, as you might want your model to sometimes tell you it doesn't recognize any of the classes that it has seen during training, and not pick a class because it has a slightly bigger activation score. In this case, it might be better to train a model using multiple binary output columns, each using a sigmoid activation.)

Softmax is the first part of the cross-entropy loss—the second part is log likelihood.

这个函数在实践中做了什么？取指数确保我们所有的数字都是正数，然后除以总和确保我们将得到一堆加起来为 1 的数字。指数还有一个很好的属性：如果我们的激活 x 中的一个数字是比其他的稍微大一点，指数会放大这个（因为它会增长，嗯……指数），这意味着在 softmax 中，这个数字将更接近 1。

直观地说，softmax 函数确实想在其他类别中选择一个类别，因此当我们知道每张图片都有明确的标签时，它是训练分类器的理想选择。 （请注意，在推理过程中它可能不太理想，因为您可能希望您的模型有时告诉您它无法识别它在训练期间看到的任何类，并且不选择一个类，因为它的激活分数略高. 在这种情况下，使用多个二进制输出列训练模型可能会更好，每个列都使用 sigmoid 激活。）

Softmax 是交叉熵损失的第一部分——第二部分是对数似然。

Log Likelihood

对数似然

When we calculated the loss for our MNIST example in the last chapter we used:

def mnist_loss(inputs, targets):
    inputs = inputs.sigmoid()
    return torch.where(targets==1, 1-inputs, inputs).mean()

Just as we moved from sigmoid to softmax, we need to extend the loss function to work with more than just binary classification—it needs to be able to classify any number of categories (in this case, we have 37 categories). Our activations, after softmax, are between 0 and 1, and sum to 1 for each row in the batch of predictions. Our targets are integers between 0 and 36. Furthermore, cross-entropy loss generalizes our binary classification loss and allows for more than one correct label per example (which is called multi-label classificaiton, which we will discuss in Chapter 6).

In the binary case, we used torch.where to select between inputs and 1-inputs. When we treat a binary classification as a general classification problem with two categories, it actually becomes even easier, because (as we saw in the previous section) we now have two columns, containing the equivalent of inputs and 1-inputs. Since there is only one correct label per example, all we need to do is select the appropriate column (as opposed to multiplying multiple probabilities). Let's try to implement this in PyTorch. For our synthetic 3s and 7s example, let's say these are our labels:

当我们在上一章计算 MNIST 示例的损失时，我们使用了：

def mnist_loss(inputs, targets):
    inputs = inputs.sigmoid()
    return torch.where(targets==1, 1-inputs, inputs).mean()

就像我们从 sigmoid 转移到 softmax 一样，我们需要扩展损失函数来处理不仅仅是二进制分类——它需要能够分类任意数量的类别（在这种情况下，我们有 37 个类别）。在 softmax 之后，我们的激活值介于 0 和 1 之间，并且对于这批预测中的每一行，总和为 1。我们的目标是 0 到 36 之间的整数。此外，交叉熵损失概括了我们的二元分类损失，并允许每个示例有多个正确的标签（这称为多标签分类，我们将在第 6 章中讨论）。

在二进制情况下，我们使用 torch.where 在inputs和 1-inputs之间进行选择。当我们将二元分类视为具有两个类别的一般分类问题时，它实际上变得更加容易，因为（正如我们在上一节中看到的）我们现在有两列，包含inputs和 1-inputs的等价物。由于每个示例只有一个正确的标签，我们需要做的就是选择适当的列（而不是乘以多个概率）。让我们尝试在 PyTorch 中实现它。对于我们的合成 3 和 7 示例，假设这些是我们的标签：

and these are the softmax activations:

这些是softmax激活：

Then for each item of targ we can use that to select the appropriate column of sm_acts using tensor indexing, like so:

然后对于 targ 的每个项目，我们可以使用它使用张量索引来选择 sm_acts 的适当列，如下所示：

To see exactly what's happening here, let's put all the columns together in a table. Here, the first two columns are our activations, then we have the targets and the row index. We explain the last column, result below:

要确切了解这里发生了什么，让我们将所有列放在一个表中。在这里，前两列是我们的激活，然后是目标和行索引。我们解释最后一列，结果如下：

Looking at this table, you can see that the result column can be calculated by taking the targ and idx columns as indices into the two-column matrix containing the 3 and 7 columns. That's what sm_acts[idx, targ] is actually doing. The really interesting thing here is that this actually works just as well with more than two columns. To see this, consider what would happen if we added an activation column for every digit (0 through 9), and then targ contained a number from 0 to 9.

查看此表，您可以看到结果列可以通过将 targ 和 idx 列作为包含 3 和 7 列的两列矩阵的索引来计算。这就是 sm_acts[idx, targ] 实际上在做的事情。这里真正有趣的是，这实际上与两个以上的列一样有效。要看到这一点，请考虑如果我们为每个数字（0 到 9）添加一个激活列，然后targ包含一个从 0 到 9 的数字会发生什么。

PyTorch provides a function that does exactly the same thing as sm_acts[range(n), targ] (except it takes the negative, because when applying the log afterward, we will have negative numbers), called nll_loss (NLL stands for negative log likelihood):

PyTorch 提供了一个与 sm_acts[range(n), targ] 做同样事情的函数（除了它取负数，因为之后应用对数时，我们会有负数），称为 nll_loss（NLL 代表负对数似然):

Despite its name, this PyTorch function does not take the log. We'll see why in the next section, but first, let's see why taking the logarithm can be useful.

尽管有它的名字，但这个 PyTorch 函数并不记录日志。我们将在下一节中了解原因，但首先，让我们看看为什么取对数会很有用。

warning: Confusing Name, Beware: The nll in nll_loss stands for "negative log likelihood," but it doesn't actually take the log at all! It assumes you have already taken the log. PyTorch has a function called log_softmax that combines log and softmax in a fast and accurate way. nll_loss is designed to be used after log_softmax.

警告：混淆名称，注意：nll_loss 中的 nll 代表“负对数似然性”，但它实际上根本不接受对数！它假定您已经获取了日志。 PyTorch 有一个名为 log_softmax 的函数，它以快速准确的方式结合了 log 和 softmax。 nll_loss 设计为在 log_softmax 之后使用。

Taking the Log

Recall that cross entropy loss may involve the multiplication of many numbers. Multiplying lots of negative numbers together can cause problems like numerical underflow in computers. Therefore, we want to transform these probabilities to larger values so we can perform mathematical operations on them. There is a mathematical function that does exactly this: the logarithm (available as torch.log). It is not defined for numbers less than 0, and looks like this between 0 and 1:

记录日志

回想一下，交叉熵损失可能涉及许多数字的相乘。将大量负数相乘会导致计算机中出现数值下溢等问题。因此，我们希望将这些概率转换为更大的值，以便我们可以对它们进行数学运算。有一个数学函数可以做到这一点：对数（可作为 torch.log 获得）。它没有为小于 0 的数字定义，在 0 和 1 之间看起来像这样：

Additionally, we want to ensure our model is able to detect differences between small numbers. For example, consider the probabilities of .01 and .001. Indeed, those numbers are very close together—but in another sense, 0.01 is 10 times more confident than 0.001. By taking the log of our probabilities, we prevent these important differences from being ignored.

此外，我们希望确保我们的模型能够检测小数字之间的差异。例如，考虑 0.01 和 0.001 的概率。事实上，这些数字非常接近——但从另一个意义上说，0.01 比 0.001 更有信心 10 倍。通过记录我们的概率，我们可以防止这些重要的差异被忽略。

Does "logarithm" ring a bell? The logarithm function has this identity:

y = b**a
a = log(y,b)

In this case, we're assuming that log(y,b) returns log y base b. However, PyTorch actually doesn't define log this way: log in Python uses the special number e (2.718...) as the base.

Perhaps a logarithm is something that you have not thought about for the last 20 years or so. But it's a mathematical idea that is going to be really critical for many things in deep learning, so now would be a great time to refresh your memory. The key thing to know about logarithms is this relationship:

log(a*b) = log(a)+log(b)

When we see it in that format, it looks a bit boring; but think about what this really means. It means that logarithms increase linearly when the underlying signal increases exponentially or multiplicatively. This is used, for instance, in the Richter scale of earthquake severity, and the dB scale of noise levels. It's also often used on financial charts, where we want to show compound growth rates more clearly. Computer scientists love using logarithms, because it means that multiplication, which can create really really large and really really small numbers, can be replaced by addition, which is much less likely to result in scales that are difficult for our computers to handle.

Observe that the log of a number approaches negative infinity as the number approaches zero. In our case, since the result relfects the predicted probability of the correct label, we want our loss function to return a small value when the prediction is "good" (closer to 1) and a large value when the prediction is "bad" (closer to 0). We can achieve this by taking the negative of the log:

“对数”会响吗？对数函数具有以下恒等式：

y = b**a
a = log(y,b)

在这种情况下，我们假设 log(y,b) 返回 log y base b。但是，PyTorch 实际上并没有这样定义 log：Python 中的 log 使用特殊数字 e (2.718...) 作为基数。

也许对数是你在过去 20 年左右没有考虑过的东西。但这是一个数学概念，对于深度学习中的许多事情都非常重要，所以现在是刷新记忆的好时机。了解对数的关键是这种关系：

log(a*b) = log(a)+log(b)

当我们以这种格式看到它时，它看起来有点无聊；但想想这到底意味着什么。这意味着当基础信号呈指数或乘法增加时，对数呈线性增加。例如，这用于地震严重程度的里氏标度和噪声级别的 dB 标度。它也经常用于财务图表，我们希望更清楚地显示复合增长率。计算机科学家喜欢使用对数，因为这意味着可以创建非常大和非常小的数字的乘法可以用加法代替，这不太可能导致我们的计算机难以处理的尺度。

观察到当数字接近零时，数字的对数接近负无穷大。在我们的例子中，由于结果反映了正确标签的预测概率，我们希望我们的损失函数在预测“好”（接近 1）时返回一个小值，而在预测“坏”时返回一个大值（接近于 0）。我们可以通过取对数的负数来实现这一点：

s: It's not just computer scientists that love logs! Until computers came along, engineers and scientists used a special ruler called a "slide rule" that did multiplication by adding logarithms. Logarithms are widely used in physics, for multiplying very big or very small numbers, and many other fields.

s：不仅仅是计算机科学家喜欢日志！在计算机出现之前，工程师和科学家使用一种称为“计算尺”的特殊尺子，通过添加对数来进行乘法运算。对数在物理学中被广泛使用，用于乘以非常大或非常小的数字，以及许多其他领域。

Let's go ahead and update our previous table with an additional column, loss to reflect this loss function:

让我们继续更新我们之前的表格，增加一列 loss 来反映这个损失函数：

Notice how the loss is very large in the third and fourth rows where the predictions are confident and wrong, or in other words have high probabilities on the wrong class. One benefit of using the log to calculate the loss is that our loss function penalizes predictions that are both confident and wrong. This kind of penalty works well in practice to aid in more effective model training.

s: There are other loss functions such as focal loss that allow you control this penalty with a parameter. We do not discuss that loss function in this book.

请注意，在第三行和第四行中，预测是有把握的和错误的，或者换句话说，在错误类别上的概率很高，损失是如何非常大的。使用对数计算损失的一个好处是我们的损失函数会惩罚既自信又错误的预测。这种惩罚在实践中效果很好，有助于更有效的模型训练。

s：还有其他损失函数，例如焦点损失，可以让你用一个参数来控制这个惩罚。我们不会在本书中讨论这个损失函数。

We're calculating the loss from the column containing the correct label. Because there is only one "right" answer per example, we don't need to consider the other columns, because by the definition of softmax, they add up to 1 minus the activation corresponding to the correct label. As long as the activation columns sum to 1 (as they will, if we use softmax), then we'll have a loss function that shows how well we're predicting each digit. Therefore, making the activation for the correct label as high as possible must mean we're also decreasing the activations of the remaining columns.

我们正在计算包​​含正确标签的列的损失。因为每个示例只有一个“正确”答案，所以我们不需要考虑其他列，因为根据 softmax 的定义，它们加起来等于 1 减去对应于正确标签的激活。只要激活列的总和为 1（如果我们使用 softmax，它们会如此），那么我们就会有一个损失函数来显示我们对每个数字的预测程度。因此，使正确标签的激活尽可能高必然意味着我们也在减少剩余列的激活。

Negative Log Likelihood

Taking the mean of the negative log of our probabilities (taking the mean of the loss column of our table) gives us the negative log likelihood loss, which is another name for cross-entropy loss. Recall that PyTorch's nll_loss assumes that you already took the log of the softmax, so it doesn't actually do the logarithm for you.

取我们概率的负对数的平均值（取我们表中损失列的平均值）给我们负对数似然损失，这是交叉熵损失的另一个名称。回想一下，PyTorch 的 nll_loss 假设您已经获取了 softmax 的日志，因此它实际上并没有为您计算对数。

When we first take the softmax, and then the log likelihood of that, that combination is called cross-entropy loss. In PyTorch, this is available as nn.CrossEntropyLoss (which, in practice, actually does log_softmax and then nll_loss):

当我们首先取 softmax，然后取其对数似然时，这种组合称为交叉熵损失。在 PyTorch 中，这可用作 nn.CrossEntropyLoss（实际上，它实际上是 log_softmax，然后是 nll_loss）：

As you see, this is a class. Instantiating it gives you an object which behaves like a function:

如您所见，这是一堂课。实例化它会给你一个行为像函数的对象：

All PyTorch loss functions are provided in two forms, the class just shown above, and also a plain functional form, available in the F namespace:

所有 PyTorch 损失函数都以两种形式提供，上面显示的类和普通函数形式，在 F 命名空间中可用：

Either one works fine and can be used in any situation. We've noticed that most people tend to use the class version, and that's more often used in PyTorch's official docs and examples, so we'll tend to use that too.

By default PyTorch loss functions take the mean of the loss of all items. You can use reduction='none' to disable that:

任何一个都可以正常工作，并且可以在任何情况下使用。我们注意到大多数人倾向于使用 class 版本，这在 PyTorch 的官方文档和示例中更常用，所以我们也倾向于使用它。

默认情况下，PyTorch 损失函数采用所有项目损失的平均值。您可以使用 reduction='none' 禁用它：

You will notice these values match the loss column in our table exactly.

您会注意到这些值与我们表中的损失列完全匹配。

s: An interesting feature about cross-entropy loss appears when we consider its gradient. The gradient of cross_entropy(a,b) is just softmax(a)-b. Since softmax(a) is just the final activation of the model, that means that the gradient is proportional to the difference between the prediction and the target. This is the same as mean squared error in regression (assuming there's no final activation function such as that added by y_range), since the gradient of (a-b)**2 is 2*(a-b). Because the gradient is linear, that means we won't see sudden jumps or exponential increases in gradients, which should lead to smoother training of models.

s：当我们考虑它的梯度时，会出现一个关于交叉熵损失的有趣特征。 cross_entropy(a,b) 的梯度就是 softmax(a)-b。由于 softmax(a) 只是模型的最终激活，这意味着梯度与预测和目标之间的差异成正比。这与回归中的均方误差相同（假设没有像 y_range 添加的最终激活函数），因为 (a-b)**2 的梯度是 2*(a-b)。因为梯度是线性的，这意味着我们不会看到梯度的突然跳跃或指数增加，这应该会导致模型的训练更平滑。

We have now seen all the pieces hidden behind our loss function. But while this puts a number on how well (or badly) our model is doing, it does nothing to help us know if it's actually any good. Let's now see some ways to interpret our model's predictions.

我们现在已经看到了隐藏在损失函数后面的所有部分。但是，虽然这为我们的模型做得好（或坏）提供了一个数字，但它并不能帮助我们知道它是否真的有任何好处。现在让我们看看一些方法来解释我们模型的预测。

Model Interpretation

模型解释

It's very hard to interpret loss functions directly, because they are designed to be things computers can differentiate and optimize, not things that people can understand. That's why we have metrics. These are not used in the optimization process, but just to help us poor humans understand what's going on. In this case, our accuracy is looking pretty good already! So where are we making mistakes?

We saw in <<chapter_intro>> that we can use a confusion matrix to see where our model is doing well, and where it's doing badly:

直接解释损失函数非常困难，因为它们被设计为计算机可以区分和优化的东西，而不是人们可以理解的东西。这就是我们有指标的原因。这些不是在优化过程中使用的，只是为了帮助我们这些可怜的人了解正在发生的事情。在这种情况下，我们的准确性看起来已经相当不错了！那么我们在哪里犯错呢？

我们在 《chapter_intro》 中看到，我们可以使用混淆矩阵来查看我们的模型在哪里做得好，哪里做得不好：

Oh dear—in this case, a confusion matrix is very hard to read. We have 37 different breeds of pet, which means we have 37×37 entries in this giant matrix! Instead, we can use the most_confused method, which just shows us the cells of the confusion matrix with the most incorrect predictions (here, with at least 5 or more):

哦，天哪——在这种情况下，混淆矩阵很难阅读。我们有 37 种不同品种的宠物，这意味着我们在这个巨大的矩阵中有 37×37 个条目！相反，我们可以使用 most_confused 方法，它只向我们显示混淆矩阵中预测最不正确的单元格（这里至少有 5 个或更多）：

Since we are not pet breed experts, it is hard for us to know whether these category errors reflect actual difficulties in recognizing breeds. So again, we turn to Google. A little bit of Googling tells us that the most common category errors shown here are actually breed differences that even expert breeders sometimes disagree about. So this gives us some comfort that we are on the right track.

We seem to have a good baseline. What can we do now to make it even better?

由于我们不是宠物品种专家，我们很难知道这些类别错误是否反映了识别品种的实际困难。因此，我们再次求助于谷歌。一点点谷歌搜索告诉我们，这里显示的最常见的类别错误实际上是品种差异，即使是专业育种者有时也会不同意。因此，这让我们感到欣慰，我们正走在正确的轨道上。

我们似乎有一个很好的基线。我们现在能做些什么来让它变得更好？

Improving Our Model

改善模型

We will now look at a range of techniques to improve the training of our model and make it better. While doing so, we will explain a little bit more about transfer learning and how to fine-tune our pretrained model as best as possible, without breaking the pretrained weights.

The first thing we need to set when training a model is the learning rate. We saw in the previous chapter that it needs to be just right to train as efficiently as possible, so how do we pick a good one? fastai provides a tool for this.

我们现在将研究一系列技术来改进我们模型的训练并使其变得更好。在此过程中，我们将更多地解释迁移学习以及如何在不破坏预训练权重的情况下尽可能地微调我们的预训练模型。

训练模型时我们需要设置的第一件事是学习率。我们在上一章中看到，要尽可能高效地训练，需要恰到好处，那么我们如何挑选一个好的呢？ fastai 为此提供了一个工具。

The Learning Rate Finder

One of the most important things we can do when training a model is to make sure that we have the right learning rate. If our learning rate is too low, it can take many, many epochs to train our model. Not only does this waste time, but it also means that we may have problems with overfitting, because every time we do a complete pass through the data, we give our model a chance to memorize it.

So let's just make our learning rate really high, right? Sure, let's try that and see what happens:

在训练模型时，我们可以做的最重要的事情之一就是确保我们拥有正确的学习率。如果我们的学习率太低，可能需要很多很多 epoch 来训练我们的模型。这不仅浪费时间，还意味着我们可能会遇到过度拟合的问题，因为每次我们对数据进行完整的传递时，我们都会给我们的模型一个记忆它的机会。

所以让我们让我们的学习率真的很高，对吧？当然，让我们尝试一下，看看会发生什么：

That doesn't look good. Here's what happened. The optimizer stepped in the correct direction, but it stepped so far that it totally overshot the minimum loss. Repeating that multiple times makes it get further and further away, not closer and closer!

What do we do to find the perfect learning rate—not too high, and not too low? In 2015 the researcher Leslie Smith came up with a brilliant idea, called the learning rate finder. His idea was to start with a very, very small learning rate, something so small that we would never expect it to be too big to handle. We use that for one mini-batch, find what the losses are afterwards, and then increase the learning rate by some percentage (e.g., doubling it each time). Then we do another mini-batch, track the loss, and double the learning rate again. We keep doing this until the loss gets worse, instead of better. This is the point where we know we have gone too far. We then select a learning rate a bit lower than this point. Our advice is to pick either:

• One order of magnitude less than where the minimum loss was achieved (i.e., the minimum divided by 10)
• The last point where the loss was clearly decreasing

The learning rate finder computes those points on the curve to help you. Both these rules usually give around the same value. In the first chapter, we didn't specify a learning rate, using the default value from the fastai library (which is 1e-3):

那看起来不太好。这就是发生的事情。优化器朝着正确的方向迈进，但它前进得太远以至于完全超过了最小损失。重复多次使它越来越远，而不是越来越近！

我们怎么做才能找到完美的学习率——不要太高，也不要太低？ 2015 年，研究员 Leslie Smith 提出了一个绝妙的主意，称为学习率查找器。他的想法是从一个非常非常小的学习率开始，它是如此之小以至于我们永远不会期望它太大而无法处理。我们将它用于一个小批量，然后找出损失是多少，然后将学习率提高一定百分比（例如，每次加倍）。然后我们再做一个小批量，跟踪损失，再次将学习率加倍。我们一直这样做，直到损失变得更糟，而不是更好。这是我们知道我们已经走得太远的地方。然后我们选择一个比这个点低一点的学习率。我们的建议是选择：

• 比达到最小损失的位置小一个数量级（即，最小值除以 10）
• 损失明显减少的最后一点

学习率查找器会计算曲线上的这些点来帮助您。这两个规则通常给出大约相同的值。在第一章中，我们没有指定学习率，使用来自 fastai 库的默认值（即 1e-3）：

We can see on this plot that in the range 1e-6 to 1e-3, nothing really happens and the model doesn't train. Then the loss starts to decrease until it reaches a minimum, and then increases again. We don't want a learning rate greater than 1e-1 as it will give a training that diverges like the one before (you can try for yourself), but 1e-1 is already too high: at this stage we've left the period where the loss was decreasing steadily.

In this learning rate plot it appears that a learning rate around 3e-3 would be appropriate, so let's choose that:

我们可以在这张图上看到，在 1e-6 到 1e-3 的范围内，什么都没有发生，模型也没有训练。然后损失开始减少，直到达到最小值，然后再次增加。我们不想要大于 1e-1 的学习率，因为它会提供像以前那样发散的训练（你可以自己尝试），但是 1e-1 已经太高了：在这个阶段我们已经离开了亏损稳步下降的时期。

在这个学习率图中，3e-3 左右的学习率似乎是合适的，所以让我们选择：

Note: Logarithmic Scale: The learning rate finder plot has a logarithmic scale, which is why the middle point between 1e-3 and 1e-2 is between 3e-3 and 4e-3. This is because we care mostly about the order of magnitude of the learning rate.

注意：对数刻度：学习率查找器图有对数刻度，这就是为什么 1e-3 和 1e-2 之间的中点在 3e-3 和 4e-3 之间的原因。这是因为我们主要关心学习率的数量级。

It's interesting that the learning rate finder was only discovered in 2015, while neural networks have been under development since the 1950s. Throughout that time finding a good learning rate has been, perhaps, the most important and challenging issue for practitioners. The solution does not require any advanced maths, giant computing resources, huge datasets, or anything else that would make it inaccessible to any curious researcher. Furthermore, Leslie Smith, was not part of some exclusive Silicon Valley lab, but was working as a naval researcher. All of this is to say: breakthrough work in deep learning absolutely does not require access to vast resources, elite teams, or advanced mathematical ideas. There is lots of work still to be done that requires just a bit of common sense, creativity, and tenacity.

有趣的是，学习率查找器是在 2015 年才被发现的，而神经网络自 1950 年代以来一直在开发中。在那段时间里，找到一个好的学习率可能是从业者最重要和最具挑战性的问题。该解决方案不需要任何高级数学、庞大的计算资源、庞大的数据集或其他任何会使任何好奇的研究人员无法访问的东西。此外，莱斯利·史密斯并不是硅谷某个独家实验室的成员，而是一名海军研究员。这一切都意味着：深度学习的突破性工作绝对不需要大量资源、精英团队或先进的数学思想。还有很多工作要做，只需要一点常识、创造力和毅力。

Now that we have a good learning rate to train our model, let's look at how we can fine-tune the weights of a pretrained model.

现在我们有一个很好的学习率来训练我们的模型，让我们看看如何微调预训练模型的权重。

Unfreezing and Transfer Learning

We discussed briefly in <<chapter_intro>> how transfer learning works. We saw that the basic idea is that a pretrained model, trained potentially on millions of data points (such as ImageNet), is fine-tuned for some other task. But what does this really mean?

We now know that a convolutional neural network consists of many linear layers with a nonlinear activation function between each pair, followed by one or more final linear layers with an activation function such as softmax at the very end. The final linear layer uses a matrix with enough columns such that the output size is the same as the number of classes in our model (assuming that we are doing classification).

This final linear layer is unlikely to be of any use for us when we are fine-tuning in a transfer learning setting, because it is specifically designed to classify the categories in the original pretraining dataset. So when we do transfer learning we remove it, throw it away, and replace it with a new linear layer with the correct number of outputs for our desired task (in this case, there would be 37 activations).

This newly added linear layer will have entirely random weights. Therefore, our model prior to fine-tuning has entirely random outputs. But that does not mean that it is an entirely random model! All of the layers prior to the last one have been carefully trained to be good at image classification tasks in general. As we saw in the images from the Zeiler and Fergus paper in <<chapter_intro>> (see <<img_layer1>> through <<img_layer4>>), the first few layers encode very general concepts, such as finding gradients and edges, and later layers encode concepts that are still very useful for us, such as finding eyeballs and fur.

We want to train a model in such a way that we allow it to remember all of these generally useful ideas from the pretrained model, use them to solve our particular task (classify pet breeds), and only adjust them as required for the specifics of our particular task.

Our challenge when fine-tuning is to replace the random weights in our added linear layers with weights that correctly achieve our desired task (classifying pet breeds) without breaking the carefully pretrained weights and the other layers. There is actually a very simple trick to allow this to happen: tell the optimizer to only update the weights in those randomly added final layers. Don't change the weights in the rest of the neural network at all. This is called freezing those pretrained layers.

我们在 《chapter_intro》 中简要讨论了迁移学习的工作原理。我们看到基本思想是，一个可能在数百万个数据点（例如 ImageNet）上训练的预训练模型，可以针对其他任务进行微调。但这究竟意味着什么？

我们现在知道，卷积神经网络由许多线性层组成，每对之间有一个非线性激活函数，然后是一个或多个最终线性层，最后有一个激活函数，例如 softmax。最后的线性层使用具有足够列的矩阵，使得输出大小与我们模型中的类数相同（假设我们正在进行分类）。

当我们在迁移学习设置中进行微调时，这个最终的线性层不太可能对我们有任何用处，因为它专门用于对原始预训练数据集中的类别进行分类。因此，当我们进行迁移学习时，我们将其移除、丢弃，并用一个新的线性层替换它，该层具有正确数量的输出，用于我们想要的任务（在这种情况下，将有 37 个激活）。

这个新添加的线性层将具有完全随机的权重。因此，我们在微调之前的模型具有完全随机的输出。但这并不意味着它是一个完全随机的模型！最后一层之前的所有层都经过仔细训练，通常擅长图像分类任务。正如我们在 《chapter_intro》 中 Zeiler 和 Fergus 论文的图像中看到的（参见 《img_layer1》 到 《img_layer4》），前几层编码了非常一般的概念，例如寻找梯度和边缘，后面的层编码仍然非常有用的概念对我们来说，比如寻找眼球和皮毛。

我们希望以这样一种方式训练模型，使其能够记住来自预训练模型的所有这些通常有用的想法，使用它们来解决我们的特定任务（分类宠物品种），并且只根据具体需要调整它们我们的特殊任务。

我们在微调时面临的挑战是用正确实现我们所需任务（分类宠物品种）的权重替换我们添加的线性层中的随机权重，而不会破坏仔细预训练的权重和其他层。实际上有一个非常简单的技巧可以让这种情况发生：告诉优化器只更新那些随机添加的最终层中的权重。根本不要改变神经网络其余部分的权重。这称为冻结那些预训练层。

When we create a model from a pretrained network fastai automatically freezes all of the pretrained layers for us. When we call the fine_tune method fastai does two things:

• Trains the randomly added layers for one epoch, with all other layers frozen
• Unfreezes all of the layers, and trains them all for the number of epochs requested

Although this is a reasonable default approach, it is likely that for your particular dataset you may get better results by doing things slightly differently. The fine_tune method has a number of parameters you can use to change its behavior, but it might be easiest for you to just call the underlying methods directly if you want to get some custom behavior. Remember that you can see the source code for the method by using the following syntax:

learn.fine_tune??

So let's try doing this manually ourselves. First of all we will train the randomly added layers for three epochs, using fit_one_cycle. As mentioned in <<chapter_intro>>, fit_one_cycle is the suggested way to train models without using fine_tune. We'll see why later in the book; in short, what fit_one_cycle does is to start training at a low learning rate, gradually increase it for the first section of training, and then gradually decrease it again for the last section of training.

当我们从预训练网络创建模型时，fastai 会自动为我们冻结所有预训练层。当我们调用fine_tune 方法时，fastai 做了两件事：

• 训练一个 epoch 的随机添加层，冻结所有其他层
• 解冻所有层，并根据请求的 epoch 数对它们进行训练

尽管这是一种合理的默认方法，但对于您的特定数据集，您可能会通过稍微不同的方式获得更好的结果。 Fine_tune 方法有许多参数可用于更改其行为，但如果您想获得一些自定义行为，直接调用底层方法可能是最简单的。请记住，您可以使用以下语法查看该方法的源代码：

learn.fine_tune??

因此，让我们尝试自己手动执行此操作。首先，我们将使用 fit_one_cycle 为三个 epoch 训练随机添加的层。如 《chapter_intro》 中所述，fit_one_cycle 是建议在不使用fine_tune 的情况下训练模型的方法。我们将在本书后面看到原因；简而言之，fit_one_cycle 所做的就是以较低的学习率开始训练，在第一节训练逐渐增加它，然后在最后一节训练再次逐渐降低它。

Then we'll unfreeze the model:

然后我们将解冻模型：

and run lr_find again, because having more layers to train, and weights that have already been trained for three epochs, means our previously found learning rate isn't appropriate any more:

并再次运行 lr_find ，因为要训练更多层，并且已经训练了三个 epoch 的权重，意味着我们之前发现的学习率不再合适：

Note that the graph is a little different from when we had random weights: we don't have that sharp descent that indicates the model is training. That's because our model has been trained already. Here we have a somewhat flat area before a sharp increase, and we should take a point well before that sharp increase—for instance, 1e-5. The point with the maximum gradient isn't what we look for here and should be ignored.

Let's train at a suitable learning rate:

请注意，该图与我们使用随机权重时略有不同：我们没有表明模型正在训练的急剧下降。那是因为我们的模型已经被训练过了。在这里，我们在急剧增加之前有一个稍微平坦的区域，我们应该在急剧增加之前取一个点——例如，1e-5。具有最大梯度的点不是我们在这里寻找的，应该被忽略。

让我们以合适的学习率训练：

This has improved our model a bit, but there's more we can do. The deepest layers of our pretrained model might not need as high a learning rate as the last ones, so we should probably use different learning rates for those—this is known as using discriminative learning rates.

这稍微改进了我们的模型，但我们还可以做更多的事情。我们预训练模型的最深层可能不需要像最后一层那样高的学习率，因此我们可能应该为那些使用不同的学习率——这被称为使用判别式学习率。

Discriminative Learning Rates

Even after we unfreeze, we still care a lot about the quality of those pretrained weights. We would not expect that the best learning rate for those pretrained parameters would be as high as for the randomly added parameters, even after we have tuned those randomly added parameters for a few epochs. Remember, the pretrained weights have been trained for hundreds of epochs, on millions of images.

In addition, do you remember the images we saw in <<chapter_intro>>, showing what each layer learns? The first layer learns very simple foundations, like edge and gradient detectors; these are likely to be just as useful for nearly any task. The later layers learn much more complex concepts, like "eye" and "sunset," which might not be useful in your task at all (maybe you're classifying car models, for instance). So it makes sense to let the later layers fine-tune more quickly than earlier layers.

Therefore, fastai's default approach is to use discriminative learning rates. This was originally developed in the ULMFiT approach to NLP transfer learning that we will introduce in <<chapter_nlp>>. Like many good ideas in deep learning, it is extremely simple: use a lower learning rate for the early layers of the neural network, and a higher learning rate for the later layers (and especially the randomly added layers). The idea is based on insights developed by Jason Yosinski, who showed in 2014 that with transfer learning different layers of a neural network should train at different speeds, as seen in <>.

即使在我们解冻之后，我们仍然非常关心那些预训练权重的质量。我们不会期望那些预训练参数的最佳学习率会和随机添加的参数一样高，即使我们已经调整了这些随机添加的参数几个 epoch。请记住，预训练的权重已经在数百万张图像上训练了数百个 epoch。

此外，你还记得我们在 《chapter_intro》 中看到的图像，展示了每一层的学习内容吗？第一层学习非常简单的基础，例如边缘和梯度检测器；这些可能对几乎任何任务都同样有用。后面的层学习了更复杂的概念，例如“眼睛”和“日落”，这可能对您的任务根本没有用（例如，您可能正在对汽车模型进行分类）。因此，让后面的层比前面的层更快地进行微调是有意义的。

因此，fastai 的默认方法是使用判别式学习率。这最初是在我们将在 《chapter_intro》 中介绍的 NLP 迁移学习的 ULMFiT 方法中开发的。就像深度学习中的许多好主意一样，它非常简单：对神经网络的早期层使用较低的学习率，而对后面的层（尤其是随机添加的层）使用较高的学习率。这个想法基于 Jason Yosinski 提出的见解，他在 2014 年表明，通过迁移学习，神经网络的不同层应该以不同的速度训练，如 《yosinski》 中所示。

fastai lets you pass a Python slice object anywhere that a learning rate is expected. The first value passed will be the learning rate in the earliest layer of the neural network, and the second value will be the learning rate in the final layer. The layers in between will have learning rates that are multiplicatively equidistant throughout that range. Let's use this approach to replicate the previous training, but this time we'll only set the lowest layer of our net to a learning rate of 1e-6; the other layers will scale up to 1e-4. Let's train for a while and see what happens:

fastai 允许您在任何预期学习率的地方传递 Python 切片对象。传递的第一个值将是神经网络最早层的学习率，第二个值将是最后一层的学习率。中间的层将具有在整个范围内乘法等距的学习率。让我们使用这种方法来复制之前的训练，但这次我们只会将网络的最低​​层设置为 1e-6 的学习率；其他层将扩展到 1e-4。让我们训练一段时间，看看会发生什么：

Now the fine-tuning is working great!

fastai can show us a graph of the training and validation loss:

现在微调效果很好！

fastai 可以向我们展示训练和验证损失的图表：

As you can see, the training loss keeps getting better and better. But notice that eventually the validation loss improvement slows, and sometimes even gets worse! This is the point at which the model is starting to over fit. In particular, the model is becoming overconfident of its predictions. But this does not mean that it is getting less accurate, necessarily. Take a look at the table of training results per epoch, and you will often see that the accuracy continues improving, even as the validation loss gets worse. In the end what matters is your accuracy, or more generally your chosen metrics, not the loss. The loss is just the function we've given the computer to help us to optimize.

正如你所看到的，训练损失越来越好。但请注意，最终验证损失的改善会减慢，有时甚至会变得更糟！这是模型开始过度拟合的点。特别是，该模型对其预测变得过度自信。但这并不意味着它必然会变得不那么准确。看一下每个 epoch 的训练结果表，你会经常看到准确率继续提高，即使验证损失变得更糟。最后，重要的是您的准确性，或者更一般地说是您选择的指标，而不是损失。损失只是我们赋予计算机帮助我们优化的功能。

Another decision you have to make when training the model is for how long to train for. We'll consider that next.

训练模型时必须做出的另一个决定是训练多长时间。我们接下来会考虑。

Selecting the Number of Epochs

选择迭代数

Often you will find that you are limited by time, rather than generalization and accuracy, when choosing how many epochs to train for. So your first approach to training should be to simply pick a number of epochs that will train in the amount of time that you are happy to wait for. Then look at the training and validation loss plots, as shown above, and in particular your metrics, and if you see that they are still getting better even in your final epochs, then you know that you have not trained for too long.

On the other hand, you may well see that the metrics you have chosen are really getting worse at the end of training. Remember, it's not just that we're looking for the validation loss to get worse, but the actual metrics. Your validation loss will first get worse during training because the model gets overconfident, and only later will get worse because it is incorrectly memorizing the data. We only care in practice about the latter issue. Remember, our loss function is just something that we use to allow our optimizer to have something it can differentiate and optimize; it's not actually the thing we care about in practice.

Before the days of 1cycle training it was very common to save the model at the end of each epoch, and then select whichever model had the best accuracy out of all of the models saved in each epoch. This is known as early stopping. However, this is very unlikely to give you the best answer, because those epochs in the middle occur before the learning rate has had a chance to reach the small values, where it can really find the best result. Therefore, if you find that you have overfit, what you should actually do is retrain your model from scratch, and this time select a total number of epochs based on where your previous best results were found.

If you have the time to train for more epochs, you may want to instead use that time to train more parameters—that is, use a deeper architecture.

在选择要训练多少个 epoch 时，您通常会发现自己受到时间的限制，而不是泛化和准确性。因此，您的第一种训练方法应该是简单地选择一些 epoch，这些 epoch 将在您乐于等待的时间内进行训练。然后查看训练和验证损失图，如上所示，特别是您的指标，如果您发现即使在最后的 epoch 中它们仍然变得更好，那么您就知道您训练的时间太长了。

另一方面，您很可能会看到您选择的指标在训练结束时确实变得更糟。请记住，我们不仅要寻找验证损失变得更糟，还要寻找实际指标。您的验证损失首先会在训练期间变得更糟，因为模型变得过于自信，然后才会变得更糟，因为它错误地记住了数据。我们在实践中只关心后一个问题。请记住，我们的损失函数只是我们用来让我们的优化器拥有可以区分和优化的东西；这实际上并不是我们在实践中关心的事情。

在 1cycle 训练之前，在每个 epoch 结束时保存模型是很常见的，然后从每个 epoch 保存的所有模型中选择具有最佳准确性的模型。这被称为提前停止。然而，这不太可能给你最好的答案，因为中间的那些时期发生在学习率有机会达到小值之前，在那里它真的可以找到最好的结果。因此，如果你发现你有过拟合，你实际上应该做的是从头开始重新训练你的模型，这次根据你之前的最佳结果的位置来选择一个 epoch 的总数。

如果你有时间训练更多的 epoch，你可能想用这段时间来训练更多的参数——也就是说，使用更深的架构。

Deeper Architectures

更深层次的架构

In general, a model with more parameters can model your data more accurately. (There are lots and lots of caveats to this generalization, and it depends on the specifics of the architectures you are using, but it is a reasonable rule of thumb for now.) For most of the architectures that we will be seeing in this book, you can create larger versions of them by simply adding more layers. However, since we want to use pretrained models, we need to make sure that we choose a number of layers that have already been pretrained for us.

This is why, in practice, architectures tend to come in a small number of variants. For instance, the ResNet architecture that we are using in this chapter comes in variants with 18, 34, 50, 101, and 152 layer, pretrained on ImageNet. A larger (more layers and parameters; sometimes described as the "capacity" of a model) version of a ResNet will always be able to give us a better training loss, but it can suffer more from overfitting, because it has more parameters to overfit with.

In general, a bigger model has the ability to better capture the real underlying relationships in your data, and also to capture and memorize the specific details of your individual images.

However, using a deeper model is going to require more GPU RAM, so you may need to lower the size of your batches to avoid an out-of-memory error. This happens when you try to fit too much inside your GPU and looks like:

Cuda runtime error: out of memory

You may have to restart your notebook when this happens. The way to solve it is to use a smaller batch size, which means passing smaller groups of images at any given time through your model. You can pass the batch size you want to the call creating your DataLoaders with bs=.

The other downside of deeper architectures is that they take quite a bit longer to train. One technique that can speed things up a lot is mixed-precision training. This refers to using less-precise numbers (half-precision floating point, also called fp16) where possible during training. As we are writing these words in early 2020, nearly all current NVIDIA GPUs support a special feature called tensor cores that can dramatically speed up neural network training, by 2-3x. They also require a lot less GPU memory. To enable this feature in fastai, just add to_fp16() after your Learner creation (you also need to import the module).

You can't really know ahead of time what the best architecture for your particular problem is—you need to try training some. So let's try a ResNet-50 now with mixed precision:

通常，具有更多参数的模型可以更准确地对数据建模。 （这种概括有很多警告，它取决于您使用的体系结构的具体情况，但目前这是一个合理的经验法则。）对于我们将在本书中看到的大多数体系结构，您可以通过简单地添加更多层来创建它们的更大版本。但是，由于我们要使用预训练模型，我们需要确保选择一些已经为我们预训练的层。

这就是为什么在实践中，架构往往有少量变体。例如，我们在本章中使用的 ResNet 架构有 18、34、50、101 和 152 层的变体，在 ImageNet 上进行了预训练。更大（更多层和参数；有时被描述为模型的“容量”）版本的 ResNet 总是能够为我们提供更好的训练损失，但它可能会遭受更多的过拟合，因为它有更多的参数需要过拟合和。

一般来说，更大的模型能够更好地捕捉数据中真正的底层关系，还能捕捉和记忆各个图像的具体细节。

但是，使用更深的模型将需要更多的 GPU RAM，因此您可能需要降低批次的大小以避免内存不足错误。当您尝试在 GPU 中安装太多内容时会发生这种情况，并且看起来像：

Cuda runtime error: out of memory

发生这种情况时，您可能必须重新启动笔记本电脑。解决它的方法是使用更小的批量大小，这意味着在任何给定时间通过您的模型传递更小的图像组。您可以将所需的批量大小传递给使用 bs= 创建 DataLoaders 的调用。

更深层次架构的另一个缺点是它们需要更长的时间来训练。一种可以大大加快速度的技术是混合精度训练。这是指在训练期间尽可能使用精度较低的数字（半精度浮点，也称为 fp16）。当我们在 2020 年初写下这些话时，几乎所有当前的 NVIDIA GPU 都支持一种称为张量核心的特殊功能，该功能可以将神经网络训练速度显着提高 2-3 倍。它们还需要更少的 GPU 内存。要在 fastai 中启用此功能，只需在创建 Learner 后添加 to_fp16()（您还需要导入模块）。

您无法真正提前知道针对您的特定问题的最佳架构是什么——您需要尝试训练一些。所以现在让我们尝试一个混合精度的 ResNet-50：

You'll see here we've gone back to using fine_tune, since it's so handy! We can pass freeze_epochs to tell fastai how many epochs to train for while frozen. It will automatically change learning rates appropriately for most datasets.

In this case, we're not seeing a clear win from the deeper model. This is useful to remember—bigger models aren't necessarily better models for your particular case! Make sure you try small models before you start scaling up.

您会在这里看到我们已经回到使用fine_tune，因为它非常方便！我们可以通过 freeze_epochs 告诉 fastai 在冻结时要训练多少个 epoch。它会自动为大多数数据集适当地改变学习率。

在这种情况下，我们没有看到更深层次模型的明显胜利。记住这一点很有用——对于您的特定情况，更大的模型不一定是更好的模型！确保在开始扩大规模之前尝试小型模型。

Conclusion

结论

In this chapter you learned some important practical tips, both for getting your image data ready for modeling (presizing, data block summary) and for fitting the model (learning rate finder, unfreezing, discriminative learning rates, setting the number of epochs, and using deeper architectures). Using these tools will help you to build more accurate image models, more quickly.

We also discussed cross-entropy loss. This part of the book is worth spending plenty of time on. You aren't likely to need to actually implement cross-entropy loss from scratch yourself in practice, but it's really important you understand the inputs to and output from that function, because it (or a variant of it, as we'll see in the next chapter) is used in nearly every classification model. So when you want to debug a model, or put a model in production, or improve the accuracy of a model, you're going to need to be able to look at its activations and loss, and understand what's going on, and why. You can't do that properly if you don't understand your loss function.

If cross-entropy loss hasn't "clicked" for you just yet, don't worry—you'll get there! First, go back to the last chapter and make sure you really understand mnist_loss. Then work gradually through the cells of the notebook for this chapter, where we step through each piece of cross-entropy loss. Make sure you understand what each calculation is doing, and why. Try creating some small tensors yourself and pass them into the functions, to see what they return.

Remember: the choices made in the implementation of cross-entropy loss are not the only possible choices that could have been made. Just like when we looked at regression we could choose between mean squared error and mean absolute difference (L1). If you have other ideas for possible functions that you think might work, feel free to give them a try in this chapter's notebook! (Fair warning though: you'll probably find that the model will be slower to train, and less accurate. That's because the gradient of cross-entropy loss is proportional to the difference between the activation and the target, so SGD always gets a nicely scaled step for the weights.)

在本章中，您学习了一些重要的实用技巧，既可以让您的图像数据为建模做好准备（调整大小、数据块摘要），也可以用于拟合模型（学习率查找器、解冻、判别学习率、设置 epoch 数以及使用更深层次的架构）。使用这些工具将帮助您更快地构建更准确的图像模型。

我们还讨论了交叉熵损失。本书的这一部分值得花大量时间阅读。在实践中，您可能不需要自己从头开始实际实现交叉熵损失，但了解该函数的输入和输出非常重要，因为它（或它的变体，我们将在下一章）几乎用于每个分类模型。因此，当您想要调试模型、将模型投入生产或提高模型的准确性时，您将需要能够查看其激活和损失，并了解正在发生的事情以及原因。如果你不了解你的损失函数，你就不能正确地做到这一点。

如果你还没有“点击”交叉熵损失，别担心——你会成功的！首先，回到上一章，确保你真正理解了 mnist_loss。然后逐步完成本章的笔记本单元，我们将逐步完成每一个交叉熵损失。确保您了解每个计算在做什么，以及为什么。尝试自己创建一些小张量并将它们传递给函数，看看它们返回什么。

请记住：在实现交叉熵损失时所做的选择并不是唯一可能做出的选择。就像我们查看回归时一样，我们可以在均方误差和平均绝对差 (L1) 之间进行选择。如果您对可能有用的功能有其他想法，请随时在本章的笔记本中尝试一下！ （但公平的警告：您可能会发现模型训练速度较慢，准确度较低。这是因为交叉熵损失的梯度与激活和目标之间的差异成正比，因此 SGD 总是得到很好的权重的缩放步长。）

Questionnaire

问题

1. Why do we first resize to a large size on the CPU, and then to a smaller size on the GPU?
2. If you are not familiar with regular expressions, find a regular expression tutorial, and some problem sets, and complete them. Have a look on the book's website for suggestions.
3. What are the two ways in which data is most commonly provided, for most deep learning datasets?
4. Look up the documentation for L and try using a few of the new methods that it adds.
5. Look up the documentation for the Python pathlib module and try using a few methods of the Path class.
6. Give two examples of ways that image transformations can degrade the quality of the data.
7. What method does fastai provide to view the data in a DataLoaders?
8. What method does fastai provide to help you debug a DataBlock?
9. Should you hold off on training a model until you have thoroughly cleaned your data?
10. What are the two pieces that are combined into cross-entropy loss in PyTorch?
11. What are the two properties of activations that softmax ensures? Why is this important?
12. When might you want your activations to not have these two properties?
13. Calculate the exp and softmax columns of <<bear_softmax>> yourself (i.e., in a spreadsheet, with a calculator, or in a notebook).
14. Why can't we use torch.where to create a loss function for datasets where our label can have more than two categories?
15. What is the value of log(-2)? Why?
16. What are two good rules of thumb for picking a learning rate from the learning rate finder?
17. What two steps does the fine_tune method do?
18. In Jupyter Notebook, how do you get the source code for a method or function?
19. What are discriminative learning rates?
20. How is a Python slice object interpreted when passed as a learning rate to fastai?
21. Why is early stopping a poor choice when using 1cycle training?
22. What is the difference between resnet50 and resnet101?
23. What does to_fp16 do?

Further Research

1. Find the paper by Leslie Smith that introduced the learning rate finder, and read it.
2. See if you can improve the accuracy of the classifier in this chapter. What's the best accuracy you can achieve? Look on the forums and the book's website to see what other students have achieved with this dataset, and how they did it.