Other Computer Vision Problems

其他计算机视觉问题

In the previous chapter you learned some important practical techniques for training models in practice. Considerations like selecting learning rates and the number of epochs are very important to getting good results.

In this chapter we are going to look at two other types of computer vision problems: multi-label classification and regression. The first one is when you want to predict more than one label per image (or sometimes none at all), and the second is when your labels are one or several numbers—a quantity instead of a category.

In the process will study more deeply the output activations, targets, and loss functions in deep learning models.

在上一章中，你学到了一些在实践中训练模型的重要实用技术。像选择学习率和世代数这样的考虑对于获得好的结果非常重要。

在这一章中，我们要看看另外两种类型的计算机视觉问题：多标签分类和回归。第一种是当你想预测每幅图像的一个以上的标签时（或者有时根本没有），第二种是当你的标签是一个或几个数字--一个数量而不是一个类别。

在这个过程中，将更深入地研究深度学习模型中的输出激活、目标和损失函数。

Multi-Label Classification

多标签分类

Multi-label classification refers to the problem of identifying the categories of objects in images that may not contain exactly one type of object. There may be more than one kind of object, or there may be no objects at all in the classes that you are looking for.

For instance, this would have been a great approach for our bear classifier. One problem with the bear classifier that we rolled out in <<chapter_production>> was that if a user uploaded something that wasn't any kind of bear, the model would still say it was either a grizzly, black, or teddy bear—it had no ability to predict "not a bear at all." In fact, after we have completed this chapter, it would be a great exercise for you to go back to your image classifier application, and try to retrain it using the multi-label technique, then test it by passing in an image that is not of any of your recognized classes.

In practice, we have not seen many examples of people training multi-label classifiers for this purpose—but we very often see both users and developers complaining about this problem. It appears that this simple solution is not at all widely understood or appreciated! Because in practice it is probably more common to have some images with zero matches or more than one match, we should probably expect in practice that multi-label classifiers are more widely applicable than single-label classifiers.

First, let's see what a multi-label dataset looks like, then we'll explain how to get it ready for our model. You'll see that the architecture of the model does not change from the last chapter; only the loss function does. Let's start with the data.

多标签分类指的是识别图像中物体的类别问题，这些图像可能不完全包含一种类型的物体。可能有不止一种物体，也可能在你要找的类别中根本就没有物体。

例如，这将是我们的熊分类器的一个很棒的方法。我们在<<chapter_production>>中推出的熊分类器的一个问题是，如果用户上传的东西不是任何种类的熊，该模型仍然会说它是灰熊、黑熊或泰迪熊--它没有能力预测 "根本不是熊"。事实上，在我们完成这一章后，你可以回到你的图像分类器应用程序，并尝试使用多标签技术重新训练它，然后通过传入一张不属于任何公认类别的图像来测试它，这将是一个很好的练习。

在实践中，我们没有看到很多人们为此目的而训练多标签分类器的例子--但我们经常看到用户和开发者都在抱怨这个问题。看来，这个简单的解决方案根本没有得到广泛的理解和重视 因为在实践中，可能更常见的是有一些图像的匹配度为零或超过一个匹配度，我们也许应该期望在实践中多标签分类器比单标签分类器更广泛地适用。

首先，让我们看看多标签数据集是什么样子的，然后我们将解释如何让它为我们的模型做好准备。你会看到，模型的结构与上一章相比没有变化；只有损失函数有变化。让我们从数据开始。

The Data

数据

For our example we are going to use the PASCAL dataset, which can have more than one kind of classified object per image.

We begin by downloading and extracting the dataset as per usual:

在我们的例子中，我们将使用PASCAL数据集，该数据集在每张图片上可以有不止一种分类对象。

我们首先按照惯例下载并提取数据集。

This dataset is different from the ones we have seen before, in that it is not structured by filename or folder but instead comes with a CSV (comma-separated values) file telling us what labels to use for each image. We can inspect the CSV file by reading it into a Pandas DataFrame:

这个数据集与我们之前看到的数据集不同，因为它不是按文件名或文件夹结构的，而是附带一个CSV（逗号分隔的值）文件，告诉我们每张图片应该使用什么标签。我们可以通过将CSV文件读入Pandas DataFrame来检查它。

As you can see, the list of categories in each image is shown as a space-delimited string.

正如你所看到的，每张图片中的类别列表被显示为以空格分隔的字符串。

Sidebar: Pandas and DataFrames

题外话：Pandas和DataFrames

No, it’s not actually a panda! Pandas is a Python library that is used to manipulate and analyze tabular and time series data. The main class is DataFrame, which represents a table of rows and columns. You can get a DataFrame from a CSV file, a database table, Python dictionaries, and many other sources. In Jupyter, a DataFrame is output as a formatted table, as shown here.

You can access rows and columns of a DataFrame with the iloc property, as if it were a matrix:

不，它实际上不是一只熊猫! Pandas是一个Python库，用于操作和分析表格和时间序列数据。主要的类是DataFrame，它表示一个行和列的表格。你可以从CSV文件、数据库表、Python字典和许多其他来源获得一个DataFrame。在Jupyter中，DataFrame被输出为一个格式化的表格，如图所示。

你可以用iloc属性访问DataFrame的行和列，就像它是一个矩阵一样。

You can also grab a column by name by indexing into a DataFrame directly:

你也可以通过直接对DataFrame进行索引来抓取一个列的名称：

You can create new columns and do calculations using columns:

你可以创建新的列并使用列进行计算：

Pandas is a fast and flexible library, and an important part of every data scientist’s Python toolbox. Unfortunately, its API can be rather confusing and surprising, so it takes a while to get familiar with it. If you haven’t used Pandas before, we’d suggest going through a tutorial; we are particularly fond of the book Python for Data Analysis by Wes McKinney, the creator of Pandas (O'Reilly). It also covers other important libraries like matplotlib and numpy. We will try to briefly describe Pandas functionality we use as we come across it, but will not go into the level of detail of McKinney’s book.

Pandas是一个快速而灵活的库，是每个数据科学家的Python工具箱中的重要部分。不幸的是，它的API可能相当混乱和令人惊讶，所以需要花点时间来熟悉它。如果你以前没有使用过Pandas，我们建议你去看看教程；我们特别喜欢Pandas的创造者Wes McKinney写的《Python for Data Analysis》（O'Reilly）这本书。它还涵盖了其他重要的库，如matplotlib和numpy。我们将尝试简单描述我们所使用的Pandas功能，但不会像McKinney的书那样详细。

End sidebar

题外话结束

Now that we have seen what the data looks like, let's make it ready for model training.

现在我们已经看到了数据的模样，让我们为模型训练做好准备。

Constructing a DataBlock

构建数据块

How do we convert from a DataFrame object to a DataLoaders object? We generally suggest using the data block API for creating a DataLoaders object, where possible, since it provides a good mix of flexibility and simplicity. Here we will show you the steps that we take to use the data blocks API to construct a DataLoaders object in practice, using this dataset as an example.

As we have seen, PyTorch and fastai have two main classes for representing and accessing a training set or validation set:

• Dataset:: A collection that returns a tuple of your independent and dependent variable for a single item
• DataLoader:: An iterator that provides a stream of mini-batches, where each mini-batch is a tuple of a batch of independent variables and a batch of dependent variables

我们如何从 DataFrame对象转换为DataLoaders对象？我们通常建议尽可能使用数据块API来创建DataLoaders对象，因为它提供了一个灵活和简单的良好组合。这里我们将以这个数据集为例，向你展示我们在实践中使用数据块API构建DataLoaders对象的步骤。

正如我们所看到的，PyTorch和fastai有两个主要类来表示和访问训练集或验证集。

• Dataset::一个集合，用于返回单个项目的自变量和因变量的元组。

• DataLoader:: 一个迭代器，它提供了一个小型批次的流，其中每个小型批次是由一批自变量和一批因变量组成的一个元组。

On top of these, fastai provides two classes for bringing your training and validation sets together:

• Datasets:: An object that contains a training Dataset and a validation Dataset
• DataLoaders:: An object that contains a training DataLoader and a validation DataLoader

Since a DataLoader builds on top of a Dataset and adds additional functionality to it (collating multiple items into a mini-batch), it’s often easiest to start by creating and testing Datasets, and then look at DataLoaders after that’s working.

在这些基础上，fastai提供了两个类，用于将你的训练集和验证集结合起来:

• Datasets:: 一个包含训练Dataset和验证Dataset的对象。
• DataLoaders:: 一个包含训练DataLoader和验证DataLoader的对象。

由于DataLoader建立在Dataset之上，并为其增加了额外的功能（将多个项目整理成一个小批量），所以通常最简单的做法是先创建和测试Datasets，然后在工作结束后再看DataLoader的情况。

When we create a DataBlock, we build up gradually, step by step, and use the notebook to check our data along the way. This is a great way to make sure that you maintain momentum as you are coding, and that you keep an eye out for any problems. It’s easy to debug, because you know that if a problem arises, it is in the line of code you just typed!

Let’s start with the simplest case, which is a data block created with no parameters:

当我们创建一个DataBlock时，我们逐步建立起来，一步一步，并使用笔记本来检查我们的数据。这是一个很好的方法，可以确保你在编码时保持势头，并留意任何问题。这很容易调试，因为你知道如果出现问题，它就在你刚刚输入的那行代码中！我们从最简单的例子开始。

让我们从最简单的情况开始，也就是创建一个没有参数的数据块。

We can create a Datasets object from this. The only thing needed is a source—in this case, our DataFrame:

我们可以从中创建一个Datasets对象。唯一需要的是一个源--本例是我们的DataFrame。

This contains a train and a valid dataset, which we can index into:

这包含一个train数据集和一个valid数据集，我们可以对其进行索引:

As you can see, this simply returns a row of the DataFrame, twice. This is because by default, the data block assumes we have two things: input and target. We are going to need to grab the appropriate fields from the DataFrame, which we can do by passing get_x and get_y functions:

正如你所看到的，这只是简单地返回DataFrame的一行，两次。这是因为在默认情况下，数据块假设我们有两样东西：输入和目标。我们将需要从DataFrame中抓取适当的字段，我们可以通过传递get_x和get_y函数来实现。

As you can see, rather than defining a function in the usual way, we are using Python’s lambda keyword. This is just a shortcut for defining and then referring to a function. The following more verbose approach is identical:

正如你所看到的，我们不是以通常的方式定义一个函数，而是使用 Python 的 lambda 关键字。这只是一个定义然后引用一个函数的快捷方式。下面这种更粗略的方法是相同的:

Lambda functions are great for quickly iterating, but they are not compatible with serialization, so we advise you to use the more verbose approach if you want to export your Learner after training (lambdas are fine if you are just experimenting).

Lambda函数很适合快速迭代，但它与序列化不兼容，所以如果你想在训练后导出Leaner，我们建议你使用更粗略的方法（如果你只是实验，lambdas是可以的）。

We can see that the independent variable will need to be converted into a complete path, so that we can open it as an image, and the dependent variable will need to be split on the space character (which is the default for Python’s split function) so that it becomes a list:

我们可以看到自变量需要转换为一个完整的路径，这样我们就可以把它作为一个图像打开，而因变量需要在空格字符上进行分割 (这是 Python 的split函数的默认值)，这样它就成为一个列表:

To actually open the image and do the conversion to tensors, we will need to use a set of transforms; block types will provide us with those. We can use the same block types that we have used previously, with one exception: the ImageBlock will work fine again, because we have a path that points to a valid image, but the CategoryBlock is not going to work. The problem is that block returns a single integer, but we need to be able to have multiple labels for each item. To solve this, we use a MultiCategoryBlock. This type of block expects to receive a list of strings, as we have in this case, so let’s test it out:

为了实际打开图像并进行张量的转换，我们需要使用一组变换；块类型将为我们提供这些变换。我们可以使用之前使用过的相同的块类型，但有一个例外：ImageBlock将再次正常工作，因为我们有一个指向有效图像的路径，但CategoryBlock却无法工作。问题在于那个块返回的是一个整数，但我们需要为每个项目有多个标签。为了解决这个问题，我们使用一个MultiCategoryBlock。这种类型的块期望接收一个字符串的列表，就像我们在这个例子中一样，所以让我们来测试一下:

As you can see, our list of categories is not encoded in the same way that it was for the regular CategoryBlock. In that case, we had a single integer representing which category was present, based on its location in our vocab. In this case, however, we instead have a list of zeros, with a one in any position where that category is present. For example, if there is a one in the second and fourth positions, then that means that vocab items two and four are present in this image. This is known as one-hot encoding. The reason we can’t easily just use a list of category indices is that each list would be a different length, and PyTorch requires tensors, where everything has to be the same length.

正如你所看到的，我们的类别列表的编码方式与普通CategoryBlock的编码方式不一样。在这种情况下，我们有一个单一的整数，根据它在我们词汇表中的位置，代表哪个类别的存在。然而，在这种情况下，我们有一个零的列表，在该类别存在的任何位置有一个一。例如，如果在第二和第四个位置有一个一，那么这意味着词汇表中的第二和第四项存在于这幅图像中。这就是所谓的one-hot编码。我们不能轻易地使用类别索引的列表，原因是每个列表的长度不同，而PyTorch需要的是张量，所有东西都必须是相同的长度。

jargon: One-hot encoding: Using a vector of zeros, with a one in each location that is represented in the data, to encode a list of integers.

jargon: One-hot 编码：使用一个零的向量，在数据中表示的每个位置都有一个一，来编码一个整数的列表。

Let’s check what the categories represent for this example (we are using the convenient torch.where function, which tells us all of the indices where our condition is true or false):

让我们检查一下这个例子中的类别代表什么（我们使用方便的torch.where函数，它告诉我们条件为真或假的所有指数）。

With NumPy arrays, PyTorch tensors, and fastai’s L class, we can index directly using a list or vector, which makes a lot of code (such as this example) much clearer and more concise.

We have ignored the column is_valid up until now, which means that DataBlock has been using a random split by default. To explicitly choose the elements of our validation set, we need to write a function and pass it to splitter (or use one of fastai's predefined functions or classes). It will take the items (here our whole DataFrame) and must return two (or more) lists of integers:

通过NumPy数组、PyTorch张量和fastai的L类，我们可以直接使用列表或向量进行索引，这使得很多代码（比如这个例子）更加清晰和简洁。

到目前为止，我们一直忽略了is_valid这一列，这意味着DataBlock一直在使用默认的随机分割。为了明确地选择我们的验证集的元素，我们需要写一个函数并把它传递给splitter（或者使用fastai的一个预定义函数或类）。它将接收项目（这里是我们的整个DataFrame），并且必须返回两个（或更多）整数的列表。

As we have discussed, a DataLoader collates the items from a Dataset into a mini-batch. This is a tuple of tensors, where each tensor simply stacks the items from that location in the Dataset item.

Now that we have confirmed that the individual items look okay, there's one more step we need to ensure we can create our DataLoaders, which is to ensure that every item is of the same size. To do this, we can use RandomResizedCrop:

正如我们所讨论的，DataLoader将Dataset中的项整理成一个小型批次。这是一个张量的元组，其中每个张量只是将Dataset项目中的那个位置的项目堆叠起来。

现在我们已经确认各个项目看起来没有问题了，我们还需要一个步骤来确保我们可以创建我们的DataLoaders，也就是确保每个项目的大小都是一样的。要做到这一点，我们可以使用RandomResizedCrop:

And now we can display a sample of our data:

现在我们可以显示我们的数据样本了：

Remember that if anything goes wrong when you create your DataLoaders from your DataBlock, or if you want to view exactly what happens with your DataBlock, you can use the summary method we presented in the last chapter.

记住，如果你在从你的DataBlock创建你的DataLoaders时出了什么问题，或者你想查看你的DataBlock到底发生了什么，你可以使用我们在上一章介绍的summary方法。

Our data is now ready for training a model. As we will see, nothing is going to change when we create our Learner, but behind the scenes, the fastai library will pick a new loss function for us: binary cross-entropy.

我们的数据现在已经准备好训练一个模型了。正如我们将看到的，当我们创建Leaner时，什么都不会改变，但在幕后，fastai库将为我们选择一个新的损失函数：二进制交叉熵。

Binary Cross-Entropy

二进制交叉熵

Now we'll create our Learner. We saw in <<chapter_mnist_basics>> that a Learner object contains four main things: the model, a DataLoaders object, an Optimizer, and the loss function to use. We already have our DataLoaders, we can leverage fastai's resnet models (which we'll learn how to create from scratch later), and we know how to create an SGD optimizer. So let's focus on ensuring we have a suitable loss function. To do this, let's use vision_learner to create a Learner, so we can look at its activations:

现在我们将创建我们的学习者。我们在<>中看到，一个Leaner对象包含四个主要部分：模型、DataLoaders对象、优化器和要使用的损失函数。我们已经有了DataLoaders，我们可以利用fastai的resnet模型（稍后我们将学习如何从头开始创建），而且我们知道如何创建SGD优化器。因此，让我们专注于确保我们有一个合适的损失函数。为了做到这一点，让我们使用visual_learner来创建一个Leaner，这样我们就可以看一下它的激活情况。

We also saw that the model in a Learner is generally an object of a class inheriting from nn.Module, and that we can call it using parentheses and it will return the activations of a model. You should pass it your independent variable, as a mini-batch. We can try it out by grabbing a mini batch from our DataLoader and then passing it to the model:

我们还看到，Leaner中的模型通常是一个继承自 nn.Module 的类的对象，我们可以用括号来调用它，它将返回模型的激活情况。你应该把你的自变量传给它，作为一个小批处理。我们可以通过从我们的DataLoader中抓取一个迷你批处理，然后将其传递给模型来进行尝试:

Think about why activs has this shape—we have a batch size of 64, and we need to calculate the probability of each of 20 categories. Here’s what one of those activations looks like:

想一想为什么activs有这样的形状--我们有一个64的批处理量，我们需要计算20个类别中每个类别的概率。下面是其中一个激活的样子:

note: Getting Model Activations: Knowing how to manually get a mini-batch and pass it into a model, and look at the activations and loss, is really important for debugging your model. It is also very helpful for learning, so that you can see exactly what is going on.

注意：获取模型的激活:知道如何手动获取迷你批次并将其传入模型，并查看激活和损失，对于调试你的模型非常重要。它对学习也很有帮助，这样你就可以看到到底发生了什么。

They aren’t yet scaled to between 0 and 1, but we learned how to do that in <<chapter_mnist_basics>>, using the sigmoid function. We also saw how to calculate a loss based on this—this is our loss function from <<chapter_mnist_basics>>, with the addition of log as discussed in the last chapter:

它们还没有被缩放到0和1之间，但是我们在<<chapter_mnist_basics>>中学习了如何使用sigmoid函数来做到这一点。我们还看到了如何在此基础上计算损失--这就是我们在<<chapter_mnist_basics>>中的损失函数，加上了上一章讨论的log。

Note that because we have a one-hot-encoded dependent variable, we can't directly use nll_loss or softmax (and therefore we can't use cross_entropy):

• softmax, as we saw, requires that all predictions sum to 1, and tends to push one activation to be much larger than the others (due to the use of exp); however, we may well have multiple objects that we're confident appear in an image, so restricting the maximum sum of activations to 1 is not a good idea. By the same reasoning, we may want the sum to be less than 1, if we don't think any of the categories appear in an image.
• nll_loss, as we saw, returns the value of just one activation: the single activation corresponding with the single label for an item. This doesn't make sense when we have multiple labels.

On the other hand, the binary_cross_entropy function, which is just mnist_loss along with log, provides just what we need, thanks to the magic of PyTorch's elementwise operations. Each activation will be compared to each target for each column, so we don't have to do anything to make this function work for multiple columns.

请注意，由于我们有一个One-hot编码的因变量，我们不能直接使用nll_loss或softmax（因此我们也不能使用cross_entropy）:

• 正如我们所看到的，softmax要求所有预测的总和为1，并倾向于推动一个激活值比其他激活值大得多（由于使用exp）；然而，我们很可能有多个我们确信出现在图像中的对象，所以将激活值的最大总和限制在1不是一个好主意。根据同样的推理，如果我们不认为任何类别的物体出现在图像中，我们可能希望总和小于1。
• nll_loss，正如我们所看到的，只返回一个激活的值：一个项目的单一标签所对应的单一激活。当我们有多个标签时，这就没有意义了。

另一方面，binary_cross_entropy函数，也就是mnist_loss加上log，正好提供了我们需要的东西，这要归功于PyTorch的元素操作的魔力。每一个激活将与每一列的目标进行比较，所以我们不需要做任何事情来使这个函数适用于多列。

j: One of the things I really like about working with libraries like PyTorch, with broadcasting and elementwise operations, is that quite frequently I find I can write code that works equally well for a single item or a batch of items, without changes. binary_cross_entropy is a great example of this. By using these operations, we don't have to write loops ourselves, and can rely on PyTorch to do the looping we need as appropriate for the rank of the tensors we're working with.

j: 我非常喜欢与PyTorch这样的库合作，使用广播和元素操作，其中一个原因是，我经常发现我可以写出对单个项目或一批项目同样有效的代码，而无需改变。binary_cross_entropy是这方面的一个很好的例子。通过使用这些操作，我们不必自己编写循环，而可以依靠PyTorch根据我们所处理的张量的等级来做我们需要的循环。

PyTorch already provides this function for us. In fact, it provides a number of versions, with rather confusing names!

F.binary_cross_entropy and its module equivalent nn.BCELoss calculate cross-entropy on a one-hot-encoded target, but do not include the initial sigmoid. Normally for one-hot-encoded targets you'll want F.binary_cross_entropy_with_logits (or nn.BCEWithLogitsLoss), which do both sigmoid and binary cross-entropy in a single function, as in the preceding example.

The equivalent for single-label datasets (like MNIST or the Pet dataset), where the target is encoded as a single integer, is F.nll_loss or nn.NLLLoss for the version without the initial softmax, and F.cross_entropy or nn.CrossEntropyLoss for the version with the initial softmax.

Since we have a one-hot-encoded target, we will use BCEWithLogitsLoss:

PyTorch已经为我们提供了这个功能。事实上，它提供了许多版本，名字也相当混乱! F.binary_cross_entropy和它的等价模块nn.BCELoss计算单次编码目标的交叉熵，但不包括初始sigmoid。通常情况下，对于单次编码的目标，你需要F.binary_cross_entropy_with_logits（或nn.BCEWithLogitsLoss），它在一个函数中同时做sigmoid和二进制交叉熵，如前面的例子。 对于单标签数据集（如MNIST或宠物数据集），目标被编码为一个整数，其等价物是F.nl_loss或nn.NLLLoss，用于没有初始softmax的版本，而F.cross_entropy或nn.CrossEntropyLoss用于有初始softmax的版本。 由于我们有一个单次编码的目标，我们将使用BCEWithLogitsLoss:

We don't actually need to tell fastai to use this loss function (although we can if we want) since it will be automatically chosen for us. fastai knows that the DataLoaders has multiple category labels, so it will use nn.BCEWithLogitsLoss by default.

One change compared to the last chapter is the metric we use: because this is a multilabel problem, we can't use the accuracy function. Why is that? Well, accuracy was comparing our outputs to our targets like so:

def accuracy(inp, targ, axis=-1):
    "Compute accuracy with `targ` when `pred` is bs * n_classes"
    pred = inp.argmax(dim=axis)
    return (pred == targ).float().mean()

The class predicted was the one with the highest activation (this is what argmax does). Here it doesn't work because we could have more than one prediction on a single image. After applying the sigmoid to our activations (to make them between 0 and 1), we need to decide which ones are 0s and which ones are 1s by picking a threshold. Each value above the threshold will be considered as a 1, and each value lower than the threshold will be considered a 0:

def accuracy_multi(inp, targ, thresh=0.5, sigmoid=True):
    "Compute accuracy when `inp` and `targ` are the same size."
    if sigmoid: inp = inp.sigmoid()
    return ((inp>thresh)==targ.bool()).float().mean()

我们实际上不需要告诉fastai使用这个损失函数（尽管如果我们想的话可以），因为它会自动为我们选择。fastai知道DataLoaders有多个类别标签，所以它将默认使用nn.BCEWithLogitsLoss。

与上一章相比，一个变化是我们使用的指标：因为这是一个多标签问题，所以我们不能使用精度函数。这是为什么呢？嗯，准确度是将我们的输出与我们的目标进行比较，像这样:

def accuracy(inp, targ, axis=-1):
    "Compute accuracy with `targ` when `pred` is bs * n_classes"
    pred = inp.argmax(dim=axis)
    return (pred == targ).float().mean()

预测的类别是具有最高激活度的类别（这就是argmax的作用）。在这里它不起作用，因为我们可以在一个图像上有多个预测。在对我们的激活值应用sigmoid之后（使它们在0和1之间），我们需要通过选择一个*阈值来决定哪些是0，哪些是1。每个高于阈值的值将被视为1，而每个低于阈值的值将被视为0。

def accuracy_multi(inp, targ, thresh=0.5, sigmoid=True):
    "Compute accuracy when `inp` and `targ` are the same size."
    if sigmoid: inp = inp.sigmoid()
    return ((inp>thresh)==targ.bool()).float().mean()

If we pass accuracy_multi directly as a metric, it will use the default value for threshold, which is 0.5. We might want to adjust that default and create a new version of accuracy_multi that has a different default. To help with this, there is a function in Python called partial. It allows us to bind a function with some arguments or keyword arguments, making a new version of that function that, whenever it is called, always includes those arguments. For instance, here is a simple function taking two arguments:

如果我们直接将accuracy_multi作为一个指标传递，它将使用threshold的默认值，也就是0.5。我们可能想调整这个默认值，并创建一个具有不同默认值的新版本的 accuracy_multi。为了帮助解决这个问题，Python 中有一个叫做partial的函数。它允许我们用一些参数或关键字参数来绑定一个函数，使该函数的一个新版本，无论何时被调用，总是包括这些参数。例如，这里有一个带两个参数的简单函数:

We can switch to a French version of that function by using partial:

我们可以通过使用partial切换到该函数的法语版本。

We can now train our model. Let's try setting the accuracy threshold to 0.2 for our metric:

我们现在可以训练我们的模型了。让我们尝试将准确率阈值设置为0.2，以衡量我们的指标:

Picking a threshold is important. If you pick a threshold that's too low, you'll often be failing to select correctly labeled objects. We can see this by changing our metric, and then calling validate, which returns the validation loss and metrics:

挑选一个阈值很重要。如果你选的阈值太低，你就会经常无法选择正确标记的对象。我们可以通过改变我们的度量值，然后调用validate，返回验证的损失和度量值来看到这一点:

If you pick a threshold that's too high, you'll only be selecting the objects for which your model is very confident:

如果你选择一个太高的阈值，你将只选择你的模型非常有信心的对象:

We can find the best threshold by trying a few levels and seeing what works best. This is much faster if we just grab the predictions once:

我们可以通过尝试几个级别，看看什么最有效来找到最佳阈值。如果我们只是抓取一次预测结果，这要快得多:

Then we can call the metric directly. Note that by default get_preds applies the output activation function (sigmoid, in this case) for us, so we'll need to tell accuracy_multi to not apply it:

然后我们就可以直接调用这个度量。注意，默认情况下，get_preds为我们应用了输出激活函数（本例中为sigmoid），所以我们需要告诉accuracy_multi不要应用它:

We can now use this approach to find the best threshold level:

我们现在可以用这种方法来寻找最佳的阈值水平:

In this case, we're using the validation set to pick a hyperparameter (the threshold), which is the purpose of the validation set. Sometimes students have expressed their concern that we might be overfitting to the validation set, since we're trying lots of values to see which is the best. However, as you see in the plot, changing the threshold in this case results in a smooth curve, so we're clearly not picking some inappropriate outlier. This is a good example of where you have to be careful of the difference between theory (don't try lots of hyperparameter values or you might overfit the validation set) versus practice (if the relationship is smooth, then it's fine to do this).

This concludes the part of this chapter dedicated to multi-label classification. Next, we'll take a look at a regression problem.

在这种情况下，我们使用验证集来选择一个超参数（阈值），这就是验证集的目的。有时候，学生们表示担心，我们可能会对验证集进行过度拟合，因为我们正在尝试很多值，看看哪个是最好的。然而，正如你在图中所看到的，在这种情况下改变阈值的结果是一个平滑的曲线，所以我们显然没有选择一些不合适的离群点。这是一个很好的例子，说明你必须注意理论（不要尝试很多超参数值，否则你可能会过度拟合验证集）和实践（如果关系是平滑的，那么这样做是没有问题的）之间的区别。

本章专门讨论多标签分类的部分到此结束。接下来，我们将看一下回归问题。

Regression

回归

It's easy to think of deep learning models as being classified into domains, like computer vision, NLP, and so forth. And indeed, that's how fastai classifies its applications—largely because that's how most people are used to thinking of things.

But really, that's hiding a more interesting and deeper perspective. A model is defined by its independent and dependent variables, along with its loss function. That means that there's really a far wider array of models than just the simple domain-based split. Perhaps we have an independent variable that's an image, and a dependent that's text (e.g., generating a caption from an image); or perhaps we have an independent variable that's text and dependent that's an image (e.g., generating an image from a caption—which is actually possible for deep learning to do!); or perhaps we've got images, texts, and tabular data as independent variables, and we're trying to predict product purchases... the possibilities really are endless.

To be able to move beyond fixed applications, to crafting your own novel solutions to novel problems, it helps to really understand the data block API (and maybe also the mid-tier API, which we'll see later in the book). As an example, let's consider the problem of image regression. This refers to learning from a dataset where the independent variable is an image, and the dependent variable is one or more floats. Often we see people treat image regression as a whole separate application—but as you'll see here, we can treat it as just another CNN on top of the data block API.

We're going to jump straight to a somewhat tricky variant of image regression, because we know you're ready for it! We're going to do a key point model. A key point refers to a specific location represented in an image—in this case, we'll use images of people and we'll be looking for the center of the person's face in each image. That means we'll actually be predicting two values for each image: the row and column of the face center.

人们很容易认为深度学习模型被划分为不同的领域，如计算机视觉、NLP等等。事实上，fastai也是这样对其应用进行分类的--主要是因为大多数人都习惯于这样思考问题。

但实际上，这隐藏了一个更有趣、更深刻的观点。一个模型是由它的自变量和因变量，以及它的损失函数定义的。这意味着，真正的模型有一个更广泛的阵列，而不仅仅是简单的基于领域的分割。也许我们的自变量是图像，而因变量是文本（例如，从图像中生成标题）；或者我们的自变量是文本，而因变量是图像（例如，从标题生成图像--这实际上是深度学习可以做到的！）；或者我们有图像、文本和表格数据作为自变量，而我们试图预测产品的购买...可能性真的是无穷的。

为了能够超越固定的应用，为新的问题制定自己的新的解决方案，有助于真正理解数据块API（也许还有中间层API，我们在本书后面会看到）。作为一个例子，让我们考虑图像回归的问题。这是指从数据集中学习，自变量是图像，因变量是一个或多个浮点。我们经常看到人们把图像回归当作一个独立的应用程序，但是正如你在这里看到的，我们可以把它当作数据块API上面的另一个CNN。

我们将直接跳到图像回归的一个有点棘手的变体，因为我们知道你已经准备好了 我们要做一个关键点模型。关键点是指图像中的一个特定位置--在这种情况下，我们将使用人的图像，我们将在每张图像中寻找人脸的中心位置。这意味着我们实际上要为每张图片预测两个值：脸部中心的行和列。

Assemble the Data

汇编数据

We will use the Biwi Kinect Head Pose dataset for this section. We'll begin by downloading the dataset as usual:

我们将在本节中使用Biwi Kinect头部姿势数据集。我们将像往常一样开始下载数据集：

Let's see what we've got!

让我们看看我们得到了什么!

There are 24 directories numbered from 01 to 24 (they correspond to the different people photographed), and a corresponding .obj file for each (we won't need them here). Let's take a look inside one of these directories:

有24个目录，编号从01到24（它们对应于拍摄的不同人物），每个目录都有相应的.obj文件（我们在这里不需要它们）。让我们来看看这些目录中的一个。

Inside the subdirectories, we have different frames, each of them come with an image (_rgb.jpg) and a pose file (_pose.txt). We can easily get all the image files recursively with get_image_files, then write a function that converts an image filename to its associated pose file:

在这些子目录中，我们有不同的帧，每个帧都有一个图像（_rgb.jpg）和一个姿势文件（_pose.txt）。我们可以很容易地用get_image_files递归地获得所有的图像文件，然后写一个函数将图像文件名转换为相关的姿势文件:

Let's take a look at our first image:

让我们看一下我们的第一张图片:

The Biwi dataset website used to explain the format of the pose text file associated with each image, which shows the location of the center of the head. The details of this aren't important for our purposes, so we'll just show the function we use to extract the head center point:

Biwi数据集网站用来解释与每个图像相关的姿势文本文件的格式，它显示了头部中心的位置。这方面的细节对我们的目的并不重要，所以我们只展示我们用来提取头部中心点的函数。

This function returns the coordinates as a tensor of two items:

该函数以两个项目的张量形式返回坐标:

We can pass this function to DataBlock as get_y, since it is responsible for labeling each item. We'll resize the images to half their input size, just to speed up training a bit.

One important point to note is that we should not just use a random splitter. The reason for this is that the same people appear in multiple images in this dataset, but we want to ensure that our model can generalize to people that it hasn't seen yet. Each folder in the dataset contains the images for one person. Therefore, we can create a splitter function that returns true for just one person, resulting in a validation set containing just that person's images.

The only other difference from the previous data block examples is that the second block is a PointBlock. This is necessary so that fastai knows that the labels represent coordinates; that way, it knows that when doing data augmentation, it should do the same augmentation to these coordinates as it does to the images:

我们可以把这个函数作为get_y传给DataBlock，因为它负责给每个项目贴标签。我们将把图片的大小调整为其输入尺寸的一半，只是为了加快训练速度。

需要注意的一点是，我们不应该只是使用一个随机的分割器。这样做的原因是，同样的人出现在这个数据集中的多张图片中，但我们要确保我们的模型能够概括到它还没有看到的人。数据集中的每个文件夹都包含一个人的图像。因此，我们可以创建一个分割函数，只对一个人返回真，从而产生一个只包含该人图像的验证集。

与之前的数据块例子唯一不同的是，第二个数据块是一个PointBlock。这是必要的，这样fastai就知道标签代表坐标；这样，它就知道在做数据增强时，应该对这些坐标做同样的增强，就像它对图像做的一样:

important: Points and Data Augmentation: We're not aware of other libraries (except for fastai) that automatically and correctly apply data augmentation to coordinates. So, if you're working with another library, you may need to disable data augmentation for these kinds of problems.

重要的是：点和数据增强:我们不知道其他库（除了fastai）是否能自动并正确地将数据增强应用于坐标。因此，如果你正在使用其他库，你可能需要为这类问题禁用数据增强。

Before doing any modeling, we should look at our data to confirm it seems okay:

在做任何建模之前，我们应该看一下我们的数据，以确认它似乎没有问题:

That's looking good! As well as looking at the batch visually, it's a good idea to also look at the underlying tensors (especially as a student; it will help clarify your understanding of what your model is really seeing):

这看起来不错! 除了从视觉上看batch，最好也看一下底层的张量（尤其是作为一个学生；这将有助于澄清你对你的模型真正看到的东西的理解）:

Make sure that you understand why these are the shapes for our mini-batches.

确保你理解为什么这些是我们的mini-batches的形状。

Here's an example of one row from the dependent variable:

下面是一个因变量的一行例子:

As you can see, we haven't had to use a separate image regression application; all we've had to do is label the data, and tell fastai what kinds of data the independent and dependent variables represent.

正如你所看到的，我们没有必要使用一个单独的图像回归程序；我们所要做的就是标记数据，并告诉fastai自变量和因变量代表哪种数据。

It's the same for creating our Learner. We will use the same function as before, with one new parameter, and we will be ready to train our model.

创建我们的Learner也是如此。我们将使用与之前相同的函数，加上一个新的参数，然后我们就可以训练我们的模型了。

Training a Model

训练一个模型

As usual, we can use vision_learner to create our Learner. Remember way back in <<chapter_intro>> how we used y_range to tell fastai the range of our targets? We'll do the same here (coordinates in fastai and PyTorch are always rescaled between -1 and +1):

像往常一样，我们可以使用vision_learner来创建我们的Learner。还记得在<<chapter_intro>>中我们是如何使用y_range来告诉fastai我们目标的范围的吗？我们在这里也要这样做（fastai和PyTorch中的坐标总是在-1和+1之间重新缩放）。

y_range is implemented in fastai using sigmoid_range, which is defined as:

y_range在fastai中使用sigmoid_range实现，其定义如下:

This is set as the final layer of the model, if y_range is defined. Take a moment to think about what this function does, and why it forces the model to output activations in the range (lo,hi).

Here's what it looks like:

如果定义了y_range，这将被设置为模型的最后一层。花点时间想一想这个函数是干什么的，以及为什么它迫使模型在(lo,hi)范围内输出激活。

下面是它的样子:

We didn't specify a loss function, which means we're getting whatever fastai chooses as the default. Let's see what it picked for us:

我们没有指定损失函数，这意味着我们得到的是fastai选择的任何默认函数。让我们看看它为我们选择了什么:

This makes sense, since when coordinates are used as the dependent variable, most of the time we're likely to be trying to predict something as close as possible; that's basically what MSELoss (mean squared error loss) does. If you want to use a different loss function, you can pass it to vision_learner using the loss_func parameter.

Note also that we didn't specify any metrics. That's because the MSE is already a useful metric for this task (although it's probably more interpretable after we take the square root).

We can pick a good learning rate with the learning rate finder:

这是有道理的，因为当坐标被用作因变量时，大多数时候我们可能会试图预测一些尽可能接近的东西；这基本上就是MSELoss（平均平方误差损失）的作用。如果你想使用一个不同的损失函数，你可以使用loss_func参数将其传递给vision_learner。

还请注意，我们没有指定任何指标。这是因为MSE对于这个任务来说已经是一个有用的指标了（尽管在我们取了平方根之后，它可能更容易解释）。

我们可以用学习率搜索器挑选一个好的学习率:

We'll try an LR of 1e-2:

我们将尝试1e-2的LR:

Generally when we run this we get a loss of around 0.0001, which corresponds to an average coordinate prediction error of:

一般来说，当我们运行这个程序时，我们得到的损失约为0.0001，这相当于平均坐标预测误差为:

This sounds very accurate! But it's important to take a look at our results with Learner.show_results. The left side are the actual (ground truth) coordinates and the right side are our model's predictions:

这听起来非常准确！但是，我们必须看看Learner.show_results的结果。但重要的是，我们要用Learner.show_results来看一下我们的结果。左边是实际的（地面真相）坐标，右边是我们模型的预测结果。

It's quite amazing that with just a few minutes of computation we've created such an accurate key points model, and without any special domain-specific application. This is the power of building on flexible APIs, and using transfer learning! It's particularly striking that we've been able to use transfer learning so effectively even between totally different tasks; our pretrained model was trained to do image classification, and we fine-tuned for image regression.

仅仅几分钟的计算，我们就创建了一个如此精确的关键点模型，而且没有任何特殊领域的应用，这真是令人惊讶。这就是建立在灵活的API上并使用转移学习的力量! 特别引人注目的是，我们能够在完全不同的任务之间如此有效地使用迁移学习；我们的预训练模型被训练来做图像分类，而我们为图像回归进行了微调。

Conclusion

总结

In problems that are at first glance completely different (single-label classification, multi-label classification, and regression), we end up using the same model with just different numbers of outputs. The loss function is the one thing that changes, which is why it's important to double-check that you are using the right loss function for your problem.

fastai will automatically try to pick the right one from the data you built, but if you are using pure PyTorch to build your DataLoaders, make sure you think hard when you have to decide on your choice of loss function, and remember that you most probably want:

• nn.CrossEntropyLoss for single-label classification
• nn.BCEWithLogitsLoss for multi-label classification
• nn.MSELoss for regression

在那些乍一看完全不同的问题中（单标签分类、多标签分类和回归），我们最终会使用相同的模型，只是输出的数量不同。损失函数是一个变化的东西，这就是为什么要反复检查你是否为你的问题使用了正确的损失函数。

fastai会自动尝试从你建立的数据中挑选出正确的函数，但是如果你使用纯粹的PyTorch来建立你的DataLoader s，当你必须决定损失函数的选择时，请确保你认真思考，并记住你很可能想要。

• nn.CrossEntropyLoss用于单标签分类
• nn.BCEWithLogitsLoss用于多标签分类
• nn.MSELoss用于回归

Questionnaire

问卷调查

1. How could multi-label classification improve the usability of the bear classifier?
2. How do we encode the dependent variable in a multi-label classification problem?
3. How do you access the rows and columns of a DataFrame as if it was a matrix?
4. How do you get a column by name from a DataFrame?
5. What is the difference between a Dataset and DataLoader?
6. What does a Datasets object normally contain?
7. What does a DataLoaders object normally contain?
8. What does lambda do in Python?
9. What are the methods to customize how the independent and dependent variables are created with the data block API?
10. Why is softmax not an appropriate output activation function when using a one hot encoded target?
11. Why is nll_loss not an appropriate loss function when using a one-hot-encoded target?
12. What is the difference between nn.BCELoss and nn.BCEWithLogitsLoss?
13. Why can't we use regular accuracy in a multi-label problem?
14. When is it okay to tune a hyperparameter on the validation set?
15. How is y_range implemented in fastai? (See if you can implement it yourself and test it without peeking!)
16. What is a regression problem? What loss function should you use for such a problem?
17. What do you need to do to make sure the fastai library applies the same data augmentation to your input images and your target point coordinates?

1. 多标签分类如何能提高熊分类器的实用性？
2. 我们如何在多标签分类问题中对因变量进行编码？
3. 如何像访问矩阵一样访问DataFrame的行和列？
4. 如何从DataFrame中按名称获得一列？
5. Dataset和DataLoader之间的区别是什么？
6. 一个Datasets对象通常包含什么？
7. 一个DataLoaders对象通常包含哪些内容？
8. lambda在Python中的作用是什么？
9. 用数据块API自定义自变量和因变量的创建方式的方法有哪些？
10. 当使用一个one-hot目标时，为什么softmax不是一个合适的输出激活函数？
11. 为什么在使用一one-hot编码目标时，nll_loss不是一个合适的损失函数？
12. nn.BCELoss 和 nn.BCEWithLogitsLoss 之间有什么区别？
13. 为什么我们不能在多标签问题中使用常规精度？
14. 什么时候可以在验证集上调整一个超参数？
15. y_range是如何在fastai中实现的？（看看你是否能自己实现它，并在不偷看的情况下测试它！）。
16. 什么是回归问题？对于这样的问题你应该使用什么损失函数？
17. 你需要做什么来确保fastai库对你的输入图像和你的目标点坐标应用相同的数据增强？

Further Research

进一步研究

1. Read a tutorial about Pandas DataFrames and experiment with a few methods that look interesting to you. See the book's website for recommended tutorials.
2. Retrain the bear classifier using multi-label classification. See if you can make it work effectively with images that don't contain any bears, including showing that information in the web application. Try an image with two different kinds of bears. Check whether the accuracy on the single-label dataset is impacted using multi-label classification.

1. 阅读关于Pandas DataFrames的教程，并实验一些你看起来很感兴趣的方法。关于推荐的教程，请看本书的网站。
2. 使用多标签分类法重新训练熊的分类器。看看你是否能让它在不包含任何熊的图片上有效工作，包括在网络应用中显示这一信息。试试有两种不同熊的图像。检查使用多标签分类是否会影响到单标签数据集的准确性。