Chapter 80

Homework 1

NotebookPython 3 (ipykernel)32 cells

Homework #1

Watch the video with explanation here.

Set up the environment

You need to install Python, NumPy, Pandas, Matplotlib and Seaborn. For that, you can the instructions from 06-environment.md.

In [2]python · cell 2

python

import numpy as np
import pandas as pd

Question 1

What's the version of NumPy that you installed?

You can get the version information using the __version__ field:

python

np.__version__

In [3]python · cell 4

python

np.__version__

Output

'1.21.1'

Question 2

What's the version of Pandas?

In [4]python · cell 6

python

pd.__version__

Output

'1.3.0'

Getting the data

For this homework, we'll use the same dataset as for the next session - the car price dataset.

Download it from here.

You can do it with wget:

bash

wget https://raw.githubusercontent.com/alexeygrigorev/mlbookcamp-code/master/chapter-02-car-price/data.csv

Or just open it with your browser and click "Save as...".

In [6]python · cell 8

python

!wget https://raw.githubusercontent.com/alexeygrigorev/mlbookcamp-code/master/chapter-02-car-price/data.csv -O hw-1.csv

Output

--2021-09-14 07:34:45--  https://raw.githubusercontent.com/alexeygrigorev/mlbookcamp-code/master/chapter-02-car-price/data.csv
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1475504 (1.4M) [text/plain]
Saving to: ‘hw-1.csv’

hw-1.csv            100%[===================>]   1.41M  1.89MB/s    in 0.7s    

2021-09-14 07:34:45 (1.89 MB/s) - ‘hw-1.csv’ saved [1475504/1475504]

Note: I have wget installed separately, so if you're on Windows without WSL, you will need to download it

Now read it with Pandas.

In [7]python · cell 10

python

df = pd.read_csv('hw-1.csv')

Question 3

What's the average price of BMW cars in the dataset?

In [8]python · cell 12

python

df[df.Make == 'BMW'].MSRP.mean()

Output

61546.76347305389

Question 4

Select a subset of cars after year 2015 (inclusive, i.e. 2015 and after). How many of them have missing values for Engine HP?

In [9]python · cell 14

python

df[df.Year >= 2015]['Engine HP'].isnull().sum()

Output

Question 5

Calculate the average "Engine HP" in the dataset.
Use the fillna method and to fill the missing values in "Engine HP" with the mean value from the previous step.
Now, calcualte the average of "Engine HP" again.
Has it changed?

Round both means before answering this questions.

In [10]python · cell 16

python

mean_hp = df['Engine HP'].mean()
mean_hp

Output

249.38607007176023

In [11]python · cell 17

python

df['Engine HP'].fillna(mean_hp).mean()

Output

249.38607007176023

Filling NAs with 0 changes the mean of "Engine HP":

In [12]python · cell 19

python

df['Engine HP'].fillna(0).mean()

Output

247.94174920261878

Question 6

Select all the "Rolls-Royce" cars from the dataset.
Select only columns "Engine HP", "Engine Cylinders", "highway MPG".
Now drop all duplicated rows using drop_duplicates method (you should get a dataframe with 7 rows).
Get the underlying NumPy array. Let's call it X.
Compute matrix-matrix multiplication between the transpose of X and X. To get the transpose, use X.T. Let's call the result XTX.
Invert XTX.
What's the sum of all the elements of the result?

In [13]python · cell 21

python

df_rr = df[df.Make == "Rolls-Royce"]
df_rr = df_rr[["Engine HP", "Engine Cylinders", "highway MPG"]]
df_rr = df_rr.drop_duplicates()

In [14]python · cell 22

python

X = df_rr.values
XTX = X.T.dot(X)

XTX_inv = np.linalg.inv(XTX)
XTX_inv.sum()

Output

0.03221232067748618

Questions 7

Create an array y with values [1000, 1100, 900, 1200, 1000, 850, 1300].
Multiply the inverse of XTX with the transpose of X, and then multiply the result by y. Call the result w.
What's the value of the first element of w?.

In [15]python · cell 24

python

y = [1000, 1100, 900, 1200, 1000, 850, 1300]

In [16]python · cell 25

python

w = XTX_inv.dot(X.T).dot(y)

In [17]python · cell 26

python

w[0]

Output

0.19989598183192342

Note: we just implemented normal equation

$w = (X^T X)^{-1} X^T y$

We'll talk about it more in the next week (Machine Learning for Regression)

Bonus

Floating point arithmetics is not exact

In [18]python · cell 29

python

0.1 + 0.2

Output

0.30000000000000004

Adding the mean value doesn't change the resulting mean:

In [20]python · cell 31

python

np.array([1, 2, 3, 4, 5, 6]).mean()

Output

3.5

In [21]python · cell 32

python

np.array([1, 2, 3, 4, 5, 6, 3.5, 3.5, 3.5]).mean()

Output

3.5

--2021-09-14 07:34:45-- https://raw.githubusercontent.com/alexeygrigorev/mlbookcamp-code/master/chapter-02-car-price/data.csv Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1475504 (1.4M) [text/plain] Saving to: ‘hw-1.csv’ hw-1.csv 100%[===================>] 1.41M 1.89MB/s in 0.7s 2021-09-14 07:34:45 (1.89 MB/s) - ‘hw-1.csv’ saved [1475504/1475504]