Chapter 14
Debugging with Evidently Test Suites and Reports
NotebookPython 3 (ipykernel)14 cells
Debugging with Evidently Test Suites and Reports
In [ ]python · cell 2
python
import datetime
import pandas as pd
from evidently import DataDefinition
from evidently import Dataset
from evidently import Report
from evidently.presets import DataDriftPreset
from joblib import dump, load
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_absolute_percentage_errorLoad data and model
In [ ]python · cell 4
python
ref_data = pd.read_parquet('data/reference.parquet')In [ ]python · cell 5
python
current_data = pd.read_parquet('data/green_tripdata_2022-02.parquet')In [ ]python · cell 6
python
with open('models/lin_reg.bin', 'rb') as f_in:
model = load(f_in)In [ ]python · cell 7
python
# data labeling
target = "duration_min"
num_features = ["passenger_count", "trip_distance", "fare_amount", "total_amount"]
cat_features = ["PULocationID", "DOLocationID"]In [ ]python · cell 8
python
problematic_data = current_data.loc[(current_data.lpep_pickup_datetime >= datetime.datetime(2022,2,2,0,0)) &
(current_data.lpep_pickup_datetime < datetime.datetime(2022,2,3,0,0))]Generate Report with Tests (ex Test Suite)
Note
Since Evidently 0.7.0 release, Test Suite is integrated into reports.
To include tests into report, you just past tests to metric or provide include_tests=True to include default tests (if provided).
In [ ]python · cell 10
python
data_definition = DataDefinition(
numerical_columns=num_features + ['prediction'],
categorical_columns=cat_features,
)In [ ]python · cell 11
python
problematic_data['prediction'] = model.predict(problematic_data[num_features + cat_features].fillna(0))In [ ]python · cell 12
python
report = Report([DataDriftPreset()], include_tests=True)
ref_dataset = Dataset.from_pandas(ref_data, data_definition=data_definition)
problematic_dataset = Dataset.from_pandas(ref_data, data_definition=data_definition)
run = report.run(reference_data=ref_dataset, current_data=problematic_dataset)In [ ]python · cell 13
python
runIn [ ]python · cell 14
python