This module contains model wrappers, dataloaders, tests and all the ingredients needed to evaluate your object detection models. In particular this module allows you to evaluate your model against the following criteria:
- Performance on images with basic image attributes.
- Performance on images with various metadata from the datasets.
- Robustness against image perturbations like blurring, resizing, recoloring (performed by
opencv
: https://github.com/opencv/opencv)
- Racoon
- 300W, using the boundary box around all face landmarks
- ffhq, using the boundary box around all face landmarks
- Living room passes through Hugging Face
Once the model and dataloader (dl
) are wrapped, you can scan the model with the scan API in Giskard vision core:
from giskard_vision.core.scanner import scan
results = scan(model, dl)
It adapts the scan API in Giskard Python library to magically scan the vision model with the dataloader. The considered metric is:
- Intersection over Union (IoU)
Currently, we only support one object detection both in the model prediction and ground truth.