otx.backend.openvino.models#

OpenVINO models implementation for all supported CV tasks.

Classes

OVModel(model_path, model_type, ...)

Base class for the OpenVINO model.

OVDetectionModel(model_path, model_type, ...)

OVDetectionModel: Object detection model compatible for OpenVINO IR inference.

OVMulticlassClassificationModel(model_path, ...)

Classification model compatible for OpenVINO IR inference.

OVMultilabelClassificationModel(model_path, ...)

Multilabel classification model compatible for OpenVINO IR inference.

OVSegmentationModel(model_path, model_type, ...)

Semantic segmentation model compatible for OpenVINO IR inference.

OVHlabelClassificationModel(model_path, ...)

Hierarchical classification model compatible for OpenVINO IR inference.

OVInstanceSegmentationModel(model_path, ...)

Instance segmentation model compatible for OpenVINO IR inference.

OVKeypointDetectionModel(model_path, ...)

Keypoint detection model compatible for OpenVINO IR inference.
class otx.backend.openvino.models.OVDetectionModel(model_path: PathLike, model_type: str = 'SSD', async_inference: bool = True, max_num_requests: int | None = None, use_throughput_mode: bool = True, model_api_configuration: dict[str, Any] | None = None, metric: MetricCallable = <function _mean_ap_f_measure_callable>, **kwargs)[source]#

Bases: OVModel

OVDetectionModel: Object detection model compatible for OpenVINO IR inference.

This class is designed to work with OpenVINO IR models or models from the Intel OMZ repository. It provides compatibility with the OTX testing pipeline for object detection tasks.

Initialize the OVDetectionModel.

model_path (PathLike): Path to the OpenVINO IR model. model_type (str): Type of the model (default: “SSD”). async_inference (bool): Whether to use asynchronous inference (default: True). max_num_requests (int | None): Maximum number of inference requests (default: None). use_throughput_mode (bool): Whether to use throughput mode (default: True). model_api_configuration (dict[str, Any] | None): Configuration for the model API (default: None). metric (MetricCallable): Metric callable for evaluation (default: MeanAveragePrecisionFMeasureCallable). **kwargs: Additional keyword arguments.

…

Setup the tiler for handling tiled inference tasks.

This method configures the tiler with the appropriate execution mode and disables asynchronous inference as tiling has its own sync/async implementation. …

Extract hyperparameters from the OpenVINO model adapter.

model_adapter (OpenvinoAdapter): The adapter to extract model configuration from.

This method reads the confidence threshold from the model’s runtime information (rt_info). If unavailable, it logs a warning and sets the confidence threshold to None. …

Customize the outputs of the model to match the expected format.

outputs (list[DetectionResult]): List of detection results from the model. inputs (OTXDataBatch): Input batch containing image and metadata.

OTXPredBatch: A batch of predictions including bounding boxes, scores, labels, and optionally saliency maps and feature vectors.

…

Prepare inputs for metric computation.

preds (OTXPredBatch): Predicted batch containing bounding boxes, scores, and labels. inputs (OTXDataBatch): Input batch containing ground truth bounding boxes and labels.

MetricInput: A dictionary with ‘preds’ and ‘target’ keys containing the predicted and ground truth bounding boxes and labels.

…

Compute evaluation metrics for the model.

metric (Metric): Metric object used for evaluation.

dict: A dictionary containing computed metric values.

…

Initialize the OVModel instance.

Parameters:

  • model_path (PathLike) – Path to the model file.

  • model_type (str) – Type of the model.

  • async_inference (bool) – Whether to enable asynchronous inference.

  • force_cpu (bool) – Whether to force the use of CPU.

  • max_num_requests (int | None) – Maximum number of inference requests.

  • use_throughput_mode (bool) – Whether to use throughput mode.

  • model_api_configuration (dict[str, Any] | None) – Configuration for the Model API.

  • metric (MetricCallable) – Metric callable for evaluation.

compute_metrics(metric: Metric) → dict[source]#

Compute metrics for the model.

prepare_metric_inputs(preds: OTXPredBatch, inputs: OTXDataBatch) → dict[str, Any] | list[dict[str, Any]] | dict[str, list[dict[str, Any]]][source]#

Convert prediction and input entities to a format suitable for metric computation.

Parameters:

  • preds (OTXPredBatch) – The predicted batch entity containing predicted bboxes.

  • inputs (OTXDataBatch) – The input batch entity containing ground truth bboxes.

Returns:

A dictionary contains ‘preds’ and ‘target’ keys corresponding to the predicted and target bboxes for metric evaluation.

Return type:

MetricInput

class otx.backend.openvino.models.OVHlabelClassificationModel(model_path: PathLike, model_type: str = 'Classification', async_inference: bool = True, max_num_requests: int | None = None, use_throughput_mode: bool = True, model_api_configuration: dict[str, Any] | None = None, metric: MetricCallable = <function _mixed_hlabel_accuracy>, **kwargs)[source]#

Bases: OVModel

Hierarchical classification model compatible for OpenVINO IR inference.

It can consume OpenVINO IR model path or model name from Intel OMZ repository and create the OTX classification model compatible for OTX testing pipeline.

Initialize the hierarchical classification model.

Parameters:

  • model_path (PathLike) – Path to the OpenVINO IR model.

  • model_type (str) – Type of the model (default: “Classification”).

  • async_inference (bool) – Whether to enable asynchronous inference (default: True).

  • max_num_requests (int | None) – Maximum number of inference requests (default: None).

  • use_throughput_mode (bool) – Whether to use throughput mode (default: True).

  • model_api_configuration (dict[str, Any] | None) – Configuration for the model API (default: None).

  • metric (MetricCallable) – Metric callable for evaluation (default: HLabelClsMetricCallable).

  • **kwargs – Additional keyword arguments.

prepare_metric_inputs(preds: OTXPredBatch, inputs: OTXDataBatch) → dict[str, Any] | list[dict[str, Any]] | dict[str, list[dict[str, Any]]][source]#

Prepare inputs for metric computation.

Converts predictions and ground truth inputs into a format suitable for metric evaluation.

Parameters:

  • preds (OTXPredBatch) – Predicted batch containing labels and scores.

  • inputs (OTXDataBatch) – Input batch containing ground truth labels.

Returns:

A dictionary with ‘preds’ and ‘target’ keys for metric evaluation.

Return type:

MetricInput

class otx.backend.openvino.models.OVInstanceSegmentationModel(model_path: PathLike, model_type: str = 'MaskRCNN', async_inference: bool = True, max_num_requests: int | None = None, use_throughput_mode: bool = True, model_api_configuration: dict[str, Any] | None = None, metric: MetricCallable = <function _rle_mean_ap_f_measure_callable>, **kwargs)[source]#

Bases: OVModel

Instance segmentation model compatible for OpenVINO IR inference.

It can consume OpenVINO IR model path or model name from Intel OMZ repository and create the OTX detection model compatible for OTX testing pipeline.

Initialize the instance segmentation model.

Parameters:

  • model_path (PathLike) – Path to the OpenVINO IR model.

  • model_type (str) – Type of the model (default: “MaskRCNN”).

  • async_inference (bool) – Whether to use asynchronous inference (default: True).

  • max_num_requests (int | None) – Maximum number of inference requests (default: None).

  • use_throughput_mode (bool) – Whether to use throughput mode (default: True).

  • model_api_configuration (dict[str, Any] | None) – Model API configuration (default: None).

  • metric (MetricCallable) – Metric callable for evaluation (default: MaskRLEMeanAPFMeasureCallable).

  • **kwargs – Additional keyword arguments.

compute_metrics(metric: Metric) → dict[source]#

Compute evaluation metrics for the model.

Parameters:

metric (Metric) – Metric object to compute the evaluation metrics.

Returns:

Computed metrics.

Return type:

dict

prepare_metric_inputs(preds: OTXPredBatch, inputs: OTXDataBatch) → dict[str, Any] | list[dict[str, Any]] | dict[str, list[dict[str, Any]]][source]#

Prepare inputs for metric computation.

Converts predictions and ground truth to the format required by the metric and caches the ground truth for the current batch.

Parameters:
Returns:

Dictionary containing predictions and ground truth.

Return type:

MetricInput

class otx.backend.openvino.models.OVKeypointDetectionModel(model_path: PathLike, model_type: str = 'keypoint_detection', async_inference: bool = True, max_num_requests: int | None = None, use_throughput_mode: bool = True, model_api_configuration: dict[str, Any] | None = None, metric: MetricCallable = <function _pck_measure_callable>)[source]#

Bases: OVModel

Keypoint detection model compatible for OpenVINO IR inference.

It can consume OpenVINO IR model path or model name from Intel OMZ repository and create the OTX keypoint detection model compatible for OTX testing pipeline.

Initialize the keypoint detection model.

Parameters:

  • model_path (PathLike) – Path to the OpenVINO IR model.

  • model_type (str) – Type of the model. Defaults to “keypoint_detection”.

  • async_inference (bool) – Whether to enable asynchronous inference. Defaults to True.

  • max_num_requests (int | None) – Maximum number of inference requests. Defaults to None.

  • use_throughput_mode (bool) – Whether to enable throughput mode. Defaults to True.

  • model_api_configuration (dict[str, Any] | None) – Configuration for the model API. Defaults to None.

  • metric (MetricCallable) – Metric callable for evaluation. Defaults to PCKMeasureCallable.

compute_metrics(metric: Metric) → dict[source]#

Compute evaluation metrics for the keypoint detection model.

Parameters:

metric (Metric) – Metric object used for evaluation.

Returns:

A dictionary containing computed metric values.

Return type:

dict

prepare_metric_inputs(preds: OTXPredBatch, inputs: OTXDataBatch) → dict[str, Any] | list[dict[str, Any]] | dict[str, list[dict[str, Any]]][source]#

Prepare inputs for metric computation.

Converts prediction and input entities to a format suitable for metric evaluation.

Parameters:

  • preds (OTXPredBatch) – The predicted batch entity containing predicted keypoints.

  • inputs (OTXDataBatch) – The input batch entity containing ground truth keypoints.

Returns:

A dictionary containing ‘preds’ and ‘target’ keys corresponding to the predicted and target keypoints for metric evaluation.

Return type:

MetricInput

Raises:

ValueError – If ground truth keypoints, predicted keypoints, or scores are missing, or if the number of predicted and ground truth keypoints does not match.

class otx.backend.openvino.models.OVModel(model_path: PathLike, model_type: str, async_inference: bool = True, force_cpu: bool = True, max_num_requests: int | None = None, use_throughput_mode: bool = True, model_api_configuration: dict[str, Any] | None = None, metric: MetricCallable = <function _null_metric_callable>)[source]#

Bases: object

Base class for the OpenVINO model.

This is a base class representing interface for interacting with OpenVINO Intermediate Representation (IR) models. OVModel can create and validate OpenVINO IR model directly from provided path locally or from OpenVINO OMZ repository. (Only PyTorch models are supported). OVModel supports synchronous as well as asynchronous inference type.

Parameters:

num_classes – Number of classes this model can predict.

Initialize the OVModel instance.

Parameters:

  • model_path (PathLike) – Path to the model file.

  • model_type (str) – Type of the model.

  • async_inference (bool) – Whether to enable asynchronous inference.

  • force_cpu (bool) – Whether to force the use of CPU.

  • max_num_requests (int | None) – Maximum number of inference requests.

  • use_throughput_mode (bool) – Whether to use throughput mode.

  • model_api_configuration (dict[str, Any] | None) – Configuration for the Model API.

  • metric (MetricCallable) – Metric callable for evaluation.

__call__(*args, **kwds)[source]#

Call the model for inference.

Parameters:

  • *args – Positional arguments.

  • **kwds – Keyword arguments.

Returns:

Model output.

Return type:

Any

compute_metrics(metric: Metric) → dict[source]#

Compute metrics using the provided metric object.

Parameters:

metric (Metric) – Metric object.

Returns:

Computed metrics.

Return type:

dict

forward(inputs: OTXDataBatch, async_inference: bool = True) → OTXPredBatch[source]#

Perform forward pass of the model.

Parameters:

  • inputs (OTXDataBatch) – Input data batch.

  • async_inference (bool) – Whether to use asynchronous inference.

Returns:

Model predictions.

Return type:

OTXPredBatch

get_dummy_input(batch_size: int = 1) → OTXDataBatch[source]#

Generate a dummy input for the model.

Parameters:

batch_size (int) – Batch size for the dummy input.

Returns:

Dummy input data.

Return type:

OTXDataBatch

optimize(output_dir: Path, data_module: OTXDataModule, ptq_config: dict[str, Any] | None = None, optimized_model_name: str = 'optimized_model') → Path[source]#

Optimize the model using NNCF quantization.

Parameters:

  • output_dir (Path) – Directory to save the optimized model.

  • data_module (OTXDataModule) – Data module for training data.

  • ptq_config (dict[str, Any] | None) – PTQ configuration.

  • optimized_model_name (str) – Name of the optimized model.

Returns:

Path to the optimized model.

Return type:

Path

prepare_metric_inputs(preds: OTXPredBatch, inputs: OTXDataBatch) → MetricInput[source]#

Prepare inputs for metric computation.

Parameters:
Returns:

Dictionary containing predictions and targets.

Return type:

MetricInput

transform_fn(data_batch: OTXDataBatch) → array[source]#

Transform data for PTQ.

Parameters:

data_batch (OTXDataBatch) – Input data batch.

Returns:

Transformed data.

Return type:

np.array

property label_info: LabelInfo#

Get label information of the model.

Returns:

Label information.

Return type:

LabelInfo

property model_adapter_parameters: dict#

Get model parameters for export.

Returns:

Model parameters.

Return type:

dict

property task: OTXTaskType | None#

Get the task type of the model.

Returns:

Task type.

Return type:

OTXTaskType | None

class otx.backend.openvino.models.OVMulticlassClassificationModel(model_path: PathLike, model_type: str = 'Classification', async_inference: bool = True, max_num_requests: int | None = None, use_throughput_mode: bool = False, model_api_configuration: dict[str, Any] | None = None, metric: MetricCallable = <function _multi_class_cls_metric_callable>)[source]#

Bases: OVModel

Classification model compatible for OpenVINO IR inference.

It can consume OpenVINO IR model path or model name from Intel OMZ repository and create the OTX classification model compatible for OTX testing pipeline.

Initialize the OVMulticlassClassificationModel.

Parameters:

  • model_path (PathLike) – Path to the OpenVINO IR model or model name from Intel OMZ.

  • model_type (str) – Type of the model. Defaults to “Classification”.

  • async_inference (bool) – Whether to enable asynchronous inference. Defaults to True.

  • max_num_requests (int | None) – Maximum number of inference requests. Defaults to None.

  • use_throughput_mode (bool) – Whether to use throughput mode. Defaults to False.

  • model_api_configuration (dict[str, Any] | None) – Configuration for the model API. Defaults to None.

  • metric (MetricCallable) – Metric callable for evaluation. Defaults to MultiClassClsMetricCallable.

prepare_metric_inputs(preds: OTXPredBatch, inputs: OTXDataBatch) → dict[str, Any] | list[dict[str, Any]] | dict[str, list[dict[str, Any]]][source]#

Prepare inputs for metric computation.

Converts prediction and input entities into a format suitable for metric evaluation.

Parameters:

  • preds (OTXPredBatch) – Predicted batch containing predicted labels and other metadata.

  • inputs (OTXDataBatch) – Input batch containing ground truth labels and other metadata.

Returns:

A dictionary containing ‘preds’ and ‘target’ keys corresponding to predicted and target labels.

Return type:

MetricInput

class otx.backend.openvino.models.OVMultilabelClassificationModel(model_path: PathLike, model_type: str = 'Classification', async_inference: bool = True, max_num_requests: int | None = None, use_throughput_mode: bool = True, model_api_configuration: dict[str, Any] | None = None, metric: MetricCallable = <function _multi_label_cls_metric_callable>, **kwargs)[source]#

Bases: OVModel

Multilabel classification model compatible for OpenVINO IR inference.

It can consume OpenVINO IR model path or model name from Intel OMZ repository and create the OTX classification model compatible for OTX testing pipeline.

Initialize the multilabel classification model.

Parameters:

  • model_path (PathLike) – Path to the OpenVINO IR model or model name from Intel OMZ.

  • model_type (str) – Type of the model. Defaults to “Classification”.

  • async_inference (bool) – Whether to use asynchronous inference. Defaults to True.

  • max_num_requests (int | None) – Maximum number of inference requests. Defaults to None.

  • use_throughput_mode (bool) – Whether to use throughput mode. Defaults to True.

  • model_api_configuration (dict[str, Any] | None) – Configuration for the model API. Defaults to None.

  • metric (MetricCallable) – Metric callable for evaluation. Defaults to MultiLabelClsMetricCallable.

  • **kwargs – Additional keyword arguments.

prepare_metric_inputs(preds: OTXPredBatch, inputs: OTXDataBatch) → dict[str, Any] | list[dict[str, Any]] | dict[str, list[dict[str, Any]]][source]#

Prepare inputs for metric computation.

Converts prediction and input entities to a format suitable for metric evaluation.

Parameters:

  • preds (OTXPredBatch) – The predicted batch entity containing predicted labels and scores.

  • inputs (OTXDataBatch) – The input batch entity containing ground truth labels.

Returns:

A dictionary containing ‘preds’ and ‘target’ keys corresponding to the predicted and target labels for metric evaluation.

Return type:

MetricInput

class otx.backend.openvino.models.OVSegmentationModel(model_path: PathLike, model_type: str = 'Segmentation', async_inference: bool = True, max_num_requests: int | None = None, use_throughput_mode: bool = True, model_api_configuration: dict[str, Any] | None = None, metric: MetricCallable = <function _segm_callable>, **kwargs)[source]#

Bases: OVModel

Semantic segmentation model compatible for OpenVINO IR inference.

It can consume OpenVINO IR model path or model name from Intel OMZ repository and create the OTX segmentation model compatible for OTX testing pipeline.

Initialize the OVSegmentationModel.

Parameters:

  • model_path (PathLike) – Path to the OpenVINO IR model.

  • model_type (str) – Type of the model (default: “Segmentation”).

  • async_inference (bool) – Whether to enable asynchronous inference (default: True).

  • max_num_requests (int | None) – Maximum number of inference requests (default: None).

  • use_throughput_mode (bool) – Whether to use throughput mode (default: True).

  • model_api_configuration (dict[str, Any] | None) – Configuration for the model API (default: None).

  • metric (MetricCallable) – Metric callable for evaluation (default: SegmCallable).

  • **kwargs – Additional keyword arguments.

prepare_metric_inputs(preds: OTXPredBatch, inputs: OTXDataBatch) → dict[str, Any] | list[dict[str, Any]] | dict[str, list[dict[str, Any]]][source]#

Prepare inputs for metric computation.

Converts predictions and ground truth inputs into a format suitable for metric evaluation.

Parameters:

  • preds (OTXPredBatch) – Predicted segmentation batch containing masks.

  • inputs (OTXDataBatch) – Input batch containing ground truth masks.

Returns:

A list of dictionaries with ‘preds’ and ‘target’ keys for metric evaluation.

Return type:

MetricInput

Raises:

ValueError – If predicted or ground truth masks are not provided.