otx.backend.native.models#

Module for OTX custom models.

Classes

Padim(data_input_params[, label_info, ...])

OTX Padim model.

Stfpm(data_input_params[, label_info, ...])

OTX STFPM model.

Uflow(data_input_params[, label_info, ...])

OTX UFlow model.

EfficientNetHLabelCls(label_info, ...)

EfficientNet Model for hierarchical label classification task.

EfficientNetMulticlassCls(label_info, ...)

EfficientNet Model for multi-class classification task.

EfficientNetMultilabelCls(label_info, ...)

EfficientNet Model for multi-label classification task.

MobileNetV3HLabelCls(label_info, ...)

MobileNetV3 Model for hierarchical label classification task.

MobileNetV3MulticlassCls(label_info, ...)

MobileNetV3MulticlassCls is a class that represents a MobileNetV3 model for multiclass classification.

MobileNetV3MultilabelCls(label_info, ...)

MobileNetV3 Model for multi-class classification task.

TimmModelHLabelCls(label_info, ...)

Timm Model for hierarchical label classification task.

TimmModelMulticlassCls(label_info, ...)

TimmModel for multi-class classification task.

TimmModelMultilabelCls(label_info, ...)

TimmModel for multi-label classification task.

TVModelHLabelCls(label_info, ...)

TVModelForHLabelCls class represents a Torchvision model for hierarchical label classification.

TVModelMulticlassCls(label_info, ...)

Torchvision model for multiclass classification.

TVModelMultilabelCls(label_info, ...)

Torchvision model for multilabel classification.

VisionTransformerHLabelCls(label_info, ...)

VisionTransformerForHLabelCls is a model designed for hierarchical label classification using ViT architecture.

VisionTransformerMulticlassCls(label_info, ...)

DeitTiny Model for multi-class classification task.

VisionTransformerMultilabelCls(label_info, ...)

DeitTiny Model for multi-class classification task.

ATSS(label_info, data_input_params, ...[, ...])

OTX Detection model class for ATSS.

DFine(label_info, data_input_params, ...[, ...])

OTX Detection model class for DFine.

SSD(label_info, data_input_params, ...[, ...])

OTX Detection model class for SSD.

RTMDet(label_info, data_input_params, ...[, ...])

OTX Detection model class for RTMDet.

RTDETR(label_info, data_input_params, ...[, ...])

OTX Detection model class for RTDETR.

MaskRCNN(label_info, data_input_params, ...)

Implementation of MaskRCNN for instance segmentation.

MaskRCNNTV(label_info, data_input_params, ...)

Implementation of torchvision MaskRCNN for instance segmentation.

RTMDetInst(label_info, data_input_params, ...)

Implementation of RTMDetInst for instance segmentation.

RTMPose(label_info, data_input_params, ...)

RTMPose Model.

DinoV2Seg(label_info, data_input_params, ...)

DinoV2Seg for Semantic Segmentation model.

LiteHRNet(label_info, data_input_params, ...)

LiteHRNet Model.

SegNext(label_info, data_input_params, ...)

SegNext Model.
class otx.backend.native.models.ATSS(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: Literal['atss_mobilenetv2', 'atss_resnext101'] = 'atss_mobilenetv2', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _mean_ap_f_measure_callable>, torch_compile: bool = False, tile_config: TileConfig = TileConfig(enable_tiler=False, enable_adaptive_tiling=True, tile_size=(400, 400), overlap=0.2, iou_threshold=0.45, max_num_instances=1500, object_tile_ratio=0.03, sampling_ratio=1.0, with_full_img=False))[source]#

Bases: OTXDetectionModel

OTX Detection model class for ATSS.

pretrained_weights#

Dictionary containing URLs for pretrained weights.

Type:

ClassVar[dict[str, str]]

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels.

  • data_input_params (DataInputParams) – Parameters for data input.

  • model_name (Literal, optional) – Name of the model to use. Defaults to “atss_mobilenetv2”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to MeanAveragePrecisionFMeasureCallable.

  • torch_compile (bool, optional) – Whether to use torch compile. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.

class otx.backend.native.models.DFine(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: Literal['dfine_hgnetv2_n', 'dfine_hgnetv2_s', 'dfine_hgnetv2_m', 'dfine_hgnetv2_l', 'dfine_hgnetv2_x'] = 'dfine_hgnetv2_x', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _mean_ap_f_measure_callable>, multi_scale: bool = False, torch_compile: bool = False, tile_config: TileConfig = TileConfig(enable_tiler=False, enable_adaptive_tiling=True, tile_size=(400, 400), overlap=0.2, iou_threshold=0.45, max_num_instances=1500, object_tile_ratio=0.03, sampling_ratio=1.0, with_full_img=False))[source]#

Bases: RTDETR

OTX Detection model class for DFine.

pretrained_weights#

Dictionary containing URLs for pretrained weights.

Type:

ClassVar[dict[str, str]]

input_size_multiplier#

Multiplier for the input size.

Type:

int

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels.

  • data_input_params (DataInputParams) – Parameters for data input.

  • model_name (literal, optional) – Name of the model to use. Defaults to “dfine_hgnetv2_x”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to MeanAveragePrecisionFMeasureCallable.

  • multi_scale (bool, optional) – Whether to use multi-scale training. Defaults to False.

  • torch_compile (bool, optional) – Whether to use torch compile. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

class otx.backend.native.models.DinoV2Seg(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: Literal['dinov2-small-seg'] = 'dinov2-small-seg', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _segm_callable>, torch_compile: bool = False, tile_config: TileConfig = TileConfig(enable_tiler=False, enable_adaptive_tiling=True, tile_size=(400, 400), overlap=0.2, iou_threshold=0.45, max_num_instances=1500, object_tile_ratio=0.03, sampling_ratio=1.0, with_full_img=False))[source]#

Bases: OTXSegmentationModel

DinoV2Seg for Semantic Segmentation model.

Parameters:

  • label_info (LabelInfoTypes) – Information about the hierarchical labels.

  • data_input_params (DataInputParams) – Parameters for data input.

  • model_name (Literal, optional) – Name of the model. Defaults to “dinov2-small-seg”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – Callable for the learning rate scheduler.

  • DefaultSchedulerCallable. (Defaults to) –

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to SegmCallable.

  • torch_compile (bool, optional) – Flag to indicate whether to use torch.compile. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

class otx.backend.native.models.EfficientNetHLabelCls(label_info: HLabelInfo, data_input_params: DataInputParams, model_name: str = 'efficientnet_b0', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _mixed_hlabel_accuracy>, torch_compile: bool = False)[source]#

Bases: OTXHlabelClsModel

EfficientNet Model for hierarchical label classification task.

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.

class otx.backend.native.models.EfficientNetMulticlassCls(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: str = 'efficientnet_b0', freeze_backbone: bool = False, optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _multi_class_cls_metric_callable>, torch_compile: bool = False)[source]#

Bases: OTXMulticlassClsModel

EfficientNet Model for multi-class classification task.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels.

  • data_input_params (DataInputParams) – Parameters for data input.

  • model_name (str, optional) – Name of the EfficientNet model variant. Defaults to “efficientnet_b0”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the evaluation metric. Defaults to MultiClassClsMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate whether to use torch.compile. Defaults to False.

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

forward_for_tracing(image: Tensor) → Tensor | dict[str, Tensor][source]#

Model forward function used for the model tracing during model exportation.

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.

class otx.backend.native.models.EfficientNetMultilabelCls(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: str = 'efficientnet_b0', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _multi_label_cls_metric_callable>, torch_compile: bool = False)[source]#

Bases: OTXMultilabelClsModel

EfficientNet Model for multi-label classification task.

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

forward_for_tracing(image: Tensor) → Tensor | dict[str, Tensor][source]#

Model forward function used for the model tracing during model exportation.

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.

class otx.backend.native.models.LiteHRNet(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: Literal['lite_hrnet_s', 'lite_hrnet_18', 'lite_hrnet_x'] = 'lite_hrnet_18', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _segm_callable>, torch_compile: bool = False, tile_config: TileConfig = TileConfig(enable_tiler=False, enable_adaptive_tiling=True, tile_size=(400, 400), overlap=0.2, iou_threshold=0.45, max_num_instances=1500, object_tile_ratio=0.03, sampling_ratio=1.0, with_full_img=False))[source]#

Bases: OTXSegmentationModel

LiteHRNet Model.

Parameters:

  • label_info (LabelInfoTypes) – Information about the hierarchical labels.

  • data_input_params (DataInputParams) – Parameters for data input.

  • model_name (Literal, optional) – Name of the model. Defaults to “lite_hrnet_18”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – Callable for the learning rate scheduler.

  • DefaultSchedulerCallable. (Defaults to) –

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to SegmCallable.

  • torch_compile (bool, optional) – Flag to indicate whether to use torch.compile. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.

property ignore_scope: dict[str, Any]#

Get the ignored scope for LiteHRNet.

class otx.backend.native.models.MaskRCNN(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: Literal['maskrcnn_resnet_50', 'maskrcnn_efficientnet_b2b', 'maskrcnn_swin_tiny'] = 'maskrcnn_resnet_50', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _rle_mean_ap_f_measure_callable>, torch_compile: bool = False, tile_config: TileConfig = TileConfig(enable_tiler=False, enable_adaptive_tiling=True, tile_size=(400, 400), overlap=0.2, iou_threshold=0.45, max_num_instances=1500, object_tile_ratio=0.03, sampling_ratio=1.0, with_full_img=False))[source]#

Bases: OTXInstanceSegModel

Implementation of MaskRCNN for instance segmentation.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters for the data input.

  • model_name (str, optional) – Name of the model. Defaults to “maskrcnn_resnet_50”.

  • optimizer (OptimizerCallable, optional) – Optimizer for the model. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – Scheduler for the model. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Metric for evaluating the model. Defaults to MaskRLEMeanAPFMeasureCallable.

  • torch_compile (bool, optional) – Whether to use torch compile. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

  • explain_mode (bool, optional) – Whether to enable explainable AI mode. Defaults to False.

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.

class otx.backend.native.models.MaskRCNNTV(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: Literal['maskrcnn_resnet_50'] = 'maskrcnn_resnet_50', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _rle_mean_ap_f_measure_callable>, torch_compile: bool = False, tile_config: TileConfig = TileConfig(enable_tiler=False, enable_adaptive_tiling=True, tile_size=(400, 400), overlap=0.2, iou_threshold=0.45, max_num_instances=1500, object_tile_ratio=0.03, sampling_ratio=1.0, with_full_img=False))[source]#

Bases: OTXInstanceSegModel

Implementation of torchvision MaskRCNN for instance segmentation.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters for the data input.

  • model_name (str, optional) – Name of the model. Defaults to “maskrcnn_resnet_50”.

  • optimizer (OptimizerCallable, optional) – Optimizer for the model. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – Scheduler for the model. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Metric for evaluating the model. Defaults to MaskRLEMeanAPFMeasureCallable.

  • torch_compile (bool, optional) – Whether to use torch compile. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

  • explain_mode (bool, optional) – Whether to enable explainable AI mode. Defaults to False.

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

forward_for_tracing(inputs: Tensor) → tuple[Tensor, ...][source]#

Forward function for export.

class otx.backend.native.models.MobileNetV3HLabelCls(label_info: HLabelInfo, data_input_params: DataInputParams, model_name: str = 'mobilenetv3_large', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _mixed_hlabel_accuracy>, torch_compile: bool = False)[source]#

Bases: OTXHlabelClsModel

MobileNetV3 Model for hierarchical label classification task.

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.

class otx.backend.native.models.MobileNetV3MulticlassCls(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: str = 'mobilenetv3_large', freeze_backbone: bool = False, optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _multi_class_cls_metric_callable>, torch_compile: bool = False)[source]#

Bases: OTXMulticlassClsModel

MobileNetV3MulticlassCls is a class that represents a MobileNetV3 model for multiclass classification.

Parameters:

  • label_info (LabelInfoTypes) – The label information.

  • data_input_params (DataInputParams) – The data input parameters such as input size and normalization.

  • model_name (str, optional) – The model name. Defaults to “mobilenetv3_large”.

  • optimizer (OptimizerCallable, optional) – The optimizer callable. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – The learning rate scheduler callable. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – The metric callable. Defaults to MultiClassClsMetricCallable.

  • torch_compile (bool, optional) – Whether to compile the model using TorchScript. Defaults to False.

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

forward_for_tracing(image: Tensor) → Tensor | dict[str, Tensor][source]#

Model forward function used for the model tracing during model exportation.

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.

class otx.backend.native.models.MobileNetV3MultilabelCls(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: str = 'mobilenetv3_large', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _multi_label_cls_metric_callable>, torch_compile: bool = False)[source]#

Bases: OTXMultilabelClsModel

MobileNetV3 Model for multi-class classification task.

Parameters:

  • label_info (LabelInfoTypes) – The label information.

  • data_input_params (DataInputParams) – The data input parameters such as input size and normalization.

  • model_name (str, optional) – The model name. Defaults to “mobilenetv3_large”.

  • optimizer (OptimizerCallable, optional) – The optimizer callable. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – The learning rate scheduler callable. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – The metric callable. Defaults to MultiClassClsMetricCallable.

  • torch_compile (bool, optional) – Whether to compile the model using TorchScript. Defaults to False.

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

forward_for_tracing(image: Tensor) → Tensor | dict[str, Tensor][source]#

Model forward function used for the model tracing during model exportation.

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.

class otx.backend.native.models.Padim(data_input_params: DataInputParams, label_info: LabelInfoTypes = AnomalyLabelInfo(label_names=['Normal', 'Anomaly'], label_ids=['0', '1'], label_groups=[['Normal', 'Anomaly']]), backbone: str = 'resnet18', layers: list[str] = ['layer1', 'layer2', 'layer3'], pre_trained: bool = True, n_features: int | None = None, task: Literal[OTXTaskType.ANOMALY, OTXTaskType.ANOMALY_CLASSIFICATION, OTXTaskType.ANOMALY_DETECTION, OTXTaskType.ANOMALY_SEGMENTATION] = OTXTaskType.ANOMALY_CLASSIFICATION)[source]#

Bases: AnomalyMixin, Padim, OTXAnomaly

OTX Padim model.

Parameters:

  • backbone (str, optional) – Feature extractor backbone. Defaults to “resnet18”.

  • layers (list[str], optional) – Feature extractor layers. Defaults to [“layer1”, “layer2”, “layer3”].

  • pre_trained (bool, optional) – Pretrained backbone. Defaults to True.

  • n_features (int | None, optional) – Number of features. Defaults to None.

  • (Literal[ (task) – OTXTaskType.ANOMALY_CLASSIFICATION, OTXTaskType.ANOMALY_DETECTION, OTXTaskType.ANOMALY_SEGMENTATION ], optional): Task type of Anomaly Task. Defaults to OTXTaskType.ANOMALY_CLASSIFICATION.

  • input_size (tuple[int, int], optional) – Model input size in the order of height and width. Defaults to (256, 256)

class otx.backend.native.models.RTDETR(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: Literal['rtdetr_18', 'rtdetr_50', 'rtdetr_101'] = 'rtdetr_50', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _mean_ap_f_measure_callable>, multi_scale: bool = False, torch_compile: bool = False, tile_config: TileConfig = TileConfig(enable_tiler=False, enable_adaptive_tiling=True, tile_size=(400, 400), overlap=0.2, iou_threshold=0.45, max_num_instances=1500, object_tile_ratio=0.03, sampling_ratio=1.0, with_full_img=False))[source]#

Bases: OTXDetectionModel

OTX Detection model class for RTDETR.

pretrained_weights#

Dictionary containing URLs for pretrained weights.

Type:

ClassVar[dict[str, str]]

input_size_multiplier#

Multiplier for the input size.

Type:

int

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels.

  • data_input_params (DataInputParams) – Parameters for data input.

  • model_name (literal, optional) – Name of the model to use. Defaults to “rtdetr_50”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to MeanAveragePrecisionFMeasureCallable.

  • multi_scale (bool, optional) – Whether to use multi-scale training. Defaults to False.

  • torch_compile (bool, optional) – Whether to use torch compile. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

configure_optimizers() → tuple[list[Optimizer], list[dict[str, Any]]][source]#

Configure an optimizer and learning-rate schedulers.

Configure an optimizer and learning-rate schedulers from the given optimizer and scheduler or scheduler list callable in the constructor. Generally, there is two lr schedulers. One is for a linear warmup scheduler and the other is the main scheduler working after the warmup period.

Returns:

Two list. The former is a list that contains an optimizer The latter is a list of lr scheduler configs which has a dictionary format.

class otx.backend.native.models.RTMDet(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: Literal['rtmdet_tiny'] = 'rtmdet_tiny', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _mean_ap_f_measure_callable>, torch_compile: bool = False, tile_config: TileConfig = TileConfig(enable_tiler=False, enable_adaptive_tiling=True, tile_size=(400, 400), overlap=0.2, iou_threshold=0.45, max_num_instances=1500, object_tile_ratio=0.03, sampling_ratio=1.0, with_full_img=False))[source]#

Bases: OTXDetectionModel

OTX Detection model class for RTMDet.

pretrained_weights#

Dictionary containing URLs for pretrained weights.

Type:

ClassVar[dict[str, str]]

input_size_multiplier#

Multiplier for the input size.

Type:

int

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels.

  • data_input_params (DataInputParams) – Parameters for data input.

  • model_name (str, optional) – Name of the model to use. Defaults to “rtmdet_tiny”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to MeanAveragePrecisionFMeasureCallable.

  • torch_compile (bool, optional) – Whether to use torch compile. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

class otx.backend.native.models.RTMDetInst(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: Literal['rtmdet_inst_tiny'] = 'rtmdet_inst_tiny', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _rle_mean_ap_f_measure_callable>, torch_compile: bool = False, tile_config: TileConfig = TileConfig(enable_tiler=False, enable_adaptive_tiling=True, tile_size=(400, 400), overlap=0.2, iou_threshold=0.45, max_num_instances=1500, object_tile_ratio=0.03, sampling_ratio=1.0, with_full_img=False))[source]#

Bases: OTXInstanceSegModel

Implementation of RTMDetInst for instance segmentation.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters for the data input.

  • model_name (str, optional) – Name of the model. Defaults to “rtmdet_inst_tiny”.

  • optimizer (OptimizerCallable, optional) – Optimizer for the model. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – Scheduler for the model. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Metric for evaluating the model. Defaults to MaskRLEMeanAPFMeasureCallable.

  • torch_compile (bool, optional) – Whether to use torch compile. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

  • explain_mode (bool, optional) – Whether to enable explainable AI mode. Defaults to False.

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

forward_for_tracing(inputs: Tensor) → tuple[Tensor, ...][source]#

Forward function for export.

NOTE : RTMDetInst uses explain_mode unlike other models.

class otx.backend.native.models.RTMPose(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: Literal['rtmpose_tiny'] = 'rtmpose_tiny', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _pck_measure_callable>, torch_compile: bool = False)[source]#

Bases: OTXKeypointDetectionModel

RTMPose Model.

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

class otx.backend.native.models.SSD(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: Literal['ssd_mobilenetv2'] = 'ssd_mobilenetv2', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _mean_ap_f_measure_callable>, torch_compile: bool = False, tile_config: TileConfig = TileConfig(enable_tiler=False, enable_adaptive_tiling=True, tile_size=(400, 400), overlap=0.2, iou_threshold=0.45, max_num_instances=1500, object_tile_ratio=0.03, sampling_ratio=1.0, with_full_img=False))[source]#

Bases: OTXDetectionModel

OTX Detection model class for SSD.

pretrained_weights#

Dictionary containing URLs for pretrained weights.

Type:

ClassVar[dict[str, str]]

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels.

  • data_input_params (DataInputParams) – Parameters for data input.

  • model_name (str, optional) – Name of the model to use. Defaults to “ssd_mobilenetv2”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to MeanAveragePrecisionFMeasureCallable.

  • torch_compile (bool, optional) – Whether to use torch compile. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.

load_state_dict_pre_hook(state_dict: dict[str, torch.Tensor], prefix: str, *args, **kwargs) → None[source]#

Modify input state_dict according to class name matching. It is used for incremental learning.

on_load_checkpoint(checkpoint: dict[str, Any]) → None[source]#

Callback on load checkpoint.

setup(stage: str) → None[source]#

Callback for setup OTX SSD Model.

OTXSSD requires auto anchor generating w.r.t. training dataset for better accuracy. This callback will provide training dataset to model’s anchor generator.

Parameters:

trainer (Trainer) – Lightning trainer contains OTXLitModule and OTXDatamodule.

class otx.backend.native.models.SegNext(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: Literal['segnext_tiny', 'segnext_small', 'segnext_base'] = 'segnext_small', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _segm_callable>, torch_compile: bool = False, tile_config: TileConfig = TileConfig(enable_tiler=False, enable_adaptive_tiling=True, tile_size=(400, 400), overlap=0.2, iou_threshold=0.45, max_num_instances=1500, object_tile_ratio=0.03, sampling_ratio=1.0, with_full_img=False))[source]#

Bases: OTXSegmentationModel

SegNext Model.

Parameters:

  • label_info (LabelInfoTypes) – Information about the hierarchical labels.

  • data_input_params (DataInputParams) – Parameters for data input.

  • model_name (Literal, optional) – Name of the model. Defaults to “segnext_small”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – Callable for the learning rate scheduler.

  • DefaultSchedulerCallable. (Defaults to) –

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to SegmCallable.

  • torch_compile (bool, optional) – Flag to indicate whether to use torch.compile. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.

class otx.backend.native.models.Stfpm(data_input_params: DataInputParams, label_info: LabelInfoTypes = AnomalyLabelInfo(label_names=['Normal', 'Anomaly'], label_ids=['0', '1'], label_groups=[['Normal', 'Anomaly']]), layers: Sequence[str] = ['layer1', 'layer2', 'layer3'], backbone: str = 'resnet18', task: Literal[OTXTaskType.ANOMALY, OTXTaskType.ANOMALY_CLASSIFICATION, OTXTaskType.ANOMALY_DETECTION, OTXTaskType.ANOMALY_SEGMENTATION] = OTXTaskType.ANOMALY_CLASSIFICATION, **kwargs)[source]#

Bases: AnomalyMixin, Stfpm, OTXAnomaly

OTX STFPM model.

Parameters:

  • layers (Sequence[str]) – Feature extractor layers.

  • backbone (str, optional) – Feature extractor backbone. Defaults to “resnet18”.

  • (Literal[ (task) – OTXTaskType.ANOMALY_CLASSIFICATION, OTXTaskType.ANOMALY_DETECTION, OTXTaskType.ANOMALY_SEGMENTATION ], optional): Task type of Anomaly Task. Defaults to OTXTaskType.ANOMALY_CLASSIFICATION.

  • input_size (tuple[int, int], optional) – Model input size in the order of height and width. Defaults to (256, 256)

class otx.backend.native.models.TVModelHLabelCls(label_info: HLabelInfo, data_input_params: DataInputParams, model_name: str = 'efficientnet_v2_s', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _mixed_hlabel_accuracy>, torch_compile: bool = False)[source]#

Bases: OTXHlabelClsModel

TVModelForHLabelCls class represents a Torchvision model for hierarchical label classification.

Parameters:

  • label_info (HLabelInfo) – Information about the hierarchical labels.

  • backbone (TVModelType) – The type of Torchvision backbone model.

  • pretrained (bool, optional) – Whether to use pretrained weights. Defaults to True.

  • optimizer (OptimizerCallable, optional) – The optimizer callable. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – The learning rate scheduler callable. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – The metric callable. Defaults to HLabelClsMetricCallble.

  • torch_compile (bool, optional) – Whether to compile the model using TorchScript. Defaults to False.

  • input_size (tuple[int, int], optional) – The input size of the images. Defaults to (224, 224).

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

class otx.backend.native.models.TVModelMulticlassCls(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: str = 'efficientnet_v2_s', freeze_backbone: bool = False, optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _multi_class_cls_metric_callable>, torch_compile: bool = False)[source]#

Bases: OTXMulticlassClsModel

Torchvision model for multiclass classification.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels.

  • data_input_params (DataInputParams) – Data input parameters such as input size and normalization.

  • model_name (str, optional) – Backbone model name for feature extraction. Defaults to “efficientnet_v2_s”.

  • optimizer (OptimizerCallable, optional) – Optimizer for model training. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – Learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Metric for model evaluation. Defaults to MultiClassClsMetricCallable.

  • torch_compile (bool, optional) – Whether to compile the model using TorchScript. Defaults to False.

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

forward_for_tracing(image: Tensor) → Tensor | dict[str, Tensor][source]#

Model forward function used for the model tracing during model exportation.

class otx.backend.native.models.TVModelMultilabelCls(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: str = 'efficientnet_v2_s', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _multi_label_cls_metric_callable>, torch_compile: bool = False)[source]#

Bases: OTXMultilabelClsModel

Torchvision model for multilabel classification.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels.

  • backbone (TVModelType) – Backbone model for feature extraction.

  • pretrained (bool, optional) – Whether to use pretrained weights. Defaults to True.

  • optimizer (OptimizerCallable, optional) – Optimizer for model training. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – Learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Metric for model evaluation. Defaults to MultiLabelClsMetricCallable.

  • torch_compile (bool, optional) – Whether to compile the model using TorchScript. Defaults to False.

  • input_size (tuple[int, int], optional) – Input size of the images. Defaults to (224, 224).

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

forward_for_tracing(image: torch.Tensor) → torch.Tensor | dict[str, torch.Tensor][source]#

Model forward function used for the model tracing during model exportation.

class otx.backend.native.models.TimmModelHLabelCls(label_info: HLabelInfo, data_input_params: DataInputParams, model_name: str = 'tf_efficientnetv2_s.in21k', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _mixed_hlabel_accuracy>, torch_compile: bool = False)[source]#

Bases: OTXHlabelClsModel

Timm Model for hierarchical label classification task.

Parameters:

  • label_info (HLabelInfo) – The label information for the classification task.

  • model_name (str) – The name of the model. You can find available models at timm.list_models() or timm.list_pretrained().

  • input_size (tuple[int, int], optional) – Model input size in the order of height and width. Defaults to (224, 224).

  • pretrained (bool, optional) – Whether to load pretrained weights. Defaults to True.

  • optimizer (OptimizerCallable, optional) – The optimizer callable for training the model.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – The learning rate scheduler callable.

  • metric (MetricCallable, optional) – The metric callable for evaluating the model. Defaults to HLabelClsMetricCallable.

  • torch_compile (bool, optional) – Whether to compile the model using TorchScript. Defaults to False.

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.

class otx.backend.native.models.TimmModelMulticlassCls(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: str, freeze_backbone: bool = False, optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _multi_class_cls_metric_callable>, torch_compile: bool = False)[source]#

Bases: OTXMulticlassClsModel

TimmModel for multi-class classification task.

Parameters:

  • label_info (LabelInfoTypes) – The label information for the classification task.

  • model_name (str) – The name of the model. You can find available models at timm.list_models() or timm.list_pretrained().

  • input_size (tuple[int, int], optional) – Model input size in the order of height and width. Defaults to (224, 224).

  • pretrained (bool, optional) – Whether to load pretrained weights. Defaults to True.

  • optimizer (OptimizerCallable, optional) – The optimizer callable for training the model.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – The learning rate scheduler callable.

  • metric (MetricCallable, optional) – The metric callable for evaluating the model. Defaults to MultiClassClsMetricCallable.

  • torch_compile (bool, optional) – Whether to compile the model using TorchScript. Defaults to False.

Example

  1. API
    >>> model = TimmModelForMulticlassCls(
...     model_name="tf_efficientnetv2_s.in21k",
...     label_info=<Number-of-classes>,
... )
  2. CLI
    >>> otx train             ... --model otx.algo.classification.timm_model.TimmModelForMulticlassCls             ... --model.model_name tf_efficientnetv2_s.in21k

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

forward_for_tracing(image: Tensor) → Tensor | dict[str, Tensor][source]#

Model forward function used for the model tracing during model exportation.

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.

class otx.backend.native.models.TimmModelMultilabelCls(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: str, optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _multi_label_cls_metric_callable>, torch_compile: bool = False)[source]#

Bases: OTXMultilabelClsModel

TimmModel for multi-label classification task.

Parameters:

  • label_info (LabelInfoTypes) – The label information for the classification task.

  • model_name (str) – The name of the model. You can find available models at timm.list_models() or timm.list_pretrained().

  • input_size (tuple[int, int], optional) – Model input size in the order of height and width. Defaults to (224, 224).

  • pretrained (bool, optional) – Whether to load pretrained weights. Defaults to True.

  • optimizer (OptimizerCallable, optional) – The optimizer callable for training the model.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable, optional) – The learning rate scheduler callable.

  • metric (MetricCallable, optional) – The metric callable for evaluating the model. Defaults to MultiLabelClsMetricCallable.

  • torch_compile (bool, optional) – Whether to compile the model using TorchScript. Defaults to False.

Initialize the base model with the given parameters.

Parameters:

  • label_info (LabelInfoTypes) – Information about the labels used in the model.

  • data_input_params (DataInputParams) – Parameters of the input data such as input size, mean, and std.

  • model_name (str, optional) – Name of the model. Defaults to “OTXModel”.

  • optimizer (OptimizerCallable, optional) – Callable for the optimizer. Defaults to DefaultOptimizerCallable.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler. Defaults to DefaultSchedulerCallable.

  • metric (MetricCallable, optional) – Callable for the metric. Defaults to NullMetricCallable.

  • torch_compile (bool, optional) – Flag to indicate if torch.compile should be used. Defaults to False.

  • tile_config (TileConfig, optional) – Configuration for tiling. Defaults to TileConfig(enable_tiler=False).

Returns:

None

forward_for_tracing(image: torch.Tensor) → torch.Tensor | dict[str, torch.Tensor][source]#

Model forward function used for the model tracing during model exportation.

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.

class otx.backend.native.models.Uflow(data_input_params: DataInputParams, label_info: LabelInfoTypes = AnomalyLabelInfo(label_names=['Normal', 'Anomaly'], label_ids=['0', '1'], label_groups=[['Normal', 'Anomaly']]), backbone: str = 'resnet18', flow_steps: int = 4, affine_clamp: float = 2.0, affine_subnet_channels_ratio: float = 1.0, permute_soft: bool = False, task: Literal[OTXTaskType.ANOMALY, OTXTaskType.ANOMALY_CLASSIFICATION, OTXTaskType.ANOMALY_DETECTION, OTXTaskType.ANOMALY_SEGMENTATION] = OTXTaskType.ANOMALY_CLASSIFICATION)[source]#

Bases: AnomalyMixin, Uflow, OTXAnomaly

OTX UFlow model.

Parameters:

  • label_info (LabelInfoTypes, optional) – Label information. Defaults to AnomalyLabelInfo().

  • backbone (str, optional) – Feature extractor backbone. Defaults to “resnet18”.

  • flow_steps (int, optional) – Number of flow steps. Defaults to 4.

  • affine_clamp (float, optional) – Affine clamp. Defaults to 2.0.

  • affine_subnet_channels_ratio (float, optional) – Affine subnet channels ratio. Defaults to 1.0.

  • permute_soft (bool, optional) – Whether to use soft permutation. Defaults to False.

  • (Literal[ (task) – OTXTaskType.ANOMALY_CLASSIFICATION, OTXTaskType.ANOMALY_DETECTION, OTXTaskType.ANOMALY_SEGMENTATION ], optional): Task type of Anomaly Task. Defaults to OTXTaskType.ANOMALY_CLASSIFICATION.

  • input_size (tuple[int, int], optional) – Model input size in the order of height and width. Defaults to (256, 256)

class otx.backend.native.models.VisionTransformerHLabelCls(label_info: HLabelInfo, data_input_params: DataInputParams, model_name: str = 'vit-tiny', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _mixed_hlabel_accuracy>, torch_compile: bool = False)[source]#

Bases: ForwardExplainMixInForViT, OTXHlabelClsModel

VisionTransformerForHLabelCls is a model designed for hierarchical label classification using ViT architecture.

Parameters:

  • label_info (HLabelInfo) – Information about the hierarchical labels.

  • model_name (str) – Name of the Vision Transformer model to use.

  • data_input_params (DataInputParams) – Parameters for data input.

  • optimizer (OptimizerCallable) – Callable for the optimizer.

  • scheduler (LRSchedulerCallable | LRSchedulerListCallable) – Callable for the learning rate scheduler.

  • metric (MetricCallable) – Callable for the metric.

  • torch_compile (bool) – Whether to use torch.compile for the model.

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.

class otx.backend.native.models.VisionTransformerMulticlassCls(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: str = 'vit-tiny', freeze_backbone: bool = False, peft: Literal['lora', 'dora'] | None = None, optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _multi_class_cls_metric_callable>, torch_compile: bool = False)[source]#

Bases: ForwardExplainMixInForViT, OTXMulticlassClsModel

DeitTiny Model for multi-class classification task.

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.

class otx.backend.native.models.VisionTransformerMultilabelCls(label_info: LabelInfoTypes, data_input_params: DataInputParams, model_name: str = 'vit-tiny', optimizer: OptimizerCallable = <function _default_optimizer_callable>, scheduler: LRSchedulerCallable | LRSchedulerListCallable = <function _default_scheduler_callable>, metric: MetricCallable = <function _multi_label_cls_metric_callable>, torch_compile: bool = False)[source]#

Bases: ForwardExplainMixInForViT, OTXMultilabelClsModel

DeitTiny Model for multi-class classification task.

load_from_otx_v1_ckpt(state_dict: dict, add_prefix: str = 'model.') → dict[source]#

Load the previous OTX ckpt according to OTX2.0.