dgs.models.embedding_generator.pose_based.KeyPointConvolutionPBEG

class dgs.models.embedding_generator.pose_based.KeyPointConvolutionPBEG(*args: Any, **kwargs: Any)

Create a short torch Module that has one convolutional layer reducing the key points using relational information and an arbitrary number of hidden fully connected layers at the end.

Module Name

KeyPointConvolutionPBEG

Description

First, the convolution of the key points is computed using a given number of :attr:nof_kernels, which will return J values after flattening the convolution output. Those values are then inserted with the four bounding box values into the first fully connected layer. There at this point, there can be an arbitrary number of hidden FC-layers.

Model Input: [B x J x j_dim] and [B x 4] Model Output: [B x self.embedding_size]

Params:

joint_shape ((tuple[int, int])) – Number of joints and number of dimensions of the joints as tuple.

Optional Params:

• hidden_layers_kp ((Union[list[int], tuple[int, …], None], optional)) – Respective size of every hidden layer after the convolution of the key points. The value can be None to use only one single convolution layer to cast the inputs before adding the bboxes. Default DEF_VAL.embed_gen.pose.KPCPBEG.hidden_layers_kp.

• hidden_layers ((Union[list[int], tuple[int, …], None], optional)) – Respective size of every hidden layer after adding the bounding boxes. The value can be None to use only one single linear NN-layer to cast the convoluted key points and bboxes to the outputs. Default DEF_VAL.embed_gen.pose.KPCPBEG.hidden_layers.

• bias ((bool, optional)) – Whether to use a bias term in the linear layers. Default DEF_VAL.embed_gen.pose.KPCPBEG.bias.

• nof_kernels ((int, optional)) – Define the number of kernels to use for convolution. Default DEF_VAL.embed_gen.pose.KPCPBEG.nof_kernels.

• bbox_format ((Union[str, tv_tensors.BoundingBoxFormat], optional)) – The target format of the bounding box coordinates. This will have influence on the results. Default DEF_VAL.embed_gen.pose.KPCPBEG.bbox_format.

Important Inherited Params:

embedding_size ((int)) – Output shape or size of the embedding.

Methods

__init__(*args, **kwargs)

configure_torch_module(module: torch.nn.Module, train: bool | None = None) → torch.nn.Module

Set compute mode and send model to the device or multiple parallel devices if applicable.

Parameters:

• module – The torch module instance to configure.

• train – Whether to train or eval this module, defaults to the value set in the base config.

Returns:

The module on the specified device or in parallel.

embedding_key_exists(s: State) → bool

Return whether the embedding_key of this model exists in a given state.

forward(ds: State) → tuple[torch.Tensor, torch.Tensor]

Forward pass of the custom key point convolution model.

Params:

ds – A State containing the key-points and the corresponding bounding boxes.

Returns:

This modules’ prediction. embeddings is describing the key-points and bounding boxes as a tensor of shape [B x E]. ids is the probability to predict a class. The ids are given as a tensor of shape [B x num_classes] with values in range [0, 1].

terminate() → None

Terminate this module and all of its submodules.

If nothing has to be done, just pass. Is used for terminating parallel execution and threads in specific models.

validate_params(validations: dict[str, list[str | type | tuple[str, any] | Callable[[any, any], bool]]], attrib_name: str = 'params') → None

Given per key validations, validate this module’s parameters.

Throws exceptions on invalid or nonexistent params.

Parameters:

• attrib_name – name of the attribute to validate, should be “params” and only for base class “config”

• validations –

  Dictionary with the name of the parameter as key and a list of validations as value. Every validation in this list has to be true for the validation to be successful.

  The value for the validation can have multiple types:

  • A lambda function or other type of callable

  • A string as reference to a predefined validation function with one argument

  • None for existence

  • A tuple with a string as reference to a predefined validation function with one additional argument

  • It is possible to write nested validations, but then every nested validation has to be a tuple, or a tuple of tuples. For convenience, there are implementations for “any”, “all”, “not”, “eq”, “neq”, and “xor”. Those can have data which is a tuple containing other tuples or validations, or a single validation.

  • Lists and other iterables can be validated using “forall” running the given validations for every item in the input. A single validation or a tuple of (nested) validations is accepted as data.

Example

This example is an excerpt of the validation for the BaseModule-configuration.

>>> validations = {
    "device": [
            str,
            ("any",
                [
                    ("in", ["cuda", "cpu"]),
                    ("instance", torch.device)
                ]
            )
        ],
        "print_prio": [("in", PRINT_PRIORITY)],
        "callable": (lambda value: value == 1),
    }

And within the class __init__() call:

>>> self.validate_params()

Raises:

• InvalidParameterException – If one of the parameters is invalid.

• ValidationException – If the validation list is invalid or contains an unknown validation.

Attributes

device	Get the device of this module.
is_training	Get whether this module is set to training-mode.
module_name	Get the name of the module.
module_type
name	Get the name of the module.
name_safe	Get the escaped name of the module usable in filepaths by replacing spaces and underscores.
precision	Get the (floating point) precision used in multiple parts of this module.
embedding_size	The size of the embedding.
nof_classes	The number of classes in the dataset / embedding.