dgs.models.combine.dynamic.DynamicAlphaCombine

class dgs.models.combine.dynamic.DynamicAlphaCombine(*args: Any, **kwargs: Any)

Use inputs and multiple alpha modules to weight the similarity matrices.

Notes

The models for computing the per-similarity alpha has to be set manually after the initialization.

Given N inputs to the alpha module (e.g. the visual embeddings of N images, or N different sized inputs like the bbox, pose, and visual embedding of a single crop), compute the alpha weights for the similarity matrices. Then use \(\alpha_i\) to compute the weighted sum of all the similarity matrices \(S_i\) as \(S = \sum_N \alpha_i \cdot S_i\).

Every \(\alpha_i\) can either be a single float value in range \([0, 1]\) or a (float-) tensor of the same shape as \(S_i\) again with values in \([0,1]\).

Params

alpha_modules (list[NodePath]):

A list containing paths to multiple BaseAlphaModule s.

Optional Params

__init__(*args, **kwargs)

Methods

configure_torch_module(module[, train])	Set compute mode and send model to the device or multiple parallel devices if applicable.
forward(*tensors[, s])	The forward call of this module combines an arbitrary number of similarity matrices using an importance weight \(\alpha\).
terminate()	Terminate this module and all of its submodules.
validate_params(validations[, attrib_name])	Given per key validations, validate this module's parameters.

Attributes

device	Get the device of this module.
is_training	Get whether this module is set to training-mode.
module_name	Get the name of the module.
module_type
name	Get the name of the module.
name_safe	Get the escaped name of the module usable in filepaths by replacing spaces and underscores.
precision	Get the (floating point) precision used in multiple parts of this module.
alpha_models	The model that computes the alpha values from given inputs.
softmax