dgs.models.similarity.pose_similarity.ObjectKeypointSimilarity.forward

ObjectKeypointSimilarity.forward(data: State, target: State) → torch.Tensor[source]

Compute the object key-point similarity between a ground truth label and detected key points.

There has to be one key point of the label for any detection. (Batch sizes have to match)

Notes

Compute the key-point similarity \(\mathtt{ks}_i\) for every joint between every detection and the respective ground truth annotation.

\[\mathtt{ks}_i = \exp(-\dfrac{d_i^2}{2s^2k_i^2})\]

The key-point similarity \(\mathtt{OKS}\) is then computed as the weighted sum using the key-point visibilities as weights.

\[\mathtt{OKS} = \dfrac{\sum_i \mathtt{ks}_i \cdot \delta (v_i > 0)}{\sum_i \delta (v_i > 0)}\]

  • \(d_i\) the euclidean distance between the ground truth and detected key point

  • \(k_i\) the constant for the key point, computed as \(k=2\cdot\sigma\)

  • \(v_i\) the visibility of the key point, with

    • 0 = unlabeled

    • 1 = labeled but not visible

    • 2 = labeled but visible

  • \(s\) the scale of the ground truth object, with \(s^2\) becoming the object’s segmented area

Parameters:

  • data – A State object containing at least the key points and the bounding box. Shape N.

  • target – A State containing at least the target key points. Shape T.

Returns:

A (Float)Tensor of shape [N x T] with values in [0..1]. If requested, the softmax is computed along the -1 dimension, resulting in probability distributions for each value of the input data.