Video

The video submodule handles video decoding and tensor-level transforms. For interactive playback and visualization, see the Video Player instead.

Decoding

sign_language_tools.video.decoding iterates over the frames of a video file using vidgear, yielding each frame together with its estimated timestamp. It's the same decoder used internally by extract_poses_from_video_file.

from sign_language_tools.video.decoding import iterate_video_frames_using_vidgear

for timestamp_ms, frame in iterate_video_frames_using_vidgear("video.mp4", show_progress=True):
    ...  # frame is an RGB array of shape (H, W, 3)

Transforms

sign_language_tools.video.transforms provides PyTorch-tensor transforms for cropping and padding video clips along the temporal dimension, mirroring the pose TemporalCrop/TemporalRandomCrop transforms but for (T, C, H, W) video tensors.

import torch
from sign_language_tools.video.transforms import TemporalCrop, TemporalPad

clip = torch.rand(120, 3, 224, 224)  # (T, C, H, W)

clip = TemporalCrop(max_width=64)(clip)
clip = TemporalPad(min_width=96)(clip)

Reference

sign_language_tools.video.decoding.iterate_video_frames_using_vidgear

iterate_video_frames_using_vidgear(
    video_path: str, show_progress: bool = False
) -> Iterator[tuple[int, np.ndarray]]

Iterates over the frames of a video file, decoded with vidgear's CamGear.

Parameters:

Name	Type	Description	Default
video_path	str	Path to the video file to decode.	required
show_progress	bool	Whether to display a tqdm progress bar while iterating.	False

Yields:

Type	Description
tuple[int, ndarray]	tuple[int, np.ndarray]: A (timestamp_ms, frame) pair for each decoded frame, where timestamp_ms is the frame's estimated timestamp in milliseconds (computed from the frame index and the video's FPS) and frame is an RGB image array with shape (H, W, 3).

Raises:

Type	Description
ZeroDivisionError	If the video's reported FPS is 0 (can happen with some codecs/backends), since timestamps are computed as frame_nb * 1000 / fps.

sign_language_tools.video.transforms.temporal_crop.TemporalCrop

TemporalCrop(max_width: int, location: str = 'start')

Bases: Transform

Crops a video along the temporal dimension to a fixed number of frames.

If the video already has at most max_width frames, it is returned unchanged.

Parameters:

Name	Type	Description	Default
max_width	int	The maximum number of frames the output video can have.	required
location	str	Where to take the frames from. Either "start", "center", or "end".	'start'

Example

import torch from sign_language_tools.video.transforms import TemporalCrop video = torch.arange(5 * 3 * 2 * 2, dtype=torch.float32).reshape(5, 3, 2, 2) crop = TemporalCrop(max_width=3, location='start') crop(video).shape torch.Size([3, 3, 2, 2])

__call__

__call__(video: Tensor) -> torch.Tensor

Crops the video down to max_width frames.

Parameters:

Name	Type	Description	Default
video	Tensor	A video tensor with shape (T, C, H, W).	required

Returns:

Type	Description
Tensor	torch.Tensor: The cropped video tensor with shape (max_width, C, H, W), or the original video if it already has at most max_width frames.

sign_language_tools.video.transforms.temporal_crop.TemporalRandomCrop

TemporalRandomCrop(max_width: int)

Bases: Transform

Crops a video along the temporal dimension to a fixed number of frames at a random offset.

If the video already has at most max_width frames, it is returned unchanged. Otherwise, a window of max_width consecutive frames is selected starting at a uniformly random offset.

Parameters:

Name	Type	Description	Default
max_width	int	The maximum number of frames the output video can have.	required

Example

import torch from sign_language_tools.video.transforms import TemporalRandomCrop video = torch.arange(5 * 3 * 2 * 2, dtype=torch.float32).reshape(5, 3, 2, 2) crop = TemporalRandomCrop(max_width=3) crop(video).shape torch.Size([3, 3, 2, 2])

__call__

__call__(video: Tensor) -> torch.Tensor

Crops the video down to max_width consecutive frames starting at a random offset.

Parameters:

Name	Type	Description	Default
video	Tensor	A video tensor with shape (T, C, H, W).	required

Returns:

Type	Description
Tensor	torch.Tensor: The cropped video tensor with shape (max_width, C, H, W), or the original video if it already has at most max_width frames.

sign_language_tools.video.transforms.temporal_padding.TemporalPad

TemporalPad(min_width: int, location: str = 'end')

Bases: Transform

Pads a video along the temporal dimension by repeating its first or last frame.

If the video already has at least min_width frames, it is returned unchanged.

Parameters:

Name	Type	Description	Default
min_width	int	The minimum number of frames the output video must have.	required
location	str	Where to add the padding frames. Either "start" or "end".	'end'

Example

import torch from sign_language_tools.video.transforms import TemporalPad video = torch.arange(3 * 3 * 2 * 2, dtype=torch.float32).reshape(3, 3, 2, 2) pad = TemporalPad(min_width=5, location='end') pad(video).shape torch.Size([5, 3, 2, 2])

__call__

__call__(video: Tensor) -> torch.Tensor

Pads the video with repeated frames until it reaches min_width frames.

Parameters:

Name	Type	Description	Default
video	Tensor	A video tensor with shape (T, C, H, W).	required

Returns:

Type	Description
Tensor	torch.Tensor: The padded video tensor with shape (min_width, C, H, W), or the original video if it already has at least min_width frames.