videoclipreassembly_mapper

September 8, 2026 · View on GitHub

Reassemble hand-action results from overlapping video clips.

When long videos are chopped into overlapping clips (e.g. 5 s with 2 s overlap via VideoSplitByDurationMapper), each clip is processed independently through the 3-D motion labelling pipeline. This operator merges the per-clip results back into one unified result per original video, including:

  • hand_action_tags — states, actions, valid_frame_ids, joints
  • video_camera_pose_tags — cam_c2w array
  • hand_reconstruction_hawor_tags — frame_ids converted to global
  • video_frames — per-clip frame path lists merged into one global list
  • camera_calibration_moge_tags — per-clip depth/intrinsics merged
  • clips — replaced with the original video path

Clip global offsets are determined automatically by pixel-matching overlapping frames between consecutive clips, rather than assuming an ideal step size. This handles ffmpeg keyframe-alignment drift that causes actual clip boundaries to differ from the nominal (split_duration - overlap_duration) * fps calculation.

Reference (paper §3.1): "To enhance efficiency, we chop long videos into overlapping 20-second clips in this stage and recompose their results."

从重叠的视频片段中重组手部动作结果。

当长视频被切分为重叠的片段时（例如通过 VideoSplitByDurationMapper 切分为 5 秒片段，重叠 2 秒），每个片段会通过 3D 动作标注管线进行独立处理。此算子将各片段的结果合并回原始视频的单一统一结果中，包括：

  • hand_action_tags — states、actions、valid_frame_ids、joints
  • video_camera_pose_tags — cam_c2w 数组
  • hand_reconstruction_hawor_tags — 转换为全局的 frame_ids
  • video_frames — 各片段的帧路径列表合并为一个全局列表
  • camera_calibration_moge_tags — 各片段的深度/内参合并
  • clips — 替换为原始视频路径

片段的全局偏移量通过像素匹配连续片段之间的重叠帧来自动确定，而不是假设理想的步长。这可以处理 ffmpeg 关键帧对齐漂移问题，该问题会导致实际片段边界与标称的 (split_duration - overlap_duration) * fps 计算结果存在差异。

参考文献（论文 §3.1）： “为提高效率，我们在此阶段将长视频切分为重叠的 20 秒片段，并重组其结果。”

Type 算子类型: mapper

Tags 标签: cpu, video

🔧 Parameter Configuration 参数配置

name 参数名type 类型default 默认值desc 说明
hand_action_field<class 'str'>'hand_action_tags'
camera_pose_field<class 'str'>'video_camera_pose_tags'
hand_reconstruction_field<class 'str'>'hand_reconstruction_hawor_tags'
frame_field<class 'str'>'video_frames'
moge_field<class 'str'>'camera_calibration_moge_tags'
clip_field<class 'str'>'clips'
video_key<class 'str'>'videos'
split_duration<class 'float'>None
overlap_duration<class 'float'>None
fps<class 'float'>None
args``
kwargs``