videohandreconstruction_mapper

Use the WiLoR model for hand localization and reconstruction.

使用 WiLoR 模型进行手部定位与重建。

Type 算子类型: mapper

Tags 标签: gpu, video

🔧 Parameter Configuration 参数配置

name 参数名	type 类型	default 默认值	desc 说明
wilor_model_path	<class 'str'>	'wilor_final.ckpt'	The path to 'wilor_final.ckpt'.
wilor_model_config	<class 'str'>	'model_config.yaml'	The path to 'model_config.yaml' for the WiLOR model.
detector_model_path	<class 'str'>	'detector.pt'	The path to 'detector.pt' for the WiLOR model.
mano_right_path	<class 'str'>	'path_to_mano_right_pkl'	The path to 'MANO_RIGHT.pkl'. Users need to download this file from https://mano.is.tue.mpg.de/ and comply with the MANO license.
frame_num	typing.Annotated[int, Gt(gt=0)]	3	The number of frames to be extracted uniformly from the video. If it's 1, only the middle frame will be extracted. If it's 2, only the first and the last frames will be extracted. If it's larger than 2, in addition to the first and the last frames, other frames will be extracted uniformly within the video duration. If "duration" > 0, frame_num is the number of frames per segment.
duration	<class 'float'>	0	The duration of each segment in seconds. If 0, frames are extracted from the entire video. If duration > 0, the video is segmented into multiple segments based on duration, and frames are extracted from each segment.
batch_size	<class 'int'>	16	Batch size for simultaneous hand inference.
tag_field_name	<class 'str'>	'hand_reconstruction_tags'	The field name to store the tags. It's "hand_reconstruction_tags" in default.
frame_dir	<class 'str'>	DATA_JUICER_ASSETS_CACHE	Output directory to save extracted frames.
if_save_visualization	<class 'bool'>	True	Whether to save overlay images.
save_visualization_dir	<class 'str'>	DATA_JUICER_ASSETS_CACHE	The path for saving overlay images.
if_save_mesh	<class 'bool'>	True	Whether to save images of the hand mesh.
save_mesh_dir	<class 'str'>	DATA_JUICER_ASSETS_CACHE	The path for saving images of the hand mesh.
args		''	extra args
kwargs		''	extra args

📊 Effect demonstration 效果演示

test

VideoHandReconstructionMapper(wilor_model_path='wilor_final.ckpt', wilor_model_config='model_config.yaml', detector_model_path='detector.pt', mano_right_path='path_to_mano_right_pkl', frame_num=1, duration=1, frame_dir=DATA_JUICER_ASSETS_CACHE, if_save_visualization=True, save_visualization_dir=DATA_JUICER_ASSETS_CACHE, if_save_mesh=True, save_mesh_dir=DATA_JUICER_ASSETS_CACHE)

📥 input data 输入数据

📤 output data 输出数据

test_mul_proc

VideoHandReconstructionMapper(wilor_model_path='wilor_final.ckpt', wilor_model_config='model_config.yaml', detector_model_path='detector.pt', mano_right_path='path_to_mano_right_pkl', frame_num=1, duration=1, frame_dir=DATA_JUICER_ASSETS_CACHE, if_save_visualization=True, save_visualization_dir=DATA_JUICER_ASSETS_CACHE, if_save_mesh=True, save_mesh_dir=DATA_JUICER_ASSETS_CACHE)

📥 input data 输入数据

📤 output data 输出数据

🔗 related links 相关链接

• source code 源代码
• unit test 单元测试
• Return operator list 返回算子列表