Research Scientist, Video & Multimodal

Overview

Innodata is seeking a Research Scientist to lead the science behind video and multimodal data design and evaluation for customers and frontier labs building video understanding, video-language, and video-generation models. This role focuses on turning model requirements into concrete data specifications, building evaluation methods, and proving experimentally which data choices improve model performance.

What You’ll Own

• Translate video and multimodal model needs into data specifications, including modality choices, annotation schemas, sampling strategies, and evaluation criteria.
• Develop evaluation methods for video understanding, including temporal grounding, long-context and long-horizon reasoning, cross-modal reasoning, and retrieval-and-grounding tasks.
• Develop evaluation methods for video generation, including fidelity, temporal coherence, and physical plausibility, especially where automatic metrics are weak and human judgment is needed.
• Determine how video should be structured, enriched, and sampled to maximize model value, including messy or domain-specific footage.
• Run experiments that test the impact of data decisions by fine-tuning and evaluating models, including ablations tied to measurable improvements.
• Design adversarial and stumping evaluations to expose failure modes in video and multimodal systems and feed those findings back into better data.
• Publish benchmarks, methodology, and papers that advance the field.
• Work with annotation teams, subject-matter experts, and synthetic-data pipelines to turn specifications into operational collection and labeling plans.

Requirements

• About 5+ years of hands-on industry experience in video understanding or multimodal machine learning.
• Bachelor’s degree in computer science, electrical engineering, or a related technical or quantitative field is required.
• Advanced degree such as an MS or PhD in a relevant field is preferred.
• Experience training and evaluating video or multimodal models, with strong PyTorch fundamentals.
• Practical fluency with video tooling and data formats such as ffmpeg, decord, temporal and COCO-style annotation, WebDataset, Parquet, Arrow, and HuggingFace Datasets.
• Experience fine-tuning large video or vision-language models using modern tools such as HuggingFace Transformers, PEFT, and efficient inference methods.
• Experience with long-form video, streaming, temporal segmentation, or synthetic video generation.
• Experience building evaluation sets, calibrating difficulty, and reasoning about what makes video data effective for a given objective.
• First-author publications or strong open-source contributions at venues such as CVPR, ICCV, ECCV, NeurIPS, or ICLR.
• Ability to explain research, data, and modeling decisions clearly to both expert and non-expert audiences, with a rigorous and reproducible approach to experiments and documentation.

Preferred Qualifications

• Interest in or hands-on experience with responsible-AI evaluation and red-teaming for video and multimodal systems.

Compensation

The expected salary range for this role is $160,000 - $185,000 per year, based on experience, skills, and qualifications.