PhysicalAI-Robotics-GR00T-X-Embodiment-Sim

Loads and streams 334,635 pre-recorded robot manipulation trajectories from NVIDIA's GR00T-X embodied AI framework, organized by task category and robot morphology. Implements HuggingFace Datasets API for efficient memory-mapped access to multi-modal trajectory data (video frames, joint states, action sequences, language annotations) without requiring full dataset download. Supports streaming mode for training on machines with limited disk space.

Extracts and parses structured annotations from trajectory records including natural language task descriptions, robot morphology metadata, environment context, and action semantics. Implements a schema-based parser that maps raw trajectory fields to standardized embodied AI representations (state-action-reward tuples, task graphs, skill hierarchies). Supports filtering and grouping trajectories by semantic attributes without loading full video data.

Extracts and decodes video frames from trajectory records with configurable temporal sampling (every Nth frame, keyframes only, or full sequence). Implements efficient frame buffering and lazy loading to avoid memory exhaustion on large trajectory sequences. Supports multiple video codecs (H.264, VP9) and output formats (RGB, BGR, grayscale) with optional preprocessing (resizing, normalization) for model input compatibility.

Aligns joint state sequences (proprioceptive sensor readings) with video frames and action sequences using timestamp-based or frame-index synchronization. Handles variable-length trajectories and missing sensor data through interpolation or padding. Outputs aligned state-action-observation tuples suitable for imitation learning, with optional filtering for physically plausible state transitions (e.g., joint velocity limits).

Organizes 334K trajectories into a task hierarchy (e.g., manipulation > grasping > pick-and-place) and enables filtering by task level, parent task, or task attributes. Implements a tree-based index structure for fast hierarchical queries without scanning all trajectories. Supports task similarity search to find related trajectories for curriculum learning or data augmentation.

Filters trajectories by robot morphology (e.g., 7-DOF arm, mobile manipulator, humanoid) and enables morphology-aware data loading that adapts trajectory representations to target robot kinematics. Implements morphology metadata indexing for fast filtering and optional trajectory morphology conversion (e.g., remapping joint indices for different arm configurations).

Implements efficient batch sampling strategies for training (random, sequential, stratified by task/morphology, curriculum-based) with support for weighted sampling to balance task distribution. Integrates with PyTorch DataLoader for distributed training across multiple GPUs/TPUs. Handles variable-length trajectories through padding, truncation, or dynamic batching.

Analyzes trajectories for quality metrics including action smoothness, state plausibility, video frame quality, and task completion indicators. Implements automated filtering to remove low-quality trajectories (e.g., with jerky motions, sensor noise, or incomplete tasks) without manual inspection. Outputs quality scores and filtering recommendations for dataset curation.

Generates synthetic trajectory variations through action perturbation, state interpolation, and video augmentation (rotation, scaling, color jittering). Implements physics-aware augmentation that respects joint limits and collision constraints. Supports trajectory mixing (blending two trajectories) for data augmentation without manual trajectory recording.