Task Specification Encoding With Language And Visual Goal Conditioning

Generalist robot policy model from Open X-Embodiment.

Unique: Supports dual task conditioning pathways (language instructions and visual goals) through separate tokenizers that feed into a unified transformer sequence, enabling the same policy to follow either linguistic or visual task specifications without architectural branching. Task tokens are simply concatenated with observation tokens, treating task specification as part of the input sequence.

vs others: More flexible than single-modality task conditioning (language-only or vision-only) by supporting both simultaneously, and more efficient than separate language and vision models by sharing the transformer backbone across conditioning modalities.