Multi Step Visual Task Composition

via “vision-task-decomposition-prompting”

A free DeepLearning.AI short course on how to prompt computer vision models with natural language, bounding boxes, segmentation masks, coordinate points, and other images.

Unique: Applies chain-of-thought and task decomposition patterns from language model reasoning to the vision domain, teaching how to structure visual analysis as a sequence of focused prompts rather than attempting to solve complex tasks in a single pass

vs others: Extends beyond single-prompt vision guidance by addressing the emerging pattern of vision-based agents and workflows, providing patterns for orchestrating multiple vision model calls to achieve complex analysis that would be difficult or impossible in a single prompt

via “multi-step-visual-task-composition”

* ⭐ 03/2023: [Scaling up GANs for Text-to-Image Synthesis (GigaGAN)](https://arxiv.org/abs/2303.05511)

Unique: Uses an LLM to decompose high-level visual requests into executable task sequences, automatically routing outputs between models and managing intermediate state, rather than requiring users to manually specify each step.

vs others: More flexible than hardcoded pipelines (which support only predefined sequences) and more intelligent than single-operation APIs (which require manual chaining).

via “complex compositional instruction following”

via “multi-step-workflow-composition”