Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold (DragGAN)

Enables real-time dragging of semantic points on generated images to deform content while maintaining photorealism and semantic coherence. Uses a feature tracking mechanism that follows user-specified points through the generative process, combined with latent code optimization that adjusts the GAN's internal representation to satisfy drag constraints. The system operates directly on the generative manifold by iteratively updating the latent code while preserving the generator's learned priors, avoiding the need for retraining or fine-tuning.

Tracks user-specified points through the multi-scale feature hierarchy of a generative model by computing feature correspondences at intermediate generator layers. Uses bilinear interpolation and gradient-based optimization to identify which features in deeper layers correspond to the dragged point, enabling the system to understand what semantic content is being manipulated. This layer-wise tracking allows the optimization to apply constraints at multiple scales simultaneously, improving coherence.

Iteratively updates the GAN's latent input code to satisfy user-specified spatial constraints (drag points) while minimizing deviation from the original latent code. Uses gradient descent on a loss function combining point position error and latent code regularization, enabling smooth optimization within the learned generative manifold. The optimization preserves the generator's learned priors by staying close to the original latent code, avoiding out-of-distribution artifacts that occur with unconstrained editing.

Provides an interactive interface where users click and drag points on generated images to specify spatial constraints, with live or near-real-time visual feedback of the deformation. The UI handles point selection, tracking, and constraint specification, then triggers the latent optimization pipeline. Supports multiple simultaneous drag points and provides visual feedback (e.g., point trajectories, constraint vectors) to guide user interaction.

Manages multiple simultaneous drag constraints by formulating them as a multi-objective optimization problem where the loss function aggregates errors from all point constraints. Implements constraint weighting and prioritization to handle conflicting constraints gracefully, allowing users to drag multiple points simultaneously while the optimizer finds a solution that satisfies all constraints as well as possible. Uses weighted least-squares formulation to balance constraint satisfaction across all points.

Prevents the optimization from drifting away from the learned generative manifold by adding a regularization term that penalizes deviation of the latent code from its initial value. This L2 regularization on the latent code ensures that the optimized result remains within the region of latent space where the generator produces high-quality, photorealistic images. The regularization weight controls the trade-off between constraint satisfaction and manifold preservation.