Abstract
Achieving streaming, fine-grained control over the outputs of autoregressive video diffusion models remains challenging, making it difficult to ensure that they consistently align with user expectations. To bridge this gap, we propose stReaming drag-oriEnted interactiVe vidEo manipuLation (REVEL), a new task that enables users to modify generated videos anytime on anything via fine-grained, interactive drag. Beyond DragVideo and SG-I2V, REVEL unifies drag-style video manipulation as editing and animating video frames with both supporting user-specified translation, deformation, and rotation effects, making drag operations versatile. In resolving REVEL, we observe: i) drag-induced perturbations accumulate in latent space, causing severe latent distribution drift that halts the drag process; ii) streaming drag is easily disturbed by context frames, thereby yielding visually unnatural outcomes. We thus propose a training-free approach, DragStream, comprising: i) an adaptive distribution self-rectification strategy that leverages neighboring frames' statistics to effectively constrain the drift of latent embeddings; ii) a spatial-frequency selective optimization mechanism, allowing the model to fully exploit contextual information while mitigating its interference via selectively propagating visual cues along generation. Our method can be seamlessly integrated into existing autoregressive video diffusion models, and extensive experiments firmly demonstrate the effectiveness of our DragStream.
Pipeline Overview
The pipeline overview of REVEL task.
Streaming Manipulation Results on Self-forcing
Streaming Manipulation Results on Causvid
Streaming Manipulation Results on Long Videos
Our method demonstrates strong performance in long video manipulation. As shown in the visualizations results below, despite the autoregressively accumulated errors, our method can still effectively support drag-based manipulation at 5s, 10s, and 20s timestamps. Drag operations induced by our approach do not degrade the quality of manipulated video frames noticeably compared with the baseline.
Streaming Manipulation Results on Occlusion and Reappearance
Streaming Manipulation Results on Moving out of frame