LiveLight is the first diffusion-based framework for real-time streaming video relighting with interactive 3D point-light control—adjust position, intensity, and color while preserving appearance and temporal coherence.
15.78 FPSreal-time streaming inference0.253 slatency with the standard VAE4 stepsdistilled denoising by default
Interactive lighting interface
Move the light. Watch the frame respond.
These screen recordings show LiveLight responding to point-light edits inside the interactive demo. Lighting position, intensity, and color can be adjusted while the relit stream updates continuously.
The LiveLight demo combines source preview, 3D drag-based light editing, intensity and color controls, and real-time streaming output in one interface.Open original PDF ↗
Case 01Library portraitLive controlCase 04Street portraitLive controlCase 08Animated drummerLive controlCase 06Stylized alleyLive control
Interactive results
Relight as the scene unfolds.
Each clip is arranged as input video, target light map, and LiveLight output. Filter the gallery by control setting, hover to preview, or open a result in the large viewer.
01Color control
Change RGB color, position, and intensity on the fly.
02Short-form
Responsive lighting with strong content preservation.
03Long-form
Rolling-window generation with bounded latency.
Show
Paper
Abstract
We present LiveLight, the first diffusion-based framework for real-time streaming video relighting with interactive 3D lighting control. Achieving this requires effectively injecting dynamic 3D lighting into a diffusion model, maintaining high-fidelity generation under an extremely low number of function evaluations, and facilitating continuous streaming for interactive control.
LiveLight combines a lightweight adapter for Multi-Plane Light Irradiance conditions, a geometry-guided feedback branch for structure-preserving few-step distillation, and a progressive rolling-window strategy that maintains temporal coherence while supporting arbitrarily long video. Experiments on real-world and synthetic benchmarks demonstrate state-of-the-art relighting quality at real-time speed.
Why LiveLight
From “render, then wait” to light as you go.
Today’s diffusion-based video relighting systems are built around an offline, clip-level workflow: the complete input video and its lighting trajectory must be known before generation begins. The result arrives only after a long wait, with no way to reshape the light while the scene is unfolding.
Relighting each frame independently removes that wait, but sacrifices the temporal coherence a video needs. LiveLight asks a different question: can relighting behave like a live creative tool—responding frame by frame while preserving geometry, appearance, and motion?
Conventional methods wait for a full clip and a predefined light path. LiveLight emits relit frames with low latency while lighting control continues.Open original PDF ↗
01
Dynamic 3D control
Inject spatially varying point-light position, intensity, and color without erasing identity or texture.
02
Fidelity at real-time speed
Preserve plausible geometry, highlights, and shadows under an extremely small denoising budget.
03
True continuous streaming
Produce frames incrementally with strict temporal coherence, bounded latency, and no fixed video length.
How it works
Three pieces, one live loop.
Structured light control enters early, geometry-aware feedback protects shading quality during distillation, and a rolling latent window turns clip generation into a continuous stream.
Overview of LiveLight. The method couples lightweight light injection, real-time distillation, and rolling-window video generation.Open original PDF ↗
01
Inject 3D light efficiently
Four depth-aware MPLI planes encode light position, intensity, and RGB color. A roughly 1M-parameter adapter injects compact lighting features into shallow diffusion layers.
02
Distill without flattening geometry
A frozen geometry estimator feeds depth- and normal-aware supervision back during training, preserving plausible highlights and shadows without inference overhead.
03
Stream through a rolling window
The oldest micro-chunk is emitted while latent history is refined and a new noisy chunk enters, enabling per-frame control and arbitrarily long relighting.
Quantitative
Real time, without giving up quality.
LiveLight leads the reported quality and temporal metrics while reducing per-video processing time from tens or hundreds of seconds to about one second.
Method
Aesthetic ↑
FVD ↓
PSNRY ↑
PSNRlight ↑
Time ↓
Temporal ↑
AnyV2V
0.4929
1082.6
11.063
15.923
89 s
0.9656
LightCtrl
0.5307
987.3
11.782
18.486
122 s
0.9559
Light-A-Video
0.5533
1053.8
11.247
17.612
115 s
0.9740
TC-Light
0.5170
862.4
12.136
19.274
38 s
0.9799
LiveLight (Ours)
0.5748
738.5
12.847
21.538
1.01 s
0.9872
Denoising steps4 steps · 15.78 FPS
Two steps are faster but introduce artifacts; eight steps offer only marginal gains at 9.43 FPS.
Micro-chunk sizeM = 4 · balanced
M = 2 weakens temporal stability, while M = 8 improves context at a substantial throughput cost.
MPLI planesP = 4 · default
Four planes capture depth-dependent illumination; eight add parameters for only marginal improvement.
Human evaluation
Users prefer LiveLight across every axis.
Participants rated light controllability, appearance preservation, temporal consistency, and motion fidelity. LiveLight receives the highest combined Good + Excellent score in all four dimensions.
Distribution of user ratings for LiveLight (Ours) and four baselines. Percent labels report cumulative preference levels.Open original PDF ↗
Public arXiv and code URLs will be added when they become available.
@article{livelight2026,
title = {LiveLight: Real-time Streaming Video Relighting
with Interactive Control},
author = {Yue Ma and Jiangming Wang and Yucheng Wang and
Xilai Wang and Zhiyuan Li and Xinyu Wang and
Hongyu Liu and Ruofan Liang and Songchun Zhang and
Yuxuan Xue and Qifeng Chen},
journal = {ACM Transactions on Graphics},
year = {2026}
}
