SPARE Heterogeneous Collaboration for Video World Models

Single-step Prediction and Action-conditioned Regeneration at the Edge: one-step cloud prediction with on-device decoding and regeneration when the action changes.

The University of Texas at Austin

Rollouts generated by SPARE: the 14B cloud model takes one denoising step per chunk, and the 1.3B device model regenerates the chunks where the action changes. Eight of the sixteen gallery scenes, 12-s excerpts; click a scene to compare it with the single models.

21.4FPS
throughput on WorldScore-300 (one RTX PRO 6000 per model), above the 16-FPS playback rate
cloud only 7.3 FPS (2.95×)
7.07/ 10
VLM-judged video quality, similar to the four-step 14B model
cloud only 7.09, device only 6.49
404J
cloud GPU energy per second of video (total energy is higher, see Results)
cloud only 1,172 J (2.90× lower)
0.595s
from an action to its first decoded frame
cloud only 1.66 s
Predefined actions

One rollout, three ways to serve it

The same input image and seed, with predefined action changes every six seconds that are not provided to the cloud in advance. The hexagon strip shows which model generated each 0.75-s chunk of the SPARE video.

syncedChoose a scene

Abstract

Interactive video world models let users explore generated environments, empowering applications from games to robotics. To achieve good quality, video world models often need to run on cloud GPUs, which can be costly for model deployers. Cloud deployment can also lead to higher response latency. A small on-device model can respond locally, but it loses details and drifts over long rollouts. Motivated by the complementary strengths of cloud and edge computing, we study how to deploy a video world model across heterogeneous edge devices and cloud GPUs. Heterogeneous deployment is challenging because current models typically require substantial cloud computation to maintain video quality, increasing serving cost and potentially delaying responses to user inputs. We present SPARE (Single-step Prediction and Action-conditioned Regeneration at the Edge), a simple collaboration method that combines one-step cloud prediction with on-device decoding and regeneration when the action changes. Compared with the original cloud-only generation, SPARE achieves 2.95× the throughput and reduces cloud GPU energy per second of generated video by a factor of 2.90, while maintaining similar visual quality.

Method

How SPARE splits the work

A chunk is a short group of video frames conditioned on the preceding video and the current action, such as a movement direction or no movement. Denoising steps predict a clean video latent from a noisy input. SPARE combines one-step cloud lookahead with local decoding and four-step regeneration when the action changes. Both pretrained models retain their weights.

chunk i−1
chunk i · action changes
chunk i+1
Cloud14B
1 step
1 step made under W · discarded
1 step continues from chunk i
Device1.3B
decode
4 steps
decode
Display
Frame of chunk i-1, generated by the cloud modelcloud
Frame of chunk i, regenerated by the device modeldevice
Frame of chunk i+1, generated by the cloud modelcloud
Action
Cloud · 14B

One-step lookahead

While the current chunk is displayed, the cloud predicts the next chunk assuming the current action persists. It sends the x0 prediction from its first denoising step. The device’s VAE decoder converts this clean-latent estimate into frames, so the cloud does not run the decoder.

Device · 1.3B

Decoding and regeneration

If the action changes, the device discards the prediction made under the previous action. Its small model regenerates the chunk with all four denoising steps under the new action, without waiting for a cloud response. A device that cannot run the small model at the playback rate keeps only the decoder and leaves regeneration to the cloud (SPARE without the device generator).

Both

Shared history

Every displayed chunk enters both models’ generation histories, so the cloud continues from the regenerated chunk. More frequent action changes reduce the number of cloud predictions that can be reused. No cache tensor is transmitted between the models.

Results

Measured quality and cost

On 300 WorldScore scenes with actions changing every six seconds and no access to future actions, SPARE scores 7.07 out of 10 under a Qwen3-VL-32B evaluator, against 7.09 for four-step cloud-only generation. Throughput rises from 7.3 to 21.4 FPS (2.95×), and cloud GPU energy per second of video falls from 1,172 to 404 J (a factor of 2.90). These ratios use unrounded measured means. The total GPU energy of SPARE, 655 J, is higher than the 567 J of one-step cloud generation. Without the device generator, throughput is nearly unchanged (21.3 FPS) and VLM quality is 7.03, so local generation does not explain the throughput gain over four-step cloud generation. All results are for LingBot-World on workstation GPUs with injected network delay and 16-FPS playback.

Methods of Table 1 are on WorldScore-300. The green design points (Low-res cloud, Cloud step + refine, Periodic cloud) show cost on the 20-scene set, with quality on the same set as in the paper’s teaser; SPARE is plotted at its Table 1 values.
Table 1. Rollouts with changing actions. WorldScore-300 (300 scenes, one seed each), one RTX PRO 6000 per model; cloud-only rows also decode in the cloud. The 30-s rollouts use predefined action changes every six seconds that are not provided to the cloud in advance. FPS is throughput in decoded frames per second; the video plays at 16 FPS. VBench Quality aggregates six dimensions and Imag. is its imaging-quality dimension (MUSIQ). RPE is the rotation error to the commanded camera path, Resp. the response score after an action change. Quality and Drift are scores of a VLM judge (Qwen3-VL-32B). Energy is per second of video, measured from NVML total-energy counters, including idle power and discarded computations but excluding host and network energy. Latency is from an action to the first decoded frame. Network is the data exchanged between the device and the cloud per displayed frame, in both directions; cloud-only rows would stream encoded video, 6 to 10 kB per frame with H.264. †Future actions known. The gray row is SPARE without the device generator. Bold and underline mark the best and second best among the two-model methods and both SPARE variants. See the paper’s appendix for details.

Longer and repeated runs. On 20 scenes × 10 seeds, SPARE and four-step cloud-only generation score 7.27 and 7.40 in VLM quality. For 120-second rollouts (12 scenes × 3 seeds) they score 6.61 and 7.00. Both gaps favor the four-step cloud model.

Comparison

Side by side

Every method receives the same input image, actions, and seed. Great Wall and Stylized forest are the two rollouts of the qualitative figure in the paper, chosen by the gap in the VLM drift score: the device model, Hybrid SD, and SDVG lose the wall and the stylized trees, whereas the cloud model and SPARE keep both scenes to the end. The visible wall and trees illustrate appearance over time; they do not by themselves establish action accuracy. EC-Diff, which is not in that figure, keeps the wall but not the stylized trees. Lake and Alley in the rain are two further scenes of the 20-scene set.

Left side:
Right side:SPARE (Ours)
Robustness

Playback and response under delay and outages

Local regeneration makes the first decoded frame available 0.20–0.41 s earlier than without the device generator, but the player’s buffer hides most of this gain in display latency. On WorldScore-300, a 2-s network outage reduces stalls from 1.33 to 0.29 s per clip, at 723 instead of 668 J total GPU energy per video second. These are injected-delay and outage tests on one host with a 16-FPS player, not a wide-area deployment, and outage video quality has not been scored.

Table 3. Local regeneration and network outages. SPARE against its variant without the device generator; both use lookahead. Latencies are medians, energy is in J per video second. (a,b) 20 scenes, 80 inputs per row. (c) Replay of (b) with a 0.3-s buffer and catch-up; only display latency and stalls are recomputed. CPU-ready latency and energy are from the source runs. (d) WorldScore-300, 1200 inputs per row, with takeover enabled for SPARE. Bold marks the best value per block, including ties.
Appendix protocol

Outage duration sweep (20 scenes)

Older appendix test on the 20-scene set, not WorldScore-300. With cloud messages withheld for 20 s, playback stalls for 0.29 s per clip with the device generator, compared with 19.33 s without it. This tests playback continuity only, not video quality during the outage.

Player stall during an injected outage (20-scene appendix protocol)
20-scene set, real-time runtime with a 16-FPS player, outage 10 s into each clip, 80 inputs per point. With 20 s the cloud does not return before the clip ends. During an outage the device generates the chunks itself, 13 to 26 of the 39 per clip, so these chunks have the quality of the device model; their videos are not scored.
Design space

Where to spend the cloud

We vary cloud sampling depth, device computation, and generation history. On LingBot-World, the one-step 14B model scores 7.03 against 7.09 for four steps and 6.49 for the four-step 1.3B model. On the 20-scene analysis set, the small model’s regenerated chunks are sharper with cloud-generated history than when it generates the rollout alone. We use one cloud step for LingBot-World and the small model for regeneration after action changes.

Collaboration designs
Quality on WorldScore-300 (left axis) and throughput on the 20-scene set (right axis), from more work on the device (left) to more work in the cloud (right). Throughput and cloud energy here are 20-scene measurements (SPARE 21.2 FPS, 398 J), which differ from the WorldScore-300 values of Table 1 (21.4 FPS, 404 J).
Table 2. Ablation. 20 scenes × 4 seeds. The scores differ from Table 1, which uses WorldScore-300. The upper block removes components one after another (rows 1–3 give identical videos, row 4 is the four-step cloud model); the lower block changes how the invalidated chunk is handled, with stale-cloud guidance meaning that device regeneration is guided by the stale cloud prediction. Lat. is the time to the first decoded frame in seconds. †Quality on 40 clips, the new action is never shown.
Table 4. Device denoising steps at action changes. MUSIQ is measured on the first chunk after each change; rotation RPE and VLM-judged scene drift cover the whole clip. 20 scenes × one seed (1234), 20 rollouts and 80 action changes per row. †The cloud reference has access to future actions. Bold marks the best observed value per column. Direction errors, later-chunk metrics, and paired intervals are in the paper’s appendix.
Second model pair

Beyond LingBot-World

We apply SPARE to Self-Forcing 1.3B on the device and Krea Realtime 14B in the cloud, both based on Wan2.1, with text prompts as actions that change four times per rollout and no advance access. The VLM quality score is 7.95 against 7.60 for one-step cloud (95% CI of the difference [0.26, 0.44]), although the reference knows future prompts; final-segment MUSIQ is 66.1 against 63.6, while whole-clip imaging quality stays below four-step cloud. Without the device generator, throughput falls from 9.8 to 8.8 FPS and cloud energy rises from 844 to 1,025 J. Four device steps take 0.48 s versus 0.94 s for one cloud step. Throughput remains below the playback rate, and this dynamic-prompt test has no network or playback clock.

Table 5. Different model backbone. Self-Forcing 1.3B on the device and Krea Realtime 14B in the cloud, dynamic prompting. Quality and Drift are VLM judge scores, Cloud J is J per video second. Best in bold, second best underlined. ‡Quality, Drift, and Imag. of the row without the device generator are the one-step cloud scores, reused: its latents match those of one-step cloud in 12 cost runs, and these three metrics were not evaluated separately. Its FPS and Cloud J are measured in its own runs.
Scope

Limitations

  • One-step chunks are softer than four-step chunks, so the imaging quality of SPARE stays below that of the cloud model (65.3 against 66.4 MUSIQ).
  • Device regeneration needs a device that runs the small model at the playback rate, here a workstation GPU (RTX PRO 6000). An RTX A5000 does not, and uses cloud regeneration with device decoding.
  • The device generator does not raise the judged quality in the reported comparison and does not explain the throughput gain. It saves 4% of cloud energy but costs 4% more total energy, and its benefits are earlier decoded-frame availability and fewer playback stalls during injected outages, subject to the playback buffer.
  • After an outage the device keeps generating until the cloud leads again, 16 of the 39 chunks of a clip on average, at the quality of the device model.
  • With an action change every 3 s the quality falls below the four-step cloud model (7.16 against 7.58), and the 120-s rollouts score 6.61 against 7.00. Action sequences differ across settings.
  • Actions are predefined piecewise-constant sequences, the visual scores come from automatic metrics and a VLM judge, and the cost is measured with both GPUs in one host, an injected network delay, and one cloud GPU per user. A human study, closed-loop play, and a wide-area deployment are future work.