HIVE Heterogeneous Collaboration for Video World Models

One-step cloud prediction with on-device decoding and regeneration after a control change.

The University of Texas at Austin

Rollouts generated by HIVE: the 14B cloud model takes one denoising step per chunk, and the 1.3B device model regenerates the chunks where the action changes. Eight of the sixteen gallery scenes, 12-s excerpts; click a scene to compare it with the single models.

21.2FPS
throughput, above the 16-FPS playback rate
cloud only 7.3 FPS
7.07/ 10
VLM-judged video quality, close to the four-step 14B model
cloud only 7.09, device only 6.49
398J
cloud GPU energy per second of video
cloud only 1,141 J
0.59s
from an action to its first decoded frame
cloud only 1.63 s
Scripted rollouts

One rollout, three ways to serve it

The same input image and seed, with scripted control changes every six seconds that are not provided to the cloud in advance. The hexagon strip shows which model generated each 0.75-s chunk of the HIVE video.

syncedChoose a scene

Abstract

Interactive video world models let users explore generated environments, empowering applications from games to robotics. To achieve good quality, video world models often need to run on cloud GPUs, which can be costly for model deployers. Cloud deployment can also lead to higher response latency. A small on-device model can respond locally, but it loses details and drifts over long rollouts. Motivated by the complementary strengths of cloud and edge computing, we study how to deploy a video world model across heterogeneous edge devices and cloud GPUs. Heterogeneous deployment is challenging because additional cloud computation can improve video quality but increases serving cost and can delay responses to user inputs. We present HIVE, a method that combines one-step cloud prediction with on-device decoding and regeneration after a control change. HIVE achieves judged video quality close to that of the four-step cloud model. Compared with four-step cloud-only generation, it achieves 2.9× the throughput and uses 65% less cloud GPU energy per second of generated video.

Method

How HIVE splits the work

A chunk is a short group of video frames conditioned on the preceding video and a control input, such as a movement direction or no movement. Denoising steps predict a clean video latent from a noisy input. HIVE combines one-step cloud lookahead with local decoding and four-step regeneration when the control input changes. Both pretrained models retain their weights.

chunk i−1
chunk i · action changes
chunk i+1
Cloud14B
1 step
1 step made under W · discarded
1 step continues from chunk i
Device1.3B
decode
4 steps
decode
Display
Frame of chunk i-1, generated by the cloud modelcloud
Frame of chunk i, regenerated by the device modeldevice
Frame of chunk i+1, generated by the cloud modelcloud
Action
Cloud · 14B

One-step lookahead

While the current chunk is displayed, the cloud predicts the next chunk assuming the control input persists. It sends the x0 prediction from its first denoising step. The device’s VAE decoder converts this clean-latent estimate into frames, so the cloud does not run the decoder.

Device · 1.3B

Decoding and regeneration

If the control input changes, the device discards the prediction made under the previous input. Its small model regenerates the chunk with all four denoising steps under the new input, without waiting for a cloud response. A device that cannot run the small model at the playback rate retains only the decoder and leaves regeneration to the cloud.

Both

Shared history

Every displayed chunk enters both models’ generation histories, so the cloud continues from the regenerated chunk. More frequent control changes reduce the number of cloud predictions that can be reused. No cache tensor is transmitted between the models.

Results

Measured quality and cost

On 300 scenes sampled from WorldScore, HIVE scores 7.07 out of 10 under a Qwen3-VL-32B evaluator, compared with 7.09 for four-step cloud-only generation. Throughput rises from 7.3 to 21.2 FPS, while cloud GPU energy falls from 1,141 to 398 J per video second: 2.9× the throughput and 65% less cloud energy.

Quality on WorldScore-300 (300 scenes, one seed), cost on the 20-scene set, one RTX PRO 6000 per model. The 30-s rollouts use scripted control changes every six seconds, not provided to the cloud in advance. Quality and Drift are scores of a VLM judge (Qwen3-VL-32B), VBench quality aggregates six dimensions, Imag. is its imaging-quality dimension (MUSIQ). RPE is the rotation error to the commanded camera path, Resp. the response score after an action change. Energy is per second of video. Latency is from an action to its first decoded frame. Network is the data exchanged per displayed frame. †Actions known in advance; these runs test the one-step cloud stream without regeneration. See the paper’s Appendix B for the scoring protocol. Bold and underline mark the best and second best among the two-model methods and HIVE.
Comparison

Side by side

Every method receives the same input image, actions, and seed. Great Wall and Stylized forest are the two rollouts of the qualitative figure in the paper, chosen by the gap in the VLM drift score: the device model, Hybrid SD, and SDVG lose the wall and the stylized trees, whereas the cloud model and HIVE keep both scenes to the end. EC-Diff, which is not in that figure, keeps the wall but not the stylized trees. Lake and Alley in the rain are two further scenes of the 20-scene set.

Left side:
Right side:HIVE (Ours)
Robustness

Playback during injected outages

The device generates chunks that cannot arrive from the cloud. With cloud messages withheld for 20 s, playback stalls for 0.29 s per clip, compared with 19.33 s without the device model. This is an injected outage test of playback continuity, not a wide-area deployment or a measurement of video quality during the outage.

Player stall during an injected outage
20-scene set, real-time runtime with a 16-FPS player, outage 10 s into each clip, 80 inputs per point. With 20 s the cloud does not return before the clip ends. During an outage the device generates the chunks itself, 13 to 26 of the 39 per clip, so these chunks have the quality of the device model; their videos are not scored.
Actions issued at wall-clock times, one-way network delay, 16-FPS player. End-to-end is the median latency from an input to its display, First frame the median time until its first frame is decoded, Stall the pauses of the player per clip, Cloud J the cloud energy per video second.
Design space

Where to spend the cloud

We vary cloud sampling depth, device computation, and generation history. On LingBot-World, the one-step 14B model scores 7.03 against 7.09 for four steps and 6.49 for the four-step 1.3B model. On the 20-scene analysis set, the small model’s regenerated chunks are sharper with cloud-generated history than when it generates the rollout alone. We use one cloud step for LingBot-World and the small model for regeneration after control changes.

Collaboration designs
Quality on WorldScore-300 (left axis) and throughput on the 20-scene set (right axis), from more work on the device (left) to more work in the cloud (right). The dashed line is the 16-FPS playback rate.
20 scenes × 4 seeds. The upper block removes components one after another (rows 1–3 give identical videos, row 4 is the four-step cloud model); the lower block changes how the invalidated chunk is handled. Lat. is the latency in seconds. †Quality on 40 clips, the new action is never shown.
Second model pair

Beyond LingBot-World

HIVE applied unchanged to Self-Forcing 1.3B on the device and Krea Realtime 14B in the cloud, both based on Wan2.1, with text prompts as actions that switch four times per rollout without being announced. This pair runs below real time in our implementation, so it tests the quality, not the runtime.

Quality and Drift are VLM judge scores, Cloud J per video second.
Scope

Limitations

  • One-step chunks are softer than four-step chunks, so the imaging quality of HIVE stays below that of the cloud model (65.3 against 66.4 MUSIQ).
  • Device regeneration needs a device that runs the small model at the playback rate, here a workstation GPU (RTX PRO 6000). An RTX A5000 does not, and leaves regeneration to the cloud.
  • The device model does not raise the judged quality in the reported comparison. It supports local regeneration after control changes and playback during injected network outages.
  • After an outage the device keeps generating until the cloud leads again, 16 of the 39 chunks of a clip on average, at the quality of the device model.
  • With an action change every 3 s the quality falls significantly below the four-step cloud model.
  • Actions are scripted, the visual scores come from automatic metrics and a VLM judge, and the cost is measured with both GPUs in one host, an injected network delay, and one cloud GPU per user. A human study, closed-loop play, and a wide-area deployment are future work.