MeteoVerse

Weather-Controllable Video World Model

MeteoVerse

Renlong Wu1, Guanqiao Wang1, Xuan Shang1, Yin Hanming1, Xiaoxiao Sheng2, Tianyu Huang2, Hui Li1, Wangmeng Zuo1

1Harbin Institute of Technology    2Huawei

Predict camera-controlled future video from a single sunny or adverse-weather image while explicitly controlling weather preservation, introduction, and removal.

Paper PDF Video Results BibTeX

Overview

Video world models aim to predict future content from an observed scene while following prescribed camera motion. Real-world scene evolution is determined not only by changes in viewpoint and object dynamics, but also by environmental conditions such as weather, which can substantially alter scene appearance and visibility. MeteoVerse explicitly represents the required weather transition between the observed and target weather states and uses a transition-aware mixture of weather experts to realize controllable weather evolution.

Explicit weather transitionContinuous observed and target weather states are estimated over rain, snow, and fog, and their difference represents the required weather change.
Transition-aware MeteoMoECategory-specific weather experts convert the transition into residual weather features while keeping scene and camera conditioning separate.
50K+ weather clipsThe dataset contains real-world rain, snow, and fog videos with generated sunny counterparts, weather annotations, and camera trajectories.
MeteoVerse overview showing weather preservation, introduction, and removal
MeteoVerse supports weather preservation, introduction, and removal from sunny or adverse-weather inputs under prescribed camera trajectories.

Method

A target-weather instruction specifies the desired condition, but the required modification also depends on the weather already present in the observation. MeteoVerse makes this transition explicit and injects only the required weather modification into the pretrained video world model.

MeteoVerse method overview
The weather-state predictor estimates observed and target states. MeteoMoE modulates rain, snow, and fog experts using the transition components and injects the fused residual features into the DiT backbone.
MeteoVerse dataset construction pipeline
Dataset construction with real-world adverse-weather clips, generated sunny counterparts, disentangled descriptions, weather-intensity annotations, and camera trajectories.

Supplementary video results

Weather Control Comparisons

10 samples · 4 weather settings · 6 methods

Input image
Sample 1
Scene description

Target weather

Fine-grained weather introduction

Intensity Control

Starting from a sunny input, only the requested target weather intensity is varied while the scene description and camera trajectory remain fixed. Each row shows the same scene and camera motion across five target intensities, making the progression easy to compare.

Citation

The arXiv identifier can be added after the preprint is posted.

@article{wu2026meteoverse,
  title   = {MeteoVerse: Unified Weather-Controllable Video World Model},
  author  = {Wu, Renlong and Wang, Guanqiao and Shang, Xuan and Yin, Hanming and Sheng, Xiaoxiao and Huang, Tianyu and Li, Hui and Zuo, Wangmeng},
  journal = {arXiv preprint},
  year    = {2026}
}