Tines The single, secure environment for agents, apps, and automations.
WorldFoundry
World Model, Infrastructure, Inference, Eval
14 followers
World Model, Infrastructure, Inference, Eval
14 followers
WorldFoundry | A Unified Inference and Evaluation Framework for World Models WorldFoundry is an open-source infrastructure that unifies inference and evaluation for generative world models. It supports video generation, interactive worlds, 3D/4D representations, and embodied models through a unified workflow with TUI, CLI, and Studio interfaces. The framework integrates a growing collection of state-of-the-art models and currently includes 58 benchmarks.
🚀We are delighted that WorldFoundry has received 150 stars within its first day. Thank you to everyone for your support and feedback.
We will continue developing and maintaining WorldFoundry, and we plan to open source our world model benchmark, WorldAtlas, in the near future.
In the next phase, we will focus on compatibility with popular frameworks such as SGLang, performance optimization, adding more model and benchmark interfaces, and developing modules for world model data preparation, training, and fine tuning.
We warmly welcome the community to star🌟 the repository, submit pull requests, open issues, and contribute new models and benchmarks.🙏
How does it actually compare results across such different model types like video generation vs 3D vs embodied models, do the benchmark scores mean apples to apples or do you have to interpret them separately for each category?
@arassald67902 You are right to be cautious. Scores across video, 3D, 4D, and embodied models should not be interpreted as strictly apples to apples. They are most meaningful within the same task and model category, while comparisons across categories need to be interpreted separately.
We expect to release WorldFoundry’s companion project, WorldAtlas Arena, next week. It will include 10,000 image and video test samples, support models that take either images or videos as input, and cover nearly all major evaluation dimensions, including interactivity, consistency, memory, and generation quality.
For the benchmark itself, we focus on generative world models, primarily video, 3D, and 4D models, rather than embodied models. For embodied AI, we provide a convenient evaluation interface that can connect to other established benchmarks. Our broader goal is to offer unified infrastructure so the community can run inference and evaluation under consistent protocols.
How does this handle the compute requirements when running multiple benchmarks simultaneously, and is there any built-in support for distributing across GPUs or clusters without extra setup?
@zeynepkluu Yes. WorldFoundry has built in support for parallel benchmark execution and distributed inference. It can schedule workloads across multiple GPUs and scale them across machines or clusters, while handling task queues, resource allocation, and result aggregation.
Users can launch evaluations through a unified interface without writing their own GPU or cluster orchestration code. Some environment specific configuration, such as cluster credentials and model dependencies, may still be required.
Curious how the Studio interface differs from the CLI in practice, especially when working with longer video generation runs. Does it handle the same model configs or is it more of a lightweight preview tool?
@nazifemeam The Studio and CLI use the same underlying execution and evaluation stack. They support the same model configurations, prompts, sampling parameters, checkpoints, and benchmark definitions, so results can be reproduced across both interfaces.
Studio is not just a lightweight preview tool. It is designed to launch and monitor long running video generation jobs, inspect outputs, compare results, and manage experiments. The CLI is more convenient for scripting, batch evaluation, automation, and cluster workflows.
The main difference is the workflow. Studio is more interactive, while the CLI is better suited to large scale and repeatable runs.
Curious how it handles the jump from offline video generation to real-time interactive worlds in terms of latency—is there a built-in caching or streaming layer, or is that left up to each model integration to figure out on its own?
@kaankundurlu This is still a challenging area. The main limitation comes from the models themselves, since many current world models are designed for offline generation rather than real time interaction.
Caching and streaming can improve system level latency, but they cannot fully overcome the generation bottleneck. At present, the exact strategy still depends partly on each model integration, so there is not yet a universal caching and streaming layer that works equally well for every model.
We will continue optimizing the infrastructure and integration interfaces to make the transition toward interactive real time worlds more efficient.
Tried the TUI yesterday and was surprised how clean the workflow feels. Jumping between different world models without rewriting code is genuinely useful.
@ozkur99489 Thank you very much for your support. We will continue improving the TUI and the overall workflow. If you encounter any issues, please open an issue or join our Slack community through the GitHub repository. We will address your feedback as quickly as possible.
We would also really appreciate it if you could star the project on GitHub and recommend it to others.
The unification of TUI, CLI, and Studio under one workflow is a really thoughtful touch, makes it easy to move from quick experiments to deeper iteration without context switching.
@fatma147337 Thank you very much for your support. We will continue improving the TUI and the overall workflow. If you encounter any issues, please open an issue or join our Slack community through the GitHub repository. We will address your feedback as quickly as possible.
We would also really appreciate it if you could star the project on GitHub and recommend it to others.