Popular games have become fields for experiments with AI.
Recently, Chinese researchers published the results of experiments with artificial intelligence. Their project was based on the Qwen2 model and was named "Lumine":
Introducing Lumine — the first open-source recipe for developing generalist agents capable of performing multi-hour complex real-time missions in complex three-dimensional open-world environments. Lumine uses a human-like interaction paradigm that end-to-end integrates perception, reasoning, and action, based on a visual language model
They trained Lumine on a special dataset. Initially, there were more than 2400 hours of recordings, and after filtering and subsequent processing, 200 high-quality data and 15 hours of manually annotated reasoning data were obtained, which contribute to the development of the model's hybrid thinking.
The authors noted that Lumine successfully completes the five-hour Mondstadt campaign in Genshin Impact, and there is also success with other games:
Without any fine-tuning, it copes with [missions] in Wuthering Waves and the full five-hour first chapter of Honkai: Star Rail. These promising results highlight Lumine's effectiveness in various worlds and interaction dynamics, marking a serious step towards creating universal agents in open gaming environments.
