We are proud to be part of 🐫 CAMEL-AI.org, the pioneer of large language model-based multi-agent frameworks. CAMEL-AI.org is inspired by the research paper "CAMEL: Communicative Agents for 'Mind' Exploration of Large Language Model Society," and aims to explore scalable techniques for autonomous cooperation among communicative agents and their cognitive processes.
This project aligns with CAMEL's mission by:
- Implementing a novel multi-agent system in the StarCraft II environment
- Exploring agent cooperation and decision-making through multimodal interactions
- Contributing to the advancement of practical AI applications
This project develops a multimodal agent for StarCraft II that processes both visual information and text descriptions to make strategic decisions. The agent leverages vision-language models and reinforcement learning techniques to understand and interact with the game environment.
- 🎮 Custom environment based on OpenAI Gym
- 🖼️ Support for multimodal inputs (images and text)
- 🤖 Automatic unit annotation system
- 🎯 Multiple agent types (Random, VLM, Test agents)
- 🗺️ Rich map selection for different scenarios
- 🛠️ Support for both ability and non-ability based gameplay
- 🤝 Support for both single-player and two-player modes
- Python 3.10
- Windows 11 (primary support)
- StarCraft II (Asia Battle.net version)
- Clone the repository:
git clone [repository-url]
cd [repository-name]- Install in editable mode:
pip install -e . --no-deps- Install dependencies:
pip install -r vlm_attention/requirements.txt- Fix potential protobuf conflicts:
pip uninstall protobuf
pip install protobuf==3.20.0
pip install pysc2 --no-deps- Single Player (No Abilities):
python vlm_attention/run_env/multiprocess_run_env.py --agent VLMAgentWithoutMove --map vlm_attention_1- Two Players (No Abilities):
python vlm_attention/run_env/multiprocess_run_env_two_players.py --agent1 TestAgent --agent2 VLMAgent- Single Player (With Abilities):
python vlm_attention/run_env/multiprocess_run_env_with_ability.py --agent VLMAgentWithAbility- Two Players (With Abilities):
python vlm_attention/run_env/multi_process_run_env_two_players_with_ability.py --agent1 TestAgent --agent2 VLMAgent- Basic maps (vlm_attention_1, 2m_vs_1z, etc.)
- Complex scenarios (2c_vs_64zg, 8marine_2tank_vs_zerglings_banelings, etc.)
- Mirror matches (2s3z, 3m)
- Balanced matchups (MMM_vlm_attention_two_players)
- Asymmetric battles (vlm_attention_1_two_players)
- Mirror matches (vlm_attention_2_terran_vs_terran_two_players)
vlm_attention/
├── env/ # Environment implementation
├── run_env/ # Running scripts
├── knowledge_data/ # RAG and knowledge base
├── utils/ # Utility functions
└── requirements.txt # Dependencies
Key configuration files:
vlm_attention/env/config.py: Environment settingsvlm_attention/knowledge_data/database/config.yaml: Database configurationvlm_attention/utils/call_vlm.py: VLM model settings
- Added new maps including 2cvs64zg vlm_attention version
- Updated model calling interface in vlm_attention/utils/call_vlm.py
- Enhanced support for multiple units of same type operations
- Implemented RGB image acquisition from game engine
- Added unit.tag retrieval functionality
- Improved multi-unit control support
Special thanks to @LLM-PySC2 for providing valuable resources and references for our environment design.
For detailed documentation:
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Copyright 2024 CAMEL-AI.org
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.