JARVIS-1: an open AI agent

JARVIS-1 is an open AI agent with multi-modal memory capabilities.

It can process visual information and written instructions simultaneously, and then make decisions and plans based on this information.

JARVIS-1 also has a memory function that allows it to remember previous experiences and learned knowledge.

In the open world game Minecraft, it performed almost perfectly in more than 200 different tasks.

Main features of the JARVIS-1:

1. Multi-modal input processing: JARVIS-1 can process multiple types of information simultaneously, including visual information (such as images in the game) and text information (such as player instructions or descriptions). This means it is able to see and understand verbal commands at the same time, just like humans.

2. Complex mission planning: In a game environment like “Minecraft”, JARVIS-1 can not only perform simple actions, but also plan and execute complex tasks. For example, it can mine specific resources or build structures based on the player’s instructions.

3. Memory enhancement model: JARVIS-1 has a special memory system that enables it to remember previous experiences and learned information. This memory function helps it adapt quickly and improve efficiency when faced with new tasks.

4. Efficient task completion rate: In “Minecraft”, JARVIS-1 has demonstrated efficient performance in more than 200 different tasks, which is perfect. On the long-term mission “Diamond Pickaxe Mission”, JARVIS-1 achieved a completion rate of 12.5%, which is a five-fold improvement over the previous record.

5. Self-improvement and lifelong learning: JARVIS-1 can continuously improve itself based on experience. This means that over time, it becomes more efficient and precise in performing tasks.

6. Adapt to different environments: JARVIS-1 can work in different environments of the “Minecraft” game, such as different terrains and biomes. This shows that it is able to adjust its behavior according to changes in its surrounding environment.

Projects and demos: https://t.co/9IZKrm09Gn

Paper: https://t.co/kX8ReYl1Wq

GitHub: https://t.co/AAw4p9xyEG

Many experimental demonstration results are given in the project:

1. Self-improvement: JARVIS-1 is able to self-improve through the life cycle learning paradigm, thanks to its growing multi-modal memory, inspiring broader intelligence and increased autonomy. It demonstrates improvements in its performance during different learning stages of the same task. For example, in the task of making scissors, JARVIS-1 lacked the furnace as a tool in the first cycle, but in the third cycle, it completed the task more accurately and efficiently.

2. Instruction execution in diverse biomes: JARVIS-1 is able to execute human instructions in different environments, such as performing tasks in different biomes such as plains, birch forests, jungles and savannahs.

3. Execution of specific tasks: JARVIS-1 demonstrated the ability to complete various tasks in Minecraft, such as making tools, equipment, decorations and food of different materials. For example, it can start by collecting raw materials, go through a series of steps, and finally make items such as diamond pickaxes, gold pickaxes, and red stone compasses.

These results demonstrate JARVIS-1’s adaptability and learning capabilities in a multi-task, open-world environment, as well as its efficiency and accuracy in handling complex tasks.

More AI News