- The Rundown AI
- Posts
- Google's new AI plays video games with you
Google's new AI plays video games with you
PLUS: New robot gives ChatGPT a body!
Rowan Cheung
March 14, 2024
Welcome, AI enthusiasts.
The future of gaming is looking more intelligent than ever, thanks to Google DeepMind’s latest creation, SIMA.
The AI agent isn’t just an impressive player — its a window into AI that can intelligently collaborate with humans for truly interactive experiences. Let’s explore…
In today’s AI rundown:
DeepMind trains AI gaming agent
New Figure robot gives ChatGPT a body
Create animations with Claude 3
AI gets a peripheral vision check-up
6 new AI tools & 4 new AI jobs
More AI & tech news
Read time: 4 minutes
LATEST DEVELOPMENTS
Image source: Google DeepMind
The Rundown: Google DeepMind just introduced SIMA, an AI agent that can follow natural language instructions to perform tasks across diverse video game environments.
The details:
SIMA was trained on 9 different video games, including titles like No Man's Sky and Teardown, to learn general skills and commands.
The AI agent doesn't require access to game code or APIs, only images and text instructions as input — mimicking how humans play.
SIMA was evaluated on ~1,500 tasks across categories like navigation, object interaction, and resource gathering.
Why it matters: SIMA offers an exciting glimpse into the future of gaming, where AI agents could become dynamic sidekicks rather than just powerful opponents. It also brings us closer to AI that can intelligently collaborate with humans not just in games, but in real-world tasks and environments too.
TOGETHER WITH IMAGINE AI LIVE
The Rundown: Time is running out to grab tickets for the Imagine AI Live conference at the Fontainebleau Las Vegas on March 27-28.
Meet leaders of the generative AI revolution like:
Mark Heaps, Chief Technology Evangelist at Groq Inc.
Bindu Reddy, CEO of Abacus AI
Div Garg, Founder of MultiOn
Learn how to future-proof your business from the AI leaders firsthand. Secure your spot now before it's too late.
FIGURE
Image source: Figure
The Rundown: Figure AI just unveiled a new demo, integrating with an OpenAI vision-language model to allow it to engage in natural conversations, visually interpret its surroundings, and autonomously execute tasks.
The details:
The demo showcases Figure 01 speaking conversationally, sorting dishes, and carrying out context-dependent actions.
All of the robot's actions are learned behaviors rather than remote-controlled and are executed at normal speed.
Figure 01 also demonstrates its ability to plan and reflect, explaining its reasoning for a certain action while remembering past context.
The demo comes just 13 days into Figure’s partnership with OpenAI, announced alongside the startup’s $675M funding round.
Why it matters: If this is the progress just 13 days into the Figure + OpenAI partnership, we’re in for a wild ride. Figure 01 shows off cutting-edge dexterity and movement, now combined with the advanced capabilities of a powerful multimodal AI.
Watch the demo here — then imagine what the GPT-5 integration will bring…
AI TRAINING
The Rundown: In this tutorial, you will learn to create animations in seconds using Claude, allowing you to explore and understand topics with visuals.
Step-by-step:
Go to Claude’s website.
Enter the following prompt: “Write manim code to animate an explanation of the [what you want visually explained]. Think step by step before coding it, and provide me with the full code”
Claude will provide you with Python code.
Lastly, follow the instructions here to obtain your animated video.
Note: If the result isn’t perfect, just ask Claude to help you out further. You can also provide it with images to describe what you want to accomplish.
AI RESEARCH
Image source: Midjourney
The Rundown: MIT researchers just created a new image dataset called COCO-Periph — allowing AI models to process visual information more like humans do.
The details:
COCO-Periph modifies images to capture the loss of detail that occurs at different degrees of visual angles.
The team tested both human subjects and AI object detection models on the images to compare peripheral vision.
AI performance dropped off as the target object moved further out of focus, while humans saw a more gradual decline.
Training on COCO-Periph closed the gap with human-level performance.
Why it matters: Research like COCO-Periph can help align tech with how we humans naturally take in and process visual information—an important skill for systems such as driver safety features that rely on anticipating what people will perceive.
NEW TOOLS & JOBS
🚀 Mixo - use AI to generate a website in seconds and bring your business concept to life*
🎮 Chaotix AI- Transform ideas into playable 2D games
🦸🏻♀️ Hero- Identifies and prices items to help you sell stuff faster
🤖 Zep- Recall, understand, and extract data from chat history
🗓️ Dola- Calendar assistant that converts voice, text, and images into organized plans
🎙️ Voiceflow- AI agent-building platform for chat and voice assistants
*Sponsored listing
QUICK HITS
Don't miss Section's free, 1-hour lecture on The Executive's Guide to Generative AI, coming up on March 19. Join to master your "AI talk track" and understand the implications for your business, workforce, and initiatives.
OpenAI CTO Mira Murati revealed in an interview with the WSJ that the viral Sora video model will be publicly released later this year, with the team currently focused on red-teaming and optimizing costs for generations.
Oracle CEO Larry Ellison said on the company’s earnings call that he and Elon Musk are collaborating to develop an AI-powered application for farms, helping to plan, predict and increase agricultural output.
GemMoE was released in beta via a Hugging Face grant, a new model based on DeepMind's Gemma architecture and open-sourced for easy training and inference.
A robotic system that enables safe and successful feeding for people with severe mobility limitations was nominated for best paper at this year’s Human-Robot Interaction conference.
LlamaIndex launched LlamaParse, a GenAI-native document parsing platform featuring LLM-powered instructions, JSON mode, image extraction, and expanded document type support.
OpenAI signed new licensing deals with European publishers Le Monde and Prisa to bring French and Spanish language news content to ChatGPT.
*Sponsored listing
THAT’S A WRAP
SPONSOR US
Get your product in front of over 500k+ AI enthusiasts
Our newsletter is read by thousands of tech professionals, investors, engineers, managers, and business owners around the world. Get in touch today.
FEEDBACK
How would you rate today's newsletter?Vote below to help us improve the newsletter for you. |
If you have specific feedback or anything interesting you’d like to share, please let us know by replying to this email.
Join the conversation