The Rundown AI
Posts
Gemini gets schooled by... GPT 3.5?

Gemini gets schooled by... GPT 3.5?

PLUS: 'Hallucinate' named Word of the Year

Rowan Cheung
December 21, 2023

Sign Up | Advertise | Jobs | Tools

Welcome, AI enthusiasts.

Google's prized AI asset Gemini just got a lump of coal in a new third-party study — achieving subpar results versus free OpenAI models like GPT-3.5.

With expectations sky-high, the tech giant looks to have some serious work to do to deliver on its massively hyped promises. Let’s investigate…

In today’s AI rundown:

Google’s Gemini lags behind GPT 3.5 Turbo
‘Hallucinate’ named Word of the Year
9 new AI tools to try
How to generate full music tracks with Copilot
LongAnimateDiff lengthens AI video outputs
Users trick ChatGPT in car dealership negotiations

Read time: 3 minutes

LATEST DEVELOPMENTS

GOOGLE

🐢 Google’s Gemini lags behind GPT 3.5 Turbo

Image source: Gemini

The Rundown: Fresh third-party testing just found that Google's new (and polarizing) Gemini Pro LLM lags slightly behind free OpenAI models like GPT-3.5 on most benchmarks, despite the hype around its launch.

The details:

On questions across academics, reasoning, math, and other tasks, Gemini solved slightly fewer problems than GPT-3.5.
Flaws seemingly responsible included struggling on longer queries, aggressive content blocking, and bias towards end choices.
One area where Gemini excelled was in non-English language generations — but content filtering resulted in responses being blocked in 10 language pairs.
A Google representative disputed the paper’s claims, pointing to the company’s own research and calling attention to Gemini Ultra’s release in early 2024.

Our thoughts: Is Gemini getting coal in its stocking this holiday season? With all eyes on Google’s flagship AI launch, the numbers in this study certainly seem to show that the tech giant is lagging behind. If Gemini Ultra doesn’t show significant upgrades early next year, things could get ugly for Google.

TOGETHER WITH YOU.AI

🧠 Bring custom AIs to your team in 2024

The Rundown: You know your business needs AI. You might already have some ideas of where AI could increase performance - or, you’ve got a list of tedious work that might be improved with AI but haven’t seen a tool that can do exactly what you need.

Using MindStudio, you can build custom, no-code AIs to solve any problem with:

Personalized, flexible workflows
Branching Automations
Multiple AI models (GPT, Claude, Llama, Mistral, and more)
Custom data sources
Integrations with current tools (Mailchimp, Slack, Salesforce)

Learn today with MindStudio’s video tutorials, and have a new AI within 24 hours. Start learning now!

AI TERMINOLOGY

💬 ‘Hallucinate’ named Word of the Year

Image source: Dictionary

The Rundown: Dictionary.com selected "Hallucinate" as its 2023 Word of the Year, pointing to its emergent AI context meaning "to produce false information and present it as fact."

The details:

Lexicographers noted lookups jumped 46% this year as articles mentioned "hallucinate" 85% more in reference to AI outputs.
The site added the novel definition in 2022, with experts drawing comparisons to past tech metaphorical extensions like "virus."
Other AI words like LLM, chatbot, generative AI, and GPT also saw upticks in searches — with AI-related words increasing by an average of 62% on the platform.

Our thoughts: While AI enthusiasts probably laugh, AI hallucinations are an important thing for the broader world to understand. The blind trust new users may put into the tech (especially young students) is a massive risk as AI continues to become more mainstream.

QUESTION OF THE DAY

Do you think Gemini Ultra surpass GPT-4 in capabilities when fully released in 2024?

Vote below (click for live results)

TRENDING TOOLS

🤖 GPT-Engineer App- Making practical, easy-to-deploy web apps in minutes (link)

🪄 InMagic- Uncover your personality based on your Instagram profile (link)

🏟️ Chatbot Arena- Compare and rank the output of 25+ LLMs (link)

🎨 LogoFast- Create stunning logos quickly & effortlessly with AI for free (link)

🤖 Knowlee AI- Integrate AI agents with various apps (link)

📘 Linnk.AI- Enhance your reading experience with AI-powered insights and efficiency (link)

🎨 Shader- Elevate your personal expression through AI (link)

❄️ Snowball- Simplify your content creation with AI assistance (link)

👩🏽‍🍳 ChefGPT - AI digital chef & kitchen companion (link)

Browse the most popular tools ever featured with our tool database.

AI TRAINING

🎼 How to generate full music tracks with Copilot

Ready to tap into your musical talent? Thanks to a new integration with Suno, Microsoft Copilot can now create full AI-generated tracks from simple text prompts — complete with lyrics, instruments, and vocals.

Create one yourself below:

Enable Suno: Navigate to Microsoft Copilot → Plugins tab. Make sure ‘Search’ is enabled, and then activate the new Suno integration.

Start Prompting: Type the details of the song you’d like to generate into the Copilot chat. Don’t overthink it — simple prompts work well. The prompt used for our example: “Make a groovy rap jazz song about AI advancing society”.

Get Your Song: After about a minute, Copilot will provide an external link to listen to your song. Re-prompt and tweak as needed!

Check out the results of our example song (it has a bit of a Kayne West vibe to it 🤣)

QUICK HITS

🤖 RSVP for free event: Can AI Be Contained? (sponsor)

Join Inflection AI co-founder Mustafa Suleyman and NYU Stern professor Scott Galloway for a discussion on whether humanity can contain the next wave of AI. Sign up for the event today!

🎥 LongAnimateDiff lengthens AI video outputs

Lightricks just introduced LongAnimateDiff, extending its AnimateDiff video diffusion model to handle 16-64 frames. Also launching a 32-frame specialist variant, the labs' updated model expands AI-generated video clips for longer and more flexible generations.

🤝 Users trick ChatGPT in car dealership negotiations

After discovering a car dealership’s chatbot was powered by ChatGPT, hilarity ensued — with users deploying prompt injections and various tricks to persuade the bot to offer cars for $1, throw in vacation packages, and complete other unrelated tasks.