Gemini gets schooled by... GPT 3.5?

PLUS: 'Hallucinate' named Word of the Year

Welcome, AI enthusiasts.

Google's prized AI asset Gemini just got a lump of coal in a new third-party study — achieving subpar results versus free OpenAI models like GPT-3.5.

With expectations sky-high, the tech giant looks to have some serious work to do to deliver on its massively hyped promises. Let’s investigate…

In today’s AI rundown:

  • Google’s Gemini lags behind GPT 3.5 Turbo

  • ‘Hallucinate’ named Word of the Year

  • 9 new AI tools to try

  • How to generate full music tracks with Copilot

  • LongAnimateDiff lengthens AI video outputs

  • Users trick ChatGPT in car dealership negotiations

Read time: 3 minutes

LATEST DEVELOPMENTS

GOOGLE

Image source: Gemini

The Rundown: Fresh third-party testing just found that Google's new (and polarizing) Gemini Pro LLM lags slightly behind free OpenAI models like GPT-3.5 on most benchmarks, despite the hype around its launch.

The details:

  • On questions across academics, reasoning, math, and other tasks, Gemini solved slightly fewer problems than GPT-3.5.

  • Flaws seemingly responsible included struggling on longer queries, aggressive content blocking, and bias towards end choices.

  • One area where Gemini excelled was in non-English language generations — but content filtering resulted in responses being blocked in 10 language pairs.

  • A Google representative disputed the paper’s claims, pointing to the company’s own research and calling attention to Gemini Ultra’s release in early 2024.

Our thoughts: Is Gemini getting coal in its stocking this holiday season? With all eyes on Google’s flagship AI launch, the numbers in this study certainly seem to show that the tech giant is lagging behind. If Gemini Ultra doesn’t show significant upgrades early next year, things could get ugly for Google.

TOGETHER WITH YOU.AI

The Rundown: You know your business needs AI. You might already have some ideas of where AI could increase performance - or, you’ve got a list of tedious work that might be improved with AI but haven’t seen a tool that can do exactly what you need.

Using MindStudio, you can build custom, no-code AIs to solve any problem with:

  • Personalized, flexible workflows

  • Branching Automations

  • Multiple AI models (GPT, Claude, Llama, Mistral, and more)

  • Custom data sources

  • Integrations with current tools (Mailchimp, Slack, Salesforce)

Learn today with MindStudio’s video tutorials, and have a new AI within 24 hours. Start learning now!

AI TERMINOLOGY

Image source: Dictionary

The Rundown: Dictionary.com selected "Hallucinate" as its 2023 Word of the Year, pointing to its emergent AI context meaning "to produce false information and present it as fact." 

The details:

  • Lexicographers noted lookups jumped 46% this year as articles mentioned "hallucinate" 85% more in reference to AI outputs.

  • The site added the novel definition in 2022, with experts drawing comparisons to past tech metaphorical extensions like "virus."

  • Other AI words like LLM, chatbot, generative AI, and GPT also saw upticks in searches — with AI-related words increasing by an average of 62% on the platform.

Our thoughts: While AI enthusiasts probably laugh, AI hallucinations are an important thing for the broader world to understand. The blind trust new users may put into the tech (especially young students) is a massive risk as AI continues to become more mainstream.

QUESTION OF THE DAY

Do you think Gemini Ultra surpass GPT-4 in capabilities when fully released in 2024?

Vote below (click for live results)

Login or Subscribe to participate in polls.

TRENDING TOOLS

🤖 GPT-Engineer App- Making practical, easy-to-deploy web apps in minutes (link)

🪄 InMagic- Uncover your personality based on your Instagram profile (link)

🏟️ Chatbot Arena- Compare and rank the output of 25+ LLMs (link)

🎨 LogoFast- Create stunning logos quickly & effortlessly with AI for free (link)

🤖 Knowlee AI- Integrate AI agents with various apps (link)

📘 Linnk.AI- Enhance your reading experience with AI-powered insights and efficiency (link)

🎨 Shader- Elevate your personal expression through AI (link)

❄️ Snowball- Simplify your content creation with AI assistance (link)

👩🏽‍🍳 ChefGPT - AI digital chef & kitchen companion (link)

Browse the most popular tools ever featured with our tool database.

AI TRAINING

Ready to tap into your musical talent? Thanks to a new integration with Suno, Microsoft Copilot can now create full AI-generated tracks from simple text prompts —  complete with lyrics, instruments, and vocals.

Create one yourself below:

Enable Suno: Navigate to Microsoft Copilot → Plugins tab. Make sure ‘Search’ is enabled, and then activate the new Suno integration.

Start Prompting: Type the details of the song you’d like to generate into the Copilot chat. Don’t overthink it — simple prompts work well. The prompt used for our example: “Make a groovy rap jazz song about AI advancing society”.

Get Your Song: After about a minute, Copilot will provide an external link to listen to your song. Re-prompt and tweak as needed!

Check out the results of our example song (it has a bit of a Kayne West vibe to it 🤣)

QUICK HITS

Join Inflection AI co-founder Mustafa Suleyman and NYU Stern professor Scott Galloway for a discussion on whether humanity can contain the next wave of AI. Sign up for the event today!

Lightricks just introduced LongAnimateDiff, extending its AnimateDiff video diffusion model to handle 16-64 frames. Also launching a 32-frame specialist variant, the labs' updated model expands AI-generated video clips for longer and more flexible generations.

After discovering a car dealership’s chatbot was powered by ChatGPT, hilarity ensued — with users deploying prompt injections and various tricks to persuade the bot to offer cars for $1, throw in vacation packages, and complete other unrelated tasks.

Browse all available AI job positions with our job board.

SPONSOR US

🦾 Get your product in front of over 400k+ AI enthusiasts

Our newsletter is read by thousands of tech professionals, investors, engineers, managers, and business owners around the world. Get in touch today.

THAT’S A WRAP

That’s it for today’s news in the world of AI!

If you have anything interesting to share, please reach out to us by sending us a DM on Twitter: @rowancheung & @therundownai

How was today's newsletter?

If it wasn't a 10/10, let us know how we can do better!

Login or Subscribe to participate in polls.

❤️ Rundown review of the day

Reply

or to participate.