- Prompt Warrior
- Posts
- I tested Gemini Ultra vs. ChatGPT and Claude
I tested Gemini Ultra vs. ChatGPT and Claude
The results might surprise you...
🚨 Quick Reminder: My ChatGPT Accelerator course is now available. If you want to:
Transform from a ChatGPT beginner to advanced user fast
Stay ahead in the fast evolving world of tech and AI
Learn how to use ChatGPT for writing, brainstorming and learning
Get the course here (offer ends this Sunday).
Hey Warrior,
Gemini Ultra, the new LLM by Google, was released a few days ago and everyone’s talking about how amazing it is.
So I decided to put it to the test…
Specifically, I wanted to test its writing abilities.
In a previous post I had done a similar test between ChatGPT and Claude and spoiler alert: Claude came out ahead in almost all categories.
This time I did a three-way head-to-head, pitting Gemini Ultra against ChatGPT and Claude by using 5 identical prompts (you’ll find all my prompts linked below as well).
Let’s jump in!
Read time: 6 minutes
Guidde is an AI-powered screen recording tool (think Loom on steroids).
You simply record your screen and then AI will automatically turn that recording into a FULL step-by-step tutorial!
I can then share this how to guide with anyone, which makes it really easy and fast to demo a workflow or show a process.
*sponsored content
I started with newsletter subject lines.
This is a prompt I use for every newsletter I write. When I’m done with the draft, I always generate subject line ideas using AI. This gives me inspiration and lets me piece together my favorite parts.
(See the full prompt I used here)
And here are the outputs of each model:
ChatGPT (GPT-4)
Claude 2
Gemini Ultra
Winner: Gemini
Why? ChatGPT's subject lines were over the top in my opinion. Claude's weren't concise enough. Gemini's were clear and compact. Therefore I’ll give this first round to Gemini.
2) Twitter Hooks
Twitter hooks are tricky, since AI is notoriously bad at this.
ChatGPT has always been slightly worse than Claude at this. Let’s see if Gemini is any better…
(Full prompt here).
ChatGPT (GPT-4)
Claude 2
Gemini Ultra
Winner: It's a draw
Why? I can't determine a clear winner here. None of them are great and stood out to me. I’ll still be writing Twitter hooks myself, and just using AI as inspiration.
3) Regular Tweets
AI is usually not great with regular tweets either. The best way to get a half-decent output is to include a specific template in the prompt.
(Here’s the simple prompt I used in this case.)
Here’s what the models came up with:
ChatGPT (GPT-4)
Claude 2
Gemini Ultra
Winner: Claude
Why? I have to give it to Claude here. ChatGPT was way too verbose (for a tweet). Gemini was good, but lost points for including the hashtags in the end. Claude did the best.
4/ Casual email
Default ChatGPT tends to sound too formal. Claude does a lot better in that aspect.
With this prompt I specifically instruct the AI to write in a casual tone. Lets see how well Gemini does here.
(Full prompt)
ChatGPT (GPT-4)
Claude 2
Gemini Ultra
Winner: Gemini
Why? ChatGPT and Claude's output both seem too long for a casual email pitch. Gemini was concise and straight to the point. I am not surprised that Gemini is good at emails, it probably has a lot of emails in its training data.
5/ Blog post
Finally, lets try out one of my favorite prompts: turning my newsletter drafts into (almost) done posts.
(Full prompt)
ChatGPT (GPT-4)
Claude 2
Gemini Ultra
Winner: Claude
Why? I like Claude's neutral and down-to-earth tone the most here. ChatGPT was a bit too much again. Gemini was ok, but the last sentence was too cringe. Ultimately I didn’t like any of them, and as you can tell from the intro of this email, I stuck with my original draft.
Wrapping up
Overall, ChatGPT struggled the most with these writing tasks. It's stronger at reasoning and brainstorming than writing. Nothing has really changed here since my comparison with Claude a few months ago.
Gemini Ultra impressed me, especially with its conciseness natural tone. I would say it’s almost on par with Claude now. I'll definitely be using it more and adding it to my workflow.
Of course, the release of GPT 4.5 may change everything soon. The AI landscape evolves quickly!
Thanks for reading!
If you enjoyed this email, consider forwarding this newsletter to a friend or colleague.
What did you think of today's email?Your feedback helps me create better emails for you! |
See you in the next one!
Reply