GPT-5 is here. I tested it for a week. Here's what surprised me.

I've been paying for ChatGPT Pro since day one. $20/month. My credit card statement reminds me every month. I justify it because it helps me debug code and write first drafts. But honestly? GPT-4 was starting to feel... stale.

Then GPT-5 dropped. I got access last week. I've been testing it every night after work (sorry, Oluwa, for the late emails). Here's what I actually think, not what the press release says.

The "million token" thing is real — and weird

GPT-5 can process one million tokens in one go. That's like reading all three Lord of the Rings books at once. Or our entire company Slack history from the past year.

I tested it by pasting our entire codebase into a single conversation. Not all at once, but chunk by chunk. Then I asked it: "Find all the security vulnerabilities in our authentication flow."

It found three things I missed. One of them was actually serious. I fixed it at 11 PM that night. Oluwa thanked me the next morning. Worth the $20 right there.

Where it still fails

• Humor: Still bad. I asked for a joke about startup life. It gave me something about "agile sprints and standups." Not funny, OpenAI.
• Math: Better than GPT-4. Not as good as a calculator. Still makes basic arithmetic errors.
• Confidence: Still too confident when it's wrong. I'd rather it say "I'm not sure" than invent an answer.

For me? I'm keeping my subscription. GPT-5 is genuinely better. But it's not magic. It still needs human oversight.