Parameter Update: 2025-16

While most labs are still not on full release mode this week, we've at least got loads of smaller announcements that, altogether, make for one of the more fun weeks in recent memory - and at least one policy development work discussing at length.

OpenAI: GPT-4o Rollback

After I complained about how bad GPT-4o felt just last week, this week was really all about the reckoning for the increasing sycophancy. After backlash kept on increasing, this week OpenAI rolled back the changes and released not one but two postmortems (the initial one immediately upon rolling back, followed by a more detailed write-up a few days later).

While I am very glad the rollback happened (and we got the write-ups we got), I can help but feel there are a few concerning factors at play here that I am not convinced anyone has fully put together until this happened. While OpenAI has talked for a while about how tuning a models personality is surprisingly hard, it seems that, for the most part, their solution consisted of, in some form, RLHFing based on user preference - which in turn, rewarded toxic sycophancy.

The fact that we are only finding out about this because people noticed something changed because Altman told us they changed something for once, then, feels entirely insufficient.

On the one hand, I am concerned that all of this is happening while both the original GPT-4 (-turbo) and GPT-4.5 are being retired - since we will lose the ability to research their behavior in the future and both of them are, in my experience, much more useful "personality models" than GPT-4o.

On the other hand, the idea that we've also just gotten confirmation that modern models now possess a superhuman ability to convince people (through an unauthorized study run on /r/changemyview), really makes it seem inevitable that we'll be dealing with the problem space this opens up for a very long time.

At the very least, it seems that OpenAI has figured this out now, seizing the moment to launch an ✨improved shopping experience✨ in ChatGPT.

Google: Gemini Image Generation Updates

Given the previous topic covered most of my timeline for the better part of last week, I would assume this one has slipped under most people's radar: After being first to launch native/autoregressive image generation, Google has made theirs actually good now?

It's still not as aesthetically pleasing as GPT-4o (meaning: it mostly sucks for creating art), but it is a lot better for editing existing pictures:

Anthropic: Claude.ai Updates

It would seem that Anthropic has finally woken up and realized that MCPs might actually be a winner for them. It might have taken them embarrassingly long, but Claude.ai now actually supports the damned things!

They also launched a better Deep Research tool that leverages these tools as part of its search process, which seems like a rather obvious thing in hindsight (which doesn't make it less cool).

Now, this doesn't fix their rate limits, but probably makes it among the easier/more useful places to get started stacking a bunch of MCPs!

Alibaba Qwen 3

I haven't gotten too deep of a look at them yet, but from initial perusal, it seems that Alibaba's new Qwen3 series consists of extremely capable reasoning models that lack too much world knowledge to be useful in most contexts. Also, the fact they haven't released the 235B or 32B base models yet seems like a weird omission? On the other hand, promoting improved MCP support seems like a nice touch, and I am really looking forward to seeing if I can get the 0.6B model to do anything useful. Also really like the "235B-A22B" notation for differentiating total and active parameters in MoEs.

Microsoft: Phi-4

The second model release this week I haven't spent too much time on, Phi-4 is (to my knowledge) the first time we've seen reasoning capabilities on such a small model released by Microsoft.ph

After Phi-3 was mostly good at benchmarkmaxxing and very little else, it would appear that the new models may actually be useful in some real-world contexts for once?

Meta: LlamaCon

While I had hoped for Meta would be able to turn things around in time for their first ever LlamaCon after the Llama4 launch, these announcements don't make me feel too optimistic about the future of Meta's LLM strategy:

• Llama API: Meta will finally start serving their own model themselves. If that's how they plan on monetizing the thing, they should work on Now having a model worth serving first.
• Meta AI App: After shoving Llama down our throats by integrating into all Meta apps, we're now getting a dedicated app. Who asked for this? I am also not terribly excited about the idea that all our AI apps might start having social feeds soon?!
• New safety & moderation tools (Llama Guard 4,...): The one announcement that felt really understated and useful - current moderation pipelines often have terribly high failure rates (especially in the false-positive direction), so it's nice to see this!

At least, Facebook still managed to make one of my favorite pieces of content this week by having Zuckerberg go on Theo Von's podcast (which seems like a legitimately terrible idea?)