this isn’t just a modeling problem. it’s also a benchmarking problem.

spurious correlations are always a pain, but in multimodal llms they become a particularly tough battle. On one hand, you want to leverage the language prior to enable better generalization; on the other, that same language prior can turn into a shortcut that makes the model effectively blind.

the irony is that humans do the same thing. We still gravitate toward language-first tasks, and the “multimodal results” in major model releases like gpt-5 reflect exactly that bias.

I mean, economically this makes most sense for LLM companies: you can claim wins in “multimodal reasoning” without investing heavily in real multimodal research.

that shortcut will come due tho. when you try to put these systems into glasses, robots, or anything else that touches the real world, the cracks will show. and they’ll be costly.