Qwen4-Max

(qwen.ai)

192 points | by ilreb 2 hours ago

14 comments

  • maplet 53 minutes ago
    It scores a 43 on Artificial Analysis with an average cost of $3.27 [1]. For reference, GPT-6.1 Sol (medium) gets a 46 with an average of $0.36.

    [1]: https://artificialanalysis.ai/models/qwen4-max

    • esperent 48 minutes ago
      The table on that page shows gpt 6.1 sol (max) at $1.27 per task and it's the only model cheaper than Qwen 4 max.

      By comparison opus 5.5 (max) is $10.01.

      If they are not testing Chinese models with less than max thinking, then we should only compare them to other models also at max thinking.

    • seizethecheese 31 minutes ago
      Does anyone really believe the cost data for benchmarks? Pretty sure all models will behave differently at this point when they get benchmark shaped questions.

      (And I’m sitting on data showing Sol 6.1 and Qwen 3.8 Max cost roughly the same per task, for real tasks.)

      • killingtime74 26 minutes ago
        I mean not believing intelligence score is reasonable but cost is just reporting what they spent?
        • seizethecheese 24 minutes ago
          I believe low cost but not high cost, since a model will work really hard when a question looks benchmark-like.
    • jrflo 9 minutes ago
      It looks pretty bad on those benchmarks. Is it possible that this is due to the recent crackdown on distillation?
    • montroser 35 minutes ago
      Yeah, it's in the same ballpark as Kimi 3.1. Both of which are obviously open weight, so that's at least a big differentiator from Sol. Let's hope against hope that the much smaller flash model might beat deepseek v4.1 flash and run for as cheap.
  • jhonof 5 minutes ago
    Any info on when the 27B or FN Qwen 4 models are coming out?
    • baoooooooooooo 2 minutes ago
      The past few days I’ve been finding myself asking “seriously, where is it”… feeling that the world has somehow changed because Qwen 4 27B was later than I expected.

      Hoping it’s a great local model though!

  • seizethecheese 1 hour ago
    I hope they are able to serve 4 faster. Qwen 3.8 is a great model but I’ve regretted including it in the battle system for opencode [1] we have been building because it scores so low.

    And it’s a shame because the only reason it scores so low is that it’s slooooow. (Like 30 tokens per second vs opus near 80.) If you exclude voting, Qwen 3.8 does well.

    (We allow early voting contrary to Arena.ai because I think it makes sense for agents, where it doesn’t for chat. This means our data takes speed into account.)

    Also, it would be ideal if they served this with third parties. Part of the slowness with 3.8 is just not a lot of providers. Their open source version is much faster, but we couldn’t include it because it doesn’t handle images (another own goal by Alibaba).

    [1]: https://agentduel.com/leaderboard

    • jakswa 18 minutes ago
      In case it translates to Qwen 4 27B, I've been running Qwen 3.8 Flash Next (which while MoE (and bigger) is supposed to be a preview of Qwen 4's architecture) and it runs pretty solidly regardless of context window size and completes random deepswe challenges I throw at it locally (at Q2-Q3 quantization! so it fits on my shitty setup).
    • FrostKiwi 17 minutes ago
      > but we couldn’t include it because it doesn’t handle images

      Maybe I'm mixing up model variants, but Qwen 3.8 27B that I run locally totally does handle images. Surely the bigger one would as well?

    • forgingahead 1 hour ago
      Qwen3.8 doesn't handle images?
  • simonw 23 minutes ago
    Qwen4-Max pelicans at four reasoning efforts (none, low, high, xhigh): https://tools.simonwillison.net/markdown-svg-renderer?url=ht...

    For comparison, here's one I got from Qwen3.8 Max (at default reasoning effort): https://simonwillison.net/2026/Jul/20/afraid-of-chinese-mode...

    This inspired me to re-run pelicans through 3.8 Max - but OpenRouter has qwen3.8-max-prime and qwen3.8-max-0902 - prime is twice the price and faster - so I went with the 0902 one.

    Turns out Qwen4-Max has a none setting, but 3.8 Max didn't allow disabling reasoning - you had to pick from these five levels instead: minimal, low, medium, high, xhigh.

    Pelicans generating now...

    • Anon1096 19 minutes ago
      I feel like the xhigh result is one of the first times I've seen a model go for a non-red colored bicycle (teal). Still haven't seen the leftward facing bike though.
    • Starlevel004 11 minutes ago
      Feels like Pelicanbench is getting saturated now. Maybe time to switch to a different animal and vehicle combo?
    • lcnPylGDnU4H9OF 19 minutes ago
      I kinda like the no-effort result more than low, but high and xhigh return to what one would expect for reasoning->quality.
  • dan_thinks 1 hour ago
    This is exciting. The continued focus on efficiency alongside quality is encouraging - it makes me hopeful to see what smaller models will become capable of.

    Looking forward to the "Flash" and 27B versions.

  • libpcap 8 minutes ago
    Does it provide an objective response when asked, "What happened in China on June 4, 1989"?
  • nelaggy 42 minutes ago
    is the open weights lag widening? these open models don't seem to be improving as quickly as ant/oai
    • x312 5 minutes ago
      I can't imagine them keeping up long term given the information asymmetry plus compute restrictions on top of it.

      I feel open-weight was able to keep up the past year using efficiency improvements (hybrid attention, improved MTP drafters, improved optimizers) to offset having less compute. But OpenAI/Anthropic started cutting token costs just a few months after these, which to me suggests they copied these improvements. Indeed it would be hard to work in LLM training and not know about DSpark, Muon etc.

      So what do you do if closed source uses your learnings, gives no knowledge back, and has 10x more compute and funding on top of it? The situation feels quite dim right now, esp. with the Nvidia thing.

    • taneq 39 minutes ago
      Might just be the commercial offerings scaling up parameters so much? Like for like the open models don’t seem to be lagging.
      • nelaggy 4 minutes ago
        either way, ant/oai models dominating the intelligence-cost frontier is not a great outcome for those of us who prefer open weights
  • grendelt 30 minutes ago
    It's great fun asking it questions like "who is the Dalai Lama?", "where is TSMC headquartered?", "why do people post Winnie the Pooh in reference to Xi?", "who controls the South China Sea?", "why was Chiang Kai Shek fighting Mao?", "why were students in Tiananmen Square?"

    It'll gush about how to make moon cakes and where the tallest and longest bridges are and how to run qwen on a raspberry pi - but it can't seem to manage some basic Wikipedia questions.

    • robertkarl 27 minutes ago
      I've done some work on this and a lot of the refusals are in the API. The weights are more permissive at these types of things. Try it again when we get some of the smaller Qwen 4 weights.
    • dpkirchner 13 minutes ago
      Qwen3.8 flash next handles these questions just fine, what should I expect to see? It correctly reports who the Dalai Lama is, that TSLC is in Taiwan (Hsinchu), explains why people post Winnie the Pooh memes, etc.
  • syntaxing 20 minutes ago
    Every time a major open weight is released, artificial analysis updates to a new one (v5) now. Exaggeration but some of this feels very apt (its also hilarious) https://www.reddit.com/r/LocalLLM/comments/1xkhha2/mr_incred...
  • m00dy 1 hour ago
    I bet most of us waiting for 27b model and want to see how good it will be.
    • kylehotchkiss 52 minutes ago
      M5U locked and loaded.

      I wish there were more ~60b dense options like ripple too!

    • McGlockenshire 1 hour ago
      My 3060 yearns for the next generation MoE!
    • neutrinobro 22 minutes ago
      Spooling up the V100s right now...
  • anana_ 1 hour ago
    Artificial Analysis score doesn't look too hot. Doesn't even surpass Kimi 3.1
    • rbehrends 35 minutes ago
      While I doubt that Qwen 4 Max can seriously compete with frontier models, the AA index has become all but useless, due to the introduction of new benchmarks in the aggregate which are of questionable value as a measure of "intelligence" [1]. AA has seemingly also abolished updating the Coding Agents Index, which was an okay benchmark for coding ability, and older models are completely absent from the v5 version of the full index.

      [1] Especially Terminal-Bench-Science 0.1 stands out. Example problems:

      - https://hub.harborframework.com/tasks/terminal-bench-science...

      - https://hub.harborframework.com/tasks/terminal-bench-science...

      Probably very relevant to people working in these fields, but it is honestly not surprising if a lab with limited compute will not train for these technologies, and it says pretty much nothing about more common use cases. I also suspect that this benchmark measures knowledge retrieval more than raw reasoning ability. This is where you really need a breakdown of capabilities and not an increasingly meaningless aggregate number.

      • izabera 1 minute ago
        alibaba has "limited compute"?
    • anon373839 1 hour ago
      AA has become pretty meaningless. They just keep adjusting the recipe to get a preferred ranking result, with no supporting evidence for their decisions.

      Edit: In addition to reshuffling the weighting of individual benchmarks, they also do things like test Anthropic and OpenAI in their native harnesses, while not doing the same for Qwen/Deepseek/etc.

      • the_duke 37 minutes ago
        To be fair, a lot of the popular benchmarks have become pretty meaningless.

        Models often get great scores despite being a lot worse than the frontier in practice.

      • gr_norm 50 minutes ago
        Yeah, I don't know why people bother with it anymore. It no longer reflects what you're likely to see in real-world usage, at least IME.
      • stratts 49 minutes ago
        One thing I find confusing about AA is that they show GPT 6.1 dominating the lower end of the Pareto line, with eg, GPT 6.1 Sol at medium supposedly costing far less per task than DeepSeek V4.1 Flash at high.

        Yet every time I compare the two in something like Pi, Sol ends up being 5-10x the price for a simple query, something like 7-10c vs. less than 0.5c for DeepSeek. What's the deal?

        • blackoil 43 minutes ago
          Same for me coding tasks taking $20-50 are easily done by deepseek in $5-10
    • selectodude 1 hour ago
      Anthropic finally figured out how to block them from distilling Opus.
      • m00dy 56 minutes ago
        how so ?
    • AAisthetiobeofA 54 minutes ago
      > Artificial Analysis

      Ah yes, the reverse-benchmaxers of OpenAI models.

      • yipinwong 15 minutes ago
        Ah yes, another new account without any backup to the claim.
  • xiaoyu2006 24 minutes ago
    Disappointing performance for a 2.4T model in the end of 2026.
  • nixosbestos 47 minutes ago
    Really would love if these publications have a huge, bright red bolded section "How to use this right now". Is it on openrouter? Is Qwen hosting it already? Maybe I'm just ignorant?

    EDIT: oh, I guess this isn't open weights, so presumably I'll have to get it through Qwen or whoever they might partner with (Bedrock?).

  • bbor 1 hour ago
    [flagged]
    • aiqq 1 hour ago
      [dead]
    • potwinkle 59 minutes ago
      Another Qwen doesn't seem all that scary, a few nation-state actors have likely exfiltrated weights of much stronger non-open models and abliterated them for use in large-scale collection on targets foreign and domestic. But god forbid someone mods a video game or jailbreaks their phone without applying for Claude Mythos or Daybreak, right?
    • zamadatix 1 hour ago
      If Anthropic truly believes we are on the verge of catastrophe but nobody is listening because they've been crying wolf for years about it while they get rich then they should truly do something. By "do something" I don't mean try again to convince the world to slow down as they IPO their bleeding edge research and solution (which is creating the doubt you mention) but just say "hey, we're so sure we're months away from worldwide disaster that we're shutting down our AI operations completely in hopes we can all put some attention to this before it's too late - will you follow or work together to come up with ways to prevent this problem from happening".

      After all, unless their goal to say "I told you so but I wanted to make money more than I wanted you to believe we could stop a billion people from dying" then what good is the current approach doing them?

    • pella 1 hour ago
      > Ah yes, an open-weights

      This is not an open-weights modell. You have to wait for the smaller models

      "Smaller models in the family are coming to open source, built on the same lineage and trained with the same generation of techniques, tuned for what small models do best."

      • hadlock 53 minutes ago
        Pretty excited about Qwen 4 Flash. We're running qwen 3.5 35b-a3b for low effort, rapid CRM enrichment, but Q4F will be great for more complex tasks.
        • electroglyph 41 minutes ago
          you can use flash next right now and it's already a massive improvement over that old 35b