Mozilla: The state of open source AI

(stateofopensource.ai)

158 points | by rellem 2 hours ago

23 comments

  • babblingfish 2 hours ago
    Speculation: open models is what will kill Anthropic and OpenAI. Hyperscalers can run the models without a licensing fee. Apple can make them smaller and put them on the device.

    The frontier models are an edge and a liability. They're astronomically expensive to train. Without them, their models will fade into obscurity. Their marketing depends on people believing the models are meaningfully different, as people have sweatily argued on this forum. Personally, I'm not convinced there's much of a difference between these models at this point. The harness is what takes these random and hallucinogenic models and make them into something deterministic and useful.

    • kurthr 43 minutes ago
      I'm a bit skeptical of the token cost/ROI for all models, but sunk costs are sunk.

      It has the feel of self-improving super-intelligence or bust to me. If you get that, the frontier model(s) run away with a faster exponential. It's a bit like semi with Moore's Law with silicon, GaAs could never catch up. If you don't get it, the fast followers crush the high investment and there's no moat. Not like they can enforce copyright!

    • ComputerGuru 2 minutes ago
      > Apple can make them smaller and put them on the device

      Someone can, but Apple has essentially admitted defeat and handed the reigns over to Google.

    • mft_ 1 hour ago
      Open models are probably also comparatively astronomically expensive to train - just less so than the frontier models because they’re somewhat smaller, +/- the creators are more incentivised to focus on getting more from less compute because they’re have to, +/- they rely on distillation of the frontier models and this is more efficient.

      But efficiencies aside; creation of open models still requires a lot of money and compute from a large organisation which is willing to accept zero return for that spend. This largesse is unlikely to continue forever; so the question is which will crack first, the frontier models’ business model or the fast followers’ generosity?

      • demosthanos 57 minutes ago
        Yes, the problem with comparing open models to open source is that open source requires humans to volunteer their time. Open models requires humans to volunteer their money.

        These two types of contributions have very different behavioral profiles, and it doesn't obviously follow that the historical success of getting people to collaborate socially on building software for fun and for the benefit of the community will translate in any meaningful way to the necessity of being able to raise enormous amounts of money to pay for enormous amounts of electricity.

        • connicpu 3 minutes ago
          Technically open source requires some amount of monetary volunteering, it's just that the electricity to run a code editor and compile (most) open source code bases is within hobby budget for most people.
      • chaosharmonic 43 minutes ago
        I don't think it needs to be framed purely as generosity. You just need a sufficiently self-interested actor that sees open ecosystems as a necessary part of reducing their own risk profile, relative to the alternative of complete reliance of a third-party business that can take an exorbitant cut and/or Sherlock them at any time.

        Valve and SteamOS are a good example of what this idea looks like in practice. (Though they may also illustrate a third thing you need: a privately-run company, that has enough profit, and enough commitment from leadership to the company's vision, that they can make long-term bets without having to eventually bow to investors seeking short-term gains.)

      • afavour 1 hour ago
        I’m not exactly sure on the “how” but it only makes logical sense for (non-AI) companies to band together to fund the training of a shared model. Apple is a great example, AI is not their core business but they still require it.

        The only thing that took us down a different path is the vast sums of VC funding pumped into the AI companies.

        • inigyou 24 minutes ago
          If not for VC-funded LLMs there wouldn't be any LLMs.
          • LunaSea 18 minutes ago
            Most of the innovations needed for LLMs came from people at Google.
            • echelon 9 minutes ago
              > came from people at Google

              Who had to leave to build anything.

            • inigyou 10 minutes ago
              So VC-funded.
          • afavour 22 minutes ago
            [citation needed]

            Historically speaking a lot of inventions have come about without things like VC investment. Either way, there’s probably little point in debating it, just because VC funded companies control the market now doesn’t mean they should indefinitely.

      • sph 45 minutes ago
        How does it work if people flock to open models but they're too expensive to train? What is the financial incentive to do so?

        I seem to understand open models are mostly coming from China, and the benefit of training and releasing them for 'free' is a powerful geopolitical weapon against the Western/US economy that at this point depends on OpenAI & co. to succeed.

        Will the West make open models illegal?

        • echelon 30 minutes ago
          > Will the West make open models illegal?

          We better not.

          > What is the financial incentive to do so?

          If we'd been sharing all along (as we should have been), we probably would have gotten even further along in the development of the tech.

          Think of everything we could do if every researcher on the planet had first class access to the frontier. No academic fallback models. No crude API access. No limits, but direct access to the weights and the ability to lobotomize, splice, and dice.

          We could pour intelligence from one container to the next without paying a tax or wearing a blindfold. All without spilling a drop.

          *Open* *Must* *Win*

          • inigyou 22 minutes ago
            > No X. No Y. No Z, but Q
        • LorenDB 22 minutes ago
          "You wouldn't download an LLM"
          • sph 7 minutes ago
            You wouldn't crash the stock market by preferring Chinese models.
    • braiamp 5 minutes ago
      The real moat aren't the models, but the tooling around the models that allow them to perform specific tasks/goals. That's what really sets apart frontier vs open. Open only has the model itself, closed have the tooling to enhance the model.
    • goolz 34 minutes ago
      Completely agree. Once I can reliably get open models doing what I am on Fable ultra I imagine I will switch for good. I am fortunate to have access to a decent bit of local RAM, 192GB of DDR5 at an OK speed. It is not enough and costs are well past absurd. In a few years time I envisage a setup that is sub $10k which can accomplish such tasks. The pace so far has been breakneck. That is all I personally need. That may change, but until true AGI I do think there will be a ceiling to how much I will pay for something frontier if it is only marginally better.
      • wosk 29 minutes ago
        This is easier to say as Fable is good (even SOTA). But people have been were saying this continuously for the current model and for now the improvement are still coming.

        A better question is would you settle for o3 now or pay 20$ or 200$/month for fable ? Because o3 quality is available OSS.

        It is like the new IPhone, in some sort. At some point come a feature many would like to have, despite diminishing returns.

        We will see how long labs can keep up and what the scaling curve look like, but I would be more worried into losing sota status to Chinese companies than letting them take the open non-sota approach.

        • Vespasian 16 minutes ago
          I think there is also the case were companies will simply use different tiers for different tasks.

          While the engineering team might need a cutting edge model (with the associated costs), the marketing department will be fine by something that can grammar correct or turn a few bullet points into prose. Likewise you already don't need Fable for Ticket -> RAG -> Reply with Faq knowledge or escalate workflows

          That's already the case with other very expensive software like CAD packages were oftentimes you have different feature sets enabled for different employees.

    • elorant 25 minutes ago
      Eventually they will kill the hyperscalers too because of privacy issues. It's better for a company to pay an uprfont cost and then run everything on premise that uploading their entire codebase to a third party service.
      • xkcd-sucks 6 minutes ago
        Would that require a watershed event to clearly establish the importance/risk of privacy though? For example, right now it seems like most big software companies w/ strong security process are comfortable uploading entire codebases to Israeli cybersecurity firms for vulnerability scanning compliance purposes
    • paulddraper 10 minutes ago
      > Their marketing depends on people believing the models are meaningfully different, as people have sweatily argued on this forum.

      They are noticeably different. Benchmarks, anecdotes, all say the same thing.

      Now, is a ~6 month lead actually worth 1 gajillion dollars? Maybe not.

    • cma 1 hour ago
      You can run the same harness on fable, opus, sonnet, and see a huge difference between them. It is true the harness is important, and openai has begun encryption its instructions to swarmed sub-agents instead of just encrypting the chain of thought, but the model is still important at this stage.
      • ActionHank 1 hour ago
        This will only delay the inevitable. Sitting on some magic prompts is hardly the moat they need.
      • Alpha3031 1 hour ago
        Referent of "the models are meaningfully different" reads as <top closed, top open> rather than <top closed, cheaper closed> to me, so I'm not sure why we'd be comparing Fable vs Opus/Sonnet or Sol vs Terra rather than the same against Kimi K3.
        • Zababa 1 hour ago
          Haven't tried Kimi K3 for now but there was a huge difference between GPT 5.6/Fable and GLM 5.2/Kimi K2.7 that were previous frontier open models.
    • sublinear 24 minutes ago
      I still strongly believe Google Gemini has the best position for one simple reason: model maintenance. Accurate information is a moving target.

      Open models are indeed very capable, but they will eventually become more specialized to the application to keep an edge. It makes perfect sense that the future shape of AI conforms to the landscape it was born out of.

      • inigyou 19 minutes ago
        Grok has the biggest advantage in current events knowledge because it's integrated with X, which enough people still use even though it isn't Twitter.
        • lukan 0 minutes ago
          Hm, when compared to all the information people with android devices share with google, or those with gmail, ..
      • lgessler 19 minutes ago
        You're saying it's important to have up-to-date facts stored in parametric knowledge? It seems to me like that's grown less and less important as agentic capabilities have grown. Even if a frontier model doesn't know something, if it's out there, it can easily find it through tool use.
    • Mistletoe 54 minutes ago
      Open models are 4k TV (or maybe 1080p tv now and 4k TV soon) and SOTA frontier models are 8k TV. Can I or the average user tell the difference? Not really. Would they pay for that difference? Not a chance. Our entire economy is teetering on some future hope that this fragile and immaterial difference will pay off, when the reality is that LLMs are a race to the bottom and eventual razor thin margins. Maybe a tiny vocal subset of programmers can use it for work and make paying for it worth it to them, but that can't prop up an entire economy, especially when said programmers are phased out, jobless, and replaced by AI with each better iteration...
    • Zababa 1 hour ago
      The thing about not much difference between models and the harness making them deterministic and useful is wrong. Also models have different strengths and weaknesses and some are better at almost everything by a large margin compared to others.

      As for your speculation, I think it's hinging on some companies releasing models for free or no big differences between models. In a world with hyperscalers and companies training models you can quickly recreate Anthropic or OpenAI by having an hyperscaler ally with a model training company, train a good/a better model, and not release it.

    • fnord77 1 hour ago
      Just like opensource search engines killed google

      oh wait

      • blanched 1 hour ago
        I don’t even know the names of any open source search engines, but the open source models perform decently on various benchmarks and in personal experience.

        Was it ever even a claim that open source search engines were trying to outperform google, let alone kill it?

        • wongarsu 22 minutes ago
          Yacy tried in the 2000s. I'm sure some magazines made headlines posing the question whether yacy is a google-killer
      • inigyou 16 minutes ago
        Marginalia already gives me better results for many queries than Google. Because Google has sunk so low.
    • gallerdude 1 hour ago
      Even in the world where all models are basically equivalent (a thesis I don’t buy, but will grant you for arguments sake) - I believe there is much more to the AI business than just training and running models.

      It’s a very new set of technologies, and understanding what is useful to customers and what isn’t is the whole game. Call it, product taste. There were a million cell phones before the iPhone took over the world. Why iPhone? Product taste. There are a million startups, and only a select few become unicorns. Why? Product taste.

      • khurs 58 minutes ago
        >There were a million cell phones before the iPhone took over the world.

        You have tripped yourself up there.

        iPhone took over as it introduced something innovative over standard phones, but then Open Source (Android) matched the multi-touch and software differences and Apple's branding, lock-in and design etc have managed to keep it as a big player in wealthier countries. IPhone also came on the back of the massive iPod success.

        ChatGPT launched the same innovation vs Google Search, but just like Android Opensource AI is moving fast now.

        Android has 72.7% market share at present, Open Source AI will do the same unless the frontier labs can continue to do something new.

        The frontier labs are saddled with enormous investor and other debts. How long they can keep innovating by spending so much on R&D and paying there staff very high wages remains to be seen.

        Once investors cash out via an IPO, the companies are back down to earth and playing in the real world again.

        • inigyou 14 minutes ago
          There were many smartphones before both iOS and Android.
        • gallerdude 48 minutes ago
          Android has market share, but Apple makes all of the money! I find it really funny when people attribute Apple’s success to “oh, the only reason they succeed is design and marketing.” Yeah, I mean factually speaking design and marketing actually do matter a lot!

          Us developer types like to pretend like specs are the only thing that matters? If you could have a 10x more powerful model you could only access running locally through your terminal, versus a weaker model through a clean web interface, normies will pick the web ui every single time. Product experience is simply everything, as much as we like to pretend like nitty technical decisions are the most important thing.

          • afavour 19 minutes ago
            > Android has market share, but Apple makes all of the money!

            So? The benefit of open source is that you don’t have to worry about making a ton of money. You just need to be viable.

            Apple: premium product a minority is willing to pay for

            Android: standard product the majority use

            I’m sure there will continue to be iPhone equivalents in the AI world, premium bespoke models. But the vast majority of people will be happy with a cheaper offering.

  • GodelNumbering 1 hour ago
    Exactly 4 months ago, the marketshare on openrouter was 60%-40% in favor of closed models. Now it's 63%-37% in favor of open models. On March 19th, the open models processed 888B tokens in aggregate, yesterday, they processed 4.19T tokens in aggregate. That's almost 5x in 4 months! I can't think of the right intensifier to describe this level of growth.

    If you are looking for more details (as inferred by openrouter data), I built a dashboard that updates daily: https://dirac.run/labs-market-share

    • buddhistdude 38 minutes ago
      would love to see a statistic by model, and maybe some sort of classification to get a sense of how good the model is and how much it costs.

      edit: this exists https://artificialanalysis.ai/

    • ofjcihen 53 minutes ago
      This right here is going to be considered one of the first major signs of the downfall of closed models years from now.

      And look, if you disagree with me PLEASE tell me why. What moat do these companies have? I genuinely want to know because looking at the spend for companies like OAI and Anthropic with no actual moat I can identify is actually driving me insane.

      • athorax 49 minutes ago
        The moat is enterprise contracts and artificial friction moving between harnesses. Moat is a strong word, more of a puddle.
  • dosinga 1 hour ago
    > Mozilla exists because one company tried to own the front door to the web, and an open community rose up to make sure it never could.

    I'd say that the front door to the web is pretty much owned by Google and Apple at this point given Firefox current marketshare. And maybe that's enough, maybe a future where a low percentage of open models keep the rest of the system honest but that doesn't seem the argument of this article

    • wongarsu 12 minutes ago
      It's also just a very cherry-picked framing

      Mozilla exists because one company owned the front door to the web, and another company abused their market position to push their free (as in beer) browser. Mozilla is the phoenix rising from the ashes of that first company.

      Then another company came along, abused their market position to push their free browser, demolished Firefox's market share but keep handing them cash to avoid the appearance of a monopoly

    • bel8 30 minutes ago
      Mozilla exists because Google gives them billions to keep Google as default search engine.
      • ifdefdebug 12 minutes ago
        And to keep around a concurrent browser with full adblock abilities so they can cripple their own without too much of an outcry.
  • Catloafdev 37 minutes ago
    As someone that's generally for the proliferation of open models, I want to take this seriously, but it's really difficult when it was clearly written by AI.

    I guess they fired whoever used to write copy for these things.

    Edit: to be clear, I'm not trying to just dunk on them, I think it's actively hurting their own point to do this, and counter-productive when people can easily clock it - it makes some percent of the audience immediately tune out.

    • s3p 9 minutes ago
      [delayed]
  • fan__glm52 1 hour ago
  • mrcwinn 0 minutes ago
    Seems quite odd to use OpenRouter as “proof” that open weights models won. If you’re using OpenRouter, you’re already looking to bypass frontier models. To suggest there’s no longer a tradeoff simply isn’t true. But this isn’t the first time I thought Mozilla was a less than trustworthy source of information.
  • amanharshx 1 hour ago
    The design and layout made it harder to read than it needed to be.

    Regardless, the inference costs dropping almost 50× is really amazing to see. And now Kimi K3 release has shown how open models are getting closer to the frontier level already. Open source AI is moving a lot faster than Anthropic and OpenAI would have expected lol.

  • osigurdson 14 minutes ago
    I don't love the appeal to romanticism portrayed in this article.
  • draxil 37 minutes ago
    Almost all about open weight, but the title says Open Source.
  • bityard 1 hour ago
    It sure is nice to see that Mozilla is still doing all that they can to keep on top of current trends, except developing a decent privacy-focused web browser for developers and power users.
    • mtone 37 minutes ago
      Firefox's AI Chatbot feature only proposes one open model provider (Mistral) and zero local option. They're not walking the talk.
    • longitudinal93 38 minutes ago
      But they supply all the source for those projects to flourish and have an ecosystem of their own: Librefox, Iceweasel, Reynard, etc, etc.
      • doodlesdev 14 minutes ago
        That's a very low bar, though. Chromium is open-source too and has a bunch of privacy-focused forks such as Ungoogled Chromium, Brave and Chromite.
  • gyulai 54 minutes ago
    Title: The state of open source AI.

    First sentence: In New Zealand's far north, a Māori broadcaster...

    ...oh boy, that's all you need to read to know what kind of media diet the writer is on.

    • jborichevskiy 47 minutes ago
      Normally I'd agree, but the rest of the sentence does actually discuss open models and their use cases.
  • hughw 51 minutes ago
    Is the CTO a bot?
  • marcuskaz 2 hours ago
    It appears open models were used to create this slop.

    That opening is so hard to understand what they are trying to say, from the font and how it's written. It took me several times rereading to even grasp.

    Plus the article is filled with cryptic things like:

        Open ships easy.
        Open deploys hard.
    
    What?! Is it a meta answer to "the state of open source AI" question?
    • azangru 2 hours ago
      From the title of a chart:

      > The venture-funded open-source ecosystem: total disclosed funding, USD M

      > Bars grow as you scroll.

      The bars, in fact, don't grow as you scroll. And I don't even see why they should.

      • gen2brain 1 hour ago
        On my device, bars grow as I scroll. I want your feature, being able to just scroll the static page without elements jumping around.
      • yjftsjthsd-h 1 hour ago
        > The bars, in fact, don't grow as you scroll. And I don't even see why they should.

        On my device, they grow as I scroll to them.

    • garretraziel 2 hours ago
      I think it’s supposed to mean “open source is easily shipped, but open source is hard to deploy”? Or perhaps “deploys hard” is a figure of speach, as in “we are deploying this open source and we are deploying it /hard/“? I don’t know, it’s not good.
    • sippeangelo 48 minutes ago
      This is truly some proper slop. The "PRODUCTION RATE BY COMPANY SIZE" graph has bars that start offset from the text underneath them, which LOOKS like a mistake that happened due to word wrap, but if you visibly compare the 54% to the 55% bars they seem to have compensated for this?! I can't tell if his was on purpose or accident and it's impossible to take the data seriously!

      This is on mobile in portrait. In landscape the text doesn't wrap or offset anything.

    • hughw 53 minutes ago
      So good at style, so weak on substance
  • latexr 1 hour ago
    Quick fix for the font, which many people are (rightly) complaining about.

      Array.from(document.getElementsByClassName("quote")).forEach(p => { p.style.marginTop = "20px"; p.classList.remove("quote", "reveal") })
    
    The issue is that all of the text is a quote, and that renders enormous. That’s probably fine for a tiny quote amongst more text, but here it is jarring.
  • Cuuugi 2 hours ago
    Maybe its the wildfire smoke in my eyes, but that font choice feels aggressive.
    • input_sh 1 hour ago
      It's their own fonts: Mozilla Headline and Mozilla Text.

      No idea why they'd be using the display font for the abstract though, that kind of defeats the whole purpose. It's supposed to be quirky and bold, but used far more sparsingly.

    • aprilthird2021 1 hour ago
      It's AI slop
  • hypfer 2 hours ago
    This new trend of content appearing while scrolling down is so terrible accessibility-wise, I do not understand how Mozilla of all institutions would do it.

    Not every trend needs to be followed. Have some backbone. You receive donations to have that.

    ___

    Apart from the website being - frankly - bullshit, the content is also - frankly - bullshit.

    It's just on the frontpage because the title says "open source AI".

    • azangru 1 hour ago
      > This new trend of content appearing while scrolling down is so terrible accessibility-wise, I do not understand how Mozilla of all institutions would do it.

      Could you explain what is wrong with the accessibility of this page? All the content is included in the html payload, so it is accessible to screen readers and text-based browsers; and as for the "reveal" effect, it seems to respect user's choice of "prefers reduced motion" and is disabled when that is user's preference.

      • hypfer 1 hour ago
        > it seems to respect user's choice of "prefers reduced motion".

        Cool, that I didn't check, because it is impossible to enable that setting, as it breaks _huge_ amounts of websites.

        I'm not aware of a way to enable it selectively, but one could also just display the content at all times. It's a static page. It's static content. None of this makes any sense.

        ___

        The idea behind that style of gradual reveal is probably some kind of storytelling, but the only story it tells is that mozilla is wasting donations on people with incorrect opinions that could be used on.. idk not building torment nexii?

  • jdw64 2 hours ago
    The UI is really hard on the eyes. Personally, I think the font size is way too big, and the animation timing feels off. If this is a benchmark page and not a product page, I feel like the information should be scannable at a glance. The UX is bad.
    • 100percentjake 1 hour ago
      I'm unsure what it is about AI developers seemingly not having eyeballs. The Hermes Agent website is absolutely eye-searing and the application itself resembles some sort of weird "RETVRN" greek-styled travel agent website.

      https://hermes-agent.nousresearch.com/

      • Jtarii 1 hour ago
        Really wish websites weren't allowed to force smooth scroll on. Hijacking basic browser functionality is so hostile.
      • fwip 1 hour ago
        Oh that's easy: they outsource design to the LLM, which doesn't have eyeballs.
      • jdw64 1 hour ago
        I agree 1000%, Mr. Jake.
      • urbsgpw 1 hour ago
        I use hermes only ever saw their repo. Atrocious. I was sure you were exaggerating.
    • farmerbb 1 hour ago
      Feels like a mobile website that was never optimized for desktop usage.
  • positron26 1 hour ago
    Just like how the web was won?

    I think Mozilla is chasing a past formula, but the projection isn't linear enough to remain consistent, and the critical parts of the outcome, utter centralization of the market dominance of the three C's, are left out of the equation.

    We might get the consolation prize, a few nerds having competitive alternatives to applaud, but we will be left with the hidden costs: stagnation by bloated market leaders, consumers and businesses pouring trillions of dollars into the commercial offerings while open development wonders where money comes from, and the leakage of these imbalances into political and social spheres.

    If we follow a Mozilla template and get to the peak of Mozilla's success at the web, look at what that really is. Facebook, Amazon, Google etc are orthogonal to that equation.

  • progx 1 hour ago
    "Open won"... to be fair cause "google paid it".
    • jdw64 1 hour ago
      I think the fact that Mozilla survival model ultimately depends on Google's money means Google is keeping a corpse propped up just to have a defense argument that a browser competitor still exists, so they don't get hit with monopoly regulations.
      • victor9000 1 hour ago
        Particularly in this day in age when the FF market share is down to low single digits
  • inigyou 41 minutes ago
    There isn't any open-source AI. There is Open AI (not to be confused with the closed company called OpenAI, which was unable to trademark its name). There's no open source AI both because the open source community doesn't have the resources to train a useful AI and because AI doesn't have source code.
    • ses1984 40 minutes ago
      Is training code and dataset not source?
      • inigyou 25 minutes ago
        Are they open?
    • Catloafdev 39 minutes ago
      This is just wrong on multiple levels, the open source model ecosystem is very much a thing.
  • urbsgpw 1 hour ago
    Haven't been following the articles and snippets we get from these labs about training their models for a while. But I'm guessing the latest chinese models are way less based on distilling? If not, then your speed of progress is still limited by the two labs (which we are collectively, in various forms subsidizing).
  • brunooliv 1 hour ago
    This is really insane to me.

    There's nothing practical about open-source models yet that makes them even remotely comparable to closed frontier models.

    All the hype around GLM, Qwen, now Kimi.... Are people really this naive that they believe these reports or, more worringly, are people NOT using these models and seeing the HUGE gap that still exists?

    Take a task, any medium-sized task, decently scoped that you'd trust to give to Sonnet to finish without a hitch. Now give it to ANY open-source frontier model and watch them struggle and go in circles while failing tool calls and randomly assuming things.

    Open-source is and has been amazing but its so hard to deploy reliably and at scale and there's still big problems in the underlying models with instruction following and tool calling that makes it basically unusable for production workloads at a decent price point...

    • foolswisdom 1 hour ago
      > Take a task, any medium-sized task, decently scoped that you'd trust to give to Sonnet to finish without a hitch. Now give it to ANY open-source frontier model and watch them struggle and go in circles while failing tool calls and randomly assuming things.

      Claude used to be much worse than it is now, just as bad the open weights models are. And the open weights were worse. The labs will also try to keep the lead, but at some point people start seeing real value from open models. Maybe you say they're not ready yet for medium tasks, but everyone sees the writing on the wall.

      • brunooliv 1 hour ago
        I hope you're right and I want you to be right, but, even seeing the current hype around local models, etc... and open-source models, I think the industry is currently under a big confusion where they see the benchmarks of things like Kimi, GLM, Qwen, they play with it via opencode, and they think like: "Wow this is pretty good, I want to deploy this". But they don't understand how the KV cache grows over time and can take almost as much memory as needed for a 30B param model, they dont understand that a quantized model WILL NOT be the same as a full precision one, and they surely don't see the engineering work needed to serve inference to even tens of customers at a decent quality and latency level.

        The biggest moat of these giant labs and models is increasingly shifting towards deployment capabilities and (debatably) having better (proprietary) harnesses.

        The models themselves can be impressive on benchmarks, but unless they can be served reliably to customers either at scale, hosted somewhere, or even on edge with predictable latency and memory usage, then frontier will always be leading.

    • seany 1 hour ago
      If your doing things the closed models won't let you do; its the whole ball game.