--- title: "Open-Weight AI Models Are Getting Scarily Good — My Experience Building a RAG System with GLM-5.3-Flash" url: "https://jaainil.com/articles/open-weight-ai-models-are-getting-scarily-good-my-experience-building-a-rag-system-with-glm-53-flash" description: "Open-weight AI models are no longer just cheap alternatives to GPT and Claude. After building a real RAG system with GLM-5.3-Flash, I think we're witnessing a major shift in how AI applications will be built." --- AI30 Aug 2026 # Open-Weight AI Models Are Getting Scarily Good — My Experience Building a RAG System with GLM-5.3-Flash ![Open-Weight AI Models Are Getting Scarily Good — My Experience Building a RAG System with GLM-5.3-Flash](/_astro/open-weight-ai-models-are-no-longer-just-cheap-alt-mtflrd3e.DVsz2tgL_1TX41n.webp) ![Jainil Prajapati](/profile.png) By [Jainil Prajapati](/about) A few months ago, the conventional wisdom around AI was pretty simple. If you wanted the best intelligence, you paid OpenAI, Anthropic, or Google. If you wanted something cheap, private, customizable, or local, you used an open model and accepted that it probably wouldn’t be quite as good. That mental model is starting to break. Not completely. Frontier models are still frontier models. When the task is genuinely difficult, when reliability matters enormously, or when getting the best possible answer has significant economic value, Claude, GPT, and Gemini still justify their premium. But here’s the thing. **Most AI tasks are not that difficult.** And I recently experienced this personally while building a Retrieval-Augmented Generation system. I used **GLM-5.3-Flash**. And honestly? Oh my God. Not because it suddenly made Claude or ChatGPT obsolete. It didn’t. But because of how absurdly capable it was relative to what it costs. That distinction matters. We’re entering an era where the question is increasingly shifting from: > “Which model is the smartest?” to: > “Which model is smart enough to solve my problem reliably at the lowest total cost?” And I think open-weight models are going to dominate that second question. * * * # The AI Market Is Splitting Into Two Very Different Economies One of the most interesting AI charts I’ve seen recently came from Vercel’s AI Gateway data. It showed something that would have sounded surprising not very long ago: **Open-weight models have rapidly gained a huge share of token usage.** The research material behind this article captured a particularly dramatic August snapshot, where open-weight models surged from roughly 28.4% of Vercel AI Gateway token volume in late June to a 62% peak on August 22, before settling around 50.3% in the subsequent snapshot. But there’s an important caveat: this was data from one gateway, and a one-day peak should not be treated as stable global market share. Still, the broader trend is difficult to ignore. Developers are increasingly routing workloads toward models whose weights are available for download. DeepSeek. Qwen. Kimi. GLM. And a growing number of others. But here’s where the chart gets fascinating. **Token usage and money are telling completely different stories.** The same research showed DeepSeek handling roughly comparable token volume to Anthropic in that snapshot, while Anthropic captured vastly more modeled spending. Anthropic and OpenAI together represented a much smaller share of tokens than their share of modeled spend. That means we’re not watching one market. We’re watching two. ### Market One: Cheap intelligence at massive scale Open-weight and lower-cost models. Huge volume. Lower margins. Used everywhere. ### Market Two: Expensive intelligence for high-value problems Frontier models. Lower volume. Much higher willingness to pay. This distinction is crucial because it explains something many people misunderstand. **Open-weight models don’t necessarily need to beat Claude or GPT at everything to win.** They just need to be good enough for most things. And increasingly, they are. * * * # My Experience Using GLM-5.3-Flash for a Real RAG System I don’t want to make this article sound like another benchmark comparison. Because honestly, benchmarks are becoming exhausting. Model A gets 78.4. Model B gets 81.2. Model C gets 83.7. Great. But then you actually put them inside a real application and discover that the difference between those numbers doesn’t always translate into a meaningful product difference. I recently built a RAG system and used **GLM-5.3-Flash** as part of the stack. And my reaction was basically: > “Wait. This costs how much?” That’s when the current open-weight AI conversation really clicked for me. GLM-5.3-Flash isn’t necessarily the most intelligent model on the planet. I’m not going to pretend it is. Z.ai positions it as a highly efficient model designed to push the capability-per-cost frontier. The model has 320 billion total parameters with 18 billion active parameters and uses a hybrid architecture intended to reduce long-context serving costs. The company says it approaches much more expensive frontier models on several coding and agentic benchmarks. But here’s what matters more to me. **It was genuinely good in a real system.** Not “good for the price.” Not “surprisingly decent.” Actually good. That’s a much bigger deal. Because once a cheap model crosses the threshold from: > “This is acceptable” to: > “I would confidently use this in production” the economics of AI applications change dramatically. * * * # The “Smart Enough” Threshold Is More Important Than People Think Let’s imagine two models. ### Model A - 98/100 intelligence - Extremely reliable - Excellent reasoning - Excellent coding - Expensive ### Model B - 92/100 intelligence - Good reasoning - Good coding - Occasionally weaker - Costs a fraction of Model A Which one should you use? The answer is: **It depends entirely on the task.** If you’re using AI to analyze a billion-dollar acquisition, maybe pay for Model A. If you’re building a medical diagnostic system, you probably want the best possible model and multiple validation layers. If you’re running high-stakes legal analysis? Again, frontier quality may be worth every dollar. But what about: - RAG question answering - document summarization - classification - extraction - customer support - content transformation - internal search - metadata generation - routine coding tasks - agent sub-tasks - knowledge base assistants Do you really need the absolute smartest model in the world for every single request? Probably not. And that’s where open-weight models become incredibly interesting. The difference between 95% and 99% intelligence can be worth billions in some industries. But for millions of ordinary tasks? It might not matter at all. * * * # GLM-5.3-Flash Is a Perfect Example of the New AI Economics GLM-5.3-Flash is interesting because it represents something bigger than one model release. It represents a trend. Z.ai released the model after anonymously testing it under the name **ox-alpha** through platforms including OpenRouter and OpenCode. According to Z.ai, it quickly became one of the most popular models being tested, and the company later publicly identified it as GLM-5.3-Flash. The model is also natively multimodal and its weights are publicly available through Hugging Face. The published repository lists an MIT license. That combination is powerful: **Good capability + low cost + downloadable weights + deployment flexibility.** Historically, you often had to choose two. You could have: ### Cheap + private But weak. ### Powerful + private But extremely expensive to run. ### Cheap + powerful But only through someone else’s API. Increasingly, open-weight models are challenging that trade-off. * * * # Why RAG Systems Are Especially Interesting for Open-Weight Models This is where I think developers need to pay closer attention. A RAG system doesn’t ask the model to know everything. That’s the entire point. A typical RAG pipeline looks something like this: **User Query** ↓ Query understanding ↓ Retrieval ↓ Hybrid search ↓ Reranking ↓ Relevant documents ↓ Context construction ↓ Language model generates an answer ↓ Citations The model is not starting from zero. If your retrieval system is good, you’re handing the model the information it needs. That changes the intelligence requirement significantly. A weaker model with excellent retrieval can sometimes outperform a stronger model with terrible retrieval. Of course, that’s not universally true. The model still needs to: - understand the query - follow instructions - interpret retrieved context - reject irrelevant information - synthesize multiple documents - avoid hallucination - generate a coherent answer But this is exactly why I found GLM-5.3-Flash so interesting. For a well-designed RAG pipeline, you may not need to spend frontier-model money on every generation. Your system architecture can compensate for raw model intelligence. Better retrieval. Better reranking. Better chunking. Better context selection. Better citations. Caching. Intent classification. Query rewriting. Confidence thresholds. All of those things reduce how much “magic intelligence” you need from the final model. And that’s a huge opportunity. * * * # RAG Changes the Question From “Which Model Knows More?” to “Which Model Uses Context Better?” This is one of my strongest opinions about the future of AI applications. The raw model is becoming less important than people think. Not irrelevant. Important distinction. But increasingly, the value of an AI application will not come solely from: > “We use Model X.” Everyone can call the same API. That is not a moat. The value will come from everything surrounding the model: - proprietary data - retrieval quality - workflow design - evaluation systems - domain expertise - fine-tuning - memory - integrations - user experience This is why open-weight models are strategically interesting. If the base intelligence becomes commoditized, then the winner isn’t necessarily the company that owns the smartest foundation model. It might be the company that builds the best system around an available model. The research included with this article makes a similar argument through a three-layer AI market: commodity generalists handling large volumes, enterprise specialists built around proprietary context, and premium frontier generalists for the hardest tasks. And honestly? I think the middle layer might become the most interesting. * * * # Cost Per Token Is the Wrong Metric Developers love comparing prices like this: > Model A costs $X per million tokens. > Model B costs $Y per million tokens. But that’s incomplete. The metric that actually matters is: # Cost Per Successfully Completed Task Imagine Model A costs twice as much per token. Sounds expensive. But what if it solves the problem using 30% fewer tokens? What if it needs fewer retries? What if it makes fewer mistakes? What if it eliminates a human review step? Suddenly the expensive model might actually be cheaper. The opposite can also happen. A model may be incredibly cheap per token but require: - longer prompts - multiple retries - complex scaffolding - additional verification That can erase the apparent price advantage. This is why evaluating AI models purely by token price is becoming increasingly silly. You should benchmark: ### 1\. Task success rate Did it actually solve the problem? ### 2\. Average tokens consumed How much reasoning and output did it require? ### 3\. Retry rate How often did you need to run it again? ### 4\. Latency Did the cheaper model make your application painfully slow? ### 5\. Infrastructure cost Can you actually host it efficiently? ### 6\. Operational complexity How much engineering does it take to maintain? ### 7\. Total cost per successful outcome This is the number that matters. * * * # Open Weight Doesn’t Automatically Mean Open Source This is an important distinction because the AI industry is becoming sloppy with terminology. An **open-weight model** generally means the trained model weights are available for download and use. That can allow developers to: - self-host the model - run inference locally - fine-tune it - deploy it privately - switch inference providers But that doesn’t necessarily mean the entire training process is open. True open-source AI, under the Open Source Initiative’s definition, involves significantly more transparency around things such as training code and data information. Publishing weights alone does not automatically make an AI system fully open source. This matters because developers should understand what they’re actually getting. **Open weights give you deployment and modification freedom.** They do not necessarily give you complete reproducibility. For GLM-5.3-Flash specifically, the published model repository makes the weights available and lists an MIT license, which is unusually permissive from a practical deployment perspective. * * * # Ownership Might Be the Biggest Advantage Nobody Talks About Enough Let’s say you build your entire company on one closed AI API. Your architecture looks like this: **Your Product → Their API → Their Model → Their Pricing → Their Rules** That works beautifully. Until it doesn’t. What happens when: - prices increase? - rate limits change? - the model behavior changes? - a model gets deprecated? - your provider competes directly with your product? - a region loses access? - you need on-premise deployment? This is platform risk. Open-weight models introduce an alternative. You can host the model yourself. Or move between providers. Or fine-tune it. Or run it in your own infrastructure. Or deploy it inside a private network. That doesn’t mean self-hosting is always better. It absolutely isn’t. Running large models is difficult. You need: - GPUs - inference infrastructure - monitoring - scaling - batching - caching - quantization - reliability engineering For a small startup, calling an API can be dramatically simpler. But having the option changes your negotiating position. And options are valuable. * * * # The Emerging AI Market Has Three Layers I think the simplest way to understand the future is to stop imagining one giant AI market. There are probably going to be at least three. ## 1\. Commodity Intelligence This is where models like cheap open-weight generalists compete. Characteristics: - massive token volume - low cost - broadly capable - increasingly interchangeable - often self-hostable These models will likely power: - internal assistants - RAG systems - automation - summarization - extraction - customer support - low-risk agents The economics here will become brutal. Margins will shrink. Competition will explode. And that’s good for developers. * * * ## 2\. Specialized Intelligence This might become the most valuable category. Imagine taking an excellent open-weight foundation model and combining it with: - proprietary enterprise data - domain-specific fine-tuning - workflow knowledge - specialized evaluations - human feedback - private infrastructure Now you don’t have a generic chatbot. You have an AI system specifically optimized for something. Legal. Medicine. Finance. Engineering. Cybersecurity. Manufacturing. This is where the real moat might exist. Not in owning the base model. But in owning the **specialized intelligence built on top of it**. The uploaded research highlights examples of companies building specialized systems on open-weight foundations, including legal AI workflows. That’s a very different business model from simply renting a chatbot API. * * * ## 3\. Frontier Intelligence This category isn’t going away. Quite the opposite. The hardest tasks will continue to justify premium pricing. Examples: - extremely complex software engineering - frontier research - difficult scientific reasoning - high-value financial analysis - advanced autonomous agents - situations where one mistake costs millions In these situations, a few percentage points of intelligence difference can be enormously valuable. If the best model costs $20 more but saves a company $2 million? Nobody cares about the token price. That’s why I don’t think the future is: > Open models kill closed models. I think that’s simplistic. The future is more likely: > Cheap open-weight models eat enormous amounts of volume while frontier labs capture premium revenue for the hardest problems. And the data we’ve seen so far is surprisingly consistent with that possibility. * * * # But There Is a Catch: Most of the Momentum Is Coming From China This is the uncomfortable part of the conversation. Some of the most exciting open-weight models are increasingly coming from Chinese companies. DeepSeek. Qwen. Kimi. GLM. And others. That’s fantastic for developers in the short term. Competition is good. Cheap intelligence is good. More choices are good. But it creates a strategic question. What happens when entire Western companies begin building critical infrastructure on top of models developed elsewhere? The concern becomes bigger when models and hardware become increasingly co-designed. A model optimized around a specific hardware ecosystem can create dependencies further down the infrastructure stack. This doesn’t mean developers should avoid Chinese models. That would be ridiculous. Technology ecosystems are global. But companies should understand their dependencies. Where does the model come from? Where is it hosted? What license governs it? Can you self-host it? Can you migrate away? Are there geopolitical or regulatory risks? Can you reproduce your stack elsewhere? These questions are becoming part of AI architecture. * * * # Why I Think Developers Should Experiment With Open-Weight Models Right Now I don’t think every production system should immediately replace Claude or GPT. That would be terrible advice. Instead, I think developers should start benchmarking open-weight models against their actual workloads. Not generic benchmarks. Your workload. Take 100 real production tasks. Remove sensitive data. Then test: - Claude - GPT - Gemini - GLM - Qwen - DeepSeek - Kimi Measure: | Metric | Why It Matters | | --- | --- | | Accuracy | Did it solve the task correctly? | | Hallucination rate | Did it invent information? | | Latency | Is it usable interactively? | | Input cost | How expensive is context? | | Output cost | How expensive is generation? | | Retry rate | How often does it fail? | | Token usage | How efficiently does it reason? | | Cost per successful task | The metric that actually matters | You might be surprised. I certainly was. * * * # My Personal Take After Using GLM-5.3-Flash I don’t think GLM-5.3-Flash replaces frontier models. That’s not my point. Claude and GPT are still operating at an extraordinary level, particularly on difficult reasoning, complex coding, and high-stakes tasks. But I think something important has changed. **The gap is becoming less relevant for a huge number of practical applications.** That’s the part people should pay attention to. Five years ago, if you wanted good AI, you basically had to use whoever had the biggest model and the biggest data center. Today? You can take a model with publicly available weights, deploy it where you want, build an entire application around it, and get performance that would have seemed absurdly good not long ago. I built a RAG system with GLM-5.3-Flash. And my reaction wasn’t: > “Wow, this is almost as good as Claude.” My reaction was: > **“Why the hell are we paying frontier-model prices for every AI request?”** That’s a very different question. And I think more developers are going to start asking it. * * * # The Real Future Might Be Model Routing Here’s another prediction. The best AI architecture probably won’t use one model. It will use many. Something like: ### Simple task → Small cheap model ### RAG answer → Efficient open-weight model ### Complex reasoning → Stronger model ### Extremely difficult task → Frontier model ### Sensitive enterprise workflow → Self-hosted specialist The application decides dynamically. This is model routing. And it makes much more sense than sending every request to the most expensive model available. Think about cloud infrastructure. Nobody runs every workload on the largest possible server. You choose resources based on the job. AI will probably evolve the same way. * * * # My Prediction: Open-Weight Models Will Win Volume, Not Necessarily Revenue If I had to summarize my view in one sentence: > **Open-weight models will increasingly become the infrastructure layer of everyday AI, while frontier models remain the premium intelligence layer for tasks where being slightly smarter is worth paying dramatically more.** And between those two? Specialized AI systems. That’s where things get really interesting. Companies will take increasingly capable base models and combine them with: - proprietary knowledge - RAG - memory - tools - workflows - fine-tuning - evaluations The model itself will become less important than the system. And that’s why I think developers should care about open weights. Not because they’re free. Not because they’re Chinese. Not because they’re “open source.” Not because they’re going to kill OpenAI or Anthropic. But because they give developers something the AI industry desperately needs: **Choice.** And choice creates competition. Competition lowers prices. Lower prices create more applications. More applications create experimentation. And experimentation is how entirely new industries appear. * * * # Key Takeaways - Open-weight models are rapidly becoming competitive for everyday production workloads. - Token volume and AI revenue are diverging: cheaper models can dominate usage while frontier labs capture premium spending. - GLM-5.3-Flash demonstrates how dramatically the capability-per-cost ratio is improving. - For RAG systems, architecture quality can reduce dependence on raw frontier-model intelligence. - Cost per token is less useful than **cost per successfully completed task**. - Open weights provide deployment flexibility, customization, and reduced vendor lock-in. - Open-weight does not automatically mean fully open source. - The AI market is likely splitting into commodity generalists, specialized enterprise systems, and premium frontier models. - The best future AI stacks may route tasks dynamically between multiple models. - Developers should benchmark models against their own production workloads instead of blindly trusting leaderboards. * * * # Frequently Asked Questions ## Is GLM-5.3-Flash open source? GLM-5.3-Flash has publicly available model weights, and its Hugging Face repository lists an MIT license. However, it’s still useful to distinguish between an open-weight model and the broader definition of fully open-source AI, which can include training code and data transparency requirements. ## Is GLM-5.3-Flash better than Claude or ChatGPT? Not universally. Frontier models remain stronger choices for many difficult reasoning and high-stakes tasks. The interesting question is whether GLM-5.3-Flash is good enough for a particular workload at a dramatically better cost-performance ratio. ## Are open-weight models good for RAG? Yes, potentially very good. A RAG system supplies relevant information to the model, meaning the model doesn’t need to rely entirely on parametric knowledge. However, retrieval quality, reranking, context construction, and generation quality still matter enormously. ## Should I self-host an open-weight model? It depends. Self-hosting provides privacy and control but introduces infrastructure complexity. For smaller teams, using a managed inference provider may be cheaper and operationally simpler. ## Are open-weight models cheaper than frontier models? Often, but not always in total system cost. Developers should measure cost per completed task, including retries, token consumption, latency, infrastructure, and engineering overhead. ## Will open-weight models replace OpenAI and Anthropic? Probably not. The more likely scenario is market segmentation: open-weight models handle enormous volumes of lower-cost workloads while frontier labs retain premium use cases where maximum capability matters. * * * * * * # CTA The AI model you use shouldn’t be a religious decision. Don’t use Claude because Claude is popular. Don’t use GPT because everyone else uses GPT. And don’t use an open-weight model just because it’s cheap. Benchmark them. On your data. For your workload. Measure actual task success. Because the most interesting thing happening in AI right now isn’t necessarily that one model is becoming smarter than another. It’s that **very good intelligence is becoming dramatically cheaper and increasingly accessible to everyone.** And once intelligence becomes cheap enough? The real competition moves from building the model to building something useful with it. That is where things are about to get really interesting. ## Margin notes The thread lives on[GitHub Discussions](https://github.com/jaainil/jaainil-2026/discussions)— sign in there to join. ## More notes [ ![How I Built My Own RAG System Into My Portfolio Website](/_astro/cover-mtbbk4k7.DtjnB2R7_10lHpb.webp) Programming ### How I Built My Own RAG System Into My Portfolio Website ![Jainil Prajapati](/profile.png) 27 Aug 2026 ](/articles/how-i-built-my-own-rag-system-into-my-portfolio-website)[ ![India’s NavIC GPS Problem Explained: Why It Can’t Currently Navigate on Its Own](/_astro/india-s-navic-gps-problem-explained-why-it-can-t-c-mt1fwj2p.D6q3J9_n_Z6fuNQ.webp) Tech ### India’s NavIC GPS Problem Explained: Why It Can’t Currently Navigate on Its Own ![Jainil Prajapati](/profile.png) 20 Aug 2026 ](/articles/indias-navic-gps-problem-explained-why-it-cant-currently-navigate-on-its-own)[ ![Why Indian Roads Crumble Every Monsoon (The Real Reasons)](/_astro/gemini-generated-image-zg1v31zg1v31zg1v-watermark--ms66cjr2.N2tG4xib_Z1gz949.webp) political ### Why Indian Roads Crumble Every Monsoon (The Real Reasons) ![Jainil Prajapati](/profile.png) 29 Jul 2026 ](/articles/why-do-indian-bitumen-roads-break-so-quickly-the-engineering-truth)