开源模型正在重塑企业 AI 的成本与基础设施

Brief Description

Y Combinator interviews Ollama co-founder and CEO Jeffrey Morgan about why open models are rapidly becoming central to enterprise AI. They discuss the cost and control advantages of open models, the growth of coding agents, the practical split between local, cloud, open, and frontier models, the security and operational challenges of adoption, and how Ollama aims to make a fragmented model ecosystem usable for developers.

Table of Contents

• The shift to open models
• Coding agents and exploding token demand
• Model releases, fine-tuning, and safety
• Launching models at production scale
• The emerging agent stack
• Open and frontier models in the enterprise
• Local execution and hybrid routing
• China, the United States, and open-model competition
• Ollama's business model and YC experience
• Building for an AI-native software world

The shift to open models

Host: Ollama makes it easy to run open-source AI models locally and in the cloud. With millions of developers using it and broad enterprise adoption, the company has a close view of what people actually download and continue to use. What is changing in the state of the art?

Jeffrey Morgan: The largest shift is toward open models, particularly in enterprise. Usage includes models originating in the United States and China, and is dominated by coding agents and AI assistants such as OpenClaw and Hermes. Ollama began as a way to run models on a MacBook and other hardware, including Nvidia, AMD, and Intel systems. Its cloud offering now sees heavy usage of Chinese-origin models by businesses worldwide, with the United States and Germany among important sources of demand.

Cost is by far the largest immediate problem that open models solve. But a business's longer-term goal is better control over AI and the ability to customize models for its own use case. Lower cost is therefore not just a savings measure; it makes it possible to experiment and later tailor models to unique workflows. AT&T, for example, was reported to have shifted 40 percent of its token consumption to open models, while evaluating models from several regions.

Coding agents and exploding token demand

Jeffrey Morgan: Coding agents account for much of the growth in token use per developer and per user. The takeoff of OpenClaw and the Hermes agent project expanded that opportunity beyond developers: people working in finance, support, marketing, or sales can delegate a difficult task to an open model and let it run over a long period. Such agents consume many tokens while choosing tools and retrieving data.

Two inflection points stood out in Ollama Cloud's token-use graph. The first, at the start of the year, came when models such as Kimi, GLM, and MiniMax became capable enough to power coding agents. The second came with OpenClaw, which helped non-developers hand harder tasks to models. Larger context windows, growing from roughly 128K to more than a million tokens for open models, helped make that expansion possible.

The result was dramatic. Per-user consumption rose several-fold for core work use cases, while aggregate cloud usage increased roughly 150-fold since the beginning of the year. In 2024 and 2025, large open models were often used as custom fine-tuned models. The out-of-the-box serving of open models began taking off only this year.

Model releases, fine-tuning, and safety

Host: Fine-tuning attracted enormous interest in early 2024, then seemed like wasted effort because the next release would overtake a custom model. Is it returning, or is this merely another cycle?

Jeffrey Morgan: Release cadence is accelerating, especially on the open side. DeepSeek Flash, for example, had already seen three iterations over a summer, where releases once followed something closer to a six-month cycle. That pace makes custom training harder to maintain, but the tooling for fine-tuning is improving and helps teams stay current.

Safety and security are a major adoption constraint. Frontier labs may slow while addressing alignment and containment, while open-weight models continue to improve. The capabilities of models such as GLM-5.3 create an opportunity for security and governance companies, because the main blocker for many enterprises considering open models is still safety. If those concerns can be addressed, using Chinese-origin model labs is a viable option for many customers in Europe and the United States.

Open models can be especially useful for authorized security testing. Closed frontier models often refuse a request to pentest a product, while security-oriented open models can be trained for such work. Even general open models retain safety training, but may better distinguish legitimate defensive work from malicious intent.

Launching models at production scale

Host: As a key distribution channel, Ollama often sees models before general release. What does operating those launches at scale require?

Jeffrey Morgan: Models are arriving faster and faster, so Ollama developed a day-zero launch playbook. A model has to work in users' preferred inference engines, perform quickly and accurately, match the reference implementation, and be paired with the right use cases and developer harnesses. A new multimodal DeepSeek model, for example, required an updated harness to expose its capabilities effectively.

Every model has distinct architecture, tool-calling mechanics, and operational challenges. The final product must be benchmarked before release to ensure that it behaves as the lab intended. Ollama packages three things: a harness or SDK, the model itself with sufficient reliability and cloud capacity, and hardware or provider support. The last part can involve inference providers, Nvidia, and Apple Silicon. A capable model that runs slowly is not a good user experience.

Access to a model may arrive only weeks before launch, and much of the integration sometimes happens in the final twenty-four hours. The company describes its role with an operating-system analogy: it connects hardware drivers, providers, the inference layer, application runtime, and harnesses. The combination is large and difficult, but reusable pieces can make it possible to develop, test, and release a common runtime that matches many models to many harnesses.

The emerging agent stack

Host: Could the layers between model and application become their own companies or providers?

Jeffrey Morgan: Yes. One useful framing separates the opportunities into knowledge, coordination, and execution. Knowledge is connecting a company's data and context to a model. Coordination is managing requests that fan out to subagents across local and cloud environments. Execution includes sandboxes and the compute required as agents increasingly operate in the cloud.

Open source historically creates best-of-breed companies around each layer. Earlier cloud platforms bundled capabilities in products such as Heroku or Google App Engine, but developers ultimately preferred specialized products. Frontier labs may seek to keep users inside managed-agent, context, and memory systems, while open-source developers will build alternatives. The likely outcome is a mix of bundling and unbundling.

Tokens are becoming abundant because many providers can serve open models. The new scarcity is the engineering required above tokens: orchestrating an agent from one task to another, storing state, managing credentials, and providing security tooling. Some harness features may move down into models, but stateful problems such as storage cannot simply become part of a periodically retrained model. Those layers remain substantial opportunities.

Open and frontier models in the enterprise

Host: Do open models and frontier models grow together, or will enterprise spend reach a steady split?

Jeffrey Morgan: Ollama expects a supermajority of an enterprise's tokens to use open models, perhaps 80 to 90 percent, while open models may account for only 10 to 20 percent of spending because they lower unit costs. That abundance lets businesses offer more access rather than rationing tokens. The hardest tasks will still belong to frontier labs with leading researchers, and many mid-range tasks will use open and closed models together.

The customer should decide where routing and orchestration occur. A frontier model may handle harder scheduling or coordination, while open models perform line-item work. Projects from Sakana AI and OpenRouter demonstrate the value of combining both. The analogy is a law firm: a partner handles the most difficult judgment and delegates other work to associates. Cloud computing already followed a similar path, mixing proprietary services such as DynamoDB with open technologies such as Postgres.

Local execution and hybrid routing

Host: Ollama operates both cloud-hosted and local models. How will those worlds develop?

Jeffrey Morgan: The answer is another hybrid. Easier tasks can run locally with lower latency and lower per-token cost, while businesses use cloud models for harder work. Newer hardware is increasingly capable of running models in the 20-billion to 40-billion parameter range, and sometimes as large as 128 billion parameters. Benchmarks suggest that Qwen 3.8 38B can approach Opus 4.6 for coding, while remaining usable on a relatively modest MacBook configuration.

Local usage includes strong American, European, and Chinese model choices, including Llama and Gemma. Coding agents tend to benefit most from large cloud models because they must solve difficult problems and write code and tests. Document-processing workflows can often run well locally because the task is more straightforward. A router can send easier work to local models and escalate difficult work to a large cloud model, reducing costs further because business hardware is already available.

China, the United States, and open-model competition

Jeffrey Morgan: Cloud coding-agent consumption is predominantly Chinese-origin models today, while local-model usage is a stronger regional mix. That difference reflects both model availability and the kinds of workloads deployed. The release of models such as Nvidia's Neotron Ultra suggests that more large U.S. open models are arriving.

The conversation also considers the competitive feedback loop between open and closed providers. Frontier labs have a highly integrated stack: applications, models, infrastructure and inference, chips, and energy. Open models create more choices at each layer, but those choices need curation. Model providers, hardware companies, inference services, and harness developers must work together to give developers a simple, reliable experience.

Ollama's business model and YC experience

Host: Ollama spent time building an open-source project before its business model was obvious. How did the team think about making money?

Jeffrey Morgan: The team saw two durable paths: privacy-focused AI products and access to open models for difficult tasks. Local execution was important but never the entire story. The company waited for product-level product-market fit comparable to closed models, and coding agents became that moment: the largest area of AI consumption could finally be served by open models.

Waiting carries risks. A team can build a culture in which monetization is never a priority, a lesson the founders learned from earlier Docker experience. The more important discipline is staying close to customers rather than treating a large open-source user base as an undifferentiated mass. Teams need to meet customers, understand their current work, and learn what they want to accomplish in six months.

Morgan says YC was valuable even for second-time founders because building a company remains lonely. Peer groups and the later YC network gave the team people with whom to share problems, learn from past mistakes, and avoid repeating failures that might otherwise threaten a company. Seeing a team deliver real product-market fit also gives young founders a practical sense of quality, release discipline, and the standards that successful software requires.

Building for an AI-native software world

Jeffrey Morgan: Experience from infrastructure and cloud companies remains valuable, but many old assumptions no longer hold. The idea that a startup is necessarily vulnerable because it sits on top of another layer is less reliable in AI; moving up the stack can bring a company closer to the customer. Traditional systems engineering seeks deterministic, tested, validated behavior, while language models are inherently imperfect and sometimes intentionally non-deterministic.

AI also changes staffing and service design. Some work that required large teams a decade ago can now be handled differently, but companies must learn how to operate services when no individual engineer fully understands every line of generated or evolving code. Long-lived software still needs accountability: teams must ask whether a product will work for customers who continue using it years later.

The central opportunity is curation. Developers do not want to navigate a fragmented universe of models, inference technologies, cloud services, and harnesses, or spend hours discovering undocumented provider behavior. They want to build software, applications, and companies. Services such as OpenRouter offer broad model selection and unified billing; harnesses such as OpenCode can connect developers to many models. As models and providers become abundant, bringing those pieces into a dependable system becomes the scarce and valuable work.