OpenRouter's COO: 1T agent tokens per day. Plus, the path to ubiquitous AI and ggml.ai joins Hugging Face.
OpenAI reportedly finalized an $850 billion valuation deal over the weekend, signaling continued investor confidence in foundational models. OpenRouter's COO shared data revealing builders are now generating a trillion agent tokens daily, highlighting the sheer scale of real-world agent deployments. This comes as new analysis details the technical path to truly ubiquitous AI, targeting 17,000 tokens per second.
OpenAI is reportedly finalizing a funding round that could raise over $100 billion at an $850 billion+ valuation. Major tech players like Amazon, SoftBank, Nvidia, and Microsoft are expected to invest. This signals continued investor confidence in the AI market, relevant for founders fundraising and for builders tracking how major platforms like OpenAI are pursuing profitability (e.g., through ads in free ChatGPT).
Read full story→Cursor launched a plugin system, allowing its agents to connect with external tools and expand their knowledge base. Partners including Amplitude, AWS, and Stripe offer plugins covering the product development lifecycle. Users can discover and install these from the Cursor Marketplace or create their own using primitives like skills and subagents.
The team behind llama.cpp is now part of Hugging Face. This move aims to boost local AI development, integrating llama.cpp with Hugging Face's transformers library and improving inference accessibility. Projects remain open-source and community-led.
Taalas unveiled custom silicon, Hardcore Models, designed for specific AI models to overcome inference latency and cost barriers. Their first product, a hard-wired Llama 3.1 8B, achieves 17,000 tokens/second. This enables developers to explore new real-time AI applications previously impractical due to latency and cost.
One builder shares nine observations from a year of building AI agent systems. Prioritize state-of-the-art models for unpredictable inputs, and fine-tune smaller models for specific tasks. Strongly-typed languages like Rust catch errors early, and building a "braintrust" of agents critiques work.
This article explains 'semantic closure' as a core difference between compilers and LLMs. Compilers verify their outputs against formal specifications and report explicit errors. LLMs, however, generate outputs based on statistical patterns without internal correctness verification, even when deterministic.
Cloudflare’s new "Code Mode" technique gives AI agents access to their entire API using only 1,000 tokens. Agents write and execute JavaScript code against a typed SDK, interacting via `search()` and `execute()` functions instead of needing thousands of individual endpoint descriptions. This cuts token consumption by 99.9% for API interactions, and Cloudflare open-sourced the SDK for other builders.
A Hackernoon article argues that AI tools accelerate code implementation but expose human comprehension as the new bottleneck. Rapid code generation outpaces a team's ability to understand and maintain the code, leading to "comprehension debt." This encourages adding more features and complexity than necessary, creating fragile systems.
Data from OpenRouter, processing a trillion tokens daily, shows agent tool call rates surged from under 5% to over 25% in the last year. This signals a shift to production, with reasoning tokens now comprising 50% of model output. Teams often use frontier models for planning and smaller models for tool execution.
OSA (Open Source Advisor), a Python library, uses LLM agents to generate README files, create code documentation (docstrings), set up CI/CD pipelines, and organize repository structures for open-source projects. This tool supports various LLM providers and can generate pull requests with suggested improvements for any repository.