Usama Fayyad

An interesting study on scaling models. Everyone knows the Scaling Law—the more parameters, the smarter the model. That law has driven the industry in recent years: GPT-3 at 175B, PaLM at 540B, Llama-3.1 at 405B. The logic is simple – if you want better, make it bigger. But Chinese researchers from Tsinghua asked a different question: what if we look not at absolute size, but at efficiency? How much “intelligence” is packed into each billion parameters? They introduced the concept of capability density – the model’s ability density. The idea is interesting: take a reference model, plot a “parameters → quality” curve, and then for any other model, compute how many parameters the reference would need to achieve the same result. The ratio of this “effective size” to the actual size is the density. Analyzing 51 open models from the last two years, they found a pattern they call the Densing Law: the maximum LLM density doubles roughly every 3.5 months. The numbers are striking. Llama-1 in February 2023 had a density of about 0.1. Modern models like Gemma-2-9B and MiniCPM-3-4B reach densities around 2.0. Over two years, efficiency has increased 20-fold. Practically, this means model size required to reach the same quality is falling exponentially. MiniCPM with 2.4B parameters, released in February 2024, performs on par with Mistral-7B from September 2023. In four months, the required size shrank threefold. Even more interesting are inference prices. GPT-3.5 in December 2022 cost $20 per million tokens. Gemini-1.5-Flash in August 2024—$0.075. A 266× drop in under two years. Moreover, cost is falling faster than density is rising—because inference infrastructure is improving in parallel. The authors make an intriguing observation about combining laws. The Densing Law captures algorithmic efficiency growth; Moore’s Law captures hardware power growth. Together, they yield a doubling of the “effective size” of the model you can run on a fixed device roughly every 88 days. This opens the path to full-fledged LLMs on smartphones and laptops! It’s also interesting that compressing models via pruning and distillation doesn’t always increase density. Compressed models often end up less efficient than natively small models – apparently due to insufficient post-training after compression. The authors’ main conclusion is fairly provocative: the parameter-count arms race is a dead-end strategy. Any SOTA model will be outclassed in price/performance by newer compact models within months. DeepSeek-V3 has already shown this path – instead of piling on parameters, they focused on an efficient architecture and achieved top-tier quality at lower cost. In essence, the Densing Law is a Moore’s Law analogue for LLMs. Moore described progress in hardware; here we’re talking algorithms and data. And as long as this law holds, every 3.5 months we get models twice as efficient as the previous generation. https://lnkd.in/d5qHtUFZ

2

AMI and Yann LeCun are betting the real future of AI is specialized. Their take: world models built on JEPA that reason causally, plan under constraints, and work in messy real-world environments. Their Feb 2026 paper coins it: Superhuman Adaptable Intelligence (SAI). The idea in short is “Outperform humans on specific tasks, tackle domains we simply can’t.” Nabla (clinical AI) is already on board as AMI’s first partner. More architectural innovation instead of more compute. Curious to see where this goes. https://lnkd.in/gmB4k4EA

19

2 Comments

Voice AI is quietly hitting an inflection point 🔊 NVIDIA just published a great deep dive on scaling speech recognition for real-world voice agents - latency, streaming accuracy, robustness, and why “ASR quality” isn’t just about WER anymore. 👉 https://lnkd.in/enDUKHvG A few things that stood out: - Streaming + low latency is non-negotiable if you want natural conversations - ASR needs to handle interruptions, background noise, accents, and partial utterances, not clean demo audio - Scaling voice agents isn’t just model quality, it’s systems engineering If you want to actually try building or running voice agents instead of just reading about them, Pipecat is a great place to start: 👉 https://pipecat.ai This is the kind of progress that makes voice agents finally feel… practical. Not demos. Not prototypes. Production. Curious to see where the ecosystem lands this year 👀

150

10 Comments

The Valyu API is the leading search API for scientific research. Why? We were the only ones crazy enough to process and index 100% of ArXiv for full-text, multimodal, and chunk-level search (not just abstracts.) Here’s what that means: - 🔍 Full-text retrieval: Full content of papers is searchable not just abstracts. - 🖼️ Multimodal context: Retrieve not just text, but figures, tables, and equations - 🧾 Complete metadata: APA citations, authors, references, citation counts. - ♻️ Fresh search: Updated as new papers are released. A must-have for any AI agents doing real research 👇

24

7 Comments

One of the biggest challenges for LLMs has been accuracy when handling massive context.  Recursive Language Models (RLMs) -- a powerful new framework from MIT CSAIL that fundamentally rethinks how AI handles context. RLMs treats context as an external resource instead of stuffing everything into a single prompt. What makes it groundbreaking: --Unlimited context capacity – Instead of overwhelming the model with massive prompts, RLM treats the full input as an external “database.” The model writes Python code to query segments, call itself recursively, and assemble insights. --Better performance at scale – On benchmarks like OOLONG and CodeQA, RLMs outperform GPT‑5 and other context agents by 10–25%, even when processing tens of millions of tokens. --Transparent & efficient – The reasoning steps happen via generated code, making the logic inspectable. Why it matters: --Transforms long-context reasoning from a memory limit issue to a software orchestration problem --Enables scalable, explainable AI—critical for applications in legal review, scientific papers, and long-form analytics --Opens the door to general-purpose AI agents that can handle truly enormous bodies of information You can read the full paper here: https://lnkd.in/e8wuwb8a

9

Postmath raised USD 4M Series A to expand its data-driven math platform “Math Secretary,” achieving rapid revenue growth and launching AI-powered “Grip” textbooks. https://lnkd.in/gGWC6kEz

1