Siddharth Patil

Truly transformative. Within the next 6 months coding will be fully commoditized. Only design, distribution, and hardware will be the moat. It is time to lean in #AI #FutureofWork #OhioTech

1

1 Comment

A thoughtful piece by our own Brian de Luna on Applied Science in marketplaces, and the unique journey we’ve taken at Faire.

17

My team is on a roll!! In our latest research, we dug into one of the hardest challenges in AI today: causal reasoning — and exposed major blind spots in how state-of-the-art models handle complex, real-world causation. 💡 Most evaluations force models to choose one answer. But real-life causes aren’t that simple. So we tested 8 frontier LLMs using multi-select (polytomous) tasks across 576 scenarios. The results? Both surprising and sobering. 🔍 Key insights: - Models adopt a conservative strategy when identifying causes but over-select confounders—revealing asymmetries in reasoning across task types. - Chain-of-thought prompting? 🚫 Not helping here. - How you score these evaluations can change which model looks best: Our results show that rankings shift significantly depending on the scoring metric used. - We uncovered behavioral signatures by model family: some are cautious and precise, others more liberal and prone to over-selection.     ⚠️ The implications for AI in healthcare, law, and policy are huge. Missing causal factors isn’t just suboptimal — it’s dangerous. 🧠 This research doesn't just offer a new benchmark. It offers a new lens for understanding model behavior under uncertainty and instruction complexity. 🔗 Full methodology, scoring analysis, and model comparisons https://lnkd.in/gvTEDhej Major kudos to the team: David Harper, Konstantinos Karageorgos & Abigail Thornton, PhD How are you measuring causal reasoning in your AI systems? #AI #LLMs #CausalReasoning #AIEvaluation #ResponsibleAI #WeloData #MachineLearning #AIRisk #ModelSelection #AIResearch

34

1 Comment

AstroBee generates ontologies...but what is an ontology, and if they're important, why don't you already have one? I've written a short piece on Kimball star schemas, ontologies, and how each relate to Bill Inmon's original approach to data warehousing in the early 90s. Dig in to find an overview of a fascinating technological debate and the re-emergence of an approach to data modeling that's been sleeping for 20 years.

18

3 Comments

Need to apply the power of graph to your business? Tired of needing an army of data scientists to do it? Rocketgraph solves this. But seeing is believing. So I've recorded a set of two-minute tutorials to highlight the platform and how it creates a user-experience that will enable your business analysts to use graph...without being data scientists. This video kicks the series off with our POV on the state of graph tools and sets the context for the rest of the tutorials. #rocketgraph #AI #graph https://lnkd.in/gapEpnC8

10

Just read a compelling article from my colleague Noel McMichael that nails an AI problem I've been thinking about a lot lately. As we move toward deploying autonomous, API-using agents, we're hitting a wall. These agents are active and adaptive, yet they often lack a verifiable identity, a complete audit trail, or clear revocation logic. This makes them a black box, which creates risk for companies of all sizes and scopes. Noel warns that the biggest risk isn't only technical... it's "institutional paralysis". Companies that can't verify their AI's behavior will refuse to deploy them for anything important. This will likely create a two-tier system where only organizations with resources to build custom solutions benefit from AI, while everyone else operates in an environment of mistrust. His solution is a "trust fabric for AI", which is essentially a unified layer that sits below the application and above the model. This fabric would bring together the technical primitives that already exist, like persistent identity, scoped permission sets, verifiable provenance, and credential chains, to make agent behavior verifiable and secure. I really like Noel's take and highly recommend reading the full article (link in the comments section) #AI #AgenticAI #Trust #DataGovernance #AIEthics

21

2 Comments

Evangelos Simoudis from Synapse Partners on how connected data is key to a functioning AI stack at companies and governments. His company consults big companies like Toyota and governments like Singapore. #cdl25 Connected Data

14

1 Comment

A 12-month professional, voluntary collaboration has finally borne fruit. This morning, I was honored to see the University of Washington News (https://lnkd.in/gRRpDqjA) and the UW Milgard School of Business (https://lnkd.in/gKJ_pwbm) highlight our Responsible GenAI Framework. Why do we care about responsible generative AI? Because GenAI systems are no longer experimental curiosities. They shape decisions, influence behavior, generate content at scale, and increasingly affect trust, fairness, privacy, and accountability in organizations and society. Without intentional design and governance, these systems can amplify bias, create opaque risks, and erode confidence rather than enable innovation. This framework is about making GenAI useful, ethical, and sustainable at the same time. It emphasizes responsibility not as a constraint, but as an enabler of long-term value, credibility, and impact. I am grateful to the collaborators, reviewers, and institutions who supported this work, and excited to see responsible GenAI move from aspiration to practice. To learn more: - Read the RGAF Framework (PDF): RGAF Framework https://lnkd.in/gTaxnndV #GenAI #ResponsibleGenAI International Society of Service Innovation Professionals (ISSIP) Milgard CBA

23

1 Comment

Yes, enjoyed this very much from Jed Sundwall. He's speaking plainly about how we must always think of the who will use our data and and how in the context of the "perfectly FAIR, utterly useless data" we keep spewing around everywhere. I've always been a fan of figuring *use* as a priority, in favour of the hard-to-know impact, especially when you hear factoids flying around that a very large portion of stored data—likely somewhere between 60-80%—is rarely or never accessed after initial storage.

4

Barbarians at the Gate (https://lnkd.in/gkEBX79s) is a very interesting new paper from some great folks at Berkeley, with some exciting results showing the potential for AI in systems research. But I think the authors aren't quite asking the hardest problem about where this takes systems as a field. I wrote a new blog post about it: https://lnkd.in/gxS3s4v4 The core point of the paper is that AI can be effective at automatically driving the 'number go up' part of systems research. Given an evaluator, AI can automate progress on implementation. I believe this is 100% correct. The paper also talks about how more focus will have to go into developing evaluators. Again, I 100% agree, but think they're understating their case. Building evaluators (basically tests) is, I believe, going to become the focus of systems and software over the next decade. Which leaves researchers with the hard problem of choosing which problems to go after. That's never been easy in academic systems, and is only going to become more of a challenge. But the best systems research is driven by choosing problems well! This vision for AI could lead to lots more of the least interesting systems research. The "number go up" kind. Which could make it harder than ever to identify the truly innovative and visionary work. We're already seeing that elsewhere in science. So, mostly, I think the paper is right, but not yet asking the biggest question: what should systems research be?

276

12 Comments

It was a pleasure sitting down with Sachi from Sierra and Leigh Marie & Josh Coyne from Kleiner Perkins to discuss what its like shipping AI at enterprise scale. We talk about the new product principles for AI-native products incl: - Why context complete workspaces are more important than raw AI capability - The impossibility of planning roadmaps in this market - Building user trust through UX - When you need (and don't need) Forward Deployed Engineers - Re-finding PMF every few months - And why vibes are sometimes all you need. Thanks for having me on Builders Ep 2, and hope this is a useful and practical guide for anyone building in AI. Full episode link here: https://lnkd.in/ejRvsUSs P.S. Recommend all podcasts provide Chick-Fil-A to their guests :)

104

2 Comments

As an American, and a Californian, I was born into a culture of can-do optimism, with a faith in our collective ability to overcome the odds and not only survive but solve big problems. We’ve done it as a society over and over in our history. We dream, we work, we fight, we compromise and cooperate, we build, and often we rebuild. We are at our best dreaming and defying impossibility, being generous with each other and welcoming the stranger. I believe in this version of us and spent much of my career in national service because of that belief. But we are going through a very pessimistic time. We have problems as big as ever, but we don’t seem to have faith in ourselves to solve them. I spend most of my time now thinking about artificial intelligence and the problems we could solve with it. Nothing is a panacea, and it will take all of us to turn a technology into new scientific discoveries, deeper knowledge and better ways to learn it, a medical system that saves lives and supports caregivers, more meaningful work and greater prosperity. Those things are hard. But we need to do them. That’s a choice we have and can make. We have always invented our way out of problems. I am really struck by our fearfulness. Countries with far fewer advantages than we have are far more bullish on the possibilities of technology. It goes beyond AI. That’s just one small focal point. Where else do we have openings and opportunities. Where else can we roll up our sleeves and come together to fix problems? Where are we deciding to be defeated before we begin? What other possibilities are in front of us? We have proven over and over that we can rise, ultimately, to any occasion. It’s what our history has passed down to us. I’ve been thinking a lot about lines from an old Ani Difranco spoken-word poem: “Our foremothers and forefathers… came through Hell and high water/ so that we could stand here and behold breathlessly this sight/ how a raging river of tears/ is cutting a Grand Canyon of light.”

59

1 Comment

My brilliant colleague, Alison Smith, published "The Great Unscaling - Part 1". Her central thesis is deceptively simple but devastating: 𝗚𝗲𝗻𝗔𝗜 𝗯𝗿𝗲𝗮𝗸𝘀 𝘁𝗵𝗲 𝗲𝗰𝗼𝗻𝗼𝗺𝗶𝗰𝘀 𝘁𝗵𝗮𝘁 𝗯𝘂𝗶𝗹𝘁 𝗲𝘃𝗲𝗿𝘆 𝘀𝗼𝗳𝘁𝘄𝗮𝗿𝗲 𝗴𝗶𝗮𝗻𝘁 𝗼𝗳 𝘁𝗵𝗲 𝗹𝗮𝘀𝘁 𝟮𝟬 𝘆𝗲𝗮𝗿𝘀. For two decades, software meant zero marginal cost. Add a customer? No incremental expense. This made "scale at all costs" the only strategy. With GenAI, every inference costs real money. Scaling is no longer free. 𝗧𝗵𝗲 𝗺𝗼𝘀𝘁 𝗽𝗿𝗼𝘃𝗼𝗰𝗮𝘁𝗶𝘃𝗲 𝗶𝗻𝘀𝗶𝗴𝗵𝘁: While everyone chased horizontal scale through intangible assets, Meta, Alphabet, and NVIDIA were quietly building physical infrastructure. They saw something coming that others missed. Now we're watching it play out in real time: Companies either go vertical (owning the full stack for specific outcomes) or risk falling into what Alison calls "𝗧𝗵𝗲 𝗖𝗼𝗺𝗺𝗼𝗱𝗶𝘁𝘆 𝗧𝗿𝗮𝗽." 𝗧𝗵𝗲 𝗰𝗮𝘀𝗲 𝘀𝘁𝘂𝗱𝗶𝗲𝘀 𝗵𝗶𝘁 𝗵𝗮𝗿𝗱: • Metropolis didn't sell parking software; they bought a $1.8B parking operator to control the full stack • Plaid didn't try to be "data connections for everyone"; they became the indispensable operating system for fintech • The pattern is clear: 𝘀𝗽𝗲𝗰𝗶𝗳𝗶𝗰𝗶𝘁𝘆 𝗯𝗲𝗮𝘁𝘀 𝗴𝗲𝗻𝗲𝗿𝗮𝗹𝗶𝘁𝘆 𝗶𝗻 𝘁𝗵𝗲 𝗔𝗜 𝗲𝗿𝗮    Alison argues we need new metrics entirely: • Forget Total Addressable Market → Focus on Value Capture Ratio • Forget Monthly Active Users → Measure outcome-based KPIs • Add Cost per Outcome to your dashboard (or you're flying blind)    For those of us implementing Agentic AI: whether in software development, business processes, or delivery models; this reframes everything. The question isn't "how do we scale this across everything?" It's "where can we become indispensable?" This is Part 1 of a 3-part series. If the next two parts are anything like this one, they'll be essential reading. 𝗢𝘂𝘁𝘀𝘁𝗮𝗻𝗱𝗶𝗻𝗴 𝗮𝗻𝗱 𝗶𝗻𝘀𝗽𝗶𝗿𝗶𝗻𝗴 𝘄𝗼𝗿𝗸, 𝗔𝗹𝗶𝘀𝗼𝗻! You've given us the framework for the next decade. Read it here: https://lnkd.in/edddWfwq #AI #GenerativeAI #AgenticAI #Strategy #ThoughtLeadership Booz Allen Hamilton

24

3 Comments

Running evals on the new Claude 4.5 Haiku this morning—seemed like a good time to share what we’ve built for evals at Boom. The core implementation: an automated evaluation pipeline that runs every supported model against an ever growing set of test cases for our critical path inference operations. Quality, latency, and cost benchmarked in real time against production workloads. But here’s what this actually delivers for our customers: **Guaranteed uptime, not just promised uptime.** When Anthropic or Google has an outage, our system automatically routes to the next best model. Our customers’ workflows don’t stop. They often don’t even notice. We’ve decoupled their reliability from any single provider’s reliability. **Immediate access to model improvements.** Claude 4.5 Haiku dropped last week. By this morning when I finally got a few minutes to add it to our model directory, I had complete performance data across our stack. If it improves customer outcomes, we can integrate it today. Our customers get the benefit of frontier model advances within hours, not months. **Protection from silent degradation.** The harder problem isn’t when services go down—it’s when they stay up but quality degrades. Our eval system monitors this continuously. If we detect accuracy dropping on a live service, we route traffic away before it impacts customer results. The chart shows what I’m looking at right now: Haiku absolutely crushing latency on most of our critical path operations (e.g. 11s vs 26-64s for other models) while maintaining 99%+ accuracy. That’s a customer experience improvement we can ship immediately. The power is in having evals that actually matter. Test cases from realistic production scenarios. Metrics that map to customer outcomes. When you can run any model against your ground truth in minutes, you stop guessing and start knowing. Model selection becomes an empirical question, not an intuitive or philosophical one. Your customers care about outcomes, not which model you’re using.

27

2 Comments

"AI Needs a Semantic Model" - Jamie Davidson, May 20, 2025 https://lnkd.in/dx7U4B_7 Loved this piece from our friends over at Omni - whether you want trusted and easy BI or AI - you need a #semanticmodel. Semantic models serve as the essential bridge between raw data and meaningful insights, ensuring that both AI and BI systems interpret information consistently and accurately. By embedding organizational context and definitions into the data layer, semantic models enable AI to provide trustworthy answers and empower business users to make informed decisions. Excited to see the excitement and interest in this space, and to share more about Snowflake's investment in this area at the upcoming #snowflakesummit.

145

2 Comments

Building on top of large language models is fun, but getting consistent performance and reliability is extremely challenging. I love this post from Kimberly Patron about how Sierra uses health and performance-based traffic routing and request hedging to get meaningfully better performance and tail latency from high scale LLMs https://lnkd.in/g4qWRCb5

284

13 Comments

New from the Wayfair Tech Blog! My team built and shipped a multimodal GenAI system to validate product dimensions at scale. The problem sounds simple - make sure the sofa dimensions on the website match reality. But, across 30M+ products from 20K+ suppliers, it gets complicated fast. Conflicting sources, inconsistent conventions, and edge cases galore (how do you measure an inflatable palm tree?). We use Gemini to cross-reference text, images, and supplier data. Results: 85%+ precision and 70%+ recall, up from <50% precision and 7% recall with earlier classical ML approaches. ➡️ Full write-up: https://lnkd.in/e7MxEjxW #MachineLearning #GenAI #Ecommerce #DataQuality #MultimodalAI

66

5 Comments

We wrote a new essay detailing our journey towards AI powered analytics in Convictional: https://lnkd.in/gX4QCgqm TLDR: With LLMs, true self-serve analytics seemed finally within reach. No more over-busy dashboards built for everyone and exactly right for no one. Even better, the LLM could help answer questions about the data. However, a serious trust problem is introduced. Most business users are already skeptical of data and its interpretations, and when you throw in LLM hallucinations (bad SQL, incorrect join logic, or poor interpretation of data definitions) the margin of trust shrinks even further. One or two errors and the solution is written off.  Data is a crucial input to decision making, so we’ve been committed to finding a solution that can provide analytics at the levels of accuracy needed. Over the last year we tried a number of approaches, with our initial achieving an unacceptable ~50% accuracy rate, and our current productionized technique approaching 100% accuracy. The key was acknowledging the human role in the solution. We found that using a semantic layer, defined by humans and queries by the LLM was the unlock. By no longer asking the LLM to write the SQL, and instead rely on robust pre-defined metrics and configurations, it could focus on actually answering the user’s questions instead of resolving SQL. If you’re using dbt’s semantic layer (or plan to) and interested in trying it out, just touch base and we can help get you set up! cc Jake Beresford Matthew H. Chequers, Ph.D.

14