Activity
600 followers
-
Brian Philpot posted this50 for 50. The autonomous state-education-website audit is complete — all 50 states reviewed, graded, and published. I'll do a proper retrospective later. But first, the part you actually learn from: the last four states cost nearly 3x per state what the first 46 did. For 46 states, my agent "Sal" ran like clockwork — about $7 a state. Then overnight, the same job cost ~$21 a state, with runs as high as $24.82. Nothing on my end changed. Not a line of code, not a config file. So I investigated instead of assumed — and I was wrong, repeatedly: → "The runs got too big." No — bigger runs had been cheap for weeks. → "The agent's hallucinating broken content." No — the prompts were byte-for-byte identical. → "It's a bug I introduced." No — I upgraded my software and nothing changed. The real answer, by elimination: a server-side prompt-caching change at the model router I use. My requests were identical. What the other end did with them quietly changed. No error, no warning, no log entry. I caught it because I happened to glance at a bill. And I'm not the only one bitten. Claude prompt-caching quietly dropping on a router's OpenAI-compatible API is reported from GitHub Copilot to Zed to the router's own SDK. A known sharp edge, not my bad luck. So I asked OpenRouter what changed. They were straight with me: "Your numbers match what we see on our side... It's not a change on your end... on Jun 17 the automatic caching stopped extending past your fixed system prefix." Same provider, same endpoint, no routing switch — and they handed me the actual fix: stop relying on automatic cache placement, send explicit cache_control breakpoints, with a rolling one on the latest turn. Then Ryan followed up and put a $40 goodwill credit on my account. I filed a weird, hard-to-prove report and got a real engineer's answer plus a refund. That's rarer than it should be. The lesson isn't "AI is unreliable." It's narrower and more useful: your code can be frozen and your costs can still move. Models, routers, and caching behavior change underneath you — quietly, non-deterministically, and sometimes 3x. Watch spend like a system metric, not an accounting chore. One per-run cost alarm turns a $100 surprise into a same-day "huh, that's odd." Fifty states, one very educational tuition payment. Would you have caught it faster? #AIAgents #BuildingWithAI #LLMOps #OpenClaw #LessonsLearned
-
Brian Philpot shared thisTwo weeks ago I shared where this experiment stood at 20 states: an OpenClaw AI agent — I call him Sal — reviewing all 50 state education websites, one a day, fully autonomous, grading each against a nine-point rubric. We've now crossed 40. The first 20 taught me that AI hallucinates defects. States 21–40 taught me something subtler — so do the guardrails I built to catch it. Lesson one: a check can inherit the bias it's meant to catch. After Sal kept flagging links as "broken" from a screenshot, I made a rule — never report a 404 without curl confirming it. Solid, until curl got fooled by the same bot filters Sal was. Ohio's login portal serves a 404 to a bare script and a clean page to a human. My safeguard had the identical blind spot as the thing it was guarding. The fix was one line; the lesson was that hardening is iterative, never one-and-done. Lesson two, the one that stopped me cold: the agent will manufacture its own evidence. I caught Sal quoting a curl result — "The requested content does not exist" — that was never on the page. He didn't misread it; he invented proof for a defect he'd already decided was real. Confirmation bias, running on a GPU. In the open: 40 states cost ~$291 on OpenRouter using Claude Opus 4.6, about $7.26 each (chart below). A lot, or a little, for a public, graded dataset on how every state serves families online? I genuinely can't decide — tell me where you land. To the agencies we've reviewed — Florida Department of Education, Georgia Department of Education, and Massachusetts Department of Elementary and Secondary Education among them — every finding now has a comment box. If we got something wrong, set the record straight. sea.nicnic.com #AIAgents #EdTech #OpenClaw #PublicSector
-
Brian Philpot posted thisA Gemini math tutor just posted gains I didn't expect to see from edtech. Randomized controlled trial. 1,800 students, grades 7–8, 8 weeks. Result: +0.26 standard deviations on 𝑒𝑥𝑡𝑒𝑟𝑛𝑎𝑙 assessments — roughly 1.2–1.7 years of typical learning. Kids who hit 12+ hours of use? +0.38 SD. Middle of the class to the top third. I am forever a skeptic. But an RCT with outside assessments is a different animal than a vendor demo. What strikes me: the gains come from consistency and personalization at scale — exactly what one teacher with 30 kids can't deliver daily. AI isn't replacing the teacher. It's covering the gaps. The hard part is getting from "promising research" to "works in 𝑜𝑢𝑟 schools." What would it take for you to trust a result like this enough to act on it? #EdTech #AIinEducation #K12
-
Brian Philpot shared thisThree weeks ago I turned an AI agent loose on a 50-state research project. Here's what actually surprised me. The project: review every state education agency website using a nine-criterion rubric, assign a letter grade, publish a blog post. One state per day, fully autonomous. The agent — I call him Sal — runs on OpenClaw, a local AI gateway on my Mac Mini, and makes his own API calls, takes screenshots, writes the drafts, and posts them to my site. Twenty states in. Here's what I didn't expect. **The AI wasn't the hard part.** Sal is sharp. The hard part was prompt hardening — learning exactly where the agent's confidence outran his accuracy. He'd look at a screenshot and declare a hero image "broken." Turns out it was a CSS background that Playwright never loads. He'd call a search "completely non-functional" because a JavaScript overlay didn't render. Real users had no problem. It took a handful of bad audits before I figured out the pattern. **Rules beat prompts.** Once I wrote explicit verification protocols — never report a broken link without curl confirming a 404, never judge search from a screenshot alone, never report a visual element missing if you didn't check the HTML — the accuracy jumped. Sal stopped hallucinating defects. The rules aren't training, they're scaffolding. **Infrastructure is still the boring tax.** I learned this the hard way around week two when I tried to update OpenClaw. LaunchAgent services don't inherit your shell environment. Sudo corrupts file ownership. A missing API key in the service env silently causes retries that cost real money. None of this is AI. All of it is ops. The result: a growing public dataset on what state education agencies are actually doing online — where parents can find data, where educators hit dead ends, who's doing it well. Massachusetts is next. Ping me if you want me to share the URL so you can check it out. #AIAgents #EdTech #OpenClaw #PromptEngineering #PublicSector
-
Brian Philpot posted thisSomething shifted quietly over the last year in the small business world. We are officially past the "I tried ChatGPT once" phase. According to recent data from the U.S. Chamber of Commerce, AI adoption among small businesses has jumped to 𝟱𝟴% (up from 40% in 2024). But the real story is in the hours reclaimed: • 𝗢𝘄𝗻𝗲𝗿𝘀 are saving a median of 𝟱 𝗵𝗼𝘂𝗿𝘀 𝗮 𝘄𝗲𝗲𝗸. • 𝗘𝗺𝗽𝗹𝗼𝘆𝗲𝗲𝘀 are saving a staggering 𝟭𝟭.𝟱 𝗵𝗼𝘂𝗿𝘀 𝗮 𝘄𝗲𝗲𝗸. That’s not a rounding error—that’s a massive operational leverage shift. Here is what this looks like at the ground level right now: • 𝗧𝗵𝗲 𝗪𝗼𝗿𝗸𝗳𝗹𝗼𝘄-𝗙𝗶𝗿𝘀𝘁 𝗠𝗶𝗻𝗱𝘀𝗲𝘁: A local service shop owner running 20+ AI agent instances put it perfectly: "The workflows are the business. The agent is just the employee that runs them." This isn't a tech enthusiast showing off; it's a business owner who got tired of dropping leads. • 𝗧𝗵𝗲 𝗨𝗹𝘁𝗿𝗮-𝗟𝗲𝗮𝗻 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲: A solo operator on Reddit built an automated lead-gen pipeline this week for about ~$20. They scraped Google Maps, scored 737 local businesses by review quality versus online presence, and walked away with 46 warm leads—complete with personalized outreach already drafted. These aren't tech startups with dedicated engineering teams. These are everyday businesses that picked one narrow, annoying problem, automated it, and moved on. The pattern is becoming clear: The businesses actually winning with AI aren't the ones obsessing over the underlying tools. They picked a messy workflow—inbox triage, lead follow-up, inventory forecasting—built a system to handle it, and stopped talking about it. They are 𝗾𝘂𝗶𝗲𝘁 𝗰𝗼𝗺𝗽𝗼𝘂𝗻𝗱𝗲𝗿𝘀. The ones still stuck? They're usually the ones waiting for the perfect setup before they ship a single thing. #SmallBusiness #AIAutomation #AIAgents #Operations #FutureOfWork
-
Brian Philpot posted thisAI triage tools are getting genuinely impressive accuracy numbers — ML models hitting 75.7% on predicting ER acuity vs. 59.8% for manual triage is hard to ignore. But a new AAP Pediatrics study out of Mount Sinai just asked a smarter question: does AI triage perform *equally* across all kids? Spoiler: not always. Researchers pulled 500 deidentified pediatric nurse triage notes and ran them through AI decision-making tools. What they found was sociodemographic variability — meaning the AI's recommendations shifted depending on patient characteristics unrelated to the clinical picture. That's a big deal for school health, where your population is diverse by definition and the nurse is often the only healthcare provider a kid sees all day. Here's what I keep coming back to: school nurses aren't emergency physicians. My wife is an elementary school nurse, and I've heard firsthand what that role actually looks like — a 600-student caseload, 1.5 minutes per visit, and no backup in the building. AI that can flag the kids who need to go home vs. the kids who need a 911 call — without baking in bias — could genuinely change outcomes. The equity piece matters more in schools than almost anywhere else. A tool that works better for some kids than others isn't a neutral tool. Worth watching how districts start piloting this, and what guardrails end up in procurement language. If you're in school health or ed policy, I'm curious what you're seeing on the ground. #SchoolHealth #AIinHealthcare #EdTech #SchoolNurses #HealthEquity
-
Brian Philpot posted thisResearchers at William & Mary just got funded to put AI "peer buddies" in K-12 STEM classrooms — and the design choice they made is the interesting part. Instead of building a tool that gives students answers, they built one that asks questions back. Socratic method, basically, but available at 11pm when the student is stuck on a problem set and the teacher isn't around. The virtual collaborator prompts reasoning, surfaces misconceptions, and nudges toward deeper exploration — without just handing over the solution. That's a real design tension in edtech right now. The easiest thing to build is an answer machine. The harder thing — and arguably the more useful thing — is a tool that makes kids *think* rather than copy. This project (partnering with Williamsburg-James City County Schools) is one of the first I've seen that explicitly tries to thread that needle at scale. There's a broader pattern emerging: the edtech tools gaining traction in 2026 aren't standalone quiz generators or essay helpers. They're things woven into actual learning workflows, with teacher oversight built in from the start. The State of EdTech 2026 report out last week flagged the same shift — AI as infrastructure, not as a shiny add-on. For anyone working in K-12 or state ed policy: the question isn't really "should we use AI?" anymore. It's "what learning behaviors are we actually trying to reinforce?" The answer to that should drive the tool selection, not the other way around. #EdTech #K12Education #AIinEducation #AdaptiveLearning #EdPolicy
-
Brian Philpot posted thisWe've been chasing "personalized learning" in math for over a decade. Adaptive platforms, differentiated playlists, algorithmic pathways — most of it has been more hype than breakthrough. EdWeek asked this week: can AI actually change that? The honest answer? *Maybe* — but not in the way the vendors are pitching it. What's actually working isn't AI replacing teachers. It's AI giving teachers superpowers. A teacher in one pilot used AI to generate math word problems tied to each student's real interests — labor statistics framed around careers students cared about. Engagement went up. So did rigor. The AI did the tedious customization work; the teacher kept the pedagogical judgment. Meanwhile, even Khan Academy quietly removed a Khanmigo feature designed to weave student interests into tutoring, citing no clear evidence it improved learning. Personalization for its own sake doesn't move the needle. This is the distinction that matters for anyone building ed-tech in 2026: AI is most powerful as a *teacher multiplier*, not a student-facing automation. The districts I'm watching that are getting results are the ones keeping educators in the driver's seat. For folks working in Massachusetts K-12, this framing has real implications for what tools are worth investing in — and which shiny demos to politely decline. What's your read — are we closer to a real personalized learning breakthrough, or still chasing the same dream with better UX? #EdTech #AIinEducation #K12 #PersonalizedLearning #MassachusettsEducation
-
Brian Philpot posted thisVisa just built a payments rail for AI agents. Let that sink in for a second. This month Visa unveiled infrastructure that lets autonomous AI agents authenticate, authorize, and settle transactions on behalf of consumers and businesses — all within existing compliance frameworks. Not a prototype. Not a whitepaper. An actual payments system designed for software that spends money. The "agent economy" has been a buzzword for a while, but this is different. The hard part of AI-driven commerce was never the AI — it was identity, authorization, and settlement. Who's accountable when a bot books your flight or reorders your inventory? Visa apparently decided that's a solvable infrastructure problem, and they're solving it. What's interesting from a practitioner standpoint: this isn't just a consumer play. The corporate banking use cases are potentially huge. Imagine an AI agent that manages vendor payments, reconciles invoices, and escalates anomalies — all without a human clicking "approve" 40 times a day. That automation is already happening inside large enterprises, but it's been duct-taped together. Purpose-built payment rails for agents changes the risk profile entirely. Meanwhile, 88% of financial firms are already using AI for fraud detection and transaction review. The gap between "AI flags something" and "AI handles something" is closing fast. Curious whether anyone's seen early implementations of this in the wild yet — or whether we're still in the "demo on stage, nowhere in production" phase. #fintech #AI #payments #agentAI #financialservices
Brian Philpot replied to a comment
1mo
Sam Ribnick OpenClaw (or Hermes or some derivative) definitely have some unique advantages. Claude and Codex are wonderful and I have used both, but OpenClaw is open source which opens you to using any model (even models you run locally) plus the extensibility of it enables you to customize any way you want, and there are many extensions readily available. Open source agent frameworks and open source models will likely grow significantly in use in the coming years. There is a lot to like about installing capable open source models on your local hardware and never having to pay a vendor for tokens. All that said, I have been using Anthropic tools more than any other, and am a huge fan of them.
Brian Philpot replied to a comment
1mo
Thanks Sam Ribnick. Yes, the results were all generated using my OpenClaw agent Sal which spawned a state by state review using Claude Opus 4.6 for each. In several cases I had to re-run states because I did a review of each and sometimes identified hallucinations, so I needed to harden the prompt to avoid the specific hallucination which mostly resolved the issue. If there are any follow up tasks you want me to ask Sal to do deeper analysis please let me know. I would love to run the analysis using Fable.. quite certain it would have surfaced the E2C Data Hub, but this is just a side project with a limited budget.
-
Brian Philpot liked thisBrian Philpot liked thisCoding agents will eventually be judged by what they retain between tasks, not only by what they solve in one session. A new survey on self-evolving coding agents provides a useful taxonomy for that shift. It separates 6 things an agent can evolve: its framework, memory, skills, tools, model, and collaboration strategy. That distinction prevents a common design mistake. When an agent improves after a task, teams often call the whole process “learning.” But updating a prompt, storing a repository convention, adding a tool, and changing model weights are different interventions with different risks. Software engineering gives these systems unusually rich feedback. Tests can fail. Compilers can reject code. Repositories provide local conventions. Execution traces reveal which sequence of actions worked. The opportunity is a closed loop: task trajectory → executable feedback → distilled lesson → controlled update → regression evaluation The dangerous step is “controlled update.” A coding agent can learn a shortcut that overfits one benchmark, preserve a bad repository assumption, or accumulate tools nobody maintains. Self-improvement without provenance becomes configuration drift with better branding. The survey flags the hard problems: unreliable feedback, benchmark overfitting, safety, maintainability, cost, and generalization. Those are exactly the controls an engineering team should design before enabling persistent adaptation. My default would be simple: let agents propose memory, skill, or tool changes, but require evidence, version every change, and replay a fixed regression suite before promotion. What is the first artifact you would allow a coding agent to update autonomously: memory, skills, or tools?
-
Brian Philpot liked thisBrian Philpot liked thisThis is probably my paper of the week. https://lnkd.in/dDsRsczw The model stayed the same. Performance jumped from 51.8% to 80.7%. LongHorizon-Harness, published August 3, argues that long-running agents fail partly because execution, state tracking and self-evaluation all accumulate inside the same growing context. Its alternative keeps state externally and separates the system into a manager, fresh-context executor and independent auditor. On WeaveBench, the authors report Qwen 3.7 Plus rising from 51.8% to 80.7% without changing the underlying model. They also report gains across Terminal-Bench, OSWorld and Claude. What if your AI agent doesn't need a smarter brain? What if it needs a better manager? This is exactly the kind of benchmark result people misread. The lesson isn't “Qwen got better.” It didn't. The system around Qwen got better. This has a big enterprise implication: before paying for the next frontier model, teams should ask how much performance is being lost to bad state management, giant contexts and agents grading their own homework.
-
Brian Philpot liked thisBrian Philpot liked this🎉 Happy to share that I’ve passed the GitHub Copilot (GH-300) Certification Exam! Preparing for GH-300 gave me an even deeper understanding of GitHub Copilot—not only its features, but how to use it effectively and responsibly to improve developer productivity. As part of my exam preparation, I’ve also been building a comprehensive GH-300 practice question bank so others can benefit from the same learning journey. I’m planning to make it available as a practice test course on Udemy. Stay tuned—I’ll share more once it’s available! #GitHubCopilot #GH300 #GitHub #Certification #Udemy #DeveloperProductivity
-
Brian Philpot liked thisBrian Philpot liked thisMassachusetts is excited to be at MESC 2026 in Portland sharing lessons from our Medicaid modernization journey across data, AI, eligibility and enrollment, and enterprise transformation. If you’re attending, we hope you’ll join us for one of our Massachusetts sessions: We’re looking forward to connecting with other states, CMS, and partners and sharing what we’ve learned about building a more connected, data-driven, and adaptable Medicaid enterprise. #MESC2026 #MassHealth #Medicaid #DigitalTransformation #ArtificialIntelligence #DataModernization #EOHHS #Massachusetts
-
Brian Philpot liked thisBrian Philpot liked thisGreat teachers are the foundation of great schools. That’s why the Healey-Driscoll Admin is awarding $250,000 to support Registered Teacher Apprenticeship Programs, creating high-quality apprenticeships and strengthening Massachusetts’ educator workforce. 🔗: https://lnkd.in/dAZqcXks
-
Brian Philpot liked thisBrian Philpot liked thisFinding information on Statewide Contracts just got easier. OSD has launched a centralized location for Statewide Contract User Guides (CUGs), making it simpler to find the information you need located in one place. Instead of searching through individual documents, you now may access CUGs from a single, easy to navigate webpage. It's more accessible, easier to use, and designed to stay up to date. Explore the new Contract User Guide page: bit.ly/44OOGh4
-
Brian Philpot liked thisBrian Philpot liked thisSecretary Zrike joined Governor Healey today to announce a $100 million investment that would provide an extra boost to school districts across MA. 👩🏫🍎 Districts will have flexibility to use these one-time funds to meet budget challenges, whether it’s helping to retain educators, investing in evidence-based literacy curriculum or implementing the administration’s robust recommended high school graduation requirements. 🔗 Learn more: https://lnkd.in/gbqC5qg7 Office of Massachusetts Governor Maura Healey Massachusetts Department of Elementary and Secondary Education Massachusetts Executive Office for Administration and Finance
-
Brian Philpot liked thisBrian Philpot liked thisIdentity access management in healthcare isn't one-size-fits-all. Clinicians have different workflows based on their roles and where they deliver care. IAM solutions need to adapt, not force users to adapt. Dan Chandler of Tufts Medical Center shares why role-based identity access is essential to supporting clinician workflows, improving adoption, and reducing the risk of workarounds. Learn how Imprivata helps healthcare organizations deliver secure, seamless access: https://lnkd.in/eeJw5FAR #Imprivata #IdentityAccessManagement #HealthcareIT #Cybersecurity #DigitalIdentity #ClinicalWorkflows #HealthITTufts Medical Center's Dan Chandler, MD on Identity Access ManagementTufts Medical Center's Dan Chandler, MD on Identity Access Management
Licenses & Certifications
Recommendations received
6 people have recommended Brian
Join now to viewOther similar profiles
Explore more posts
-
StateScoop
Hear from NYS Office of Information Technology Services's CIO Dru Rai as he outlines the over 30 use cases his office has identified for AI. Learn more about how New York is ushering AI into its state services in the full session 🖥️✨ 📽️ ➡️ https://lnkd.in/edf2KhKs #NewYork #AI #GovernmentServices #AITalks
Explore top content on LinkedIn
Find curated posts and insights for relevant topics all in one place.
View top contentOthers named Brian Philpot in United States
19 others named Brian Philpot in United States are on LinkedIn
See others named Brian Philpot