How to Tell an AI Win From an Anecdote

A win nobody defined in advance is an anecdote. That was the standard set for the From Pilot to Program panel at In the Round 2026. A real win has three parts. Someone decided ahead of time that it would count, a number defined it, and it earns the next decision.

By that measure, much of healthcare AI is still collecting anecdotes. 

AI ideas become programs because of decisions leaders make before the work starts, and the room they give people once it does. Those early decisions are where Optura spends its time with healthcare organizations, which is why the topic was on the In the Round 2026 agenda.

The first decision is where the impact should show up and how soon. That sounds obvious, until an organization tries to compare 37 initiatives graded 37 different ways and finds it can’t rank, fund, or stop any of them with confidence. One standard, applied to everything, is what makes the portfolio readable.

Panelists pushed the question further back. The usual debate over whether boards or operators should drive AI leaves out the customer. For a health plan, that’s the member, the employer client, and the provider partner, and the best opportunities come from working across all three. A project mindset counts a win as finishing on time and rarely measures what happened after. A product mindset spends more time in discovery, so the team solves the real problem instead of a symptom.

Sometimes the real problem is that the work shouldn’t exist. One leader questioned celebrating an automation that turned hours into minutes before anyone had asked whether the process was needed. Much of what organizations do, especially after mergers, carries legacy steps that wouldn’t survive a fresh design. Asking how the work would look if it started from scratch is part of deciding what counts.

One technology leader described how their company makes those decisions concrete. Before building anything, the team writes a one-page mock press release, with a customer quote. FAQs follow, covering the customer, partners, and the business, the metrics for each stage, the alternatives considered, the criteria for shutting it down, and how to roll it back. Timelines get the same treatment. “I don’t want to know four weeks from now that it’s not working, but it’s still running,” they said.

The moderator put the ownership rule plainly. The business is accountable, and IT provides the guardrails. If there’s no name next to an initiative, there’s no real budget and no real accountability.

One leader asked the room to stop calling it IT. It’s tech, and the people in it are engineers who bring ideas and write product proposals like anyone else. What a team is called, and how the room is set up, shapes how people behave. Organizations with large legacy IT teams are now bringing them in at step one, so they can say early whether an idea will work given everything else happening in the enterprise.

Many healthcare workflows run on what was called “osmosis or heroics,” with no written process behind them. People resist change done to them, not change done with them.

Co-creation takes time. Teams that grew up on the frontline often expect a checklist, while product work lives in the gray. Hospital culture runs on “see one, do one, teach one,” so questions and feedback can feel like criticism. Staffing and scheduling are especially emotional, since a parent who has to pick up children at four can’t take a random night shift. The tools that hold up are designed alongside the frontline through every phase, before they scale.

The payoff can be large. AI cut one pharmacy team’s administrative work by about 90%, so its pharmacists now work at the top of their license and are available around the clock. With demand for care rising and staff in short supply, the goal is extending people, not replacing them.

The organizations moving fastest treat experiments as normal. They run them, learn quickly, and cut their losses. Hackathons help. One engineer solved a pharmacy testing problem that used to require asking coworkers whether they had a prescription to refill that month.

Leadership sets the tone. One health system leader took their executive team offsite for two days of AI training and now puts an AI initiative on every leadership agenda. Another gives sharp people a quarter of their time to find AI opportunities and report back. The panel’s advice was to free people to fail fast without it counting against them, and to build on wins that are meaningful even when they aren’t transformative yet.

Those wins still have to meet the initial test. Someone decided in advance that they would count, a number defined them, and they earned the next decision. Setting that standard before the work starts, and tracking it while the work runs, is how Optura keeps every win countable.

Those wins still have to meet the initial test. For the themes that ran across every panel and roundtable, read Full Measure: The Set List, Optura’s insights report from In the Round 2026.