The Real Reason Your AI Investment Isn’t Paying Off

 By Katy Feldner, VP, AI Transformation and Value Creation

During my time leading clinical product strategy at the second largest health plan in the United States, I watched a team ship an AI model that worked. By every technical measure, it cleared the bar. Validation was clean. The data science team was proud of it, and they had every right to be.

Then it sat there, working in the background to prove its value. Nurses overrode the prior authorization recommendations. I noticed it wasn’t because the model was wrong. It was because we, as a program, hadn’t fully addressed the nurses comments and concerns in change management training on why they should trust what technology had built.

That’s the moment that changed how I think about AI transformation. The technology wasn’t the problem. The organization’s appetite for change was.

I tell every new client some version of this on day one: “I care a lot less about your tech stack than I care about whether your people are ready to change how they work.” That tends to catch people off guard. They’ve bought into the platform. They want to talk about agent workflows and data integrations. Those things matter. They’re just not what determines whether you see a return.

Change management is the single biggest determinant of whether an organization sees AI returns, or watches the investment disappear. It’s harsh, but true. And I’ve seen it happen too many times to soften the blow.

The argument that AI can drive value in healthcare is won. Nobody’s debating that anymore. What’s left is harder: changing how an organization operates, and that work is political and human in a way no platform fixes on its own.

Saying that honestly means telling your finance team their annual planning cycle doesn’t connect to the decisions they’re making by August. It means admitting the system you spent a decade building produces outcomes that don’t work anymore. I get why most teams skip that conversation and buy more technology instead. It’s the easier room to sit in.

I’ve sat with IT teams who became roadblocks for tools they could have championed, because the change conversation started after the contract was signed instead of before. I’ve watched executive sponsors treat adoption as something HR would handle after launch, not as the work itself. None of that is rare. It’s the default, and it’s expensive.

The people who determine whether an AI initiative pays off aren’t always who you’d expect. Your CTO matters, but the organizations I see get real returns bring two other groups in early:

• HR, because they know who’s going to resist, how to build a communication plan that meets people where they are, and how to frame job change as opportunity instead of threat.
• Operations, because they know which workflows break under a new process, and what it takes to get a nurse, a claims processor, or a prior-auth reviewer to adopt it.

Without both at the table from day one, you’re trying to change behavior with technology alone. I haven’t seen that work yet.

This is also where the job-threat conversation needs to happen out loud, not around. Nurses are weighing AI recommendations against their own license. IT teams are watching their relevance shift under them. Coders are wondering what happens to a five-day job that AI can do in two hours. Those fears are legitimate. Left unaddressed, they don’t go away. They go underground and quietly sabotage adoption. The honest answer is that the goal is upskilling people, not replacing them. That answer has to come from leadership, not a slide.

ROAI™ (Return on AI Investment) is the discipline I keep coming back to with clients. I help leadership teams see which initiatives move the financial metrics their board is watching, then build the change case around those specific outcomes. Once an organization commits to removing a defined amount from its operating budget by a defined date, the conversation stops being abstract: here’s what changes, here’s who owns it, here’s when.

That accountability is what makes the change real. Not the platform. The moment people sign on to real numbers and measurable impact.

When I think about the models we shipped – and that sat unused, despite working well – in my time on the health plan side of the industry, the solutions all had a common theme. The fixes, when we found them, were never another tool. They were finally treating the nurses’ trust as the project. Figuring that out remains the first step to ROAI in any healthcare enterprise. Many people still miss it, but you don’t have to.

If your AI investment is stuck, it’s likely that’s the reason why. Let’s talk about how to build trust and boost adoption so you can start seeing the returns you expected.