The Algorithm Was Probably Right.

That's the Worse Story.

A follow-up to last week’s post on Pope Leo XIV’s encyclical, structural hobbling, and the death of Gene Lokken. “The algorithm was wrong” is the easier but wrong diagnosis.

---

Photo by Palle Knudsen on Unsplash

Last week I called Gene Lokken’s death a case of structural hobbling. While I focused on AI algorithms given the Pope’s encyclical on AI, I was a bit disingenuous in how much blame I placed on the AI, which I noted in a footnote. The truth is that while faceless algorithms do create the potential for real harm, the Lokken case actually points to something even more disturbing: the algorithm that cut off his skilled nursing coverage was almost certainly correct on its own terms.

The research is reasonably clear that frail elderly patients with the kind of clinical picture Lokken presented rarely benefit from rehabilitation in the technical sense. They often decline functionally during the rehab stay itself. In one cohort of patients seventy-five and older admitted to a geriatric rehabilitation unit, roughly one in six measurably lost function between admission and discharge, leaving in worse shape than they arrived. UnitedHealth’s nH Predict almost certainly identified a real pattern when it flagged Lokken’s case for early termination.

While this doesn’t excuse the denial, it changes what the denial is a denial of. And notice what the denial could and could not do. If rehab really was the wrong setting, the humane response would have been to move Gene Lokken somewhere better: home, with paid support, or hospice alongside comfort care. The algorithm had no power to do any of that. Its only lever was the checkbook. So even granting that it read the clinical picture correctly, the one action available to it was financial. It could stop paying. It could not move him to a better setting, and it did not. He stayed exactly where he was, now on his family’s dime. Being right and doing good came apart completely. That distinction matters, because it points to a deeper problem than any of the available villains can account for, and to a slower fix than the lawsuit Lokken’s family is rightly pursuing will produce.

The algorithm did not create the harm

A skilled nursing facility was almost certainly not the right place for Gene Lokken to spend the last year of his life. The trouble is that, given the structure of Medicare and the structure of the post-acute care market in the United States, an SNF was very plausibly his family’s only option that anyone was willing to pay for.

Custodial home care, the level of care most elderly people actually need at the end of life, is not covered by Medicare.

Hospice, which would have covered comfort-focused care at home, requires that the patient surrender further curative treatment for the same diagnosis, and Medicare regulations prohibit the use of hospice and SNF benefits simultaneously for the same condition. Hospice also typically covers only 4-8 hours of care per day, which is much less than working families need to care for a terminally ill loved one.

Hospitalization, the alternative covered by Medicare without restriction, is one of the most dangerous places an elderly person can be: roughly a third of hospitalized patients over seventy acquire a new disability during the stay that they did not have when admitted, often even when the illness that put them in the hospital is successfully treated. To even qualify for the SNF coverage Lokken received, his family first had to clear Medicare’s three-day rule, which requires a three-day inpatient hospitalization before SNF benefits trigger at all.

What we end up with is a complex web that captured Gene Lokken more effectively than any intentionally built net could have: An elderly patient with a fracture cannot receive paid-for care at home. He cannot combine comfort-oriented hospice with rehabilitation. He must be hospitalized for three days to qualify for the SNF that is his only post-acute facility option, and he will likely leave that hospital with new disabilities he did not arrive with. The structure offers, for most families, exactly one realistic post-acute pathway, and that pathway is the one nH Predict’s data correctly indicated was not, in the strict clinical sense, where the patient needed to be.

The nH Predict algorithm answered one question: does continued rehabilitation clinically benefit this patient? It answered correctly. But the structure had quietly made that question stand in for a different one: should anyone keep paying to care for this man at all? Those are not the same question. A frail ninety-one-year-old can be past the reach of rehabilitation and still need round-the-clock care that someone has to fund. By collapsing the second question into the first, the structure let a valid clinical finding settle a question it does not actually answer. The algorithm was right about rehab. It was never asked the question that mattered.

The obvious objection is that no insurer should fund care that does not work. That objection is right. The harder thought is that no humane society should let an elderly man with a broken leg discover that the only paid-for care available is the kind that does not work. Both are true at once. Medicare is structured so that the second truth has no leverage on the first.

This is not the algorithm’s failure. The algorithm read the structure correctly. The structure is the failure.

Structural hobbling and rigidity cascades

In the academic world, I write on a pattern I call structural hobbling: the constellation of well-meaning, procedurally legitimate rules that, in the aggregate, produce outcomes no one designed and no one is positioned to fix. Jennifer Pahlka, whose work I leaned on in last week’s post, calls a related dynamic the cascade of rigidity: policy intent, however flexible at the top, becomes more rigid at every step of the bureaucratic descent until the person who actually has to act, whether the discharge planner, the SNF administrator, the family member, or the algorithm, has no authority to recognize a misfit.

Both patterns are visible in the bind that killed Gene Lokken’s options. The three-day rule was written in the 1960s, when the structure of post-acute care looked entirely different than it does today; it has survived because rescinding it would require Congress to do something rather than the absence of action. The hospice-and-SNF exclusion was written to prevent double-billing, which is a reasonable thing to prevent; the people who designed it did not intend to force families to choose between rehabilitation and comfort care for a dying parent, but in operational reality that is what it does. The absence of custodial home care coverage in Medicare reflects the original statutory line between “medical” and “non-medical” services, a line that made sense in the political environment of its drafting and makes no sense in the clinical environment of a twenty-first-century aging population.

Every one of these rules has a defensible origin story. Every one of them is followed conscientiously by people doing their jobs. The aggregate effect of all of them, on the family of a ninety-one-year-old man with a broken leg, is that the only humane option is one the algorithm correctly identifies as clinically inappropriate. The cascade does not produce this outcome through any single bad decision. It produces it through the patient summation of defensible ones.

Four dimensions on a single case

In my public work I talk about four-dimensional thinking (4D for short). It’s a discipline for refusing the false binaries that political and policy conversation keeps trying to force onto situations that have more sides than that. Lokken’s case is a good one for it.

The first dimension is where you are standing. Gene Lokken was ninety-one, recovering from a fracture, frail, in nursing care his physicians had asked for and his family was paying for after his coverage ended. From where he was standing, the question was not “rehab or not” in the abstract. It was “rehab or hospital readmission or inadequate home support or a hospice transition his family had not been counseled toward and probably did not know was on the table.” Each option had a different sticker price, a different probability of survival at one year, a different probability of returning to the hospital, and a different probability of dying with people who loved him in the room.

The second dimension is the breadth of the social world the decision sits inside. The same decision looked different to everyone touching it: a family juggling jobs and a crash course in healthcare bureaucracy; a discharge planner inside a workflow built to move patients out on Medicare’s clock; an SNF administrator inside a reimbursement structure that pays more for rehabilitation than for hospice referral (and research suggests SNFs are reluctant to refer dying residents to hospice partly because it cuts their revenue); physicians trained to default to cure because cure is what the system pays for; and an algorithm contracted to flag patients unlikely to benefit from more rehab. The next section returns to these actors one at a time.

The third dimension is the pits and peaks, the actual tradeoffs, which do not line up the way intuition expects. Comfort and continued life are not the same thing; surviving longer in a setting that produces hospital-acquired disability is not always better than surviving less long in your own bed. The clinical literature now talks routinely about rehab and death, the pattern in which elderly patients are rehabilitated up to and through their last days because the structure does not support transitioning them to comfort care. Every actor in Lokken’s case was navigating those tradeoffs without the vocabulary or the incentives to surface them honestly.

The fourth dimension is time. The question for Gene Lokken in the spring of 2022 was not the question for Gene Lokken in the spring of 2023. The question for the Lokken family is not the question for the next family in their situation. The question for Medicare in 1965, when the post-acute care system was being built, is not the question for Medicare today, with a population whose median life expectancy is more than a decade longer and whose end-of-life trajectory is differently shaped. Each of these time horizons points to a different decision. The system is calibrated to almost none of them.

In four dimensions, the situation does not have a right answer that anyone in it failed to find. It has a structural shape that makes the right answer almost unreachable, regardless of the integrity, training, or judgment of any individual inside the structure.

Three goods: no way to honor all three

The four-dimensional view helps us see what was true about the case from multiple angles. It does not yet name why each actor in the case kept making the decision they made. To get to that, you have to name the three competing goods that were live in every step of Gene Lokken’s discharge.

The first good is getting it right in the clinical sense. Ninety-one-year-olds with Lokken’s profile rarely benefit from rehabilitation. The algorithm’s job, on its own terms, was to identify exactly that. From this angle the algorithm was honoring a real good. Refusing to put frail elderly people through treatment that will not help and may harm them is a moral commitment, and it is the commitment geriatricians have been arguing for in the medical literature for decades.

The second good is honoring the expectations of patients and the people who treat them. When a patient’s own physician recommends a course of care, the patient and the family build their planning around the assumption that the care is necessary and that the system will pay for it. Neither of those things may be true or justifiable in the cosmic or even macro-moral sense, but expectations matter. Overriding that recommendation in software, without conversation, with no human reachable to push back, breaks an implicit contract that runs deeper than what any insurance policy spells out. From this angle the algorithm was failing a real good, the one the physician was honoring by making the recommendation in the first place.

The third good is making the most humane choice available given the system you actually have. The discharge planner, the SNF administrator, and Lokken’s family were not choosing between rehab and a better option. They were choosing between rehab and repeat hospitalization, between rehab and inadequate home support, between rehab and a hospice transition the family may not have been able to manage or have been counseled toward. From this angle the SNF stay was the most humane realistic path even if it wasn’t a terribly good one, and cutting it off was an attack on the only working option the family had.

These three goods are all legitimate. None of them is a mask for self-interest (at the very least it’s complicated). Clinical correctness is what good geriatric medicine looks like. Honoring expectations is what good doctor-patient relationships look like. Choosing the most humane option under constraint is what conscientious caregivers and discharge planners do every day.

The problem is that the system Gene Lokken was inside made it impossible to honor all three at once. Every actor was honoring the good closest to their role, and the aggregate outcome honored none of them. The algorithm got the clinical question right and the relational question grotesquely wrong. The physician got the relational question right and was clinically constrained by what the system would pay for. The family got the humane-choice question as right as they could and paid for it out of money they had hoped to leave to grandchildren. The system as a whole produced an outcome that no one in it could defend.

This is what the combination of structural hobbling and the cascade of rigidity produces inside an actual human life. It is not bad people choosing wrongly among clear options. It is good people choosing rightly among options each of which sacrifices real good(s). The structure is the only actor in the room with the standing to make a different choice, and the structure does not have a face you can address.

Lokken’s case indicts much more than the algorithm

This is the part of the argument that the lawsuit cannot reach, and that the encyclical I wrote about last week is in fact pointing toward when it talks about subsidiarity and the design tests for institutional accountability. In fact, it points to the reality that the principles of the encyclical go far beyond AI. The problem in Gene Lokken’s case is not that an algorithm overrode a doctor. The problem is that the institutional structure inside which both the algorithm and the doctor were operating produces the same outcome under any allocation of authority that respects the rules. Give the doctor full authority and the doctor is still constrained by what Medicare will pay for. Give the discharge planner full authority and the discharge planner is still constrained by SNF capacity and reimbursement structures. Give the family full authority and the family still runs out of money in three months. Replace the algorithm with a human reviewer and you get a human reviewer making the same call slightly more slowly, with no more authority to recommend anything better.

This is the pattern that physicians caught in prior authorization, teachers caught in IEP compliance, social workers caught in CPS protocols, and front-line workers in nearly every American institution will recognize on first read. Good people. Defensible rules. Structurally produced harm. Pahlka’s career is built on the observation that the answer to this is not to find a better person or write a better algorithm. The answer is to reach back into the architecture that produces the bind and rebuild it, slowly, with the people who actually have to live inside it as partners in the redesign. But (sadly) that’s not how American institutional reform usually works. It is how it would have to work for Gene Lokken’s grand-niece, in 2055, to face a fundamentally different choice than his family did in 2022.

I’ve spent most of my academic career on the cluster of questions this case sits inside. My book The Medicalization of Birth and Death (Johns Hopkins University Press, 2019) argues that we have made medical settings the default home for the first and last weeks of American life, and that the structure of insurance, professional licensure, and liability has made the alternatives nearly impossible to choose. A bioethics article I just shipped makes the structural-injustice version of the same argument: the four principles of medical ethics, autonomy, beneficence, nonmaleficence, and justice, are taught and applied as if they were independent inputs that a thoughtful clinician can balance.

They are not.

The structure forces tradeoffs that consume all of them simultaneously, and good ethical reasoning inside a broken structure produces broken outcomes regardless of how careful the reasoning is.

That is the harder claim, and it is the one this case has been making to me since I first read about it. In my bioethics work I keep returning to a figure I call Joseph, a ninety-one-year-old who wants to die at home and cannot find a way to do so because the web of regulations and rules makes the most simple and most justified request impossible. Gene Lokken is one more Joseph, and there will be others. The bind was not built by the algorithm, or the clinician, or the family. It was built by the structure of the system and the structure is often the thing we avoid talking about.

Structural vs. process reform: No villains, just rebuilding

Last week I argued that the encyclical’s extension of subsidiarity to private digital power was the move that mattered, because it gave us a vocabulary for naming what is wrong with bureaucracies whose decisions cannot be reached. I stand by that, but now I want to add to it.

The vocabulary of subsidiarity, of human-scale accountability, of legibility and reachability, is necessary. It is not sufficient.

There is no version of the Lokken case in which putting a human on the appeal line, or making the algorithm legible to the family, would have produced a humane outcome.

The humane outcome required, among other things, that custodial care at home be paid for, that hospice be available alongside continued rehabilitation or that hospice covered more than four hours of care a day, that the three-day hospitalization rule not exist in its current form, and that SNFs have palliative training and incentive structures that don’t penalize them for transitioning dying residents to comfort care. Each of these is a structural reform that has to be won at the level of Medicare statute, agency rulemaking, professional training, and reimbursement design.

This is slower work than naming a villain, and not a substitute for it. The villain account is satisfying because it points at someone we can demand a different decision from. It’s also deeply satisfying to blame Big Tech because they’re a villain everyone agrees to villainize. The structural account is harder because the people we would have to demand different decisions from are, in the relevant sense, all of us, slowly, over the course of years, in the rule-making processes and professional bodies and statute revisions that almost no one is paying attention to. High-profile litigation is often the precondition for the harder structural fix, and my quarrel is not with naming a villain but with stopping there and with treating the verdict as the end of the work rather than the moment that makes the rest of it possible.

I have been making this argument in academic venues for a decade. It is the argument most likely to get a piece rejected from the journals that publish on AI ethics, on the grounds that it’s “not about AI.” It is also the argument most likely to be correct. The encyclical does not say it in this form, but the encyclical’s deeper claim, that the moral character of an institution is in its design choices and not only in its outputs, points the same direction.

Gene, and the families we have not yet failed

Gene Lokken’s family is in litigation against UnitedHealth, and they should win. They will win on a narrower (and in some ways inaccurate) claim than the one I have been making here. They will win on the claim that an algorithm should not override a treating physician’s judgment, which is mostly true, and which is the claim the lawsuit is best positioned to vindicate. The claim I am making, that the structure inside which the algorithm and the physician were both operating produces the wrong answer either way, is the claim that the lawsuit cannot vindicate and that will not be vindicated by any single court decision.

This isn’t a counsel of despair. It’s a counsel of where the actual work is. The structural reform of Medicare’s post-acute care provisions, of the hospice and SNF benefit boundary, of the three-day rule, of custodial care coverage, of palliative training in nursing facilities, and of the reimbursement structures that make all of the above durable, is exactly the kind of work that civic institutions, scholarly research, professional associations, and the slow grind of bureaucratic reform are positioned to do. None of it will produce a headline, but at least some of it would be transformational for the next ten thousand families in Gene Lokken’s situation.

The algorithm did not kill Gene Lokken. The structure inside which the algorithm and his family and his physicians were all making defensible decisions did. That is the worse story. It is also the true one.

Next week I’ll walk through what trustworthy institutions actually take, and why the strategy of telling Americans to trust more is the institutional version of victim-blaming. But for now, I’ll leave some links to additional reading in the Notes and Citations section below. And before you get too engrossed in that work, leave me a comment below!

---

Notes and citations

• On the previous post: “An Algorithm Denied Gene Lokken’s Care. The Pope Just Named the Pattern.” — link to be added at publish.

• On the pattern of rehabilitation up to and through end of life, see Singh et al., “Rehab and Death: Improving End-of-Life Care for Medicare Skilled Nursing Facility Beneficiaries” (Journal of the American Geriatrics Society, 2026), and the related work on rehabilitation therapy for nursing home residents at end-of-life.

• On functional decline during a geriatric rehabilitation stay, see “Functional decline in geriatric rehabilitation ward”. On hospitalization-associated disability in older adults more broadly, see Covinsky, Pierluissi, and Johnston, “Hospitalization-Associated Disability” (JAMA, 2011).

• On Medicare’s three-day rule for SNF coverage, see the CMS guidance. On Medicare’s general non-coverage of custodial / long-term care, see Medicare.gov on long-term care.

• On inappropriate SNF payments and quality concerns, see the HHS Office of Inspector General reports and the Center for Medicare Advocacy on opportunistic nursing home ownership.

• On the broader argument about medicalization of birth and death and its costs to patients and families, see my book The Medicalization of Birth and Death (Johns Hopkins University Press, 2019).

• On structural hobbling, see my article “Structural Hobbling: Regressive Harm, Diffuse Responsibility, and Structural Injustice” (Social Philosophy and Policy, 2025).

• On the cascade of rigidity, see Jennifer Pahlka’s Recoding America (2023) and her AI Meets the Cascade of Rigidity (Niskanen Center, 2024).

Your Turn

If you have been a family member navigating a parent’s post-acute care, you have lived inside the structure I am describing. What did you have to fight for that, in a sane system, should have been the default? Drop it in the comments. I read them.