Mini Post #11: Water Under the Bridge

Does forgiving people's medical debts help their finances?

I’ve been, for around three months, writing something different: a shorter, narrower post focusing on one specific study. Last week, I wrote about how the Catholic Church chooses its saints. This is the tab with all the previous posts.

Onto the actual question of interest for the post: does forgiving people's medical debts help their future finances?

---

banger cap inscription ngl

“Debt forgiveness” is a major political issue in the United States, since wiping out college loans was a campaign promise by President Joe Biden which he eventually implemented - after some legal issues. This sparked debate on the moral and economic merits of the policy, such as its economic benefits, as well as whether it was a reasonable use of government money or whether a problem existed at all.

In the middle of all of this, some people (namely on Twitter dot com) demanded to cancel medical debt too/instead. The problem is fairly large. two in five (40%) of Americans have some medical debt, and the median amount is around $2,500 dollars. Almost two thirds of debt holders report reducing expenditures on clothing, food, and half even mention wiping out their savings. In response, around 15 local and state US governments rolled out programs that have funded around USD 8 billion in medical debt relief, and five other localities were considering a further 5 billion in programs. Private donors, meanwhile, have eliminated around 10 billion to date.

Does this work? And in what areas? The paper in question is “The Effects of Medical Debt Relief: Evidence from Two Randomized Experiments” (2024) by Raymond Kluender, Neale Mahoney, Francis Wong, and Wesley Yin. The question seems fairly straightforward - less debt would obviously be better for financial health, and “moral hazard” considerations would be kind of irrelevant considering that the debt resulted from healthcare expenses. Or would it?

The authors utilize a simple framework: to study the effects of medical debt forgiveness on financial outcomes, health, and healthcare use they use two randomized experiments in partnership with RIP Medical Debt, a non profit that buys and cancels medical debt. The cancelled debt in question belonged to 83,401 patients, and has a face value of $169 million.

The study conducted two separate experiments: firstly, one for debt that hadn’t been sold by hospitals to collections agencies¹ yet (called hospital debt). The second study concerns people whose debts are in the hands of debt collectors (called, unsurprisingly, collector debt). The two experiments are fairly similar, and were conducted between March 2018 and October 2020. In both cases, debtors were divided between a treatment group (aka, the people who had their debt completely forgiven) and a control group (those who got bupkis). To ensure a balanced composition of both groups, they were stratified by location, amount of debt, insurance status, and a collections score - to avoid the effects being driven by one group being substantially different than the other. Individuals who has their debt forgiven were informed three weeks after, and reminded of this six weeks later.

For both experiments, the debt collector provided the researchers with a dataset including the amount owed, information on the debtor (name, age, address, and phone number) and limited information on the medical services they received. For people in the hospital debt experiment, they also received health insurance information. Additionally, they obtained credit score and overall credit and debt data from credit bureau information, as well as a representative random sample of 58,669 people to contextualize the findings in the broader US. Some of the data on the experiments’ sample’s credit was not usable due to incongruities with other data sources for the study. Finally, the University of Chicago conducted a multimodal study of the sample group to collect information on mental and physical health, healthcare use, and financial wellbeing. This study was sent as a survey to a subset of the sample of 14,922 individuals (a number that was selected due to a lack of funding), of which 19.4% responded (68% through web surveys sent to their email).

The average person in the sample is a woman in her 40s, and has a 43.7% chance of being White, 30.9% chance of being Hispanic, and 18.8% chance of being Black - which means they are disproportionately likely to be non-white compared to the average American. In addition, they are also more likely to be low income, but less likely to be elderly, due to the availability of health insurance programs for that group specifically. The sample’s credit scores are lower than average - compared to the national population, it is at the 20th percentile (that is, the lowest 20%). This is particularly because approximately 63% of the sample has medical debt in their credit report, versus 18% of the total population.

The sample was examined and was truly randomly assigned, with balance in both demographics and financial conditions on any of the data sources. However, because neither experiment had treatment be actually randomly assigned to the participants (to ensure balance in sample and avoid skewing the results) the effects estimates contain fixed effects for person-level characteristics. Both experiments also conduct heterogeneity analysis by examining individuals across four baseline characteristics: elegible medical debt, age, time span between medical debt and experimentation, and the amount of other, non-elegible debt. These are divided between quartiles and examined in particular.

For the hospital debt experiment, the total sample consists of 75,873 people owing $103 million of debt at face value. The treatment group was 18.9% of the sample (that is, 14,377 people) and they received relief amounting to $19 million in value, or $1.0 million in cost to RIP Medical Debt. The average person in this sample owed $1352 dollars. Additionally, a sub-experiment was conducted where a random sub-sample of the treatment group (4,232 people out of the 8,160 treated individuals) received additional, phone-based outreach to raise the salience of the intervention - only 19% of this group reported seeing the initial letter.

Now, to the results. The treated hospital debt experiment group showed no statistically significant differences with the control group on the number of accounts past due (that is, debt), as well as on any other number of measures of financial distress (accounts in default, debt delinquency, dollar value of balances past due and in default, debt sent to collections, and bankruptcy). There is also no effect detected on credit access, their credit score if they have access to credit, and their combined credit card limit, as well as on credit card and auto loan borrowing.

People in the treatment group recall having their debt rleieved at three times the rate of the overall treatment group (24.2% versus 8.1%) and reported having twice as much debt relieved as the rest of the control group. However, the sub-sample of the treatment group that was phoned to remind them of the intervention saw no statistically significant changes in the quality of their recollection.

For the collector debt experiment, the total sample consists of 137,038 people where 69,024 received treatment (50.4% of the sample), whose debt had a total face value of $150 million, with an average of $2,167 per person, but which was purchased at less than 1% of said value (so around $1.5 million). The accounts were also grouped by location, age of debt, person’s age, and amount of debt to ensure a balanced sample.

In terms of results, they’re broadly similar to the hospital debt sample: there is no statistically significant effect of medical debt forgiveness on any measure of financial distress or on the converse, financial wellbeing.

One aspect of the experiment is that, unrelatedly to the study, credit bureaus stopped including medical debt in credit reports. For a segment of the sample, however, this resulted in effects on their credit scores separate from this policy change, since they had preexisting debt that was relieved and included in the reports. For this group, medical debt relief reduced the count of debt in collections by 0.98% at high levels of significance (or, in dollar amounts, $1,206 - 29% of the control mean amount). The number of people with credit scores decreases by 4.2% after relief, pointing that these people would not be scored otherwise. For individuals in this group that have credit scores in all periods, their score is raised by 3.6 points, a very small amount. This, however, increases their access to credit by raising their credit limits by 8% immediately after, and 15% a year afterwards. These effects are all highly significant.

One unexpected result is that medical debt relief caused a 1% increase in the probability of having future unpaid medical debt, with adequate statistical significance. The explanation is that this could stem from either lower payment of already existing debt, or from greater utilization of future medical care - but controlling for whether the relevant medical service ocurred before or after the intervention, the authors find that the increase in future unpaid bills is caused by reduced payment of preexisting, non-cancelled debts.

In terms of mental health outcomes, there was no significant changes in self-reported feelings of depression, and moderate levels of depression increased by statistically insignificant levels. There is also no effect on depression in the lower 75% of debt holders; however, the most indebted quartile reported a large and very statistically significant increase of 12.4% in feelings of depression. These effects are mirrored for other mental health conditions, such as anxiety and stress: non-significant effects in basically any direction, insignificant increases in moderate rates of stress and anxiety, and a large rise of mental unwellness among the most indebted individuals. Examining wellbeing, there is a similar trend for happiness and self-reported general health, although the top 25% of medical debt havers show a moderately significant increase in positive feelings. In addition, the individuals who received phone calls reminding them of their medical debt cancellation had worse mental health outcomes than the overall treatment group, which was statistically signficiant. This is true for depression and anxiety, but less clear for stress or wellbeing.

Feelings of financial distress, in addition, did not improve either - there was a small but non significant increase in odds of having trouble paying other bills, and no statistically significant change on self-reported increases in borrowing or cutbacks in spending. This is true across all four quartiles.

Finally, as it pertains to healthcare use, there are no significant effects either. There is a non-statistically significant drop in the chances of using necessary healthcare, and a similar non-significant drop in the odds of receiving necessary prescriptions. There was a small positive impact on the most indebted quartile, but the data is not precise enough to make any rigorous statements.

Put together, this points to extremely moderate effects: no detectable changes in financial status, wellbeing, or distress, as well as no changes in healthcare conditions. There were small improvements in credit conditions for people whose forgiven debt showed up in their credit reports, a now-moot impact. Additionally and counterintuitively, the key effects were small reductions in payments of medical bills and worse mental health outcomes for the most indebted individuals and for those who were reminded of their indebtedness. The explanation by the authors for the first issue is that the recipients of relief reduce payment rates by mis-estimating the probability that more of their debts will be forgiven, or alternatively, that they have a certain amount of debt that they will (not) pay, and relief simply causes them to respond according to those mental amounts. Secondly, it is probable that the intervention reducing mental and subjective wellbeing by raising the salience of financial troubles, or by raising the salience of the act of receiving charity - which many consider humilliating in some ways - particularly when “feeling like a failure” was the most heavily affected negative statement.

The results are considered quite surprising, especially since previous, similar experiments for other kinds of debts yielded largely positive results. One technical issue pointed by the authors is that debt relief programs purchase the cheapest debt available (13 billion in forgiveness costs around 137 million in funding, so around 10.5 cents for each dollar), which is also the least likely to be recouped. Because of “assets 101”, the lower the value of debt, the less likely it is considered plausible to collect on this. If the study had targeted more expensive debt, it is possible that there would have been positive results, since this debt does have more active and thorough attempts at collecting and might impact less overall deprived individuals.

---

To finish the post, some links

• The paper in question

• This week’s post, about the efficacy and ethics of randomized control trials

• A J-PAL write up of the paper

• A Vox article by Dylan Scott about the study and its implications

• A Urban Institute post about the decision to erase medical debt from credit scores

• A 2011 paper by Finkelstein et al. on whether randomly receiving health insurance improves healthcare conditions, as well as an overall list of results from other papers using the same healthcare lottery

• A similar 2017 paper by Will Dobbie & Jae Song about credit card relief

• A similar 2020 paper by Di Maggio, Kalda, and Yao about student debt relief

1. As a note to non-Americans, when an American has medical debt they cannot pay, after a certain period of time it gets sold as an asset from the hospital to a collections agency, who then try to secure payment by, more or less, harassing the debtor into compliance. Their methods are frequently quesitonable, and sometimes even abusive and coercive.