Susan Athey on biggish data and machine learning

 Susan Athey is interviewd in JAMA:

How an Economist’s Application of Machine Learning to Target Nudges Applies to Precision Medicine   by Roy Perlis and Virginia Hunt  JAMA. Published online May 16, 2025. doi:10.1001/jama.2025.4497 

"A recent study by economist Susan Athey, PhD, and her colleagues may shed light on how best to target treatments using machine learning. The investigation, published in the Journal of Econometrics, focused on the effectiveness of text and email reminders, or nudges, sent to students about renewing their federal financial aid. The researchers compared causal targeting, which was based on estimates of which treatments would produce the highest effects, and predictive targeting, which was based on both low and high predicted probability of financial aid renewal.

"In the end, the study found hybrid models of the 2 methods proved most effective. However, the result that may be most surprising to Athey was that targeting students at the highest risk of nonrenewal was actually less effective.

...

"Dr Athey:When I first started working on this, I was like, “Oh, there’s going to be a gold mine. I’m going to go back and reanalyze all of these experiments that have already been run, and we’re going to be doing new scientific discoveries every day.” It didn’t quite work out that way. We had some big successes, but there has been a lot of lack of success.

What are the cases where this doesn’t work? Machine learning is using the data to learn about these treatment effects. You have to do a lot of sample splitting. There’s always a cost to using the data to discover the model. You can do it without sample splitting, but then you have to adjust your P values. There’s no free lunch. If you have a very small dataset, you probably know what the most important factors are. You might be better off prespecifying those and just doing your subgroup analysis. If [there are] hundreds of observations, it’s just unlikely. These techniques are too data hungry to work.

Generally, you need thousands of people in the experiment. Then more than that, the statistical power needed to get treatment effect heterogeneity is large. And even treatment effect heterogeneity is easier—trying to get differential targeting is another thing. Imagine you have 3 drugs. It’s hard enough to say that something works relative to nothing. If you’re trying to say that one drug works better than another drug where both work, that’s hard. Usually you need really large, expensive trials to do that.

Then you add on top of that that I want to say, “This drug is better for these people, and this other drug is better for these other people.” You need 10 times as much data as you would for the basic “is there a treatment effect at all?” Now, of course, sometimes there’s a genetic thing: this drug literally doesn’t work or it has this terrible side effect for some people. That will pop out of the data.

For more subtle effects, you do need larger studies. That’s really been the main impediment. And as an economist, it’s like, why are all these things just barely powered? Why are there so many clinical studies with a t-statistic of 2? Of course, people did the power calculations, and they had some data already when they planned the experiments. If you have more data, maybe you add another treatment arm or something else. You don’t actually overpower an experiment. In my own research, I’ve ended up running my own experiments that are designed to get heterogeneity. I’ve also had a lot of luck when there’s very big administrative datasets, and there’s a really good natural experiment. Then you have lots of data. But former clinical trials are selected to not be good because the researcher themself didn’t overpower their own experiment. That’s why this isn’t so useful.

But nonetheless, that’s not to say it’s not out there. Like in any discovery, if it’s going to save lives and money, it’s worth doing. It’s just that there’s not a whole bunch of low-hanging fruit. There’s no dollars lying on the sidewalk."

There's a gray market for diabetes supplies

 High prices and inadequate medical insurance force some patients to get their diabetes monitoring supplies second hand. Here's a suggestion for some redesign...

Medpage has the story:

The Diabetic Supplies Gray Market Is Ripe for Disruption by Jacob Murphy 

""CA$H 4 SEALED & UNEXPIRED DIABETIC TEST STRIPS -- CALL NOW."

This message, printed in black on a neon yellow poster board, hangs just outside the $1.1 billion Charlotte R. Bloomberg Children's Center at Johns Hopkins Hospital. The sign represents a peculiar offshoot of American healthcare: the diabetic supplies gray market. Here, blood glucose test strips are exchanged within an informal network.

This market operates through handwritten signs and websites like QuickCash4TestStrips.com, often flourishing in areas with high rates of poverty and uninsured patients. How does it work? Insured patients obtain excess test strips at little personal cost through insurance, then sell them to reseller companies. These companies profit by selling the strips below original retail prices, which far exceed manufacturing costs, to uninsured individuals. And, surprisingly, this is all legal.

...

"Diabetes costs have steadily risen to thousands of dollars annuallyopens in a new tab or window for individuals. These expenses can be even higher for the 1.5 million diabetic Americans without health insurance, especially if they experience diabetes-related complications. Without proper monitoring, these individuals face life-threatening risks of ketoacidosis, hypoglycemia, and long-term complications including vision loss, kidney failure, and amputations. In the end, they're left with a tough choice: turn to an unregulated "gray" market or potentially face major health complications and financial consequences. 

...

"[ academic medical centers] should establish formal redistribution programs that incentivize donations of excess diabetic supplies. These programs would provide safer alternatives to unregulated gray market exchanges, leveraging institutional scale to deliver essential supplies at minimal or no cost to those most in need.

"Incentives to donors could include copay waivers for downstream care, free diabetes check-ups such as eye and foot exams, or connections to food assistance programs. Conditional cash transfers could also serve as effective motivators, particularly for the lowest-income donors. Additionally, centers could help patients transition from test strips to continuous glucose monitors "

"Pocket listings" in residential real estate

 After recent antitrust rulings regarding residential real estate sales, big brokers are regrouping:

Yahoo finance has the story

The real estate world is fighting over secret listings and the future of how homes are sold

"Barely a year after the National Association of Realtors settled a lawsuit and rewrote the rules for how agents get paid, the powerful trade group is examining the fate of another policy that could change how homes are bought and sold across the country.

"The policy, known as Clear Cooperation, requires agents to list homes on shared databases known as multiple listing services (MLS) within one business day of beginning to market the properties. The rule is designed to cut down on what are known as “off-market” or “pocket” listings, where a home for sale is marketed semi-privately to small pools of potential buyers without being advertised widely on the MLS. 

"The NAR is currently reviewing whether the rule should be repealed, remain in place, or be changed.

...

"Where most real estate companies fall on the issue tends to align with whether or not they benefit from the rule. Executives at Zillow and Redfin, which aggregate home listings from the MLS, are for it. Compass, a luxury-focused brokerage that touts its access to “Private Exclusives,” is against it. Anywhere Real Estate, the parent company of franchises including Century 21, Coldwell Baker, and Sotheby's International Realty, has said it would like to see the rule remain but with changes.

Pocket listings are a relatively niche way to sell a home in most parts of the country. Data is scarce, but estimates generally put the deals at 5% or less of sales nationwide. Using deals that were listed as “sold” the same day they first appeared on the MLS as a proxy, Redfin estimates that they made up 1.8% of deals in mid-2024."

Notes from Messina (kidney exchange and the Accademia Peloritana dei Pericolanti)

 We traveled last week from Prague to Sicily, for the Matching in Practice workshop in  Messina, which was rich in kidney exchange. I was glad to reconnect with the Director of the National Transplant organization, Dr. Giuseppe Feltrin, and with Professors Antonio Nicolò and Antonio Miralles.

But a funny thing happened first, at the University of Messina. I was inducted into the university's Accademia Peloritana dei Pericolanti, founded in the early 1700's, at a time when autocrats didn't look fondly on universities (imagine that!). The symbol of the Academy is a ship sailing in the Strait of Messina (between Scylla and Charybdis) with the motto "Inter utramque viam periclitantes," "Taking risks between both paths," reflecting (I was told) the perils of navigating the strait between scholarship and politics.  It seemed very appropriate for the times.

Here's the story in the local news: with a picture:

 

Notes from Prague (kidney exchange, market design, and progress on a new book)

I flew back to California yesterday, after spending some time in Czechia and Italy talking about kidney exchange.  Here is a video of the public talk I gave at Prague Castle.  Among other things it highlights the Czech kidney exchanges with Israel. (I had the pleasure of meeting  Prof. Jiri Fronek, the distinguished surgical pioneer who led the Czech side of that effort.)

https://youtu.be/jrrlNWMkQyE?feature=shared

I also had the privilege of visiting CERGE-EI  (Center for Economic Research and Graduate Education - Economics Institute) where my host was Prof  Štěpán Jurajda.  He and I first met when we were both at the University of Pittsburgh in the 1990s.

And here's an interview with the Economic newspaper  Hospodářské noviny  that starts off with the optimistic notion that I may have just (largely) completed the draft of a new book:)

Zkoumá trhy, kde peníze nevládnou. Ledvinu ani lásku si za ně většinou nekoupíte, říká nobelista Alvin Roth [ He explores markets where money doesn't rule. You can't usually buy a kidney or love with it, says Nobel laureate Alvin Roth]

“Before flying from the USA to Prague, economist and Nobel laureate Alvin Roth managed to send the publisher a draft of his new book, which he is currently finishing. He calls it Controversial Markets. Between an afternoon lecture for students at the CERGE-EI Institute in Prague and an evening lecture at Prague Castle, he also found time for an interview with Hospodářské noviny, in which he outlines what his new book will be about. One of the controversial markets he deals with, for example, is the organ transplant market."

Wine barrels, barrel brokers, and tariffs

 Wine barrels are traded and reused internationally.

The Guardian has the story:

Stained, warped and terroir rich: the global and shockingly sustainable lives of wine barrels
Wood barrels circle the world and can be used for more than a century. They tell a story, but they’re imperiled by tariffs  by Kiki Aranita

"In the alcohol industry, when ageing liquor can easily take decades, the vessels that house them can also become more covetable over the years. In an age of disposable materials and dire news of plastics polluting our environment, reused wooden barrels exist in stark contrast. The lives of barrels are long, shockingly sustainable and currently imperiled by trade war.

"Many circumnavigate the globe and end their days in distilleries in remote corners of the world, originating in the forests of Hungary and moving from mountain towns in Canada to distilleries in the Caribbean and Mexico. At Hamilton, new American oak barrels hold fresh distillate, alongside the dinosaurs: French cognac barrels that show their age

...

“We think of barrels as teabags. It gets used first for bourbon, like the first steep of a teabag. You get a lot of color and flavor from the barrel quickly. If you use the barrel again, it’ll take longer to impart, so maybe it’s used for scotch, which sits and ages longer. You mute the barrel’s flavors along the way.”

...

"The international use of barrels is part and parcel of the global liquor industry. Large conglomerates like LVMH, Brown-Forman and Suntory have multiple spirits brands in their portfolios, and barrels make their rounds internally. A Kentucky-made barrel might end up in Scotland to age scotch because Brown-Forman owns both the Jack Daniel’s and Glendronach brands.

...

"Distilleries that aren’t owned by large conglomerates enlist the help of a barrel broker who can source unique barrels. Mara Smith sources old pinot noir barrels from France through a broker, as they give her Inspiro tequila a rosy hue and flavors like “berries, some nuttiness, a floral [aroma] on the nose”.

...

"Rizzo outlines a Laws Whiskey barrel’s typical lifespan. “We age our Four Grain Bourbon and send those used bourbon barrels [after four to 10 years] to a local apiary, Bee Squared, in Berthoud, Colorado. They age their honey in those used barrels for 90 days to produce a glorious local barrel-aged honey. We then get those barrels back and put more bourbon into them to make a natural honey-aged bourbon [which takes a year and a half of ageing]. One went to our friends at Lady Justice Brewing, who aged a honey bock beer in the honey barrel. Once that was finished in six months to two years, they put malted barley grain inside the barrel for another six months to two years to flavor the grain to produce another beer. And then the barrel is made into furniture.” These barrels had seven lives.

"Recent tariffs are making the very aspect of manufacturing barrels, distilling alcohol and selling its finished product more expensive. In American winemaking, particularly in California, French oak barrels will be affected by the EU’s retaliatory tariffs. Since barrels are essentially an ingredient in any given type of alcohol’s recipe, a winemaker would not be able to easily switch out a type of wood they have used previously for something available domestically."

Chronic diseases in the U.S.

 The top three chronic diseases--Hypertension, Obesity, and Diabetes--all contribute to the fourth, Kidney Disease.

How Chronic Disease Became the Biggest Scourge in American Health
Americans live shorter and sicker lives than people in other high-income countries   By  Brianna Abbott  | Graphics by  Josh Ulick