Experiment on Medicare auctions

Charlie Plott and two of his students have a recent QJE paper that employs an experiment to make a point about market design. They study the Medicare procurement auction that has been criticized by Peter Cramton: see my previous posts here, here and here.

 The CMS Auction: Experimental Studies of a Median-Bid Procurement Auction with Nonbinding Bids Brian Merlob Charles R. Plott Yuanjun Zhang
Quarterly Journal of Economics Volume 127, Issue 2Pp. 793-827

 Abstract We report on the experimental results of simple auctions with (i) a median-bid pricing rule and (ii) nonbinding bids (winning bids can be withdrawn)—the two central pillars of the competitive bidding program designed by the Centers for Medicare and Medicaid Services (CMS). Comparisons between the performance of the CMS auction and the performance of the excluded-bid auction reveal the problematic nature of the CMS auction. The CMS auction fails to generate competitive prices of goods and fails to satisfy demand. In all proposed efficiency measures, we find the excluded-bid auction significantly outperforms the CMS auction

Here's a brief news story about the article: Medicare auction will face severe difficulties, research shows 


"Unlike standard auctions, however, the CMS auction was designed with two unorthodox rules. First, the eventual selling price is set at the median of all of the winning bids. Second, bids are nonbinding, so companies can change their mind once the prices are set.

"Critics say that these rules cultivate a harmful bidding strategy. To ensure a winning bid, each company will make a very low offer; this carries no risk, because the companies can cancel their bid if the median price turns out to be too low. The result is that participants in the auction will tend to make low-ball bids, assuring that the median price will also be very low—so low, in fact, that few of the companies can actually afford it, leading them to cancel their offers. At the extreme, nothing is bought or sold and, Plott says, "the auction crashes. It's just not an effective auction."

"And what will happen then, critics warn, is that the government will end up negotiating prices with individual companies—negating the whole point of a competitive-bidding scheme in the first place. "You can see immediately from theoretical arguments that the potential for disaster is built right in the strategic structures," Plott says.

"To determine if such theoretical predictions translate into real-world behavior, it is essential to examine experimentally how people behave in an actual auction, says Plott—who, for the last 30 years at Caltech, has been developing experimental techniques to test economic theories. He posed the problem to his experimental economics (EC 160) class. Two of his students, Merlob and Zhang, took up the challenge and spent a year researching, designing, and conducting experiments to test that behavior.

"The team used computers at Caltech and the University of Maryland to run a simplified version of the CMS auction and several other auction types; one, for example, followed more standard rules, with binding bids and prices set at the lowest bid that did not win, instead of the median of all winning offers. Each auction involved 12 or 16 bidders (student volunteers from Caltech and the University of Maryland), who first had to pass a quiz showing that they understood how the auctions worked. The volunteers were given just one item to sell—a generic "thing" (since the bidders' behavior should be the same in a given auction type, regardless of the item being sold)—each at a different cost to them.

"The Caltech team also examined the effect of other auction features, such as whether the costs of each item for each bidder are public knowledge and the effect of charging bidders to participate.

"The results, the researchers say, convincingly support critics of the CMS auction design. "It's pretty disastrous what the bidders ended up doing," Zhang says. In the simulated CMS-type auction, some people bid $0, and the "government" was not able to buy all the items it needed. The experiments also showed that a standard auction is much more efficient and successful: the government was able to buy all the items it needed, and the bidders who had the lowest costs were the winners.

"Using this experimental approach, the researchers were able to pinpoint the fundamental problem of the CMS auction design: the two rules. "If you just get rid of one of those two rules, it doesn't help—you still have problems," Plott explains. "So you have to get rid of both of them."