Rotten Cherry Picking

According to Idiom Origins, “cherry picker” was originally a British nautical expression from the late 19th century for an inferior seaman who would pick the easiest tasks on the ship, with an allusion to picking the best fruit from a bowl of cherries. 

This definition is interesting because we now have two different aspects of the problems we have in science: 1) it is done by inferior scientists (or scientific organizations); and 2) choosing only those papers that support a preferred outcome.

The problem with the term “cherry picking” for science is that picking out the good cherries makes sense. It is picking out the rotten cherries that leads us astray. Picking out good cherries that establish causation instead of correlation is a good example. Correlation studies are useful, but only as suggestions for things that need to be investigated further to find causation.

Of course, it’s not just correlation, we also need to eliminate poor studies. In 2015, Richard Horton, editor of the science magazine Lancet, said,

 “The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.”

That’s the literature but what about the inferior seamen? In the case of tort law, this would refer to scientists who offer unsupported opinions, rely on correlation or junk science that should not be used as evidence. To prevent this, judges are supposed to act as “gatekeepers” (Federal Rule of Evidence 702) to keep out weak or biased scientists (inferior seamen) who don’t know the subject, offer their opinion without evidence, or offer biased evidence. These are the rotten cherries of trials. This may include “expert witnesses” who routinely testifyfor plaintiffs or defendants. One professor of family medicine, who has testified in more than 600 cases, described his testimony as having“the potential to save millions,” i.e., he was on a mission.

Why do scientists and science-based organizations rotten cherry pick? The answer that most economists give is incentives. For an academic scientist, there is need for both funding and tenure and they are interrelated. Funding sources, whether from industry, nonprofits, or government, are too often looking for specific answers that match their needs or beliefs. In order to get or keep funding flowing, scientists (and others) must “rotten cherry pick” data and models in order to come up with the “desired” answers. With funding and publications, they are more likely to get tenure. 

Regulatory agencies fund the science that they need to continue to pass regulations. That means they must find scientists willing to discover new health and safety issues. 

Of course, we also see this kind of biased reporting from political actors and news organizations. For example, both reporters and politicians appeal to listeners and voters using confirmation bias. This is a well-known concept from behavioral economics that suggests that people choose to pay attention to and believe information that conforms to their existing thinking and preconceptions, which is often rotten cherry picking.

Wherever it occurs, rotten cherry picking should be a practice that is called out and condemned as it leads to poor policies and an unhealthy and uninformed society.