Problems With Scientific Research: How Much Of it Is Just Wrong?

Researchers headed into their studies wanting certain results—and, lo and behold, they were getting them. We think of the scientific process as being objective, rigorous, and even ruthless in separating out what is true from what we merely wish to be true, but in fact it’s easy to manipulate results, even unintentionally or unconsciously. “At every step in the process, there is room to distort results, a way to make a stronger claim or to select what is going to be concluded,” says Ioannidis. “There is an intellectual conflict of interest that pressures researchers to find whatever it is that is most likely to get them funded.”

...as much as 90 percent of the published medical information that doctors rely on is flawed...the field of medical research is so pervasively flawed, and so riddled with conflicts of interest, that it might be chronically resistant to change—or even to publicly admitting that there’s a problem...an obsession with winning funding has gone a long way toward weakening the reliability of medical research. _theatlantic

In the opinion of Al Fin, those assertions are largely true. But not just for medical research. For all scientific research where funding is influenced by big money interests -- whether corporations, governments, or "philanthropic" organisations -- the issue of conscious and sub-conscious bias must be met head-on.

The problem is particularly acute in the field of climate science -- where $trillions are at stake. The severity of the problem was highlighted recently by the resignation of renowned and respected physicist Hal Lewis from the American Physical Society. Here is why Lewis resigned from the formerly prestigious society:

the money flood has become the raison d’être of much physics research, the vital sustenance of much more, and it provides the support for untold numbers of professional jobs. For reasons that will soon become clear my former pride at being an APS Fellow all these years has been turned into shame, and I am forced, with no pleasure at all, to offer you my resignation from the Society.

It is of course, the global warming scam, with the (literally) trillions of dollars driving it, that has corrupted so many scientists, and has carried APS before it like a rogue wave. It is the greatest and most successful pseudoscientific fraud I have seen in my long life as a physicist. Anyone who has the faintest doubt that this is so should force himself to read the ClimateGate documents, which lay it bare. (Montford’s book organizes the facts very well.) I don’t believe that any real physicist, nay scientist, can read that stuff without revulsion. I would almost make that revulsion a definition of the word scientist. _WUWT

In fact, the gut instinct of revulsion to mercenary science -- science corrupted by funding bias and influence seeking -- is clearly a better gauge of a true scientist than any degree of technical sophistication of research tools or supercomputer simulations. And yet climatology -- which is a newborn infant science in comparison to medical research -- may never escape the trap of politicised funding and publication selection in which it is snared.
Medical science is often biased at all stages by the desire of researchers and biomedical corporations for monetary gain. But the sheer amount of money at stake in climate science -- $trillions, as mentioned earlier -- dwarfs any profits ever to be gained by drug companies, medical insurance companies, or medical device companies. The gargantuan political influence combined with the relative tiny size of the climate science community, combines to provide the public with a thoroughly corrupted, biased, and unreliable look at global climate.

How many other areas of science share the same weaknesses? Follow the money. Big money interests (including governments) that get too deeply involved in the funding and publication process for any science, will introduce a corrupting influence on that science.

The dumbing down of schools from K-University does not help matters any. The replacement of meritocracy in the selection process for some graduate programs by affirmative action and other anti-meritocratic tools of political correctness, further degrades the grand enterprise of dispassionate scientific research.

All humans have their biases, which must be faced at every level of discovery. If human biases are ignored, or submerged beneath an overlying layer of "consensus" or political correctness, the end result of such an enterprise will be foul excrement.

Perhaps only a minority of researchers were succumbing to this bias, but their distorted findings were having an outsize effect on published research. To get funding and tenured positions, and often merely to stay afloat, researchers have to get their work published in well-regarded journals, where rejection rates can climb above 90 percent. Not surprisingly, the studies that tend to make the grade are those with eye-catching findings. But while coming up with eye-catching theories is relatively easy, getting reality to bear them out is another matter. The great majority collapse under the weight of contradictory data when studied rigorously. Imagine, though, that five different research teams test an interesting theory that’s making the rounds, and four of the groups correctly prove the idea false, while the one less cautious group incorrectly “proves” it true through some combination of error, fluke, and clever selection of data. Guess whose findings your doctor ends up reading about in the journal, and you end up hearing about on the evening news? _theatlantic

PLOS Medicine 2005 Why Most Published Research Findings are False

PLOS Medicine 2008 Why Current Publication Practices May Distort Science  

More: (via SDA_via_Maggie'sFarm)

1. 
   Whenever possible there must be independent confirmation of the “facts”
2. 
   Encourage substantive debate on the “evidence” by knowledgable proponents of all points of view.
3. 
   Arguments from authority carry little weight as “authorities” have made mistakes in the past. They will do so again in the future. Perhaps a better way to say it is that there are no authorities; at most; there are “experts”.
4. 
   Spin a variety of hypotheses. If there’s something to be explained, think of all the different ways in which it could be explained. Then think of tests by which you might systematically disprove each. The ones that survive are the ones to do in depth study on.
5. 
   Do not become attached to any hypothesis just because it’s yours. Find reasons for rejecting all, including your own, hypothesis.
6. 
   Quantify. If whatever you are explaining has a measure, quantify it so that measurement is more possible.
7. Vague hypothesis, or those difficult to quantify will be the most difficult to prove or disprove.
8. 
   If there is a chain argument, then each and every link must work, including the premise.
9. 
   Use Occam’s Razor; which is to choose the hypothesis that explains the data in the simplest terms.
10. 
     Ask: is the Hypothesis testable and falsifiable. Hypothesis that are not testable are not worth much. Could you duplicate accurately, at least theoretically, the hypothesis?

__Carl Sagan