18 October 2010

Problems With Scientific Research: How Much Of it Is Just Wrong?

Researchers headed into their studies wanting certain results—and, lo and behold, they were getting them. We think of the scientific process as being objective, rigorous, and even ruthless in separating out what is true from what we merely wish to be true, but in fact it’s easy to manipulate results, even unintentionally or unconsciously. “At every step in the process, there is room to distort results, a way to make a stronger claim or to select what is going to be concluded,” says Ioannidis. “There is an intellectual conflict of interest that pressures researchers to find whatever it is that is most likely to get them funded.”

...as much as 90 percent of the published medical information that doctors rely on is flawed...the field of medical research is so pervasively flawed, and so riddled with conflicts of interest, that it might be chronically resistant to change—or even to publicly admitting that there’s a problem...an obsession with winning funding has gone a long way toward weakening the reliability of medical research. _theatlantic

In the opinion of Al Fin, those assertions are largely true. But not just for medical research. For all scientific research where funding is influenced by big money interests -- whether corporations, governments, or "philanthropic" organisations -- the issue of conscious and sub-conscious bias must be met head-on.

The problem is particularly acute in the field of climate science -- where $trillions are at stake. The severity of the problem was highlighted recently by the resignation of renowned and respected physicist Hal Lewis from the American Physical Society. Here is why Lewis resigned from the formerly prestigious society:
the money flood has become the raison d’être of much physics research, the vital sustenance of much more, and it provides the support for untold numbers of professional jobs. For reasons that will soon become clear my former pride at being an APS Fellow all these years has been turned into shame, and I am forced, with no pleasure at all, to offer you my resignation from the Society.

It is of course, the global warming scam, with the (literally) trillions of dollars driving it, that has corrupted so many scientists, and has carried APS before it like a rogue wave. It is the greatest and most successful pseudoscientific fraud I have seen in my long life as a physicist. Anyone who has the faintest doubt that this is so should force himself to read the ClimateGate documents, which lay it bare. (Montford’s book organizes the facts very well.) I don’t believe that any real physicist, nay scientist, can read that stuff without revulsion. I would almost make that revulsion a definition of the word scientist. _WUWT

In fact, the gut instinct of revulsion to mercenary science -- science corrupted by funding bias and influence seeking -- is clearly a better gauge of a true scientist than any degree of technical sophistication of research tools or supercomputer simulations. And yet climatology -- which is a newborn infant science in comparison to medical research -- may never escape the trap of politicised funding and publication selection in which it is snared.
Medical science is often biased at all stages by the desire of researchers and biomedical corporations for monetary gain. But the sheer amount of money at stake in climate science -- $trillions, as mentioned earlier -- dwarfs any profits ever to be gained by drug companies, medical insurance companies, or medical device companies. The gargantuan political influence combined with the relative tiny size of the climate science community, combines to provide the public with a thoroughly corrupted, biased, and unreliable look at global climate.

How many other areas of science share the same weaknesses? Follow the money. Big money interests (including governments) that get too deeply involved in the funding and publication process for any science, will introduce a corrupting influence on that science.

The dumbing down of schools from K-University does not help matters any. The replacement of meritocracy in the selection process for some graduate programs by affirmative action and other anti-meritocratic tools of political correctness, further degrades the grand enterprise of dispassionate scientific research.

All humans have their biases, which must be faced at every level of discovery. If human biases are ignored, or submerged beneath an overlying layer of "consensus" or political correctness, the end result of such an enterprise will be foul excrement.
Perhaps only a minority of researchers were succumbing to this bias, but their distorted findings were having an outsize effect on published research. To get funding and tenured positions, and often merely to stay afloat, researchers have to get their work published in well-regarded journals, where rejection rates can climb above 90 percent. Not surprisingly, the studies that tend to make the grade are those with eye-catching findings. But while coming up with eye-catching theories is relatively easy, getting reality to bear them out is another matter. The great majority collapse under the weight of contradictory data when studied rigorously. Imagine, though, that five different research teams test an interesting theory that’s making the rounds, and four of the groups correctly prove the idea false, while the one less cautious group incorrectly “proves” it true through some combination of error, fluke, and clever selection of data. Guess whose findings your doctor ends up reading about in the journal, and you end up hearing about on the evening news? _theatlantic

PLOS Medicine 2005 Why Most Published Research Findings are False

PLOS Medicine 2008 Why Current Publication Practices May Distort Science  

More: (via SDA_via_Maggie'sFarm)

  1. Whenever possible there must be independent confirmation of the “facts”

  2. Encourage substantive debate on the “evidence” by knowledgable proponents of all points of view.

  3. Arguments from authority carry little weight as “authorities” have made mistakes in the past. They will do so again in the future. Perhaps a better way to say it is that there are no authorities; at most; there are “experts”.

  4. Spin a variety of hypotheses. If there’s something to be explained, think of all the different ways in which it could be explained. Then think of tests by which you might systematically disprove each. The ones that survive are the ones to do in depth study on.

  5. Do not become attached to any hypothesis just because it’s yours. Find reasons for rejecting all, including your own, hypothesis.

  6. Quantify. If whatever you are explaining has a measure, quantify it so that measurement is more possible.
  7. Vague hypothesis, or those difficult to quantify will be the most difficult to prove or disprove.

  8. If there is a chain argument, then each and every link must work, including the premise.

  9. Use Occam’s Razor; which is to choose the hypothesis that explains the data in the simplest terms.

  10.  Ask: is the Hypothesis testable and falsifiable. Hypothesis that are not testable are not worth much. Could you duplicate accurately, at least theoretically, the hypothesis?

__Carl Sagan

Labels: ,

Bookmark and Share


Blogger kurt9 said...

This is very true about medical research. 95% of medical research is criminal fraud, including the much ballyhooed Framing-ham study on cardio-vascular disease.

Monday, 18 October, 2010  
Blogger Greg said...

Einstein wasn't paid to discover the Theory of Relativity. Just saying.

Monday, 18 October, 2010  
Blogger KGould said...

This is something I have mentioned to others - I have friends who have always been on the 'Global Warming bandwagon' and tell me that too many scientists agree etc etc... but then at a later date they will say that health care is being over-run by greed in the pharmacuetical companies and even go as far as saying that they will never willingly find a cure for cancer because it is too big a money-income for them from medications, treatments, equipment, etc. Okay - but who is at the heart of these pharma companies? Scientists of various forms create the medications, treatments, and medical equipment. So - if my friends do not trust the pharma companies, why do they blindly trust companies/groups employing the climate scientists?? It makes no sense to me. They will whole-heartedly agree that medical science is being taken over by greed, but refuse to admit that climate science could be as well. They will say it's a conspiracy to halt a cure for cancer, but say I am a 'nutcase' for wondering about a conspiracy with climate-gate....???? Again - it makes no sense to me.

Monday, 18 October, 2010  
Blogger gtg723y said...

I have always wondered why medical sciences haven't been pushing for the nanites of star trek.

On a more serious note while in school I have noticed that most of the research at GaTech was geared towards BME. Hopefully Engineers will continue to take of the world and fix things.

Monday, 18 October, 2010  
Blogger Kinuachdrach said...

The destruction of science was foreseen. See President Eisenhower's 1961 farewell speech, where he described concerns about the "military-industrial complex". And then went on to talk about the influence of government money on scientfic research:


"Akin to, and largely responsible for the sweeping changes in our industrial-military posture, has been the technological revolution during recent decades.

"In this revolution, research has become central; it also becomes more formalized, complex, and costly. A steadily increasing share is conducted for, by, or at the direction of, the Federal government.

"Today, the solitary inventor, tinkering in his shop, has been overshadowed by task forces of scientists in laboratories and testing fields. In the same fashion, the free university, historically the fountainhead of free ideas and scientific discovery, has experienced a revolution in the conduct of research. Partly because of the huge costs involved, a government contract becomes virtually a substitute for intellectual curiosity. For every old blackboard there are now hundreds of new electronic computers.

"The prospect of domination of the nation's scholars by Federal employment, project allocations, and the power of money is ever present and is gravely to be regarded."

Tuesday, 19 October, 2010  

Post a Comment

“During times of universal deceit, telling the truth becomes a revolutionary act” _George Orwell

<< Home

Newer Posts Older Posts