Misuse of the scientific method has led to peer review failures with significant implications

By Joseph D’Aleo, CCM, AMS Fellow

THE SCIENTIFIC METHOD

The scientific method in science is an iterative process. The scientific method starts with a theory or hypothesis. The data needed to test it and all possible factors involved are identified and gathered. The data is processed and the results rigorously tested. The data and methods are made available for independent replication. Reviewers for the proposed theory must have the requisite skills in the topic and in the proper statistical analysis of the data to judge its validity. If it passes the tests and replication efforts, a conclusion is made and the theory may be turned into a paper for publication. If it fails the tests, the hypothesis or theory must be rethought or modified.

SC1.png

It should noted a refutation of a previously accepted theory even one that has been published and widely accepted can follow the same route to review and publication as Albert Einstein observed:

SC2The peer review process is failing due to political and economic pressures that have altered the scientific method to virtually ensure a politically correct or economically fruitful theory can never fail.

When the tests fail, instead of rethinking the theory or including other factors, there is an alarming tendency to modify input data to more closely fit the theory or models.

SC3.png

Also often, the authors and reviewers do not to have the proper understanding of all the factors involved and often the needed mathematical skills to properly evaluate the results. And even if they do, the input data and methods are generally not made available to the reviewers for replication. And in many cases, forecasts are made for many decades or even centuries into the future, so true validation is not possible, a luxury those of us who must forecast in shorter time frames (days to seasons) do not enjoy.

Also too often, the reviewers that then serve as final gatekeepers are often not only not fully capable of this kind of rigorous review, they are often biased and speed politically correct or economically beneficial work to publication while blocking or at least ‘slow walking’ work that challenges the so-called consensus science or their own often ideologically driven beliefs.

As Dr. Michael Crichton wrote “Let’s be clear: the work of science has nothing whatever to do with consensus. Consensus is the business of politics. In science, consensus is irrelevant. What is relevant is reproducible results. The greatest scientists in history are great precisely because they broke with the consensus. (Galileo, Newton, Einstein, etc)”

SCIENTIFIC METHOD FAILURE IN THE CLIMATE SCIENCES

So when greenhouse climate models fail, they don’t revisit the theory but instead try and find the right data to fit that model. All data today is adjusted with models with a goal of addressing data errors, changes in location or instrumentation or changing distribution or filling in for missing data or station closures. Once you start this adjustment process, there is always the risk to adjust the find ways to mine from the data the desired results.

With the climate models there is an increasingly large divergence with balloon, satellite and surface reanalysis data sets the last 20 years. The one model that follows the temperature is a Russian model that has roughly half the greenhouse forcing and improved ocean modeling.

SC4.png

John Christy 2017 has shown models without greenhouse warming agreed perfectly with atmospheric (tropical) observations.

SC5.png
This kind of refutation should, if scientists abided by the scientific method, spark an effort to revisit the theory but that is too politically incorrect. This kind of ideologically or politically or economically driven thinking is pervasive across the sciences (atmospheric and medical).

EVIDENCE THAT TRADITIONAL PEER REVIEW IS FAILING

There is increasing proof that the traditional journal peer review process is broken. This is true in the Medical and Scientific areas. See examples here and here.

This story in Forbes by Henry Miller says “A number of empirical studies show that 80-90% of the claims coming from supposedly scientific studies in major journals fail to replicate”.

Another recent paper in Nature showed 70% of the papers in medical journals had studies that could not be replicated, many not even by the original authors. See also this example of one such falsified report that the author worries is a part of an epidemic of agenda-driven science by press release and falsification that has reached crisis proportions.

Other reports show an alarming number of papers having to be retracted. Springer is retracting 107 papers from one journal after discovering they had been accepted with fake peer reviews (here).

Result-oriented corruption of peer review in climate science was proven by the Climategate emails.

In the journals, there are a small set of gatekeepers that block anything that goes against the editorial biases of the journals. Conversely, these journals and their reviewers do not provide a thorough due diligence review of those that they tend to agree with ideologically. They are engaged in orthodoxy enforcement.

Indeed, Henry Miller wrote: “Another worrisome trend is the increasing publication of the results of flawed ‘advocacy research’ that is actually designed to give a false result that provides propaganda value for activists and can be cited long after the findings have been discredited.”

A prime example of this is the hideously flawed but endlessly repeated “97% of climate scientists” paper by Cook and Lewandowski. EPA’s own Inspector General found that EPA’s Endangerment Finding was never properly reviewed, yet it is the basis of all EPA GHG regulations that imposed hundreds of billions in costs on the U.S. economy.

The scientific method requires the data used be made available and the work must be capable of being replicated. This should be required of all journals (in virtually all cases, as shown above, it is not). Peer review has become pal review with gatekeepers that prevent alternate unbiased data analyses and presentation but rush new papers that support their ideology or view on the science.

Our team chose to apply the same research report procedures used in industry, which is to assemble the most qualified authors with the skills required to compile the data and rigorously perform the correct analysis. They draft a report and share the draft with a team of experts chosen for their expertise in this field to provide feedback. In our research reports, we identify the reviewers, who have lent their names to the conclusion, and provide full access to the data for others to work with and either refute or replicate, with and instructions on the analytical methods used.

Almost no journals require that and their failure and rejection numbers speak for themselves.

Wegman suggested one of the common failures in climate papers is the lack of necessary statistical expertise. For our research reports we assembled the highest qualified data experts, econometricians/statisticians and meteorologists/climatologists to draft the research project, do the rigorous statistical/econometric analyses, and then submitted their work to the best qualified scientists/econometricians for review. Attempts to discredit this report are now of course being made because it raises critically important questions about the quality and trustworthiness of the global surface temperature data sets.

The facts and statistical reasoning of this paper cannot be refuted merely by carping peer review. Instead, demonstration of a factual or mathematical error is required.

The scientific method must be reestablished. Bad science, bad medicine, bad economics lead to bad policies. Bad policies hurt good people

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s