Irreproducible Science and US Authorities Regulation – Watts Up With That?

Guest contribution by S. Stanley Young and Warren Kindzierski

In April, WUWT published several articles by Dr. Judith Curry’s Climate Etc. website on How We Deceive Ourself. One of the articles – Part II: Scientific Consensus Building – indicated that… “Researcher Degrees of Freedom” allows researchers to extract statistical significance or other meaningful information from almost any data set. Last week, WUWT republished one of our articles from Climate Etc., which is about how epidemiologists try to deceive us with flawed statistical practices.

The elephant in the room that fuels these types of discussions is the irreproducibility crisis that is affecting modern science. This is not a trivial matter. We believe this problem has over time infected a wide range of government regulations and policies that are hugely damaging to the American public.

These are the questions people should be asking … How many federal regulations reflect irreproducible, inaccurate, and ill-founded research? How many grants have funded non-reproducible research? In short, how many government regulations based on irreproducible claims harm the common good?

Non-reproducible science

Academic researchers, fellows, and journals (and peer reviewers) all help contribute to the crisis of irreproducibility. We have previously shown that incentives for academic researchers reward exciting research with new positive (significant association) claims – but not reproducible research. This generally encourages scientists in general to humorously or negligently use a variety of questionable research practices to make claims that cannot be replicated.

Well-published university researchers earn tenure, promotion, relocations to more prestigious universities, salary increases, scholarships, professional prestige and public appreciation – primarily by publishing exciting research results. The National Academy of Sciences, Engineering, and Medicine (NASEM) recently reported a survey of academic researchers. NASEM stated that the researchers themselves believe they are responsible for addressing reproducibility issues, but that supporting institutional infrastructure (e.g. training, mentoring, funding, and publication) is required.

That’s all we need, more tax dollars to be used to fund academics so they can learn how to do the work they are supposed to know. What does this tell you about the current academic mentoring process? Do not turn to university administrations to solve the problem of irreproducibility. American university administrations want to receive scholarships, which they benefit from by receiving overhead expenses – often a large part of the total cost of research scholarships.

The problem of irreproducibility extends outside of science. In our experience, there is little incentive to change current academic research practices unless there is also an obligation to change the way in which fellows and journalists (and reviewers) end the Conduct the research publication process.

Funders (funding agencies and research sponsors) tend to emphasize the novelty of the research. These organizations want to fund the same exciting research. In addition, state donors have the additional incentive that exciting research with positive results also supports the expansion of their organizational mission. The downside is that this encourages many researchers to keep playing around with their research until they find a publishable, novel claim (publish or go under) that still won’t repeat itself.

Magazines are also looking for innovations in research, partly because of the competition for influencing factors. An impact factor is a measure of citation frequency that provides a quantitative metric for evaluating journals and their relative impact on the scientific community. The same incentives that drive academics impact journalists, who receive recognition and personal reputation awards for their magazine by publishing exciting research – even if the research has not been thoroughly reviewed. Editors are often rewarded for actions that increase the impact of their journal. Research reporting novel discoveries is more cited, adding to the stature of a journal and increasing rewards for journal editors.

Academic researchers, university administrators, scholars, journal editors – everyone has an incentive to find exciting research that earns money, status, and prestige, but little or no incentive to review their work. Most importantly, they have little incentive to reproduce the research and see if an exciting claim holds up – because if they don’t, they lose money, status, and prestige.

Compromised regulation by the US government

As we discussed in our post on Climate Etc., how the Environmental Protection Agency uses research to regulate outdoor air quality can show how irreproducible science affects government regulation. Over the past 40 years, the EPA has continuously examined and introduced increasingly restrictive air quality regulations.

That’s not necessarily a bad thing. As part of its mandate, the EPA must develop air quality criteria based on expert opinion and describe their effects individually and in combination on the health and well-being of American citizens. In addition, the EPA must define the National Ambient Air Quality Standards (NAAQS) as a benchmark for the measurement of their own air quality by states and municipalities and as a legal requirement for the enforcement of the NAAQS.

However, the debate about whether the EPA should make a particular regulatory decision based on scientific evidence raises questions that are central to the irreproducibility crisis – data accuracy, research logs, statistical analysis, publication bias, sponsorship bias, etc. This is done by bureaucrats in government makes fund research difficult – these bureaucrats rely on regulation to support their existence. Why would they want to acknowledge an irreproducibility crisis if it would affect their careers?

There is a burden on society that lurks behind the intentions of academic researchers and government bureaucrats who use their research to develop regulation. Together they distort the process to justify increased regulation. The costs of insufficiently justified environmental regulation cannot be dismissed.

An example is the estimated cost of ships using “cleaner fuel” with less sulfur to reduce sulfur dioxide emissions. The EPA argues that switching to low-sulfur marine fuel could prevent 14,000 deaths annually by 2020, based on epidemiological studies in which sulfur dioxide kills. Let’s ignore the great uncertainties surrounding their estimate (which must be taken with a piece of salt). In addition, the EPA estimates the statistical lifespan to be worth $ 7.4 million ($ 2006). Let us also ignore the major uncertainties associated with this estimate.

Using the EPA figures and an approximate 28% inflation from 2006 to 2020, the health-related benefits for this requirement are estimated to be more than $ 130 billion per year in 2020 (14,000 x 7.4 million x 1.28). That might sound convincing to an environmentalist, but is it real? A growing body of research does not support the EPA’s position. In particular, there is research showing that sulfur dioxide in the outside air is not associated with death, heart attacks, asthma or lung cancer.

Comprehensive regulatory systems are always associated with additional costs. Who knows what the real extra costs will be for ships to use fuel with less sulfur? However, these are not zero. Depending on the type of ship and service, fuel costs account for up to 50-60% of total ship operating costs. On the other hand, there is a clear possibility that health-related benefits (ie statistical lives saved) derived from this requirement are meaningless.

The EPA enacts an extraordinary number of regulations that affect all areas of the economy and limit everyday freedoms. Comprehensive regulatory systems can give large companies a competitive advantage over small companies. Large companies have greater capacity to comply with a comprehensive legal framework. In addition, the cost of regulation will ultimately be borne by American consumers. These costs can also have negative health effects, such as: B. due to increased unemployment, e.g. B. Depression, substance abuse, and domestic violence.

It can be shown that the development of air quality regulation by the EPA reflects irreproducible, erroneous and ill-founded research results. How many other federal regulations suffer this fate? That is the trillion dollar question.

S. Stanley Young is the CEO of CGStat in Raleigh, North Carolina and the director of the Shifting Sands Project for the National Association of Scholars. Warren Kindzierski is an Associate Professor in the University of Alberta’s School of Public Health in Edmonton, Alberta.

Like this:

To like Loading…

Comments are closed.