This failure in economic literature is relevant to climate science because not only are the dynamics we’ve seen in economic science demonstrated similar to climate science, but also both fields have large influences on public policy — in part due to politicians confidence in peer reviewed literature.
One thing I found out as a grad student, particularly while I was working on my Ph. D., is that one should never start to investigate a subject unless one already knows what the answers will be.
That’s basically how one obtains research funding. Grants are rarely obtained simply because of curiosity or a desire to “see what’s out there”. Funding agencies reward success and punish failure. Therefore, whatever gets investigated (i. e., funded) is that which has a higher chance of success at finding what’s expected.
peer review should be done on a double blind bases, that way much of the shenanigans are stopped. Then again, the best peer review is a published paper that is widely disseminated to the market place. And then only “qualified” after much public input.
That’ll never happen because it slows down the rate of publication.
In research, particularly in academe, everything depends upon one’s publication rate. The more one publishes, the greater one’s chances of getting tenure and, thereby, a seat on the gravy train. Funding agencies love high publication rates because it gives them the impression that their money is well-spent and it also increases a researcher’s chances at getting more when it comes time to renewing a grant. Bring in the moolah, and one’s chances of promotion go up.
I’m sure you see how it works. Never mind that most people on tenure or promotion committees rarely read those publications, if at all. Forget that many funding agencies are run by people who don’t understand the research for which they give out their money.
I saw this first-hand. One of my graduate supervisors took a set of data and got at least 3 mediocre papers out of it, rather than one real good one. It looked to me that he simply divided that data set in half, producing one paper from one of them. The other half looked like it was divided again, and one paper was written from each portion.
It was quite profitable–3 publications for one data set. Now do the same for each new run, and one starts to look quite productive.
See how it works? The department administration can brag at how much is being done. The funding agency would likely have been impressed and, perhaps, given him even more money when he renewed his grant. He could all a lot more titles to his CV. His ranking in the publication surveys goes up, particularly if a lot of people (particularly his own grad students) cited them in their own papers and reports.
Oh, and don’t forget the publishers. They stand to make a lot through journal subscriptions and sales of reprints of individual papers.
Compared with this arrangement, Ponzi schemes begin to look amateurish.
“What Would We Do Without Peer Review?”
Real science? You know, the kind with reproducible results? The kind of science where the statistical factual input is questioned.
The kind of science where everyday world observations, empirical observations, don’t contradict the hypothesis.
The kind of science where if an hypothesis is falsified, then they scrap the hypothesis.
That’s what we’d do.
And arguments to the contrary, it would cost all of us less if we didn’t have peer review even if the experimental science were subsidized because they wouldn’t be building wind/solar/ethanol infrastructure and subsidizing that to the tune of $Trillions on the basis of mere peer review.
“…in part due to politicians confidence in peer reviewed literature…”
BULL F****** SHIT! Methinks you give them far too much credit.
Do you thing for one f****** minute that the average politician reads or cares about “peer reviewed literature” as it relates to Global Warming…or anything else for that matter?
With few exceptions, they are sheep. And they simply vote the way turdo la doo, or Patrick Brown, or any of the other arseholes tell them to. And that direction is solely dependant upon the advice of little putzes with poli-sci degrees who have been thoroughly schooled in the arts of acquiring and retaining power… by Marxist profs who despise and oppose every aspect of capitalism because its dollar-per-value system, if adhered to, would represent a threat to, not only their tenure, but their veritable existence too.
Truth doesn’t matter; damage to their province, country, or future generations doesn’t matter….just toss out whatever bullshit appeals to the greatest number of stupid/selfish/lazy bastards, and ride the wave – while all the time pretending that the kicked can doesn’t exist.
Samuel Johnson had it right: truth is the first casualty in war.
The same can be said for the profitable business of ‘academic research’.
That’s particularly true if that research is conducted on behalf of a third party and done so under contract.
About 20 years ago, a physician in Toronto was conducting a clinical trial of a new medication for a large pharmaceutical corporation. She noticed that there may be some harmful side effects with the drug and she publicized the potential hazard as she felt it was her duty as a physician.
Guess who got hauled into court for breach of contract, unethical conduct, and violation of confidentiality?
With all due deference to B, NME, and Oz’s descriptions and explanations of the peer review process, how it should work and does work, you and Jean describe perfectly how it works in government.
The peer review system–or, for that matter, the entire research process–is largely nothing but politics and, in many ways, it works very similar to government.
That’s one thing I noticed while I was working on my Ph. D. Originality is discouraged, if not punished. A good idea, however novel, not only confuses people, it means that they have to make an effort to figure it out, even though it may be presented in a logical and methodical manner.
Whoever happens to be cozy with one’s supervisor gets supported. If that supervisor takes a dislike to a particular grad student, that person will have a difficult time. (That wasn’t just my experience. I’ve read on-line accounts of other people who went through the grad studies circus and many said something similar.)
It’s been that way for centuries, even after Copernicus and Galileo. For example, Sir Isaac Newton was reluctant to publish because he was afraid that someone else would pinch his ideas. In addition, the often caustic rivalry between Newton and Sir Robert Hooke was well-known.
Closer to the present day, the Manhattan Project didn’t go smoothly because of interpersonal rivalries and squabbling between researchers. Dr. Robert Oppenheimer and Gen. Leslie Groves had their hands full managing the many large, and often sensitive, egos of the people who worked on it.
Quite so. I guess everyone would like their name in the history books even if unethically accomplished.
I also noticed that the primary goal of the entire research process was to preserve the status quo. Deviations from ‘acceptable thought’ were only marginally allowed for those with Ph.D.s and a long list of publications. Such thoughts from lowly undergrads were considered heresy. Actual advancement of science was low priority; it was mostly politics. I met the most duplicitous people in academia. They would appear all smiling and affable in public and look for every possible way to assassinate your character in private. They were the nastiest people I ever had the displeasure of working with.
Not quite everyone. It doesn’t matter to me; I’m just visiting.
And the bigger the prize, the greater the temptation for that sort of thing.
For example, why wasn’t Best a co-winner of the Nobel that Banting and Macleod won? Banting and Best did most of the initial work in isolating insulin and identifying its metabolic role. As well, Banting, apparently, thought that Macleod’s contribution was less than that of Best.
Me too.
I used to think that academics were worthy of admiration because they were dedicated to the pursuit of truth and knowledge. Then I started grad school…..
Shortly after I started, I got roped into being the grad student representative in the department faculty meetings. Now *that* was an eye-opener. It didn’t take long to find out what certain profs thought about their colleagues. Some cases, the “discussions” became so heated that I was sure that a fistfight was about to break out.
Years later, I started my Ph. D. My supervisor turned out to be a jackass. I completed my previous grad degree under him, so I thought that I could continue with him as my advisor. Boy, was I wrong!
Things didn’t go well. I got little support and certain things happened to block my progress. (It wasn’t until several years after I finished my degree that it occurred to me that he deliberately wanted to kill my project, hoping that I’d work on what *he* was interested in.)
One of the members of my final committee (the first one quit on me, which is story in itself) didn’t do a lot aside from being a crap artist. Whenever I wrote a paper, I was “encouraged” by my supervisor to add this chap’s name to the author list, even though his contribution was next to nothing. (I couldn’t figure out how his answering a simple question could be considered to be vital work.)
When I submitted my thesis, I raised a lot of eyebrows when I didn’t specifically acknowledge my supervisor. I had spent enough time in industry to realize that one doesn’t reward a colleague who sat on his backside while everyone else worked. Since my supervisor was next to useless, I didn’t name him and many people took umbrage at that.
It’s not called the academic circus for nothing.
Being remembered in the history books was never my objective. If I am, it would be nice, but not at any price or by any means.
Not all or even most are in it for the fame it can bring. Probably the vast majority are just doing work they love.
Peer Reviewer “shopping” … is no different than Judge “shopping” … to obtain your desired outcome. BTW … we can expect quite a bit more Judge “shopping” … the longer the SCOTUS remains at 8 members
the truth does not need a lot of peer review.