In Pakistan, the military dictatorship in ~2002 decided they wanted to jump start academic research. The scientist who led this effort had the "brilliant" idea that the government pays professors per published paper, but decided there was no need to restrict this to publishing in high quality journals. Simultaneously, promotions were dependent on number of published papers.
The end result was natural. People first started published in really low quality journals, then started plagiarizing high papers from others and submitting in lower quality journals. Then got smart, and created their own "fake" journals to publish their "fake" papers in. At the forefront of this corruption was the scientist himself [1]. Things have become dire. Recently, a famous university in Pakistan offered a candidate professor, a contract stating that they would hire him if the published ~10 papers a month. And the government money from that would be his salary.
The point of the story is that fraudulent science is not often not just because people are "evil". It can emerge from bad incentives set up by governments or larger institutions.
> Discussions with different stakeholders suggest that many currently perceive systematic fraudulent science as something that occurs only in the periphery of the “real” scientific enterprise, that is, outside OECD countries. Accumulating evidence shows that systematic production of low quality and fraudulent science can occur anywhere.
From supplement (section about the output of the "ARDA" paper mill):
> We obtained 20,638 documents and were able to impute country of authorship for 13,288 documents (64.4%). Of these documents, more than half were solely from India (26.4%), Iraq (19.3%), or Indonesia (12.2%).
The identity and reputation of the authors, and the publication venue, is (for now) still a strong signal when evaluating the credibility of an article.
The article is spot-on though in that there is a real risk of paper mills infecting formerly reliable journals, and this is not helped by the publishers' commercialism. For example, it used to be easy to ignore Hindawi journals (they are characteristically low quality); then Wiley started publishing them under its own brand. The good is now mixed with the bad under the same label. Practicing scientists can fall back on whether they know the authors personally but that doesn't really help non-practicing professionals or the general public.
My experience is that incentives within the two are converging: monetization drives all activities.
Higher ed is allowing professors to slump teaching responsibilities - as it has for many years - in favor of high-flying research. What is new is that now: entire departments struggle to find chairs or people to run basic functions at teaching or research levels. Professors are rarely on campus except at the bare minimum and are largely diverting resources (people & research) to their companies.
> (philosoraptor template:) Maybe paying professors more could help?
who is going to pay them more? institutions? do you think that institutions are acting against their own incentives somehow? institutions are acting on their incentives and it's producing suboptimal outcomes for society because spoiler alert the free market isn't the correct mechanism for absolutely everything in life.
Generally i agree that paying more is unlikely to resolve much.
Fixing the incentives might, however funding comes from very few places and they have their own incentives => lobbied goals or profit-motivated.
One might argue that money is a prioritization method.
I'd argue that having one metric (amount of money) makes you susceptible to single-metric problems: optimization of everything to the point of meaninglessness, critical dependence on the stability of that metric's underlying present & future value.
Defects in the currently implemented financial system (exploitable by evolving technologies) might lead to larger systematic failures and less robustness.
Had similar thoughts from hobby work in game development. Bit like a system where you have many game mechanics, yet most have become meaningless as the money mechanic is the only metric that actually means anything.
Politics doesn't mean anything, money. Academia is starting to mean less, money. Culture is mostly, money. Health care is having serious issues, money. Military industrial is having issues, money. Even farming is having issues, money.
«The trends we expose forecast serious risks ahead for the scientific enterprise. Large groups of editors and authors appear to have cooperated to facilitate publishing fraud (Fig. 1). Networks of linked fraudulent articles suggest industrial scale of production (Fig. 2). Organizations selling contract cheating services anticipate and counter deindexing and other interventions by literature aggregators (Fig. 3). The literature in some fields may have already been irreparably damaged by fraud (Fig. 4). Finally, the scale of activity in the enterprise of scientific fraud already exceeds the scope of current punitive measures designed to prevent fraud (Fig. 5). Currently implemented punitive measures are not addressing the tide of fraudulent science.»
This sounds pretty damning, and the figures mentioned (closer to the end of the article) are pretty revealing.
Weirdly specific targeting of something much like a cancer of academia that then specifically targets "miRNAs, cancer" and "LncRNAs" with the note that much "higher rates of retractions for subfields focused on cancer than on development".
Maybe we should not trust authors that make vague unsubstantiated claims?
The ingress: "Some suggest that the ease of communication provided by the internet and open-access publishing have created the conditions for the emergence of entities..."
Having unsubstantiated statements in an abstract of a paper is deeply worrying from a scientific standpoint. The methodology in the paper seems ad hoc, sprinkling it with statistics does not help if there is no firm ground to stand on, ie a sound method. Many of the reported issues have other explanations.
Publishing has been broken for years and the system built by academia on top (publish or perish) has exacerbated the problem. Science is well and all, and as you mentioned, results should be reproduced, not only this paper's (with a better methodology preferably), but ALL science that has even the faintest hallmark of being published for publishing's sake.
“ In a 2022–2023 survey of medical residents at tertiary hospitals in southwest China, 46.7% of respondents self-reported buying and selling papers, letting other people write papers, or writing papers for others”
This line gave me pause. So basically everyone is deceiving everyone else all the time? At least on the plus side, it did make me feel a little more optimistic about AI slop. A machine-written report rife with errors and fabrications is apparently at a parity with real human performance.
New generations spend more time, money and effort than ever to educate themselves... Only for the system to become increasingly less meritocratic. Not only is the education becoming less valuable because of more competition, but it's also becoming less valuable because of less demand for it... Why study the real thing when you can just fake it and get rewarded better?
In Pakistan, the military dictatorship in ~2002 decided they wanted to jump start academic research. The scientist who led this effort had the "brilliant" idea that the government pays professors per published paper, but decided there was no need to restrict this to publishing in high quality journals. Simultaneously, promotions were dependent on number of published papers.
The end result was natural. People first started published in really low quality journals, then started plagiarizing high papers from others and submitting in lower quality journals. Then got smart, and created their own "fake" journals to publish their "fake" papers in. At the forefront of this corruption was the scientist himself [1]. Things have become dire. Recently, a famous university in Pakistan offered a candidate professor, a contract stating that they would hire him if the published ~10 papers a month. And the government money from that would be his salary.
The point of the story is that fraudulent science is not often not just because people are "evil". It can emerge from bad incentives set up by governments or larger institutions.
[1] https://loksujag.com/story/bentham-science-publishers-scam
Many evils are caused by bad incentives like this.
From main text:
> Discussions with different stakeholders suggest that many currently perceive systematic fraudulent science as something that occurs only in the periphery of the “real” scientific enterprise, that is, outside OECD countries. Accumulating evidence shows that systematic production of low quality and fraudulent science can occur anywhere.
From supplement (section about the output of the "ARDA" paper mill):
> We obtained 20,638 documents and were able to impute country of authorship for 13,288 documents (64.4%). Of these documents, more than half were solely from India (26.4%), Iraq (19.3%), or Indonesia (12.2%).
The identity and reputation of the authors, and the publication venue, is (for now) still a strong signal when evaluating the credibility of an article.
The article is spot-on though in that there is a real risk of paper mills infecting formerly reliable journals, and this is not helped by the publishers' commercialism. For example, it used to be easy to ignore Hindawi journals (they are characteristically low quality); then Wiley started publishing them under its own brand. The good is now mixed with the bad under the same label. Practicing scientists can fall back on whether they know the authors personally but that doesn't really help non-practicing professionals or the general public.
I find going by citation good for established work. Harzing's publish or perish is useful for this.
I've been in industry and academia.
My experience is that incentives within the two are converging: monetization drives all activities.
Higher ed is allowing professors to slump teaching responsibilities - as it has for many years - in favor of high-flying research. What is new is that now: entire departments struggle to find chairs or people to run basic functions at teaching or research levels. Professors are rarely on campus except at the bare minimum and are largely diverting resources (people & research) to their companies.
(philosoraptor template:) Maybe paying professors more could help?
Tuition is at all-times high, but apparently only 1/4 to 1/3 of this money goes to the actual teaching staff.
> (philosoraptor template:) Maybe paying professors more could help?
who is going to pay them more? institutions? do you think that institutions are acting against their own incentives somehow? institutions are acting on their incentives and it's producing suboptimal outcomes for society because spoiler alert the free market isn't the correct mechanism for absolutely everything in life.
Generally i agree that paying more is unlikely to resolve much.
Fixing the incentives might, however funding comes from very few places and they have their own incentives => lobbied goals or profit-motivated.
One might argue that money is a prioritization method.
I'd argue that having one metric (amount of money) makes you susceptible to single-metric problems: optimization of everything to the point of meaninglessness, critical dependence on the stability of that metric's underlying present & future value.
Defects in the currently implemented financial system (exploitable by evolving technologies) might lead to larger systematic failures and less robustness.
Had similar thoughts from hobby work in game development. Bit like a system where you have many game mechanics, yet most have become meaningless as the money mechanic is the only metric that actually means anything.
Politics doesn't mean anything, money. Academia is starting to mean less, money. Culture is mostly, money. Health care is having serious issues, money. Military industrial is having issues, money. Even farming is having issues, money.
As long as the institutional incentives are monetization, it will never be enough.
Entities enabling _______ fraud at scale are large, and growing rapidly.
Give me something that isn't being enabled by the current technology progress.
Empathy and humility.
virtue, peace.
«The trends we expose forecast serious risks ahead for the scientific enterprise. Large groups of editors and authors appear to have cooperated to facilitate publishing fraud (Fig. 1). Networks of linked fraudulent articles suggest industrial scale of production (Fig. 2). Organizations selling contract cheating services anticipate and counter deindexing and other interventions by literature aggregators (Fig. 3). The literature in some fields may have already been irreparably damaged by fraud (Fig. 4). Finally, the scale of activity in the enterprise of scientific fraud already exceeds the scope of current punitive measures designed to prevent fraud (Fig. 5). Currently implemented punitive measures are not addressing the tide of fraudulent science.»
This sounds pretty damning, and the figures mentioned (closer to the end of the article) are pretty revealing.
Goodhart's Law gives rise to Sturgeon's Law :(
Weirdly specific targeting of something much like a cancer of academia that then specifically targets "miRNAs, cancer" and "LncRNAs" with the note that much "higher rates of retractions for subfields focused on cancer than on development".
Maybe we should not trust authors that make vague unsubstantiated claims?
The ingress: "Some suggest that the ease of communication provided by the internet and open-access publishing have created the conditions for the emergence of entities..."
The article: nothing
This article seems more like part of the problem.
The article does some real-looking statistical analysis. I'd love it if somebody tried to reproduce the results, as is customary in science.
Having unsubstantiated statements in an abstract of a paper is deeply worrying from a scientific standpoint. The methodology in the paper seems ad hoc, sprinkling it with statistics does not help if there is no firm ground to stand on, ie a sound method. Many of the reported issues have other explanations.
Publishing has been broken for years and the system built by academia on top (publish or perish) has exacerbated the problem. Science is well and all, and as you mentioned, results should be reproduced, not only this paper's (with a better methodology preferably), but ALL science that has even the faintest hallmark of being published for publishing's sake.
“ In a 2022–2023 survey of medical residents at tertiary hospitals in southwest China, 46.7% of respondents self-reported buying and selling papers, letting other people write papers, or writing papers for others”
This line gave me pause. So basically everyone is deceiving everyone else all the time? At least on the plus side, it did make me feel a little more optimistic about AI slop. A machine-written report rife with errors and fabrications is apparently at a parity with real human performance.
New generations spend more time, money and effort than ever to educate themselves... Only for the system to become increasingly less meritocratic. Not only is the education becoming less valuable because of more competition, but it's also becoming less valuable because of less demand for it... Why study the real thing when you can just fake it and get rewarded better?
Like OpenAI?
[dead]