> Just as academics must cite any AI-generated machine outputs they rely upon in their research and writing, they can also cite Wikipedia if they use it in their work. Doing so can highlight Wikipedia’s value to other academics as a source of knowledge and provide greater transparency in one’s research processes. Despite longstanding skepticism of Wikipedia among some scholars, citations to Wikipedia in academic journals appear to be increasing – even in high-impact publications in subject areas like the health sciences.
As someone who's contributed to Wikipedia and seen how the sausage is made: that's stupid. Even Wikipedia itself considers wikis (including itself) to lack sufficient authority to be cited as a source.
Such citations could be easily gamed (if you cite a particular change) or are useless (the content you cite may not be there in the future), and are obvious pathways to "citogenesis."
* one source of truth. Not useful for controversial topics
* On topics like graph databases (where I've spent 10+ years), the content doesn't reflect best practices (e.g. VLDB papers)
* Someone with sufficient resources can remove content in the name of "clean up"
* Google <-> wikipedia effect that others discuss in the thread.
Academics citing any website should consider also pointing towards (or at least creating) an archived version via something like the Wayback Machine. Wikipedia is not the only website that changes over time, and most don't have a easily accessible revision history.
> Academics citing any website should consider also pointing towards (or at least creating) an archived version via something like the Wayback Machine.
It's probably better for the user to search the original URL themselves on the Wayback Machine. If you cite with a Wayback Machine link when the original link still works, you're just creating needless work for a non-profit. An archive isn't a CDN.
Academics require primary source evidence - e.g., experimental or documentary. Wikipedia is a tertiary source; by policy it forbids primary sources and requires secondary sources.
Recently I did a deep dive on some minor historical event in the 1800s that I won't bother to go into detail here, except to say that it has been claimed by an activist editor who was made it a personal mission to re-frame the event as an example of "westerners" committing sexual violence against helpless "easterners".
While certainly this kind of story is common throughout history, in this case, there's absolutely no documentary evidence of the event, and what little evidence exists points in the other direction. This is not just the weight of evidence -- it's what all current historical experts believe. It's what all major encyclopedias say about the event. Nonetheless, the Wikipedia editor clings to a bad reference (in an extremely biased book) that makes an off-hand assertion. This assertion, in turn, is sourced with only a single citation to a 1960s-era historical novel that a) didn't actually support what the author claims, and b) essentially made up signficant details of the event in question, based on popular movies and rumors of the time. It's garbage all the way down.
Nevertheless, anyone who edits this article to reflect accurate current knowledge is immediately embroiled in a war with the editor, because unfortunately, there's no "trusted source" that says the opposite of the bad citation. And so the lie remains. To fight this battle would require escalating the issue and possibly getting the editor removed. Who has time?
Again, this is a minor historical event that nobody cares about, and you still cannot trust it as a source. I have seen enough things like this across Wikipedia that I no longer trust it for anything that might intersect with modern political opinions.
If you look up something on Wikipedia and include it in your work without citing it, it's an even worse pathway to citogenesis, since someone trying to check whether you can be used as a source for Wikipedia won't know that that's where you got it from in the first place.
Of course for most things you should have no need to refer to Wikipedia at all, and hence also no need to cite it.
What academics need to defend is wikipedia in latent space. Not walls of text that reflect the opinions of a few who have the time to edit that the rich and famous can optimize.
Defense of open knowledge will appear when they attack the monopoly of Elsevier and other publishers through innovative systems for peer-evaluated papers, not by trying to resuscitate the body of the spook-filled Wikipedia.
Open knowledge systems like Wikipedia are not really defensible. Letting anybody edit any page is a great way to get AI and human slop everywhere. It makes it extremely easy to insert knowledge with dubious quality and sources.
We need "open access" knowledge systems with known authors who are sponsored by non-profit and educational organizations and its their duty to defend their reputation.
Humanity needs more openly accessible curated and factually checked information and "open to modify" systems are bad at in all.
Something like Wikipedia is a wonderful idea in theory, but it's tough to defend when their policies allow factually incorrect content to remain just because it appears on a third-party source that the editors consider reliable. Most media sources contain a pretty high percentage of fake news and when this slop makes its way into Wikipedia that makes the problem even worse. Can we find a better solution?
> it's tough to defend when their policies allow factually incorrect content to remain just because it appears on a third-party source that the editors consider reliable.
I feel this is the kind of discussion that would really benefit from a number of concrete examples.
Philosiphers have been trying to work on how to define what is "True" since forever without much to show for it.
Wikipedia tries to kick the bucket by going with what "reliable" secondary sources say. Thus the goal is not to say the capital T Truth, but to essentially sumnarize academic knowledge.
To steal a saying - its the worst method except for every other known method.
If anyone has a better idea how to determine what is true and what isn't - i'd certainly be very interested.
I think the Gell Mann effect applies on Wikipedia just as much as in the papers. I’ve often come across articles in an area where I’m an expert and they have issues. I used to try and have them changed, but many years ago I just got exhausted by fanatics who seem to have just gotten really attached to their (incorrect) prose.
Sorry but scientific method has been the default for "Truth" for many academic subjects. It is reliable and idempotent. If all of the human information and most of the knowledge workers are removed from the Earth tomorrow, many will arrive in the same conclusion in most of the natural sciences and maths. The language and representation will be different but prime numbers will still be prime numbers and the matter will still be made of one neutral and two "interactive" elements.
Social sciences like politics, history and economy are harder to replicate with the scientific method, however repeated observations, statiscial information as well as qualitative analysis with evidence-based research can still provide replicable analysis.
> If anyone has a better idea how to determine what is true and what isn't - i'd certainly be very interested
This kind of excuse is often cited to shut down any discussion, especially in history, politics and economics. I find it deeply anti-intellectual and dogmatic. As long as evidence and counter-evidence is followed, it should be possible to discuss alternative interpretations and solutions.
While I agree with your sentiment, I must add to this that to say that something is unequivocally true that holds in all conditions everywhere is difficult to ascertain in general. So we have to deal with that two (or more) things can be true at the same time (sometimes). This may be less the case for things such as prime numbers but more the case when the underlying principle is void of simple cause and effect (in the linear sense).
The scientific method can be used to falsify hypotheses. It cannot prove that something is true. Fields that use the scientific method typically rely on social consensus for determining the truth. Empirically speaking, the social consensus seems to converge to a reasonable approximation of the truth with a high probability, but the process can take decades.
I understand that "truth" and "facts" are squishy philosophical concepts at the edges, but I'm talking about something much more basic. There are Wikipedia pages which are flat out wrong (by any common definition of the term) and yet the editors won't allow any corrections because the primary sources are also wrong.
One thing that might help a bit would be to tag or color code specific statements with a confidence factor. So a statement sourced only from a single trash site like the NY Times or Fox News would be red, whereas another statement sourced from multiple independent scholarly journals would be green. Or something like that.
Fair to assume that the library of Alexandria contained books written by the most erudite scholars and philosophers of their day, not Joe and Jane and their cohort of terminally online gatekeepers.
Why is that fair to assume? "Books" were extremely rare then. Some random librarian had to include those books, and likely leaned towards taking all written works due to scarcity.
Likely some other librarian disagreed with some inclusions and certainly many readers disagreed with the content. They must have had a system for all of this. It was far less open than Wikipedia.
IIRC, they seized all scrolls from all ships in the port, in many languages. They likely had many "errors" according to other librarians and many readers.
Wikipedia is existentially threatened by AI, which does the same thing but better.
> most users of AI products do not click the links to Wikipedia, choosing instead to accept machine-generated outputs without further verification. If this practice continues, readership on Wikipedia will continue to decline.
This is the same mechanism by which Wikipedia drained readers from news outlets and academic papers that it cites for information.
I play the tiniest violin in the world for Wikipedia editors that now suffer from the same thing they did to the news media.
> Just as academics must cite any AI-generated machine outputs they rely upon in their research and writing, they can also cite Wikipedia if they use it in their work. Doing so can highlight Wikipedia’s value to other academics as a source of knowledge and provide greater transparency in one’s research processes. Despite longstanding skepticism of Wikipedia among some scholars, citations to Wikipedia in academic journals appear to be increasing – even in high-impact publications in subject areas like the health sciences.
As someone who's contributed to Wikipedia and seen how the sausage is made: that's stupid. Even Wikipedia itself considers wikis (including itself) to lack sufficient authority to be cited as a source.
Such citations could be easily gamed (if you cite a particular change) or are useless (the content you cite may not be there in the future), and are obvious pathways to "citogenesis."
Problems with wikipedia:
Latentpedia will not have any of these problems.Academics citing any website should consider also pointing towards (or at least creating) an archived version via something like the Wayback Machine. Wikipedia is not the only website that changes over time, and most don't have a easily accessible revision history.
> Academics citing any website should consider also pointing towards (or at least creating) an archived version via something like the Wayback Machine.
It's probably better for the user to search the original URL themselves on the Wayback Machine. If you cite with a Wayback Machine link when the original link still works, you're just creating needless work for a non-profit. An archive isn't a CDN.
> or at least creating
I agree on creating.
[delayed]
> they can also cite Wikipedia
Academics require primary source evidence - e.g., experimental or documentary. Wikipedia is a tertiary source; by policy it forbids primary sources and requires secondary sources.
Recently I did a deep dive on some minor historical event in the 1800s that I won't bother to go into detail here, except to say that it has been claimed by an activist editor who was made it a personal mission to re-frame the event as an example of "westerners" committing sexual violence against helpless "easterners".
While certainly this kind of story is common throughout history, in this case, there's absolutely no documentary evidence of the event, and what little evidence exists points in the other direction. This is not just the weight of evidence -- it's what all current historical experts believe. It's what all major encyclopedias say about the event. Nonetheless, the Wikipedia editor clings to a bad reference (in an extremely biased book) that makes an off-hand assertion. This assertion, in turn, is sourced with only a single citation to a 1960s-era historical novel that a) didn't actually support what the author claims, and b) essentially made up signficant details of the event in question, based on popular movies and rumors of the time. It's garbage all the way down.
Nevertheless, anyone who edits this article to reflect accurate current knowledge is immediately embroiled in a war with the editor, because unfortunately, there's no "trusted source" that says the opposite of the bad citation. And so the lie remains. To fight this battle would require escalating the issue and possibly getting the editor removed. Who has time?
Again, this is a minor historical event that nobody cares about, and you still cannot trust it as a source. I have seen enough things like this across Wikipedia that I no longer trust it for anything that might intersect with modern political opinions.
If you look up something on Wikipedia and include it in your work without citing it, it's an even worse pathway to citogenesis, since someone trying to check whether you can be used as a source for Wikipedia won't know that that's where you got it from in the first place.
Of course for most things you should have no need to refer to Wikipedia at all, and hence also no need to cite it.
> Even Wikipedia itself considers wikis (including itself) to lack sufficient authority to be cited as a source.
Those two things are not alike. Nobody can view themselves as sufficient authority to be cited as their own source.
(The late Victor Mair was an exception; he was happy to "support" his own claims by citing wikipedia articles that cited unsourced claims from him.)
What academics need to defend is wikipedia in latent space. Not walls of text that reflect the opinions of a few who have the time to edit that the rich and famous can optimize.
That is why we are building latentpedia.org.
Defense of open knowledge will appear when they attack the monopoly of Elsevier and other publishers through innovative systems for peer-evaluated papers, not by trying to resuscitate the body of the spook-filled Wikipedia.
Open knowledge systems like Wikipedia are not really defensible. Letting anybody edit any page is a great way to get AI and human slop everywhere. It makes it extremely easy to insert knowledge with dubious quality and sources.
We need "open access" knowledge systems with known authors who are sponsored by non-profit and educational organizations and its their duty to defend their reputation.
Humanity needs more openly accessible curated and factually checked information and "open to modify" systems are bad at in all.
Something like Wikipedia is a wonderful idea in theory, but it's tough to defend when their policies allow factually incorrect content to remain just because it appears on a third-party source that the editors consider reliable. Most media sources contain a pretty high percentage of fake news and when this slop makes its way into Wikipedia that makes the problem even worse. Can we find a better solution?
> it's tough to defend when their policies allow factually incorrect content to remain just because it appears on a third-party source that the editors consider reliable.
I feel this is the kind of discussion that would really benefit from a number of concrete examples.
Philosiphers have been trying to work on how to define what is "True" since forever without much to show for it.
Wikipedia tries to kick the bucket by going with what "reliable" secondary sources say. Thus the goal is not to say the capital T Truth, but to essentially sumnarize academic knowledge.
To steal a saying - its the worst method except for every other known method.
If anyone has a better idea how to determine what is true and what isn't - i'd certainly be very interested.
I think the Gell Mann effect applies on Wikipedia just as much as in the papers. I’ve often come across articles in an area where I’m an expert and they have issues. I used to try and have them changed, but many years ago I just got exhausted by fanatics who seem to have just gotten really attached to their (incorrect) prose.
Sorry but scientific method has been the default for "Truth" for many academic subjects. It is reliable and idempotent. If all of the human information and most of the knowledge workers are removed from the Earth tomorrow, many will arrive in the same conclusion in most of the natural sciences and maths. The language and representation will be different but prime numbers will still be prime numbers and the matter will still be made of one neutral and two "interactive" elements.
Social sciences like politics, history and economy are harder to replicate with the scientific method, however repeated observations, statiscial information as well as qualitative analysis with evidence-based research can still provide replicable analysis.
> If anyone has a better idea how to determine what is true and what isn't - i'd certainly be very interested
This kind of excuse is often cited to shut down any discussion, especially in history, politics and economics. I find it deeply anti-intellectual and dogmatic. As long as evidence and counter-evidence is followed, it should be possible to discuss alternative interpretations and solutions.
While I agree with your sentiment, I must add to this that to say that something is unequivocally true that holds in all conditions everywhere is difficult to ascertain in general. So we have to deal with that two (or more) things can be true at the same time (sometimes). This may be less the case for things such as prime numbers but more the case when the underlying principle is void of simple cause and effect (in the linear sense).
The scientific method can be used to falsify hypotheses. It cannot prove that something is true. Fields that use the scientific method typically rely on social consensus for determining the truth. Empirically speaking, the social consensus seems to converge to a reasonable approximation of the truth with a high probability, but the process can take decades.
I understand that "truth" and "facts" are squishy philosophical concepts at the edges, but I'm talking about something much more basic. There are Wikipedia pages which are flat out wrong (by any common definition of the term) and yet the editors won't allow any corrections because the primary sources are also wrong.
One thing that might help a bit would be to tag or color code specific statements with a confidence factor. So a statement sourced only from a single trash site like the NY Times or Fox News would be red, whereas another statement sourced from multiple independent scholarly journals would be green. Or something like that.
yeah and for some reason primary literature can't be used, except when it can, but review articles are allowed. makes no sense.
I guess Larry Sanger was right all along about experts vs amateurs?
Of course it has faults, but so did the Library of Alexandria. However, can anyone argue that Wikipedia is not our best attempt at collective facts?
Of course agreeing on collective facts is extremely messy.
Show me a better attempt, please.
As long as we acknowledge its limitations and biases
Fair to assume that the library of Alexandria contained books written by the most erudite scholars and philosophers of their day, not Joe and Jane and their cohort of terminally online gatekeepers.
Why is that fair to assume? "Books" were extremely rare then. Some random librarian had to include those books, and likely leaned towards taking all written works due to scarcity.
Likely some other librarian disagreed with some inclusions and certainly many readers disagreed with the content. They must have had a system for all of this. It was far less open than Wikipedia.
IIRC, they seized all scrolls from all ships in the port, in many languages. They likely had many "errors" according to other librarians and many readers.
Wikipedia has been the default result on Google for most things for years now.
Wikipedia is existentially threatened by AI, which does the same thing but better.
> most users of AI products do not click the links to Wikipedia, choosing instead to accept machine-generated outputs without further verification. If this practice continues, readership on Wikipedia will continue to decline.
This is the same mechanism by which Wikipedia drained readers from news outlets and academic papers that it cites for information.
I play the tiniest violin in the world for Wikipedia editors that now suffer from the same thing they did to the news media.