Hunting for Controversy in the Archives: Wikipedia, Truthiness, and Historical Research on Dialectics in the Modern West

I’ve been staying up all night shaking the virtual archive tree into dropping fruit. Humanities researchers have an increasing number of possible trees: collections of biographies, collections of images, machines for sorting texts, and machines for identifying parts of speech. Alas, any of these trees are only useful when we can ask good questions of them. Different researchers have different questions. The historian’s major problem -- discerning watersheds in social practice -- appears to defy such inquiries. But I might have found a place to start. When you’re looking for controversy over truth in the archives, the best place to start is where one group of thinkers tells another to shut up.

Here’s the question: can I ask an online archive to point me to the most controversial topics about politics between 1789 and 1832? That would be great, right? I’m not there yet; I can’t train my sophisticated search tools on the eighteenth/nineteenth-century collections yet. But I do have Wikipedia. How do I ask Wikipedia about the places where its volunteer-writers stand in line to punch each other?

Searching for Truthiness in Wikipedia

I started on a hunch, and that hunch involved one of my favorite twentieth-century words for starting a fight. The word “pseudoscience” was one of the twentieth century’s most powerful tools for turning a pleasant argument into an all-out jam-throwing mud-in-your-eye punch fest. Establish something as a “pseudoscience” and you dismiss the evidence offered out of hand. The processes to which pseudoscience refers are very modern: expert communities, institutional truth, public discourse, and the body of scientific thinking. Twentieth-century philosophers of science like Karl Popper of course fashioned us the tool, discerning communities of disciplinary inquiry, methods, and paradigm shifts as the symptom of science an its distance from its opposite, pseudoscience, where evidence could be marshaled into the simulacrum of established knowledge without the actuality of the same authority. To call something “pseudoscience” is therefore to challenge whether a conversation is relevant to the other people hanging out in the same era. Your sixteenth-century worldview, my friend says, It’s so pseudoscientific. Yes.

I suppose I’m attracted to things labeled pseudoscience in part because they represent communities whose thinking is out-of-time or out-of-place, and twenty-first-century interlocutors frequently have problems determining which and why. Take acupuncture. The elemental metaphors relating bodily organs to the seasons appear to be whacked out-of-time from the perspective of twentieth-century western science. But acupuncture’s reliance on communities of practice, responding to the experience of the patient, frequently surpasses western medicine in its skills of communicating with the patient, treating the physical ailment, the psychological, the psychosomatic, and the patient-doctor relationship as parts of an entire whole. If anything, acupuncture isn’t out-of-time, it’s out-of-place, the creature of a tradition external to the west and frequently misunderstood by western practitioners. That doesn’t stop western pracitioners from labeling acupuncture a “pseudoscience,” or cultural anthropologists from accusing the category “pseudoscience” of incorporating western bias. Pseudoscience is a great term for pointing to fissures.

Armed with nice, personalized search-engine, all of Wikipedia, and a few sort-and-visualize master tools, I set upon the canon of twentieth-century knowledge about science, determined to suss out Truthiness and subject it to the steely lens of machine-based AI.

Here’s what I did. I asked DevonAgent (a personalizable search agent for automatically pulling big sets of text from the web without cut and paste, ~$25 education rate) to find me all the Wikipedia articles that reference pseudoscience, either in the class heading, the article itself, or the backboard discussion. Those articles range from articles explaining Popper’s definition of pseudoscience to clearly discredited theories (phrenology, flat-earth theory) to controversial subjects from the borderlands of western institutional knowledge (e.g. acupuncture).

The resulting database of articles is a collection of knowledge fissures: places where one group of researchers has attempted to tell another to shush.

So I asked the machines who was involved in the fights. ManyEyes, a free collection of visualization tools from IBM, will let anyone look for recurring grammatical relationships between certain words in a given piece of text. When ManyEyes looked at the Wikipedia Pseudoscience database, it quickly recognized certain names coming up regularly. It knew that they were names only from the grammatical construction of a personal relationship, i.e., the word was followed by an apostrophe-s, e.g. Freud‘s book, Sheldrake’s claim, Popper’s hypothesis. So I asked ManyEyes for a list of the players who appear in the fights over pseudoscience. Here they are, to the best of the machine’s knowledge.



Next, I asked the machines to tell me about where such fights occurred. Sort of interesting. Irvine, California: home of pseudoscience! (or is it its refutation?)



Finally, I asked the machines to tell me about what the claims of pseudoscience were. The pseudoscience database was too fuzzy to produce good results, but a narrower database, a list of articles referencing “pseudoscience”, “modern”, and “history” successfully produced a tight list of articles about controversial scientific battles in their historic context. Once I had that database, I could ask for a list of phrases that began “claims that…”. Even more powerful was looking at those individual claims, for instance, statements about geology, taken from the set of sentences beginning with “the earth…” The resulting set is a fairly good list of discredited early-modern to modern- theories about the nature of the globe: the theory that the earth is flat, the myth of the hollow earth, and so on:



One can even usefully extrapolate the data from the pseudoscience data to a set of basic centers of consensus that comprise what c21 wikipedia users believe to be true of science in general:



* * *

Now the real gambit I'm after isn’t Wikipedia; it’s the revolutions of knowledge in the eighteenth- and nineteenth centuries.

I’m imagining searching for dialectics in discourse – fissures like the ones indicated by the word “pseudoscience.” I’m imagining being able to look into points when alternative intellectual discourse erupts.

The eighteenth- or nineteenth-century term wouldn’t be pseudoscience, of course. It might be “retrograde” for science; for politics it might be accusations of “interest.”

I spend a lot of my time reading endless pamphlet and newspaper wars, trying to figure out who the sides are in the railway debates and what they care about. While I don't think that I'll ever be freed from reading, algorithms like these promise to give me some of the keywords that can offer me a short cut direct to some of the major controversies in question. The paragraphs on railroads that accuses others of “interest” reference which place-names, for instance? I've had limited luck searching the nineteenth century for "Birmingham interest", although I read paragraphs about railway interests relating to Cornwall from time to time. I'd love a short-cut that takes me to, say, the unknown bulk of references to the "railway interest" in Darlington. Or again: What personal names come up most frequently? Or would the phrase "the claim that" turn up a list of counter contentions and contentious claims, once I fed in the top ten treatises from both side of the railway arguments? When enlightenment figures accuse each other of being “retrograde”, which claims are most violently under contention?

The result will hardly be a Skinnerian automatic history-generating machine, but it nevertheless offers an important tool. Finding the names and places associated with a controversy as common as the railroads, which I study, is time-consuming, the work of reading hundreds of treatises and following up every place and name in infinite keyword searches. A machine that can predict who the controversialists were and where the places they argued about were located would save the historical researcher untold time.

What I'm imagining is essentially a finding aid for zeroing in on *some* of the contentious claims, names, and interests from any given discourse. It will not eliminate the need to read in context, but it can offer some short-cuts straight to the heart of the controversy.

We’re only a few months away from having these tools and being able to apply them to pointing out controversies for a select set of texts. When we have them, we’ll be able to look at a set of all the texts that reference the phrase “political economy” published between 1820 and 1830, say, and then ask for the major personal names and place names.

What I need next: a series of plug-ins to make the DevonAgent search machine talk to eighteenth- and nineteenth-century databases (EEBO, ECCO, Making of the Modern World, Google Books, Hansard’s Debates, Parliamentary Papers, Nineteenth Century Newspapers, and so on). We’re guessing that it’ll cost about $200/each to hire a programmer to design each plug-in.

So: we’re passing the hat to historians with research budgets to collect the necessary funds. Interested? Get in touch. Know a DevonAgent programmer? We’re hiring. Rewriting the history of modern institutions of knowledge? Stick around; it’s about to get interesting.

(Cred where cred is due: Thx, Simon DeDeo, for helping me think through pseudoscince, and thx to my brilliant colleague Fredrik Albritton Jonsson, for helping me think through "retrograde" political economy and the tortured legacy of Quentin Skinner!!)