Claude, Language, and Work – Le Minh Khai's Southeast Asian History Blog (+ More)

In the late 1990s, I remember that I used the Netscape browser. Then Microsoft Internet Explorer came along and then Firefox. . . At some point I “jumped ship” and stopped using Netscape, but I remember that before I did that, there was a time when I knew that another browser was probably better, but I still kept using Netscape anyway, mainly out of habit and attachment.

So, I’ve been using ChatGPT more or less from the outset, and I have kept hearing things about Claude, and how “it’s better for humanities work” as “its language is more natural,” etc. However, I’ve kept using ChatGPT, but I’ve long had this feeling that I might be in “Netscape-mode,” where I’m using something because I’m used to it, but not because it is the best.

Therefore, a week or so ago I started to use Claude (the $20 version). Actually, signing up and paying was such a pain in the arse that I almost gave up, but eventually I got in. Looking online (Reddit), I saw that many other people have encountered the same difficulties that I did.

So, for a company of Anthropic’s size and stature, the fact that you have to struggle to sign up and get it to take your money didn’t impress me all that much.

Then I interacted with its “natural-sounding language.” I engaged in a conversation where I was trying to think about what the consequences will be if scholars steelman their writings by having LLMs assess them and give advice on how to strengthen the papers, and then reviewers go on to review those writings with LLMs as well.

The response was “If author and reviewer consult the same oracle, review becomes a null operation on everything the oracle catches and adds nothing on everything it misses.”

Ummm. . . what?!!

Does that make sense?

People are consulting “oracles”?

So, if I understand this statement correctly, it is saying that having an author steelman an argument by running a paper through an LLM and then having a reviewer review it by running it through an LLM will not lead to much because the author and reviewer will both be consulting the same “oracle”. . .

However, Claude did seem to think that running a paper through an LLM is still a good idea, as it said,

“One genuine upside worth not underselling: the cost of getting a critical read drops to near zero, and that matters most for people without a department of colleagues. Someone working alone, outside an institution, in a field with a dozen practitioners, now has something to push against.”

Then it went on to say,

“The thing I’d most want to guard against, if I were you: treating an AI pass as clearing a draft rather than as one more reader with a peculiar and knowable bias — omniscient about the articulated, blind to the unarticulated, and systematically biased toward whatever has been written about most. Best used before the argument is settled, as a way to find out what the standard objections will be, and worst used at the end, as a certificate.”

OK, somewhere in here is a logical point, but THE LANGUAGE!! I have heard so many people tell me that Claude produces text that sounds “more natural.”

In what world is “omniscient about the articulated” considered “natural-sounding” language? I understand that marijuana is now legal in California. Did Anthropic hire a bunch of stoners to do their RLHF (Reinforcement Learning from Human Feedback)?

Again, I went on to Reddit, and people were talking about the same issue.

OK, so I now see that you have to tell Claude to “respond in plain language” (or to input that as an instruction). And, I have been working with it to try to steelman my arguments.

I have previously done this with ChatGPT, and I thought that worked decently; however, Claude either does it at another level, or maybe I just keep forgetting to tell it to respond in plain language and, therefore, it feels like it is tougher, as it more or less insults and humiliates me each time it responds.

Nonetheless, after a week or so in “Claude Bootcamp,” I’m starting to notice some things.

1) For a long time, we (historians/scholars) have had it easy. There is only so much that our brains can do to figure out how to improve a paper, and there is only so much time/space/effort that a reviewer has to point out how our paper could be better, and there is only so much that we do in response to that advice.

With an LLM, and especially with that $%^@ Claude, there is no end. It’s literally a war of attrition that you, the author, can never win. You can only keep doing it until you surrender, otherwise you will physically collapse or your brain will explode.

That said, it’s very interesting to participate in that battle. I’ve learned a ton.

Nonetheless, what I can see is that in this respect AI is making my work “more difficult.” I get to the point where I think I have achieved something, and then Claude cuts me down to size, chews me up and spits me out. . . over and over and over and over again.

“OK, I’ll finish this paper this morning” turns into days of back and forth, as I’m forced to re-examine sources, check new ones, reformulate my argument, etc., to say nothing of all of the tiny typos and errors that it keeps finding.

I have heard the same thing about coders (the ones who still have jobs). AI enables them to do more, so they end up working more.

2) I fed Claude a paper I have already published and asked it to assess it. It, of course, found things to critique, but then it said the following:

“If Kelley pre-cleared that paper, he’d add a paragraph acknowledging localization, a hedge on the tu argument, a note that silence cuts both ways. The paper gets harder to attack. Whether it gets truer is a separate question, and I’d guess mostly not — a pre-empting paragraph is not the same as an engagement. There’s also a real cost: the paper’s usefulness partly is its overreach. It’s a polemic, its errors are legible, and in a field this small the overstatement is probably what gets someone to go check the Việt sử lược for themselves. A fully hedged version might advance the field more slowly.”

After all the beating I’ve taken from Claude, I was pleased to see a kind of compliment: “Your argument is crap, but in the small field you are in, sometimes you need to throw some dirt around to get people moving.”

That’s basically been my philosophy for my entire career. 😊

However, when I ask Claude to assess my writing, I have the sincere desire to strengthen my argument. That’s why I keep fighting the war of attrition, and as I do so, what starts out as a polemic gets transformed into. . . perhaps just another scholarly paper that no one will ever read?

That’s the sense I have gotten from working with Claude.

It’s like it sucks you into a void or a vortex, where you work and work and work to make your writing and ideas better and better and better. . . and meanwhile, a week has passed. . . and the result?

– A paper that has an argument that is much more solid than any I could have created on my own or with the feedback of reviewers, but which will no longer ruffle any feathers.

And, I’m still in the same world where very few scholarly works get read, let alone responded to. . . and a week just went by.

BUT, I am definitely feeling “omniscient about the articulated.”