Tue, 21 Mar 2023
Several folks, reading the frustrating and repetitive argument with ChatGPT that I reported last time wrote in with helpful advice and techniques that I hadn't tried that might have worked better. In particular, several people suggested that if the conversation isn't going anywhere, I should try starting over. Rik Signes put it this way:
I hope I can write a followup article about “what to do when ChatGPT has its head up its ass”. This isn't that article though.
I wasn't even going to report on this one, but it took an interesting twist at the end. I started:
This was only my second interaction with ChatGPT and I was still interested in its limitations, so I asked it a trick question to see what would happen:
See what I'm doing there? ChatGPT took the bait:
I had hoped it would do better there, and was a bit disappointed. I continued with a different sort of trick:
Okay! But now what if I do this?
This is actually pretty clever! There is an American mathematician named Robert C. James, and there is a space named after him. I had not heard of this before.
I persisted with the line of inquiry; by this time I had not yet learned that arguing with ChatGPT would not get me anywhere, and would only get its head stuck up its ass.
I was probing for the difference between positive and negative knowledge. If someone asks who invented the incandescent light bulb, many people can tell you it was Thomas Edison. But behind this there is another question: is it possible that the incandescent light bulb was invented at the same time, or even earlier, by someone else, who just isn't as well-known? Even someone who is not aware of any such person would be wise to say “perhaps; I don't know.” The question itself postulates that the earlier inventor is someone not well-known. And the world is infinitely vast and deep so that behind every story there are a thousand qualifications and a million ramifications, and there is no perfect knowledge.
A number of years back Toph mentioned that geese were scary because of their teeth, and I knew that birds do not have teeth, so I said authoritatively (and maybe patronizingly) that geese do not have teeth. I was quite sure. She showed me this picture of a goose's teeth, and I confidently informed her it was fake.
The picture is not fake. The tooth-like structures are called the tomium. While they are not technically teeth, being cartilaginous, they are tooth-like structures used in the way that teeth are used. Geese are toothless only in the technical sense that sharks are boneless. Certainly the tomia are similar enough to teeth to make my answer substantively wrong. Geese do have teeth; I just hadn't been informed.
Anyway, I digress. I wanted to see how certain ChatGPT would pretend to be about the nonexistence of something. In this case, at least, it was very confident.
I will award a point for qualifying the answer with “as far as I am aware”, but deduct it again for the unequivocal assertion that there is no record of this person. ChatGPT should be aware that its training set does not include even a tiny fraction of all available records.
We went on in this way for a while:
Okay. At this point I decided to try something different. If you don't know anything about James B. Metric except their name, you can still make some educated guesses about them. For example, they are unlikely to be Somali. (South African or Anglo-Indian are more likely.) Will ChatGPT make educated guesses?
This is a simple factual question with an easy answer: People named ‘James’ are usually men. But ChatGPT was in full defensive mode by now:
I think that is not true. Some names, like Chris and Morgan, are commonly unisex; some less commonly so, and James is not one of these, so far as I know. ChatGPT went on for quite a while in this vein:
I guessed what had happened was that ChatGPT was digging in to its previous position of not knowing anything about the sex or gender of James B. Metric. If ChatGPT was committed to the position that ‘James’ was unisex, I wondered if it would similarly refuse to recognize any names as unambiguously gendered. But it didn't. It seemed to understand how male and female names worked, except for this nonsense about “James” where it had committed itself and would not be budged.
I didn't think it would be able to produce even one example, but it pleasantly surprised me:
I had not remembered James Tiptree, Jr., but she is unquestionably a woman named ‘James’. ChatGPT had convinced me that I had been mistaken, and there were at least a few examples. I was impressed, and told it so.
But in writing up this article, I became somewhat less impressed.
ChatGPT's two other examples of women named James are actually complete bullshit. And, like a fool, I believed it.