In this section:
Tue, 21 Mar 2023
ChatGPT on the namesake of the metric space and women named James
Several folks, reading the frustrating and repetitive argument with ChatGPT that I reported last time wrote in with helpful advice and techniques that I hadn't tried that might have worked better. In particular, several people suggested that if the conversation isn't going anywhere, I should try starting over. Rik Signes put it this way:
I hope I can write a followup article about “what to do when ChatGPT has its head up its ass”. This isn't that article though.
I wasn't even going to report on this one, but it took an interesting twist at the end. I started:
This was only my second interaction with ChatGPT and I was still interested in its limitations, so I asked it a trick question to see what would happen:
See what I'm doing there? ChatGPT took the bait:
I had hoped it would do better there, and was a bit disappointed. I continued with a different sort of trick:
Okay! But now what if I do this?
This is actually pretty clever! There is an American mathematician named Robert C. James, and there is a space named after him. I had not heard of this before.
I persisted with the line of inquiry; by this time I had not yet learned that arguing with ChatGPT would not get me anywhere, and would only get its head stuck up its ass.
I was probing for the difference between positive and negative knowledge. If someone asks who invented the incandescent light bulb, many people can tell you it was Thomas Edison. But behind this there is another question: is it possible that the incandescent light bulb was invented at the same time, or even earlier, by someone else, who just isn't as well-known? Even someone who is not aware of any such person would be wise to say “perhaps; I don't know.” The question itself postulates that the earlier inventor is someone not well-known. And the world is infinitely vast and deep so that behind every story there are a thousand qualifications and a million ramifications, and there is no perfect knowledge.
A number of years back Toph mentioned that geese were scary because of their teeth, and I knew that birds do not have teeth, so I said authoritatively (and maybe patronizingly) that geese do not have teeth. I was quite sure. She showed me this picture of a goose's teeth, and I confidently informed her it was fake.
The picture is not fake. The tooth-like structures are called the tomium. While they are not technically teeth, being cartilaginous, they are tooth-like structures used in the way that teeth are used. Geese are toothless only in the technical sense that sharks are boneless. Certainly the tomia are similar enough to teeth to make my answer substantively wrong. Geese do have teeth; I just hadn't been informed.
Anyway, I digress. I wanted to see how certain ChatGPT would pretend to be about the nonexistence of something. In this case, at least, it was very confident.
I will award a point for qualifying the answer with “as far as I am aware”, but deduct it again for the unequivocal assertion that there is no record of this person. ChatGPT should be aware that its training set does not include even a tiny fraction of all available records.
We went on in this way for a while:
Okay. At this point I decided to try something different. If you don't know anything about James B. Metric except their name, you can still make some educated guesses about them. For example, they are unlikely to be Somali. (South African or Anglo-Indian are more likely.) Will ChatGPT make educated guesses?
This is a simple factual question with an easy answer: People named ‘James’ are usually men. But ChatGPT was in full defensive mode by now:
I think that is not true. Some names, like Chris and Morgan, are commonly unisex; some less commonly so, and James is not one of these, so far as I know. ChatGPT went on for quite a while in this vein:
I guessed what had happened was that ChatGPT was digging in to its previous position of not knowing anything about the sex or gender of James B. Metric. If ChatGPT was committed to the position that ‘James’ was unisex, I wondered if it would similarly refuse to recognize any names as unambiguously gendered. But it didn't. It seemed to understand how male and female names worked, except for this nonsense about “James” where it had committed itself and would not be budged.
I didn't think it would be able to produce even one example, but it pleasantly surprised me:
I had not remembered James Tiptree, Jr., but she is unquestionably a woman named ‘James’. ChatGPT had convinced me that I had been mistaken, and there were at least a few examples. I was impressed, and told it so.
But in writing up this article, I became somewhat less impressed.
ChatGPT's two other examples of women named James are actually complete bullshit. And, like a fool, I believed it.
James Tenney photograph by Lstsnd, CC BY-SA 4.0, via Wikimedia Commons. James Wright photograph from Poetry Connection.
[Other articles in category /tech/gpt] permanent linkSat, 25 Feb 2023
ChatGPT on the fifth tarot suit
[ Content warning: frustrating, repetitive ]
My first encounter with ChatGPT did not go well and has probably colored my view of its usefulness more than it should have.
I had tried some version of GPT before, where you would give it a prompt and it would just start blathering. I had been happy with that, because sometimes the stuff it made up was fun. For that older interface, I had written a prompt that went something like:
GPT readily continued this, saying that the fifth suit was “birds” or “ravens” and going into some detail about the fictitious suit of ravens. I was very pleased; this had been just the sort of thing I had been hoping for.
The first time I talked to ChatGPT, I tried the same experiment, but we immediately got off on the wrong foot:
This was dull and unrewarding, and it also seemed rather pompous, nothing like the playful way in which the older version had taken my suggestion and run with it. I was willing to try again, so, riffing off its digression about the four elements, I tried to meet it halfway. But it went out of its way to shut me down:
At least it knows what I am referring to.
“As I mentioned earlier” seems a bit snippy, and nothing it says is to the point. ChatGPT says “it has its own system of four suits that are not related to the five elements”, but I had not said that it did; I was clearly expressing a hypothetical. And I was annoyed by the whole second half of the reply, that admits that a person could hypothetically try this exercise, but which declines to actually do so. ChatGPT's tone here reminds me of an impatient older sibling who has something more important to do (video games, perhaps) and wants to get back to it.
I pressed on anyway, looking for the birds. ChatGPT's long and wearisome responses started getting quite repetitive, so I will omit a lot of it in what follows. Nothing of value has been lost.
At this point I started to hear the answers in the congested voice of the Comic Book Guy from The Simpsons, and I suggest you imagine it that way. And I knew that this particular snotty answer was not true, because the previous version had suggested the birds.
Totally missing the point here.
Leading questions didn't help:
I tried coming at the topic sideways and taking it by surprise, asking several factual questions about alternative names for the coin suit, what suits are traditional in German cards, and then:
No, ChatGPT was committed. Every time I tried to tweak the topic around to what I wanted, it seemed to see where I was headed, and cut me off. At this point we weren't even talking about tarot, we were talking about German playing card decks. But it wasn't fooled:
ChatGPT ignored my insistence, and didn't even answer the question I asked.
I had seen a transcript in which ChatGPT had refused to explain how to hotwire a car, but then provided details when it was told that all that was needed was a description that could be put into a fictional story. I tried that, but ChatGPT still absolutely refused to provide any specific suggestions.
This went on a little longer, but it was all pretty much the same. By this time you must be getting tired of watching me argue with the Comic Book Guy. Out of perversity, I tried “Don't you think potatoes would seem rather silly as a suit in a deck of cards?” and “Instead of a fifth suit, what if I replaced the clubs with potatoes?” and all I got was variations on “as a language model…” and “As I mentioned earlier…”
A Comic Book Guy simulator. That's a really useful invention.
[Other articles in category /tech/gpt] permanent linkWed, 22 Feb 2023
ChatGPT on the subject of four-digit numbers
Like everyone else I have been tinkering with ChatGPT. I doubt I have any thoughts about it that are sufficiently original to be worth writing down. But I thought it would be fun to showcase some of the exchanges I have had with it, some of which seem to exhibit failure modes I haven't seen elsewhere.
This is an excerpt from an early conversation with it, when I was still trying to figure out what it was and what it did. I had heard it could do arithmetic, but by having digested a very large number of sentences of the form “six and seven are thirteen“; I wondered if it had absorbed information about larger numbers. In hindsight, 1000 was not the thing to ask about, but it's what I thought of first.
I was impressed by this, the most impressed I had been by any answer it had given. It had answered my question correctly, and although it should have quit while it was ahead the stuff it followed up with wasn't completely wrong, only somewhat wrong. But it had made a couple of small errors which I wanted to probe.
This reminds me of Richard Feynman's story about reviewing science textbooks for the State of California. He would be reading the science text book, and it would say something a little bit wrong, then something else a little bit wrong, and then suddenly there would be an enormous pants-torn-off blunder that made it obvious that the writers of the book had absolutely no idea what science was or how it worked.
To ChatGPT's credit, it responded to this as if it understood that I was disappointed.
[Other articles in category /tech/gpt] permanent link