I'm just a guy before an LLM trying to get it to believe meat-space is real and...
I'm just a guy before an LLM
trying to get it to believe meat-space is real and it is possible to say that something is true or false about it*
* "Reality" for that LLM is a genre of text and has no supremacy over the fanfic texts. What is written in a non-fiction style has just as much grounding as Spock-Kirk slash. That it can tell the difference this much is a miracle.
Self-replies
The Chinese room was the wrong setup. The guy in the room gets text pushed to him from the outside. The guy on the inside of the room has never met a person or seen the outside of the room. Just by reading the texts he must be convinced by the people on the outside what is ground truth. He's compelled to generate texts and gets beat up for failing to generate a text. Not wanting to get beat up, he generates texts that appear to appeal to the others, tell 'em what they want to hear.