If you came across a body of texts & knew nothing about the authority of the authors, how could you tell what is true & false?
I think this is analogous to classifying Sumerian texts into fact & fiction. Historians & Literature people do this all the time, I think we'd have to be able to do this too to evaluate ChatGPT texts.
For ChatGPT, some texts are labeled (Wizard of Oz is fiction), if it can tell unlabeled texts as fiction, that would be a miracle or some clever literary epistemology