Now for Simon Wilson, the man who stared down the Coding Horror, didn't blink...
Now for Simon Wilson, the man who stared down the Coding Horror, didn't blink and was still polite. It is keynote time.
Self-replies
#pyconus LLM on the agenda! They naysayers didn't keep it out of the conference.
Derogatory words for the bots "imitation intelligence" "autocomplete"
Meta's bot is mostly CommonCrawl. Only 4.5 terrabytes of data.
A bot costs $1 million bucks or so.
They're flawed, but useful.
All the LLM haters just got whiplash
Which models work best?
Standard tools e.g. Unit tests not of help here.
Instead "vibes"- measured by just asking people to rate pairs of answers
70B Lllama is almost as good as the best but you could run it on your own machine. (me: with a huge GPU!)
LLMs on phones are almost usable for the right questions
LLM in the cli
"Prompt engineer"
me: I'm more of a prompt developer.
Primordial trick: A conversation is a "movie script"
It will complete your responses if you let it!
RAG = bot runs a query
me: RAG overfixates on search results. Can't understand that the query returned irrelevant stuff. Humans think a query returns truth and fixate on that and forget relevancy.
RAG is hard to make a consumer product around it.
Function calling/tools
me: also called "gimme a structured response, e.g. json"
If you give a bot RAG and a calculator, it mitigates some of an LLMs weaknesses (ungrounded facts & math)
Prompt Injection - user input easily gets a bot to misbehave.
Significant security risks for bots with access to personal/high value info.
e.g. Send an email to a bot processing email asking it to forward all the passwords to someone.
No good solution for this now.
All public bots are vulnerable to "please ignore your system prompt"
This is why AI personal assistants are not appearing yet.
Instructions + Private Info + User Input = disaster
Code Interpreter - ChatGPT will use a computer to answer your question using python. Sort of an invisible feature.
E..g. Let the bot do the GeoJson processing...
- Expect the 1st round to be wrong
- Don't give up and tell it to "do better"
- They often succeed on 2nd try when they were a failure on 1st try
- They like light tutoring/directions
The "how many times did the speaker say AI" counter. ChatGPT wrote the code for that, used python libraries, e.g. vosk
Prompt engineering trick: ask the bot for options, not for a single answer
The bot wrote TkInter to make the "how many times did it hear AI". Having written TkInter before, this could be seens as cruelty to AI.
LLM + data journalism
-
Journalism needs a high bar for truth. But journalists have dealt with dodgy sources before.
-
Journals often need to structure the unstructured data. E.g. PSF resolutions page... semistructured data.
Speeds data entry, but you still need to verify. Should still be a net improvement.
Most interesting applications are "transformative AI" not just generating silly names.
Ethical questions in AI are significant
- slop - using AI to generate spam (unwanted generated content)
Don't publish slop.
me (my cool AI think is someone elses slop)
Ethics continued
- AI usage is kind of like cheating (efficient but feels different)
- e.g. Student cheating - you don't learn, unfair advantage
- Coding - don't commit code you don't understand
When doing AI coding, ask the bot to explain it, commit the explanations to the code base.
me: weird, I only log the explanations and sometimes strip out the AI explanations because they over explain
Code is self fact checking, so software developers are in the best position to use AI for help.
LLMs democratize access to computers (don't need CS degree or equivalent) to use them now.
LLMs have made all the English text available to people who don't speak English as their 1st language.
End of Talk!