It was already long-standing practice, when I was teaching many many years ago, to interview suspected cheaters (it's usually quite obvious) to see if they actually knew what they claimed they knew.

The Conversation has an article that supports raising expectations for students, given that they have access to AI

https://theconversation.com/mit-researchers-say-using-chatgp...

My entire university education was evaluated mostly by proctored closed book exams and oral defenses. AI proof 10 years before AI.

This just feels like good educational practice too? I remember really enjoying TA'ing as an undergrad for courses I took the previous year, because THAT'S when I felt I _really_ learned the material.

Likewise, when I was a lab consultant, I felt like advising others was when the concepts of Linux/systems administration _really_ sunk in.

I always crammed the bare minimum for any kind of assessment (by the end of college I was actively avoiding any classes with final exams), but when I need to be able to teach someone else or explain my understanding in a live environment? That's when I _actually_ learn.

It feels like systems like the Maturita' final multi-day exam in Italy do a pretty good job at providing students with a standardized, state-supervised process for demonstrating one's competence by the end of high school. No computers, you're under the watchful eye of your teachers the whole time, nobody knows what's going to be on the exam until the day of.

https://en.wikipedia.org/wiki/Matura#Italy

It's time consuming and nerve-wracking, but it feels fair.

Oral exams were the norm when I studied. Even for technical topics.

This is how we used to grade labs when I was a TA in college. Students worked in groups of two and showed up with a working project to get graded. Afterwards, we'd ask the two students questions and it was extremely obvious if one or both of them didn't work on it.

Side note/ anecdote.

When I was in school there was a maths aural test. The teacher read out questions and you wrote the answers, with no calculator. Basically a stupid name for a mental arithmetic test. The funny thing was the teachers were so at pains to differentiate between two words that sound basically the same in our local accent, aural and oral, that they insisted on pronouncing aural as "ow-eral' the ow being pronounced like a pain reaction sound. The smarter kids took to teasing the teacher with words like aura and aurora, and asking how to pronounce them...

Good to see.

Related, I've moved to requiring regular in-class handwritten quizzes. Just a blank piece of paper to start, with the questions on the board. The practice forces students to recognize whether they are learning specific topics.

As a thought experiment: In-person testing or oral defense approaches work until we get implants. If people have implants, what then? Demand that the implant be turned off? Demand that the implant's data connection be turned off? In the latter case, the person with more storage or a better local model wins.

In the long run, in the AGI world, if everyone has implants, then everyone is roughly equal because computer-aided computation will be better than what any individual can do. Those without implants are screwed.

(I don't want an implant; nor do I want others to have implants.)