I remember being very interested in programming in middle/high school, but all the environments in our school computer lab had windows (this was in India), and I think at that time (maybe 2001-2003) I didn't even know there were other operating systems.
Our school was participating in something called International Cyber Olympiad, and of course I gave the eligibility exam.
They sent all students who passed a Knoppix Live CD to prepare for the actual competition. We did not have a PC at home until a couple of years later, but I used that CD in any PC I could find anywhere - the school computer lab, the school library computers, and my dad's office computers. It was my first experience with a Linux system (and I found it awesome). Also my first experience with gcc instead of borland c++.
After that I always had a CD wallet thing with copies of sysresccd and supergrubdisk and others (including I think an old knoppix cd from a linux magazine).
When I first started going towards Linux I tried, in this order:
* Puppy linux, because I liked puppies.
* College linux, because it was for education, and I was in secondary school, and college sounded fancy.
* Adriane Knoppix, because it's what came up when you did a web search for "knoppix download" -- that was interesting, if you didn't know, ADRIANE is for blind people.
* Whoppix (which became Whax) -- because I could actually find the download.
* Backtrack linux (because that was apparently better than WHAX)
* Slackware, because backtrack was based on this and "only script kiddies use Backtrack".
I did the same as you, tried to keep things to liveCDs but I always got the urge to install them, so would do it periodically until everything broke. This also meant I had to deal with whatever was broken (usually wifi).
One thing I remember very fondly though, which isn't a linux, is the leaked Geek Squad rescue CD... I'd give a decent chunk of change for an updated one of those..
I remember there being a sliding puzzle game in the theme of assembling molecules. I remember this because I remember a very classic argument between two teenagers over "propene" being a typo of "propane" vs. being an actual chemical. If only they were sitting in front of a device that could help them find the answer.
Since then, a lot of Live Linux distros emerged, with various features offered; Debian got a much better installer; and then Knoppix dropped KDE Plasma as their desktop environment. All of that made me to move over to better "Live Linux" distros.
I remember using this when it first came out. It was a game changer for doing forensics back before full disk encryption was a common thing.
Enter Knoppix and persisting any state I cared about on a thumb drive.
Of course, since RAM was so limited on devices, just installing packages and leaving the modifications taking up valuable RAM was inconvenient to do, so I went down a rabbit hole of customizing the image builds with various nonsense.
Useful dozens of other times before Ubuntu popularized live images just being a thing you supplied as table stakes, but that window of going down a customization rabbit hole and running a diskless laptop is what I remember.