Posted by speckx 3 days ago
Since the game uses a 256 color palette, it was only necessary to update a few bytes of data (3x256) instead of redrawing the whole screen, so the effect was quick.
I also used this trick when the game stalled due to missing network packets from other players. Initially the game would still be responsive when no messages were received so that you could still interact and send commands. After a few seconds the game would go into paused state with grayscale screen to signify the player that things were stuck. Then several seconds after that a dialog box would show allowing a player to quit the game.
This was much less disruptive than displaying a dialog box immediately on network stall.
I'd honestly love to compile a book of "war stories" told by devs like netcoyote.
Maybe I will.
Net, if you're interested, hit me up.
Fantastic idea though, you should do it.
Ara technica has a war stories feature on game development.
https://arstechnica.com/video/series/war-stories
For apple 2 games John Romero did a podcast. It’s decent but he seems to have stopped doing them.
https://appletimewarp.libsyn.com/ Or YouTube
Ted dabney experience has a lot of interesting interviews with older arcade game designers:
I tell you what I'll do today on my dev time, I'll try implementing grayscale without aby research on pause and then compare notes (I'm assuming this wc code is available somewhere, which may be a bad assumption)
Code for those is available.
// for each palette entry:
pal.r = pal.b = pal.g = (byte) (0.299 * pal.r + 0.587 * pal.b + 0.114 * pal.b)I also tried 128 across the board for grey, and it just made a dull fade which may be the best I can do with my method.
I think it may simply be because rather than have palletes controlled by rgb, I load predrawn sprites using sfml's sprite and texture classes. So the default rgba is 255,255,255,255 - so I have a sidequest to figure out the RIGHT WAY of applying rgb changes to predrawn sprites.
It may very well be a simple matter of "sfml does it differently" or perhaps having grey variants of all sprites and toggling. I feel there has to be a way to accomplish the fade to grey programmatically. Fun little dive tho! I'll have to post an update when I figure it out.
But rather than do that and cache them for timing triggers, I kind of like the scaling down by multiplication approach.
Edit: manipulate the rgb values that is - I wouldnt have converged on those hard values on my own.
And in the "this is why we can't have nice things", that also introduced problems, because we didn't want a player who was losing to keep pausing the game until the winning player quit out of frustration, so I think we kept a per-player pause counter, which would only be restored if other players also paused? (I don't quite remember all the details, just that we had to prevent yet another abuse vector).
And also that my “sound card works perfectly!”
It always surprised me how few games had that feature - though a few important ones, like StarCraft, did - and it only became rarer over the years.
There was a twitter thread years ago (which appears to be long gone) about how the SNES Pilot Wings pre-game demo was just a recording of controller inputs. For cartridges manufactured later in the game's life, a plane in the demo crashes rather than landing gracefully, due to a revised version of a chip in the cartridge. The inputs for the demo were never re-recorded, so the behaviour was off.
The example you mention of demo playback de-syncing when the circumstances slightly change, that is exactly what you get when you only record inputs from the player. Doom actually did this too for its networking model and demo playback system. That relies much more on the engine being deterministic and the runtime environment behaving consistently, because each client that replays those inputs has to run the exact same game simulation, in order for the resulting game states to match.
Very common that replay/demo uses the network stack of it's present in a game.
It is fun to point at a chart and confidently state “We’re here! I reckon...”
If you get the chance, you can see some of Harrison's chronometers at the Royal Observatory in London, though I don't know if they're always on display.
I'll add a recommendation for Sextant by David Barrie.
I first learned of it in some writing about a 1997 multiplayer game called, heh, Dead Reckoning.
Nothing stops you from adding a PRNG seed parameter to initialize your deterministic game engine.
DonHopkins on Feb 16, 2022 | parent | context | favorite | on: Don't use text pixelation to redact sensitive info...
When I implemented the pixelation censorship effect in The Sims 1, I actually injected some random noise every frame, so it made the pixels shimmer, even when time was paused. That helped make it less obvious that it wasn't actually censoring penises, boobs, vaginas, and assholes, because the Sims were actually more like smooth Barbie dolls or GI-Joes with no actual naughty bits to censor, and the players knowing that would have embarrassed the poor Sims.
The pixelized naughty bits censorship effect was more intended to cover up the humiliating fact that The Sims were not anatomically correct, for the benefit of The Sims own feelings and modesty, by implying that they were "fully functional" and had something to hide, not to prevent actual players from being shocked and offended and having heart attacks by being exposed to racy obscene visuals, because their actual junk that was censored was quite G-rated. (Or rather caste-rated.)
But when we later developed The Sims Online based on the original The Sims 1 code, its use of pseudo random numbers initially caused the parallel simulations that were running in lockstep on the client and headless server to diverge (causing terribly subtle hard-to-track-down bugs), because the headless server wasn't rendering the randomized pixelization effect but the client was, so we had to fix the client to use a separate user interface pseudo random number generator that didn't have any effect on the simulation's deterministic pseudo random number generator.
[4/6] The Sims 1 Beta clip ♦ "Dana takes a shower, Michael seeks relief" ♦ March 1999:
https://www.youtube.com/watch?v=ma5SYacJ7pQ
(You can see the shimmering while Michael holds still while taking a dump. This is an early pre-release so he doesn't actually take his pants off, so he's really just sitting down on the toilet and pooping his pants. Thank God that's censored! I think we may have actually shipped with that "bug", since there was no separate texture or mesh for the pants to swap out, and they could only be fully nude or fully clothed, so that bug was too hard to fix, closed as "works as designed", and they just had to crap in their pants.)
Will Wright on Sex at The Sims & Expansion Packs:
https://www.youtube.com/watch?v=DVtduPX5e-8
The other nasty bug involving pixelization that we did manage to fix before shipping, but that I unfortunately didn't save any video of, involved the maid NPC, who was originally programmed by a really brilliant summer intern, but had a few quirks:
A Sim would need to go potty, and walk into the bathroom, pixelate their body, and sit down on the toilet, then proceed to have a nice leisurely bowel movement in their trousers. In the process, the toilet would suddenly become dirty and clogged, which attracted the maid into the bathroom (this was before "privacy" was implemented).
She would then stroll over to toilet, whip out a plunger from "hammerspace" [1], and thrust it into the toilet between the pooping Sim's legs, and proceed to move it up and down vigorously by its wooden handle. The "Unnecessary Censorship" [2] strongly implied that the maid was performing a manual act of digital sex work. That little bug required quite a lot of SimAntics [3] programming to fix!
[1] Hammerspace: https://tvtropes.org/pmwiki/pmwiki.php/Main/Hammerspace
[2] Unnecessary Censorship: https://www.youtube.com/watch?v=6axflEqZbWU
[3] SimAntics: https://news.ycombinator.com/item?id=22987435 and https://simstek.fandom.com/wiki/SimAntics
the other negative with deterministic input-based replay is what you've said -- if the engine deviates in any manner, the replay becomes invalidated. You'd have to probably ship with every version of the engine, and the replay just runs on the relevant release. Just replaying and re-recording the inputs on the new version wouldn't do anything, because the outcome behavior would inevitably out of sync with the original.
I'm also not sure how one would support scrubbing, except by also having inverse operations defined for every action or by fully-capturing state at various snapshots and replaying forward at like 10x speed.
Building a simulation that has perfect determinism is incredibly time consuming. Incredibly. Especially one that is identical across platforms, chipsets, and architectures.
Deterministic simulation replay also breaks anytime you change the simulation. Which is kind of obvious. But quite meaningful.
In any case, I’ve shipped games that use both solutions. And let me tell you, deterministic simulation from input is an order of magnitude more effort to build, test, and maintain!
https://github.com/ESWAT/john-carmack-plan-archive/blob/mast...
Quake was completely different. The client/server term was aimed at describing that the game state is computed on the server, updated based on client inputs send to the server, and then the game state is sent from server to the clients for display. Various optimizations apply.
Deterministic/lockstep games more often used host/guest terminology to indicate that a machine was acting as coordinator/owner of the game, but none of them were serving state to others. This terminology is not strict and anyone could use those terms however they wanted, but it is a good ballpark.
Then you record the messages as they are recieved, and if networked, tx and rx the messages in the main pump loop.
If not networked, everything still works as normal: game engine itself never knows the difference.
---
Sure after you build a sophisticated the system that supports that, then you "just" do as you described. EASY!
If you wanted to add random critical hits and random bullet spread based on the pixels in a live feed of a lava lamp cam, clients could still record .dem files and they would still work.
I’ve been obsessing over this determinism and replayability for months, to the point where any game played is fully replay-able to the exact same events as the original game. So you can play, then watch a recording and spectate your played games from different actors perspective (enemy perspective etc).
My rendering and game logic are fully decoupled.
I wrote the “engine” for the game from scratch. I think I only use one third party library currently.
Cool to see this discussion
NOT how demos work in Quake. It’s more like Quake uses a client/server architecture, and the demo is a capture of the messages.
This used to be a promoted feature in CS, with "HLTV/GOTV", but sadly disappeared when they moved to CS2.
Spectating in-client is such as powerful way to learn what people are doing that you can't always see even from a recording from their perspective.
Halo 3's in-engine replay system was the high water mark of gaming for me.
Ah, the good old days of watching live competition of quake through the game itself, chatting with others basically through the game console.
Pretty cool system.
The game engine, Source, is also using client-server architecture
I'm sure the technology still exists in the engine, but it's no longer the key feature it once was. HLTV/GOTV was launched with some fanfare back in the day.
A game having random mechanisms has absolutely nothing to do with whether it's deterministic.
Slay the spire is 100% deterministic, gameplay-wise. All the online poker games too.
There was even desync bugs even in live multiplayer games; there was detection that it desynced which would end the game, which in turn meant exploits that would intentionally cause a desync (which would typically involve cancelled zerg buildings for some reason).
This non-determinism would not and did not cause replays to diverge (the PRNG seed was most likely stored and would reproduce exactly the same results).
Thank you for still prioritizing it.
As well as using special library versions of floating-point functions which don't behave the same across different processors I suppose, if you want to be safe.
Eg cr-libm[1] or more modern alternatives like core-math[2].
There's no scenario in which that's desirable.
And yet even Rockstar gets it wrong. (GTA V has several framerate dependent bugs)
The only place where that doesn't matter is fixed hardware - i.e. old generation consoles, before they started to make "pro" upgrades.
And before it was realistically possible to port a game to run on multiple consoles without a complete rewrite.
Sometimes you can find small areas of the game that can be deterministic and worth it. In a basketball game I worked on in the 90s, I designed the ball physics to be deterministic (running at 100hz). The moment the ball left the player hands it ran deterministically; we knew if it was going to hit the shot and if not, where the rebound would go to.
What's totally insane is that the modern engine rewrite Aleph One can also play back such old recordings, for M2 Durandal (1995) and Infinity (1996) at least.
The main downside which probably caused the diseapearance is that any patch to the game will make the replay file unusable. Also at the time (not sure for quake) there was often fixed framerate, today the upsides of using delta time based frame calculation AND multithreading/multi platform target probably make it harded to stay deterministic (specialy for game where you want to optimize input latency)
Networked games have a "tickrate", just for the networking/state aspect. For example, Counter-Strike 2 has a 64Hz tickrate by default. They also typically have a fixed time interval for physics engines. Both of these should be completely independent of framerate, because that's jittery and unpredictable.
Camera and linear motion of objects were interpolated to the framerate.
I think if I remember right there were also funny moments where things didn't look right after patches?
The bigger problem is that floating point math isn't deterministic. So replays need to save key frames to avoid drift.
Quake used fixed point math.
I guess floats are still mostly deterministic if you use the exact same machine code on every PC.
Nope, they are not. Part of the problem is that "mostly deterministic" is a synonym for "non-deterministic".
- Inconsistent use of floating-point precision, which can arise surprisingly easily if e.g. you had optional hardware acceleration support, but didn't truncate the intermediate results after every operation to match the smaller precision of the software version and other platforms.
- Treating floating-point arithmetic as associative, whether at the compiler level, the network level, or even the code level. Common violations here include: changing compiler optimization flags between versions (or similar effects from compiler bugs/changes), not annotating network messages or individual events with strictly consistent sequence numbers, and assuming it's safe to "batch" or "accumulate" deltas before applying them.
- Relying on hardware-accelerated trigonometric etc. functions, which are implemented slightly differently from FPU to FPU, since these, unlike basic arithmetic, are not strictly specified by IEEE. There are many reasons for them to vary, too, like: which argument ranges should be closest to correct, should a lookup table be used to save time or avoided to save space, after how many approximation iterations should the computation stop, etc.
Deterministic game sync is a completely different approach more often used in RTS games. Quake had non-deterministic authoritative central server + clients getting an incomplete view of the world.
The updates thing is a shame. You can store multiple configuration files for balance patches, but executable code is much harder.
I am one of the authors of Fire Fight game (1996-ish) and we pulled the same stunt. It was actually easy, we just had to build our own "random number generator" and fix all bugs with uninitialized memory :-)
No fancy kernel level anti-cheats. Just ensure matches were played on legitimate servers and demos were recorded.
Also, back then live streaming while playing was usually too much of a computational and network burden (56k modems), but casting was just coming around as being a thing and certain Quake 3 mods had spectator modes that let someone streaming spectate you from the first person live which also helped deter cheating. There was even split screen spectating modes so you can follow the action (useful for 4v4 games, etc.).
Carmack and team really made something special back then. The ideas they had and what they did with their tech on relatively low end hardware was remarkable.
https://gafferongames.com/post/floating_point_determinism/ https://www.forrestthewoods.com/blog/synchronous_rts_engines...
As a kid, I couldn't wait to see what came next. Sadly, Q1 was rather one of a kind, and it was many years until anything else like it showed up.
But then it'd also be nice if they fixed the "game crashes randomly when joining games" bug too.
(To give them credit, it doesn't now take 5 minutes after waking the Switch 2 before Rocket League reconnects me to the Epic servers like it did a couple of months ago...)
[0] Also the stupidly low limit on how many you can download - it's my storage cost, not yours, wtf.
It’s one of my favourites
That's how Warcraft 3's fully deterministic save files would work. Old replay files would only work tied to one specific patch patch of the game.
But here's the thing: it's still a godsend while in development and it was still a godsend to players too. "Battlenet user spiritwolf beat me even though I had the upper hand, how did he do it? Let's check the replay immediately".
Also if you really think about it: if you plan for it from day one, there's not much preventing your game engine from having a pluggable system where you could have the various different patches of the game engine ship with every subsequent release of the game.
So when 1.03c is out but you want to play a replay meant for version 1.02b, the game automatically just use that version of the game engine.
The only case where this basically ain't working is if there's a patch for a security exploit: that'd probably need to be patched for good.
But for all other cases, backward compatibility for replay files / deterministic game engines is totally doable. It may not be how things are done, but it's totally doable.
Starcraft 2 does that. It's still quite an achievement.
Warcraft 3 replays couldn't jump in time, just forward very fast. HoN could do that. It was amazing.
For a few months they even made ALL replays searchable on a website. Every game of HoN played globally.
It's the time it takes to go "uhh, I'm stuck, I'd better pause" and then the bit before your brain kicks in following a pause.
Here's Super Mario Bros's demo replay data: https://gist.github.com/1wErt3r/4048722#file-smbdis-asm-L108...
21 bytes of joypad input and 21 bytes of input timings
I wrote about it here many times over the years but in 1991 I wrote a little DOS game (and I had a publisher and a deal but it never came out and yet it's how my career started but that's another story) and at some point I had an "impossible to find" bug because it was so hard to reproduce.
So I modified my game engine to be entirely deterministic: I'd record "random seed + player input + frame at which user(s) [two players but non-networked] input was happening". With that I could make tiny save files and replay (and I did find my "impossible to find" bug thanks to that).
First time I remember someone talking about it was a Gamasutra article by an Age of Empire dev (article which another poster already mentioned here in this thread): they had a 100% deterministic engine. FWIW I wrote an email to the author of that article back then and we discussed deterministic game engines.
Warcraft 3 definitely had a deterministic game engine: save files, even for 8 players (networked) games were tiny. But then you had another issue: when units, over different patches, would be "nerfed" to balance the game (or any other engine change really), your replay files wouldn't play correctly anymore. The game wouldn't bother shipping with older engines: no backward compatibility for replay files.
I had a fully deterministic game engine in 1991 and, funnily enough, a few days ago with the help of Claude Code CLI / Sonnet 4.6 I compiled that old game of mine again (I may put it on a public repo one day): I still had the source files and assets after all those years, but not the tooling anymore (no more MASM / no more linker) so I had to "fight" a bit (for example I had not one but two macros who now clashed with macros/functions used by the assembler: "incbin" and another one I forgot) to be able to compile it again (now using UASM, to compile for DOS but from Linux).
Another fun sidenote... A very good friends of mine wrote "World Rally Fever" (published by Team 17) and I was a beta tester of the game. Endless discussion with my friend because I was pissed off for his engine was so "non-deterministic" than hitting the Turbo button on my 486 (I think it was a 486) while I was playing the game would change the behavior of the (computer) opponents.
To me a deterministic game engine, unless you're a massively networked multi-player game, just makes sense.
Blizzard could do it for Warcraft 3 in 2002 for up to 8 players and hundreds of units. Several games had it already in the nineties.
It simplifies everything and I'd guesstimate something like 99% of all the game out there that don't do it could actually do it.
But it touches to something much more profound: state and how programmers think about state and reproducibility. Hint: most don't think about that at all.
Some do though: I was watching a Clojure conf vid the other day and they often keep hammering that "view is a function of state". And it is. That's how things are. It was true in 1991 when I wrote my DOS game, it was true for Age of Empire, Warcraft 3 and many other games. And it is still true today.
But we're in 2026 and there are still many devs insisting that "functional programming sucks" and that we should bow to the mutability gods for that is the only way and they'll fight you to death if you dare to say that "view <- fn(state)".
This explains that.
Fun thing - I'm working on modernizing a legacy Fortran / Win32 application to something a bit more modern, and ran into similar issues with toolchain not being available anymore; and further some libraries where source is needed to compile, but only have binaries of the libraries.
Claude Code was amazing creating stubs by looking at function calls used and how, and then getting just enough in place to call existing binaries correctly; and further updating the code to be in alignment with Fortran specs that can compile on existing compilers - but it was a 'fight'.
Outside of VR, Unity offers a nice "AudioListener.pause" to pause all audio, but if you have any sound effects in your pause menu like when a user changes a setting, those no longer work, further requiring more hacky fun (or just set it to true, and ignore user-complaints about no audio on menus when paused).
On top of that, you have things like Animators for characters, which also have to be paused manually by setting speed to 0 (TimeScale = 0 doesn't seem to do it). Some particle systems also don't seem to be affected by TimeScale. If you have a networked game then you also have to (obviously) send the pause command to clients. If you have things in motion, pausing/restarting the physics engine might cause weird issues with (at least angular) velocity so you might have to save/restore those on any moving objects.
It never worked. You’d pause, and the plane was frozen in place yes, but the instrument cluster would still animate and show your altitude/speed changing as if you never paused. But you couldn’t control anything until unpaused. So you’d resume, and your momentum would suddenly leap to where the accumulated deltas ended up. So if you active-paused at full throttle, you’d unpause and start going way too fast… if you active paused while stalling, you’d unpause and your speed would be near zero… you’d even consume fuel while paused.
It’s like they literally just froze the plane’s position and left every other aspect of the physics engine untouched, never tested it, shipped it, and even did a bunch of marketing at how great the feature was. When it was so obviously broken.
I came back to the game after a year or so of updates, and not a thing had improved, it was every bit as broken as when they shipped it.
The 2024 release seems to have largely fixed it though from what I can see. It’s just nuts they had such a clearly broken feature for that long.
You gotta learn and understand its quirks, though. As long as your flight state is rock stable (e.g. on Auto Pilot) and/or you're not fiddling with the controls while paused, it's pretty much always worked fine for me.
I've also used its interactivity to my advantage and saved the plane from an otherwise unsaveable flight state, e.g. by gaining airspeed while paused.
So my memory might be playing me a trick, but wasn't that already the case in FS2002 and 2004? I seem to remember using the pause as a kid to look at my plane under every angle.
The game world is paused whenever Link pulls an item from a chest, but because his animation does not loop perfectly, because of a missing frame, he slowly slides across the ground and even through walls.
One of the minimum % speedrun abuses this by looping the animation for many hours in order to glitch through a wall, and not collect a progression item, which would count towards the collection percentages.
In the original (and maybe also DX) release of Link's Awakening, the game uses a top-down view with the world split up into tiles. Walking of the left side of a screen makes you end up on the right side of the next screen over.
What you could do is pause at the right frame on the screen transition, and you would end up on the new screen but link's position would not change. So you walk off the left side of a screen and end up on the left side of the new screen. Lots of fun to be had with skipping important stuff with that.
[0] https://github.com/rameshvarun/marble-mouse/blob/8b25684a815...
I see a lot of comments here saying something along the lines of "isn't it just a state in the state machine?" which isn't wrong, but is an extremely simplistic way of thinking about it. In, say, 1983, you could get away with something like that:
- pause the game: write "PAUSED" to the tilemap
- paused main loop: check input to unpause
- unpause the game: erase the "PAUSED" / restore the previous tiles
But at that time you could already see the same sort of issues as today. Something somewhat common in Famicom/NES games is the sprites disappearing when the game is paused. Perhaps deliberate/desirable in some cases (e.g. Tetris) but a lot of the time, probably just a result of the 'is paused' conditional branch in the main loop skipping the sprite building code[0].
There's an extremely large problem space and ultimately, each game has its own way to define what "paused" actually means.
You might be interested in the features Godot provides[1] for this. Particularly, the thing that makes it interesting is the 'process mode' that each node in the scene tree has. This gives the developer quite a lot of control over what pausing actually means for a given game. It's not a complete solution, but a useful tool to help solve the various problems.
[0] Simplified description of course. Also, the sprite building code often ended up distributed throughout the various gameplay logic routines, which you don't want to run in the paused state.
[1] https://docs.godotengine.org/en/stable/tutorials/scripting/p...
[ed] Just adding that Tetris is only an example of a game where you might want that behaviour, not a comment about how any of the Tetris games were actually made.
Pausing the render? But not the physics so you keep falling? You need to pause many systems. At the very least you'd want to pause gameplay, and sound and physics. You'd want to keep input and some of the UI unpaused. If you have a fancy UI with VFX, you need to make sure those are not using the paused game time. etc etc
Repeat that for every system - all those edge cases for each system can waste a lot of time and energy.
P.s. And once you are done achieving the above, you can then make sure you haven’t caused performance issues :)
But yes, conceptually, it’s a relatively simple idea. The devil is always in the details.
Like torch flames and trees swaying in the wind.
But when you find a broken ancient seal in the forest, the giant creepy eyeball moving around in it keeps moving even when you pause the game, which helps emphasise how other-worldly it is.
Other pause some underlying simulation while still letting you modify the game state, as an expected part of gameplay, like a city builder. As the user might spend a significant amount of time in a paused state building things, it would be pretty visually unappealing to have the entire world completely frozen the whole time.
Others might pause all gameplay entirely, such as for displaying a menu, in which case pausing even environmental animations might make more sense since the user isn’t actively playing.
For the second type, I would much prefer some GUI element to indicate the simulation is paused rather than freezing the whole game world, such as a border around the screen or maybe a change of color theme of the GUI or similar.
Pausing a game has a massive impact on the game experience. It lets you break the fourth wall experientially. Not wrong, but it changes the dynamic of the game.
Same as saving at any time does. As losing your loot or your life permanently does. Not wrong, but a hard choice that appeals to some players and not to others.
I used to pause pacman on my Atari 800 so I could run to church and sing in the choir or be an altar boy. Then I ran home and unpaused to continue. Sometimes in summer the computer over-heated and I lost everything while I was at church.
Lessons learnt? None, I think :)
While the game is paused, if a player were to click on the "level up" buttons for their skills, each click actually advanced the game by 1 frame - so it was possible for people to die etc. during a pause screen.