Repton 3 "Replay" patch

[gfoot]

Hi guys,

I've been playing a lot of Repton 3 variants recently, and thought it'd be nice to be able to record input-level replays of games, and then maybe even try to make some tool-assisted speedruns by hand-editing the control data in the replay files.

Since Matthew Atkinson posted the source code, this was actually really simple to do, at least as a proof-of-concept. Here's a video demo:

repton3replays.mp4

Example of recording and playing back replays in Repton 3

(6.22 MiB) Downloaded 58 times

The demo there shows me first recording a bit of Repton 3 Screen C - I picked this one because it has a couple of monsters near the beginning, to demonstrate that random behaviour works too - and then playing back the recording. I'm using the main menu's keyboard/joystick selector to control recording and playback - keyboard mode records your inputs while you play, and joystick mode just plays back the recorded inputs.

I made a mistake halfway through this video when playing back by starting the game from Screen A instead of entering the password, so it tried to play the inputs on the wrong level, which obviously doesn't work well! But after restarting on the right level, you can see that it plays through the exact same sequence as when recording.

For anyone interested in what the changes were, the code to store and restore inputs fit exactly on top of the old joystick code, to the byte, which was nice as that code block is otherwise completely full. The replay data is stored in sideways RAM, and there's no saving or loading support built in, but you can do that after pressing Break.

Other than that code, I hooked into where the spirit initialization fix happens, at the start of each level, and made it initialize the replay pointer and patch a couple of other things to make the random number generation deterministic.

I was able to constrain all the changes to the REPTON4 file on disc, by making it patch the existing already-loaded code rather than editing the code directly. If anyone is interested in a copy of my current REPTON4 file (which you could drop on to any decrypted late-version Repton 3 disc), or the source code changes to B.REPTN, then let me know! Hopefully I'll get time to improve this further though as there are a bunch of issues I think need resolving.

[scarybeasts]

Hi guys,

I've been playing a lot of Repton 3 variants recently, and thought it'd be nice to be able to record input-level replays of games, and then maybe even try to make some tool-assisted speedruns by hand-editing the control data in the replay files.

Somehow, I've only just noticed this really interesting post!

One feature I added to beebjit a long time ago, is input capture and replay at the emulator level. I've really been meaning to use it to capture some full game runthroughs. It's so much more fun than a video -- you can take over the replay at any time, and use it to play / access parts of the game that are hard to get to.

beebjit is fast so it's able to replay very fast. I was thinking of making some "speedrun squared" demos, i.e. someone good at a game speed runs it at normal speed. And then I keep trying to make beebjit faster until a full replay is measured in milliseconds

There also a (possibly buggy) rewind key combo that rewinds by in fact just replaying to the state 5 seconds ago. The JIT in beebjit is sufficiently fast that it's an acceptable way to undo any mistakes and continue.


Cheers
Chris

[Diminished]

One feature I added to beebjit a long time ago, is input capture and replay at the emulator level. I've really been meaning to use it to capture some full game runthroughs. It's so much more fun than a video -- you can take over the replay at any time, and use it to play / access parts of the game that are hard to get to.

Oh, I didn't realise you'd already added this capability. I should probably RTFM occasionally.

I should maybe start working on Citadel runs in that case.

[scarybeasts]

One feature I added to beebjit a long time ago, is input capture and replay at the emulator level. I've really been meaning to use it to capture some full game runthroughs. It's so much more fun than a video -- you can take over the replay at any time, and use it to play / access parts of the game that are hard to get to.

Oh, I didn't realise you'd already added this capability. I should probably RTFM occasionally.

I should maybe start working on Citadel runs in that case.

It's not mentioned in -help or -more, but there is a simple example in EXAMPLES.
Summarized, there's -capture to log keys and timings to a small file; -reply to replay it; Alt-F as usual to toggle a replay between real-time speed and turbo speed; and Alt-E to "exit" the replay and take over with the real keyboard.

However, I've been deliberately a bit quiet about the feature because I'm worried about it: it's somewhat fragile. We're simply capturing a log of non-deterministic events (human key pressed) with timestamps. Replay a log "correctly" requires a lot of things to be lined up:

- Emulation has to be deterministic. Fortunately, most (all?) emulators have this property.
- The exact same software has to be used (e.g. no mix and matching tape Citadel with disc Citadel, or cracked Citadel with original Citadel).
- The machine type has to be the same; a Master has different timings to beeb.
- The machine config has to be identical; e.g. a 1770 vs. 8271 beeb takes a different number of cycles to boot up.
- The replay emulation has to be cycle-exact to the capture emulation.

It's that last point that's the killer. As of the most recent master, I'm happy with beebjit's emulation accuracy.

That said -- and it's almost always the pesky Bitshifters -- there's always some new accuracy fix that might change the timings in ever so slight a way. There's at least one 6845 video emulation change I'll need to make to beebjit, to support a new demo idea, because we've found yet another extreme corner case where emulators don't match real hardware. The risk here is that the change might slightly throw off game timing so that some of the earlier capture files drift out of sync and don't replay as per original capture.

I'm curious if anyone has any thoughts here.

I'd love to get my hands on a Citadel capture, if only to replay a fast run at fast speed and see how awesome the video looks. But I'm worried about forward compatibility of the capture file.

If you did want to play with this, I'd recommend to keep things simple and use -mode interp (to avoid any undiscovered JIT timing problems). Any feedback welcomed!


Cheers
Chris

[tom_seddon]

One of my goals with b2 was always to simplify (or indeed actually make possible...)the creation of full playthrough videos of stuff like Citadel, Exile, Repton 1/2/3/etc., and this is supported to some slightly inadequate extent by its timeline functionality. I used an early version of b2 to produce this playthrough of Thrust: https://www.youtube.com/watch?v=GlevFyQvD9k - took about 30-40 minutes to make as I recall.

I do have some plans for an overhaul of this feature, that will hopefully turn it from inadequate to, ideally, at worst merely a bit ropey...

Replay a log "correctly" requires a lot of things to be lined up:

- Emulation has to be deterministic. Fortunately, most (all?) emulators have this property.
- The exact same software has to be used (e.g. no mix and matching tape Citadel with disc Citadel, or cracked Citadel with original Citadel).
- The machine type has to be the same; a Master has different timings to beeb.
- The machine config has to be identical; e.g. a 1770 vs. 8271 beeb takes a different number of cycles to boot up.
- The replay emulation has to be cycle-exact to the capture emulation.

My plan for b2, once I get it saving replay logs to disk, is for the log to include the initial state, including disk contents/machine type/ROM contents/FDC type/etc. (anything that might affect reproducibility, basically). That should bypass many of the above issues. Record/replay will also become a specific UI mode, entered only at the user's request, that disables anything that's inconvenient or impractical to support.

To cater for forward/backward version compatibility, I'm intending to have a simple version check. Maybe check the emulator version, maybe have some explicit number that I only increment when I change the emulator code rather than the UI or whatever - haven't decided yet.

My working assumption/forlorn hope is that 99.99% of the time, even with emulation changes, things will be fine. The majority of nondeterministic actions are keypresses, and that's something that happens at a completely different timescale from the sort of microsecond-level differences that I'm hoping will be the result of the majority of improvements to b2's accuracy. But, we'll see

--Tom

[TobyLobster]

To cater for forward/backward version compatibility, I'm intending to have a simple version check. Maybe check the emulator version, maybe have some explicit number that I only increment when I change the emulator code rather than the UI or whatever - haven't decided yet.

If you want to get more sophisticated, maybe have a major and minor version number for the emulation layer. If the minor version changes that means technically some small thing has changed, your replay probably still works but there's a small chance it won't. You can load a replay with only a difference in minor version with a warning. However for a change that will definitely break all existing replays, use a major version change. You can't load a replay with a different major version.

[scarybeasts]

My plan for b2, once I get it saving replay logs to disk, is for the log to include the initial state, including disk contents/machine type/ROM contents/FDC type/etc. (anything that might affect reproducibility, basically). That should bypass many of the above issues. Record/replay will also become a specific UI mode, entered only at the user's request, that disables anything that's inconvenient or impractical to support.

Logging initial state sounds like a fruitful avenue.
I wonder if it'd be easy to productionize "initial state" meaning "initial state after game load" or "initial state after game start"? This could eliminate change in the FDC / tape subsystems from affecting replay. beebjit has seen substantial changes in FDC / tape timing since its release.

My working assumption/forlorn hope is that 99.99% of the time, even with emulation changes, things will be fine. The majority of nondeterministic actions are keypresses, and that's something that happens at a completely different timescale from the sort of microsecond-level differences that I'm hoping will be the result of the majority of improvements to b2's accuracy. But, we'll see

Video emulation changes worry me, and I've been bitten by them in the past when experimenting with replays. Surprisingly, mode changes are a killer. The 6845 registers are reprogrammed at an arbitrary time and this will often put the chip in a strange state (e.g. various counters can become larger than the max value in the registers; flipping interlace is an operation we likely still don't have down 100%; etc.) Some games -- like an Exile, Castle Quest, etc. have a very stable core 6845 state once started, so it's unlikely that emulator fixes would affect timing after the game has started. Other games might be less stable, e.g. Citadel puts the 6845 in an unusual state when the main character is between rooms. And many other games flip modes as part of starting the game, e.g. Acornsoft titles often do MODE7 -> MODE2/5.


Cheers
Chris

[Diminished]

Would it be a miserable endeavour to dump some of the machine state along with the inputs in the replay file? Perhaps you could use that to resynchronise, even in the case where some change breaks the causality of the thing.

I suppose you might have to dump more state than you'd like (worst case: the entire machine's state) in order to get much benefit.

Given the above, I might just try to do a fast play of Citadel sometime, without bothering trying to optimise it aggressively.

[scarybeasts]

Would it be a miserable endeavour to dump some of the machine state along with the inputs in the replay file? Perhaps you could use that to resynchronise, even in the case where some change breaks the causality of the thing.

As per the other thread on Castle Quest, I think I might try to keep any "old" timing models around in order to support older captures. I think this'll work if any future timing fixes are going to be one-liners here or there to the disc, video, etc. systems. We'll see.

Given the above, I might just try to do a fast play of Citadel sometime, without bothering trying to optimise it aggressively.

I'd love to see a competent Citadel capture. The Castle Quest capture with -mode interp is holding up pretty well; probably a Citadel one would also be usable.


Cheers
Chris