beebjit-0.9.8 release: a story in one picture, one video and multiple facepalms

[scarybeasts]

Hi,

Following closely on from v0.9.7 is v0.9.8! This is the version you want to use if you're experimenting with captures.

GitHub: https://github.com/scarybeasts/beebjit
Windows builds: https://github.com/scarybeasts/stuff/tr ... win_builds

What is worth mentioning?

1) Black and white PAL hack support!

I was a little mind blown to hear that back in the day, Fire Track had a "black and white" mode that actually worked on colour TVs!
The technical details appear to be that if you massively delay an hsync just after the vsync, it confuses the TV into not using the colour encoded into the PAL signal.

Given such a cool hack, it'd be remiss to not emulate it.

2) "seek" command in the console.

This command enables seeking around inside a replay file. It demonstrates the concept that a capture file, plus a fast replay engine, is kind of like an infinite series of saved game files.

Perhaps best shown with a video: https://www.youtube.com/watch?v=PT9aGamnbrI

[With thanks to all who recently contributed to the Castle Quest play throughs, glitching and world record attempts!]


3) The facepalms

- D'oh! Since v0.9.4 or so, the interpreter has been mis-counting the cycles taken by DEC $50,X. This isn't a common opcode but Exile uses tons of unusual opcodes including ones like this. And so capture files were "off" when you capture with the interpreter and replay them with the JIT.

- Argh! One of the optimizations in v0.9.7 broke JIT execution in some cases. The sideways RAM version of Exile seems adept at hitting this. The optimization has been reverted for now.

- Oops! A model B configured with sideways RAM (e.g. for Exile!) would crash on ARM64 macOS.


4) Other

- Fix CMP in ARM64 inturbo mode.
- Fix crash with Exile in ARM64 inturbo mode.
- Fix a video assert regression in debug builds (probably harmless in optimized builds, but not 100% sure).
- Re-jig the memory management setup on Intel macOS, so that it now should start reliably on older macOS versions.


Cheers
Chris

[bob147]

Well done mate, whilst castle quest was short enough with enough lulls in the action that it really wasn't hard to exit an existing replay and just carry on playing, the other games I'm messing with aren't so accommodating, so this feature is very welcome!

Edit - pondering how it would be used in practise (without having it in front of me yet), how about a running timer on a hud or debug window so you know roughly where to seek to? And I assume this works in combination with -capture so you can seek to 10 seconds into an existing replay, then add your own new inputs, just like you can with ctrl-E. With apologies if this stuffis already present!

[gfoot]

Awesome. Regarding captures and playback, I have found - at least in versions up to 0.9.7 - that in order to get fast-forward playback to work, I need to do the playback in "interp" mode. I don't think this is documented anywhere, is it an intentional limitation? It seems to be fine to do the recording in jit mode, it's just fast playback that goes out of sync unless it's in "interp" mode.

Edit - I just checked again, and this applies to use of Alt-F and Alt-Z - they don't work correctly unless it's in interp mode. -fast doesn't work even in interp mode, it goes wrong much sooner, so I guess that's implemented in a different way. This is all tested in 0.9.5 and 0.9.7 - I don't think my capture ports to 0.9.8. I'll try to record a new capture later on to verify if the issues still exists in 0.9.8.

Edit again - I recorded a new capture in 0.9.8, and the above bug did not occur now for Alt-F playback. Playback with -fast still doesn't work though, it doesn't even get past the title screens.

I also tried to debug why my old 0.9.7 capture wasn't working in 0.9.8, and edit it to fix it - I think it was either due to pressing two keys simultaneously, or not having a long enough gap between pressing and releasing keys. I don't think we should expect old captures to work in new versions though, especially when instruction timing bugs have been fixed!

The "seek" command is very useful. I think it would be better if it dropped to the debugger after the seek, rather than continuing the replay at normal speed. It also doesn't seem to work if the end of the capture file has already been reached - it would be nice if it would still work in that case, and start from the beginning again.

[scarybeasts]

Edit again - I recorded a new capture in 0.9.8, and the above bug did not occur now for Alt-F playback. Playback with -fast still doesn't work though, it doesn't even get past the title screens.

I'm glad that the Alt-F bug is fixed. You were probably hitting the terrible v0.9.7 regression.

If you want to launch a fast playback from the command line, you need -fast -accurate. With only -fast, that launches an entirely different execution model that runs the CPU as fast as possible but locks the VIA, CRTC etc. to wall time. It's good for running BASIC very fast (the BBC Micro Bot uses it) but it's not deterministic.

I also tried to debug why my old 0.9.7 capture wasn't working in 0.9.8, and edit it to fix it - I think it was either due to pressing two keys simultaneously, or not having a long enough gap between pressing and releasing keys. I don't think we should expect old captures to work in new versions though, especially when instruction timing bugs have been fixed!

Ideally, I'd like to try keeping the old captures working. beebjit timing accuracy should be pretty settled by now -- the incorrect cycle counting bug fixed in v0.9.8 was a major embarrassment, but it showed up easily enough when I started really testing with Exile.
Of the "classic" captures made with v0.9.6, these replay verbatim in v0.9.8:

- Nightworld playthrough (nw16.cap)
- Castle Quest world record (cq_wr10.cap)

And this replays with a hack flag I added to tweak video timings:

- Castle Quest (normal play through)

The "seek" command is very useful. I think it would be better if it dropped to the debugger after the seek, rather than continuing the replay at normal speed. It also doesn't seem to work if the end of the capture file has already been reached - it would be nice if it would still work in that case, and start from the beginning again.

Good ideas. I'll fix these.


Cheers
Chris

[gfoot]

If you want to launch a fast playback from the command line, you need -fast -accurate. With only -fast, that launches an entirely different execution model that runs the CPU as fast as possible but locks the VIA, CRTC etc. to wall time. It's good for running BASIC very fast (the BBC Micro Bot uses it) but it's not deterministic.

Is there a way to choose the replay speed, while still remaining accurate in emulation? I've been doing a lot of speedrun recording and it'd be great to play back at e.g. 2x speed, for review purposes. -fast and Alt-F are too fast to see what's going on!

The "seek" command is very useful. I think it would be better if it dropped to the debugger after the seek, rather than continuing the replay at normal speed. It also doesn't seem to work if the end of the capture file has already been reached - it would be nice if it would still work in that case, and start from the beginning again.

Good ideas. I'll fix these.

Another issue with "seek" that I came across today is that if I'm replaying one capture, capturing into another, planning to take over at a certain point with Alt-E, and using "seek" to get to near that point, seek doesn't actually work. I think the fact that a new capture is being recorded is confusing it.

[Diminished]

Hi Chris. More UEF compatibility problems I'm afraid. Unless I'm wrong, it seems beebjit won't load UEFs that have been saved out by BeebEm. Unexpectedly, I can't actually decide whose fault this is. An example and some discussion here:

viewtopic.php?p=362215#p362215

[scarybeasts]

Hi,

Hi Chris. More UEF compatibility problems I'm afraid. Unless I'm wrong, it seems beebjit won't load UEFs that have been saved out by BeebEm.
viewtopic.php?p=362215#p362215

Thanks. That's an interesting one. It doesn't appear to be related to the UEF chunks at all, but rather it's about the gzip container.

BeebEm appears to be emitting strange gzip files:

[chris@localhost Downloads]$ file simple.uef
simple.uef: gzip compressed data, from TOPS/20

The "system" field of TOPS/20 doesn't really affect much but I found it interesting because I'd never seen TOPS/20 before.

The actual issue is that BeebEm is emitting UEF files that are the concatenation of multiple gzip files. I don't know if that's intended in the UEF spec or not, but since these files exist I might as well just support them, which I did here:

https://github.com/scarybeasts/beebjit/ ... ceb73098db


Cheers
Chris

[Diminished]

Thanks. That's an interesting one. It doesn't appear to be related to the UEF chunks at all, but rather it's about the gzip container.

Sorry. I should have checked. When you have a hammer, everything looks like a nail.

The actual issue is that BeebEm is emitting UEF files that are the concatenation of multiple gzip files. I don't know if that's intended in the UEF spec or not, but since these files exist I might as well just support them

I didn't remember running into this at all, but it seems I encountered it in my own UEF tool also:

Code: Select all

  // need to be able to deal with concatenated gzip files (thanks beebem)

Thanks for the fix.

[chrisn]

The actual issue is that BeebEm is emitting UEF files that are the concatenation of multiple gzip files. I don't know if that's intended in the UEF spec or not, but since these files exist I might as well just support them, which I did here:

https://github.com/scarybeasts/beebjit/ ... ceb73098db

Thanks for tracking this issue down. I don't think it is intended, so I'll fix it in the next BeebEm release.

[vanekp]

is it not got something to do with BeebEm is adding extra CRC bytes on each file block.
Reported here https://github.com/stardot/beebem-windows/issues/51
Also mentioned it here viewtopic.php?p=246952#p246952

[sydney]

I seem to have hit a compile bug.
Here is the error:

In file included from asm/asm_jit.c:6:0:
asm/null/asm_jit_null.c: In function ‘asm_jit_enter’:
asm/null/asm_jit_null.c:124:10: error: ‘jump_addr’ undeclared (first use in this function)
(void) jump_addr;

I'm using Bodhi linux on a (very!) slow laptop.
gcc is version 7.5.0
I'm pretty sure I've compiled successfully on other versions of linux (mint and bunsenlabs) on this laptop.
Any ideas or should I ditch Bodhi linux and go back to something that definitely works?

[scarybeasts]

I seem to have hit a compile bug.

...

I'm using Bodhi linux on a (very!) slow laptop.
gcc is version 7.5.0
I'm pretty sure I've compiled successfully on other versions of linux (mint and bunsenlabs) on this laptop.
Any ideas or should I ditch Bodhi linux and go back to something that definitely works?

Thanks for the report and sorry about that.
I think this wonderful laptop might be running a 32-bit version of Linux? I broke the build in this case. Should be fixed on as of recent commit: https://github.com/scarybeasts/beebjit/ ... cdb2ab03b9

On a 32-bit build, you'll get beebjit but without the JIT! Still, the interpreter is reasonably zippy as far as 6502 interpreters go, so hopefully your slow laptop will be ok.


Cheers
Chris

[sydney]

Thanks Chris. I'll give that a try this evening.
I just arrived at work and the first thing I did was try compiling beebjit and it worked fine. The only differences were the version of gcc and that fact my home laptop is 32 bit. Has the bug crept in recently? I'm sure I had and earlier version of beebjit running on it before.
I'm sure beebjit will be plenty fast enough without the jit. The linux versions of BeebEm and B-Em are just too slow on that laptop and in the past I've resorted to installing wine and running the windows versions which seemed to work fine.

I should probably get some work done now.