B-Em Tape Overhaul v3 (SAVE support, autotests)

[Diminished]

I botched yesterday's attempt at releasing my latest B-Em tape overhaul effort. So, with less bungling, hopefully:

I've included ZIPs of the source code patch (toh3-1_rc3_diff.zip) for Linux and MacOS, and also a highly experimental unsigned Windows build (toh3-1_rc7_win32build.zip). There are also a couple of hundred automated tests (toh3-1_rc6_tape_tests.zip), including full loads of many protected games (you'll need PHP 8 for this). SHA256 sums are on pastebin here (do check the date on that).

You should read the readme.txt, but the two most pertinent pieces of information are as follows:

1. If you are trying to save data to tape, nothing will be recorded unless you have enabled Record mode (Tape -> Record and Append to Tape). Similarly, if you are trying to hear tape noise from a SAVE operation, you won't hear anything unless Record mode is activated.

2. I had trouble with the recommended 5.2.2.0 version of Allegro, so I had to use the latest 5.2.9.1 instead for the Windows build. There may be bugs in the UI, particularly related to ticks appearing in menus. Sorry -- I have run out of time to mess about with it.

I've also included a slightly updated version of the Hoglet Histogram (TDRE timing) tests, allowing selection of 300 or 1200 baud. TOH-patched B-Em is the only emulator that passes these tests (and everything else except beebjit crashes). [edit: I am a liar; jsbeeb also runs the tests without crashing, although it doesn't really pass them, and particularly not at 300 baud.]

[Diminished]

I put together a video demonstrating some of the improvements I've made.

https://youtu.be/_t_N63wcsGk

[Sazhen86]

Excellent work and a very enjoyable video, thanks. I’m looking forward to finding some time to have a play around with this.

[vanekp]

thanks for the post will play with it some was having problems with the UI ticks as you mentioned it will not let me change the emulator speed instead it ops up with a error about attaching mixer to voice for Music 5000.

Music5000.jpg (9.69 KiB) Viewed 728 times

[Diminished]

thanks for the post will play with it some was having problems with the UI ticks as you mentioned it will not let me change the emulator speed instead it ops up with a error about attaching mixer to voice for Music 5000.
Music5000.jpg

This may just be a bug in the latest B-Em master branch. If you go to Settings -> Sound and switch off Music 5000, this problem may go away.

[Diminished]

Here is part 2 of my video demonstration, which mainly focusses on B-Em-TOH's automated testing capability:

https://www.youtube.com/watch?v=LtMhvoOsCf8

[SteveF]

I've probably said it before, but this is some very nice work. A lot of the technical detail is over my head but it is neat to see emulation accuracy being improved so dramatically, and the prospect of having unit tests for b-em is intriguing. Thanks for doing this!

[Diminished]

I've probably said it before, but this is some very nice work. A lot of the technical detail is over my head but it is neat to see emulation accuracy being improved so dramatically, and the prospect of having unit tests for b-em is intriguing. Thanks for doing this!

Thank you. I appreciate this.

There is an element of duty involved. I have learned that tape is apparently unglamorous; grunt labour on tape infrastructure seemingly even more so. I concluded long ago that nobody else was going to bother doing the work I have done on B-Em (or Quadbike, Cornfield ...)

I think nearly all of the protected software in the tests already loaded in B-Em (with the possible exception of The Music System?) but things are definitely more accurate now.

The big feature here IMO is in writing to tape. I have needed to do this on several occasions (most famously to test the mastering software we found on Michael Jakobsen's Citadel source discs), and every time I had to resort to booting up Windows and using BeebEm. Publicly grumbling about this situation didn't help, so I had to apply the old maxim about wanting something done properly.

(I am jaded, yes.)

Ironically, using B-Em-TOH to save an Elk version of Citadel at this time is troublesome: B-Em-TOH routinely inserts baud-select chunk &117 to confirm 1200 baud. Elkulator petulantly calls exit() when it encounters this chunk type. I 100% consider this an Elkulator problem, but there are apparently about half a dozen Elkulator forks. The chance that I can get them all fixed is zero, so there probably should be a setting in B-Em to suppress chunk &117. A workaround is to save to TIBET and use tibetuef.php to generate the UEF, as this has an option to suppress chunk &117 generation.

I agree the tests are worth having. They are murder to write, but they save effort in the long term. There is much that can be improved in this area; I haven't even written any proper unit tests for CSW. BTW, I spotted one bug -- the test that should load ultron114.uef instead loads ultron.csw (slightly too enthusiastic copy & paste).

[davidb]

Ironically, using B-Em-TOH to save an Elk version of Citadel at this time is troublesome: B-Em-TOH routinely inserts baud-select chunk &117 to confirm 1200 baud. Elkulator petulantly calls exit() when it encounters this chunk type. I 100% consider this an Elkulator problem, but there are apparently about half a dozen Elkulator forks. The chance that I can get them all fixed is zero, so there probably should be a setting in B-Em to suppress chunk &117.

Please create an issue or submit a pull request and we can try to improve Elkulator.

[Diminished]

Seems there is a problem loading The Haunted Abbey. I'll discuss this here to avoid messing up the other thread.

This is an interesting one. I've never seen this before -- there are carefully-timed bursts of leader tone:

There is no data (i.e. no 1200 Hz) anywhere in this sequence -- it is all 2400 Hz. Presumably this produces a signature series of DCD pulses, or something.

I would be interested to know whether this game can load on a SERPROC rather than a Serial ULA.

[Sazhen86]

I don't know if the CSW would help in this case, because it seems beebjit can load the UEF. Neither the vanilla master branch nor my TOH patch of B-Em will load it; so it appears to be a B-Em problem. Bah!

The reason I asked for the CSW was because I could load the CSW of Dr Who successfully whereas the UEF fails, and I figured it was worth a try.
However, looking a bit further, the Dr Who UEF fails with the following error in b-em:

Code: Select all

uef: 
chunk &114: illegal pulse/wave char combination P, P
tape: code 336; disabling tape! (Eject tape to clear.)

Which does seem to violate the UEF specification.

[Diminished]

I don't know if the CSW would help in this case, because it seems beebjit can load the UEF. Neither the vanilla master branch nor my TOH patch of B-Em will load it; so it appears to be a B-Em problem. Bah!

The reason I asked for the CSW was because I could load the CSW of Dr Who successfully whereas the UEF fails, and I figured it was worth a try.
However, looking a bit further, the Dr Who UEF fails with the following error in b-em:

Code: Select all

uef: 
chunk &114: illegal pulse/wave char combination P, P
tape: code 336; disabling tape! (Eject tape to clear.)

Which does seem to violate the UEF specification.

Hm. Something else to look at. Is that the HQ UEF from the acornpreservation ZIP? Could be a bug in B-Em-TOH, or the UEF might genuinely be illegal. B-Em-TOH is unique in that it actually takes chunk 114 seriously.

vanekp has a transcription of this also, which might work out better:

https://drive.google.com/drive/folders/ ... qg296kyyeF

I know I tested one of the Dr. Who images -- I think it was the one from Chris's beebjit set. I don't know its origin. The reason it isn't in the unit tests is because I could never find a copy of the ROM.

[Sazhen86]

I don't know if the CSW would help in this case, because it seems beebjit can load the UEF. Neither the vanilla master branch nor my TOH patch of B-Em will load it; so it appears to be a B-Em problem. Bah!

The reason I asked for the CSW was because I could load the CSW of Dr Who successfully whereas the UEF fails, and I figured it was worth a try.
However, looking a bit further, the Dr Who UEF fails with the following error in b-em:

Code: Select all

uef: 
chunk &114: illegal pulse/wave char combination P, P
tape: code 336; disabling tape! (Eject tape to clear.)

Which does seem to violate the UEF specification.

Hm. Something else to look at. Is that the HQ UEF from the acornpreservation ZIP? Could be a bug in B-Em-TOH, or the UEF might genuinely be illegal. B-Em-TOH is unique in that it actually takes chunk 114 seriously.

vanekp has a transcription of this also, which might work out better:

https://drive.google.com/drive/folders/ ... qg296kyyeF

I know I tested one of the Dr. Who images -- I think it was the one from Chris's beebjit set. I don't know its origin. The reason it isn't in the unit tests is because I could never find a copy of the ROM.

Yes, this is the one from the Acorn Preservation set. I'll take a look at the files in vanekp's transcription which looks like it could keep me busy for a while. Thanks.

EDIT: I tried DoctorWhoAndTheMinesOfTerror(MicroPower).uef from vanekp and that one worked fine in b-em, so it does look like the Acorn preservation UEF file is damaged/invalid.

Also, can I suggest that the following define is increased to, maybe 64, so that we can see the whole of the filename in the eject entry of the tape menu?

Code: Select all

#define GUI_MENU_TAPEFILE_MAXLEN 16

[Diminished]

EDIT: I tried DoctorWhoAndTheMinesOfTerror(MicroPower).uef from vanekp and that one worked fine in b-em, so it does look like the Acorn preservation UEF file is damaged/invalid.

Good to know. I should probably still check the HQ-UEF just to make sure B-Em is decoding it correctly. It doesn't look like I wrote a test for the [P, P] combination even though there is a dedicated error code for it (TAPE_E_UEF_0114_BAD_PULSEWAVE_COMBO), so I should probably rectify that.

Also, can I suggest that the following define is increased to, maybe 64, so that we can see the whole of the filename in the eject entry of the tape menu?

Code: Select all

#define GUI_MENU_TAPEFILE_MAXLEN 16

I can certainly increase that somewhat, yes. I don't know how far it can be safely pushed.

Some notes on the Haunted Abbey issue:

I was surprised to discover that using PlayUEF to load the game on hardware actually works, even though PlayUEF's leader lengths are embarrassingly broken. This is a convincing demonstration that the loader doesn't seem to care how long the leader bursts are; indeed the UEF has them all of fixed length, at ~714 cycles each.

The variation is actually in the gaps between them; here is the sequence (gaps in seconds):

0.225918
0.387392
0.224989
0.225442
0.226281
0.224762
0.224966
0.224762
0.387415
0.388526
0.388685
0.225533
0.388753
0.225102
0.388844
0.223764

So there are two discrete lengths there, and the loader can obviously differentiate between them, and expects the correct sequence. I don't yet know why B-Em-TOH fails this.

Turns out that Haunted Abbey also breaks Quadbike, which is useless at detecting that fast leader/silence switching, so I'll have to look at that too. >:-/

[Sazhen86]

I was surprised to discover that using PlayUEF to load the game on hardware actually works, even though PlayUEF's leader lengths are embarrassingly broken. This is a convincing demonstration that the loader doesn't seem to care how long the leader bursts are; indeed the UEF has them all of fixed length, at ~714 cycles each.

It seems that PlayUEF can be made a bit more sane by changing:

Code: Select all

  
  // Create mini-samples of audio bit encoding
  const carrier = generateTone("carrier", baud*2, 2, phase, sampleRate);

to

Code: Select all

  
  // Create mini-samples of audio bit encoding
  const carrier = generateTone("carrier", baud*2, 1, phase, sampleRate);

[vanekp]

DoctorWhoAndTheMinesOfTerror the protected parts I created by hand working out the bits and the 300/1200 switching and then created the inter file protected blocks with the data.

[Diminished]

Haunted Abbey

DCD pulses

As I suspected. B-Em-TOH only pulses DCD twice during the leader sequence; on hardware it pulses something like 17 times. I thought I had the DCD behaviour correct, but I've clearly missed something.

edit: Pretty sure my UEF decoder is generating silence at twice the length it should be (chunk &116), so that'll be another thing to fix.