Using OSFIND

[emiliom]

I just wanted to confirm that I'm understanding how to use OSFIND because I'm trying to create a log file for debuging and I'm getting bad corruption of not just the disc I have "inserted" from my MMB file but sometimes other discs as well. I could just stop my code (the Space Invaders clone I've been working on) and print some debug values to the screen, but it would just be more convenient to write stuff to a log file.

When I initially create my log file I LDA with #&80 (write access), load X & Y with the location where the filename is stored, and check A for zero to make sure there was no error opening it, then TAY so that I can close the file with:

LDA #0
JSR OSFIND

...Y, having already been loaded with the file ID. This works and creates the file. When I want to re-open the file to append, I LDA #&C0 and load X & Y with the location where the filename is stored again. My understanding is that I not only don't need to load Y with the previous file ID, but that it would be pointless because I need to load X & Y with the file name location as mentioned above, so Y would just be overwritten anyway. So executing OSFIND will again return a file ID for the current session that I can use to close the file when I'm done. I don't need to remember the original file ID at all because it's not used when reopening the file.

Is that correct?

If I run just a short test program that opens a log file and appends a dummy bit of data (10 '1's) this seems to work fine. If I try and call the (almost) same code from my Space Invaders program, but LDA with the actual values that I want to save before calling the append routine my disc is completely corrupted. You can't even get the catalogue. If you try, it displays some rubbish at the start and then it hangs.

At the moment the only data I'm trying to write is either 1 or 0 as a test, and before I load A with the value to be appended I add 48 to it so that I am writing character "0" or "1" using BPUT. As I say, standalone, this works and doesn't seem to be corrupting the disc.

The only 2 things I can think of is that I'm either not opening the file correctly, or it's filling up very fast. But would it just not fail to open or fail to be written to if the disc was full? The first time the corruption happened I was able to load about half the files on the disc, luckily all the critical ones that I had made changes to. But with all the others I just got "bad program", and with one I got the error "disc overflow". Since then though when the corruption happens I can't even list the catalogue as I said above.

I'd be glad for any input. But just for a start, am I using OSFIND correctly?

[tom_seddon]

Yes this use of OSFIND sounds right. You never need a previous file handle number when opening - that value becomes useless once you've passed it in to OSFIND to close the file. (You may get the same value back from a future open call of course. Handle numbers do get reused.)

If you're getting particularly weird behaviour from filing system operations, make sure you aren't using any of the following regions, particularly the zero page ones: $a0-$a7, $b0-$cf and $d00 to (wherever OSHWM is after Ctrl+BREAK - probably $1900). Easiest to assume for now these are all off-limits, and you can think about being a bit less cautious about this when things are working.

My own advice would be to recommend finding some alternative means of debugging... the throughput of random-access files is quite poor, and it'll be even worse if you flush often enough to ensure no data loss if the program crashes or you press BREAK. I've found writing data to sideways RAM to work pretty well, though reading it can be a bit annoying.

--Tom

[sweh]

I've found writing data to sideways RAM to work pretty well, though reading it can be a bit annoying.

EXMON is what I use for for raw reading SWR; use !<bank> (eg !07) and then reads from 8000-BFFF are from that bank; D8000 will do a hex dump, for example.

[emiliom]

Yes this use of OSFIND sounds right. You never need a previous file handle number when opening - that value becomes useless once you've passed it in to OSFIND to close the file. (You may get the same value back from a future open call of course. Handle numbers do get reused.)

Thanks, at least I know I'm using it right.

If you're getting particularly weird behaviour from filing system operations, make sure you aren't using any of the following regions, particularly the zero page ones: $a0-$a7, $b0-$cf and $d00 to (wherever OSHWM is after Ctrl+BREAK - probably $1900). Easiest to assume for now these are all off-limits, and you can think about being a bit less cautious about this when things are working.

I have definitely got code between &d00 - &1900 so that could be my problem. I'll move it up to &1900 and check to see what happens.

My own advice would be to recommend finding some alternative means of debugging... the throughput of random-access files is quite poor, and it'll be even worse if you flush often enough to ensure no data loss if the program crashes or you press BREAK. I've found writing data to sideways RAM to work pretty well, though reading it can be a bit annoying.

I did notice that in my standalone file append test it was pretty slow so I think you're right. Thing is I'm using a MiSTer system in Model B mode, so I want to say that didn't have sideways RAM? But having said that the boot prompt actually says "Model B MMFS SWRAM (FEA0)" so now I'm not sure. TBH I don't know much about how to use SWRAM so I'll have to look into that, but if it's available it seems like it it would be a good solution.

[hoglet]

I did notice that in my standalone file append test it was pretty slow so I think you're right. Thing is I'm using a MiSTer system in Model B mode, so I want to say that didn't have sideways RAM? But having said that the boot prompt actually says "Model B MMFS SWRAM (FEA0)" so now I'm not sure. TBH I don't know much about how to use SWRAM so I'll have to look into that, but if it's available it seems like it it would be a good solution.

Sounds like MiSTer is already pre-loading the SWRAM version into Sideways RAM.

The SWRAM version uses the 2.5KB at the end of the 16KB Sideways RAM slot as filesystem workspace. You can verify this is the case, if PRINT ~PAGE returns E00 rather than 1900.

What version of MMFS is being used? *HELP should tell you this. I suspect it wll be rather old (1.39? which is 6 years old now).

The other possible issue here is because MMFS is in SWRAM, it's more prone to getting corrupted if the application mis-behaves and randomly writes to memory.

Dave

[BeebMaster]

The SWRAM version uses the 2.5KB at the end of the 16KB Sideways RAM slot as filesystem workspace. You can verify this is the case, if PRINT ~PAGE returns E00 rather than 1900.

Dave

Is there a build of (8271) DFS which can do this?

[hoglet]

Is there a build of (8271) DFS which can do this?

There was a thread on this a couple of years ago:
viewtopic.php?t=17904

[emiliom]

The SWRAM version uses the 2.5KB at the end of the 16KB Sideways RAM slot as filesystem workspace. You can verify this is the case, if PRINT ~PAGE returns E00 rather than 1900.

Yes, it is &E00. I thought that was odd as my original BBC Model B by default had PAGE set to &1900. That explains why it's &E00.

What version of MMFS is being used? *HELP should tell you this. I suspect it wll be rather old (1.39? which is 6 years old now).

1.50

The other possible issue here is because MMFS is in SWRAM, it's more prone to getting corrupted if the application mis-behaves and randomly writes to memory.

Well I have DIMmed all the chunks of memory I use, and other than page zero user mem (&70-&8F) I don't deliberately write anything anywhere else, but then I have seen some odd random behaviour recently when I try and blow up an alien, so there's definitely something amiss. I did find a bug where I'd messed up my position in the stack which solved a big issue so I thought at first that might have been the problem, but it doesn't seem so.

[hoglet]

1.50

Ah, OK... Are you using Iain Sharp (LushProject)'s experimental core?
viewtopic.php?t=24343
https://github.com/sharpie7/BeebFpga_MiSTer/tree/master

MMFS 1.50 and earlier version of MMFS place several restrictions on the usage of the FAT32 file system on the SSD Card.
(1) - BEEB.MMB must be one of the first 16 files in the root directory
(2) - BEEB.MMB must be contiguous (i.e. it must not be fragmented)
(3) - BEEB.MMB must start in the first 128MB of the SD Card (*)
(4) - BEEB.MMB must fit within the first 8GB of the SD Card (it uses a 24-bit sector address)

(*) This was addressed in MMFS 1.51

(1) results in an Image? error which you are not getting.

If (2), (3) or (4) is violated, then you can get serious corruption of other disc images in the BEEB.MMB file, and of other files on the SD Card.

I'm not familiar with how MiSTer makes the SD Card visible to the core, but if the card is shared with Mister then you need to check you are not running into one of these constraints.

How large is the SD Card?

How large is the BEEB.MMB file?

Do you have multiple copies of the BEEB.MMB file on the Card?

Do ypu routinly re-write the BEEB.MMB file, e.g. switching versions, or restoring from a backup?

I suspect most users of the Beeb core on MiSTer are not developing new software, so are much less likely to be writing to the disk.

Dave

[emiliom]

Ah, OK... Are you using Iain Sharp (LushProject)'s experimental core?

I thought I was. I've been running the alternative core that's on the MiSTer system which I thought was Iain's core, but when I check the ROMs list I don't see 'Acorn DFS' which I thought was an option on Iain's core from what I read in the documentation, so now I'm not sure. The core file is dated 20/03/2022.

MMFS 1.50 and earlier version of MMFS place several restrictions on the usage of the FAT32 file system on the SSD Card.
(1) - BEEB.MMB must be one of the first 16 files in the root directory
(2) - BEEB.MMB must be contiguous (i.e. it must not be fragmented)
(3) - BEEB.MMB must start in the first 128MB of the SD Card (*)
(4) - BEEB.MMB must fit within the first 8GB of the SD Card (it uses a 24-bit sector address)

(1) results in an Image? error which you are not getting.

If (2), (3) or (4) is violated, then you can get serious corruption of other disc images in the BEEB.MMB file, and of other files on the SD Card.

I am good on (1),(3), and (4). (2) I have to check, but the .MMB file is the only file on the card so I don't think it would be fragmented.

I'm not familiar with how MiSTer makes the SD Card visible to the core, but if the card is shared with Mister then you need to check you are not running into one of these constraints.

It's my understanding that the MiSTer doesn't use the second SD card (where the .MMB file resides) directly for any general 'MiSTer' stuff, it's just there in case a core requires it for the specific core to use, as in the case of the Beeb core. So nothing else is being written to the card by the MiSTer as far as I know. There are certainly no other files on the card.

How large is the SD Card?

8GB

How large is the BEEB.MMB file?

102,208KB

Do you have multiple copies of the BEEB.MMB file on the Card?

No. As I said above it's the only file on the card.

Do ypu routinly re-write the BEEB.MMB file, e.g. switching versions, or restoring from a backup?

I've only restored it twice and that was after this recent corruption. I just overwrote the MMB file with the backup copy.

[hoglet]

I thought I was. I've been running the alternative core that's on the MiSTer system which I thought was Iain's core, but when I check the ROMs list I don't see 'Acorn DFS' which I thought was an option on Iain's core from what I read in the documentation, so now I'm not sure. The core file is dated 20/03/2022.

That does match the most recent release of Iain's core:
https://github.com/sharpie7/BeebFpga_Mi ... r/releases

As you are using the second SD Card slot, it should behave very like an MMFS installation on a real Beeb.

I'd be happy to take a look and try to reproduce the issue, if you could provide a raw image of the whole SD card.

Dave

[emiliom]

I'd be happy to take a look and try to reproduce the issue, if you could provide a raw image of the whole SD card.

Thanks for the offer Dave but it seems like tom had the answer all along:

If you're getting particularly weird behaviour from filing system operations, make sure you aren't using any of the following regions, particularly the zero page ones: $a0-$a7, $b0-$cf and $d00 to (wherever OSHWM is after Ctrl+BREAK - probably $1900).

When I build the code I was loading the larger files at &F00, so I moved things up to &1900 as per tom's advice. I just re-enabled the log file and now I'm not getting corruption any more. I can catalogue the disc ok, I checked all the files on the disc, plus other discs that had previously got corrupted and they're all ok. I ran it multiple times just to check it wasn't a one off, but no, that seems to have been the issue.

Of course it's turned out to be completely useless for my purposes as the file access is so slow compared to the game speed that the game just hangs. But it does actually write data to the log file, so it does work. At least I know to avoid &E00-&1900 when working with files.

[hoglet]

When I build the code I was loading the larger files at &F00, so I moved things up to &1900 as per tom's advice. I just re-enabled the log file and now I'm not getting corruption any more. I can catalogue the disc ok, I checked all the files on the disc, plus other discs that had previously got corrupted and they're all ok. I ran it multiple times just to check it wasn't a one off, but no, that seems to have been the issue.

Of course it's turned out to be completely useless for my purposes as the file access is so slow compared to the game speed that the game just hangs. But it does actually write data to the log file, so it does work. At least I know to avoid &E00-&1900 when working with files.

That doesn't really make sense to me, as you are using the SWRAM version of MMFS. The whole point of that is to free up the memory from &E00-&!900 for use by programs.

So either their is a bug in the SWRAM version of MMFS (that means it's still using memory from &E00 - &1900), or something else is going on.

Dave

[emiliom]

That doesn't really make sense to me, as you are using the SWRAM version of MMFS. The whole point of that is to free up the memory from &E00-&!900 for use by programs.

Ok, so I tried it again with a new SD card. I formatted it and I copied my current .MMB file over to it and tried it with PAGE set first to &1900 and then &0F00 and they both worked. The only difference I can see is that with my original card the file has been overwritten twice and this second card was newly formatted.

So would it be a good idea to reformat my SD card if I have to restore the .MMB from a backup?

[hoglet]

So would it be a good idea to reformat my SD card if I have to restore the .MMB from a backup?

That is probably the safest option.

I wouldn't have thought overwriting the same file with one of the same length would cause fragmentation, but it might depend on the client doing the writing. I do this all the time on Ubuntu 18.04 without issue. Ubuntu has a command (filefrag) that will report if a file is fragmented. I've not found a simple way to check on Windows yet.

Dave

[SKS1]

So would it be a good idea to reformat my SD card if I have to restore the .MMB from a backup?

That is probably the safest option.

I wouldn't have thought overwriting the same file with one of the same length would cause fragmentation, but it might depend on the client doing the writing. I do this all the time on Ubuntu 18.04 without issue. Ubuntu has a command (filefrag) that will report if a file is fragmented. I've not found a simple way to check on Windows yet.

Dave

You can use Sysinterals contig command (now MS: https://learn.microsoft.com/en-us/sysin ... ads/contig)

e.g.

Code: Select all

contig -a foobar

[emiliom]

I wouldn't have thought overwriting the same file with one of the same length would cause fragmentation, but it might depend on the client doing the writing. I do this all the time on Ubuntu 18.04 without issue. Ubuntu has a command (filefrag) that will report if a file is fragmented. I've not found a simple way to check on Windows yet.

Nor me, but best be safe. I tested the files I managed to save from the MMB file with the corrupted disc on a reformatted SD card and again no issues so all seems good at the moment. But as I said I've abandoned that debug code now anyway because it was too slow so I don't imagine the problem will crop up again.

[emiliom]

You can use Sysinterals contig command (now MS: https://learn.microsoft.com/en-us/sysin ... ads/contig)

e.g.

Code: Select all

contig -a foobar

Cheers, downloaded that.