PLASMA virtual machine

[shifters74]

Hi,

I will try the +x permission on HELLO tomorrow as suggested and let you know.

I also have another server which is running a true FS3 implementation (FSEM2) but under emulation so will try the same with that and see what the behaviour is and let you know.

I will also try

Code: Select all

OSCLI "SAVE NEW "+STR$~data+"+100"
*DUMP NEW

cheers

shifters

[SteveF]

Thanks, that will be interesting!

[shifters74]

Hi,

I will try the +x permission on HELLO tomorrow as suggested and let you know.

I also have another server which is running a true FS3 implementation (FSEM2) but under emulation so will try the same with that and see what the behaviour is and let you know.

I will also try

Code: Select all

OSCLI "SAVE NEW "+STR$~data+"+100"
*DUMP NEW

cheers

shifters

Adding +x made no difference at the PiFS file system level. The .inf file has file permissions which are 15 which is used for LIB and all other files.

Running the TESTLD and doing OSCLI and the NEW file saved has the Hello World text within it!!

Running PLASMA and +Hello from FSEM2 (a real FS3 software server) gives the same as running from PiFS - Hello not found

Hope that helps!

shifters

[SteveF]

Adding +x made no difference at the PiFS file system level. The .inf file has file permissions which are 15 which is used for LIB and all other files.

Running the TESTLD and doing OSCLI and the NEW file saved has the Hello World text within it!!

Running PLASMA and +Hello from FSEM2 (a real FS3 software server) gives the same as running from PiFS - Hello not found

Hope that helps!

Thanks very much for trying that!

I can't help feeling NFS (obviously nothing to do with the PiFS implementation) is technically wrong here but even if I'm right about that, it's obviously just one of the quirks you have to be aware of with OSFILE. There are lots documented on BeebWiki, I don't see this one so maybe we could add it if someone can give me a second opinion that this is technically wrong.

I suspect in practice that because loading a file with A=&FF is supposed to generate an error if it fails and every filing system will implement that correctly, checking the return value in A afterwards is very rare.

I just experimented with ADFS 1.50 and although you can use OSFIND to open a directory for byte access, you can't use OSFILE A=&FF to load it into memory. This makes me think that checking the return value in A after OSFILE A=&FF to see if we just loaded a directory is probably pointless, especially after we already checked the &6502 magic word in the module header. This will also shrink the VM by a few bytes as a bonus.

Does anyone have any thoughts on this before I go ahead and make the change?

[Coeus]

I just experimented with ADFS 1.50 and although you can use OSFIND to open a directory for byte access, you can't use OSFILE A=&FF to load it into memory....

What does it do when you try?

I have re-read the BeebWiki article and I can't find it there but I think I remember JGH saying somewhere previously that while filing systems should return the type of object in response to other calls, in actual implementation you can only rely on getting the right object type code in response to the call with A=05 (read file information).

The other thing is that the level of checking what the user is attempting and making sure they don't do something silly has gradually grown over time. There wasn't as much BITD and there are plenty of crazy things the machine does let you do, like *LOAD a file too big to fit in memory so it writes to the I/O area then, if it hasn't crashed by then, overwrite page zero and the stack. Or put a language ROM in the main machine that gets copied and relocated to a tube processor that overwrites the RAM copy of the tube ROM on the Z80.

So I think it will be fine to stop checking the return value of OSFILE and assume, if it returns at all, that the file is loaded and can be run.

[shifters74]

If you want me to test a beta version over econet with a fix - just let me know!

I can also test over DFS and MMFS also.

shifters

[SteveF]

I just experimented with ADFS 1.50 and although you can use OSFIND to open a directory for byte access, you can't use OSFILE A=&FF to load it into memory....

What does it do when you try?

It just generates a "Not found" error. (I tested this using the TESTLD program I posted further up the thread, but with HELLO being a directory instead of a regular file, because I'm lazy. )

I have re-read the BeebWiki article and I can't find it there but I think I remember JGH saying somewhere previously that while filing systems should return the type of object in response to other calls, in actual implementation you can only rely on getting the right object type code in response to the call with A=05 (read file information).

Well, it certainly seems like he's right if he did say that!

The other thing is that the level of checking what the user is attempting and making sure they don't do something silly has gradually grown over time. There wasn't as much BITD and there are plenty of crazy things the machine does let you do, like *LOAD a file too big to fit in memory so it writes to the I/O area then, if it hasn't crashed by then, overwrite page zero and the stack. Or put a language ROM in the main machine that gets copied and relocated to a tube processor that overwrites the RAM copy of the tube ROM on the Z80.

I do know what you mean, although if it's not too painful I always think it's nice to check. I agree this isn't a particularly problematic case though, especially as we already have the magic number check earlier.

Incidentally, what *does* feel like a booby-trap in PLASMA is that while +FOO will sanity-check that FOO is a valid module, if you type *FOO the OS will just execute FOO and if it's actually a PLASMA module it will crash. But short of adding the delay caused by a trial open of FOO and checking it *doesn't* have the magic number before passing the call through to OSCLI, which would probably by annoying and/or tricky (I suspect you'd need to play games to only do this if a sideways ROM doesn't pick the call up, to do it right), I don't see much of an alternative. (Edit: I suppose PLASMA could temporarily hook OSFSC immediately before running a module - a little fiddly over the tube but not terrible - and do this check if it's entered with A=3, then restore the original OSFSC. I don't think it could install a handler on startup as it would probably be un-installed if the user changes filing system via a * command, hence just installing it temporarily before the module load. I might have a play with this.) (Edit 2: I suppose an alternative might just be to make a convention of putting PLASMA modules in a specific directory so they can't be found by accident via an implicit *RUN in the first place. This just feels a little intrusive as it "hides" them on a hierarchical filing system, and the self-hosted compiler would need tweaking to generate them in the right place.) (Edit 3: I suppose the FSCV handler we install could simply refuse to *RUN anything implicitly. The user can always do */FOO or *RUN FOO if they really mean it, and at the PLASMA prompt it's unlikely they will be wanting to use the implicit *RUN behaviour anyway. Then again, it's probably no big deal to just open the file and check the magic number.)

So I think it will be fine to stop checking the return value of OSFILE and assume, if it returns at all, that the file is loaded and can be run.

Thanks, I've done that now and pushed the change to the merge16 branch. This doesn't check the return value of OSFILE and as a result should work on NFS and has an extra 18 bytes free.

If you want me to test a beta version over econet with a fix - just let me know!

I can also test over DFS and MMFS also.

Thanks, if you can get the latest version of the merge16 branch and test that that would be great! I've given this a very quick test myself and I don't think I've broken it, but I haven't tested on anything except Acorn DFS. If you're already on the merge16 branch you should just be able to do:

Code: Select all

git pull
make

If you want to double check you have the fix, you can do:

Code: Select all

git log

and the top entry should mention "Don't check return value of OSFILE".

[Coeus]

...(Edit 2: I suppose an alternative might just be to make a convention of putting PLASMA modules in a specific directory so they can't be found by accident via an implicit *RUN in the first place. This just feels a little intrusive as it "hides" them on a hierarchical filing system, and the self-hosted compiler would need tweaking to generate them in the right place.)

I have been caught out by this. It doesn't help that the keys are adjacent on the BBC micro keyboard and the symbols even look slightly similar.

I was going to mention FSCV but you beat me to it and had obviously thought about it in more detail than I had.

Trying to solve this is working around a problem that exists elsewhere. CP/M had file name extensions which were often used as a file type and could consequently be used so the command interpreter could execute COM modules as machine code and BAT files as its own batch files, even without any header on the file signifying what it was. It is not perfect but it is better than blindly executing any file at all.

Directories on DFS tend to be used the same way, as file types, but as they weren't there on TAPE, there is no convention for a directory for executable code. Strangely DFS does have a library directory - it would make more sense if it only executed code from this directory, at least when looking for a file to match an unrecognised command - *RUN could still run things from the current directory.

Then with the same separator, and the same *DIR command, directories become proper directories on hierarchical filing systems and the "file type" disappears again. So I can see why the filing system doesn't check the directory before executing anything though, as it is the filing system, each could have their own convention.

Then there is typically no header on executable code. Later it became a thing to put a standard ROM header on a file to mark it as executable and communicate more about it, for example which tube processor it is supposed to run on but that hasn't replaced executing headerless files. The Acorn filing systems make up for the lack of header by having the load/exec address in the directory entry for the file so this is another check that could have been done: reserve an exec address to mean "not executable" but again it would be hard to introduce retrospectively.

Back to PLASMA, does the VM accept any other commands? It may make the mistake less likely if an otherwise unrecognised command to the PLASMA VM is run as a module, i.e. without needing to type either the '+' or the '*'.

[Coeus]

And interesting after mentioning ROM headers on disc files, this should turn up: viewtopic.php?p=357295#p357295

[SteveF]

It is unfortunate that there's no mandatory executable header, but I guess we are where we are. I've been following the OSBYTE &A4 thread with interest.

Back to PLASMA, does the VM accept any other commands? It may make the mistake less likely if an otherwise unrecognised command to the PLASMA VM is run as a module, i.e. without needing to type either the '+' or the '*'.

It does accept "M"/"MODE" (or variants; the parsing is deliberately crude to keep the size of the code down) to change mode but nothing else. This isn't a bad suggestion, but what puts me off is that the "+" prefix is something Apple PLASMA uses too and it just feels like part of the language environment.

I think I will experiment with the FSCV approach at some point. My current thinking is to install a handler for USERV which temporarily points FSCV at an "extra logic on A=3" handler on *FX136,1 and restores the previous FSCV on *FX136,0. This way the core VM only has to have a little extra code (and thus bloat) to call these around its OSCLI call when executing "*" commands from the prompt, and I can make the filing system responsible for installing the code in the host if we're on a second processor. What seems a little unsatisfactory here is that it uses an extra file up on disc and the PLASMA VM will have to execute that on startup. It would also be mandatory, but if I just make it the user's job to *RUN the code to implement this if they want it, a) they will forget most of the time when it is probably good to have *on* by default b) the *FX136 calls will generate an OS error if the FSCV wrapper hasn't been *RUN rather than being no-ops, so the VM has to have extra code to trap and ignore those errors.

[SteveF]

Picking up on something I said over in the Sideways ROM software development thread:

I did intend (I suppose I still do, eventually ) to make it possible to bundle the PLASMA VM (which is a few K in size) into a language ROM with the user's compiled bytecode, which would make it possible to write a standalone language ROM in PLASMA.

I've been having a look into this and subject to a few caveats (you can't just take an arbitrary PLASMA program and expect it to run from ROM; most notably, all global variables are read-only) I think this is feasible. I have a prototype "no language environment" language ROM written in PLASMA that just accepts * commands from the keyboard and feeds them to OSCLI. That occupies about 3.25K of 16K available, which includes the VM, a bit of the PLASMA library code and a simple service handler to accept a * command to start the language and respond to *HELP. So I think there'd be enough space to write a serious application in PLASMA in a 16K ROM.

PLASMA is currently covered by LGPL3 and building a standalone ROM image using the PLASMA VM/library is essentially a form a static linking, which (as I understand it) can be tricky to do in an LGPL3-compiliant way. I've spoken to Dave and he has very kindly offered to relicence the code if it will help. Both of us currently seem to lean towards the MIT licence, does anyone have any thoughts on this? I believe with the MIT licence anyone would be able to write a language ROM in PLASMA and distribute it without having to release their source code if they wished.

Would that mean the 16K ROM holds the VM and your plasma byte code and then main memory is used for data (of the VM and your app)?

The entire VM in 3.5K WOW!!

I forgot to reply to this earlier with all the NFS excitement. Yes, that's right - the PLASMA VM and your PLASMA code would live in the 16K ROM and main RAM would be free for the user data (the PLASMA heap and frame stack). It should also be possible to write parts of the ROM in assembly language if you need the extra speed and/or some things can be more compactly expressed in machine code than PLASMA bytecode.

[shifters74]

Hi,

did a git pull, rebuilt the ssd's, exported them and moved to the econet server.

Ran *PLASMA, then ran +HELLO - works fine!!! Ran +TEST - works fine!!!

Fired up the co-pro (6502 JIT with 64K default) - tried again - all ran fine.

One thing i did notice on the co-pro - there only seems about $4CEE bytes free of the 64K - is this expected???

Good job on the fix, thanks for your work on that!!!

shifters

[Coeus]

It does accept "M"/"MODE" (or variants; the parsing is deliberately crude to keep the size of the code down) to change mode but nothing else. This isn't a bad suggestion, but what puts me off is that the "+" prefix is something Apple PLASMA uses too and it just feels like part of the language environment.

Ok, I wasn't suggesting that +module be made not to work, so it would still feel natural to someone with a virgin VM. More like how just the filename as a command can be used in place of *RUN so a bare filename could be considered as a module to run if it doesn't match anything else.

I think I will experiment with the FSCV approach at some point. My current thinking is to install a handler for USERV which temporarily points FSCV at an "extra logic on A=3" handler on *FX136,1 and restores the previous FSCV on *FX136,0. This way the core VM only has to have a little extra code (and thus bloat) to call these around its OSCLI call when executing "*" commands from the prompt, and I can make the filing system responsible for installing the code in the host if we're on a second processor.

Assuming OSBYTE 136 does call the code pointed to be the user vector in the I/O processor, that is a neat solution.

What seems a little unsatisfactory here is that it uses an extra file up on disc and the PLASMA VM will have to execute that on startup. It would also be mandatory, but if I just make it the user's job to *RUN the code to implement this if they want it, a) they will forget most of the time when it is probably good to have *on* by default b) the *FX136 calls will generate an OS error if the FSCV wrapper hasn't been *RUN rather than being no-ops, so the VM has to have extra code to trap and ignore those errors.

It would be possible to install a small amount of code on the I/O processor with OSWORD A=6 but at one byte at a time that is a bit clumsy. The other possibility, as the VM is so small, is to have it load into the I/O processor even if there is a tube by setting the exec address accordingly, then it can transfer itself over the tube. Obviously that would mean checking the tube is there, then making calls into the tube host code, i.e. the entry point at 0403 so start the transfer. Actually sending the data is a case of writing directly to the tube hardware. This is what the filing systems do and it would be possible to inspect one for which we have the source to see how it is done. Of course, it would not have to transfer either the code hooked into USERV or the code to do the copy over the tube, just the actual VM. That would then be one file on disc.

[SteveF]

did a git pull, rebuilt the ssd's, exported them and moved to the econet server.

Ran *PLASMA, then ran +HELLO - works fine!!! Ran +TEST - works fine!!!

Great, thanks for testing that!

Fired up the co-pro (6502 JIT with 64K default) - tried again - all ran fine.

One thing i did notice on the co-pro - there only seems about $4CEE bytes free of the 64K - is this expected???

This doesn't seem right. I have $D793 free with "PLASMA" and $978E with "PLASJIT" on a 6502 second processor. Which version of the VM are you using? It doesn't sound like you are, but do note that *PLAS128/*P128JIT will silently disable the second processor and run on the host instead. I should perhaps change that so they just generate an error, I thought I was being clever at the time but it's perhaps just confusing.

Can you do *INFO PLASMA (or whichever VM(s) you see this on)? If the load/execution addresses don't start with "00" they will probably try to run on the host even when the second processor is present.

Good job on the fix, thanks for your work on that!!!

You're welcome, thanks again for your help. Although maybe we're still not quite there given this second processor problem.

[SteveF]

It does accept "M"/"MODE" (or variants; the parsing is deliberately crude to keep the size of the code down) to change mode but nothing else. This isn't a bad suggestion, but what puts me off is that the "+" prefix is something Apple PLASMA uses too and it just feels like part of the language environment.

Ok, I wasn't suggesting that +module be made not to work, so it would still feel natural to someone with a virgin VM. More like how just the filename as a command can be used in place of *RUN so a bare filename could be considered as a module to run if it doesn't match anything else.

Oh, I see what you mean. I think I prefer the approach of stopping *FOO causing a crash where FOO is a module, but if I that didn't look like it would be possible this might be worth trying.

Assuming OSBYTE 136 does call the code pointed to be the user vector in the I/O processor, that is a neat solution.

I'm fairly sure it will; I use it in Ozmoo when running on a second processor to control a cache running in the host processor to take advantage of spare sideways RAM. But I haven't tried it in this context yet. It's just a shame there's no similar variant which will be a no-op by default instead of generating an error by default. That said, given the next bit, this is probably not really a concern.

It would be possible to install a small amount of code on the I/O processor with OSWORD A=6 but at one byte at a time that is a bit clumsy.

To be fair, this isn't necessarily that big a deal. I could do it in the discardable init code so the clumsiness wouldn't waste runtime RAM, just a little disc space (and not much more than having a separate *RUNnable file to install this - perhaps even less if you think in terms of disc sectors used instead of actual code size). Thanks, I will maybe have a look at trying it this way! (Edit: Not a big deal, but I need to remember to set up USERV again on a soft reset when running on a second processor. This isn't specific to this particular OSWORD A=6 technique, of course.)

The other possibility, as the VM is so small, is to have it load into the I/O processor even if there is a tube by setting the exec address accordingly, then it can transfer itself over the tube. Obviously that would mean checking the tube is there, then making calls into the tube host code, i.e. the entry point at 0403 so start the transfer. Actually sending the data is a case of writing directly to the tube hardware. This is what the filing systems do and it would be possible to inspect one for which we have the source to see how it is done. Of course, it would not have to transfer either the code hooked into USERV or the code to do the copy over the tube, just the actual VM. That would then be one file on disc.

This is clever! It feels like overkill in this situation though, if I can possibly avoid it. Definitely an idea to keep in mind for any future tube-related shenanigans though.

[SteveF]

On a random note: I am tinkering with the code for building ROMs with PLASMA, and I am seeing really bizarre behaviour (which isn't obviously related to PLASMA) where calling OSRDCH from inside a language ROM just hangs - the "JSR OSRDCH" never returns, getting stuck in some OS code. It's probably just a random bug, but is there any known gotcha in this area?! (Edit: I can use OSWORD 0 to read the keyboard just fine...)

Edit: It turns out language ROMs have to enable interrupts. The AUG does say this, but I hadn't noticed it/had just assumed it meant "don't go turning interrupts off in your language ROM". Still, at least I figured it out eventually.

[shifters74]

Can you do *INFO PLASMA (or whichever VM(s) you see this on)? If the load/execution addresses don't start with "00" they will probably try to run on the host even when the second processor is present.

False alarm.

I did *INFO PLASMA and it was starting with 00 - tried running *PLASMA on co-pro again and it came up $D793 free.

I must have done something stupid!

shifters

[Coeus]

Oh, I see what you mean. I think I prefer the approach of stopping *FOO causing a crash where FOO is a module, but if I that didn't look like it would be possible this might be worth trying.

I have just though of another fly in the ointment with the FSCV approach. Although filing systems are paged ROMs and therefore get offered the service call for an unknown command, they may not search the full table of commands in the ROM at this point but only those that make sense even when that filing system is not the current one.

That would therefore, obviously, include the command(s) to select that filing system as well as any general utilities that have been written to work with any filing system. That may therefore also include the likes of *BUILD, *DUMP etc.

For commands that only make sense when that filing system is selected, it may well delay the search until the call to FSCV. That saves including a test for "am I the current filing system" as, if it isn't, the call to FSCV never comes. So *ACCESS would be handled by this second call as it does something slightly different depending on whether it is DFS, ADFS, NFS etc.

So by intercepting this and searching for a file on disc, this would introduce a small delay to commands that would otherwise have executed from ROM. It may not be that significant because many of the filing system commands will access the disc anyway and by then the disc will already be spinning and ready with the catalogue cached after the first access. That would be slightly quicker if you could encode that a file is a PLASMA module in the load/exec addresses so that access to the actual file (and this a seek) isn't necessary, at least unless it looks likely it could be a PLASMA module.

This is clever! It feels like overkill in this situation though, if I can possibly avoid it. Definitely an idea to keep in mind for any future tube-related shenanigans though.

It would probably be more work that using OSWORD to read/write I/O processor memory. That OSWORD would have been more friendly for this if it had been specified to read/write a variable number of bytes but it shouldn't be hard to write code to increment the address directly in the control block and call it as many times as necessary.

Back to sending from the I/O processor to the tube, I wondered at the call to the tube host only being to claim the tube initiate the transfer, not to do the actual work, but this is so filing systems don't have to store the data temporarily in the I/O processor before transmission. For disc-based filing systems, for example, the NMI code can read from the FDCs data register and write straight to a tube register (or the other way round when writing to a file).

[SteveF]

False alarm.

I did *INFO PLASMA and it was starting with 00 - tried running *PLASMA on co-pro again and it came up $D793 free.

I must have done something stupid!

Thanks for trying that, I'm glad it worked. If it it happens again please let me know; you might have done something wrong, but it may be there's a weird intermittent glitch in the PLASMA code.

[SteveF]

I have just though of another fly in the ointment with the FSCV approach. Although filing systems are paged ROMs and therefore get offered the service call for an unknown command, they may not search the full table of commands in the ROM at this point but only those that make sense even when that filing system is not the current one.

That would therefore, obviously, include the command(s) to select that filing system as well as any general utilities that have been written to work with any filing system. That may therefore also include the likes of *BUILD, *DUMP etc.

For commands that only make sense when that filing system is selected, it may well delay the search until the call to FSCV. That saves including a test for "am I the current filing system" as, if it isn't, the call to FSCV never comes. So *ACCESS would be handled by this second call as it does something slightly different depending on whether it is DFS, ADFS, NFS etc.

So by intercepting this and searching for a file on disc, this would introduce a small delay to commands that would otherwise have executed from ROM. It may not be that significant because many of the filing system commands will access the disc anyway and by then the disc will already be spinning and ready with the catalogue cached after the first access. That would be slightly quicker if you could encode that a file is a PLASMA module in the load/exec addresses so that access to the actual file (and this a seek) isn't necessary, at least unless it looks likely it could be a PLASMA module.

Curses, you're right. I had implemented this as an experiment (not perfectly, but enough to test the principle) and I've just tried it and at least with DFS 1.20 (and probably others) "*INFO X" doesn't work at the PLASMA prompt now, presumably because of precisely the issue you've pointed out (even though "X" exists).

My current implementation just automatically generates a "Bad command" error if it sees FSCV called with A=3 during a * command issued from the PLASMA prompt. So it looks like a proper fix is going to involve parsing (admittedly very minimally) the command line to see if the first "word" is actually the name of a PLASMA module and only generating "Bad command" if that's the case, otherwise passing the FSCV call through to the parent. (It actually gets a little worse than this, because if the user has done *LIB, there might be a file BAZ which can't be opened as "BAZ" using OSFIND but which will be executed if we allow the FSCV A=3 call to succeed. But I guess that isn't very likely to be a problem, as there's probably no reason the user should dump non-executables in a directory selected with *LIB.)

The load/exec address idea is interesting. We could give PLASMA modules "distinctive" load/exec addresses (they are otherwise meaningless) and if a *FOO command is executed and file "FOO" matches the distinctive load/exec addresses, we could probably assume it's a PLASMA module without bothering to open it, especially if the load/exec addresses were something which make no sense for a real executable to have. If it doesn't match, we *could* open it and check the magic number, but it might be as well just to allow it through.

I did wonder if it might be possible to give PLASMA modules "harmless" load/exec addresses, e.g. load at &FFFFFF and execute at the address of an RTS somewhere and sidestep the whole issue, so they *are* safe to *RUN. But unless I'm missing something this doesn't seem to be possible, even just doing "*LOAD JUNK FFFFFF" hangs, presumably because the filing system is wrapping around to zero page. On a B/B+ you can of course safely load 14-15K at &FFFFC000 before you hit the I/O space, but that won't work on a Master. (Edit: I suppose loading at FFFF8000 might not be too bad a compromise, if I do decide to give up on the FSCV approach. This would break on SWMMFS or any other filing system running in a RAM bank, but at least some users would be protected against accidentally doing *FOO where FOO is a module. But on further reflection, this is *worse* for SWMMFS-and-similar users as currently doing *FOO would probably crash the machine, but with this change it will trash their filing system and require it to be reloaded. So probably not a good idea.)

It's also a good point that a lot of the affected commands are going to spin up the disc anyway.

This is starting to feel like a lot of effort, but it's quite interesting as well and the usability gotcha does annoy me. Except for the slightly annoying fact I'm currently using &900 for this code when that was previously left untouched by PLASMA (and thus free for other uses, e.g. transient commands or the PLASMA program if it wants to access that region or whatever), there's not too much of a runtime penalty for writing "complex" code here, as long as it (say) fits in the 256 bytes at &900 - it's not like the VM itself is growing and the free RAM for programs is decreasing accordingly.

It would probably be more work that using OSWORD to read/write I/O processor memory. That OSWORD would have been more friendly for this if it had been specified to read/write a variable number of bytes but it shouldn't be hard to write code to increment the address directly in the control block and call it as many times as necessary.

Yes, I wish it had been capable of doing multiple bytes at a time, but it's nice that it at least exists all the time (i.e. you can use it without caring if you have a second processor or not, rather than the transfer OSWORD implementation being in the tube host code and failing on machines with no copro).

Edit: There's nothing terribly exciting there, but if you're interested you can see my prototype code on the no-implicit-run branch in the repo. The USERV code and the FSCV handler it temporarily installs can be seen at USERVHANDLERCOPY here. I think this is buggy quite apart from what you've just pointed out: it will probably get in a mess if filing systems are changed while PLASMA is running under some circumstances. I had been sketching out a tweaked version in my head but I haven't implemented this yet.

[Coeus]

...So it looks like a proper fix is going to involve parsing (admittedly very minimally) the command line to see if the first "word" is actually the name of a PLASMA module and only generating "Bad command" if that's the case, otherwise passing the FSCV call through to the parent.

Although the filing system calls are documented as taking a filename terminated by CR, most seem to also treat space as terminating the filename so you could be lazy and just put the address of the whole command line tail into a call to search for the file.

(It actually gets a little worse than this, because if the user has done *LIB, there might be a file BAZ which can't be opened as "BAZ" using OSFIND but which will be executed if we allow the FSCV A=3 call to succeed. But I guess that isn't very likely to be a problem, as there's probably no reason the user should dump non-executables in a directory selected with *LIB.)

I don't think that many people use a lib direcory, certainly not on DFS. It would also be less common on ADFS floppies, probably more common on an ADFS hard disc but by then someone has presumably chosen carefully what to put in there.

I was thinking you might use OSFILE with A=5. Like OSFIND, this returns even if the file is not found rather than generating an error, or at least it is supposed to. Perhaps I was worrying about the delay too much as the most common case is that a file will not be found that matches the name of the command and thus only the catalogue/current directory would need to be read. If you did go for a distinctive load/exec address combination then you could probably also run out some files without incurring a seek to read the file data.

Edit: but perhaps that's an argument for doing the full header check. It will only happen when a file exists with the name of the command concerned so either when the user has a real executable with that name or has mistyped the + as *.

I did wonder if it might be possible to give PLASMA modules "harmless" load/exec addresses, e.g. load at &FFFFFF and execute at the address of an RTS somewhere and sidestep the whole issue, so they *are* safe to *RUN.

I think that's what BASIC is doing when it uses PAGE and its own entry address but that is far from foolproof. But of all the options, loading into language RAM and re-starting the current langauge us probably safest. For PLASMA you could add the size of the VM to page but then the address probably wouldn't be distinctive. I believe RISCOS uses the load and execution addresses as filetype.

[SteveF]

...So it looks like a proper fix is going to involve parsing (admittedly very minimally) the command line to see if the first "word" is actually the name of a PLASMA module and only generating "Bad command" if that's the case, otherwise passing the FSCV call through to the parent.

Although the filing system calls are documented as taking a filename terminated by CR, most seem to also treat space as terminating the filename so you could be lazy and just put the address of the whole command line tail into a call to search for the file.

Thanks for the tip! I have taken advantage of this for now but it feels a bit like asking for trouble with obscure filing system X so I may tweak it to temporarily patch the first character <=32 to CR before calling OSFIND and then reverting it afterwards.

I was thinking you might use OSFILE with A=5. Like OSFIND, this returns even if the file is not found rather than generating an error, or at least it is supposed to. Perhaps I was worrying about the delay too much as the most common case is that a file will not be found that matches the name of the command and thus only the catalogue/current directory would need to be read. If you did go for a distinctive load/exec address combination then you could probably also run out some files without incurring a seek to read the file data.

Edit: but perhaps that's an argument for doing the full header check. It will only happen when a file exists with the name of the command concerned so either when the user has a real executable with that name or has mistyped the + as *.

I was tempted to go with OSFILE A=5, but what put me off was that BeebWiki says:

• You can't rely on filing systems to put standard values on newly-created files, so I'd have to add code to the self-hosted compiler to set the load/exec addresses correctly.
• Some filing systems might generate an error instead of returning with A=0 when using OSFILE A=5.

So for the moment I've written some code to use OSFIND-open-for-input, call BGET four times and then close the file. If the last two BGETs return the magic number we generate a "Bad command" error, otherwise we forward to the parent FSCV. It's on the same branch I mentioned before (no-implicit-run) if anyone would like to try it out or just look at it.

As you say, the fact this will only access non-catalogue data when the file actually exists probably reduces the practical performance impact of this. Doing something like "*INFO FOO" will be fractionally slower (at least on a B; I think on a Master this doesn't go via FSCV, but I could be wrong, I just thought I observed it not being broken on the Master before I made the code changes to fix this) but probably not so as you'd notice.

Edit: I also tweaked the code so it is better about not getting into loops where it is its own parent vector handler. There may be a corner case I've overlooked but you can now do "*PLASMA" from the PLASMA prompt to re-launch the VM without it hanging, for example.

[Coeus]

What you've settled on seems sensible.

...(at least on a B; I think on a Master this doesn't go via FSCV, but I could be wrong, I just thought I observed it not being broken on the Master before I made the code changes to fix this) but probably not so as you'd notice.

I am not sure this is specific to the Master but there is an FSCV call specifically for *INFO. Certainly on the Master there has been an effort to de-duplicate some of the filing system common commands that were previous implemented by both DFS and ADFS so the OS parses *INFO and the calls FSCV with A=10 rather than with A=3.

[SteveF]

What you've settled on seems sensible.

...(at least on a B; I think on a Master this doesn't go via FSCV, but I could be wrong, I just thought I observed it not being broken on the Master before I made the code changes to fix this) but probably not so as you'd notice.

I am not sure this is specific to the Master but there is an FSCV call specifically for *INFO. Certainly on the Master there has been an effort to de-duplicate some of the filing system common commands that were previous implemented by both DFS and ADFS so the OS parses *INFO and the calls FSCV with A=10 rather than with A=3.

That makes sense. In hindsight, I was being overly general and talking about FSCV when I really should have said "FSCV with A=3". The fact the Master was avoiding the buggy behaviour trap I'd put on the A=3 case would have made it work there.

[SteveF]

I've had a go at adding support for writing standalone language ROMs with PLASMA. This means that you can create an "application ROM" like Wordwise or ViewSheet using PLASMA, with the fact it is written in PLASMA instead of machine code being invisible to the user. (Not that it's anything to be ashamed of. My point is that it "just works", like any other language ROM.)

The code to do this is a little bit hacky and experimental but I think it should be reasonably solid. You can find it on the rom branch in my repository: https://github.com/ZornsLemma/PLASMA/tree/rom I've also created a stardot9 tag with the current works-for-me-at-least version of the code.

Dave has very kindly altered the PLASMA licence to MIT and that change has also been merged in. Thanks again Dave!

Given a compiled PLASMA module pnle.mo, you can turn it into a langage ROM pnle.rom by doing:

Code: Select all

python plasmatool/link.py -o rom-full.a samplesrc/acorn/pnle.mo
acme --setpc 0x8000 -o pnle.rom rom-full.a

The service code, PLASMA bytecode interpreter and the "must have" PLASMA library routines take about 3.25K of the 16K address space in the ROM. I suspect that PLASMA bytecode is a lot denser than 6502 machine code in general, so that should help to compensate for the space taken up by this support code.

As with the standard PLASMA VM, you can mix PLASMA code with hand-written assembler in your program. The standard "asm" functions work as usual, and there's also support for what I'm calling "raw" assembler, which is a bit more flexible; I will probably do a follow-up post at some point with more on how this works. So you can use whichever is best for any particular task within your program, and if you wanted to you could even start off with a prototype of your language ROM written in PLASMA and gradually convert it to pure machine code a function at a time.

There are three examples on the "rom" branch in my repo, which are automatically built by the makefile and put on the demo.ssd disc. I've attached a copy of that if anyone wants to play with it without building the code for themselves. They are:

• R.PNLE - a simple "no language environment" language, which sits in a loop accepting * commands from the keyboard and passing them to the OS. Start this with *PNLE. If you want to write your own language ROM, this is probably a good place to start - I have tried to comment the source code thoroughly, and in particular it shows how to install an error handler, which almost all language ROMs will need. (In fact, the bulk of the source code is comments, so please don't let the apparent verbosity put you off - there are only 33 lines of actual code in there.)
• R.TESTROM - a convoluted test/demonstration of some corner cases in the static linking process. Start this with *TESTROM. This might be useful as an example if you want to start taking advantage of some of the more advanced features of link.py. It shows the use of "raw" assembler, as opposed to PLASMA "asm" functions, for example.
• R.HANOI - a *hack* of the existing Towers of Hanoi sample code built as a language ROM. Start this with *HANOI. Unlike the previous two examples, this assumes it is running from sideways RAM and will not work if it's run from unwritable memory.

There are a few restrictions on PLASMA code being used in a language ROM, unfortunately.

Global variables are read-only

The main restriction on PLASMA code in a language ROM is that global variables are read-only. If your ROM will *only* be used from sideways RAM which follows the standard Acorn convention (so if it's paged in, it's implicitly writeable), you can ignore this.

link.py will give a warning for "simple" attempts to write to global variables, although this will not catch everything.

One workaround for the lack of writeable globals is to call a main() function immediately from the init code and put your would-be globals in there as local variables. You'll need to pass pointers to them to any functions that need to access them.

You can also use global pointers to point to memory allocated on the heap. For example, instead of:

Code: Select all

word myglobal // 16-bit word
...
myglobal = 42
...
if myglobal < 60
    ...

you can do:

Code: Select all

const myglobal_ptr = $0070 // PLASMA VM leaves $70-$8F inclusive free for user (as BASIC does)
...
*myglobal_ptr = xheapalloc(2) // do this in init or a "main()" function ASAP
...
*myglobal_ptr = 42
...
if *myglobal_ptr < 60
    ...

With either of these approaches, a useful technique is to define a structure which holds all the globals:

Code: Select all

struc t_globals
    word global1
    word global2
    byte global3
    ...
end
...
// In "init" or early in main()
*myglobal_ptr = xheapalloc(t_globals)
...
myglobal_ptr=>global1 = 42
...
puts("global1=")
puti(myglobal_ptr=>global1)

as this means you don't have to worry about allocating them all individually (and, if you chose the option of passing pointers into functions which need the globals, avoids having to pass multiple pointers). (samplesrc/acorn/testrom3.pla has an example of this.)

Globals are not completely useless; you can use pre-initialised globals (particularly arrays) for read-only tables of data. In general you should probably use "const" instead of simple scalar read-only globals you don't plan to take the address of, as the generated code will be smaller with "const".

There aren't that many standard libraries for Acorn PLASMA at the moment anyway, but this restriction on globals is the main reason you need to be careful which libraries you include. All the standard features of "cmdsys", "cmdsysac" and "acornos" are available, and the 32-bit arithmetic "int32" module should work fine. If there's anything else you're interested in please ask and/or give it a try and let me know if it doesn't work.

This restriction is why I describe R.HANOI as a hack. I didn't rewrite the code to get rid of global variables as it's supposed to be an example of normal PLASMA code using a few Acorn features and I wanted to keep it simple. It could be tweaked to work from ROM if you really wanted to, of course.

Constant strings are read-only and live in the ROM address space

If your code uses a constant string (e.g. mystr = "foo"), that constant string will be located within the ROM and never be copied into RAM automatically. This means:

• You shouldn't try to pass it to an OS call which might involve paging in another ROM (e.g. writing it to a file using OSGBPB), as that ROM won't be able to see the string in your language ROM's address space.
• You shouldn't try to modify it, assuming you want your language ROM to still work when it's not in sideways RAM. This is probably bad practice anyway, but maybe if you know what you're doing it has its place when running on the standard RAM-based VM.

The standard "fileio" module won't work in ROM

There are various reasons for this, and it's not quite production-ready even when used with the normal PLASMA VM anyway. If anyone is particularly interested in this let me know and I'll see what I can do, or at least explain why it isn't practical. Anyone writing a language ROM in PLASMA should probably just use the standard Acorn OS filing system calls for I/O, though; there is some support for this in the acornos module, and you can use the standard PLASMA call() function to call any OS routine.

The program should never terminate

In a normal PLASMA environment, when control falls off the end of the the top-level module, control returns to the PLASMA VM prompt. A language ROM takes over the machine and there is nothing to hand back control to, so if your program stops running the machine will just hang. If you want to allow a way out, you can provide a mechanism for the user to enter * commands, and that will allow them to transfer control to the language of their choice. Or of course they can press CTRL-BREAK to re-enter the default language.

I could have made the code do "*BASIC" if control falls off the end of the top-level module, but that would have wasted a little bit of space for something that isn't all that useful, and would also assume that BASIC is installed. You can always do an explicit:

Code: Select all

call_oscli("BASIC")

at the end of your program if you really want to have this behaviour.

All that said, I don't think these restrictions are too onerous. If you try it please let me know how you get on; if you run into problems please post your source code here (e.g. as a zip file or a link to github) and I'll try to help. At the risk of stating the obvious, it's probably best to make small incremental changes to your code and test at each step, then if something breaks it will be a bit more obvious where the problem is (whether that problem is in your code or something in the PLASMA world itself, such as link.py being buggy).

I will probably make an even-more-rambling technical post at some point about how this works internally and some alternative approaches I could have taken (and still might take), etc, but I think this is probably long enough as it is already so I will stop here for now. Questions welcome, as always!

[shifters74]

This is a very exciting development!!

I am hoping to have a play around with it next week!

cheers for the hard work!

shifters

Oh yeah i like the explanation posts you do too!! I think i have spent more time looking at the source of plasma than writing plasma so far - its a fascinating project and language.

[shifters74]

Hi SteveF,

i downloaded the stardot9 source, did a make clean, make to build the compiler and demo disk fine.

But when trying to build the ROM code pnle as you mentioned in your post i get the following:

Code: Select all

twri@twri-ROGB450F:~/Downloads/6.BBC/8.Languages/PLASMA-stardot9/src$ python2 plasmatool/link.py -o rom-full.a samplesrc/acorn/pnle.mo
twri@twri-ROGB450F:~/Downloads/6.BBC/8.Languages/PLASMA-stardot9/src$ acme --setpc 0x8000 -o pnle.rom rom-full.a
Error - File rom-full.a, line 7 (Zone <untitled>): Cannot open input file "vmsrc/plvmzp.inc".
Error - File rom-full.a, line 173 (Zone <untitled>): Value not defined (DROP).
Serious error - File rom-full.a, line 173 (Zone <untitled>): Value not defined.
twri@twri-ROGB450F:~/Downloads/6.BBC/8.Languages/PLASMA-stardot9/src$ 

The error 'Cannot open input file "vmsrc/plvmzp.inc" ' is because the path is actually vmsrc/acorn/plvmzp.inc

I didnt clone the branch, just downloaded the tagged source you linked too. I am sure i am doing something wrong probably git related as the paths are not quite right for the build process with acme.

Any ideas?

shifters

[SteveF]

Thanks for giving this a try, and for your kind words about my rambling technical posts! I'm glad you're looking at the source too, it's really a very neat piece of software - Dave created something very clever and usable on an 8-bit system, and if you look over the history of the (Apple) VM source you can see various cool optimisations and tweaks being submitted by other people too.

You have almost solved the problem yourself, congratulations! I don't have a "good" solution, but the simple fix is to do:

Code: Select all

make vmsrc/acorn/plvmzp.inc

Half-baked technical wafflings follow...

The root problem is the makefile's attempts to be "cross-platform" and work with Acorn, Apple and C64 targets. Various source files which are themselves portable need to access target-specific assembly constants (e.g. the zero page locations used are different on each system, because they depend on that system's OS and what it leaves available), so they do:

Code: Select all

!SOURCE "vmsrc/plvmzp.inc"

and vmsrc/plvmzp.inc is supposed to be a symlink to the relevant version for the platform we're building for. The makefile sets this symlink up, but it also removes it at the end of some builds because (in principle) you might subsequently go and do a build for one of the other targets.

I have wondered about setting up an Acorn-only makefile but it feels a little bit of a cop-out. (As I write that I'm starting to feel some appeal in this though.) I'm also not very good at writing makefiles that really work well, although hardly anyone else seems to be either, so that's some consolation. (To be clear, I'm not blaming upstream for these makefile woes. I have a feeling I might have had a hand in adding this symlink myself anyway.)

I do wonder if I should just get rid of the "rm vmsrc/plvmzp.inc" at the end of the build. That would mostly solve the problem in practice and I suspect if really "nicely" supporting multiple platforms from the same source tree becomes anyone's focus they will need to make quite a few tweaks anyway. There are all sorts of awkwardnesses here, e.g. I've had to tweak the self-hosted compiler to work on Acorn, and those tweaks would currently stop it being buildable for Apple from my fork anyway, so even if it worked perfectly the "cross-platform" support in the Makefile is effectively useless in my fork.

Edit: I don't know exactly why you ran into this problem and I didn't. I suspect it's got something to do with some makefile invocations *not* removing the symlink whereas some do.

[shifters74]

Hi SteveF,

Yup that worked fine.

From src folder;

Code: Select all

make vmsrc/acorn/plvmzp.inc

python2 plasmatool/link.py -o rom-full.a samplesrc/acorn/pnle.mo

acme --setpc 0x8000 -o pnle.rom rom-full.a

Leaves pnle.rom in the src folder.

cheers!

shifters

[SteveF]

Great, thanks!