basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

handy tools that can assist in the development of new software
User avatar
scruss
Posts: 653
Joined: Sun Jul 01, 2018 4:12 pm
Location: Toronto
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by scruss »

BeebMaster wrote: Mon Jul 12, 2021 9:55 pm … for some reason I no longer have any select or copy & paste function, so I can't do anything with the listing except look at it.
That reason might be Wayland, the X11 replacement that all the system people love but it breaks a lot of user tools.

Matrix Brandy has the tbrandy and sbrandy console interpreters, and they work very well.
SteveF
Posts: 1695
Joined: Fri Aug 28, 2015 9:34 pm
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by SteveF »

I'm pleased to announce the release of v0.05 of basictool. There's nothing too earth-shattering here, just some small improvements and tweaks:
  • There's now a man page (man/basictool.1). This is probably pointless, but I was watching a video on Unix architecture evolution and it somehow made me think it would be cool to create one. I'd never written one before and it was an interesting experience getting to grips with all the associated macros - I hope I got it all right! I have to be honest that there's a lovely pseudo-retro quality to the basictool man page when properly "typeset" (e.g. as a PDF) which makes happy even if no one else ever looks at it. :-)
  • The --keep-spaces-start and --keep-spaces-end options have been removed. I hadn't realised BASIC 4 always strips trailing spaces when tokenising, so --keep-spaces-end didn't do anything. The -k/--keep-spaces option has been retained to control stripping of leading spaces.
  • Warnings will be emitted if you invoke basictool with --keep-spaces or --pack-singles-n in ways that will cause those options to be ignored.
Please report any problems here or on github and I'll take a look.
User avatar
lurkio
Posts: 4351
Joined: Wed Apr 10, 2013 12:30 am
Location: Doomawangara
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by lurkio »

SteveF wrote: Sun Aug 08, 2021 6:30 pm I hadn't realised BASIC 4 always strips trailing spaces when tokenising
Ah. Neither had I, and it's slightly unfortunate because BASIC 4 on a Master even seems to strip spaces off the ends of DATA values:

http://bbcmicro.co.uk//jsbeeb/play.php? ... 2C13%22%0A

That's not ideal because sometimes you might need to squeeze as many bytes as possible onto a DATA line -- in a big text adventure program, for example -- and it's conceivable you might need to end-pad a DATA value with spaces, and you might not always be able to afford the luxury of enclosing the DATA string in quotes.

It's a shame, because BASIC 4 is perfectly happy to run a program that's been tokenised by BASIC 2 (and BASIC 2 doesn't strip spaces off the ends of DATA values):

http://bbcmicro.co.uk//jsbeeb/play.php? ... Aspace.ssd

I know it's an edge case, but I think it's a viable one, and not totally unrealistic.

I wonder if you could offer the option to use BASIC 2 instead of BASIC 4 in basictool..?

:?:
SteveF
Posts: 1695
Joined: Fri Aug 28, 2015 9:34 pm
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by SteveF »

Thanks lurkio, that seems like a sensible suggestion.

Can you (and anyone else interested) please give v0.06-pre a try and see how you get on? I've updated the code but not touched the documentation yet.

This still defaults to BASIC 4 but you can tell it to use BASIC 2 by specifying the --basic-2 option. --keep-spaces-start and --keep-spaces-end have been restored and --keep-spaces is now shorthand for specifying both of those, as it was before v0.05. A warning will be given if you use a --keep-spaces* option which will be ignored.
User avatar
lurkio
Posts: 4351
Joined: Wed Apr 10, 2013 12:30 am
Location: Doomawangara
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by lurkio »

SteveF wrote: Mon Aug 09, 2021 8:15 pm Can you (and anyone else interested) please give v0.06-pre a try and see how you get on? ... This still defaults to BASIC 4 but you can tell it to use BASIC 2 by specifying the --basic-2 option.
I tried the shorthand -2 option with the attached textfile, but the spaces at the end of the DATA value weren't retained. I used the following command:

Code: Select all

./basictool -2 -t p.txt p.tok
:?:

p.txt
(45 Bytes) Downloaded 65 times

EDIT: I wonder if the problem is that leading and trailing spaces are stripped by default when tokenising, even when using BASIC 2? I rather think that that perhaps shouldn't be the case and that the default should be to retain all spaces, with an option to strip them..?

(The Pack routine, of course, is independent of the basictool "pre-processor" (or whatever you call it!) and rightly has its own separate option to remove spaces -- but that option doesn't strip trailing spaces from DATA lines. Which is fine as is.)

:?:
User avatar
TobyLobster
Posts: 622
Joined: Sat Aug 31, 2019 7:58 am
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by TobyLobster »

Yes, this does keep the trailing spaces:

Code: Select all

./basictool --basic-2 --keep-spaces-end -t p.txt
but with --basic-4 doesn't.
SteveF
Posts: 1695
Joined: Fri Aug 28, 2015 9:34 pm
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by SteveF »

Thanks guys. So I think the current situation is that v0.06-pre is working fine, but keeping spaces should maybe be the default?

If memory serves I originally made stripping spaces the default because I imagined myself using basictool to tokenise programs I'd written in a text editor on a modern PC where I'd like to indent the program to reflect its logical structure. That said, it's no major hardship to require an option to strip off those spaces and preserve them by default. I can certainly see some logic in a "make minimal changes to the input unless told to do so" approach.

The one thing that slightly worries me about not stripping spaces by default is that with the default of BASIC 4, spaces at the end of lines will be stripped *anyway*. I don't know if this is a big problem, and I suppose I could issue a warning if this case occurs and a line actually has spaces at the end which will be stripped.

(I don't think it's a good idea to use BASIC 2 by default, or to change the default space-stripping behaviour depending on which BASIC is chosen; that seems likely to cause confusion.)

So I'm leaning towards following lurkio's suggestion, but I'm open to further thoughts on this before I do.
User avatar
lurkio
Posts: 4351
Joined: Wed Apr 10, 2013 12:30 am
Location: Doomawangara
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by lurkio »

SteveF wrote: Mon Aug 09, 2021 9:17 pm Thanks guys. So I think the current situation is that v0.06-pre is working fine
Yes.

SteveF wrote: Mon Aug 09, 2021 9:17 pm but keeping spaces should maybe be the default?
Yes. I think that would be more intuitive. I think it’s what most users would expect. I certainly did!

SteveF wrote: Mon Aug 09, 2021 9:17 pm I can certainly see some logic in a "make minimal changes to the input unless told to do so" approach.
Exactly. Again, I suspect that that’s how users would assume the program behaves.

SteveF wrote: Mon Aug 09, 2021 9:17 pm The one thing that slightly worries me about not stripping spaces by default is that with the default of BASIC 4, spaces at the end of lines will be stripped *anyway*. I don't know if this is a big problem,
I don’t think it is a big problem.

SteveF wrote: Mon Aug 09, 2021 9:17 pm and I suppose I could issue a warning if this case occurs and a line actually has spaces at the end which will be stripped.
That’s a good idea.

SteveF wrote: Mon Aug 09, 2021 9:17 pm(I don't think it's a good idea to use BASIC 2 by default, or to change the default space-stripping behaviour depending on which BASIC is chosen; that seems likely to cause confusion.)
Agreed.

SteveF wrote: Mon Aug 09, 2021 9:17 pm So I'm leaning towards following lurkio's suggestion, but I'm open to further thoughts on this before I do.
Yes, it would be good to hear what others have to say.

:idea:
SteveF
Posts: 1695
Joined: Fri Aug 28, 2015 9:34 pm
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by SteveF »

Thanks guys. Please give v0.06-pre2 a go and let me know how you get on. I've updated the code but not the documentation in the repo, I'll do that once we're happy with the behaviour.

In this version:
  • Spaces in text input are preserved by default.
  • You can tell basictool to strip spaces on text input using the new --strip-spaces, --strip-spaces-start and --strip-spaces-end options.
  • If the default of preserving spaces at the ends of lines hasn't been changed and there are some spaces which BASIC 4 will strip off regardless, a warning will be given. The warning suggests using --basic-2, but if you don't care you can also suppress the warning by using --strip-spaces or --strip-spaces-end. Or of course by editing the input to remove the spaces at the end of lines. :-)
  • The --keep-spaces* options have been removed. I think basictool's user base is small enough there's no need to support these as dummy options for backwards compatibility. :-)
  • An incorrect warning claiming that keeping/stripping spaces was irrelevant except for the --tokenise option has been removed. (If the input is text rather than tokenised BASIC, these options have and always had an effect.)
User avatar
TobyLobster
Posts: 622
Joined: Sat Aug 31, 2019 7:58 am
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by TobyLobster »

SteveF wrote: Tue Aug 10, 2021 9:02 pm Spaces in text input are preserved by default.
You can tell basictool to strip spaces on text input using the new --strip-spaces, --strip-spaces-start and --strip-spaces-end options.
This seems to work as expected.
But it can't strip spaces on already tokenised input: "warning: --strip-spaces* have no effect with pre-tokenised input" I guess the workaround would be to run the tool once to convert to ascii, then run the tool a second time to strip the spaces.
SteveF wrote: Tue Aug 10, 2021 9:02 pm The warning suggests using --basic-2, but if you don't care you can also suppress the warning by using --strip-spaces or --strip-spaces-end.
Perhaps the warning message could say something like "...use --basic-2 to preserve them. Use --strip-spaces or --strip-spaces-end to suppress this warning"

I found a separate bug. It doesn't seem to like programs that start with line zero

Code: Select all

test1.txt:1: error: line number 0 is less than previous line number 0
test1.txt
(26 Bytes) Downloaded 51 times
SteveF
Posts: 1695
Joined: Fri Aug 28, 2015 9:34 pm
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by SteveF »

Thanks! I've tweaked the warning roughly as you suggested, and I think I've fixed the line number 0 bug. v0.06-pre3 should have these fixes. The test suite is broken by the natural fix for the line number 0 bug but I'll fix that up along with the documentation when we're otherwise happy.

Edited to add: You're right about stripping spaces from already tokenised input and the workaround. I could potentially make this work, but it would complicate the internals a bit and (perhaps?) make the internals harder to mentally model. Serious, genuine question: How useful do you/anyone think this would be?

Edited to add further: I suspect - I have't tried it - the workaround would fail if the program has been packed and can't be typed in as-is. I had (perhaps stupidly, in hindsight) thought the natural implementation of this would be for basictool to "re-type" the program internally, but I guess it wouldn't be too hard to operate on the tokenised representation of the program in memory to strip leading/trailing spaces.
User avatar
TobyLobster
Posts: 622
Joined: Sat Aug 31, 2019 7:58 am
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by TobyLobster »

SteveF wrote: Tue Aug 10, 2021 11:12 pm You're right about stripping spaces from already tokenised input and the workaround... How useful do you/anyone think this would be?
It is a very minor issue, and there is a workaround for all but an edge case. I only found it by accident. I would hold off trying to fix this unless/until more people report it as a problem.
User avatar
lurkio
Posts: 4351
Joined: Wed Apr 10, 2013 12:30 am
Location: Doomawangara
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by lurkio »

SteveF wrote: Tue Aug 10, 2021 11:12 pmv0.06-pre3 should have these fixes.
Thanks! I've only had a quick play, but that version seems to work fine on my short test program with the terminally space-padded DATA string.

SteveF wrote: Tue Aug 10, 2021 11:12 pmabout stripping spaces from already tokenised input ... How useful do you/anyone think this would be?
I personally have never had the need to do that, so I can't say it's a feature I'm desperate for, but, on the other hand, I do think it would be nice (but perhaps not essential) if space-stripping worked on both tokenised and untokenised input alike.

SteveF wrote: Tue Aug 10, 2021 11:12 pmI guess it wouldn't be too hard to operate on the tokenised representation of the program in memory to strip leading/trailing spaces.
If you're going down the road of more involved parsing, I'm tempted to mention a feature I've kind of been wishing for for a while now: GOTO <label>, GOSUB <label>, and RESTORE <label>.

Would it be possible for those constructs to be implemented either as an option in basictool or in some sort of separate pre-processor for basictool? To be clear, what I'm proposing is that in a modern text-editor the user would be able to mark a subroutine by prefacing it with (say) %%myroutine%%, and then call it with something like GOSUB %%myroutine%% -- and then basictool or the pre-processor would take the plaintext BASIC program as input and the first thing that would be done is that the program would be given line-numbers and the labels would be converted to numbers as appropriate.

But actually, the more I think about it, the more I feel that maybe that sort of functionality really falls outside the scope of basictool, which I would summarise roughly as "bringing BASIC ROM utilities to the modern commandline". So this sort of thing probably isn't a good fit for basictool after all...

Still, it would be a useful thing to be able to do in one way or another because when you're writing and editing 8-bit BBC BASIC in a modern text-editor the one stumbling block that prevents you from being able to avoid using line-numbers entirely is the pesky ON ERROR problem. At some point -- usually when you're using filesystem commands, e.g. to load or save a savegame -- you'll want to trap a filesys error, and you'll then have to use ON ERROR, which clears the BASIC stack and forces you to use GOTO in order to recover without crashing. Therefore, you'll end up faffing around and adding line-numbers to code that otherwise could have been structured beautifully with PROCs instead of GOSUBs, and with nary a line-number in sight. Which is a shame. (I know Jonathan Harston has actually implemented ON ERROR LOCAL for 8-bit BBC BASIC, but you may not always have the space to include the machine-code patch.)

And adding line-numbers makes it harder to re-order blocks of code because doing so now requires you to remove the line-numbers in your modern text-editor, rearrange the code, let BASIC tokenise and renumber it, reimport it into the text-editor, and then change all the arguments of any GOTOs and GOSUBs, etc., to their new renumbered values. And repeat the whole procedure every time you need to restructure parts of the prog. Quite a pain.

:idea:
SteveF
Posts: 1695
Joined: Fri Aug 28, 2015 9:34 pm
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by SteveF »

lurkio wrote: Thu Aug 12, 2021 2:53 am
SteveF wrote: Tue Aug 10, 2021 11:12 pmv0.06-pre3 should have these fixes.
Thanks! I've only had a quick play, but that version seems to work fine on my short test program with the terminally space-padded DATA string.

SteveF wrote: Tue Aug 10, 2021 11:12 pmabout stripping spaces from already tokenised input ... How useful do you/anyone think this would be?
I personally have never had the need to do that, so I can't say it's a feature I'm desperate for, but, on the other hand, I do think it would be nice (but perhaps not essential) if space-stripping worked on both tokenised and untokenised input alike.
Thanks for testing, I'm glad it's working. I've added a TODO to the source code to maybe look at stripping spaces from tokenised input in the future; I will sleep on it but I'll probably not implement this for v0.06 since no one seems overly keen.
lurkio wrote: Thu Aug 12, 2021 2:53 am
SteveF wrote: Tue Aug 10, 2021 11:12 pmI guess it wouldn't be too hard to operate on the tokenised representation of the program in memory to strip leading/trailing spaces.
If you're going down the road of more involved parsing, I'm tempted to mention a feature I've kind of been wishing for for a while now: GOTO <label>, GOSUB <label>, and RESTORE <label>.

Would it be possible for those constructs to be implemented either as an option in basictool or in some sort of separate pre-processor for basictool? To be clear, what I'm proposing is that in a modern text-editor the user would be able to mark a subroutine by prefacing it with (say) %%myroutine%%, and then call it with something like GOSUB %%myroutine%% -- and then basictool or the pre-processor would take the plaintext BASIC program as input and the first thing that would be done is that the program would be given line-numbers and the labels would be converted to numbers as appropriate.

But actually, the more I think about it, the more I feel that maybe that sort of functionality really falls outside the scope of basictool, which I would summarise roughly as "bringing BASIC ROM utilities to the modern commandline". So this sort of thing probably isn't a good fit for basictool after all...
I agree this would be nice but that it also does somewhat fall outside the scope of basictool; I like your summary. Stripping spaces from tokenised programs goes a little beyond this, but it's probably a relatively simple thing to do.

Did you know that you can already do partial line numbering with basictool? This is kind of what you want, but with purely numeric "labels" which have to appear in order. For example:

Code: Select all

REM Do we have handy ROM FOOBAR which we can turn on with *FOOBAR ON?
REM Check, then we can use it if it's installed and do our best if it isn't.
have_foobar=FALSE
ON ERROR GOTO 500
*FOOBAR ON
have_foobar=TRUE
500ON ERROR PROCerror
PROCdo_cool_stuff
REM blah blah
RESTORE 1000
READ foo$
PRINT foo$
END
:
DEF PROCerror
REPORT:PRINT " at line ";ERL
END
:
1000DATA "hello!"
As long as you leave big gaps between the handful of line numbers you have to allocate, you can edit fairly freely without worrying about them. basictool will give an error if you edit them into the wrong order as well.

Given this, adding label support might not be that hard, if the labels were something that could be identified naively without complex parsing (such as your "%%"-delimited ones). basictool would make a first pass over the text input, replacing label definitions with line numbers and remembering the (label name, line number) pairs, then make a second pass replacing references to the labels with the corresponding line numbers, which would transform the labelled program to something like the above. It does feel a bit hacky, but it would probably be a fairly effective hack. Your idea of doing this as a preprocessor would probably work well, at least for initial experimentation.
User avatar
lurkio
Posts: 4351
Joined: Wed Apr 10, 2013 12:30 am
Location: Doomawangara
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by lurkio »

SteveF wrote: Thu Aug 12, 2021 4:20 am Did you know that you can already do partial line numbering with basictool? …
Ah. I think I did know but had forgotten! Thanks. That’s good to bear in mind. It’s not quite as flexible as what I was proposing, but it’s definitely a handy feature. I’ll ponder it.

SteveF wrote: Thu Aug 12, 2021 4:20 am adding label support might not be that hard
You must stop encouraging me like this..! But it occurred to me that a handy optional extra to my proposal might be to allow the user to specify the line-number increment (which is conventionally 10) as a parameter to the preprocessor, so that they could then write code like this (assuming a text adventure prog in which room descriptions are stored in sequentially numbered DATA lines):

Code: Select all

DEFFNget_room_description(r)
RESTORE %%string_start%%+r*[[INCREMENT]]
READ a$
=a$
…
%%string_start%% DATA a small antechamber
DATA a dark, dank cave
…
Funny, isn’t it, how feature-creep seems so harmless when you’re not the one doing the actual implementing..!

:idea:
SteveF
Posts: 1695
Joined: Fri Aug 28, 2015 9:34 pm
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by SteveF »

I've knocked up a fairly hacky preprocessor, which you can get from here. (That link will always access the latest version, it's not a fixed tag.)

Given this silly example (test.txt):

Code: Select all

PRINT "Hello"
GOSUB %%Greet%%
25PRINT "Bye!"
GOSUB %%ReallyBye%%
PRINT "PS: We don't exempt string literals yet: %%ReallyBye%%"
END
%%Greet%%:PRINT "How are you?"
RETURN
%%ReallyBye%%:
PRINT "Are you still here?"
RETURN
we can run the preprocessor like this and get usable BASIC output:

Code: Select all

$ python basiclabel.py test.txt | basictool -
    0PRINT "Hello"
    1GOSUB 29
   25PRINT "Bye!"
   26GOSUB 31
   27PRINT "PS: We don't exempt string literals yet: 31"
   28END
   29PRINT "How are you?"
   30RETURN
   31:
   32PRINT "Are you still here?"
   33RETURN
A few notes:
  • "basictool -" will just tokenise standard input (the output of the preprocessor) and then output the program as ASCII text. Really you'd do something like "basictool -t - output.tok" to write the tokenised program to a file, but that wouldn't be human readable for this example.
  • This has had minimal testing - please experiment and let me know how you get on.
  • You can edit the hard-coded values next_auto_line_number and auto_line_number_increment around line 50 in basiclabel.py to try out your idea of allowing these to be controlled by the user. A less hacky version would accept command-line arguments for these, of course. I default these to 0 and 1 to reduce the chances of occasionally user-supplied line numbers causing problems.
  • As demonstrated by the "PS" in test.txt above, this hacky prototype doesn't try to recognise BASIC string literals ("foo") and label substitution will occur inside string literals as well.
  • The user-supplied line number 25 in test.txt is just to show it's possible to supply line numbers manually, it isn't necessary.
Edited to add: Following on from your latest example, the preprocessor could define a %%INCREMENT%% label and you could use that, rather than having to hard-code it as 10. I'm not sure if that's what you were suggesting or if your "[INCREMENT]"" was a meta-variable
SteveF
Posts: 1695
Joined: Fri Aug 28, 2015 9:34 pm
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by SteveF »

OK, I've added an INCREMENT pseudo-label. Given this example (test2.txt):

Code: Select all

INPUT "Item? (0-2) " n
RESTORE (%%string_start%%+n*%%INCREMENT%%)
READ s$
PRINT s$
END
%%string_start%%:DATA zero
DATA one
DATA two
the preprocessor will turn it into:

Code: Select all

$ python basiclabel.py test2.txt | basictool -
    0INPUT "Item? (0-2) " n
    1RESTORE (5+n*1)
    2READ s$
    3PRINT s$
    4END
    5DATA zero
    6DATA one
    7DATA two
If you edit basiclabel.py at line 50-ish to set next_auto_line_number to 10 and auto_line_number_increment to 10, the same input will turn into:

Code: Select all

$ python basiclabel.py test2.txt | basictool -
   10INPUT "Item? (0-2) " n
   20RESTORE (60+n*10)
   30READ s$
   40PRINT s$
   50END
   60DATA zero
   70DATA one
   80DATA two
This is a very neat idea, thanks for the suggestion!

I'm sure there's going to be a catch somewhere as this seems to be working too well in these trivial tests, but please let me know how you get on!
User avatar
TobyLobster
Posts: 622
Joined: Sat Aug 31, 2019 7:58 am
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by TobyLobster »

This reminds me of perhaps my favourite error message in BASIC:
Screenshot 2021-08-12 at 14.30.32.png
Screenshot 2021-08-12 at 14.30.32.png (4.5 KiB) Viewed 5880 times
User avatar
lurkio
Posts: 4351
Joined: Wed Apr 10, 2013 12:30 am
Location: Doomawangara
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by lurkio »

SteveF wrote: Thu Aug 12, 2021 2:05 pm I've knocked up a fairly hacky preprocessor
Oh, wow! I will try this out ASAP.

SteveF wrote: Thu Aug 12, 2021 2:05 pm Edited to add: Following on from your latest example, the preprocessor could define a %%INCREMENT%% label and you could use that, rather than having to hard-code it as 10. I'm not sure if that's what you were suggesting or if your "[INCREMENT]"" was a meta-variable
What I meant was that you could invoke the preprocessor with something like

Code: Select all

basiclabel -inc 5 …
to specify that line-numbers should go up in increments of 5. The preprocessor would then number the lines of the program with 5, 10, 15, etc. And whenever it came across the [[INCREMENT]] tag it would replace it with 5.

So in my example the output of the preprocessor would then be

Code: Select all

…
200 RESTORE 1000+r*5
…
1000 DATA a small antechamber
1005 DATA a dark, dank cave
…
:idea:

EDIT: I see we’ve just crossed posts..!
SteveF
Posts: 1695
Joined: Fri Aug 28, 2015 9:34 pm
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by SteveF »

Yes, I think we just crossed over.

I've just pushed a newer version to github (the same link as before, since it's not tagged). This now adds basic command line parsing - use "python basiclabel.py --help" to see the available options, but the most interesting one is "-i" or "--increment" to specify the line number increment.

The default line number increment is still 1 but I'm prepared to change this to 10 if you think it's important; I agree it's more conventional. (I'd really rather keep it at 1 in basictool though, since this default minimises the chance of clashes between auto-generated and user-supplied line numbers when using partial line numbering. I think this is less of an issue with basiclabel.py as the whole point of labels is to reduce the need to have user-supplied line numbers in the first place.)

The newer version also doesn't generate any BASIC output if errors occur, which is a little bit neater, particularly when piping the output to basictool.

Edited to add: perhaps stating the obvious, but if you do "chmod +x basiclabel.py" you can run it without needing the "python" at the start of the command, as long as it's on your PATH.

Edited again to add: I've just pushed a newer version to github which should correctly avoid substituting labels inside string literals. I won't be too surprised to find there's a subtle bug in this, but as far as I know it works.
Last edited by SteveF on Thu Aug 12, 2021 5:09 pm, edited 2 times in total.
SteveF
Posts: 1695
Joined: Fri Aug 28, 2015 9:34 pm
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by SteveF »

TobyLobster wrote: Thu Aug 12, 2021 2:31 pm This reminds me of perhaps my favourite error message in BASIC:
Hah, I'd forgotten about that! Try specifying "-i 0" with the latest version of basiclabel.py. :-)
User avatar
lurkio
Posts: 4351
Joined: Wed Apr 10, 2013 12:30 am
Location: Doomawangara
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by lurkio »

Had a quick play with basiclabel.py.

Looks to be working great -- except that when the final char of the input plaintext BASIC program isn't a newline, that final char gets trimmed!

:!:
SteveF
Posts: 1695
Joined: Fri Aug 28, 2015 9:34 pm
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by SteveF »

Thanks for testing, I've just pushed a change to github which I hope will fix that.
User avatar
lurkio
Posts: 4351
Joined: Wed Apr 10, 2013 12:30 am
Location: Doomawangara
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by lurkio »

SteveF wrote: Thu Aug 12, 2021 8:24 pm Thanks for testing, I've just pushed a change to github which I hope will fix that.
Yes. That fixes the problem. Many thanks!

=D> =D> =D>

This whole thing feels like a surprisingly massive upgrade to the experience of writing 6502 BBC BASIC in a modern text-editor. The ability to do the following, for example, is just great:

Code: Select all

CLS
PRINT"Start"'
%%main_loop%%:REPEAT
REM do lots of stuff
INPUT "Filename: "f$
PROCfile(f$)
UNTIL FALSE
END

DEFPROCfile(f$)
ON ERROR ON ERROR OFF:PRINT"Recovering from error"':GOTO %%main_loop%%
PRINT"Attempting to access file"
OSCLI "ACCESS "+f$
ON ERROR OFF:PRINT"Successfully accessed file"'
ENDPROC
I think it was the need to trap filesys errors that was the main thing that prevented me from being able to remove all the line-numbers from Raven Wood when I was helping to tidy up the code, which was a shame because the author John had really taken to heart all the advice from the User Guide (etc.) about "structured programming" and had used PROCs and FNs wherever he could. But, in the end, the need for ON ERROR to trap the saving and loading of savegames made the use of line-numbers essential.

:idea:

EDIT:
SteveF wrote: Thu Aug 12, 2021 4:21 pm The default line number increment is still 1 but I'm prepared to change this to 10 if you think it's important; I agree it's more conventional. (I'd really rather keep it at 1 in basictool though, since this default minimises the chance of clashes between auto-generated and user-supplied line numbers when using partial line numbering. I think this is less of an issue with basiclabel.py as the whole point of labels is to reduce the need to have user-supplied line numbers in the first place.)
Keep 1 as the default. I have a vague memory of reading in some book or article that an increment of 1 was less than ideal (for some sort of program-efficiency reason which escapes me, and not just because it made it slightly harder to interpolate lines while editing -- because you'd have to use RENUMBER) -- but I can't remember why it was supposed to be less than ideal, and now I can't think of a plausible reason for why the increment shouldn't be 1. And, as you say, a 1-increment works better with any user-supplied line-numbers.

:idea:
Last edited by lurkio on Thu Aug 12, 2021 11:22 pm, edited 1 time in total.
Deleted User 9295

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by Deleted User 9295 »

lurkio wrote: Thu Aug 12, 2021 10:10 pm But, in the end, the need for ON ERROR to trap the saving and loading of savegames made the use of line-numbers essential
You don't need line numbers, even in that case, if the 'recoverable' filesystem errors can be distinguished by the value of ERR. This kind of program structure was one I used very commonly BITD:

Code: Select all

CLS
PRINT"Start"'
ON ERROR IF ERR<189 REPORT : PRINT " at line ";ERL : END ELSE PRINT "Recovering from error" 
REPEAT
  REM do lots of stuff
  INPUT "Filename: "f$
  PROCfile(f$)
UNTIL FALSE
END
If an error occurs, even inside a PROC or FN, there is an 'implied' GOTO to the ON ERROR statement; if it's a recoverable error execution simply 'falls through' to the subsequent code, otherwise it aborts. If it's not convenient to use ERR to distinguish recoverable errors from others, a global variable can be used instead.

I obsessively avoided GOTOs, as a matter of principle, and this technique made that possible even when error trapping was in use.
User avatar
lurkio
Posts: 4351
Joined: Wed Apr 10, 2013 12:30 am
Location: Doomawangara
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by lurkio »

Richard Russell wrote: Thu Aug 12, 2021 11:22 pm… I obsessively avoided GOTOs, as a matter of principle, and this technique made that possible even when error trapping was in use.
Good tip. Thanks!

:idea:
User avatar
Lardo Boffin
Posts: 2979
Joined: Thu Aug 06, 2015 7:47 am
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by Lardo Boffin »

You can error trap code in PROCs and FNs without having to resort to line lines thanks to JGH:

viewtopic.php?p=151688#p151688

Disclaimer - not read every word in this thread so this may be of no relevance. :D
Adventure Language on GitHub
Atom, issue 5, YARRB + video noise killer
Elk
A number of econetted (is that a word?) Beebs
BBC Master, Datacentre + HDD, pi co-proc, econet, NULA
User avatar
lurkio
Posts: 4351
Joined: Wed Apr 10, 2013 12:30 am
Location: Doomawangara
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by lurkio »

Lardo Boffin wrote: Fri Aug 13, 2021 10:55 pm You can error trap code in PROCs and FNs without having to resort to line lines thanks to JGH: viewtopic.php?p=151688#p151688 Disclaimer - not read every word in this thread so this may be of no relevance.
Relevant but already mentioned:
lurkio wrote: Thu Aug 12, 2021 2:53 am (I know Jonathan Harston has actually implemented ON ERROR LOCAL for 8-bit BBC BASIC, but you may not always have the space to include the machine-code patch.)

Richard Russell wrote: Thu Aug 12, 2021 11:22 pm If an error occurs, even inside a PROC or FN, there is an 'implied' GOTO to the ON ERROR statement; if it's a recoverable error execution simply 'falls through' to the subsequent code, otherwise it aborts. If it's not convenient to use ERR to distinguish recoverable errors from others, a global variable can be used instead.
Developing the example a bit further:

Code: Select all

CLS
PRINT"Start"'
err=0
ON ERROR PROCerr
REPEAT
 REM main loop
 PRINT"File or time? (F/T): ";:REPEAT K$=GET$:UNTIL K$="F" OR K$="T":PRINT K$
 IF K$="F" PROCfile
 IF K$="T" PROCtime
UNTIL FALSE
END

DEFPROCfile
 INPUT "Filename: "f$
 PRINT"Attempting to access file"
 err=1
 OSCLI "ACCESS "+f$
 PRINT"Successfully accessed file"'
ENDPROC

DEFPROCtime
 err=2
 *TIME
 PRINT
ENDPROC

DEFPROCerr
 IF err<>1 ELSE IF ERR=201 PRINT"Can't write to disc" ELSE PRINT"Filesystem error"
 IF err=2 PRINT"Can't tell time"
 IF ERR=17 PRINT'"There's no escape"
 err=0:PRINT
ENDPROC
I hadn't thought of placing the ON ERROR in that precise spot. I now think that that's probably the best way to do error handling if you're trying to avoid using line-numbers in the sort of BASIC program that has a main loop, such as a text adventure.

So perhaps there's now one less reason for basiclabel.py to exist! Apologies, SteveF! But the RESTORE-with-increment feature is still useful, so hopefully basiclabel.py wasn't made entirely in vain..?!

:idea:
SteveF
Posts: 1695
Joined: Fri Aug 28, 2015 9:34 pm
Contact:

Re: basictool - a command-line tool to tokenise, de-tokenise, pack and analyse BBC BASIC

Post by SteveF »

Some informative stff here; I don't write a lot of 6502 BBC BASIC these days, but I hope I will remember this when I do. (Is there any kind of central repository of these advanced tips and tricks?)
lurkio wrote: Sat Aug 14, 2021 4:21 pm So perhaps there's now one less reason for basiclabel.py to exist! Apologies, SteveF! But the RESTORE-with-increment feature is still useful, so hopefully basiclabel.py wasn't made entirely in vain..?!
It's OK, although I'd probably be annoyed if I'd incorporated it into basictool and had to implement it in C. :-) I did find myself thinking last night that a tweaked basiclabel.py could be used to implement "compile-time constants", e.g. tweaking your example:

Code: Select all

%%ERR_NONE=0%%
%%ERR_FILE=1%%
%%ERR_TIME=2%%
CLS
PRINT"Start"'
err=%%ERR_NONE%%
ON ERROR PROCerr
REPEAT
 REM main loop
 PRINT"File or time? (F/T): ";:REPEAT K$=GET$:UNTIL K$="F" OR K$="T":PRINT K$
 IF K$="F" PROCfile
 IF K$="T" PROCtime
UNTIL FALSE
END

DEFPROCfile
 INPUT "Filename: "f$
 PRINT"Attempting to access file"
 err=%%ERR_FILE%%
 OSCLI "ACCESS "+f$
 PRINT"Successfully accessed file"'
ENDPROC

DEFPROCtime
 err=%%ERR_TIME%%
 *TIME
 PRINT
ENDPROC

DEFPROCerr
 IF err<>%%ERR_FILE%% ELSE IF ERR=201 PRINT"Can't write to disc" ELSE PRINT"Filesystem error"
 IF err=%%ERR_TIME%% PRINT"Can't tell time"
 IF ERR=17 PRINT'"There's no escape"
 err=0:PRINT
ENDPROC
Post Reply

Return to “development tools”