Hacker needed ... for Zarch ;-)

[sirbod]

Holy moly! That looks incredible!! How smoothly does it run? I'm gobsmacked.

What machine would you like it tested on? ARM2 will probably be about 4fps at a guess. StrongARM will probably manage 25fps, ARM710 possibly 12fps.

I've yet to add any optimisations, so lots of coding yet to do.

[tricky]

Looking great, might there be a sneak peek at the RiscOS show next weekend?

[trixster]

Holy moly! That looks incredible!! How smoothly does it run? I'm gobsmacked.

What machine would you like it tested on? ARM2 will probably be about 4fps at a guess. StrongARM will probably manage 25fps, ARM710 possibly 12fps.

I've yet to add any optimisations, so lots of coding yet to do.

Hi Jon,

I've only got a 4mb a3020, alas!

[trixster]

For me the holy grail would be seeing these optimisations/improvements running at roughly the same framerate as the original on a3000/a3020 spec hardware. Requiring much beefier hardware to get the additions kind of takes away some of the magic of what your achieving (but not too much!), but I appreciate there's only so much you can wrestle from the arm2 and arm250! The a3020's higher clock speed might help it over the a3000.

[sirbod]

Looking great, might there be a sneak peek at the RiscOS show next weekend?

Unfortunately not, I had to pull out, as it clashed with a hospital visit.

[trixster]

Any further progress Jon? I'm itching to see what you've been up to!

[sirbod]

Any further progress Jon? I'm itching to see what you've been up to!

Not made much progress, I'm still trying to get it 100% stable when the plot range is increased. When it's stable, I'll put a video of it on the JASPP YouTube channel.

[trixster]

Have you got a Christmas present for us Jon?!

Does it look like this yet?

[sirbod]

It does look like that, but isn't ready for release, I'm planning to start working on it again over Christmas. That is a handy reference screenshot to check the shading and render depth against though. I remember seeing something similar in Edge around 15 years ago.

I've been pondering how best to handle deep plot depth, as with the way Zarch is currently coded, its very unoptimized. For plotting at any kind of depth, ideally you'd want to calculate which are the visible left and right landscape tiles for a given Z depth and only plot between them. This immediately knocks out 1/3 of the landscape tiles, which are offscreen in the foreground and currently calculated but not displayed.

Zarch currently pre-builds a list of objects (trees, houses, ships, dust etc. as a triangle list) into a table, they're then plotted during the landscape tile plot stage, two rows back from the current Z row of landscape tiles being plotted. I suspect it was coded separately like this to save registers, but it would use less cycles to calculate and plot one row of objects as they're required when plotting at depth. Stacking and restoring 14 registers around this call, would undoubtedly be less costly than storing all the object triangles for a given Z depth in memory cycles.

Anyone want to count how many X and Z rows there are in that picture

[sbadger]

It does look like that, but isn't ready for release, I'm planning to start working on it again over Christmas. That is a handy reference screenshot to check the shading and render depth against though. I remember seeing something similar in Edge around 15 years ago.

yep - in this thread http://www.stardot.org.uk/forums/viewto ... rch#p91388

[jms2]

Anyone want to count how many X and Z rows there are in that picture

It's not an exact answer, but... I make it 16 deep x 24 wide.

[sirbod]

I make it 16 deep x 24 wide.

It looks like 32x32 to me.

The last screenshot I posted, I believe I had it running at 24x24, but it looks like I need to sort out the lighting which needs to fade to black a lot quicker.

[sirbod]

I had a look at Zarch's depth shader earlier and compared against the Edge screenshot; it's using a completely different algorithm. The method the finished game is using, starts at a base colour and then adds white based on depth and height, so it's only using around half the palette and can't fade to black.

Zarch currently takes 33 cycles for every triangle to work out the palette entry and I'm sure this could be reduced with lookup tables.

Is there anyone knowledgable on the VIDC palette that could help here? I need to take an RGB value (in the form &000 to &FFF, ie &RGB), apply a point lightsource to it based on height and depth (both of which range from 0-32) and end up with the palette entry to write to screen.

[steve3000]

Really looking forward to seeing this!

Is there anyone knowledgable on the VIDC palette that could help here? I need to take an RGB value (in the form &000 to &FFF, ie &RGB), apply a point lightsource to it based on height and depth (both of which range from 0-32) and end up with the palette entry to write to screen.

I have numerous RGB to 256 colour vidc tables from past endeavors (including &RGB to default vidc 256, at various levels of brightness, with and without dithering) and happy to share these, if suitable...or recreate something from scratch if needed.

I'll have some time over Christmas to new year. Will ping you.

[davidb]

Although I probably just used bit-twiddling for various projects to compose colour values for the standard RISC OS palette, I used a colour table when I added 8-bit colour support to Mesa. The src/RISCOS/c/colours file in the rosm.zip archive from Drobe contains a 16 x 16 x 16 x 2 array that might be useful. There are two colour values for each RGB value because I used dithering to try and soften the rendering output.

[sirbod]

I'll have some time over Christmas to new year. Will ping you.

Thanks, saves me trying to figure it out and can instead concentrate on getting the code stable at 32x32 tiles and look at adding optimization.

colour table when I added 8-bit colour support to Mesa.

I had a quick look at the file, but am none-the-wiser

Looking at the Edge screenshot, its approximating a strip lightsource on the landscape, that's just above and behind the screen. I have a feeling the ships are using a point lightsource, as they flare to white when the face vector is pointing at the lightsource. To match the Edge screenshot, the landscape needs to be 32x32 tiles, so it would need the dot product of the landscape Z and height, which could be approximated using a table 32x16 (ZxY). The &RGB colour would then need to be adjusted based on this value.

Looking at the code, only certain colours are used with shades of grey for the landing pad and calculated earthy hues for the landscape based on height and infection:

Code: Select all

&111 .. &888 - landing pad
&440 - bitmask taken from the height, to which &40 (not infected?), or &400 (infected?) are added
&880 - sand (can't be infected)
&004 - water (can't be infected)

So the only colours its using are:

Code: Select all

&111
&222
&333
&444
&555
&666
&777
&888
&040
&080
&400
&440
&480
&840
&880
&004

Based on this, lookup tables could be used for each colour, relative to the dot product.

[davidb]

It defines this array:

Code: Select all

GLubyte rosmesa_colourtable [16] [16] [16] [2] = {{{{0, 0}, ..., {255, 255}}}};

You can see that the first element (red=0, green=0, blue=0) accessed using:

Code: Select all

rosmesa_colourtable[red][green][blue]

gives {0, 0} (black and black in the standard VIDC palette). The last element (red=15, green=15, blue=15) gives {255, 255} (white and white).

Or did you mean something else?

[sirbod]

I have numerous RGB to 256 colour vidc tables from past endeavors (including &RGB to default vidc 256, at various levels of brightness, with and without dithering) and happy to share these, if suitable...or recreate something from scratch if needed.

I've now been through the code and rewritten both the object and landscape colour code to lookup from a table, so what's needed is a palette table that's 16*16*16*16 (distance*R*G*B ie %ddddRRRRGGGGBBBB).

At distance=8 it needs to return the same colour (ie &8888 will return red=50%, green=50%, blue=50%), below that it needs to lower the luminosity (ie &0800 will return red=6%, green=0%, blue=0%), above it needs to increase the luminosity (ie &F800 will return red=100%, green=0%, blue=0%).

[sirbod]

We could also alter the VIDC palette to better suit the game, as earthy colours are more prominent with very little use of bright blues for example.

[sirbod]

Whilst I wait for Steve's to do his magic...and the Turkey to cook...I thought I'd knock up a very rough palette conversion table using ColourTrans.

Using the default palette:

zarch_default_palette.png (24.34 KiB) Viewed 7947 times

And using the second "earthy" palette from this thread:

zarch_earthy_palette.png (22.26 KiB) Viewed 7947 times

And now the issues:

• ColourTrans is not very good at getting the closest colour as it's based on distance, not perception, so some of it's colour choices are rather odd
• The last two rows of the landscape are grey, possibly due to ColourTrans or simply the lack of low tints
• It doesn't fade to black, without turning the contrast down on your display
• 32x32 tiles doesn't look like the Edge screenshot, I've increased it to 48x32 for the screenshots above
• It looks awful when you're flying around - due to the lack of graduated tints in the palette (or possibly ColourTrans' choices)

I'm hoping Steve's palette table can rescue the situation.

What we really want, is a decent bit of code that can convert an R,G,B value to the nearest palette entry using colour perception, then we can try tweak the palette to see if it can improve the look of it.

[trixster]

Well I think both scenes look absolutely stunning! Tremendous stuff!

So what machine do you have that running on?

I was reading the article on the making of zarch and was interested to read that DB had originally wanted the terrain to be deformable. That would be an interesting challenge to implement!

[trixster]

Blimey, it makes such a difference!

[sirbod]

Now back from family visits, I've extended the plotting so it goes below the camera view.

Progress so far in images:

1. Zarch as is
2. Plotting extra width and depth
3. Reducing luminance at depth
4. Plotting below the camera view

zarch_extra_3_front_default_palette.png (24.47 KiB) Viewed 7947 times

[trixster]

So what sort of fps are you seeing with these extended views? Close to the 15ish fps of the original?

[steve3000]

I'm hoping Steve's palette table can rescue the situation.

What we really want, is a decent bit of code that can convert an R,G,B value to the nearest palette entry using colour perception, then we can try tweak the palette to see if it can improve the look of it.

Great first attempt Jon!

I've located my 12bit RGB to 8bit vidc code today, just need to tweak for depth multiplier before posting up. Will get this done over next couple of days.

[sirbod]

I've just noticed that Zarch's method for calculating the luminance of objects (trees, houses etc) doesn't work. It looks like two static variables (I've not found any direct writes to them) were added to fudge the issue, meaning objects are a constant brightness. I've now stripped all that code out and changed it to be based on Z depth, image 4 above has now been updated. The difference is clear if you look at the trees in the distance between images 3 and 4.

So what sort of fps are you seeing with these extended views? Close to the 15ish fps of the original?

This is being developed under emulation, so I'm not worrying about speed. I'll get it looking how we want it, then start optimizing, currently there's no optimization.

However...I've just tried it under Arculator to give you some figures. On an ARM2 @ 8Mhz and it's ~4fps, ARM3 @ 33MHz ~7fps - so it's memory bandwidth limited.

I wouldn't be too shocked by these as it's currently calculating and plotting a lot that not visible. Rewriting and combining the object/landscape plot code so it starts just off the left of the screen and plots until just off the right would probably double the framerate. What's really slowing it down is pre-calculating the objects to a triangle plot list, which is then read back and plotted during the landscape plot stage. That's 7 reads and 7 writes for every object triangle (X1,Y1,X2,Y2,X3,Y3,C), which is a lot of wasted memory accesses.

I've located my 12bit RGB to 8bit vidc code today, just need to tweak for depth multiplier before posting up. Will get this done over next couple of days.

[SteveF]

Now back from family visits, I've extended the plotting so it goes below the camera view.
...

• Plotting below the camera view

That extra plotting really makes all the difference, and the image is getting close to the screenshot from the magazine above ("how it could have looked"). Really impressive work!

Steve

[sirbod]

A few more "features" I've noticed today, whilst trying to get the code stable:

• Although the object lighting doesn't work, the code is there and adjusting the luminance for every triangle. Unfortunately, its currently only making objects slightly brighter as the landscape height changes. The code that adjusts for depth doesn't work, because it LSR's the depth by 25 twice, resulting in 0 luminance being added
• Where the landscape is clipped on the left and right sides to keep them in a constant position, it doesn't calculate the clipped coordinates correctly. As the cutoff gets closed to a zero width tile, the Y value starts to increase.
• Enabling cheatmode (hold "T", "U" and press "Q") is near impossible, due to the OS_Byte it's using to detect the keys (it's using 129 instead of 121). To enable you currently have to hit "Q" first and then "U" and "T" together, within about 1000nS
• The landscape lighting and object lighting use two completely different methods. The former is based on tile depth from the back of the screen and the former is based on distance from the lightsource
• The landscape tile colours are fixed, based on their X,Z position

[sirbod]

I'm afraid that despite spending over two weeks on the original code, I've not made much progress. I have now named most of the variables and figured out what some of the main functions are, the problem however is that its very unstable to the point that I can't change the code much before it breaks.

During the year I went through the data within the code and replaced all hardcoded addresses with pointers, so everything should be using relative addresses. The stability issues could also be alignment related, as adding strategically placed NOP's gets it working again.

The next issue is the table sizes, there's various tables for the map, ships, ship map locations, particles, pre-calculated triangles for trees, ships, buildings etc. Some of these are embedded within the code, the majority are above the code. I've increased the size of these appropriately I believe, but there's definitely an issue with one of the tables above the code that's causing crashes. There may also be a crash caused by variables not being set, as it randomly crashes when starting a game.

That's the major issues. Minor issues I still need to resolve include:

• Increasing the plot range of Ships to match the landscape
• Correcting the plot range of particles

Optional things I'd like to look into:

• Increase the plot range of particles
• Alter the ship, alien ship and particle code to move them based on time from the last frame, instead of being fixed at 1/25th of a second
• Support different resolutions
• Support a 24bit palette

Then there's the ARM2/3 optimizations:

• Replacing the triangle based landscape plotting with a quadrilateral routine
• Optimizing the line fill routine
• (optional) Merge the object plot code (ships, buildings, trees) into the landscape routine, instead of pre-building a separate triangle plot list for them

I have however resolved what I believe to be all the legacy bugs/issues:

• Now requests the screen address
• The OS_Byte used for checking keys (which fixes the cheat mode)
• One of the functions had a typo on the exit LDM, which could corrupt the stack
• Selects the correct sounds, without the order of WaveSynth and Percussion needing to be swapped
• Speed regulation to 25fps
• Non-32bit code corrected
• Self-modifying code removed

Finally, I've got to figure out how to provide a way to patch the original Zarch. I've tried using David Pilling's !Refreshen to create a diff patch, but it fails to worth with Zarch. Unless I can find an alternative, I'll have to create a diff patcher from scratch.

[tricky]

Great work, it is a huge pain fixing bugs that you introduce and don't notice immediately! I have submitted 8 times to my local perforce today (to make this easier) and I've only done a few hours mixed with other stuff.
It wouldn't be pretty, but you could just EOR the old and new files and ship the diff that way, not small but simple and would only work with the original files.