Disc Image Manager

[geraldholdsworth]

I'm happy to report that Disk Image Manager saved my bacon the other day

That's good to hear.

[geraldholdsworth]

It's been a while since I've done anything towards DIM. I was working on making it multi-partition...then gave up because I need to basically rewrite the entire structure of how the image is held in memory. This is mostly driven by wanting to support the RDSK Commodore Amiga hard drive format, but also, for example, if DIM opens an ADFS image with multiple DOS image files inside.

I've also been, for quite a while now, looking at making myself a UPURS cable for my Master. Also, in another project, I can control a VISCA enabled camera (e.g., pan/tilt, zoom, focus, etc.) and this is all done over serial...and I've managed to figure out how to do this on macOS (before it was only Windows). So, this led me to thinking - I wonder if I can add a feature to Disc Image Manager to interface with UPURS on the BBC side. Seems quite simple.

I have also been made aware of an issue with RISC OS hard drive images, specifically using Arculator. Annoyingly, the latest versions of Arculator are only for Windows, so this becomes a bit of a hinderance for me (I am now pretty much fully macOS). But what I have found is that it is likely to be something on the RISC OS side...they work fine with RPCEmu (RISC OS 5), so could just be a RISC OS 3 thing. One addition I am going to add in is the option to add the 512-byte 'emulator header' to ADFS hard drive images.

[geraldholdsworth]

Earlier today, I made version 1.44 available. Main changes this version:

• When creating ADFS hard drive images, some more options have been added: option to add the 512-byte blank header for emulators; option to specify whether the image should be geared towards IDE or ST506 (this changes the sector size and heads, emulating what HForm does).
• There are more error codes for the other types of ADFS broken directories.
• Fixed some rather glaring bugs in ADFS New Map coding.
• Now creates and uses shared fragments in ADFS New Map.
• You can pick and choose which columns you want when outputting a CSV file of the image. I've also added the '0x' prefix to hex numbers so that the file can be imported into Excel, Numbers, or other spreadsheet much easier.

Plus tons more, and a load of bug fixes too.

Gerald.

[geraldholdsworth]

I've uploaded an rc1 and rc2 version of 1.45. I haven't done much with this as I've been off writing Repton Map Display. But Mark Usher has noticed a few bugs, plus there's some more I fixed previously, which are included.
I've been working on making the tickboxes and radio boxes more RISC OS style - this includes some which have been changed over (click on 'Save as CSV' to see them).
RC2

[geraldholdsworth]

Version 1.45 rc3 is now available via the above link. This includes some bugs and suggestions from Mark Usher (thank you Mark), along with some more of the OS standard check boxes being changed over for RISC OS style tick boxes. One of the major bugs was some of the controls still not getting scaled correctly in high DPI situations.

rc3 has only been made available for Windows 64 bit, as this covers the majority of the user base.

[Rob_hawk]

rc3 has only been made available for Windows 64 bit, as this covers the majority of the user base.

Hi Gerald,

Hope all is well.

Does this mean no updated versions on Mac OSX going forward? I use your great tool quite a lot so it would be a big miss

Look forward to news.

Rob

[geraldholdsworth]

Does this mean no updated versions on Mac OSX going forward? I use your great tool quite a lot so it would be a big miss

That's good to hear - thank you.

I'll never abandon macOS - I also use a Mac (in fact, I've completely ditched Windows, aside from having to use it for work...on occasions), but it is easier to package up a Windows executable than a macOS DMG - I use a script for full releases, but I can just right click over the Windows exe and select 'Compress <filename>'. When I do the next full release, it will be on all three platforms (Windows, macOS and Linux). Saying that, if you want I can manually package up a macOS DMG for 1.45 rc3 for you.

[baz4096]

I'll never abandon macOS - I also use a Mac (in fact, I've completely ditched Windows, aside from having to use it for work...on occasions), but it is easier to package up a Windows executable than a macOS DMG - I use a script for full releases, but I can just right click over the Windows exe and select 'Compress <filename>'. When I do the next full release, it will be on all three platforms (Windows, macOS and Linux). Saying that, if you want I can manually package up a macOS DMG for 1.45 rc3 for you.

Another macOS (occasional) Disc Image Manager user here.
Have you seen DropDMG? Nice free utility that has a GUI for drag and drop DMG creation, and also can be driven from the command line.
Not quite as easy as "right click, compress" but closer:

https://c-command.com/dropdmg/

[Rob_hawk]

Does this mean no updated versions on Mac OSX going forward? I use your great tool quite a lot so it would be a big miss

That's good to hear - thank you.

I'll never abandon macOS -

HI Gerald - This is great news that you’ll be supporting Mac OSX going forward for final releases, thanks very much! I’m more than happy to wait for the full release but thanks for the offer.

Like you there’s not a Windows PC in the house even for work for me! I used to be a Sun Solaris guy many years ago and always liked having the Terminal at home!

Keep up the work on the great tool.

[geraldholdsworth]

I've just released the full version of v1.45. This version has had a slight makeover on it's look. There's also some bug fixes and improvements:

• Removed the file type filter from the 'Add Files' dialogue box.
• Rewrote the function to reconsolidate the ADFS Old Map Free Space Map, sorting entries into order and removing unused ones.
• Styled the application more towards RISC OS style.
• Changed the toolbar on the main window to tabbed toolbars, as the 'coolbars' were not scaling correctly.
• Now creates an 'inf' file for the root directory when extracting to the host system.
• The 'inf' file created for directories now includes an extra field for the directory title.
• Made the scroll bars for the directories display now automatic. Now, they only appear when required.
• Adding files to an ADFS Old Map would sometimes result in a 'Bad FS Map' when read by the MOS.
• When displaying an ADFS disc, the root directory title would be replaced with the disc name instead.
• When displaying BBC BASIC files, line numbers (after GOTO, RESTORE, GOSUB, etc.) were not getting displayed.
• Ignored the attribute setting, from an 'inf' file, when importing directories.
• Defrag would lose all the sub-directory's entries.
• Updating the free space map of an old map ADFS would blank off the root directory's 'StartSeq' number thereby causing a Broken Directory for the root.
• Deleting objects, from Old Map ADFS, did not update the Free Space Map correctly all the time. This was mostly noticeable on Defrag when many files got lost because of lack of space.
• Deleting a directory, in ADFS, did not remove the reference from the internal array. The resultant image was usually unaffected.
• After a defrag, the application would sometimes crash for no apparent reason.
• The modified icon on the Image Details status bar was not being scaled correctly.
• The Image Details status bar was not getting fully repainted after a non-saved image was closed.
• The application would crash if an ADFS image had zero free space.
• Creating a 640KB ADFS L image, then adding a DOS partition would not result in the partition being visible.

As usual, binaries are available for Windows, macOS and Linux in 32bit, 64bit and ARM variants.

[daveejhitchins]

Just tried latest version on MacBook with latest MacOS Ventura 13.1 - Not working - Says App should be updated

Dave H.

[brianmartin_2001]

Excellent stuff Gerald .. amazed you sorted all those issues .. It's a great utility
and has helped me no end .. Thanks for your endeavour and skill in developing it.

BM

[geraldholdsworth]

Just tried latest version on MacBook with latest MacOS Ventura 13.1 - Not working - Says App should be updated

Oh my gawd...I wonder what Apple have done now!

I suspect I maybe need to find the latest Xcode libraries.

[geraldholdsworth]

Excellent stuff Gerald .. amazed you sorted all those issues .. It's a great utility
and has helped me no end .. Thanks for your endeavour and skill in developing it.

Thank you - it means a lot to hear that.

I've found another few bugs - it was a single issue (defragging a New Map ADFS image) that opened a can of worms...the defragging itself was fine, but there were additional bugs which affecting deleting files and directories, and it wasn't reading the FSM correctly (I'm still not entirely convinced I've fixed it as my free space count still differs from RISC OS's free space count).

[lurkio]

On macOS Monterey (ARM), DIM v1.45 says it's "broken" and needs to go to the bin, just like the Repton editor! Had to do the xattr trick to get the app to run.

Is it no longer possible to combine .SSDs into a DSD (or split a DSD into two .SSDs)? Can't figure out how to do it if so.

[geraldholdsworth]

On macOS Monterey (ARM), DIM v1.45 says it's "broken" and needs to go to the bin, just like the Repton editor! Had to do the xattr trick to get the app to run.

Ah, yes. Need to add that line to my script for DIM.

Is it no longer possible to combine .SSDs into a DSD (or split a DSD into two .SSDs)? Can't figure out how to do it if so.

Yep - it's now add and delete partitions. DFS is probably the only partitions that work properly - other partition types are a bit of a muddle. I'm needing to overhaul the internal array structures to deal with them better. It's a big job, and I keep putting it off.

EDIT: OK...it's greyed out. Bugger. It became this.

EDIT 2: You can split a DFS DSD image, just can't join them with DIM...currently.

[geraldholdsworth]

I've fixed a few bugs on Disc Image Manager and made available an interim release (1.45.1). Bugs fixed:

• When deleting directories, the internal directory references were not getting updated.
• When deallocating free space on a New Map ADFS, the wrong idlen was sometimes being used.
• The New Map ADFS free space map was not getting read correctly.
• Renaming an ADFS file in a ADFS/DOS hybrid image could crash the application on Windows. Renames on the DOS or Acorn FS side could also fail.

Thank you to those that found these.

I'm still to fix that issue where macOS Ventura doesn't like it, and I'm not convinced that the fix for Monterey is working...might need to move my development folder back to OneDrive, from iCloud, where it worked much better.

[geraldholdsworth]

Is it no longer possible to combine .SSDs into a DSD (or split a DSD into two .SSDs)? Can't figure out how to do it if so.

I've just been editing the user guide for DIM for the new version and remembered that I had indeed kept in the ability to join two SSDs to make a DSD - just simply open an SSD (which will become drive 0), then, with it open, open another SSD (to become drive 2) and click on Convert when asked.

This will get added into a button on the partition toolbar/menu.

[geraldholdsworth]

Just tried latest version on MacBook with latest MacOS Ventura 13.1 - Not working - Says App should be updated

I've asked about this over on the Lazarus forum, and have been informed thus:

There are new security checks every time the app is run (Gatekeeper used only check the first time an app was run). One of the new security checks blocks apps being run from unexpected locations.

What is the exact error message that the OS emits for your app? Is there anything in the system log?

[tom_seddon]

Got round to trying this out. I'm intending to add ADFS-style hierarchical filing system support to BeebLink, something that I'd like to have interoperate usefully with Disc Image Manager.

I extracted an ADFS disk - the Master Welcome disk - by extracting its $ folder. The only surprising thing about the output: the $.inf has no load address, execution address or size, but other directories do.

Code: Select all

tmbp ~/tmp/MasterWelcome % cat \$.inf
$            OPT=03 TITLE=             DIRTITLE="80T Welcome & Utils"%
tmbp ~/tmp/MasterWelcome % cat \$/LIBRARY.inf
LIBRARY      00000000 00000000 00000500 09 CRC32=E60A9F18 DIRTITLE=LIBRARY              %

I take it the numeric values in the non-$ case are there because the directories have attributes, and so all the numeric values have to be present in order to make the parsing unambiguous?

--Tom

P.S. the most recent .inf thread was here: viewtopic.php?f=2&t=21319&p=301457&hilit=inf#p301457 - if you'd rather discuss this there, I don't mind!

[geraldholdsworth]

It was requested that an export of the root included a *.inf file describing the disc, as the root can't have load/execution addresses or attributes. I was going to post this in the *.inf thread until I realised it was already discussed there anyway, albeit hidden away.

The structure of the root inf file is thus:
<root name> <boot option> <disc title> <directory title>

In actual fact, the last three can be in any order as long as they have the tags. I then added the DIRTITLE (or just TITLE) tag for directory *.inf files to specify the directory title (which is no longer used with the ADFS Big Dirs...e.g., E+ and F+).

[tom_seddon]

It was requested that an export of the root included a *.inf file describing the disc, as the root can't have load/execution addresses or attributes. I was going to post this in the *.inf thread until I realised it was already discussed there anyway, albeit hidden away.

The structure of the root inf file is thus:
<root name> <boot option> <disc title> <directory title>

In actual fact, the last three can be in any order as long as they have the tags. I then added the DIRTITLE (or just TITLE) tag for directory *.inf files to specify the directory title (which is no longer used with the ADFS Big Dirs...e.g., E+ and F+).

I noticed you've updated the Disc Image Manager manual on github with a note about the root dir .inf file - thank you

I've got started on coding some of this stuff up in BeebLink now, and it now (hopefully) correctly handles standard .INF files as produced by Disc Image Manager. I'll be posting a couple of notes about this in the .inf thread.

--Tom

[BeebMaster]

I think there might be a bug in the ADFS interleaving option in Tools, changing the setting doesn't seem to actually change anything. (This is the same as it was in the previous version I was using as well as the latest version). In Preferences, I had it set to Sequential, but then changing in Tools from sequential to interleaving doesn't do anything. To successfully read an interleaved disc I had to change the preferences to auto.

Also I noticed that files added to ADFS images all have the master sequence number of 0 instead of the "current" number.

[geraldholdsworth]

I think there might be a bug in the ADFS interleaving option in Tools, changing the setting doesn't seem to actually change anything. (This is the same as it was in the previous version I was using as well as the latest version). In Preferences, I had it set to Sequential, but then changing in Tools from sequential to interleaving doesn't do anything. To successfully read an interleaved disc I had to change the preferences to auto.

Are you wanting to change the interleave of an image, or just force a certain interleave when loading an image?

Also I noticed that files added to ADFS images all have the master sequence number of 0 instead of the "current" number.

I've been looking into this and implementing it. But yes, so far, new files are written with zero, as is the master sequence number for directories. These are stored as BCD, so is a tad more complex than just storing a number...but nothing I can't do.

[BeebMaster]

Are you wanting to change the interleave of an image, or just force a certain interleave when loading an image?

Yes, just force a certain interleave for an image already loaded. Maybe I've got it wrong and the tool is actually to re-organise the layout for when it's saved again?

So what I'd like to do: load image....some files are all ZZZZZZ...press a button to change interleave....files are now the files.

[geraldholdsworth]

So what I'd like to do: load image....some files are all ZZZZZZ...press a button to change interleave....files are now the files.

Ah. I see.

No, it does it this way: change setting to force interleave, load image. You can then press another button to change the interleave.

The interleave setting just forces DIM to use that interleave for loading future images. The change interleave button moves the data around so it is interleaved differently, but requires the image to be read in OK already (or things can get confusing). Having another button to change the interleave, without moving data around, and then re-read the image is possible and shouldn't be too much trouble to add for the next version.

[BeebMaster]

I have made a giant leap forward today with rebuilding my Econet file server discs after the Great CF Card Crash of February 2013. It's only taken me 10 years!

After spending some time earlier trying to fix the bad sectors on a different Econet disc (the USB stick in my PC which is one of my NFS shares) I thought maybe it's time to have another go at the CF cards, and start again from scratch. So I made a new image of the card which I abandoned using a number of years ago, and thought I would try it in DIM. At first I thought it was stuck on "please wait working...clearing empty entries" but it did get past that after a few minutes. Then I used the extract file option on the L3 root and it did the whole lot of one file server disc in a few seconds, and the second one, known to be full of broken directories and FS error 53, it said "Range error". Curiously I traced this to the !RunImage file in !65Host I have on the file server disc, so it's not important. Even though I selected OK to continue, it still stopped at this point, so with a bit of manual intervention I skipped that file and after about 5 minutes I had a full logical copy of the disc, which counts up as 245,527 files (probably half of that number taking into the account the .inf files, which I reckon is about right on the basis of my oft-quoted "rescue of 100,000+ Teletext pages).

So now "all" I need to do is simplify the Teletext disc structure (it's over 20 levels deep in places, as my grabber just made a new directory inside the current directory every time it counted that it had done about 250 saves) and then I'm sure I can use DIM to add the whole lot back to a new blank image and I'll be finished!

One thing I wasn't clear on - in the .inf files, is the date stamp retained? It looks like it only has load, exec, length & cycle number. In RISC OS the date stamp is contained in the load/exec addresses but for AFS0 it's a separate part of the directory entry.

[geraldholdsworth]

and the second one, known to be full of broken directories and FS error 53, it said "Range error". Curiously I traced this to the !RunImage file in !65Host I have on the file server disc, so it's not important. Even though I selected OK to continue, it still stopped at this point

Curious...must take a look at that bit of code.

One thing I wasn't clear on - in the .inf files, is the date stamp retained?

Not in the current version - I'll put it in on the next version/update.

[BeebMaster]

Date stamps would be really good.

With that !RunImage file, it's inaccessible from the file server with FS Error 53, which means the SIN doesn't point to the JesMap sector. There are plenty of other files like that on the disc, but I didn't get any other errors. I'll try to fish out another FS Error 53 file and see what DIM made of it.

[BeebMaster]

In the !65Tube directory, !RunImage has a SIN of &1AE45E, the contents of which are:

Code: Select all

1AE4:5E00 | 4A 65 73 4D 61 70 00 00  4B 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 | JesMap..K.......................................................
1AE4:5E40 | 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 | ................................................................
1AE4:5E80 | 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 | ................................................................
1AE4:5EC0 | 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 01 | ................................................................

So it is a JesMap sector, but it's corrupted, as there are no pointers to the sectors making up the file, and the JesMap sequence number and copy don't match. This is probably what's giving the "out of range" error in DIM.

Two files in another directory with FS Error 53 do have SINs which don't point to a JesMap, they actually point to the first two sectors of a BASIC program, so on the extracted set of files, both files are empty, so I think DIM has done that correctly by ignoring them.