Disk Preservation and Writing Tools

[danielj]

This is really to act as a placeholder for all the options for backing up your disks! There are some other options like the DataCentre, but this is no longer available generally, so we're aiming to stick to things that are current. Two sets of tools exist - those that use a Beeb/Master, and those that are independent hardware devices. The latter are able to read, and frequently write, multiple disk formats/densities including protected formats. I'd very much recommending following the links given to get more detail, and ask questions of the relevant community to see if the product/tool will meet your needs! And do please let me know if there's anything missing or wrong!

A note as to formats:

SCP/Flux - these are the highest resolution captures of disks - they measure the time between flux reversals on the disk surface. They must be further processed to yield the data, but they hold a representation of pretty much everything on the disk and can be used to create nearly exact copies of originals.

HFE - this is an intermediate format, used with floppy emulators. It's agnostic as to the system it came from. It does a good job of preserving unusual features and consequently most protection mechanisms. GreaseWeazle can write HFE files directly back to floppy.

FSD/JFD - these are formats that hold sector data alongside additional information about sector headers and error codes generated by the floppy controller. FSD is DFS specific, JFD is Archimedes ADFS specific to work with !ADFFS.

SSD/DSD/ADF/ADL/ADM etc - these are sector formats. They just hold the sector data, no metadata/headers, most emulators use these, but they fail to maintain many features which are important in copy protection schemes.

No separate hadware (PC/BBC based)
UPURS - Read/Write:
https://www.retro-kit.co.uk/UPURS/
High speed serial reading and writing of DFS disks. Works with 8271 and 1770 controllers. Needs a special cable (easy to DIY or sometimes purchasable for £10ish) and PC/Mac-side serial port. Tools are ROM Based (BBC/Master/suitably equipped Electron) or Disk based (BBC/Master only). Generates sector (ssd/dsd) images. Also allows direct loading of tape images over serial. Lots of support on Stardot.

DiskBeast - Read only:
https://github.com/scarybeasts/misc/tre ... /discbeast
viewtopic.php?f=2&t=19953#p278720
Open source, can capture HFE images of protected BBC Micro DFS discs using a BBC Micro/Master with 1770 floppy controller. Some method of getting the disc images off your BBC Micro is required. Lots of support on Stardot.

Omniflop - Read/Write
http://www.shlock.co.uk/Utils/OmniFlop/OmniFlop.htm
Requires a PC (running up to Windows 7) with a floppy controller and floppy drive. Will not work with USB floppy drives. Can read/write multiple sector formats but not protected images. Whilst reading single and double density formats tends to work well, not all PC Motherboard controllers work with single density formats.

Dedicated hardware:
General note:
If one piece of software doesn't recover the data well, try another! HxC Floppy Emulator Software, FluxEngine and Disk-Utilities can read a variety of native flux formats, and each decodes the data slightly differently. Whilst one might see a bit as marginal, another might successfully decode it. You can mix-and-match sectors/tracks into a sector image! SCP/HFE images for writing can be created from sector images with Disk-Utilities or the HxC Floppy Emulator Software.

GreaseWeazle - Read/Write (price: £3 for a basic F1 kit, up to around £30)
https://github.com/keirf/Greaseweazle/wiki
Open source, no restrictions on use. Gerbers available. Connects via USB, runs on Windows/MacOS/Linux. The very simplest version which can be self built is a cheap BluePill development board connected to a floppy drive with Dupont cables. Captures to SCP, HFE and sector image formats which can be converted to other formats using Disk-Utilities, FluxEngine (Command Line), HxC Floppy Emulator Utilities (GUI) or the AppleSauce sosftware. Can write SCP/HFE and some sector formats back to disk. Wiki has latest details. Cheapest versions are not buffered and sometimes don't work with very old 5.25" drives. Active community support.

BBC-FDC - Read Only at present(price: £8.50 built or approx £2 for a PCB plus cost of Raspberry Pi)
https://github.com/picosonic/bbc-fdc/
Open source hat for RaspberryPi, no restrictions on use. Gerbers available, PTH parts. Software runs on Linux on the Pi. Software captures to flux formats (native and SCP) and Acorn formats directly with no conversion step needed. SCP capture enables other formats to be derived using Disk-Utilities, FluxEngine or HxC Floppy Emulator Software. Buffered. Active community support.

FluxEngine - Read/Write (price: the cost of a Cypress PSoC5LP CY8CKIT-059 development board - $10-$20ish plus headers)
http://cowlark.com/fluxengine/index.html
Open source, no restrictions on use. Can be self built. Connects via USB, software runs on Linux/Windows/MacOS. Captures and writes multiple formats, including SCP, and sector images from a large variety of systems including Acorn and Victor 9000. Software can be used with other boards to convert between formats, and natively supports greaseweazle hardware.

Supercard Pro - Read/Write (price: US $99.95)
https://www.cbmstuff.com/proddetail.php?prod=SCP
Propriatary. No restrictions on use. Windows software only, connects. Captures to SCP format, can capture to SD card directly under serial control. SCP files can be manipulated with all the software listed above.

KryoFlux Read/Write (price approx 100 euro)
https://kryoflux.com/
License covers non-commercial use. Connects via USB, Windows/Mac/Linux Software. Connects via USB. There is a GUI and command line program which can convert to most sector formats. Software hasn't been updated since 2018. Able to read-write flux images. Can write Kryoflux Raw/IPF formatted files back to disk. Buffered.

AppleSauce- Read/Write (price: approx 300 USD)
https://applesaucefdc.com/
Proprietary. No restrictions on use. software is macOS only (minimum OS version 10.11), it can be run through virtualbox on other systems but YMMV. Connects via USB. Originally designed to work with Apple drives (5.25 and 3.5") but has been expanded to work with PC/Shugart hardware and the software extended massively to boot. Captures to native A2R format when in flux mode. GUI software provides powerful analysis capabilities. Frequently updated with support available via Discord. Numerous formats are supported, including hard sectored disks, most sector formats and many copy-protection methods. Software can import SCP from other sources. Support for various obscure computers such as Victor 9000 and MTU. Buffered.

[scarybeasts]

Hi,

Thanks for collecting this in one place!

Re: discbeast, it does not actually handle double density yet. The current discs of interest -- protected beeb software -- all seem to come on single density 5.25" discs. It's possible double density could be added in the future if need arises.

Re: file formats. It's great that there are so many hardware tools. I think we should be decisive on a recommendation for which file format we prefer people to capture if they are helping with archiving. Is there any reason not to go for SCP, 80 track, inclusive of the 81st track, both sides of a flippy? From there, it's a downsample to HFE. I think that's in line with the great ABUG talk you gave?


Cheers
Chris

[BigEd]

Is it worth a note in the head post about cleaning disks, or the risk of trying to read a mouldy disc, or anything like that? It would be a shame to ruin the very disk you plan to preserve, just because you didn't realise the risk. I suppose dirty heads would be bad too. Is there such a thing as sticky sleeves? I don't know...

[danielj]

Scope creep this was really just meant to outline hardware/methods and not practicalities - perhaps another pinned post with some general practical stuff? "So you want to preserve your floppy?" a flow chart would possibly be a good way of presenting it.

[BigEd]

Yes, perhaps just a "take care - dirty disks damage themselves - see here". Where we link to the best summary or thread we can find.

[flaxcottage]

And don't forget OmniFlop for straight conversion to a variety of sector formats - SSD, DSD, ADL - for both 8-bit and 32-bit Acorn stuff. It only works for unprotected discs and needs an on-board floppy drive interface and needs Windows XP.

It is fast and good for batch archiving.

OmniFlop also reverses the process to make physical copies of software from an image.

[danielj]

And don't forget OmniFlop for straight conversion to a variety of sector formats - SSD, DSD, ADL - for both 8-bit and 32-bit Acorn stuff. It only works for unprotected discs and needs an on-board floppy drive interface and needs Windows XP.

It is fast and good for batch archiving.

OmniFlop also reverses the process to make physical copies of software from an image.

Good point - also note that anything in the above list that says "Read/Write" can make a physical copy from an image.

d.

[JasonStonier]

So, my trusty Pace 5.25" (with BBCFDC) did this to a disc last night:

I've been really diligent about cleaning the heads before imaging every disc - using a Q-Tip and IPA, but clearly that wasn't doing the job.

Thinking it through today, I came up with this:

That's the sleeve from the dead disc with a normal "triangular" coffee filter paper inside - hole cut out so the motor doesn't try to drive it - and a good slug of IPA.

Using Picosonic's newest bbcfdc code allows you to do:

Code: Select all

sudo ./drivetest -tmax 40 -noindex

...which forces the heads to seek to the last track and back without expecting a disc index pulse to be present. I chained that command 5 times in a script to wipe the heads back and forth over the filter paper, and you can see the amount of crap that came off the heads.

The next disc read was fine.

Now, clearly a proper spinning floppy cleaner would be better, but in the absence of one this seemed to work.

[flaxcottage]

That cleaner idea looks great.

[tingo]

Ah! Nice solution!

[iancr77]

I may be wrong, but I’m not sure whether BBC-FDC will capture to SCP. Might just be me skim-reading and missing something, though.

[danielj]

I may be wrong, but I’m not sure whether BBC-FDC will capture to SCP. Might just be me skim-reading and missing something, though.

Just skimming the code in the Repo it does I believe!

d.

[iancr77]

I may be wrong, but I’m not sure whether BBC-FDC will capture to SCP. Might just be me skim-reading and missing something, though.

Just skimming the code in the Repo it does I believe!

d.

Thanks! I’ve been running discbeast recently, but wondering about upping my archive game...

[JasonStonier]

Looks like I’ll be ordering another batch of boards then...

[iancr77]

Looks like I’ll be ordering another batch of boards then...

I’d certainly be interested in putting my name down if you do!

[apellew]

Hi all,

I've just managed to purchase a greaseweazle setup and am currently working my through the Archimedes cover discs I picked up a while ago.

Quick question; SCP files - what's the easiest way to convert them to use with the various Archiemedes emulators out there?

Thanks,

Andy

[danielj]

Depends on the disk - if it's protected, I don't think there's an emulator that will work with HFE at present. If not, the HxC Floppy Emulator software can convert it to a sector image (img/adf - be careful, adf is sometimes an amiga image) that will work, or else you should be able to use Keir Fraser's Disk Utilities (command line) https://github.com/keirf/Disk-Utilities - if using the latter for 800k formats the name is "acorn_adfs_d_e" and for 1.6MB formats it's "acorn_adfs_f" - if you look in https://github.com/keirf/Disk-Utilities ... se/formats you can see all the sector formats it knows about and can output.

Given it's a stack of cover disks, I'd probably use disk-analyse and set up a script just to churn through?


d.

[scarybeasts]

Thanks! I’ve been running discbeast recently, but wondering about upping my archive game...

Been capturing anything interesting?


Cheers
Chris

[iancr77]

I've got a big pile of discs (c.300) which I acquired with a Master 128 a couple of years ago. The Master had been in a local FE college, and the discs include a lot of educational titles, some of which are original discs, some are copies. The discs are in varying conditions - some of course have rotted, but others are afflicted by having been stored in a damp environment (South Wales) for many years, and need a good clean. Unfortunately, none have instructions. I've been meaning to submit them to the Flax Cottage archive, but I'm only just getting around to it.

One issue I've come up against (because I'm a newbie to this) is how to convert HFE images of normally-formatted DFS discs to ssd/dsd if there are missing sectors or tracks. AFAIK, the HxC conversion tool and Keir Fraser's Disk Utilities don't fill missing tracks with anything (e.g. zeros) in the ssd image, so that if (for example) Track 2 is blank, Track 3 will appear in its place (and so on). This obviously causes problems (unless I have misunderstood).

[danielj]

I *think* if you're telling disk utilities what you're asking it to output, it knows what it should be finding on each track and will just enter zero'd sectors as appropriate. However, I've never actually tried this in practice. If it doesn't, it should - so it'd be worth raising a ticket on github. Keir's support is second to none.

d.

[billcarr2005]

One issue I've come up against (because I'm a newbie to this) is how to convert HFE images of normally-formatted DFS discs to ssd/dsd if there are missing sectors or tracks. AFAIK, the HxC conversion tool and Keir Fraser's Disk Utilities don't fill missing tracks with anything (e.g. zeros) in the ssd image, so that if (for example) Track 2 is blank, Track 3 will appear in its place (and so on). This obviously causes problems (unless I have misunderstood).

Chances are if the HFE becomes SSD then it won't work anyway / will require further work to remove any checks for unformatted tracks, etc.
At least all the data will be there though!
Easiest thing is to load the HFE into beebjit and check it functions as it should

[danielj]

Chances are if the HFE becomes SSD then it won't work anyway / will require further work to remove any checks for unformatted tracks, etc.

That's what disk-utilities should do! - it's pretty good at blatting between formats and decoding things:
https://github.com/keirf/Disk-Utilities/

[scarybeasts]

Chances are if the HFE becomes SSD then it won't work anyway / will require further work to remove any checks for unformatted tracks, etc.

That's what disk-utilities should do! - it's pretty good at blatting between formats and decoding things:
https://github.com/keirf/Disk-Utilities/

I think billcarr was saying that if there's something like an unformatted track on the HFE, then SSD fundamentally cannot represent it.


Cheers
Chris

[scarybeasts]

One issue I've come up against (because I'm a newbie to this) is how to convert HFE images of normally-formatted DFS discs to ssd/dsd if there are missing sectors or tracks. AFAIK, the HxC conversion tool and Keir Fraser's Disk Utilities don't fill missing tracks with anything (e.g. zeros) in the ssd image, so that if (for example) Track 2 is blank, Track 3 will appear in its place (and so on). This obviously causes problems (unless I have misunderstood).

Could you post a few HFEs (or TRKS-filled-DSD or SCPs) here that exhibit the challenge? I'd be happy to take a look. If any of those files were of original discs, all the better -- I'd add them to the original disc archive.


Cheers
Chris

[danielj]

I think billcarr was saying that if there's something like an unformatted track on the HFE, then SSD fundamentally cannot represent it.Chris

That's true - most things should just render that as 00 in a sector image though. It won't actually be accurate, but it leaves something ostensibly "complete".

[flaxcottage]

I've got a big pile of discs (c.300) which I acquired with a Master 128 a couple of years ago. ...

... I've been meaning to submit them to the Flax Cottage archive, but I'm only just getting around to it. ...

That sounds great, thanks. I always like getting new stuff - it gives me something to do, ha ha!

[iancr77]

One issue I've come up against (because I'm a newbie to this) is how to convert HFE images of normally-formatted DFS discs to ssd/dsd if there are missing sectors or tracks.

Could you post a few HFEs (or TRKS-filled-DSD or SCPs) here that exhibit the challenge? I'd be happy to take a look. If any of those files were of original discs, all the better -- I'd add them to the original disc archive.

The use-case is this: I have an ordinary, user-formatted 40T DFS disc, but some of the disc is damaged or corrupted (e.g. Track 3 is unreadable). I've been using discbeast to image discs like this because it will repeatedly read problematic tracks until a consistent result is obtained, and so it will record whatever can be obtained from the damaged disc. Previously, I would image such a disc using *BACKUP to a blank ssd image on a Gotek, but it's much less transparent re. damaged data.

What I then want to do is extract the files (and any other data) from the image on my 'non-retro' computer. For SSD images, I've been using https://github.com/jamesyoungman/beebtools, so I need to convert the HFE image to SSD. For this to work, the SSD image needs to have all tracks present, even if they are absent on the original disc. Hence the need to fill Track 3 with zeros (or something).

I can't run beebjit, as I don't have a suitable machine (Mac only, and I haven't had time to get a Linux distro working well on VirtualBox).

However, I think I know what to do - disk-analyse will extract one track at a time if requested, so I'll wrap that in some code which will fill in the blanks.

Hope that all makes sense.

OK - ignore all that. I hadn't clocked that beebtools will handle HFE images... Sorry!

[iancr77]

I've got a big pile of discs (c.300) which I acquired with a Master 128 a couple of years ago. ...

... I've been meaning to submit them to the Flax Cottage archive, but I'm only just getting around to it. ...

That sounds great, thanks. I always like getting new stuff - it gives me something to do, ha ha!

I think I emailed you about this a long time ago, but am only now getting round to it. Life intervened...

[flaxcottage]

I think I emailed you about this a long time ago, but am only now getting round to it. Life intervened...

Yes, I remember. Life has a habit of doing that doesn't it?

[Jeff_HxC2001]

AFAIK, the HxC conversion tool [] don't fill missing tracks with anything (e.g. zeros) in the ssd image, so that if (for example) Track 2 is blank, Track 3 will appear in its place (and so on). This obviously causes problems (unless I have misunderstood).

It does if you want.

Use the "reffile" option to copy one image to another using a reference image as disk geometry. The bad/missing sectors will me tagged or set to the value you want.

HxC Floppy Emulator : Floppy image file converter v2.0.3.1
Copyright (C) 2006-2020 Jean-Francois DEL NERO
This program comes with ABSOLUTELY NO WARRANTY
This is free software, and you are welcome to redistribute it
under certain conditions;

libhxcfe version : 2.13.2.2

Options:
-help : This help
-license : Print the license
-verbose : Verbose mode
-script:[filename] : Script file to execute
-modulelist : List modules in the libhxcfe [FORMAT]
-rawlist : Disk layout list [DISKLAYOUT]
-interfacelist : Floppy interfaces mode list [INTERFACE_MODE]
-finput:[filename] : Input file image
-foutput:[filename] : Output file image
-conv:[FORMAT] : Convert the input file
-reffile:[filename] : Sector to sector copy mode. To be used with -conv : specify the format reference image
-uselayout:[DISKLAYOUT] : Use the Layout [DISKLAYOUT]
-usb:[DRIVE] : start the usb floppy emulator
-infos : Print informations about the input file
-ifmode:[INTERFACE_MODE] : Select the floppy interface mode
-singlestep : Force the single step mode
-doublestep : Force the double step mode
-list : List the content of the floppy image
-getfile:[FILE] : Get a file from the floppy image
-putfile:[FILE] : Put a file to the floppy image