Stardot Github

[simonm]

Hi folks,
Just been doing a bit of housekeeping on our Stardot Github org, and a couple of things to share:

Firstly, 27 of our 45 contributors do not yet have two-factor security enabled on their accounts, and really this is something we need to enforce. So on 1st February 2023 I will change the org settings to make this a requirement for contributors. Anyone on that date without 2FA enabled, will be removed from the org and will be sent an email from Github explaining the steps they need to take before we can add them back.

More info on how to do that here: https://docs.github.com/en/authenticati ... cation-2fa

Secondly, there's now a profile section on our organisation page: https://github.com/stardot
Owners can edit this and add more info as you like.

Next, David Boddie, if you read this, I've combined your various game map tool projects into one repo now: https://github.com/stardot/GameMaps


Lastly, thanks everyone for keeping this going, it's turning out to be a really useful resource.


Cheers
Simon

[paulb]

Is there any way of interacting with GitHub without providing a phone number or using a mobile "app" under this new regime?

[simonm]

Is there any way of interacting with GitHub without providing a phone number or using a mobile "app" under this new regime?

Probably not. There's various options for authentication: https://docs.github.com/en/authenticati ... entication but they all seem to be either app or sms based.

I didn't think this would be controversial tbh, in this day and age 2FA is a pretty important tool to keep your online accounts more secure.

[simonm]

Oh and I also uploaded an export from the retrosoftware.co.uk MediaWiki site I'd been meaning to do for a while.
It's a snapshot of all the raw files from the site - perhaps someone out there might be interested in seeing if its possible to convert the content to Github markdown/wiki format.... https://github.com/stardot/retrosoftware.co.uk

[SteveBagley]

Is there any way of interacting with GitHub without providing a phone number or using a mobile "app" under this new regime?

If it's a standard TOTP 2FA then you'd probably be able to write a BBC Basic program to generate the required codes -- it's already been done with a C64

https://oldvcr.blogspot.com/2022/11/mee ... cator.html

Steve

[simonm]

This is all kinds of awesome.

[paulb]

If it's a standard TOTP 2FA then you'd probably be able to write a BBC Basic program to generate the required codes -- it's already been done with a C64

https://oldvcr.blogspot.com/2022/11/mee ... cator.html

Well, just a simple program that does the job would be adequate. I actually have a device that supposedly does U2F stuff, but I don't know whether such technology keeps up with the latest shiny or whether it has been swept off the table by petulant Valley hipsters, particularly since Microsoft is involved. It looks like U2F is no longer supported by GitHub.

[sa_scott]

Oh and I also uploaded an export from the retrosoftware.co.uk MediaWiki site I'd been meaning to do for a while.
It's a snapshot of all the raw files from the site - perhaps someone out there might be interested in seeing if its possible to convert the content to Github markdown/wiki format.... https://github.com/stardot/retrosoftware.co.uk

I'm having a go at this, but am using a different method - let me see how I get on

[davidb]

Next, David Boddie, if you read this, I've combined your various game map tool projects into one repo now: https://github.com/stardot/GameMaps

Thanks for merging them. I was going to ask if you considered making them subrepositories instead of importing the code, but perhaps that doesn't really simplify the repository situation in the stardot organisation.

[sweh]

Is there any way of interacting with GitHub without providing a phone number or using a mobile "app" under this new regime?

They used to (and indeed were one of the first) to use U2F physical keys, and in 2015 sold octocat branded yubikey tokens for $5 plus shipping. I got two of them somewhere...

Ah... https://docs.github.com/en/authenticati ... entication claims they still support security keys and TOTP (which has a number of implementations; eg google authenticator, microsoft authenticator, authy...)

[sweh]

If it's a standard TOTP 2FA then you'd probably be able to write a BBC Basic program to generate the required codes -- it's already been done with a C64

https://oldvcr.blogspot.com/2022/11/mee ... cator.html

Here's a perl program...

Code: Select all

#!/usr/bin/perl

use strict;

# This is a simple TOTP generator; the seed is passed on the command line
# and this algorithm will output the current and next values in a loop
# Yes, it's bad to put secrets on the command line so only use this on
# a machine you trust.
# The generator function main algorithm came from
#   https://github.com/j256/perl-two-factor-auth/blob/master/totp.pl
# but using MIME::Base32 do convert base32 to binary
#
# That routine is Copyright 2015, Gray Watson
#
# I'm not using the code directly, but I can't claim this is a clean
# room re-implementation!

use Digest::HMAC_SHA1 qw( hmac_sha1_hex );
use MIME::Base32 qw( RFC );

# In the more general case this should be an input, but I've never
# seen one that isn't 30 seconds.  So... lazy me!
my $TIME_STEP = 30;

my $pname=$0; $pname=~s!^.*/!!;

my $seed = join("",@ARGV);
die "Syntax: $pname base32seed\n" unless $seed;

$seed=~s/ //g;

sub do_totp($$)
{
  my ($seed,$time) = @_;

  # For more details of this magic algorithm, see:
  # http://en.wikipedia.org/wiki/Time-based_One-time_Password_Algorithm

  # Round the time down to the stepping value and convert to hex
  $time = sprintf("%016x", int($time / $TIME_STEP));

  # And convert it to binary
  my $data = pack('H*', $time);

  # Get the real binary seed
  my $key =  MIME::Base32::decode($seed);

  # encrypt the data with the key and return the SHA1 of it in hex
  my $hmac = hmac_sha1_hex($data, $key);

  # Now this is where I do a lot of trust in Gray's code...
  my $offset = hex(substr($hmac, -1));
  my $encrypted = hex(substr($hmac, $offset * 2, 8)) & 0x7fffffff;

  # the token is then the last 6 digits in the number
  my $token = $encrypted % 1000000;
  return sprintf("%06d", $token);
}

$|=1;
while (1) {
  my $time=time();
  my $code = do_totp($seed,$time);
  my $nextcode = do_totp($seed,$time+$TIME_STEP);
  my $diff = $TIME_STEP - ($time % $TIME_STEP);
  print "Current = $code - Next = $nextcode - Change in $diff seconds  \r";
  sleep(1);
}

[davidb]

I just discovered 2fa, which seems to do the trick.

[Arcadian]

Thanks Simon for your continued work on the Stardot Github page

Oh and I also uploaded an export from the retrosoftware.co.uk MediaWiki site I'd been meaning to do for a while.
It's a snapshot of all the raw files from the site - perhaps someone out there might be interested in seeing if its possible to convert the content to Github markdown/wiki format.... https://github.com/stardot/retrosoftware.co.uk

I'm having a go at this, but am using a different method - let me see how I get on

Stephen, Simon - the Wiki files are still present on the Retro Software server, so if either of you would like to take a look at the FTP site and Database (with the intention of exporting a dump of the most recent version of the Wiki) please let me know and I can sort access.

[sa_scott]

Thanks Simon for your continued work on the Stardot Github page

Oh and I also uploaded an export from the retrosoftware.co.uk MediaWiki site I'd been meaning to do for a while.
It's a snapshot of all the raw files from the site - perhaps someone out there might be interested in seeing if its possible to convert the content to Github markdown/wiki format.... https://github.com/stardot/retrosoftware.co.uk

I'm having a go at this, but am using a different method - let me see how I get on

Stephen, Simon - the Wiki files are still present on the Retro Software server, so if either of you would like to take a look at the FTP site and Database (with the intention of exporting a dump of the most recent version of the Wiki) please let me know and I can sort access.

So far, I've been using pandoc on its own, using the following command (I'm using Mac OS Terminal):

Code: Select all

pandoc -f mediawiki -t markdown retrosoftwarecouk_wiki-20160918-history.xml

I keep hitting formatting errors, but have corrected them as I go along, as best I can. I'm about 90% of the way through, having had a break of 12 hours.

UPDATE - Ah, I've been thwarted by missing closing </text> elements (about 400+). I'm attempting to reformat the document using one of the copious XML plugins available for VS Code. It hasn't finished yet, as I type, so I don't know what the outcome will be.

Given it's an old backup, I don't think I'll have time to go through the more recent site backups.

[ctr]

I just discovered 2fa, which seems to do the trick.

Cheers, that's worked for me too.

[elk1984]

Oh and I also uploaded an export from the retrosoftware.co.uk MediaWiki site I'd been meaning to do for a while.
It's a snapshot of all the raw files from the site - perhaps someone out there might be interested in seeing if its possible to convert the content to Github markdown/wiki format.... https://github.com/stardot/retrosoftware.co.uk

I'm having a go at this, but am using a different method - let me see how I get on

Can you let me know how you get on with that please? I'm happy to help out too.

[simonm]

Nice work Stephen, feel free to commit your solution to that repo. Even WIP is good to have in there.
I understand that the wiki's for a given repo are just another repo. so it's possible to clone/commit changes directly
eg. for the retrosoftware.co.uk repo, the git remote is:

Code: Select all

git clone https://github.com/stardot/retrosoftware.co.uk.wiki.git

https://docs.github.com/en/communities/ ... r-computer

[sa_scott]

Nice work Stephen, feel free to commit your solution to that repo. Even WIP is good to have in there.
I understand that the wiki's for a given repo are just another repo. so it's possible to clone/commit changes directly
eg. for the retrosoftware.co.uk repo, the git remote is:

Code: Select all

git clone https://github.com/stardot/retrosoftware.co.uk.wiki.git

https://docs.github.com/en/communities/ ... r-computer

I'm afraid I hit a block, too many formatting errors, and childcare issues make it impossible for me to devote more time. I may have been a bit foolish to have considered it a good idea to do this. Clearly converting such large files is not for the faint hearted, especially when it's XML

I'll have to stop on this, my apologies. Hopefully this can get sorted out - the retrosoftware archive was, and is, a great resource!

[simonm]

I'm afraid I hit a block, too many formatting errors, and childcare issues make it impossible for me to devote more time. I may have been a bit foolish to have considered it a good idea to do this. Clearly converting such large files is not for the faint hearted, especially when it's XML

I'll have to stop on this, my apologies. Hopefully this can get sorted out - the retrosoftware archive was, and is, a great resource!

Totally understand - I took a pop at this a few years ago with some python scripts (which I can no longer find anywhere) - and yeah I remember the initial dump of XML -> MD was ok, but a LOT of edge cases. Thanks for giving it a shot tho. No rush on this anyway, can be a slow burner this one.

[paulb]

Totally understand - I took a pop at this a few years ago with some python scripts (which I can no longer find anywhere) - and yeah I remember the initial dump of XML -> MD was ok, but a LOT of edge cases. Thanks for giving it a shot tho. No rush on this anyway, can be a slow burner this one.

I would imagine that Markdown is a pretty bad format for representing various aspects of MediaWiki syntax, being a rather poor format generally, but the cool kids are all using it, so we all apparently have to live with it (and reStructuredText). And I imagine that parsing MediaWiki syntax can be challenging in some respects, too, even though the Pandoc people have probably given it a go.

My own experience of informal markup language conversion is that even after covering what might seem to be a decent subset of any markup language, there will be plenty of exotic things that creep in and diminish the fidelity of the final result. But it is nevertheless important to do as good a job as possible and not throw unconverted markup into people's laps, compelling them to do tedious busy work. Such experiences have only reinforced my own beliefs in interoperability and that content migration should be undertaken warily.

[simonm]

Markdown was merely a suggestion really, so if there's a WikiMedia -> HTML option out there that would be fine too, since we could host this on Github pages.
I don't have a strong opinion on the how, but the why would be to have as much of the retrosoftware site content resurrected to be readable as we could reasonably achieve given the spare time nature of the work.

[simonm]

Thanks for merging them. I was going to ask if you considered making them subrepositories instead of importing the code, but perhaps that doesn't really simplify the repository situation in the stardot organisation.

Yeah, just optimizing the number of org repos really as it seemed like these projects were all closely related, so I moved them into separate root folders and merged all the repos using `git remote add` and `git merge` - this way it preserved all of your commit histories. I've archived the source repos for now so they are read only, but at some point we can simply delete them and nothing will be lost.

[davidb]

Yeah, just optimizing the number of org repos really as it seemed like these projects were all closely related, so I moved them into separate root folders and merged all the repos using `git remote add` and `git merge` - this way it preserved all of your commit histories. I've archived the source repos for now so they are read only, but at some point we can simply delete them and nothing will be lost.

I have local copies, so it's no big deal if GitHub removes them at some point. I could even transfer them out to my account if that would make things easier to manage.

[paulb]

Markdown was merely a suggestion really, so if there's a WikiMedia -> HTML option out there that would be fine too, since we could host this on Github pages.
I don't have a strong opinion on the how, but the why would be to have as much of the retrosoftware site content resurrected to be readable as we could reasonably achieve given the spare time nature of the work.

I'm not so familiar with MediaWiki archiving, I'm afraid. Obviously, people sometimes pursue the wget-style approach which is usually far from optimal, but it is likely that if you actually have a MediaWiki instance running, there may be support for generating a static version of a wiki. I know from my own experience that other wiki solutions have HTML archiving capabilities. In the case of MoinMoin, the implementation is superficially adequate but so deficient in practice that I wrote my own tools to do the job in a more capable way, to the point that I actually maintain documentation in that markup format and don't even use MoinMoin to edit it or even process it any more.

I guess that any attempts to convert the site will inform whether Markdown might be sufficient to represent the MediaWiki markup actually used, but my suspicion is that Markdown's capabilities will probably be exceeded fairly easily, even with random extensions supported by GitHub. That's the unfortunate outcome of people peddling a markup language that is good enough for their own limited needs (typically README files with limited typographic enhancements) without being aware of what other markup languages are capable of.

I had a brief look for HTML archiving solutions for MediaWiki but ended up down the usual rabbit hole of hints and tips that have aged badly and were never particularly satisfactory in the first place. However, this might be a reasonable place to start:

"Publishing from MediaWiki"

Ideally, though, you'd use the XML export content, which might well be a job for Pandoc, instead of driving an active MediaWiki instance.

[simonm]

I've started up a new topic for the retrosoftware site conversations here: viewtopic.php?f=67&t=26218

[Coeus]

GitHub had already hassled me for 2FA and I have set it up, but I still don't see why it is needed. I don't doubt it is a more secure way if authenticaing and, for access to my bank account, my heath records and my employer's VPN it seems entirely sensible. But who would be interested in masquerading as me in order to contribute to some BBC Micro related open source projects?

[BigEd]

The threat is not people intending to contribute, it's people intending to do damage. And it might not be people, it might be automated. And they might not be intending to damage this repository, but any and all repositories.

Without 2fa, any computer used by any of the contributors is a possible vector for damaging the repository. Likewise any account of any contributor who happens to be using a password which has been compromised somehow.

[simonm]

As BigEd says, 2FA just addresses the concern that someone's account gets compromised and then the stardot org repos are at risk of bad actors doing irreversible stuff like deleting repos/accounts/orgs.

There's still 23 folks yet to switch, so I'm going to push this down the schedule to April because I dont want to be removing valued team members unnecessarily , but please folks, if you are on this list, it is highly recommended that you setup 2FA on your accounts:

https://github.com/orgs/stardot/people? ... 3Adisabled

Thanks!

[simonm]

Just a quick followup, Github are rolling out 2FA enforcement anyway over the remains of the year so will leave it at that.
https://github.blog/2023-03-09-raising- ... -march-13/