Best practices developing with git and github

[hoglet]

Ed's suggested we have a thread to capture best practices when developing with git and github.

As a starter, here are a few references

Some existing threads:
- Using GitHub - introduction & tutorial
- Github "Pull-Request" workflow
- Github Token Authentication
- Stardot on Github
- Request membership of the Stardot organisation on GitHub here

External links:
- TODO

Dave

[BigEd]

Thanks Dave - nice selection of links there.

Perhaps I'll try to write just a little orientation and motivation:

Using git allows us to keep track of our work, to save every meaningful change, and also to share our work. It's a way of having backups and having infinite undo to your edits.

It's possible and it can be useful to use git locally on your own machine, but most people make use of a web provider like Github (others are available) as a form of backup, as a way to put the work online, and as a way to share a project and get contributions from others.

I believe git can be useful with the use of just a half-dozen commands, but it's worth noting that it has great hidden depths too, which allow for quite sophisticated use, but with the hazard of getting into a bit of a mess. So, stay within the set of commands and actions which you understand, and if you get a bit stuck, remember that there's always someone with more knowledge who can help out. You can pretty much always get back to where you wanted to be, with the right steps.

In particular, if working on a project in parallel with others, or working on a project making changes which you intend to get "pulled" into the parent project, be sure to do only one thing at a time, and don't have too much change going on without resynchronising with the others or with the parent.

Also, talk to the project owner before embarking on something major, so your work is likely to be accepted and likely to arrive in good shape.

And don't make lots of trivial changes, like adjusting white space or comments, without making sure that's going to be appreciated.

[guesser]

In particular, if working on a project in parallel with others, or working on a project making changes which you intend to get "pulled" into the parent project, be sure to do only one thing at a time, and don't have too much change going on without resynchronising with the others or with the parent.

And work on your feature in a new branch off the main code.

[sweh]

This is a long video (1hr40), but it's well worth watching. "Git for ages 4 and up" presentation at LinuxConfAu in 2013.

https://www.youtube.com/watch?v=1ffBJ4sVUb4

It explains a lot of the basics of git, including branches and tags.

[IanS]

An open source game about learning Git!

https://ohmygit.org/

[shifters74]

Thanks for those links!

My git knowledge is rather basic and (always) needs some work - maybe this thread should be stickied as proper source code control is rather important to all projects going on by stardock members and stuff like like this gives the noobies (like me) the incentive to do it properly from the start!

cheers

shifters

[shifters74]

An open source game about learning Git!

https://ohmygit.org/

That puts a new definition on OMG!

shifters

[IanJeffray]

git is horrendous, complicated, impossible. I use it commercially and privately every day and recommend to all -- but I only use a very limited set of features and almost always use GUIs such as SmartGit and SourceTree. And I self-host using GOGS. Git is also integrated in to tools such as Eclipse, VisualStudio and Altium which is handy. IMO if you ever find yourself using any of the insane and obscure references to objects in git, you're doing something wrong, and that's where it becomes horrendous.

A favourite tip for using git (or any SCCS) - use the commit procedure to enforce good self-review - I've saved myself from committing things I didn't mean to (bugged code, mis-edits, debug code still present, missed files) many times.

I wish there was a git port for RISC OS.

[hoglet]

Continued from here

I think I am getting the hang of making changes using Github, this is to bring it in line with the various changes I have made to my own local copy over the last few days. Thereafter I'm going to have to look into how I can send local edits straight through to the repository without manually having to edit the files online. (Or possibly I just paste a whole replacement file edited elsewhere over the file in the online editor - can it still track the changes if I do that?)

If you are manually editing files on the "github website" to replicate changes you made locally, then please, please just stop! This is not how git/github is meant to be used.

You need to get properly setup so that your git client can push changes back from your local repository to your repository on github (using the git push command).

Github supports two means of authenticating push requests:
- SSH Keys
- Personal Access Tokens

Personally I have always used SSH keys to access my repositories. Creating and uploading these is a something you just do once, then you forget about it. It's a long time since I've done this myself (over 10 years), but there is a guide here:
https://www.theserverside.com/blog/Coff ... untu-Linux

(There are lots of guides to this process, this was just the first I found that looked reasonable.)

I've never used Personal Access Tokens (they were added a couple of years ago to replace passwords), but Stephen talks about them here:
viewtopic.php?f=12&t=22974

Please pick one of these approaches and get it properly setup and tested. You can make your own test respository to verify everything works, and you can successfully push back changed files.

Note: When using SSH keys, you need to clone the repository using the SSH protocol. When using PATs you need to clone the repository using the HTTPS protocol. Both URLs are easily findable in github.

Dave

[BigEd]

I have sometimes edited files directly on GitHub. It works when a meaningful change is to a single file and needs no testing - like an update to a README. The problems with doing more complex changes include:
- inability to have a single change affect multiple files
- the difficulty of testing exactly the code you intend to commit
- the possibility of changing whitespace and getting a bloated change

[BeebMaster]

Personally I have always used SSH keys to access my repositories. Creating and uploading these is a something you just do once, then you forget about it. It's a long time since I've done this myself (over 10 years), but there is a guide here:
https://www.theserverside.com/blog/Coff ... untu-Linux

(There are lots of guides to this process, this was just the first I found that looked reasonable.)
Dave

Thanks Dave, that looks simple enough. It even says so at the end:

Quickly set up GitHub SSH example

.
.
.
.
2,000 steps later...
.
.
.

And that’s how easy it is to setup GitHub SSH keys in Ubuntu for Git.

I'll have a go at that at the weekend, hopefully I won't delete the whole internet or download the Kremlin.

I'm sure it was even simpler when the only copy of anything was on floppy disc. (Actually for everything else I do, it still is.)

[BeebMaster]

I think before I get to any of that it looks like I am going to have to choose & install my "git client". The most obvious choice looks to be Github Desktop application, but there isn't a Linux version. So I am looking for recommendations before I start wading through this sort of list:

https://linuxhint.com/best_git_gui_clients_ubuntu/

[IanJeffray]

I think before I get to any of that it looks like I am going to have to choose & install my "git client".

I do recommend a GUI for minimal pain - and SmartGit is my favourite for Linux. I actually run it on headless Linux server systems, with the X11 display pointed at XMing running on my desktop Windows PC.

But so long as you know the key basic commands, "most stuff" is actually straightforward at the command line - clone, status, add, commit, push.
If you're not working with anyone else on the same tree/branch, that's probably all you'll need.

[BeebMaster]

Yes, minimal pain is my watchword. Doing things in command line is maximal pain usually! Will look at SmartGit, thank you.

[hoglet]

Yes, minimal pain is my watchword. Doing things in command line is maximal pain usually! Will look at SmartGit, thank you.

My advice to anyone getting started with git is to learn the basics of the git command line, because it is universal and it helps build an understanding of what's actually going on. Then later, if you feel the need, try a GUI based client.

A basic git "workflow" on a local repository starts with:

Code: Select all

git clone <Repository URL>

The Repository URL should be to one of your repositories on github (which might be your fork of someone else's repository). The important thing is that it's a repository you have permission to write back to. If using SSH keys it will look like git@github.com:<username>/<repository>.git

You then work locally on the files, make changes, build, test etc.

When you are at a stable point that you want to explicitely record, you start by reviewing which files have changed.

To see what files have changed, use git status

Code: Select all

git status

You then "stage" the files you want to commit using "git add". Staging is telling git what changes you would like to include in the next commit. It allows you to include certain files, and exclude others.

You can stage files one at a time:

Code: Select all

git add <file1> <file2> ...

Or you can stage all changed files:

Code: Select all

git add -u

Or you can stage all changed files and any newly created ones.

Code: Select all

git add .

(this is rarely a good idea, because it's easy to include files you don't want)

At any time you can see the list staged files using git status:

Code: Select all

git status

At this point you have a commit "staged" (i.e. almost ready), it's worth reviewing each and every change in detail. I typically just use the gitk tool for this, because it's fast, quick and graphical:

Code: Select all

gitk

Take your time and verify every change is intentional.

If you accidentally stage files you don't want to commit, then you can unstage them using git reset:

Code: Select all

git reset HEAD <file>...

Once you have the right files staged, then you commit them using git commit:

Code: Select all

git commit -m "Summary of what this change is about"

These changes are now included in your local repository.

You can check that with git log:

Code: Select all

git log

Finally, to publish these changes to others, or just so you have a backup, you push them back to github:

Code: Select all

git push

That's pretty much all you need to know to get started:
- git clone
- git status
- git add
- git reset
- git commit
- git log
- git push
- gitk

There are a few more advanced commands you will need later:
- git revert
- git branch
- git checkout
- git fetch
- git merge

But they are not critical for getting started.

These 10 or so commands cover 99% of everything you will ever need to do with git.

Dave

[guesser]

I use TortoiseGit on Windows, an explorer shell extension which I find very convenient. I've not found anything quite as polished on Linux - I'm using RabbitVCS at the moment.

[shifters74]

I use VSCode on linux for all sorts of languages (including PLASMA but that has no colour syntax hi-lighting) and its built in GUI tool (including diff etc) makes working with github a fair bit easier. But i totally agree with hoglet - understanding the basic command line options of git actually makes using a gui more 'obvious'!

Git kraken (https://sourceforge.net/software/product/GitKraken/) is also not bad on linux.

shifters

[BigEd]

One of many guides I've previously bookmarked:
Github guide: How to pull, branch, squash and fork in Github

As git is a complex and sophisticated tool, I've always found it difficult to find or describe the 'level 1' knowledge that a beginner might need. But I think it's good to
- understand why it's worth using
- get the basics down
before going too deep into the advanced stuff.

[IanJeffray]

My advice to anyone getting started with git is to learn the basics of the git command line, because it is universal and it helps build an understanding of what's actually going on. Then later, if you feel the need, try a GUI based client.

I'll still disagree with that Your post is spot on for basic commandline use, but speaking even as someone who's gone from sccs, cvs, svn, etc over 30 years ... git still seemed impossibly confusing until I could see what was going on with a GUI and play with making changes and commits there - far easier to understand what's going on with SmartGit/SourceTree, then you can learn how to use the command line if you need to.

[hoglet]

I'll still disagree with that

Yes, clearly there isn't a one right answer here. Some people have more of an affinity with graphical interfaces, others with command line based interfaces. Both are valid options, or they wouldn't exist.

[lovebug]

i use github as a backup and difference tool and do upload a lot of changes which is bad for anyone trying to follow it

thanks for the links, im gonna have a good read

im doing my dev in windows 7, ive tried the command line git commands but prefer the simplicity of the github desktop app

[Mince]

I'll still disagree with that

Yes, clearly there isn't a one right answer here. Some people have more of an affinity with graphical interfaces, others with command line based interfaces. Both are valid options, or they wouldn't exist.

I use VS Code when working on more complicated things and haven't had any experience of any other GUI tool, so my opinion is probably very biased, but I agree with hoglet that learning the concepts in a command line is really important to know what's actually going on: I tend to mix the two when working on the same commit - perhaps using the '+' button to 'git add' specific files but then 'git commit' in a shell to actually do the commit and write the message. VS Code does a lot of clever things which only really make sense to me when I can map them to underlying 'git' command lines but I find it quite handy when doing things like a merge with conflicts as it highlights what needs to be done. For small things, I just edit files using vim and and use the command line.

Maybe other GUI tools are better and help you understand and visualise what is going on better. The one thing I think would be really handy is some sort of commit and branch tree to show what's going on and to explain things like rebases and merges: I've done a few talks on git and I always draw diagrams showing the chains to highlight what's going on as it can get really confusing when you're at the command line and get presented with a seven-digit hex number and a large warning saying "you are in a detached HEAD state!".

I found working through the interactive tutorial on git-scm.com really handy. Also, that you can create a repository on a standalone machine and even clone from another local directory, which is really handy to test out push and pull without needing servers and stuff, although github/gitlab make that easy nowadays.

[guesser]

Maybe other GUI tools are better and help you understand and visualise what is going on better. The one thing I think would be really handy is some sort of commit and branch tree to show what's going on and to explain things like rebases and merges: I've done a few talks on git and I always draw diagrams showing the chains to highlight what's going on as it can get really confusing when you're at the command line and get presented with a seven-digit hex number and a large warning saying "you are in a detached HEAD state!".

One of the things TortoiseGit has is the "revision graph" which tries to show how branches and tags are related to each other:

It's not something that I ever really look at though, but I assume it's useful to somebody!

[Coeus]

One of the things TortoiseGit has is the "revision graph" which tries to show how branches and tags are related to each other:

GitHub has something similar as part of the web user interface and I find that really useful. I also find that a graphical diff tool is also really useful - it is much easier to see changes side by side with colour highlighting rather in the traditional unix way.

One thing that makes git a little confusing if you have come from Subversion is some of the commands seem to do something slightly different in git compared to their Subversion equivalent, mostly because of git being distributed rather than centralised. Many people use git with a public repository (like GitHub) in the way they would have a working copy and a central repository, i.e. where sending your work to the remote/central repository is what you do when something is ready for others to see and you want it backed up, so in that respect 'git clone' is a bit like 'svn checkout' and 'git push' is a bit like 'svn commit' because the git versions of commit and checkout work locally with your own repository, i.e. the directory you're working in is a working copy (just like svn) but also part of a complete local repository so 'checkout' and 'commit' are working between the working copy and the local repository and clone, push and pull between the local repository and the remote one.

[guesser]

GitHub has something similar as part of the web user interface and I find that really useful. I also find that a graphical diff tool is also really useful - it is much easier to see changes side by side with colour highlighting rather in the traditional unix way.

Yeah the side by side diff tool is what I miss most when on another system (and being able to add files from the right click context menu or modify which files are staged via checkboxes in the commit window helps me think about what I'm about to commit in a way that just seeing a listing and issuing commands in the terminal doesn't for some reason, I'm not sure why!)

[BigEd]

Even on the command line, git can give you a graphical representation of recent history - see for example
https://stackoverflow.com/a/18287861

One thing that makes git a little confusing if you have come from Subversion is some of the commands seem to do something slightly different in git compared to their Subversion equivalent...

I'd be a bit careful with this line of thinking, although it's natural enough. I'd like to say

Any git repository is as good as any other, whether it's on your disk, your friend's disk, or at some web-connected service like github. That is, once you've got hold of a copy with "git clone" and kept it up to date with "git pull", it has the whole of history for everything.

but it turns out that replicating all the branches from another repository doesn't happen automatically.

At minimum, I think, a git user's mental model needs to contain
- I have my working copy, which might contain some very recent edits on top of some specific state of play
- I have my local repository, which contains the whole history of at least the branch I'm working on and to which I expect to commit my changes when they are ready

But that leads to the frequent problem of having committed changes locally but neglecting to push them upstream so others can play along
- There is an upstream online repository to which I will push my changes for backup and to share them

But when dealing with other people and needing to offer a pull request to a maintainer of a project
- There is an official online repository which will take (and hopefully accept) my changes in the form of pull requests

And for some purposes in day to day workflow, one needs to be aware of an intermediate local area
- There is the staging area, or index, or cache, which sits between your working copy and your local repository, where you accumulate the files which will be modified in the next commit
(I'm a little unclear on this, and that usually doesn't matter, because it's not beginner level knowledge.)

[hoglet]

Here's a short (6 minute) video that explains the basic git workflow quite well. It covers the working directory, the staging area (aka the index), the local repository, the remote repostory, and the git commands used at each stage:
https://www.youtube.com/watch?v=3a2x1iJFJWc

Dave

[tingo]

It's possible and it can be useful to use git locally on your own machine,

One good reason for doing this (a local git repository on your machine) - if you later decide to put the repository somewhere shared (github, gitlab or another git-type server) you can just add remotes and push. And now you will have your history along.

[tingo]

You can even use git to push updates to web sites https://toroid.org/git-website-howto
great for those static web site generators.

[BeebMaster]

I'm set up now and have been using it for a while (with a lot of help from Hoglet). But something is going wrong with my security keys.

When I make local changes and try to push them, at this point it all goes wrong:

Code: Select all

git push origin HEAD
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

The only way past this is for me to generate a new key using ssh-keygen and ssh-add and uploading the keyfile contents to my online Github account. Then the push will work, till the next time.

That's not right, surely? What am I doing wrong?