Signing app commits


#1

Would be awesome to be able to sign commits as having come from an app.

Any plans to add support in the git commits API?


#2

Hi,

I’m one of the engineers who worked on commit/tag signature verification for github.com. Signing commits is actually a bit complicated. After writing any new blobs, you need to prepare the commit as it would be written to the ODB. This is the string that you need to sign with your GPG key. For example, this might look like

tree 71b1c5d9c74f05a5a2c144c7f0140c8533a2efa9
parent 3ed3dd431075fe0a395bd7ec8637d4c6b327af26
author Ben Toews <mastahyeti@users.noreply.github.com> 1506613756 -0600
committer Ben Toews <mastahyeti@users.noreply.github.com> 1506613756 -0600

my commit

Once you’ve created a signature over this string, you add another header, containing the signature and then write the whole thing to the ODB.

It would be pretty trivial for us to add a signature parameter to the create-a-commit API, but it seem like it would be a bit complicated to the integrator (you) to build out the string that needs to be signed. I’m curious for your input on whether this would be too complicated to actually be useful.

Cheers,
Ben Toews


#3

Hi Ben,

Hmmm, the tree sha and parent sha are easy, but those unix timestamps are going to be difficult - when creating a commit through the git data API we don’t know what they’re going to be in advance, right?

I imagine adding an additional endpoint to sign a commit that’s been created through the API is a total no-go, as it would be a rebase and the original commit would have caused hooks to fire, etc… Is there another way? Perhaps an option to include author information in the create commit API, too?

Grey


#4

It looks like you get to specify the date already in the create-a-commit API with the author/committer parameters.


#5

Should have checked that before commenting! :man_facepalming:

In that case I don’t think it’s too much to ask for integrators to build out the string, and the logic to do so could easily live in the octokit client libraries (I’d be happy to add it to the Ruby one).


#6

Ok. No promises, but I’ll look into what would be required on our end to support this.


#7

I added a signature parameter to the create-a-commit API. I’ll be curious to hear any feedback on how it works for you.


#8

Thanks @mastahyeti!

We’re having a little trouble getting this working with GitHub app commits – signed commits from apps currently still come up as unverified, and say “No user is associated with the committer email.” To fix this, we need to associate a GPG key with the app, and specify an email address that identifies the app (we’re currently using the auto-generated email address that’s used by default for app commits). Is there any way we can do this, or will it require a change on your end?

Thanks again for jumping on this so quickly – we’re really excited about feature :slight_smile:


#9

Signatures are only considered verified when the can be associated with a user. It would probably be easiest to create a “bot account” associated with your app. You’ll need to give the account an email address and verify that address with GitHub. This email address will need to be added as an identity in your PGP key and will also need to be the committer email address. If you are making commits on behalf of a user, you can still use that user’s email as the author email. Then, in the GitHub UI, the commit will show up as @someuser committed with @MyAppBot. The Verified badge will show that the signature was made by @MyAppBot. I hope that helps.

The other option is to create a PGP key for each of your users and add that key to their account with the GPG Keys API. Again, the key would need to include an identity matching a verified email address on the user’s account. I wouldn’t recommend this approach though, as users might find it to be invasive for an app to create a key on their behalf using their identity.


#10

Do you have any plans to provide first-class support for GPG key management for GitHub Apps? It would be great if we could manage a set of keys for each app, as well as allow each App to manage their keys via the API similar to the existing User-level key management.


#11

Hi @mastahyeti,

Thanks for the clear explanation regarding signing commits with PGP keys.
I’m currently working on a lightweight Node module to generate the signature automatically given a commit’s payload for the API.
However, I need to generate the string that will be used to sign on your side and one possible pain point is calculating the timezone offset for the author’s and committer’s dates.
It seems that the API is always returning the value +0000, but can you assure me that it will always be the case?
I’m in France, so is this due to the server receiving my request or is this value always set to +0000 on your side, in which case generating the string is quite easy.

Thanks in advance!
Johan


#12

I’m not aware of any such plans. Sorry.

I’m not sure off the top of my head, but my guess is that we do +0000 for all commits. If you’re worried about this changing though, it looks like you can specify timezone in the “Create a Commit” API.


#13

Hi @mastahyeti

Thanks for the info, it’s very helpful.

I have a GitHub App called ImgBot (https://github.com/apps/imgbot)

This app optimizes images and opens Pull Requests with the image optimizations.

The commits are done with a ‘bot account’ @ImgBotApp

The way the app is set up is the following:

  1. Use the private key and AppID to get an installation token from the access_tokens endpoint for a given installation of the app

  2. Clone the repo using username ‘x-access-token’ and password is the token from step 1

  3. Run the image optimizations on the local clone

  4. Make a branch and commit describing the optimization

  5. Push the branch to the remote using username ‘x-access-token’ and password is the token from step 1

  6. Open a Pull Request comparing the new branch to the default branch using username ‘x-access-token’ and password is the installation token from step 1

This is all working out great so far, now I am trying to sign the commit my bot makes and I am getting a weird message in the GitHub UI.

The key whose key-id is in the signature did not sign this commit. Someone may be trying to trick you.

image

Here is an example of a PR I was using to test: https://github.com/dabutvin/GitFixed/pull/4

The commiter and the author are both ImgBotApp ImgBotHelp@gmail.com.
This is the same username/email I used to generate the gpg key
I have uploaded the public key to ImgBotApp’s profile (https://api.github.com/users/ImgBotApp/gpg_keys)

Can you give me more information about what the message means?
How can the key-id be in the signature, but the key didn’t sign the commit?
Can the commiter/author be set the @ImgBotApp when authenticating with an InstallationToken and still get verified?

Here is an example of what the commit looks like with format=raw in the git log:

commit ad9e63b3139e0ff8c6a63473e2553bd3347d7f48
tree df4ed36f1ba8fffe4f5518b3cd2dd58fb994752c
parent 3c51d8ccf9f07df68d9bc133560888b78f3bd738
author ImgBotApp <ImgBotHelp@gmail.com> 1518506033 +0000
committer ImgBotApp <ImgBotHelp@gmail.com> 1518506033 +0000
gpgsig -----BEGIN PGP SIGNATURE-----
Version: BCPG C# v1.8.1.0

iQI+BAEBCAAoBQJagpAxIRxJbWdCb3RBcHAgPEltZ0JvdEhlbHBAZ21haWwuY29t
PgAKCRA/oWDp6dbwreCdEADGbFv4oijplJujaRZ0pL4dEunsgIVj0VICTcPByRVW
sfHhOhXgxkwWsqabkToKrs8dM/9/b+S4Dk1fMqZrbN0NmsJLnJ/jorrj5UmmAdsD
ZRNO0ApH9MVgnLQJt0toKd/BALwtyxtZ0iKiorxEiMG2RgiuRrSSXBdtPJ0C5yKv
Y2c0NQON6GBkzrXLGH/7rce6Hq+Lmj5nqsMvQ+6Cv/qrkXstvDx4tx/HoT2aopSl
4LcopDQ4LBDAuGPuO+vkIHSwjKfkLDc/FtiNa5DAPsePB1CguR80IpF5igsgAzes
Bd4OyPpDk/cZqrXrpH2583GoD2YdnIRX4di/kgOWZdPmZlWgyIfS1dv/XEhItb4O
bV3NmpE49rF4LlpIk6U9N7P1vJKRQm4MyIbfrnn224AkVm1cozAKkwuLOjOMPUpP
kwjqcfgZXCNcdHrUxLu2/GFxUhppPBy/5V1bOAERlPbWJbqQH92M4qLbMmRjcLs4
UScnV3lk2hi1KZEck4+A3W+8fMATd//U/IXrZSg5jrmE3jyEdvFfSUsfA2CN1810
5YiYZ6s5qwEhEYhZBLDM4gDhoo0M763KUwSf8TSRsV2awjA0cKxqXsAPl5fNAzln
o0HJfQUanA0widDB0NeJ2dAwvx01cDoSSUckzE8ZoP1deuFQ5uhQVwnksUsotnYE
Hg==
=Egij
-----END PGP SIGNATURE-----


   [ImgBot] optimizes images

   /Screen Shot 2018-02-10 at 3.09.59 PM.png -- 82.33kb -> 51.16kb (37.86%)

Any help is appreciated. I have spent many hours getting this far and I feel truly stumped at this point.

Thanks!


#14

It looks like the signature is invalid. You can see this with GPG on the command line also:

$ git verify-commit ea7c577e4248fd3a2150d594d25a5c64aa3dc5d1
gpg: Signature made Mon Feb 12 23:38:48 2018 MST
gpg:                using RSA key 3FA160E9E9D6F0AD
gpg:                issuer "ImgBotApp <ImgBotHelp@gmail.com>"
gpg: BAD signature from "ImgBotApp <ImgBotHelp@gmail.com>" [unknown]

I don’t have any experience working with BCPG, but it might be a bug in that tool.


#15

I played around with manually verifying this signature for a few minutes. It looks like you’re missing a newline character in the data you’re generating the signature over.

Here’s the commit as it is in Git’s object database:

$ git cat-file -p ea7c577e4248fd3a2150d594d25a5c64aa3dc5d1
tree f9819b11f5537b127cdec3b9f89499d7917aa0c3
parent 3c51d8ccf9f07df68d9bc133560888b78f3bd738
author ImgBotApp <ImgBotHelp@gmail.com> 1518503925 +0000
committer ImgBotApp <ImgBotHelp@gmail.com> 1518503925 +0000
gpgsig -----BEGIN PGP SIGNATURE-----
 Version: BCPG C# v1.8.1.0

 iQI+BAEBCAAoBQJagof4IRxJbWdCb3RBcHAgPEltZ0JvdEhlbHBAZ21haWwuY29t
 PgAKCRA/oWDp6dbwrcQ+D/9oPYpaRjGYky8UiaMDfTYVAUUblspDqJ26e0nmKOGC
 DB8W13fCUWmDkKKRJ1GnfI9YTLnXsWP76iSXW4KmRbnjbaV0Inb0VKW3n1Ryfgki
 n0d/gpbZxPGtZ5Kwiev1I36M+OXBG0YEUkyatMBLIawpUzJXHcqYKq/rL+1OCSLB
 BsGyMuFcHVyL+b9nY5jqvI/DQF3K0QWevWr8KQegfFH/yx1/1D/q11YLINqiSdYE
 Fn4HocbG9ni/QficyggLZCUf9PyxlY6aONXTDXlXBKLK8U4HoNbz/d+WrhbwfAg8
 knRvNjPcghFAb3Gycrb14ZmJttZ87anQszUwOgvR9IrvOfbBeIKCdvOCQ4CMne9N
 YmZj8iINWihLqpGgDJn5Shg0CCcqOaWp+mrGgGvlnyC9QUgl8aoqZ9KnH2vmqHpM
 p8u5W4wJn+Ol7gCpunbX+t3xgSHaAx2vuoqwJy2J8dOT64mxas8dRiNDIqvRJUkt
 1LLID+j6cpVQaEbgfx5jlue8/MGRyZQP70KfE2sWiP2zTifesclj3Yd1xyRhYDsj
 VgcMQsybsBVSZ3vpXVBzS3GHQGtu7WpKUd03sO88JI4nniJXE/chLn2RvX74cozI
 wzlhNgqwSFe6Wts4tfUFKoxXZe9x3vZ5zrmZPNmGo+bWe47cpsc680k8PB5DyPQx
 ag==
 =qOOu
 -----END PGP SIGNATURE-----


[ImgBot] optimizes images

/Screen Shot 2018-02-10 at 3.09.59 PM.png -- 82.33kb -> 51.16kb (37.86%)

So, the signed data is:

$ git cat-file -p ea7c577e4248fd3a2150d594d25a5c64aa3dc5d1
tree f9819b11f5537b127cdec3b9f89499d7917aa0c3
parent 3c51d8ccf9f07df68d9bc133560888b78f3bd738
author ImgBotApp <ImgBotHelp@gmail.com> 1518503925 +0000
committer ImgBotApp <ImgBotHelp@gmail.com> 1518503925 +0000

[ImgBot] optimizes images

/Screen Shot 2018-02-10 at 3.09.59 PM.png -- 82.33kb -> 51.16kb (37.86%)

I wrote this to one file, and the signature to another file. I then tried verifying the commit with GPG:

$ gpg --verify ./sig ./commit
gpg: Signature made Mon Feb 12 23:38:48 2018 MST
gpg:                using RSA key 3FA160E9E9D6F0AD
gpg:                issuer "ImgBotApp <ImgBotHelp@gmail.com>"
gpg: BAD signature from "ImgBotApp <ImgBotHelp@gmail.com>" [unknown]

This file ends in \r\n\n. I edited the file, removing the last newline, after which the signature is verifiable:

$ gpg --verify ./sig ./commit2
gpg: Signature made Mon Feb 12 23:38:48 2018 MST
gpg:                using RSA key 3FA160E9E9D6F0AD
gpg:                issuer "ImgBotApp <ImgBotHelp@gmail.com>"
gpg: Good signature from "ImgBotApp <ImgBotHelp@gmail.com>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 3051 EB06 721E 8F31 B1B1  8444 3FA1 60E9 E9D6 F0AD

#16

@mastahyeti

Thanks so much digging into this.

I followed your steps and was able to reproduce getting the Bad signature and then Good signature by removing the last newline from the commit.

I’ve been playing around with different commit message formats and trying to get the right combination of newlines here - no luck so far, but I feel close.

This is where the signing is happening right now https://github.com/dabutvin/ImgBot/pull/78/files#diff-d7ab1bbd466440b1ab431978f112a6f0R97

I am backing up a little bit to try and get to the bottom of this.


#17

I got a verified signature in the commit by adding an extra newline to the data I am signing!
This newline must be getting created along the way in the library I am using.

I am still seeing the warning in the UI though

Do you have any idea why I could get the message
The key whose key-id is in the signature did not sign this commit. Someone may be trying to trick you. even though it is a good signature?

commit ad007a77d3a010a7fa36b5885271cd3ec84b9bd3 (HEAD -> imgbot-4, origin/imgbot-4)
gpg: Signature made Fri Feb 16 11:00:12 2018 PST
gpg:                using RSA key 3FA160E9E9D6F0AD
gpg:                issuer "ImgBotApp <ImgBotHelp@gmail.com>"
gpg: Good signature from "ImgBotApp <ImgBotHelp@gmail.com>" [ultimate]
Author: ImgBotApp <ImgBotHelp@gmail.com>
Date:   Fri Feb 16 19:00:11 2018 +0000

    [ImgBot] optimizes images
    
    /Screen Shot 2018-02-10 at 3.09.59 PM.png -- 82.33kb -> 51.16kb (37.86%)

I also tried removing the version header with no luck


#18

FWIW, we’ve got commit signatures running nicely for Dependabot now - thanks a lot for adding the feature @mastahyeti!


#19

@mastahyeti is there anything you can see from the commit to help me out here?

I fixed the newline issue and now I am seeing a verified commit signature on the command line, but still seeing an unverified badge in GitHub UI. https://github.com/dabutvin/GitFixed/pull/5

Really appreciate it.
Thanks


#20

Sorry for not getting back to you sooner. I looked into this and traced the verification failure back to our openpg library. I didn’t dig any further, but it seems that there is some difference between how Go’s openpgp library is validating the signature and how GnuPG is.

A good next step might be to try to reproduce this in a more isolated way and open an issue with the openpgp library. Better yet would be to write a failing test for the library. It is worth noting that the github.com/keybase/go-crypto Go package is a fork of the golang.org/x/crypto/openpgp package, so it would be worth testing against the upstream version as well. This is the first time I’ve seen the BouncyCastle PGP library being used, so I’m not terribly surprised so see that it doesn’t play well with Go’s implementation.