February 11, 2023


authentication github api curl

The GitHub gh CLI rocks

While working on a side project I needed to download a file containing some data to process. The issue? It was on a private GitHub repo within my organization (which also happens to be GitHub!). I was dreading the usual song-and-dance around getting that file using various auth mechanisms but then realized that the GitHub CLI tool can actually do that already.

NOTE: I was using this inside a Taskfile.yaml file that you can read more on the linked blog post above. Ignore the {{ }} bits…that’s just string interpolation.

Where is the file?

Step one is to consult the GitHub API to figure out the correct name of the file. Because the API refers to things differently that the path you see in the URL for repo browsing. Here’s what I ended up with:

https://api.github.com/repos/github/thehub/contents/docs/_data/hubbers.yml
  1. api.github.com/repos – that bit is consistent.
  2. github – fooled you! I work at GitHub, so that’s the name of my organization. The name of a typical repo at work is www.github.com/github/the-repo-name. See the github bit in there? That’s my organization, not your favorite Git hosting website!
  3. thehub – that’s the name of our internal repo where we keep our intranet The Hub. Replace with your repository name.
  4. contents – use that. It’s a consistent part of the API name for your files in your repo.
  5. docs/_data/hubbers.yml – that is the directory/filename of something in my repo. So replace that with whatever it is you want to download.

That is all probably superbly well documented somewhere. But it took me an age to figure it out (and the org name GitHub didn’t help!!!!)

So now to the actual download.

1

curl $(gh api {{.EMPLOYEE_SOURCE}} --jq .download_url) --output {{.EMPLOYEE_DATA}}
  1. EMPLOYEE_SOURCE is the full URL from above. You could just type that whole thing out or use an environment value. Again, I’m using Taskfile.yaml syntax so cut’n’paste will not work in your shell.
  2. EMPLOYEE_DATA is just where I want curl to save the file.
  3. The --jq .download_url is MAGIC. That’s the bit that yanks out just the URL of the file you need to download. When the gh api URL bit runs, it gets a JSON response from the GitHub API. We only need the actual file URL, not the other metadata and bits. So that whole $(gh api...) bit is just to get the real URL of the file resource I want.

So you tried that and it failed?! Yeah, me too. AuthN will get you every time. You have to remember that in order to ‘act as Dylan’ the gh CLI tool needs to know I’m Dylan.

1

gh login

With that taken care of, gh is happy and curl gets the URL to the file I want.

See Also