Review and decide on the Netlify CMS backend & auth options

[Tim Wisniewski]

Netlify CMS offers several backends. The ones that are relevant to us are:

• Git Gateway
• GitHub
• Local

We'll want to aim for:

• An easy installation/setup experience
• One that doesn't require you host your site on Netlify
• A way to work on the site locally (though I expect this will be a relatively small subset of users)

Once we decide on this, we'll likely need to create a ticket for updating the installation docs.

[Tim Wisniewski]

Wow, I just tried using the GitHub backend, and it was astonishingly easy... even easier than the documentation says. I'm wondering if I've missed something, as it seems too good to be true.

All I did was set this in the config.yml file:

backend:
  name: github
  repo: timwis-test/jkan-demo
  branch: gh-pages

I then ran my JKAN instance locally (with local_backend disabled), and it let me login with my GitHub account, using what appears to be a shared "Netlify CMS" app on GitHub. I then updated a dataset, and it committed my changes on behalf of my github.com user correctly.

If I've understood this correctly, it removes the "setting up an oauth application on github" step completely (and instead uses Netlify.com's shared one). I just want to verify that we have the option of not using Netlify.com at all, and running it ourselves, if we want to.

[Tim Wisniewski]

Ah, okay, so this is slightly more complicated than I thought—Netlify must have had something cached when I did it before, but on a fresh fork of JKAN (netlify-cms branch), the login doesn't work, saying I need to setup the GitHub credentials in the Netlify config. So that means the installation options for a JKAN admin would be:

1. Host JKAN on Netlify
2. Host manage/index.html on Netlify, and point it to the JKAN repo
3. Run Netlify Identity on a heroku-like alternative, similar to Gatekeeper now
4. Don't use the editing UI

Just want to think through these a bit more to make sure we stick to our guiding principle of 'easy setup process'.

[Tim Wisniewski]

Apologies for the lack of progress on this PR—I've been a bit stuck on this topic, and haven't had a lot of time to think it through.

It seems like this could amount to a significant change to the 'default' for JKAN. I'd imagine the setup instructions would have 3 options for users:

A. Deploy on GitHub pages (via forking), but with no editing UI (edit the yaml files manually)

B. Deploy on Netlify (via a one-click deploy button), with this new editing UI

C. Self-host anywhere, and optionally run gotrue to enable the editing UI

I think option B would end up being the one we highlight as 'recommended', which is what feels like a significant change. I just wish there were an easy way to host it on GitHub pages and still get the editing UI running.

I'm currently trying to get gotrue running on render.com, to see how complex it will be. So far, it's a bit more complex than I expected (e.g. it requires a database).

@BryanQuigley let me know if you have any ideas on this.

[BryanQuigley]

I don't know if we need to invest in C.

I'd revise A as "Deploy on GitHub pages (via forking) or Gitlab Pages, but with either no editing UI (or built-in) but any user who runs docker can get access to the netlify-cms editing UI locally. " (Or follow these instructions to get netlify-cms running locally - I'm happy to work on getting the docker bit working for netlify editing)

It does seem like B would be the nice default though. I think that's OK to stop there - all the pieces if someone wants to pick it up and self-host anywhere are open source.

[Tim Wisniewski]

I don't know if we need to invest in C.
. . .
It does seem like B would be the nice default though. I think that's OK to stop there - all the pieces if someone wants to pick it up and self-host anywhere are open source.

Good point. I think I just want to validate that it's possible, but I agree we don't need to go down a rabbit hole of optimising for it.

Okay, so we'll make Netlify the recommended hosting option for JKAN, and also make it easy to disable the Editor UI if you just want to host it on Git(hub|lab) Pages and edit the YAML files manually. And perhaps we'll have a blurb at some point on how to self-host the Editor UI, but I agree it's not critical.

I'll translate these tasks into issues for the v2 milestone.

[Tim Wisniewski]

Sorry to open this back up, but as I've been knee deep in #220, I've learnt a bit more about the authentication providers that come with Netlify CMS. I just want to document what I've learnt, and the decision making process around it. It turns out there are two relevant approaches:

1. Git Gateway: You can effectively authenticate with Netlify's Identity widget (like Auth0), using email+password, or OAuth such as Google, GitHub, etc., and then Netlify will perform the git operations on your behalf, using your email address as the author if it knows it. In this scenario, you don't need to have access to the repo on GitHub.com; you just need to be invited as a 'user' on the Netlify site from the Netlify dashboard. The 'users' database is held within Netlify (unless you run gotrue yourself) rather than GitHub.

2. GitHub OAuth: You can point Netlify CMS to an OAuth2 server, exactly as the current JKAN works with Gatekeeper. There are plenty of compatible OAuth2 servers (I tested this one with vercel), or you can use the one that comes with any Netlify site by going to Settings > Access control > OAuth and putting in your Client ID and Secret. This option requires site editors to have write access to the repo on GitHub.

Trying it out

1. Fork the JKAN repo (untick the box that says 'only gh-pages branch')
2. Add a new site on Netlify by clicking 'Import an existing project' and selecting your fork
3. Select the netlify-deploy branch of the repo
4. Change the build command to jekyll build and deploy

Git Gateway

1. Edit editor/netlify-cms.yml, changing backend's name property to git-gateway (Netlify should automatically re-deploy your site)
2. Go to Site Settings > Identity and make sure it's enabled
3. Scroll down to Registration preferences and change it to invite-only
4. Scroll down to External providers and add the GitHub provider (optional)
5. Go to Integrations in the top menu, select Identity on the left menu, then View, and invite yourself
6. Click the activation link in your email. You may have to modify the URL to append /editor/ (only because I haven't added the redirect code)
7. Set a password or sign in with GitHub

GitHub OAuth

1. Edit editor/netlify-cms.yml, changing backend's name property to github (Netlify should automatically re-deploy your site)
2. Register an OAuth application on GitHub. Use https://api.netlify.com/auth/done as the Authorization Callback URL
3. Generate a Client Secret on the OAuth Application page and make a note of it for later
4. On the Netlify dashboard, go to Site Settings > Access control > OAuth, and click Install provider
5. Copy the Client ID and Client Secret from the GitHub OAuth application page.
6. Go to your site's /editor page and sign in with your GitHub account

Pros & Cons

Git Gateway

• Pro: Easier to setup (likely no configuration required at all with the Deploy to Netlify button)
• Pro: Site editors don't need GitHub.com accounts
• Pro: Admin has control over invitation email templates
• Con: Users database & access control is separated from the GitHub repository
• Con: It's much harder to self-host with this option

GitHub OAuth

• Pro: Easy to self-host and run the site on GitHub Pages or elsewhere
• Pro: Users database & access control is handled by the GitHub repository
• Pro: Supports guests proposing changes (via open authoring)
• Pro: Very similar to how JKAN currently works (less significant change between JKAN versions)
• Con: More steps to setup (register github oauth app, copy/paste the client id and secret)

Proposal

Ultimately I think the portability, technical simplicity, and similarity (of the way JKAN works currently) makes GitHub OAuth the winner. We can document how to use the Git Gateway option for those who are happy to go 'all in' on Netlify or don't want to require editors have GitHub accounts.

I'm going to proceed with that in #220, but I'll leave this open for a bit in case anyone would like to share their views (I'd certainly welcome them!).