FHIR / auto-ig-builder

Hello,
What would be the difficulty in adding support for building from GitLab repositories? I tried building an IG from https://gitlab.mitre.org/ this week, and wasn't successful. I'm assuming that https://github.com is hardcoded in the codebase at the current moment.

Update: It appears that GitLab is sending data via plain/text and doesn't offer an option to send via application/json.

how wasn't it successful?

Well, I have a HelloImplementationGuide example here:
https://gitlab.mitre.org/awatson/hello-implementation-guide

Set up the webhook like so:

But nothing shows up at https://build.fhir.org/ig/awatson/hello-implementation-guide/

I'm receiving an error that says: Hook executed successfully but returned HTTP 500 Error: could not handle the request.
One thing I noticed is that the GitLab webhook configuration interface doesn't have the option to specify application/json.

Basically followed the same instructions that got the IG working on GitHub, but it doesn't work from GitLab. My assumption is that it's a URL whitelisting sort of thing, but maybe it's a content type?

POST https://us-central1-fhir-org-starter-project.cloudfunctions.net/ig-commit-trigger
Push Hook
Completed in 0.32 seconds (17 minutes ago)

Response
500

Error: could not handle the request

Headers

Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
X-Cloud-Trace-Context: 2715da26da58bd9fb7e3bc671a5c2a2c;o=1
Date: Thu, 06 Jan 2022 19:38:31 GMT
Server: Google Frontend
Content-Length: 36
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Connection: close

Request

{
  "object_kind": "push",
  "event_name": "push",
  "before": "1a6886fcc828a733da862163f002fa19f151e867",
  "after": "440fd068da094572e581288e568431409b9cf46b",
  "ref": "refs/heads/master",
  "checkout_sha": "440fd068da094572e581288e568431409b9cf46b",
  "message": null,
  "user_id": 7418,
  "user_name": "Abigail R Watson",
  "user_username": "awatson",
  "user_email": "",
  "user_avatar": "https://gravatar.mitre.org/avatar/124ff4d7b84b33c1cad2067a910d6246?s=80&d=identicon",
  "project_id": 36398,
  "project": {
    "id": 36398,
    "name": "HelloImplementationGuide",
    "description": "",
    "web_url": "https://gitlab.mitre.org/awatson/hello-implementation-guide",
    "avatar_url": null,
    "git_ssh_url": "git@gitlab.mitre.org:awatson/hello-implementation-guide.git",
    "git_http_url": "https://gitlab.mitre.org/awatson/hello-implementation-guide.git",
    "namespace": "Abigail R Watson",
    "visibility_level": 20,
    "path_with_namespace": "awatson/hello-implementation-guide",
    "default_branch": "master",
    "ci_config_path": "",
    "homepage": "https://gitlab.mitre.org/awatson/hello-implementation-guide",
    "url": "git@gitlab.mitre.org:awatson/hello-implementation-guide.git",
    "ssh_url": "git@gitlab.mitre.org:awatson/hello-implementation-guide.git",
    "http_url": "https://gitlab.mitre.org/awatson/hello-implementation-guide.git"
  },
  "commits": [
    {
      "id": "440fd068da094572e581288e568431409b9cf46b",
      "message": "updated ig.ini\n",
      "title": "updated ig.ini",
      "timestamp": "2022-01-04T00:44:04-06:00",
      "url": "https://gitlab.mitre.org/awatson/hello-implementation-guide/-/commit/440fd068da094572e581288e568431409b9cf46b",
      "author": {
        "name": "Abigail Watson",
        "email": "awatson@mitre.org"
      },
      "added": [

      ],
      "modified": [
        "ig.ini"
      ],
      "removed": [

      ]
    },
    {
      "id": "358068ca44c2a226a550f217b76441fb7e67befb",
      "message": "updated index.md\n",
      "title": "updated index.md",
      "timestamp": "2022-01-04T00:33:28-06:00",
      "url": "https://gitlab.mitre.org/awatson/hello-implementation-guide/-/commit/358068ca44c2a226a550f217b76441fb7e67befb",
      "author": {
        "name": "Abigail Watson",
        "email": "awatson@mitre.org"
      },
      "added": [

      ],
      "modified": [
        "input/pagecontent/index.md"
      ],
      "removed": [

      ]
    },
    {
      "id": "1a6886fcc828a733da862163f002fa19f151e867",
      "message": "initial commit\n",
      "title": "initial commit",
      "timestamp": "2022-01-04T00:19:00-06:00",
      "url": "https://gitlab.mitre.org/awatson/hello-implementation-guide/-/commit/1a6886fcc828a733da862163f002fa19f151e867",
      "author": {
        "name": "Abigail Watson",
        "email": "awatson@mitre.org"
      },
      "added": [
        ".gitignore",
        "_genonce.bat",
        "_genonce.sh",
        "_updatePublisher.bat",
        "_updatePublisher.sh",
        "ig.ini",
        "input/fsh/patient.fsh",
        "input/ignoreWarnings.txt",
        "input/pagecontent/index.md",
        "sushi-config.yaml"
      ],
      "modified": [

      ],
      "removed": [

      ]
    }
  ],
  "total_commits_count": 3,
  "push_options": {
  },
  "repository": {
    "name": "HelloImplementationGuide",
    "url": "git@gitlab.mitre.org:awatson/hello-implementation-guide.git",
    "description": "",
    "homepage": "https://gitlab.mitre.org/awatson/hello-implementation-guide",
    "git_http_url": "https://gitlab.mitre.org/awatson/hello-implementation-guide.git",
    "git_ssh_url": "git@gitlab.mitre.org:awatson/hello-implementation-guide.git",
    "visibility_level": 20
  }
}

Headers

Content-Type: application/json
User-Agent: GitLab/14.6.0-ee
X-Gitlab-Event: Push Hook

Mmmm.... yeah, upon closer inspection, it appears that GitLab doesn't allow us to specify an application/json type, and is sending it over via text/plain. :/

There are several places in the pipeline today where we assume there's a GitHub repository. Keep in mind that this infrastructure not only builds but also publishes the results of builds in a fashion that right now transparently maps GitHub repository names to build.fhir.org URLs, so any approach we take to generalize the repository hosting would need to be careful about avoiding name collisions on the publication side as well.

As a practical matter, I should note that we've struggled to find help even for maintenance (e.g, our most requested feature right now is eliminating the hardcoded dependency on a default branch name of master, which I think is a prerequisite to trying to generalize support for repository hosting services -- so I would suggest that as a place to start; background in slides and a YouTube video at https://docs.google.com/presentation/d/12JykZwSdQ1pwSuzP2fGZSXr3jYMmvEcwVgNAy3dWr_U/present)

(Depending on what your goals are, and if you're not looking to publish results at build.fhir.org but only to get automated builds and status information, there are some much simpler things that you could do using GitLab actions. Ping me on zulip if you want to discuss!)

GitLab sends via text/plain