Let's try a monorepo (or two)!

[Stephen Gutekanst]

Hi all! I've noticed over a long period of time that the excessive amount of repositories that Azul3D has is quite confusing for new users. Additionally, our versioning scheme does not match other Go packages and has been a huge source of confusion for users as well.

How it's caused us trouble

• Feature request from user for wireframes (already fixed in latest, but not released, version of azul3d.org/gfx.v2).
• Confusion from a long-term user of Azul3D (since we used Google Code, even!) about the repository situation.
• Manual patch (instead of PR) from user for an issue that is already fixed in unreleased version of gfx.v2 because he couldn't find the right repository!
• And last but not least, long-awaited promises to @nwidger for nintengo about the new V2 of the gfx API which we couldn't release yet. (no citation, mostly over Slack)

We've also been dealing with this issue in various ways for a long time:

• azul3d/issues#33 Too many tiny repositories.
• azul3d/issues#34 Move legacy repositories to a separate organization.
• azul3d/issues#28 [trouble] tracking all issues

And also not listed above, is the high development cost on me personally to ensure that we are keeping repositories in-sync with one another. Changes to azul3d.org/gfx.v2 require changes in a separate repo at azul3d.org/keyboard.v2, we must use govers etc. to update version numbers, and it's all quite confusing for newcomers.

What the future looks like

• One github.com/azul3d/engine repository, and one github.com/azul3d/examples repository.
• go get -u azul3d.org/engine
• go get -u azul3d.org/examples/...
• No more package versions.
  • The official way to do this by the Go team is to use vendoring. I highly approve of this approach, and it means users can get new fixes and features when they want them (rather than relying on me to release them first).
  • We will NOT try to avoid API-breaking changes. We will document these, and maybe even continue to use semver outside of our import paths, but otherwise not.
  • This means PRs that are merged will instantly be usable by users, instead of being delayed for long periods of time until the next release (sometimes months).
• One issue tracker for all repositories.
  • I view this as a huge positive, but others might not like the noise. "Closed as unfortunate." :)

Potential Downsides:

• The repository is large once merging all commits, 80MB+. This is because we were naive and committed binary files like wav files, textures, etc.
  • To fix this, we will switch to proper storage for these binary files, in specific https://git-lfs.github.com/
  • This does mean we will lose all old commit history (or would otherwise have to rewrite all of it, too much work / not worth it).
  • Users will now have to install git-lfs in order to: run the examples, tests, install native/al or native/freetype packages. Other packages are not affected. git-lfs is useful for end-user projects, too, for storing resources in git like textures, etc.
• One issue tracker for all repositories.
  • I actually view this as a positive, but YMMV.

Other notes:

Installing git-lfs is actually very useful outside of Azul3D, too. If you're doing gamedev, you will not want to commit your assets to your repository in order to avoid the size growing outrageously each time you modify an asset. And it's very easy to install:

Linux:

curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
sudo apt-get install git-lfs
git lfs install

OS X (requires brew install):

brew install git-lfs
git lfs install

Windows (requires choco install):

choco install git-lfs.install
git lfs install

Current Status

1. The plan is in action! We're moving forward!
2. The monorepo at https://github.com/azul3d/engine is finalized.
3. The monorepo at https://github.com/azul3d/examples is finalized (old repo at https://github.com/azul3d-legacy/examples).
4. Old repos have been migrated to https://github.com/azul3d-legacy.
5. Old repo's issues have been migrated to https://github.com/azul3d/engine/issues.
6. The website has been updated to remove all versioning-related content, plus added a new blog post describing what has happened.

[Dmitri Shuralyov]

👏 👍

[Robin]

+1 indeed.

This is definitely moving in the right direction, and while we don't know yet of all the hassles it will create, it certainly solves a number of long-standing issues with the old repo management.

Specifically, I approve of dropping the use of versioned import paths. They have caused much more pain than that created by the potential problems they may have been set out to solve.

I'll dig in a bit deeper in the next month or so, when I start to get more time on my hands.

Until then, best of luck and thanks for keeping us up to date :)

Cheers

• u

[Robin]

Another potential downside to the merge of all repos into a single repository is that it decreases the granularity with which permissions may be distributed to developers and contributors of Azul3d.

In the previous setup, my brother and I had read/write access to the audio repositories of the Azul3d GitHub organization. Now, I can no longer close audio related issues, which creates an unnecessary administrative overhead that I have to ping @slimsag each time (e.g. #79 (comment), #76 (comment) and #71 (comment)).

Any thoughts on how we may resolve this without giving contributors to Azul3d full access to the entire project?

[Stephen Gutekanst]

@mewmew Good point. I am not too concerned about contributors doing anything crazy, so I've gone ahead and given all prior contributors access to the engine repository. I think we are all good/responsible OSS citizens, so we should be fine.

[Stephen Gutekanst]

Closing for now, if anyone runs into new issues regarding this please feel free to create a new one :)