[RFC]: protocol for creating and maintaining repositories
joeyAghion opened this issue · comments
No clear proposal, but an interesting question surfaced about what protocol, if any, to have about creating new repositories. Some broad options we have include:
Keep it chill
No protocol whatsoever.
Pros: Move quickly. Many things won't get adopted broadly anyway, and the ones that do will evolve organically toward familiar technology and greater support from the team.
Cons: Divergent or duplicative solutions can emerge from different developers or teams. There's little review of project goals or tech choices, which can become problematic when more critical systems depend on a repo or the authors move on. Without a clearly associated team, maintenance or even security patches don't happen reliably.
Lightweight
Aim to document the purpose, choices, questions, and risks in a tech-plan or plan-like document (maybe even just the repo's README).
Pros: Sharing the plan at least in big stand-up (like other tech plans) creates a chance for input, ideation, buy-in, and visibility from the wider team.
Cons: Overhead. In some cases when the "right" thing to do is to extract and share functionality, the tech plan is an obstacle and the "easier" thing may be to just duplicate or diverge (debt) instead.
Heavyweight
All projects demand a plan, review, and buy-in. (Not especially fond of this one, but mentioning it for completeness.)
Pros: Little risk of duplication or divergence. All repos have a clearly associated team a criticality level, and buy-in from the wider team.
Cons: Exploration and experiments are burdensome. As a result, the team misses opportunities for technology improvements that would otherwise end up being compelling wins.
Dimensions we could use to tailor any approach:
- Whether a repository is hosted on the
artsyGithub org or not. E.g., developers freely hack on libraries or projects under their personal namespaces, but others aren't expected to maintain them. If/when a certain level of adoption is reached (say, a critical system depends on it or more than 1 non-critical system), it may merit "upgrading" to theartsyorg where others should contribute maintenance. This includes "adopting" whatever the repo's tech choices or debt may include. - Whether a system rises to any particular criticality level. E.g., level "0" (unsupported) systems don't require any technical plan or review, but also don't get support. Systems depending on support (level "1" and up) require at least an adopted technical plan covering the motivation, goals, and risks of a project.
- Whether a repository makes any atypical technology choices as per our "radar." E.g., maybe repositories that stick with well-supported technology from the "adopt" quadrant are unhindered, but any "trials" or certainly any uses of "on hold" tech would require buy-in.
Additional Context:
I'm +1 to the lightweight approach.
"Keep it chill" is nice in theory, but in practice it would discourage engineers who haven't done this kind of work before. A lightweight approach gives enough guidance to make engineers feel confident that they're not messing anything up, with enough freedom to let them use their best judgement 👍
We also already have a de facto lightweight approach. Things like having point-people named in repos, the Artsy OSS footer that @jonallured keeps up-to-date, etc. We have this approach but it's not really documented, so reifying what we do already seems like a good first step.
+1 to the lightweight approach, I agree with @ashfurrow that giving guidance also aids confidence. Regarding readme, we have some expectations spelled out in the production-ready checklists's readme section: https://www.notion.so/artsy/Template-Production-Ready-Checklist-5aca3cea5c05488baf426c5ed1bf9178
Side-note re the "keep it chill approach", which is kinda in place right now -- what artsy/<repos> are currently available that aren't being used at the moment, or are in development? How do we track such things? It's a burden to not be sure if one can clean up or not. We should at least audit and assign point people if not already set.
Maybe we can list all of them in a notion page and assign point people that can also put a status, like active, or not needed anymore, or stalled or whatever?
Many are included in the project list, but, since there's little process in place now, new repos (especially libraries) don't necessarily get added or adopted by a team.
I'd like to shift this conversation a bit if folks don't mind. It'd be nice if we could outline and align on the problems we'd like to solve before deciding on a solution.
Some examples pulled from the conversation/context above:
- It's not always clear if a repo is still used so it's hard to make decisions about what to clean up
- It's not always clear who owns a repo so it may not be getting the security updates or maintenance it needs, especially if the creator leaves
- The process is mostly implicit which may not give folks the confidence they need to create repos
What are some other problems and what would we like to see solved?
What are some other problems and what would we like to see solved?
Unclear which repos are WIP. Process around visibility of in-progress work and / or transfer of ownership from personal to engineering org.
I'd add that, if there's no coordination or communication around the start of projects:
- Divergent or duplicative solutions can emerge from different developers or teams
- There can be insufficient consideration of early tech choices
As a potential step forward to resolving this RFC, I propose these action items:
- Add peril automation to announce all new repo creations (and maybe deletions?) in #dev to raise visibilty of activity in the org
- Add peril automation to create a repo creation checklist. Much like our private repo automation, this can help folks who are creating repos understand what the next steps are.
- Augment our existing documentation with best practices if they're not already listed. This includes things like having point people, referring to the technical choices doc for choosing the right tech or gaining alignment, etc. This document can be linked to in the initial message created by peril.
- Eventually, (if necessary) we can add automated checks for compliance. Are there repos that aren't in the projects list? Are there repos that haven't been touched in over a year? Are there repos that have only ever been touched by one person? Whatever things we want to flag or raise visibility of, we can figure out some threshold for that. The compliance isn't MVP scope though.
To summarize the immediate steps, we automate raising visibility and providing resources and then trust people to do the right thing. If we have clear deviations or compliance issues, then we apply stricter checks until we're sufficiently satisfied with the state of things.
Thoughts on this proposal and next steps?
I think automation should only follow a manual process that's been adequately exercised and "works." E.g., if there's agreement about adding all repos to the list (with associated point people and teams), we should try to do that first. Similarly, we could try to document the new-repo-checklist.
Since there isn't obvious agreement about adopting more process and we already have so many recommendations in this space, I suggest we err on the side of autonomy rather than risk excessive bureaucracy.
To encourage anyone who'd like to start new repos going forward, I can add an explicit suggestion in the tech choices FAQ summarizing how the general recommendations apply:
- Make sure the project README provides necessary context as per the documentation playbook
- Mention [and invite feedback on] any new repositories in open stand-up as per its agenda
- Add any new projects to the project list 🔒 and specify which team will support.
- For any technology choices not "adopted" as per the radar, tech plans and reviews are highly recommended.
I'll resolve this next week depending on feedback.