Battleplan Rust
A prototype tool for organizing the activity of The Rust Project.
Rust is a huge effort. There's a lot going on. But it's difficult for the casual observer - or even those deeply involved - to grasp the big picture of where Rust is going, and how it will get there. This tool is an attempt to create clarity by collectiing key information already produced by Rust's development process, and presenting it in an easily-digestable format.
Goals
- Give users and contributors a picture of what's happening in Rust and where it's going.
- Illuminate current project initiatives and the RFC feature pipeline.
- Inspire users and contributors with a common vision for the future.
- Progressive detail - on first glance one should sense the scope of the project, and the key deliverables for the coming releases across all corners of the project. Drilling down, both users and contributors should be able to discover detailed information about the state of the project's goals.
- Link all relevant issues, RFCs, pull reuests, etc. associated with any particular feature or initiative.
- Provide entry points for contributors.
- Foreshadow future initiatives and features.
- Demonstrate progress towards the project's goals.
- Visualize the finished and unfinished sub-tasks for any particular initiative.
- Show what's in the next release.
- Living updates - everybody can help maintain it.
How?
Fundamentally, this tool helps organize the tracking issues already produced by the Rust development process. Tracking issues of strategic importance are registered with the tool as goals, and they are stated as concrete, limited deliverables. The tool automatically derives important data about the progress of each goal and presents it to the user in a way that is easier to understand than attempting to discover the same information through the usual GitHub, etc. UI.
There are three major components to this:
- A data schema defined by yaml files in
_data, maintained by humans, and in_data/gen, generated by machines. - A static webapp that presents that data.
- A Rust application that keeps the data up to date by scraping sources, and guiding the user to improve the data set.
It is intended that manual curation is kept to a minimum, but achieving that will be an ongoing matter of discovery, tool improvement, and process improvement.
Process implications
Goals are categorized into themes, which represent the overarching direction of the project. These are the subjects that we emphasize most frequently in our PR/marketing/outreach. There are relatively few of them (5-7), and they change infrequently. They tie in to the long 'narrative arc' of Rust, and can be decided every 18-24 months. We'll need a process for that takes a global, long-term view of the project, and allows for participation accross the entire project.
Goals are short-term deliverables that should take only a few cycles to complete. They can established by teams as part of a periodic planning process that does not need to involve the entire project. Goals that go on the roadmap must fit into a theme that serves the project's long-term goals. This potentially gives us an somewhat objective measure of features that are in-scope for considiration during any time period - and a more objective rationale for saying 'no', steering our precious resources in the most productive direction.
The quality of information we can present is only as good as the quality of data generated by the Rust development process. In some places - most notably RFC-driven features - that data is quite complete already; in others we'll need to improve. But hopefully making the changes necessary is mostly a matter of being more consistent in our process, and the tool can be designed to help discover places where the data is poor.
From what I've seen so far I think these are the places we need to improve in order to make the right data available:
Tracking issues should be consistently tagged as such. Right now
we have B-RFC-approved and B-unstable that can serve this purpose,
but they are only used for things that go through the RFC process
or are otherwise tagged with stability attributes (i.e. things that
go through the 'feature pipeline'). We need to similiarly tag
other key tracking issues. We can get by without proper data for this,
but then the task of putting 'goals' into the tool is dependent
on somebody knowing a priori what they are - the tool can't help
'discover' goals from the repo.
All tracking issues should be assigned a T- team tag, and preferably assigned to a person so we can list a 'point of contact'.
All tracking issues that have an associated RFC should mention the RFC number or link in the OP. We're pretty good about that already.
Tracking issues for tasks with multiple steps should have them tasked out in the OP, and we should lean toward having more subtasks because that lets us give better progress estimates. We sometimes do this now, but not consistently, and they are often not up to date. This is a fair bit of maintenance, but there are things we can do to make it easier.
The hardes thing to do is to start estimating the release in which a goal will be completed. We have no system for this at all right now, but one could easily imagine using milestones for it. Obviously estimates are hard, and it will take work to keep up to date, but this is one of the highest-impact data points we can provide for our users and we should try.
Finally, maintaining the roadmap presented by this tool itself will take effort. As part of the process for establishing the roadmap, teams might be strongly encouraged to submit their plans directly to the tool repo, or at least write up descriptions suitable for integration into the tool. The tool must be designed in a way that encourages contribution, and somebody must periodically refresh the data. This is unlikely to become a completely-automatable process since the data will never be perfectly reliable.
Additional concepts
The tool is intended to provide 'progressive detail' in a few ways. First, it divides both themes and goals into 'top' and 'bottom'. The top goals are displayed by default, but the bottom goals must be toggled on. This allows us to track important work being done that isn't part of our primary 'narrative', without overwhelming on first glance.
Second, both concepts support 'details'. These are not displayed in e.g. the overview, or the list view of all goals, but are displayed when drilling down to specific goals. Details can also be turned on globally with a toggle.
The tool includes a third, less important concept - 'problems'. These are areas of improvement that are on our radar, and we want to see solutions to, but have no concrete plans for. This fills a significant gap in our tracking and planning right now, where some important subjects that we talk about a lot informally don't have a clear place for tracking. Once the solution for any given problem becomes reasonably clear, they can be converted to concrete goals. Examples of these types of topics might be 'bundling our own linker', 'specification', 'HKT'.