Monday 5 March 2012

New development and contribution model for illumos-userland

We had a great discussion at FOSDEM about where we wanted to take the development and contribution model for userland, and that was followed up by direct conversations with Alasdair, OI's fearless leader, and Andy Stormont. We also took feedback from recent group discussions on #oi-dev and userland@, and we hope you'll be as excited about these changes as we are.

What we're rolling out is essentially a variation on scrum. We chose this for a couple of reasons, mostly to do with resource and priority management.

Under the new model, development is timeboxed around fixed-period dev cycles. At the beginning of each cycle, we pick a list of priorities (e.g. sync'ing with userland, bringing in a new default compiler). We then have a three week "walk" where contributions are worked on in a shared WIPing post repo. Packages would potentially be available for anything that builds
successfully, and things like compiler changes would need to be available at the outset of the cycle.

At the end of the three week walk, there's a one-week sprint. The sprint starts with a sanity-check of the priorities against the work that's been done (i.e. to make sure that anything that's clearly overreaching is set aside for a later cycle). Then all the components are polished and prepared, reduced to minimum requisite change and rationalized as changesets, and lined up for commit. At the end of the sprint, there's a review and commit cycle that looks at the changes as a block.

We've asked Andy to take on the role of userland scrum master, responsible for negotiating deliverables for each cycle, clearing blocker issues, and shepherding changes through the review process.

I'll be stepping in as development manager for OI generally. As such I'll be on the other end of the negotiations. Andy's job is to advocate for userland and its deliverables, whereas my job is to think about getting releases out the door and to provide an alternate perspective on resources and priorities. Andy works for Nexenta, who employs all of our paid contributors, and has unparalleled experience in developing FOSS distributions around illumos and its successor projects via StormOS, NCP, and illumian, our sibling distro and userland partner.

Supporting both of us as project manager will be Lou Picciano, whom most of you know as DrLou. His role is to make sure that we don't lose track of our priority deliverables or completely forget about the things we said we would defer.

So, those are the people, but what we really care about is getting the community working together more effectively to deliver product. We have a mix of full-time and part-time developers, so we wanted a model that could work for both. Having the ability to drop code that's still a WIP and pick it up from a teammate is a model designed to accommodate people who are juggling other obligations and want to contribute in time that they can't organize around userland.

Given the more collective approach to development, one area where we'd like to hear back is in terms of how to attribute changes and recognize work. We want people to feel a clear sense of accomplishment as contributors, and we don't think that a team model should diminish that. If you have thoughts on this, please provide comments or trackback responses on your own blog.

Under the new model, final reviews and commits are handled by the dev manager and the scrum master. Nothing will be committed from the WIP to the dev repo unless it can build and test cleanly. Changes that may be handed off between people without being able to clean or test will be reduced to a single changeset before by the time they are migrated between repos.

We want to have peer review throughout the process, so the scrum master will also have the ability to set up peer review arrangements between cycle participants to make sure that quality is maintained and that changes are clearly conceived, implemented, and documented. The best measure of this is whether peers with a reasonable background in the change area agree that this is the case.

Dev cycles needn't target just porting and packaging: tools, test suites, and CI in particular seem fair game, as they can enhance productivity. Part of the scrum master role is to identify these issues and advocate their priority.

I've mentioned testing a few times, so I'll underscore this before concluding: quality is a core value of the OI project. We're solidifying this value by making sure that we're running available test suites, tracking known issues, and working with the upstream to fix test suite issues or clean up any issues with the port. We hope to engage the wider OI community as this model matures to broaden our definition of quality and expand it based on the use cases people build around the technology.

Finally, with the concept of a separate WIPing post and dev/canonical repo, we expect that we will no longer be constrained to use mercurial exclusively. We know that there is a strong interest in introducing git, and we expect that the WIPing post is the place to do it. We also expect that this more collaborative dev model will adopt an SCM workflow closer to this much-discussed git model, and Andy will be primarily responsible for fleshing this out.

No comments:

Post a Comment