Responsible module maintainership

by Larry Garfield

Recently, there was a minor flap about a security hole found in one of the modules that's part of the Drupal install running whitehouse.gov. The actual issue itself was rather uninteresting (if you give someone an "administer X" permission they can do evil things to your site; um, duh), and the only reason it attracted attention at all was sensationalist reporting by some online news sources, but it did bring up one interesting and important detail that many Drupal folks forget: You really shouldn't be using pre-release modules in production.

Of course, that becomes a problem given the number of Drupal modules that are in a pre-release state forever. Which is, of course, the underlying problem: Too many module maintainers are not being responsible maintainers.

Part of the problem with the whitehouse.gov site is that it was using the Context module, which at the time was standing at RC3 state. That's "release candidate, 3rd version", meaning "almost done but not really released yet". Drupal's security team, as a policy, doesn't handle security issues for modules until they have an actual stable release. That meant when someone noticed the issue and reported it to the security team, they did what they said they would do: It's a minor issue in a pre-release version of a module, let's handle it in the issue queue like anything else.

What's that? You didn't realize that the security team doesn't deal with pre-release modules? That's right, if you are using a pre-release module the security team won't deal with it. They're already over-worked enough as it is with the thousands of modules with stable releases. Security issues in such modules get directed to the issue queue and are not handled through the usual security announcement process. So it seems like the safe thing to do is to just not use modules that don't have a stable release, right, at least not for high-end sites?

Well, that would be the natural response, and it would be the responsible response. Unfortunately, Context is far from the only popular module to not have a stable release yet. In fact, having an RC puts it ahead of a lot of modules. Nearly every site that we've built with Drupal 6 has includes at least one module that is, technically, in pre-release state, even though we know it's perfectly stable and usable. I'd wager that if you looked at the modules page of any Drupal site built in the past year you'll probably also find at least one module still listed as pre-release despite it being release-quality in practice.

Beta, by definition, means "this is feature complete but may still have bugs, let's find 'em." "Release candidate", similarly, means "I think this is done and ready to release, but let's do a final check first". That's what Drupal core uses, as does the rest of the software industry (except Google, who uses "beta" as an excuse to not provide support). Yet how many Drupal modules violate those definitions or ignore them entirely, endangering their users? Sadly quite a few.

There are modules that have had ten "beta" releases with no significant new development in over a year, yet are so essential that they've been installed on hundreds of thousands of sites and have even been integrated into Drupal 7 core. Even though modules like these are clearly production-worthy, they get no security monitoring from the community because their maintainers haven't bothered to call any of the releases "stable".

On the other side of the coin, there's another module that was doing serious feature additions for most of the Drupal 6 life-cycle despite being in an RC state for most of it. That's right, active feature additions in an RC. None of which were getting any attention from the security team despite it being a module that leverages Drupal's security system. And it's not even the only one that did so.

There's even one rather high-profile module that hasn't had a stable release in over a year, despite continuous active development. That module's maintainer recently told me quite directly "Hey, my -dev releases are pretty stable. I'm not going to tag another release until I get around to porting to Drupal 7."

There are many more, too. All of these modules are, by definition, self-declared as not ready for production. People are ignoring the risks of pre-release modules because they have to. The "stable" modules don't provide that functionality because it's already provided. That makes using would-be "unstable" modules mandatory to get anything significant done with Drupal. How is keeping your production-ready code from getting proper security monitoring and training people to ignore the very good warnings about using pre-release code being a responsible maintainer? It isn't.

It's time for us to stop being irresponsible.

I'll own up to one such module myself, Views Cycle, which is currently at beta4. No, it's not "done". There will be more work done to it, both bug fixes and new feature additions. There may even be API changes in the future. There are systems in place for handling that, which people know to expect. But it is in a state where people can use it, and over 1500 people are using it. And treating them as cheap beta testers without security attention is irresponsible. So, consider this a pledge that the next version of Views Cycle will be a 1.0 stable release, and then continued development from there will be released 1.1, 1.2, and so forth, as it's supposed to be.

If you maintain a module that is still in pre-release state, and people are actually using it in production, do the responsible thing. Follow a standard, accepted release system. Follow the only release system that the security team supports. Don't try to hide behind "well Google does it". If you're going to release code to the community, support it responsibly.

This is our community, and our code. We should be responsible members of the community.

Comments

and most of the story was how poorly it was handled by the self-proclaimed security "expert" that reported the

I don't think that was "most of the story." It's potentially damaging to our relationship with that expert to make claims like that - it's worth ammending the article, IMO.

I guess Dave Reid is a Palantiri on the Security Team - you could get his opinion on the topic.

I really do wish that saying module owners should be more responsible, however wishing that to be the case simply won't get it done. Is there a better way to enforce some sort of release policy / standard?

The beta dance is a silly game of not taking responsibility, and it makes the user, not the maintainer, responsible for grokking the quality of the module.

Maintainers should state their intentions and beliefs about the state of the module, not hide behind "beta".

And maintainers should strive to get a stable release out there.

Thanks for the link, Dries! I knew you'd written something on this subject before. I think the money quote is:

Rather than forcing people to do something, we need a culture that uses peer pressure to encourage responsible maintainership.

I agree entirely. But we also need to better define "responsible maintainership".

All too often, "beta" means "might be production-ready, but I had more in mind for this module -- even though I will never have the time to actually get those features in."

That's a trend in many open source projects, I think, spurred on by Google and the like. I am not sure where the idea came from that 1.0 is the last release of a project. It's not. If it's a successful project, it's the first of many.

A 1.0 release should be reasonably stable and complete enough that it works, even if it doesn't yet make your coffee. It should have a reasonably stable API. But there's no reason it has to have every feature imaginable. Those can come later in 1.1, 1.2, and if there's a lot of new features even a 2.0.

It seems like we need another set of standards in our community.

We have a fantastic coding standard.
There's been work on a CSS and HTML standard.
There's been talk, at least, of an ethics/behavioral/community standard.

There isn't really a maintainership standard. Maybe there should be.

The problem is that proper module maintenance is a lot of work. Sometimes it's an unbelievable amount of work. It's easy to burn people out. Some people just want to give back to the community on their schedule. I respect that, though I also try not to use those modules.

We might want to watch out with this. I'd rather have a module that has one beta release too many, then a module that should really be in dev but has a 1.0 release already. Dev, alpha, beta, rc and 1.0 are very subjective terms on d.o modules and they vary from maintainer to maintainer. I am always very careful with my modules and I rather put in "just this last rc" before releasing a 1.0 version.

Just a note on the security team policy. While issues with beta/rc modules are handled in the public queue and don't get an SA, that doesn't preclude the maintainer from tagging the next release as a security release so that update status will warn site maintainers. I'm not sure how many actually do that but it's an option.

There's also an issue of maintainers who immediately release 1.0 versions of brand new modules with security holes in, which while they might only have 50 users still go through the SA process, so like Wim says it does go both ways a bit.

Personally, of the very few contrib modules i've posted on Drupal.org, none have a stable release, but I do make a point of putting huge disclaimers on the project pages telling people not to use them ;)

Agree, Agree, Agree.
BUT
Calling people names like "irresponsible" is not the solution IMO. People are limited by their time commitments, and we need to respect that. As MoC says, people need to be able to be creative and contribute on their schedule. The more we make it a command and control relationship, the more we kill the creativity.

Getting everyone with an RC3 module to move it to 1.0 tomorrow with no other changes would have the following effects:

- Security team email eats up all the remaining disk space on the mail server.
- Many maintainers with security holes get made to feel like jerks and don't want to contribute.
- Users get all pissed off about bugs.

The problem here sounds like a problem of curation. d.o. could do more to indicate which modules are stable, how often they get releases, what kind of security coverage they have, etc. It would be nice to more clearly delineate the differences between different release states and describe them on the release form, perhaps ask maintainers to check them off a list when they make a release.

There is a one-way negative communication channel only. Thousands of irresponsible users stand against the claims of this post. drupal.org hardens the conflict.

You do alpha, beta, RC releases, you try to manage a product. Various issues get reported and fixed. Product management requires quality feedback. Especially positive feedback, or otherwise you are irresponsible, and guilty for releasing broken crap that works for 0.001% of all users only. Especially in a modular system like Drupal.

How often did you report back to maintainers that their pre-release code works on your site(s)?

In the unlikely case that you ever did, which non-existing, impossible way did you choose to communicate?

In the very likely case that you never ever did, how much worth is your single feedback anyway?

We should rather ask ourselves why we try to use and adopt product cycles without providing the essentially required environment and tools for user testing and reporting feedback.

Realistically, the tools we provide only allow for alpha and "stable" releases. The latter can be anything from hella broken to might work for you.

Assigning blame is too easy.

It is you who is responsible.

I agree with merlinofchaos, we need a standard on this, a simple document to instruct new and old maintainers.

I became a maintainer somewhere around 2007, and don't remember reading any instructions on what a responsible maintainer should be/do. Learned most of it through community involvement (just "being there")

I know that most maintainers do this on their own free time (they are not paid to do it), which means that they sometimes don't have the time needed to make a stable release, which is a whole new responsibility.

Perhaps having more co-maintainers would help, but I've seen (hopefully, rare) cases where people didn't want to share their "baby", or just admit that they need help.

So, it all comes down to raising awareness of the issue, and this is a good start.

"... and most of the story was how poorly it was handled by the self-proclaimed security "expert" that reported the bug ..."

I was a bit put off of this article thanks to the statement shown above. Sounds like an attack on someone who is merely trying to help. This type of wording with no context is what makes module maintainers and would-be module developers question why they would want to be a part of this type of "community".

Actually the issue-reporter in question has since stated publicly that he didn't handle the issue well and vowed to do better in the future. That's a good outcome there.

There was also a bit of flame-bait from The Register, but then it's well-know for sensationalist stories about whatever the subject is they're writing about. I didn't want to get into that, either. The entire matter was mostly making a mountain out of a mole hill.

I assume you are referring to http://www.madirish.net/justin/mea_culpa

That blog post refers to a totally different issue. It also has some general statements, but those are about some of the dozens of issues Justin has reported over the past few years.

In the case of the Context module and as it relates to the Drupal Security Team, Justin did exactly the right thing.

You really should edit your blog post and retract your claims about the security issue.

Are users ready to pay for responsibility ? Read GPL, the software provided "as is without warranty of any kind blah blah".
It's free/opensource software. And if users don't help with testing alphas and betas, software stays beta forever.
Most users dont even bother with testing, they suppose stable releases come out of nowhere like a freaking magic. Sure, maybe 10% of good, quality software that is sponsored by clients is released stable from the start, but for other 90% there is no such guarantee. That's why calling people "irresponsible" blah blah won't help - most software is guaranteed to have bugs, including critical ones, including security ones, unless there's some big company behind supporting it.
And for anyone who is depending on Drupal security team to keep his site safe - you are naive if you think you don't need to do the same job yourself. Sure, professional audit helps, but it's always a site admin/developer who is reponsible for everything that happens with the site.
Calling people "irresponsible" will only piss developers off - they already do us a favor by releasing the software for free. What can you expect from a hobbyst project ?

Sure, all software has bugs. And just because a big company is behind it doesn't mean there aren't any. Megacorp software has security holes and bugs all the time. That's why Microsoft and Apple have regularly scheduled update cycles; they have a lot of bugs to fix, even years after a program ships.

As far as a hobbyist, project, Drupal isn't a hobby project anymore. It's an enterprise-class system, just like Linux or Apache or other top-tier open source projects. Some contrib projects may be side projects or hobbies for some, while others are the maintainer's full or mostly-fully time job. That does create an interesting and sometimes complex dynamic.

No, module maintainers don't "owe" people support. On the other hand, as others in this thread have noted if you put a module out for people to use you are in effect endorsing it. There is some level of responsibility that comes with that, even if it's ill-defined at present.

It should perhaps get better defined.

I spotted that many developers (won't call names here) tend to release a beta when software barely has an alpha quality - but I won't blame them - sometimes it's the only way to get your software tested. "Release early, release often"..

Larry,

Sun's comment above pretty well covers my general thoughts here, and I'm not sure leading with the Context module, which I co-maintain, is the right way to make the point you seem to be getting at. In my view the Context module is an example of things working great, let me explain...

Context 2.x was in beta so long because we, the original authors, needed to make architectural and API changes to the module so that it would meet our needs. Given the size of module's user base, this meant incrementing the major version. After we had the 3.x series in reasonable shape we realized that not every one would want to migrate immediately and that the 3.x branch needed more work. As we focused, and as we continue to focus, on the 3.x series there were still many users on 2.x. So we posted a call for a 2.x branch co-maintainer, and had a supremely capable person step up, dive into the 2.x issue queue, and who has now brought it to a stable release.

As best I can see, this is how Open Source is supposed to work.

I have a question about my module maintainership problem.

The company where I work sometimes gets some really custom requests from the client. To fulfill these requests, I often ends up developing a module. The client usually agrees to publish the module.

What to do with these modules?
I don't have the time to maintain properly these modules. However I don't want to lose the module ownership, because if the client comes back, and asks for a new feature, or reports bug, then I should also put the further development into the open source version.

What I did is that I released a dev tarball with a notice, that it is used in one a site without any known problems, but I need community feedback to mark it stable. Moreover I set the "seeking for co-maintainer" status.

One example of the silly errors, what I encountered, and I would never discovered without the community: sometimes I forget the curly braces in SQL queries, because NetBeans autocompletes the SQL queries, but it can't understand the curly braces. That is why I omit them first, and write later, what I seldom forget. Since we almost never use table prefixing neither on our dev/testing sites nor our deployed sites, this bug haven't affected us.

Sometimes the users request more functionality or detailed documentation about the module. What is the best practice in this situation? I try to free some hours to develop my modules, but sometimes I can't. Because of that it can happen that I forget to answer, and the issue is left open for months (while I can fix it in 2 hours).

I'm guilty of dragging out the D6 version of Advanced Profile Kit. I've waffled about what it was going to do for a very long time but it's fairly stable, now, and not likely to change much. Still I hesitate to move off beta. A lot of it is the Google mindset, I admit. As long as it's beta, I can say "use at your own risk" and offer support as I have time. If I put out a full release then it feels more official and I feel like I'm obligated to support it more.

Frankly, it's a PITA module to support because it glues together so many modules and requires at least a working knowledge of those modules to use. Maybe what would help here is a guide about what sort of support maintainers are expected to give for modules. How much responsibility do we have? Fixing security issues and critical bugs seems like a no brainer but what about more trivial bugs? What about module incompatibilities? What about when someone basically has no clue and wants you to talk them through everything?

Better labeling of modules would help as well. I'd like to slap a label on APK that says I'll support it as far as fixing bugs but offer minimal support for anything else. Sure I can put that in the project page but I'll still get people coming back saying it's my responsibility to hand hold them since I put the module up for download. With no clear guide of what, exactly, is expected of maintainers, it's hard to argue against that. If I point out that I'm not getting paid for this and help when I can, I get called nasty names. It's very frustrating and is just much easier to leave it as beta and say it's "as is".

Michelle

How often did you report back to maintainers that their pre-release code works on your site(s)?

In the unlikely case that you ever did, which non-existing, impossible way did you choose to communicate?

In the very likely case that you never ever did, how much worth is your single feedback anyway?

We should rather ask ourselves why we try to use and adopt product cycles without providing the essentially required environment and tools for user testing and reporting feedback.

How does one report back to a maintainer that the module is working well? Create an issue? "Hi, I'm cluttering up the issue queue here just to say I have no issues. okthanksbye."

Our tools on drupal.org fail us here. All of us, because everyone, including the n00b with no expertise, can contribute positive feedback when things are working swimmingly. The ratings and such planned for the drupal.org redesign can help, but I wonder if we can't find a possibly better way to provide that feedback. (Crazy idea: Perhaps an extension to update that allows site admins to "recommend" a module that's working well, with those stats available to the maintainers.) It would be nice to have a tool designed to provide positive feedback for more than just a patch needing testing.

We actually were kicking that question around a bit today at the office after seeing the amazing response to this posting, and one of the ideas that came up was very similar. Right now the only positive feedback maintainers get is seeing the number of sites running their module, which is often times more scary than useful. :-) Having some way to know that a beta release is ready for prime time would be great feedback.

So your "crazy idea" may be crazy but it just may work. :-)

Another crazy idea we had was the ability to rate not modules but module releases on d.o. So if a given beta or RC or whatnot is getting lots of +1s on it, I know that it's popular and doing pretty well, so it should be safe to make a "real" release out of it.

Consider the usage stats that all projects on D.o have. I think that's one way that we can count the number of people who are silently satisfied with a module; otherwise, they'd uninstall it and it wouldn't be counted anymore and/or file an issue (hopefully). If a module's usage count is in the four digits or more but the number of open issues in the queue are in the lower two digits or less, it's probably safe to assume that it's a pretty solid module, at least from the perspective of average site administrators.

Several people have pointed out that maintainership can be a non-trivial task, but we don't warn people of that before they submit a module. In fact, there is no real documentation anywhere about what it means to be a "responsible maintainer", aside from scattered blog posts like this one or Dries' post linked above.

So, let's start making one. Who wants to start the handbook page for it?