What should go into a WordPress maintenance release?

Now that WordPress 2.9 has hit the streets there are inevitably going to be a number of bugs which slipped through testing which people will identify and we need a good process for deciding which of these bugs should be included in an upcoming 2.9.1 release and which of them can wait until 3.0 to be fixed.

In the past I think we have included to many changes into the maintenance releases.  This has partly been due to the lack of end-user testing which has taken place during the beta phase and partly based on a drive to fix bugs sooner rather than later.  This has had the unfortunate downside of diverting the time of the core contributors, that they could have been spending on working on features for the next major release, and has meant that the release cycle has been extended to longer than we would have liked.  This means that you don’t get the cool new features like Trash as soon as you would have done.

This post is an effort to summarise my thoughts on this subject and I would welcome discussion in the comments below – hopefully we can come up with a succinct set of criteria which will may it clear to everyone what the community thinks should be included in a maintenance release and what should not.

First of all I think it is important to understand the purpose of a maintenance or point release.  To me these releases are about fixing critical bugs which affect a large percentage of end-users and disrupt their ability to use the software. They need to have a small number of changes so as to make them easy to test, lightweight to develop and to reduce the impact on the next major release. This also means that they will be easy to upgrade too and are less likely to break plugins / themes in the process so making the upgrade to them even easier and more worry free.

If we let the content of these release grow too large then we will never have the time to work on the new features for the next major release which will then mean that release takes longer to arrive and may end up with less features.  We also need to remember that every change has the risk of introducing a new bug and this is another reason we should limit the number of changes in a maintenance release.

Now we need to identify what sort of bugs fit into the category of critical bugs.  The most obvious and easiest to identify are the security bugs – fixes for these need to be available as soon as possible. Once we get beyond security bugs it can get more difficult to quantify the affect of the issue and therefore how soon a fix needs to be in the hands of the users.

The following list of criteria are my straw man proposal on this subject:

  • The issue is easily reproducible and testable.
  • The issue is severe – e.g. it makes a whole feature unusable and therefore the fix cannot wait for the next major release.
  • The issue is a regression from a previous major release – for example if an unintentional/incompatible change to an API or UI was made.
  • The issue is not just that warnings or notices appear when WP_DEBUG is enabled the area of code in question must also fail to work correctly.

What do you think?

57 thoughts on “What should go into a WordPress maintenance release?

  1. First of all, I think it’s very important to set a time frame, such as this:

    A point release is launched:

    - right after a security exploit is found and patched
    - otherwise, after a month from the last release

    As for your proposed categories:

    - “The issue is easily reproducible and testable” AND it has a patch already

    “The issue is severe (entire feature is unusable)” – agreed.

    “The issue is a regression from a previous major release” AND it has a patch that can be tested well

    “The issue is not just that warnings or notices appear when WP_DEBUG is enabled the area of code in question must also fail to work correctly.” – heh, know what you mean by that ;)

    1. We do need to think about time frame for point releases as well.

      It is easy for a security release as the time frame is yesterday is not soon enough as for normal maintenance releases I think it is harder to pick a time frame and that needs more discussion after we have reached a consensus on the content.

  2. My feeling is simple: WordPress should never, ever, ever ship with a known, potentially major bug. If we need to delay shipping by 6 months, so be it. But never, ever, ever with a major bug.

    1. This is the whole point behind the beta and rc phase of the development process – to discover these bugs so we can fix them before release.

      Hopefully the set of criteria we reach agreement on can be applied at this point as well so that everything is consistent.

      1. It is very hard to test every end user configuration and when we come up against PHP bugs which only show in certain server configurations unless a lot of people test on there configuration then bugs are unfortunately going to creep past.

        The best ways to avoid having to write posts like that are:

        1. Encourage people to get involved in testing.
        2. Writing detailed bug reports which show how to reproduce the problem.
        3. Take bugs with 3rd party libraries upstream as well
      2. Once again missing the point, Peter. The bugs Denis referred to were all either reported before release or caused by code changes that were committed after the RC and never released for testing. These are workflow failures that have nothing to do with bug reporting.

      3. The bugs Denis referred to were all either reported before release or caused by code changes that were committed after the RC and never released for testing. These are workflow failures that have nothing to do with bug reporting.

        For the first of these bugs the feedback from the upstream developer on the suggested patch was that is was not recommended.

        For the second of these bugs a fix for the majority of the issues was in the builds from weeks before release and none of the testers noticed the missed patch.

        For the third of these. Yes we made a mistake. We had a patch which looked like it would provide an improvement to the behaviour of wp-cron and deal with issues that had been reported including the fact that cron was completely broken for one of the HTTP transports:

        This prevents spawn_cron() (which sets a timeout of 0.01) from spawning anything if the WP_Http_ExtHTTP transport is used.

        Unfortunately we have subsequently found that there are a number of issues with the curl implementation in php which make the patch not a good idea and the changes to the curl transport will be reverted for 2.9.1

  3. First of all I think this topic needs to be re-stated on several points:

    1. Please differentiate between bugs and patches. Throwing away perfectly good patches is worlds different from setting aside a bug report.

    2. Resources always need to be balanced between bug work and feature work. If you can’t even apply all of the patches that are available for a major version (2.9) then you need to shift resources to a minor version (2.8.7). This is common-sense quality assurance. Fix bugs first, then add features, and do both using branches when feasible.

    3. If there are too many patches to satisfy your standard, “content of these release grow too large”, then you need two minor versions (2.8.8).

    4. Please explain why we are defining “critical bugs” only? Is it your intention to not fix bugs that don’t meet that standard?

    5. Your straw man proposal is unreasonable. It puts the burden of reproducibility on the reporter. As it stands now, the burden of patching is also usually on the reporter. The bar is set far too high.

    6. Requiring a bug to be a regression from a major release is nonsense. If a bug can be identified and fixed, then placing blame is never a factor.

    With that said, I look forward to some constructive ideas.

    1. Firstly, thank you for your detailed feedback.

      With regard to (1) – I am not suggesting we throw away good patches, all I am suggesting is that if the bug that is fixed is not severe enough then those fixes will get included in the next major release rather than then next maintenance release.

      With regard to (2) – If we focus solely on fixing bugs for a major release then people will not upgrade. A large majority of people upgrade because of the new features not because of the number of bugs fixed.

      With regard to (3) – Splitting out the large number of non-urgent patches into multiple maintenance releases doesn’t help. In fact you make the problem worse because you then have to run multiple beta programs which themselves take up a chunk of time.

      With regard to (4) – Yes. My personal belief is that a maintenance release should only be for critical bugs (I see an exploitable security bug as critical).

      With regard to (5) – We need the reporter to provide enough information so that we can reproduce the bug. Without that there is little chance of it being fixed as we can’t test any suggested fix against the problem. The burden of patching is not on the reporter, anyone in the community can and does provide the patch – there ability to do so is driven by this reproducibility.

      With regard to (6) – A regression from a previous major release is a critical bug in my mind. We value backwards compatibility and we try and maintain previous functionality so that the upgrade process is safe and plugin/theme developers can then move over to the recommended new functions / methods.

  4. I think bugs that affect functionality should be fixed first. Bugs that throw out errors that actually affect usage of the software should be the priority.

    A simple example: a bug in backdating posts which affects their being displayed or not displayed at all is a serious bug that affects the functionality of a blog, and it should be fixed a.s.a.p. A bug that causes a key/value to display at the bottom of a post which should not be displayed is a less serious type of bug, and that can wait until later.

    I do like your list of criteria too. It shows you’re thinking of the end user and their ability to use or not use the software as is.

  5. First of all I think most parts of your post are pretty much misleading. You’re doing wrong assumptions and then you follow this up with some “if then that would have been” sentences. Highly problematic IMHO.

    Version numbering in WordPress is mostly cosmetic. I won’t call for round numbers not swear on what a RC or Beta is nor what a full release or a maintenace release. The longer WordPress exists the more it’s code is testing code. That’s living on the razors edge and infact that makes fun with WP.

    Except that it looks like it’s not totally clear to the core developers that WordPress code is mainly prototype code. The other half is legacy code. Now I said it. :)

    You named it in your post: There was lack of testing with the current release and that lead to problems. But why was there lack of testing? Why were there not so much testers? You leave this unanswered.

    It has a long tradition in WordPress development to say: This or that bug can wait until (put in next release number) to be fixed. I’m not shure that this is adviseable forever.

    Mew features will per se create new bugs.

    To reduce the number of bugs we need to patch them out before introducing new features.

    The current codebase does has a output rate of X new bugs per week even if we do not put new features in. We only get fixed X/2 bugs per week, so the number of unfixed bugs grow by X/2 per week.

    As 2.9 introduced new features, additionaly Y bugs per week have been introduced with the release.

    So each week we gain X/2 + Y new bugs unfixed.

    A statistically analysis of trac tickets could gather concrete numbers here. I haven’t done this so I can only assume that currently this might be new 10-20 Bugs per week we leave unfixed in code per week I assume.

    From what I feel, this number is growing not shrinking. That makes sense because the user-base of wordpress is still growing as far as I’m concerned so more users will output more bugs as well.

    So talking about point or maintenance release does not get the root of the problem why actually there aren’t enough testers: There is just too much to test with all these bugs. Unless we do not keep up with fixing bugs properly, this will not help us with the actual problems.

    Your straw man proposal is only lowering the bar based on the appearance of the problem (not enough testers; too long release cycles) instead of figuring out what the root of the problem is: too many bugs. So your proposal will only make the release of a single milestone look better and I doubt that it actually will, it will only look right now theoretically better to certain persons sharing your point of view.

    For example the suggestion that notices can be spread here and there and you give a damn on WP_DEBUG. If I as developer / tester use WP_DEBUG to find problematic areas you make my job actually harder not easier to test code.

    1. Version numbering in WordPress is mostly cosmetic.

      I disagree. We have major version numbers x.y where the x and y have no special meaning they just increment decimally such that when y would reach 10 x is incremented by 1 and y reset to 0. Maintenance releases are then named for the major release they maintain.

      There was lack of testing with the current release and that lead to problems. But why was there lack of testing? Why were there not so much testers? You leave this unanswered.

      I actually said there was lack of testing in previous beta/rc phases not this one. This one seemed to go very well and we got a good amount of feedback and fixed a large number of bugs with the new features during this process.

      For example the suggestion that notices can be spread here and there and you give a damn on WP_DEBUG. If I as developer / tester use WP_DEBUG to find problematic areas you make my job actually harder not easier to test code.

      WP_DEBUG is a very useful feature and I use it all the time. In fact all my developer installs have it turned on. However a bug which causes notices/warning to be shown when WP_DEBUG is enabled is not a severe enough bug, in my opinion, to need fixing in a maintenance release unless it causes an actual behavioural defect in the code. These fixes can be roled into the next major release.

      With regard to the bug level and bug injection rate you can never have bug free software. Therefore you have to accept that there will always be some bugs in released code and try and work on prioritising the resolution of the more severe bugs.

      With regard to the idea that the number of bugs is growing overtime at a faster rate than we fix them. I would disagree on this point – I think what we are seeing is a large number of people reporting issues that they have discovered. In quite a few cases people are finding issues that have existed for a long time and I think this actually shows that WordPress is getting more testing and feedback as the community grows than it was before.

      1. You mean 1000+ open tickets isn’t a sign that we have problems to cope with the bug discovery rate versus the bug close rate?

        Next to weight wich bugs to close first, it should be weighted which changes to NOT introduce close to a new release. I mean, take a look which bugs are fixed right now. I do not see enough testing on the important changes, sorry.

        How about to have one trunk for the current release number (as we have), then havin one fixing trunk that is taking care to fix stuff for all future versions and one trunk that is interchanged with the fix trunk to be there to integrate new features.

        Then there would be one base we actually can take care of bugs, existing users can go with the blocker / security fixes, other users can go with the overall fixes version.

        Then it should be no problem to do a maintenance release and next time a feature release. Right now I see that this mixes a bit because we have these long release cycles and user want to get new features in. As you wrote: never bug free. So when do we actually take the bugs out?

  6. It doesn’t take a lot of time to backport a change into the current branch from trunk, however if a bug is introduced from doing so, then we have to release yet another release.

    On the other hand the time between major releases can be a long time.

    I think it’s worth fixing all bugs a user on a clean install would encounter. This maintains a good reputation for WordPress as keeping the number of bugs that users deal with low.

    Bugs that only plugins would encounter can wait unless they are trivial in their complexity to fix.

    1. Back-porting a change does take extra time as the testing of the change needs to be done twice.

      Part of the reason why there has been a large gap between the last two major releases is the number of changes and amount of developer effort that went into the early 2.8.x maintenance releases.

      1. But that would mean, that 2.8 introduced bugs should have been get out in maintenance relase in 2.9 for the bugs priority, right?

        And in case for 2.9 a decision is made to not fix a thing, does that mean that the bug is to be fixed in 3.1 because 3.0 is for new features only? Or is a feature relase a maintenance release in your meaning as well?

      2. For well understood and reproducible regressions in behaviour the general rule would be – “Fix them at the earliest opportunity.”

        A maintenance release (i.e one numbered like x.y.z) would only provide fixes an not new features.

        A major release (i.e one numbered like x.y) provides new features and bug fixes.

  7. I totally agree that security bugs are the most urgent, and need to be addressed quickly.
    After that, I found two serious regression bugs:
    1. Scheduling doesn’t work: this is quite a basic feature, and seeing “Missed schedule” again and again is quite annoying. I’m not the only one complaining.
    2. Trackbacks and Pingbacks aren’t sent, not internally nor externally. I didn’t see too many people complaining about it yet.

    Those are my two cents (bugs).

    1. Thank you for this feedback Yohay have you created trac tickets for either of these issues?

      They both sound like the wp-cron system is not working correctly on your host so we will need some extra information to investigate them.

      1. Thanks for your quick response. I haven’t opened tickets, just whined in the forums :) I’ll do that soon.
        I thought of switching to another host anyway, so WordPress 2.9 will accelerate the process…

      2. Yes there is already a ticket for this. My first glance at that issue revealed there was no call to set_time_limit() even though the version checking timeout was increased from 3 seconds to 30 seconds. So unless I’m mistaken, this bug only affects servers that fail the version checks.

  8. My suggested general rule of thumb: If a bug is only irritating to users and not preventing them from using WordPress in the way that it was intended, then the fix can wait till the next full release.

    1. Seriously?

      So it’s not a big deal that WordPress is irritating to the user?

      Isn’t WordPress meant to be NOT irritating to the user?

      WordPress boasts such users as The New York Times, CNN.com, Wall Street Journal blogs, Reuters, Rolling Stone and many more that you can find listed right here: http://en.wordpress.com/notable-users/

      Are you seriously saying that you don’t care that your irritating the users, they can wait for fixes for maybe months until a major release just because you find it more fun to work on new cool features that are just going to potentially cause more irritations to users?

      Please don’t get me wrong. I love what you guys are doing. I admire all of you. I admire your capabilities and your dedication to making WordPress the fantastic publishing platform that it is. I just think that you are so far above me as an end user that you don’t get that what may seem like a minor irritation to you can be more like a horrific nightmare to me.

      Having said all of that I would just like to add that you guys are my heroes. There is nothing out there that I have found that even comes close to WordPress even though it can feel like a double edged sword sometimes with releases coming out so often.

      I can’t speak for all users but I personally get a big knot in my gut every time there is a new major release, precisely because of the things that only seem like minor irritations to you.

  9. to the question : What should go into a WordPress maintenance release?

    i would answer : well! bug fixes only …

    then comes the question of bug/fix priority …

    If you want to encourage the community to fix bugs, think about the following workflow in trac

    community open bug => priority review => community fix bug

  10. All of this must be qualified.

    Before developing and coding WP to achieve certain goals, there must be agreement on WHAT those goals are in the first place, e.g., v2.8 is going to do this and that, and while v2.9 is going to do all of what 2.8 already does, but with added features such as X and Y.

    Now when there’s a problem with (a) security or (b) meeting the predetermined goals, then a point release is warranted.

    Releasing the next version while there are still open tickets with previous versions doesn’t make sense to me. Granted, I’m not a coder, but I think it’s like building a house on an unstable foundation. In other words, I think the next version of WP should be put off indefinitely, until all open issues have been addressed.

    Star with a clean slate (close out all tickets), have agreed-upon goals (community polling), and release patches when people have problems using WP in relation to the goals stated.

    1. Granted, I’m not a coder, but I think it’s like building a house on an unstable foundation. In other words, I think the next version of WP should be put off indefinitely, until all open issues have been addressed.

      It would be wonderful if we could fix every single open issue with WordPress before the next major release.

      However there are a number of reasons why this is unlikely to be achievable:

      A percentage of the tickets that exist do not have enough information in them to reproduce the issue or test the supplied patch.

      Every change has the possibility of introducing a new bug – the larger the change the more likely this is.

      Some areas of the code do not have well defined rules for the expected behaviour and do not have good unit test coverage.

      For the software to more forward and be desirable to use it needs to have new features added.

      1. I think you’re leaving out the single most important point: the commit team is so understaffed that it doesn’t have enough time to drill through the tickets in the first place.

      2. Personally, I don’t think that commit team staffing level is necessarily the issue.

        I find that I have to spend a lot more time that should be necessary on reviewing the contents of a patch and testing the fix before it can be committed because people do not leave enough detail in the ticket to make the committers task easy.

        Tickets without detailed reproductive tests and a good explanation of the end user issue are much harder to review and commit than those which do have this information.

        Unfortunately too many people set the commit and or tested flags on tickets without giving a good commentary on what they have tested and why the change is commit worthy so the committers have to spend time on testing the changes in detail themselves.

        Adding extra committers does nothing to resolve this problem and could easily make things worse than they currently are with things getting committed that were not ready or not well enough tested.

        If as a community member anyone wants to help drive the number of bugs down then they should strive to provide detailed relevant information in the tickets they raise, patch, test and comment on.

        That will help accelerate the process significantly.

      3. “You have to spend a lot more time that should be necessary on reviewing the contents of a patch and testing the fix before it can be committed because people do not leave enough detail in the ticket to make the committers task easy.”

        To me, that translates directly to “We need to double the staff on the commit team, in order to deal with our huge amounts of unintelligible tickets.”

        But to you, it means something more like: “We’ve far too much work on our hands; and so we need others to work harder when they help us.”

        That’s a fine reading too. But in this case you need to look into how we keep the helpers interested.

        On the one hand side, you’ve potential contributors who get discouraged because their pet patch gets punted or ignored. After a few tickets, they decide it’s not worth their effort to massage the tickets they open, or the patches they submit.

        After some time, they stick to submitting a vague report (if any at all), occasionally with a patch, and they move on without even worrying about whether it gets fixed or not. I would know this first hand, since I occasionally do exactly that.

        You’d of course argue here that to sort this out we need more trustworthy bug wrangler. Just to make things clear, I tried to buy that argument this year. Let’s take a closer look on this front.

        A bug wrangler spends an hour or two of his valuable time, helping and encouraging a newbie into massaging a commit-worth patch. The tested or commit keyword gets added, and the patch sits there for a few weeks with little if any feedback. It happens once, twice, … many times.

        And then all of a sudden, the tickets and patches get punted too. Maybe it’s lack of time, or trust, or both. I don’t want to know. I do know it happens, however — first hand again. And your bug wrangler decides it’s not worth his time to help newbies either.

        In the end, the gist of your argument is the situation is screaming for more more helpers, in order to take work off of your plate. The gist of mine (and a handful of others) is that these very helpers are losing interest from lack of attention, which in term lands more work on your plate.

        In the meanwhile, every week sees an increase in the total number of bugs in trac, which means you’ve even less time to give helpers attention. Don’t you think it’s time we finally move WP out of this vicious circle?

    2. To me the big point is you don’t have to be a committer to be a big contributor to WordPress.

      We have a large community of core contributors which do a lot of work to make WordPress what it is yourself included.

      We are thankful for the work that everyone puts in and the time that people volunteer to help make WordPress better for everyone.

      What we need is for people to help and encourage the new contributors to mentor them and show that there work is appreciated.

      If everyone acts in a positive and encouraging light, is welcoming and helpful towards new contributors then they will want to stay.

      1. “To me the big point is you don’t have to be a committer to be a big contributor to WordPress.”

        I’m quite sure everyone will agree on that. But it’s not the question I raised.

        “What we need is for people to help and encourage the new contributors to mentor them and show that there work is appreciated.”

        And as I mentioned further up, the trouble here is that these mentors’ desire to encourage newcomers eventually withers away too from lack of traction.

        I’ve spent quite a lot of time, earlier this year, cleaning up the ticket list. I also took the time to create new trac reports. I did so because I took your argument at face value — that what’s really needed are a couple of wranglers and better tools to push the correct tickets to them.

        As far as I can tell, the reports were mostly ignored. The ticket keywords have in turn returned to being their usual mess. And the workflow-related reports have become completely useless as a result. It’s discouraging.

        So, yes, I’d be happy to be positive. But I’d be a lot more so, if I didn’t see people in trac who openly get discouraged by the lack of traction. I’d be even more so, if I didn’t get the impression that someone is sticking his head under the sand.

        If memory serves, Windows NT (4M lines of code?) was mocked at when it got released with 65k or so pending tickets. But nobody seems to think that it’s a problem when WP (30k lines?) gets released with 1.1k tickets. That’s a bug ever 30 lines of code, vs 800.

        Neither you or any of the core devs are at fault here. You simply have far too much on your plate, and we need a couple more of you to bring this back to a manageable level. There are two dozen or so regulars in trac. Why is it so hard to pick one or two, and see how it goes?

      2. As far as I can tell, the reports were mostly ignored. The ticket keywords have in turn returned to being their usual mess. And the workflow-related reports have become completely useless as a result. It’s discouraging.

        The reports were and are useful. I use them regularly and I am grateful for the time you put into writing them.

        If memory serves, Windows NT (4M lines of code?) was mocked at when it got released with 65k or so pending tickets. But nobody seems to think that it’s a problem when WP (30k lines?) gets released with 1.1k tickets. That’s a bug ever 30 lines of code, vs 800.

        According to Ohloh WP has ~128k LOC and currently we have 499 open tickets which are marked by the reporter as bugs.

        Yes this is a high level per LOC. I don’t think it is particularly bad for a popular open source project that makes it really easy to raise a bug report.

        However, a portion of those bug reports will either be unverifiable as they don’t have enough information in the.

        Also based on my experience few years I suspect that a portion of those tickets will actually be feature requests.

        Neither you or any of the core devs are at fault here. You simply have far too much on your plate, and we need a couple more of you to bring this back to a manageable level.

        I don’t see a big bottleneck in getting stuff committed so I don’t see that this would help significantly.

        The bottleneck I see as I have already said is in the availability of well tested and documented patches – You can throw all the committers you like at trac but without these they will not make any bigger dent in the number of tickets.

      3. “The reports were and are useful. I use them regularly and I am grateful for the time you put into writing them.”

        You’re most welcome. But in this case, please consider this report:

        http://core.trac.wordpress.org/report/6

        The first page starts by tickets that are marked as tested/commit or dev-feedback. I’d refer to those two sections as the core dev’s todo list, and they should stay trimmed to a handful of tickets at all times.

        It’s perfectly fine if things that are marked as tested do not get committed on the spot. “Did you test this too?”/”It might be better this way” are a perfectly normal part of the workflow.

        What is needed, however, is that the tested/commit and/or dev-feedback keywords quickly get removed by core devs, complete with any applicable feedback (wontfix is fine too), so that whoever is looking into the ticket knows whether or not he should massage things further, and how.

        I’ll very honestly start to see the light if I see that happening on a consistent basis. (And yes, I’ll stop beating the dead horse if I do.)

  11. Personally, I think every bugfix should go in, no matter how severe. You don’t have to have a new maintenance release for every bugfix – but every bugfix should go into the next point release, unless that particular fix involves changing things which are very likely to introduce more bugs.

    In particular, once there is a tested patch for a bug, it should be committed to the next maintenance release. Once there is already a patch submitted by a member of the community, all your arguments about wasting dev time on patching the bug go out of the window. The time has already been spent.

    Whether developers should be encouraged to spend their time on fixing bugs for point releases or working on the next major release is a different matter, and I would argue that they should work mostly on the next major release.
    But if they choose to work on a maintenance release instead, there is no reason why their patch shouldn’t be used.

    1. Can you explain why it is so important that every bug fix goes into the next maintenance release?

      What is the problem with waiting till the next major release for these bug fixes?

      If we are to put every bug fix in the next maintenance release then they no longer become minor things with a small amount of changes and this makes them much harder to test and is going to make it much more difficult to convince people to upgrade as they will be worried by the number of changes and the increased chance that a new bug has been introduced.

      1. >Can you explain why it is so important that every bug fix goes into the next maintenance release?

        Peter, honestly, are you soliciting opinions, or do you just want to argue with everyone?

        caesar gave several valid explanations in his comment, and they should be given serious consideration.

      2. Because bugs should be fixed as soon as possbile. By definition, a bug is going to impact at leat some people to at least some extent, even if just to by annoying the hell out of them.

        The longer you make people wait for bugfixes, the more they are affected by the bug.

        Since maintenance releases would contain only bugfixes, there’s not a huge amount to test – as I said before, any bugfixes which do have a high probability of introducing bugs should be punted, but that doesn’t mean simple ones need to be punted too.
        There is no hard-and-fast rule about how likely a patch is to introduce a new bug. It’s often obvious to people familiar with the code whether it will introduce a bug or whether there is absolutely no chance of it doing so. We don’t need a rule about how to work out whether a patch can introduce a bug. We need to apply common sense to each case.
        Furthermore, as I mentioned in my previous comment, it is often the case that a patch which could casue bugs gets tested thoroughly by the developer or other community members. Once they have done this testing, why not commit the patch?

      3. @miqrogroove:

        Peter, honestly, are you soliciting opinions, or do you just want to argue with everyone?

        Yes I am honestly soliciting opinions. I want to understand why, from an end users perspective, we should be including every bug fix in every maintenance release.

        To me the whole point of maintenance releases is to make a limited number of changes so that it can easily be tested and end users feel comfortable upgrading to it.

      4. @caesarsgrunt:

        Because bugs should be fixed as soon as possbile. By definition, a bug is going to impact at leat some people to at least some extent, even if just to by annoying the hell out of them.

        Not all bugs have any visible impact or actually annoy people.

        Since maintenance releases would contain only bugfixes, there’s not a huge amount to test

        If we were to commit a patch for every single open ticket currently marked as a bug in the WordPress trac (http://bit.ly/wpopenbugs) then we would have a maintenance release with as many changes as a major release which would need just as much testing and we would be wrong to call that a maintenance release.

        Once they have done this testing, why not commit the patch?

        Once the testing is done this patch would be committed for the next major release (as long as that release was not at the point in the cycled where the change was seen as not appropriate)

  12. I’ll have to agree with the points you make Peter. But I want to bring to your notice a Usability Issue with a key feature in WordPress 2.9 “Carmen” that can use your attention..

    If someone upgrades WordPress right from the DashBoard and has some plugins waiting to be updated – he’s most likely going to miss the “Upgrade All” button/link under the Tools > Upgrade section. I think it’d be much better and sensible to have the “Upgrade All” feature inside the “Plugins >Installed > Upgrade Available” as well !

    I posted it as a topic in WordPress Support > Request and Feedback
    Usability Bug – Batch-Upgrade Plugins Feature in WP 2.9
    http://wordpress.org/support/topic/343535

    Regards..

  13. In retrospect, are there any examples of bug fixes from previous point releases that you feel should not have been addressed? If so, why?

    I’m not even sure that this post is asking the right question. Taking a step back, everyone surely has their own priorities. My belief is that, too frequently, WordPress behaves like a ship without a rudder.

    It seems to lack a long-term vision. Greater clarity on this would help to define the development priorities, like what should be addressed in point-releases.

    Effective future development requires some consensus from the community and a bug can not be classified as critical without a clear idea of what WordPress is trying to be; until then, it will likely remain a haphazard production.

    Some Specific Thoughts
    The WordPress about page references blogging software and the evolution toward becoming a CMS solution. Since blogging software is a subset of CMS software, WP should probably focus more on core improvements that enable development as a CMS.

    The open source nature of the WP framework is great. Because WP allows users to customize it for their own needs through plugins and themes, there should probably be far less focus on improvements around any particular type of CMS (e.g. photo editing, blogging, etc.).

    Conclusion
    All in all, WP is awesome, but a little direction might make for an even awesomer (?) piece of software and a faster release cycle.

    P.S. Sorry if this is all a bit too abstract, but I find that addressing problems within the context of some broad and generally agreed upon principles is usually helpful.

    1. In retrospect, are there any examples of bug fixes from previous point releases that you feel should not have been addressed? If so, why?

      I don’t have any specific examples to hand. I am more interested in going forward and getting a consensus on how the make up of maintenance releases can be decided so that we can avoid wasting time on discussion the priority of bugs every time a single bug is not included in a maintenance release. If we have a clear set of guidelines that all the contributors and bug gardeners can use when assigning bug fixes to milestones it will make the process much faster and more productive work will done on improving the quality of the software as a whole.

      It seems to lack a long-term vision. Greater clarity on this would help to define the development priorities, like what should be addressed in point-releases.

      One of the things I hope we can get out of the 3.0 feature discussion is a roadmap of possible feature make up of future releases as well so that we can say 3.0 is focused on “X and Y” and 3.1 with be focussed on “A and B”. Of course these priorities will be reviewed at the start of the cycle for 3.1 but it will help show the forward progress.

  14. “Tested” patches in trac introduce a lot of bugs.

    No bug fix is without risk. See 2.8 for what you get when you pump lots of patches into a release.

    There will always be tickets that won’t be fixed in the current release. To not punt tickets is to never release. It also allows your release to be stalled by whoever takes to the bug tracker with an agenda or a grudge.

    Any corporate engineer has gone through the trials of release engineering, bug scrubs, and the software release cycle. Near the end of a release the lower priority stuff is postponed so that you can actually release something. Triage is a necessity of release management. Those who say every bug must be fixed do a disservice to their customers because they never get around to releasing what has already been fixed. Delaying by six months means delaying the delivery of fixes and features and making the release bigger and buggier.

    WP maintenance releases should fix only high impact bugs affecting many users. Pumping fixes for lots of bugs into a maintenance release will make it buggier. A maintenance release is carefully considered fixes applied to a known base. There should be no surprises in a maintenance release.

    1. Nonetheless, I honestly believe that we should take things up to a 2.9.10 release if necessary, before we focus on 3.0 and the WP MU merge.

      WP is nice as it currently is, but it has quite a load of bugs in it. So, let’s get the ticket list trimmed to something manageable first, in order to stop the bleeding, possibly adding a minor enhancement here and there while we do, and then — only then — should we focus on adding new major features.

    2. Also, re: “Tested” patches in trac introduce a lot of bugs.

      If those same patches got marked as needs-patch with a comment on why, instead of simply being ignored or dismissed without the slightest comment, the core team would be miles ahead of where you are now.

      As things stand, someone who marks a patch as tested frequently ends up watching that patch get punted without the slightest amount of feedback. And as suggested further up in my reply to Westi, this can discourage some who write the patches in the first place.

      1. If those same patches got marked as needs-patch with a comment on why, instead of simply being ignored or dismissed without the slightest comment, the core team would be miles ahead of where you are now.

        Where we can easily give feedback we always try to.

        The problem is that we cannot rely on the Tested flag as often not enough detail is provided. We therefore have to either pick and choose things which have enough information to easily test or have an obvious effect or spend a long time testing the patch ourselves and fixing the bugs in it.

        It would improve things a lot if people always provided detail of how they tested the change.

      2. I’d agree with Ryan. Many of the patches marked as “tested” and “commit” may introduce new bugs or change in behaviour that breaks backwards compatibility (plugins).

        Let’s say we introduce a new requirement: a patch can only be committed after is has been tested by at least three developers and each leaves a brief comment on what was tested and how. How many of the outstanding patches would pass that requirement? Very few or perhaps none.

        Considering that maintenance releases are tested by the community a lot less than major releases, it would be very bad to start introducing new bugs and incompatibilities in them. Then we would be locked in an unending cycle of “a maintenance that fixes the bugs introduced by the previous maintenance” etc.

  15. What should go in a maintenance release?

    These would be my priorities for scheduled maintenance releases:

    1. Fix any bugs introduced by new features added at older major releases – which weren’t fixed at the corresponding maintenance release, or latest major release. In an ideal world these wouldn’t have escaped the earlier maintenance release, but c’est la vie!

    2. Fix any bugs introduced by new features added at the previous major release;

    Leave new features for major releases only. Deciding on the priority of new features is another discussion altogether!

    Security vulnerabilities should be fixed with their own maintenance release as and when necessary.

    Inevitably, #1 above will require some picking and choosing in view of the number of bugs/enhancements in trac, especially since many so-called bugs are not bugs, they’re just intended behaviours that don’t suit 100% of users. It’s unreasonable to expect every “bug” to be fixed before moving on to new features – software development just doesn’t work like that, certainly not in Open Source projects IMO.

    Just my two centimes worth…

  16. This is something that’s been an issue with WordPress since I first discovered it five years ago, and one I’ve experienced in other OSS projects. I doubt it’s going to go away any time soon.

    I personally think that it wouldn’t harm WP at this point to focus a “major” release on simply fixing a lot of the minor bugs that tend to get punted. Perhaps 3.1, as 3.0 seems to be focused on merging WPMU into the core software (among other things). There’s going to inevitably be some issues that need cleaning up, either from a code standpoint or from a UI standpoint that might not be minor point worthy. This could also help limit the number of minor releases to the 3.0 branch to just security releases and major bug fixes.

    With the talk of mentors and building a stronger connection amongst new/seasoned developers, focusing primarily from the beginning on these types of bugs rather than any heavy lifting, new features would allow for more time to foster these new developers and get them more familiar with the code base and the procedures of how a release is done. Also, the discussion of canonical plugins could dove tail into this type of a release as again, it could provide additional areas for newer developers to really get involved, as they may not necessarily have to digest as much core code to really get involved and see an immediate return on their contributions.

    Anyway, certainly not a solution to the long term question, just some random thoughts on how to possibly satisfy some goals I’ve seen tossed out for community building while also fulfilling the goal of making WordPress suck less.

  17. I agree with your four ‘rules’ for what should be included in a maintenance release, with the question of if adding hooks / filters or adjusting the variables sent to hooks and filters would qualify as a “severe” problem. I view the extendability of WordPress to be it’s number one feature and a missing hook / filter would qualify as a severe bug in many situations to me.

Comments are closed.