Design debt, Drupal.org, and the Drupal project itself

Published in: 

The community will begin to realize (in part due to the above stalled project) how much design debt Drupal has accrued over the past several years of rapid expansion and development. While the problem will not be solved in 2008, the realization that it IS a problem will help prevent a long-term meltdown. - Jeff Eaton January 4, 2008

Design Debt and Drupal.org
Indeed as Jeff suggests, this "realization" seem to be a bit of a challenge at the moment for parts of the Drupal community right now, at least if the reactions to Harry Slaughter's post regarding a Drupal module ratings site which John Forsythe just launched are taken into account.

Given the possible reactions it was disappointing to me that some people chose to attack Harry and John (e.g., the messengers) rather than the processes which have lead to a situation whereby one of the largest open source software projects on earth can't get SOMETHING up to help it's users deal with their number one feature request for Drupal.org. As John, myself, and any number of people have proven there has been no lack of volunteers, or lack of resources, but rather just grid lock on this issue. (for instance, this thread was just one of many that tried to get the ball rolling).

Contrast how this scenario with module ratings works with how core development works. With core - people submit patches, other people give opinions and review, but eventually one person just makes a judgment call on whether or not the patch is committed and life moves on. With the module ratings it's been more a game of 'waiting for consensus' of a handful of well intentioned people who all each seem to have the power to hold things from going forward but none of whom have the authority to 'just call it' - and thus move the issue on toward *actual* implementation. Thus more time passes by without a solution...

While on the subject of design debt - design debt and Drupal 6/7
Drupal 6, a great release in it's own right, is unfortunately not production-ready for many site-types due to the lack of some very important contributed modules. Since my time with Drupal (4.5) I've never seen anything quite resembling the lag between a Drupal release and it's 'production readiness' that I'm seeing now (Views is not even out of alpha - and CCK is waiting for views to be done before finalizing itself).

Won't pretend to know what needs to be done to fix it all, but from my own personal perspective I can't fathom being concerned with anything regarding Drupal 7 and probably won't be able to until I've got some productive months of being able to use Drupal 6 under my belt. This is NOT to say that current development for Drupal 7 is a waste of time - I do see the value of forward-thinking development, but I also feel there is a rapid rate of diminishing returns for it at the moment.

If there is any point to the above statements, it's that a) I don't think that I'm alone in my feelings and, b) I'm somewhat concerned about Drupal achieving an 'Emperor has no clothes' moment in the near future if these issues aren't eventually addressed.

17 March, 2008

Comments

Note that the lag is actually due to Views and CCK both making the hard choice to undergo a major rewrite during the D5 -> D6 upgrade cycle.

Views' rewrite got held up by the fact that I have two major major projects, and Panels got a higher priority due to my employer, who is basically funding most of my work right now.

CCK is held up partially by Views and partially by its complexity.

Views should be out of alpha by early April, and I hope will be reasonably solid in its first beta. Because of its popularity, Views is getting quite a lot of alpha testing and that's helping a lot.

Anyway, my point is that I feel these situations are reasonably unique, but not as completely out of the blue as it sounds. ECommerce, for example, has always had a pretty big lag from Drupal version to Drupal version, because it's a big project and porting it can take time.

And no contrib modules in history have had the Importance to Drupal's adoption as Views and CCK have. Which is funny -- Drupal prior to 4.7 didn't even have Views; there was flexinode but it wasn't anywhere near as powerful as CCK. That right there is a really interesting dynamic to study, and it's important to think about.

(That said, I do wish some of the D7 development going on would take 2 weeks off and help finish the Views task list, but developers scratch their own itches).

Perhaps you were unaware of the Drupal.org redesign presentation at DrupalCon, or the fact that the Drupal Association has made the Drupal.org redesign a #1 priority right now, or the fact that the suggestion-gathering phase is past and now we're moving into implementation.

When I look at http://groups.drupal.org/node/... -- "" -- I don't see gridlock, I see 3 comments, with only 2 of them from people stepping up. Some of us (like me) have joined the effort without commenting (in fact, just before that post went up), but there are always lots of ideas but very few people who are willing and able and available to do the work.

-- The presentation giving the overview. Video is here.

Then there are all the vast improvements in theming Drupal available already in Drupal 6. It just takes some people to step up. The framework is there. The core is solid.

So what "issues" aren't being addressed? (Maybe you can help address them?)

People will always scratch their own itch, but that doesn't mean they can expect everyone to follow along. I'm interested in making Drupal.org better, not running off to an obscure site with orders of magnitude less pagerank, reach and visibility. If we're going to improve this situation in any meaningful way, I feel we should take care of our own home.

I had not seen that particular post, which is apparently the latest out of the many posts regarding the subject. If this one results in something productive then that's great. But as far as this being 'a #1 priority now' - with all respect it HAS been a number one priority for a long time already. :-)

Also, regarding resources to work on this. I don't see that as being the problem - there are plenty of people who would be willing to implement any/all of it (myself and John makes at least 2), but there have been roadblocks in the form of other people with competing ideas....which means that nothing moves forward. In the thread I was involved in for instance, dww and Kieran seemed to have specific ideas about things, others had other ideas and/or objections - so there wasn't much room edgewise to volunteer to do anything but watch everything be stuck. It was also seemed sort of assumed that dww or someone already working on Drupal.org frequently would be taking care of the implementation details. Which is fine, but it also means that scheduling is limited to those folks schedules.

Finally I mentioned a few issues about core "issues" already - namely the fact that it's not usable in a production environment for many website types at the moment due to core development being so far ahead of contrib development. Certainly there are other design debts that I'll leave to the issue threads, other articles, and/or other people - this is enough worms out of the can for me at the moment. :-)

You want to help see things done with project? Then see DWW's list on groups for the work and patches needed for project.

Being willing to 'help' implies a willingness to collaborate instead of going off and 'doing your own thing'.

Perhaps instead of doing their own things, maybe, just maybe, people could have been contributing patches on the road map already listed out in great detail.

Claiming that people are attacking others isn't helpful, just annoying.

http://groups.drupal.org/node/...
http://groups.drupal.org/issue...

I give you a lot of leeway in regard to being frustrated since I know you're knee deep in dealing with people that do not contribute and/or have no intention of contributing, but they want to gripe a lot.

However - did you miss the part whereby I said that I am willing to contribute to this, have tried to contribute, and other have done the same - BUT - because of process have been unable to?

....so what would happen to a dance if everyone showed up and things never got started because they couldn't agree on what to get the dj to play and no one had the authority to make a decision. I think that best describes the situation - not that people (including myself) aren't interest in being helpful.

And rating themes would be absolutely useful, too. And that fancy module finder at DrupalModules.com could search among themes, too.

Sure there might be people p*** off, if they feel attacked because of Calebs comments. But be sure he only names a structural problem - neither merlin nor anybody replying to his post are to be blamed personally - much to the contrary.

As the tendency is - at least as I read it - to congratulate each other how great "our" Drupal is - I find a bit of open criticism very valuable. For Drupal has just outgrown the phase of a mere love project. Trying to mature and to improve reliability as a whole brings up obstacles - things have to be done faster, more people are waiting for it and may even depend on in with their business.

So maybe we are experiencing a phase of consolidation, where less effort has to be put in adding new features into drupal but instead make the whole package more rounded and business-ready. This sure brings up a lot of pain. The people who work the hardest even get nagged - because they are out in the focus very much like merlin. I wonder if you can keep up as a main developer for both panels and views in the future, but I doubt it.

So - I'd like to see some open criticism more often, I think we can handle it. The boat will still be rockin' and hopefully we will draw the right conclusions.

This might seem pedantic or obvious but there are other open source options available for content management. Joomla and Plone come to mind, and I know that both have solid support communities. Perhaps it is worth exploring other communities that might be more responsive to user feedback.

I also agree with eigentor. Open-source is built upon calm and well-founded criticism.