Agile Development is in danger of slipping into the ditch already occupied by brainstorming sessions, train-the-trainer, and team-building: that of management double-speak and great ideas used as excuses to continue the same old practices.

I know I’m not the first person to point all this out – however, unlike many of them, I don’t see Agile as a panacea or a ploy.  It is precisely because I believe in the principles of Agile Development that I feel a little twinge of pain when I see it going wrong.  Rather than trying to forecast why it must fail or trumpet its invincibility, this post is simply trying to paint some yellow stripes on the bumps in the road, in the inevitable web list format:

1) Working From Small To Large – Agile Development aims to put practical decision-making in the hands of the people doing the work.  This sounds dangerous to the people who ostensibly have the “vision”, as they realize that real-world concerns are shaping “their” project.  However, this is only a problem when there are errors in the vision in the first place.  As Getting Real aptly points out, this style can only work when a motivated team understands and embraces the vision of the project.  If management thinks, “well, I’ve got a superb and agile team of developers over here and a great concept over here – now all I have to do is start assigning tasks”, the project is doomed to snarl.  After all, agile development teams don’t need reams of documentation and planning, right?  Sometimes that’s true, and then only if they deeply understand the goals of the project.  Agile Development is like high-end audio gear.  It can make everything shine, but it will also highlight all the flaws.  Make the effort to get your direction aligned before you start your project running.  You want a nimble bobsled team, not a game of tag.

2) Agile Development on a Non-Agile Project – How many workstations have felt the concussion of a developer’s forehead after the project manager (or the project manager’s manager) stops by to report that the glorious, elegant solution the developer came up with is nixed for a reason that the developer herself had pointed out weeks, maybe months ago.  Agile teams are supposed to respond to change quickly and enthusiastically.  However, agile precepts are corrupting when applied to non-agile projects.  If the business isn’t sure what they want, it means trouble for most projects, but an immediate fail for an agile project, which is better adapted to change, but also more vulnerable to it.  Even cases where the business side is focused may not be suitable for agile methods.  For example, one of the main tenets of Agile Development is that developers and business units collaborate to find the best approach for the customer.  If you are Apple, you make your living by exceeding the curve and leading your customers (in theory) to the right place.  However, if you are in an industry that requires stable, trusted solutions (banking would be one that comes to mind), you would do well to temper that agile enthusiasm with a solid dose of risk assessment.

3) Teams Not Independent – Unlike the previous examples, this one is a prime example of giving lip service to an idea while ignoring it completely in the day-to-day life of a project.  Micromanagement and agile do not go well together.  Though this is a well-known problem in many types of projects, it actually goes against everything agile stands for.  The Agile Manifesto points out that face-to-face discussions are the most effective means of conveying information.  But what do you do when you’re having a face-to-face conversation with someone who wants to insert personal preferences into his developers’ work?  Or when management cannot protect the team from peripheral, time-sapping requests?  In these cases, it may be best to put your nose to the grindstone and polish up your resume.

4) Over-Employment – So far, we’ve touched on common problems, but this point is rooted directly in the areas where agile excels.  The concepts of Agile Development have proven so successful that many developers expect to find them on day one: being able to call on business personnel to clarify / collaborate over requirements and solutions; the use of small teams to accomplish discrete tasks; flexibility in planning; experimentation and prototyping as integral to the development process.  When you’ve got it, you don’t need to push it.  You don’t want to show off the strings you just cut and find yourself holding the strings in the process.  If your developers are in the comfort zone, don’t push them out of it just to try and fulfill what someone says is “properly” agile.

5) Scope Defenses Are Down – We’ve all heard that change is key to success, and in the world of software development, the ability to rapidly change direction can mean a decisive edge on the competition.  But there is a big difference between an agile redirection of resources and a stumbling, careening project that no longer knows where it’s going.  Part of this involves scope.  Imagine a current of water traveling through a gulley.  You realize you need to divert the gulley 90 degrees to the left.  What happens if you dig out a leftward channel but also leave the gulley intact?  Imagine doing this every time the project needs a change – you are left with a flooded delta, not an effective change.  If developers or managers are thinking “this is an agile project, so scope creep shouldn’t matter”, they are lining themselves up for a waste of resources and a messy, unfocused project – there is nothing agile about that.

6) Weak Push-Back – If Getting Real gets unrealistic at points, it’s probably in the area of push-back on feature requests.  Sure, if you’re a crack team of multitalented developers and designers with a shared vision and a successful product, you can afford to ride out an initial wave of resistance as customers adjust to your latest release.  When you have a corporate customer relations manager whose phone banks are blowing up, and he’s coming over to speak with your boss, things “get real” in a hurry.  For Agile Development to fulfill its promise of responsiveness, developers must – somewhat ironically – be allowed to provide context to the business.  As usual, communication early and often is key.  Remember that good developers want their applications to be effective and well-received.  Imagine that your team and the business are on the same page, you think you have a brilliant answer, you roll it out, and the users hate it – and continue hating it.  Agile Development can really shine for you here.  Now imagine the same situation, except that the developers knew users would hate it, and they said so, but you didn’t listen?  How is the fix going to go?

7) Paying The Bills – An agile team should, according to The Agile Manifesto, be able to run indefinitely, making changes and adjustments, keeping up a regular pace.  This does not mean, however, that kludges and workarounds should be allowed to fester when they can be cleaned up.  Get Real puts it well: “The same way you should regularly put aside some of your income for taxes, regularly put aside some time to pay off your code and design debt. If you don’t, you’ll just be paying interest (fixing hacks) instead of paying down the principal (and moving forward).”  If your team is maintaining a constant pace, they’ll be fixing these issues eventually.  Better to account for it.

8) How Do We Do This – Agile Development requires not just developer feedback on the project itself, but developer feedback on the process.  The Agile Manifesto calls this “tuning”, and requires it at regular (and, we can assume, scheduled) intervals.  Failure to pay heed to this tenet of the manifesto results in what I like to call “Casual Development”, with everyone plugging along, ignoring opportunities to create efficiency and save time and money.  Can there be a better rationale to the business side than “this will save you time and money”?  A regular reflective breather helps create more smoothly running teams and exposes issues that may be silently eating away at a project.  It also gives deeper meaning to the term “agile” itself: the ability for a team to self-diagnose and fix issues before they arise.

9) Docu-drama – As someone who enjoys writing, making diagrams, and having sensible project outlines, I’ve never had a problem with documentation, unless it’s bad documentation.  Some developers and managers act like documentation is a pure waste of time, or else as archaic as stone tablets.  This is hubris.  Thinking that having an agile project means documentation is superfluous is to ignore the experience and wisdom of others – is your project so cutting edge that no one has ever attempted anything like it before?  Then maybe you don’t need documentation, and you can just wing it on the strength of your genius.  Will no one ever touch your application besides you and your team?  Surely if they do, they will simply read through your sublimely intuitive and graciously commented code and instantly intuit exactly what is going on.  Just like pre-writing is a great creative exercise for all kinds of non-linguistic projects, documentation is immensely helpful to produce, even if you think you’ll never need it.  Go ahead, get your thoughts, your vision, your risks and assumptions, you technical specs down on paper.  You may be very glad you did.

10) Users?  What users? – Oh, the poor, poor users.  Who is looking out for them in the world of Agile Development?  One of the main creeds of Agile Development is that everyone should be looking out for the souls who have to use our wonderful applications.  Too often, no one is looking out for the user, so caught up are they in the idea that in the modern, agile era of development, all the user has to do is wait until the next release.  The truth is that sometimes the business and the development teams get together and end up with zero idea of how users are going to react.  Here we come full-circle: some projects are well cut out for Agile Development, and some are not.  If you are a small startup and you’re desperately trying to get your app to market, agile may be what you’re looking for.  If you’re in the midst of transitioning a legacy application that thousands of users are relying on daily, you may want to (shudder) think of your users first and your developers second.  As I’ve pointed out, many agile habits have already worked their way into the mainstream, so don’t be afraid to be a bit selective about how far you want to push the methodology on any given project.  At 37 Signals, users can marvel at the process, but that doesn’t mean your users necessarily want to be made aware of your process.  It pays to spend the time and effort to get to know your users.

Agile Development is neither a gift from above nor a mere trend – it’s an important crystallization of ideas that can often provide real clarity of purpose for developers and help projects adjust to a fast-changing world.  But it can also be a destructive force if used carelessly.  It’s like business casual: it can be fun and freeing, but in some circumstances and some hands, it can be disastrous.  I hope this post helps you choose your outfit wisely.

With the iPod and iPhone, Apple looked like world-beaters, and when the app store opened, developers flocked to pay Apple $100 for the privilege of downloading the app SDK; they began furiously pumping out apps and biting their fingernails while Apple’s team personally verified the worthiness of each app.  Okay, we get it: part of the iPhone experience is knowing that every app has been tested and approved; it’s quality control.

It seems that much of the debate right now is whether Flash is an open standard and/or whether apps written in objective C are open and/or which company is behaving like the biggest crybaby.  Really, neither is an open standard.  If you want to develop an iPhone app on a Windows machine or if you’re looking to compile some Actionscript without running Flash, you have to use third-party software in either case.  These companies are like two toddlers whining at one another for not wanting to share their toys.

Jobs does have some points – Flash is not the best for use in iPhone apps, and not just because adopting it for the app store would punish all of Apple’s loyal developer community, who plunked down dollars for the SDK.  Having developed in Flash, I’m willing to believe Apple’s claims about Flash’s memory issues (remember ye olde garbage collection issues?).  However, WebKit is the best mobile browser anyone has ever seen, and it doesn’t support Flash.  On this point, there seems to be no good rationale.

Let Apple keep Flash out of the app store.  That’s their bread and butter – okay.  But creating a mobile web browser that doesn’t support the plugin that’s installed on over 95% of desktop browsers – including safari – is just bitterness manifested.  Other mobile browsers support Flash, and yet Apple seems too content to let the market pass them by in this regard.

The worst part is that Jobs and his supporters are offering HTML5 as a panacea.  Sure, it’ll be great – whenever it’s fully supported – but its video component will lack much that Flash is able to do fairly easily now (real interactivity, for example), and its codec of choice, H.264, is proprietary.  Having most players agree on a proprietary format is not the same thing as an open standard.  Oh, and it leaves out Firefox, which is, by the way, open.   It’s like telling commuters that everything will be great when we get our hovercars that run on solar power.  In the meantime, your car is not welcome on our road.  Jobs and his backers may have a point that Adobe treated Apple like a second-class citizen back in the 90s.  Which they were.  Now, I love my Apple products like anyone else drinking the multimedia development koolaid, but wasn’t Adobe doing the same thing to Apple back then that Apple is trying to do to Adobe now?  If the open revolution proved anything, it’s that having a zen-like indifference to such petty squabbles produces winners.  Did Zend worry about Microsoft?

Jobs’ blog post hints at the decades of recrimination by recalling the early days when Adobe and Apple were both based in similar garages.  Get over it.  It’s Final Cut Pro and Aftereffects.  No one will ever use the combination of Premiere and Motion.  Flash is better for the web.  Mac apps are better for Apple devices.

It seems that offering Flash support in Apple’s mobile WebKit browser while keeping Adobe out of the app store market would make everyone relatively happy, but unfortunately, it seems such realism is lacking in the relatively closed and high-ego world of large-scale software development.

update: proof Flash can hang on Apple mobile devices after all?

But for now, the rel tag is easy, supported, and makes getting a little bit of meta information to your script quite easy.

Two uses I’ve written recently are:

(1) Creating abstracted div collapse scripts.  The script just looks at the rel tag for the link and finds the matching rel tag among a class of divs on the page.  Anything matching is shown, anything not matching is hidden.  I never have to go back and touch the script, and I only have to add a rel tag to my divs.

(2) Letting my css-based navigation know which page we’re currently on.  With includes, the idea that your navigation should reflect your current page location fell out of favor a bit.  Kind of suspicious, if you ask me.  It often seems designers and front-end developers adapt their pet styles to suit the latest technology.  Regardless, it’s dang easy with the rel tag to revert to the “old” style of location-aware navigation.  I just drop a rel tag in my nav links and a matching rel tag in my content div within the content include file.  They search for each other.  They find one another.  It’s like an interface component romance paperback.

Part of the reason that HTML 5 is going to include some developer-defined tags is that people have been using rel and a few other tags to include meta data (and sometimes some not-so-meta data) in their front-end code.  I completely understand the cold chill that comes over some folks when they start contemplating gobs of data gooping up the front end.  I also think it’s possible to fret too much.  As the linked article above points out well, it’s possible to include some meta data inline and still be semantically useful, degrade nicely, and not offend the browser.

If people hadn’t been expanding on the uses of the rel attribute so effectively, we may not have had the push for user-defined attributes in HTML 5.  This is a good thing.  It’s important to remember this truism: no amount of standards will stop people from writing ugly code.  But, paying attention to what developers are doing in practice and honoring those practices in new standards definitions can benefit everyone.

Hangin at the Bottom

We’re just psyched to be in a google result page with the likes of James O’Reilly and actionscript.org.

Hooray!  If you haven’t checked out the code yet, by all means please do.

It looks like CSS3 will have some pretty cool new attributes available for us developers.  Of course, it’s all proposed and not released yet, but let’s start salivating anyway:

One really cool thing with the potential for hilarious abuse of power is a proposed scale attribute.  Use your own imagination on that one.

But that’s not the best.  The best (if truly implemented) would be a set of CSS specs for creating a multicolumn layout without having to resort to javascript trickery, fakey background images, or ridiculous amounts of div nesting.  Great news, everyone!

Found on these sixrevisions posts.

Dagnabit!  What is the point of combining xml and html standards if I can’t use them together all the time?  I know, you may be thinking, but xml is not html , no matter what you throw into your nodes.  You’re right, and this is exactly what jQuery tells us.  You can grab the text contents of any node, but if you have html tags in there, the .text() method in jQuery will just look right past them.  As it should.  This works fine, for little snippets like my titles, but what if I want to put some (ostensibly) well-formatted html code in my data?  Further, what about all those applications where you want to store html snips in a database and read them out in XML?  How do you get around this?

For that, we’d want to use the .html() method of jQuery, and for that, my friends, we suddenly get quite strict.  XML, according to jQuery, is not html, and so has no innerHTML quality for the .html() method to grab.  It doesn’t matter, says jQuery, if your html code is so well formatted that it can be parsed as xml…it’s just not possible to access an html-specific property that does not exist.

Like fun, I say!

But before we get to the solution, let’s heartlessly go over some available options:

1) We could punt on the whole jQuery thing and use Flash, which knows just what to do with a string of html parsed from within an xml node.  Furthermore, Flash has a nifty .htmlText property on its text fields, and although the formatting of said html pales compared to real css, it will faithfully render as html anything you tell it to, caveat codeur.  Since the project in question did involve some Flash, this was an option, but one of the points of the project was to take the same xml data and have both flash and jQuery parse it for different purposes, and thereby provide ease for content providers and improve accessibility.  So doing the whole thing in Flash was out.

2) We could escape the heck out of all our characters and turn the whole node into a mess of html character codes.  Un-fun to code and messy for the content people.  Also, I tried that and it rendered out a block of actual html onto my screen.  I’m sure I could have reconverted those characters back into actual html behind the scenes, but again, un-fun and a hassle.

3) Use a fancy xml consuming plugin or script.  There are some neat ones out there, and I tried one in particular that looked pretty good, but it seems they have a common issue – you have to know what tags you’re looking for in order to seek and replace within the script.  Some of them will hunt for anything that looks like an html tag in the string and convert that, but mostly these plugins seem like scripts that people wrote to perform functions for themselves, and then extracted and shared.  Which I applaud.  But these were not what I was looking for, and I found the implementation sometimes troublesome.

4) [Warning: bad solution] Hand jQuery a bag of grapes and say it’s “nature’s jellybeans” (at least that’s what my mom used to do to us).  It goes like so: grab your xml file with an ajax() call, and use the optional key/value pair:

dataType: “html”

And then you’d do ike so:

$.ajax({ url: “myXMLdoc.xml”,
dataType: “html”,
success: function(xml){
var myHtmlContent = $(‘node’,xml).eq(id).find(“nodeName”).html();
$(“#targetdiv”).html(myHtmlContent);
}
});

…and there you have it!  Or, would…if…

…can you see where this is going?

…search your feelings, Luke…

…if only Internet Explorer handled the XMLHTTPRequest result like the rest of the world!  Like most Microsoft stuff, IE, with its own version of the XMLHTTPRequest (someone correct me if I’m wrong on that), wants to get too clever and won’t let you, the developer, tell it what kind of data you’d like to see (even if it’s either of several types that are verging toward common standard).  No siree!

Supposedly, recent versions of IE (7+) have their own XMLHTTPRequest, not the previous Microsoft HTTPRequest.  However, the Microsoft XMLHTTPRequest doesn’t behave like the other girls and boys, even in the recent versions of IE.  Now, this post didn’t start off to be a discussion of the supposed evils of Microsoft’s proprietary habits.  But who else has almost 11 million results on a google search of how much they suck?  Hint: not Chrome (3 M), Firefox (1 M), or Safari (1 M).  (Can’t evaluate Opera on this critrion because sadly a lot of people out there think that real, actual opera – the music – sucks too).  For comparison: “yanni sucks” (400 K), “death metal sucks” (700 K), “paris hilton sucks” (1 M), “the lions suck” (1 M), “tv sucks” (11 M), and “new york sucks” (12 M).  So, yeah.

Anyway, after that turgid digression, finally, we come to the solution.  And…

It’s simple!  Maybe I’m, like, the last one to figure this out, but you just use XML’s CDATA declaration, and wrap it around our html code within our xml node.  If I had found that idea earlier, it would have saved me some time, but I would have missed a brain-scratching moment or ten for sure.

Anyway, this is fascinating to me for several reasons:

a) When oh when is Microsoft going to just give up and give us a truly standards-compliant web browser?  Maybe they should spin off a company whose software is intuitive and flexible and plays nice with others.  Oh wait, they already did that.

b) With the XHTML standard in full force, it seems totally silly that one would have to pass off perfectly valid XHTML within an xml document as CDATA.  Isn’t the whole point of XHTML to be able to one day have a semantic web, where code itself can be meaningful?  How far away are we when it’s trouble to get even well-formatted html out of my xml nodes without basically commenting it out?  In theory, that means I can’t parse through my html elements, which seems to go against the whole ethos of XHMTL.

Well, heck.  I remember when CSS was going to put an end to bad code.

I mean, you could simply get a USB plug power adapter with cord, plug that sucker into your existing wall outlet, and staple-gun the cord to the wall.  But that would just be ugly.  This is way more elegant, more fun, and more dangerous.

The author of the article also makes a fascinating point, however dubious it may be, that one day all wall outlets will be USB.  This makes me think of a smart grid with smart appliances sharing data on energy consumption and using sophisticated algorithms to allocate power just where it’s needed, when it’s needed.  Since 26% of the cost of our electricity comes from distribution (not counting loss through transmission), this could be a real boon.

However, it also makes me a little nervous to think of my household appliances having the ability to share data.  While it’s totally innocuous, do I want the electric company (or, theoretically, any good hacker with a USB connection) knowing how many times I used my toaster this month?  With the immense amount of personal information that is now being bartered without our consent from our internet use, TV viewing habits, and credit card purchases, I kind of cringe when I think of data on every household appliance use being available for some marketer to collect and sell.

Imagine your washing machine saying, “hello!  I notice it’s been a few days since you’ve done the wash.  Welcome back!  You haven’t been using fabric softener lately.  Did you know that Snuggle fabric softener is on sale at your local supermarket?  Push the green button to have some Snuggle fabric softener shipped to you right now!”

To some people, this may seem like we finally have reached the Jetsons-esque world of the foodarackacycle and appliances that are our pals.  To me, it seems like digital marketing hell.

If we ever get to the days of data sharing over the power lines, you might see hackers doing the opposite – posting DIY articles about converting their ubiquitous USB smart power outlets back into good ol’ fashioned “dumb” power.

In the mean time, though, this outlet looks pretty sharp.  There’s nothing wrong with being ahead of the curve, until the curve catches up with you.

By this time, I’ve worked my head around the idea that, to many developers (and clients) out there, CSS is the one true way and tables are only good for displaying data – that is, data that they consider data and not data that someone else might consider data.  See my previous post on semantic html for more carping on this topic.  But enough negativity – I would hereby like to add my voice to the growing number of developers calling for a new future in CSS: [paraphrase] “If CSS is the golden chalice of front-end web development, how come it can interpret between pixels, point sizes, and em sizes, but it can’t add 1 + 1?”

Now, I know, purists will say that Javascript is for math, and CSS is for styling.  Just simply get a good Javascript library and do your math there.  The problem is that other purists will say that using Javascript to generate your CSS positions is pure hackery.  And yet other purists will say “everyone come to my website to check out my awesome CSS hacks!”

Let’s face up to it: there are some things that CSS is just not that good at.  The idea that you are ruining your SEO by adding three lines of table code to get your column heights equalized seems dubious.  But, I’m willing to go with that and check out some truly weird and wonderful tricks to arrive at the same solution that would take literally half a dozen lines of code using old clunky tables.  But yeah, I’m sure google’s robots are completely fooled by a table and a couple of <tr> tags… [sarcsasm alert].

Anyway, if CSS is going to be responsible for our layout and can have access to and control some of the DOM, why can’t we just do:

#layerid {
width:20%;
margin:10px auto;
height:width * 0.5;
}

That would really open things up and allow for some more creative, interactive layouts.  Hopefully, CSS 4 will allow for some of this.  It should be, in my opinion, a no-brainer.

Hope you enjoy.

The real dream has been that one day, HTML and XML would be combined (not just in terms of the XHTML standard), and that the web would turn into a big soup of meta information.  This would allow all sorts of wonderfully customizable content, searching, cross-threading, and possibly, someday, even help create some kind of internet based solely on context, rather than infrastructure and IP.  Positives: super-personalized web-browsing, advertisements, and content aggregation.  Negatives: super-personalized web-browsing, advertisements, and content aggregation.

But I digress.  The flock to div tags as cleaner and more semantically meaningful has been a beneficial one, but for we web developers who remember the [good?] [bad?] old days, there are more than a few cases in memory of the development community rushing en masse to the next greatest thing, and then flipping it into reverse with the release of the next IE browser / web standard / greatest thing.

It has gotten to the point where I have seen postings looking for people who write div- and css-based code – no tables allowed!

This is madness.  What is XML, after all, than a method of organizing data?  When you’re getting an XML feed from a backend, you are converting your data from a table-based format (database) to a markup-based format.  Same data (we hope).  The real question here is page layout.  Surely web developers who are using and editing one another’s code would rather see:

<div id=”header”><!– header –></div>

than

<table border=”0″ class=”table” cellpadding=”0″ cellspacing=”0″>
<tr>
<th><!– header –></th>
<tr>
etc.

But while that div is nice and neat and it allows for better accessibility, it does not do as well when it comes time for, let’s say:

this.

Also, despite the wonderful blessing that is CSS (it can be quite soothing), when something goes wrong with the style sheet, the result can be disastrous.  Of course, we all want to write code that degrades gracefully, but the fact is that, the more we ask of our code – especially as we utilize style sheets – the more risk we run that if something breaks on the backend (certainly our code would never break), or a user is tied to a hopelessly outdated browser, the result will be unintelligible.

But back to semantics, for their own sake.  Perhaps it comes down to a particular aesthetic, but the notion of a fully semantic web makes me think of those kitschy items that have the word for the item printed on them, for example a mug that says “MUG”.  Maybe there is a bit too much big-picture here, for a change.  Net neutrality’s goal is access and publishing rights for anyone with an internet connection.   It’s not hard to imagine a semantic web that wants to take choices out of our hands and push certain content in front of our eyeballs for us.

As for us developers, if the goal is good, solid code, the answer seems to be, like with much else, a healthy mix of new and old, with an eye towards every tool and its best use.