Brainstorming Done Wrong

Brainstorming - a group problem-solving technique in which members spontaneously share ideas and solutions.

Notice that the definition doesn't use the word critique anywhere in it. That's because brainstorming is meant to be an activity in which all participants, even if it is just one person, are meant to throw out as many ideas as possible without shooting any down under existing premises and allow the organization and planning of those ideas be placed on the back burner.

Sounds like a wonderful way to come up with new ideas and possibilities, eh? It is the day dreaming portion of being a knowledge worker - where you get to think without reality. But that's the part where brainstorming can go awry if you let it.

Brainstorming must be done in the overall context of a plan or a project. If one is just brainstorming willy-nilly and simply for the sake of brainstorming alone, it's a rather pointless activity that is orthogonal to anything that we as knowledge workers should be focusing on. In fact, if brainstorming is conducted without the results of sessions being used to impact an already existing project or goal then it is, in the best-case of a single person, a huge waste of time and mental energy used to flutter out ideas that won't be used. In the worst case, it can demoralize a whole team if they are coerced in to a highly collaborative activity that sucks up double-digit man hours, all to produce nothing if no actionable steps or follow through is executed on the residual items left over from a session. One can guarantee this does not happen if brainstorming is put in to the context of an overall plan or project and is usedas the basis of forming next steps in a project.

A case point that I've experienced at the professional level involves a brainstorming session we had close to the initial launch of a high visible web development/services project I was a part of. Toward the end of that project, when frustrations were high and deadlines were starting to slip, management thought it would be a good idea to have all of the key participants gather in a room and have a brainstorming session on identifying problems in the project and how to correct them. As an exercise in how to conduct a brainstorming session, it was very well done. Someone in management had been doing his/her reading well on how to conduct a successful brainstorming session. Lots of ideas and thoughts were tossed out there, displayed wonderfully as sticky notes on a whiteboard (there were literally hundreds of them). Unfortunately, what management in this case did not do was use this brainstorming session in the context of the full project. There were no actionable items were created as a result of the session and no follow up meetings were held - contrary to the lip service given by those organized the affair. It was as if hundreds of good ideas and thoughts suddenly cried out in terror and were suddenly silenced. The voices of all who participated were just simply ignored.

This particular session ultimately did more harm than good. To this day, that brainstorming session still gets mentioned in the office in very negative connotations as a useless waste of time and no one today wants to engage in any formal brainstorming sessions out of distrust from the original. This is due to the fact there was no follow-through - the brainstorming was not done in the larger context of a plan. Had the ideas used in that brainstorming been used for actionable items immediately as part of our project plan, two things might have occurred: 1) the rest of project may have gone much more smoothly (it didn't, no problems were addressed) 2) the folks doing the work would have had confidence in the brainstorming activity and would have engaged in more.

Things - Part 2 of How I Get Things Done

This entry expands on my previous post on managing communication to Get Things Done (GTD). In that post I stated

I set the timer for 48 minutes and kick it off and focus solely on items in my GTD list - nothing else exists in the world if I can help it. I ignore all non-urgent electronic communication (anything mentioned above and email) and I only communicate with others on purpose if its relevant for the task at hand that I'm working on. What is considered urgent or not is often for me to decide on the fly.

That sounds fine and dandy - but just how does my GTD list get populated? I basically try to follow David Allen's instructions (at least those outlined in the first three chapters or so) to a tee. This includes having a system that I can trust that I put every actionable item I need to take in it. The system that I trust is a piece of software called Things created by Cultured Code.

It is essentially setup the same way that Allen instructs well. Any actional item that I encounter that I can't get done in a *very* short amount of time (2-5 minutes) typically gets placed in my Inbox so that I can decide what to do with it later. For the entire time that I'm working, I am focused solely on those tasks that I have given myself in the Today section, giving higher priority to those makred as Due on that day. Every single item is assigned to an Area of work (e.g. work for my Employer, Personal Projects, Housework, Developer Communities, etc. etc.) and within those Areas can be projects - which are items that require two or more actionable steps to complete.

I have found that using this system has had two effects on me:

1. I'm much better organized and prioritized in my daily work. Organizing all of my actionable items in such a way allows me to focus on one thing at a time knowing that the sequential order based on priority has already been taken care of for me, thus negating my ADD.
   
2. My mental burdens are very low - I no longer obsess about work or what I haven't done when I'm not working because I have my things to-do ready for me when it's work time again.

The book and the software itself comes out to around $70 total, but given the boost in productivity and quality of work/life if its had on me, $70 is one hell of a deal.

Mysql Backup Script

[Note: This script was not written originally by me. I am just pasting it for others to consume.]

Here is a quick and nice mysql backup script that one can add to their crontab to produce nightly mysql backups. It is not completely turnkey as you need to edit some of the variables (or write some code and pass some arguments such that they can be specified on the command line) and also have a dblist.txt file ready with the names of the databases you wish to backup.

#!/bin/bash

# Backup mysql databases

# Set a value that we can use for a datestamp
DATE=date +%Y-%m-%d

# Our Base backup directory
BASEBACKUP="/folder/to/store/mysql/backups"

for DATABASE in cat dblist.txt
do
       # This is where we throw our backups.
       FILEDIR="$BASEBACKUP/$DATABASE"

       # Test to see if our backup directory exists.
       # If not, create it.
       if [ ! -d $FILEDIR ]
       then
               mkdir -p $FILEDIR
       fi

       echo -n "Exporting database:  $DATABASE"
       mysqldump --host=localhost --user='YourMySQLUser' --password='YourMySQLPW' --opt $DATABASE | gzip -c -9 > $FILEDIR/$DATABASE-$DATE.sql.gz
       echo "      ......[ Done ] "
done

# AutoPrune our backups.  This will find all files
# that are "MaxFileAge" days old and delete them.
MaxFileAge=30
find $BASEBACKUP -name '*.gz' -type f -mtime +$MaxFileAge -exec rm -f {} \;

OSX Java Update Surprise

On Friday of last week I installed Apple's latest Java update to Leopard, Update 4, which gives us 1.6.0_13 of Java. This patches a pretty critical vulnerability in Java that Apple was lagging behind on addressing in their repackaging of Java.

After installing the update, I notice that my version of Java, as reported by `java -version` was a 1.5 version, not 1.6. However, the Java version I set per Java Prefrences.app was indeed 1.6. I did not muck with any of the symlinks found within /System/Library/Frameworks/JavaVM.framework and there certainly wasn't a java bin directory on my path.

Turns out that Java on OSX now honors your JAVA_HOME environmental variable, which I did have set in my .bash_profile file. I originally had this set to Java 1.5 for a project I work on that requires a 1.5 JRE (the bundled version of JAXB in 1.6 conflicts with a version of CXF we depend on).

This is kind of neat actually. I no longer have to open a gui app to change the version of Java I am using... I can use my ~/.bash_profile file as I always have on any other system. Here are the relevant contents of my ~/.bash_profile file:

#export JAVA_HOME=$(/usr/libexec/java_home)
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
#export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Home

I basically leave the version of Java I want to use uncommented and then `source ~/.bash_profile` to make that take effect. If I wanted to actually use the Java version that takes priority via what's set on the Java Preferences.app utility, I would uncomment the first line - /usr/libexec/java_home simply returns the directory of the currently active Java version set by Java Preferences.app. I personally chose to forgo that route since it requires multiple steps and opening that utility instead of just allowing me to simply edit my ~/.bash_profile and re-source it.

Managing Communication - Part 1 of How I Get Things Done

This post was prompted by another blog post by my friends over at Mystic Coders. In short, it discusses the topic of communication overload. This post is simply a re-edit of a comment I made there. I will base a series of posts off of the general subject of how I Get Things Done (GTD) of which this post will be the first.

I have been keeping this thought inside for quite sometime, only ever preaching to choir when a certain style of communication annoys me. Here is that thought for the rest of the world to consume...

EMAIL IS NOT INSTANT AND SYNCHRONOUS! STOP USING IT AS SUCH!!

We live in a world where friends, family, and coworkers are all constantly sending each other emails and somehow the protocol was established that replies and threads must be replied to and read in an almost instantaneous fashion. The coworker bunch is the worst offender of this. My favorite example of this is from a former operations director who would send a (non-critical) email to you and within five minutes later he would be at your desk to make sure you read it. That, my friends, is a double-whammy to The Flow. Unfortunately, everyone seems to have forgotten that the final 80% of characters in the word "email" spells "mail" - a communication mechanism that we all find acceptable to check once a day and let pile up when we are out of town.

Of course, given the world of constant interaction that we find ourselves in now, it would be pretty implausible to just trivialize email the way I just did by comparing it to snail mail. It wouldn't work practically in some organizations. Given the fact that I work remotely, I really would have a hard time eschewing it.

Of course, email isn't the only distraction that we face. I personally engage and benefit from using 4 other electronic communication mechanisms:

• IRC
• IM
• Skype
• Twitter

If one is not careful in their usage of any electronic communication protocol, it can become a timesuck just like, if not bigger than, the barrage of email piling up in your inbox. Of course there's always those interactions you must and will have with people on the phone and in person as well.

So... here's what I do to circumvent this. For the part of my days that I've partitioned to Get Things Done (aka work), I put myself on a timing system. I'm on a Macintosh and I use the simple but functional Apimac Timer application. I set the timer for 48 minutes and kick it off and focus solely on items in my GTD list - nothing else exists in the world if I can help it. I ignore all non-urgent electronic communication (anything mentioned above and email) and I only communicate with others on purpose if its relevant for the task at hand that I'm working on. What is considered urgent or not is often for me to decide on the fly. Roughly, if it's direct coworker who is pinging me via a synchronous communication protocol (IM/IRC/Skype) or in person or on the phone about something important to my tasks or their tasks and I'm not in The Flow, I'll pause to answer them; otherwise... they must wait. I do not even look at my email window at all unless a high priority email came. After the timer finishes and the 48 minutes is up, I reset it to 12 minutes, and allow myself to check email, irc, hit the bathroom, read a random article from my rss feed, and so on. After the 12 minutes is up, I'm back to focus mode again for 48 more minutes... Rinse, and repeat.

This 4:1 ratio allocation of one hour blocks of time has the triple effect of allowing me to:

• synchronously communicate and email enough when it is warranted
• concentrate in almost 50 minute blocks of time - long enough to establish and keep a flow on short tasks
• take enough mental breaks from tasks during the day so as not to be drained by 2PM
  

I only ever void this schedule if the work I'm doing is either super critical, if I'm really in the zone and it doesn't seem like a need to take a short mental breather, or if I've scheduled an event previously (a meeting, or phone appointment, for instance).

To help me stick to this schedule the I do the following things with my email and email client which delivers the communication that is often the nosiest and responsible for the biggest chunks of time that could potentially be wasted

• Turn off pop-up windows and noises that occur when ANY new messages arrive
• Create a filter rule that automatically gives a low priority status to any message where my name, or one of my distribution lists/groups are not in the TO: line. Low priority emails are read at the beginning of the day on the next day.
• Keep my own email messages under 3 sentences, unless the piece of communication really deserves more. Unless an email message will contain a moderate amount of technical detail, it must be typed in 3 sentences and under 5 minutes. Otherwise, it probably ought to be communicated through some other means.
• Bounce the dock icon on my Mac if there is a high priority email - I allow myself to at least skim those at any time to determine if there is an action item I need to perform right away. This form of notification is subtle enough to not distract my eyes until I've come to an intermediate stopping point in whatever it is I'm doing (e.g. finishing writing a statement in a piece of code).
• Strive to keep the Inbox at 0 unread messages... I never want to have the feeling that I'm behind or missing anything.
  

What I've found in applying these techniques to email is that the signal/noise ratio of the email I check constantly is very acceptable. I probably only read about 10 during the normal workflow of each day, as I only permit myself to read/write them once per hour. Ignoring emails you are CC'ed on until the next day also has the side effect of making you resist the urge to peanut gallery on threads you were merely CC'ed on - thus preventing (or at least keeping in check) the negative feedback loop of everyone trying to shove their opinion and thoughts in to a gigantic email thread.

With the other communication protocols, I just play it by ear. I don't want to eliminate folks from being able to find me, at will, on synchronous communication protocols unless I really need the focus. I do nothing fancy with them outside of turning off growl and setting my skype to DND status (which prevents me from seeing when others call).

The end result of all of this is that I now probably spend about 1 hour a day on communication during my work time instead of the 3 hours or so I estimated I did before - and that's an extra 2 hours a day one can devote to doing the real work that one needs to do to stay productive and valuable.

What are some things that you do to help facilitate getting in The Flow and keeping your communication avenues available but without getting in your way?

Ivar Jacobson's Smart Practices

Here's a neat little article that provides several quotes from Ivar Jacobson regarding current Agile practices and software methodologies in general.

The really cool part is the juxtaposition of what Jacobson considers to be smart and un-smart practices - and I tend to agree with him. Here are the top three I can pick from that list (in the order they came in at that list).

• Unsmart with Projects – Trying to follow a waterfall approach
• Smart with Projects – Build a “skinny system” to demonstrate that you have eliminated all the critical risks, then add more capabilities on top of that skinny system as needed.
  
  “Think big, build in many steps”

I work at a shop that essentially uses a waterfall approach. Nothing is really done iteratively with very little to no prototyping - outside of a rare alpha delivery here and there. What unfortunately ended up happening was lots of debate and wasted time with up front planning, no one group owned the full architecture (as there wasn't buy-in up front), implementers of a project shoved everything out in one go using the same technologies that were selected up front without knowing, through the use of a prototype, whether they were the right tools for the job. Time then gets wasted during the "death march" period fighting problems that could have been identified up front with a prototype.

That basically results in two instances of wasted time and contention - once up front because everyone tried to get the implementation correct on the whiteboard in their own way instead of in code, and a second time because everyone ends up dealing with a design problem during the construction process because the design wasn't correct. This has been written about at last as far back as Steve McConnel's first edition of Code Complete in 1993 - and probably even further back than that. It seems we haven't learned much as an industry since that point if well known names in the software industry are still having to give talks decrying the waterfall methodology as a common pitfall in today's practices.

• Unsmart with Testing – having two classes of people – developers and testers. Unsmart projects testers are “the cleaners in the software world” – picking up the mess left by the developers
• Smart with Testing – the whole team is jointly responsible for quality and testers are first-class citizens
  
  “Whatever you do, you are not done until you have verified that you did what you wanted to do”

In a shop I've worked at, I have seen the "Unsmart with Testing" scenario crop up as well. I've heard straight from the mouths of developers that "our job is to write code, we don't need to test our code at all, that is QA's job", to the point that 10 minutes would not be spared for a member of that time to run a quick smoke test on a build before it was delivered to QA. Additionally, no form of automated unit testing was being performed by this team, essentially leaving all formal testing up to QA folks. This is the epitome of not owning your own work. It also did not help that at this shop many QA folks were not especially savvy at describing and reporting bugs or helping developers find issues. It was the stereotypical example of a place where bugs were just tossed back and forth over a very thick brick wall with no communication.

The point here is that developers should own the testing of their code and QA folks should be proactive in helping resolve bugs. To provide concreteness to that assertion - developers need to 1) write automated tests for their code up front 2) write automated tests that reproduce bugs deliver by QA 3) take at least some time to work with QA members on the resolutions of bugs. QA folks also need to 1) make the developers lives as easy as possible by not only owning the reporting of bugs, but by helping see them to resolution b) provide concrete steps and extremely detailed pieces of information in every peice of communication given to developers - "It doesn't work" is always the wrong piece of communication in this case.

• Unsmart with Documentation – slavishly filling in a document template because some process rule says it has to be there
• Smart with Documentation – recognize the “law of nature: people don’t read documents”. Document only what is absolutely needed
  
  “Focus on the essentials – the placeholders for conversations – people figure out the rest for themselves”

I'm wary of this, as I think documentation of any form is better than no documentation than all (which is debatable). However, I feverishly oppose enforcing documentation by template any chance I get - if the documentation writing is manual. The only documentation that ought to be placed in a template needs to be something automatically generated - such as bug resolution reports from an issue tracker, or something like javadoc or maven site generation.

Beyond that, documentation should be treated like like creating the software itself - make it an iterative process. Document only what is necessary and when the gaps need to be filled in, they get filled in. This is why wiki's as documentation are a powerful thing. Developers need to spend time in their tools and in conversation with other co-workers, not time in Microsoft Word or whatever the editor of choice is.

What are your thoughts and takes on Ivar's list?

Continuous Deployment - Neat Idea, but not generally applicable

Continuous Deployment, as I interpret it from this particular article, is basically automating your production pushes to come straight from your Continuous Integration environment with NO human intervention in between. I will leave it to you, the reader, to check out the full article before continuing reading my post.

To paraphrase the author, this method of development involves building and testing all commits in your Continuous Integration environment followed immediately by an automated deployment in to your production environment automatically, and that such a methodology actually encourages very strict and rigorous automated testing and discipline from developers. I believe him. It is a lofty goal and is an extremely neat concept. There are cases where I'd probably see myself recommending and implementing such an approach - think of a smaller start up with limited resources whose production systems don't necessitate 24x7 availability or correctness according to a specification (perhaps Twitter in the early days?).

However, in the general case, I don't know if Continuous Deployment is such a wise goal - at least to a production system. I for one don't even encourage it in my own shop to our development environment since I want to maintain some semblance of stability in that environment where deployments are timed and roughly scheduled simply due to the number of disparate groups who rely on them for testing and working on their own components and systems (this is a side affect of an SOA architecture). Continuous Integration encourages constant integration somewhere, which means deployment ought to be happening somewhere for automated integration testing - to me that environment shouldn't be to your production environment. I for one think that automated deployment followed by automated integration testing should be segregated from live environments consumed primarily by humans, be they developers, testers, business users, or external users/customers. Most organizations have environments containing specific versions/combinations of their systems, each with different roles according to their development process. Updating one automatically at the point in which any developer makes a commit is too loose for an organization of larger size with many systems and many teams where there is usage and testing (manual or not) of a given environment going on seemingly at all times.

That's not to say that deployment shouldn't be automated... it should. But the ability to deploy, in the general case, should be held in escrow by a human to at least enact a deployment on a push button basis. Otherwise, I believe too much burden (that being almost all) falls on the developers for owning deployments to an environment - a task they should be mostly decoupled from.

There is something else that rubs me wrong about Continuous Deployment. There aren't multiple validation factors being applied to the sanity of a given build. Automated testing, be it unit/integration testing at build time or monitoring via an alert system (e.g. Nagios) is indeed quite awesome, but it shouldn't be the only check. Automated testing, much like manual testing, is fallible because it too is ultimately orchestrated by humans (someone has to write those tests!). One method of testing by itself can't be trusted - I have seen it all too often where an automated test picked up what QA guys did not catch (or picked it up much more quickly) and when QA guys spotted failures that the automated tests did not account for. Going full hog on Continuous Deployment gives you a single point of failure on testing, and in my honest opinion, is no more reliable than manual human testing by itself where correctness is concerned.

Again to summarize I think Continuous Deployment to production is a neat idea and applicable in certain cases - but I wouldn't suggest getting all hot about it unless you are willing to accept the risks it might bring with it to your organization.