Agile Thinkers

Welcome to AgileThinkers

Subscribe to this blog to learn more about the Agile communities many great thinkers.

Jim Shore - Q1

Jim Shore is my first guest on the AgileThinkers blog.  He's just co-authored a wonderful book "The Art of Agile Development" along with Shane Warden.  Shane's also going to answer the questions but I'll be posting them separately.  We're going to be giving a way a copy of the book shortly.

Q1.  Hi Jim,  I'm really enjoying your book.  I honestly think it is the best agile book I've read - and I've read a good few of them.  Good job!  Can you tell me a bit about yourself - both personally and professionally?  What got you to the point where you could produce such a wonderful book?

Thank you!  I should start by recognizing my coauthor, Shane Warden.  Shane and I collaborated closely on the book, so that's part of the reason it turned out so well.  Another reason would be the sheer amount of time and effort we put into it--I put nearly a year of full-time work into it myself, and that's not counting Shane's effort or the effort of all the people at O'Reilly.  I think our open review process helped a lot, too.  We put nearly every section of the book online and we got tons of feedback--over 1200 messages were posted to the reviewer list. Our best work started as drafts that got brutal beatings.

So: blood, sweat, and tears.  You know, the usual.

As for how I personally got here... you know, it's hard to say.  I've always liked reading and writing, even though I was born a hardcore geek.  (I placed in my first programming contest at age ten or eleven.  I think I would have come in first if my program had actually loaded off the tape drive.)  I actually did better on the English portion of the SATs than the Math portion.  I also wasted a huge amount of time on Fidonet and Usenet, talking about programming and (in retrospect) being a general pain in the ass.  I could spend hours composing careful explanations of why OS/2's Worksplace Shell was better than the Windows desktop.  I expended far more effort than the topics were worth, except for two things: I had fun, and I had a lot of practice writing and seeing how my writing was received.

So that's how I learned to write.  On the software side, the really core experience was when I had an AppleSoft BASIC program dissolve on me.  Applesoft BASIC was old-school: line numbers, two-letter variables, GOTO's, the works.  When I was in high school, the complexity of one of the programs I was writing overwhelmed my capacity to keep track of it all.  The program just fell apart in my brain.  Ever since, I've been fascinated with the human side of programming: if programming is partly an exercise in making the unbelievably complex comprehensible, how do we do that?  That led me to structured programming, then OOP, then software engineering texts, then Ward's Wiki, then XP and agile development, and now to questioning what makes software successful.  Along the way, I did the same thing I did with writing: I spent hours every day just playing with the ideas, writing programs, writing about programming, and so on.  (As well as my professional experience.)

So really, the answer is that, for most of my life, I've had a pretty single-minded dedication to writing, programming, and software development.  (The word you're looking for: Nerd.)  It's amazing I ever married, let alone reproduced.

Jim Shore - Q2

Q2 - One of the things which annoys me most in the agile community is when I hear people saying things like "That's not pure agile" when what they really mean is "That's not XP, like I read about in a book".  For me Agile is about getting better and XP is just one of the  ways to do so - albeit, a very good way!  I recall that you received a bit of flak while we were reviewing your book because the focus was on XP, but the book title indicated it was about Agile in general.  In retrospect I think you made the right choice and you've explained your reasoning very well in the book.  Could you share your reasoning here?

Sure.  We wanted to make a book that was very practical and useful, but we didn't want to fall into the trap of watering down agile development.  In my consulting work, I've seen that when people pick and choose agile practices, they usually pick the ones they like (iterations, mostly, and sometimes test-driven development) and ignore the ones they don't like (pair programming, sitting together, cross-functional teams, evolutionary design).

That's not really that surprising, is it?  People only choose practices they like--big insight!  But the problem is that they're leaving out essential practices without realizing what they're missing.  It's okay to leave a practice out, but you have to replace it with something, and they're not.  Then they have trouble with technical debt and making reliable plans, but don't know why.  After all, what do pair programming and on-site customers have to do with estimating?  Everything, as it turns out.

Now, we agree that each team needs a customized approach to agile development.  The problem is that teams new to agile make poor customization choices.  So we decided to describe a single method, with a fairly limited set of choices, that will work well for most teams. (And we describe when it won't work well.)  The idea is that the team can follow the method we describe, practice it a bunch, talk about their experiences, learn from it, and then use that experience to make good customization decisions.  One of my favourite aspects of the book is our inclusion of a third part, designed to be read months after the rest of the book, that talks about customizing your agile method.

Shane Warden Q1

Shane Warden is co-author of "The Art of Agile Development" and author of the Extreme Programming Pocket Guide (and others, going by the name "chromatic").

Q1- My first question is the same as I asked Jim,  I'm really enjoying your book.  I honestly think it is the best agile book I've read - and I've read a  good few of them.  Good job!  Can you tell me a bit about yourself - both personally and professionally?  What got you to the point where you could produce such a wonderful book?

Like Jim, I started programming when I was very young.  I must have been six or seven years old when I saw my first minicomputer, and I sat down, opened the manual, and typed the first program I came to.  (For some reason I ignored the line numbers in the book and typed the program directly, which didn't exactly work.  I suppose some of us learn to debug early.)

For a long time, nothing happened.  My undergraduate degree is in music, but I realized that I wanted something with more stable job prospects, so I returned to technology after I graduated.

During a stint as a system administrator as the dot com era heated up, I rediscovered programming and spent a lot of time reading the source code of free software projects, trying to improve my skills and discovering that successful projects had development processes very different from those I was seeing in our corporate IT group.  When I read the first XP book, I realized that there were important commonalities I'd seen in both, as well as practices that were missing (in particular, refactoring and test-driven development).

Through the next phase of my career (consultant, writer, software developer) I continued to explore the ideas of agility especially as they related to software quality.  I wrote a lot of tests, test frameworks, test tutorials, and testing advice, and I started to see how adopting even just one or two practices improved software development.  I also saw first-hand how adopting just one or two practices without considering their context made it difficult to succeed in software development.  (In particular, the lack of stakeholder support can doom almost any project.)

I wrote the XP Pocket Guide for O'Reilly, and in the year of research and writing (and admittedly arguing with Jim occasionally about how strict or lenient to be and how people were likely to pick and choose what they wanted to do) I really understood how everything fit together.  If you look at the Art of Agile Development, you'll notice that almost every practice we discuss has one or more contraindications.  Sometimes you really can't practice the practice precisely the way we recommend, and that's fine -- but you need something besides a tremendous amount of luck to fill in the gaps left behind.

Shane Warden Q2

Q2 - One of the things which annoys me most in the agile community is when I hear people saying things like "That's not pure agile" when what they really mean is "That's not XP, like I read about in a book".  For me Agile is about getting better and XP is just one of the  ways to do so - albeit, a very good way!  I recall that you received a bit of flak while we were reviewing your book because the focus was on XP, but the book title indicated it was about Agile in general.  In retrospect I think you made the right choice and you've explained your reasoning very well in the book.  Could you share your reasoning here?

My very high level goal is to have better software and a better environment for people involved in creating and using software.  When I work on a project, I want a pleasant work experience with high morale, fulfilling work, and personal and professional satisfaction.  I think agile development helps achieve that, and I think XP is the most effective and cohesive approach within the agile universe, but if projects achieve organizational and personal success without strictly adhering to what one book or website or consultant or another says, I count that as success for the team.

What we're trying to do is achieve organizational success as well as personal success.  Jim and I tried to explain what we think is the best way for a team fairly new to agile development to start achieving that success.  Ultimately we believe that teams who follow our advice and eventually start changing the process to meet their own environments more effectively will have mastered agility, even if they end up doing certain things very differently from how they started.

We did (and do) feel very strongly that we have a responsibility to give the best advice we have, though, and we tried to do that even if it wasn't always fun or pleasant.  For example, a lot of people hate the idea of pair programming, and so I occasionally see a lot of agile advocates back away from recommending pair programming.  That's a mistake; it's a powerful practice that has many benefits you can't get in gestalt anywhere else, not to the same degree.

With that said, there are ways to make pair programming more difficult and ways to make it less difficult.  I believe we explored the discussion honestly, but our recommendation is that no matter how awkward it seems for the first few days, if you can, you should try it for at least a month and measure carefully how it affects your work.

Shane Warden Q3

Q3 - What has been your most satisfying Agile experience?

I think this story's in the book, near the end.  A good friend of mine has been skeptical about agile development and testing and pair programming for many years.  She's actually a great reviewer of drafts and manuscripts because she's honest and analytical and asks good questions.

We sat down a couple of years ago to try pair programming for the first time. She had said "I don't think I'm ever going to understand this deeply unless I do this with someone who already knows how."  We spent a couple of hours ping-ponging back and forth, where I'd write a couple of lines of a test case and she'd write a couple of lines of code to make the test pass, then talk
about any refactorings, and then she'd write a test and I'd write some code. She didn't care for that at all.  She kept saying that it felt like we weren't making any progress, like we were taking baby steps and doing screechingly obvious things.

I won't say that that project was much of a success; we managed to get in about 90 minutes of pairing and could have gone another hour or two to start seeing really great results, but when she went back to work the next week, a handful of other people suddenly asked her to show them how pairing worked, and she had some experience to draw on.

Maybe you want a more successful experience.  I have to say that every time someone asks me to write some code and hands me a failing test, I feel plenty proud.

Shane Warden Q4

Q4 - What has been your least satisfying Agile experience?

My last full-time software development job before I wrote the first book was underwhelming.  I had a couple of great co-workers, but our development process was... ambitious and underwhelming at the same time.  We used a mixture of Scrum and XP where we dutifully broke up all of our work into stories and engineering tasks, estimated the necessary work, let our tests drive our design and refactoring, and did pretty well at all of the development practices.  (I wish we'd paired more, but....)

The problem was that it was a very small consulting shop with lots of different small customers to satisfy, so we'd work on multiple projects every week. We did have a single grand project to work on, but only when we had had enough billable work for the month for us to spend time on a new product. Mostly we lacked a grand cohesive vision for our work, at least one that could predict what we should work on at a high-level on a weekly basis.

It wasn't an unpleasant experience -- I do remember favorably the experience of customizing a F/OSS point of sale system to the point where our largest customer could use it successfully in a month's time -- but it was unsatisfying because we had to spend so much time just staying in business that we didn't have as many opportunities to succeed at our business. Despite that, our software quality improved dramatically and we were able to meet the needs of our customers much better, thanks in part to much better development practices.

Jim Shore - Q3

Q3 - What has been your most satisfying Agile experience?

My favorite aspect of agile development is the collaboration and camaraderie that builds up over time.  The really great teams have this intense energy to them.  People laugh and joke and really have a lot of fun.  The work fires on all cylinders and there's this feeling of immense productivity.

I love that feeling.

I first experienced a hyper-productive team with my second XP project.  I first tried XP out on a small project (three developers) and had moderate success.  The manager asked me to come coach a larger dev team and I convinced them to give XP a try.  We worked together for nearly a year, and then I moved out of state, set up a satellite team, and we continued working on the same codebase for another six months.

That project was probably the most fun I've ever had at work.  We all got along very well.  Most of the team would go out for lunch together every day.  We pair programmed, wisecracked, and produced some of the best code I've seen.

The code didn't actually start out all that well.  It's not like we were super-geniuses or anything.  A lot of it was my fault: I didn't totally trust XP's ideas of evolutionary design, so I worked with one of the leads to create an up-front architecture.  It was terrible!  We didn't know that, of course.  But we gradually improved it, and by the end of the first year we had a nice, elegant abstraction.  That project taught me that code can actually get easier to modify over time, not harder, so long as you commit to constantly improving it.

(I wrote a paper about the experience of constantly improving design for the first XP Universe, here: http://www.xpuniverse.com/2001/pdfs/EP203.pdf.  That was seven years ago... I'd probably be embarrassed to read it now.)

Jim Shore - Q4

Q4 - What has been your least satisfying Agile experience?

Well, just as when it's great fun to have everything firing on all cylinders, it's very frustrating when the team struggles, particularly when you've had the opposite experience.  For me, that's usually because people dismiss an idea, like pair programming, without trying it.  They just decide that it can't possibly work, and that's all there is to it.  Also, because I'm a consultant, some people form this instant dislike of me before I even walk in the door.  You've seen the Dilbert cartoons: the consul"tick".  That's me.  It can be disheartening.

One of the worst moments for me was during the .com crash, shortly after I went independent.  Work dried up, so I found a job through a headhunting firm and ended up as an "in-the-trenches" programmer on a web app in bug-fix crunch mode.  I ended up getting them to try agile development (after about six months of soul-destroying effort), but it didn't work out.  That project taught me the importance of making sure a company is ready for agile before convincing them to try it.  In retrospect, I was pretty arrogant in my approach, too.  (What can I say--excessive humility has never been my problem.)

I wrote about that experience in my "Change Diary"--on my website at http://jamesshore.com/Change-Diary/.  It's one of the most popular pieces on my site.

(There's a silver lining to it all.  That company has now switched over to Scrum+XP, at least in part, and they're apparently having success with it.)

Shane Warden - Q5

Q5: Can you tell me something odd, unexpected or just plain fascinating?  Something your friends know ...

Jim knows this, but not everyone does.  My undergraduate work was in music, specifically performance, with a smattering of history and almost zero mathematics or science.  It's not that I hate either math or science (I don't), but I decided against studying engineering the year before college and went the other direction to the arts and things you can't prove with the empirical method.

Once in a while I do wish that I'd taken a compilers class though.

Q1 - Johanna Rothman

Today I kick off a series of questions and answers with another big-hitting agile thinker Johanna Rothman.   Johanna has recently published the excellent Manage It! and she'll be giving away a copy as we near the end of the questions.  She also writes one of my favourite blogs.

Q1: Hi Johanna, I know you mostly through your (excellent) blog and your other writing.  Could you please tell us a little more about yourself - both personally and professionally?

Okay, this is a tougher question than it should be :-)

Personally, I'm married to a wonderful guy who can actually fix stuff around the house. When I "help," he always asks me if I've read step 2, because I tend to gloss over some of the early details in my impatience to get to the meat of the problem. We have 2 daughters, 19 and 15. I no longer know anything about small children, but I know a lot about teenagers :-)

Professionally, I have a BS in Computer Science and an MS in Systems Engineering. For you BAs, that MS is about how to develop systems, not how to write requirements. I started working professionally in 1977 as a developer. About 8 years later, I became a tester for a couple of  years, and then focused on project management, program management, and people management. I write code for fun still, but it's definitely not production quality code. I started my consulting business Labor Day of 1994. (For my non-US readers, that's early September, 1994.)

Q1 - Johanna Rothman

Today I kick off a series of questions and answers with another big-hitting agile thinker Johanna Rothman.   Johanna has recently published the excellent Manage It! and she'll be giving away a copy as we near the end of the questions.  She also writes one of my favourite blogs.

Q1: Hi Johanna, I know you mostly through your (excellent) blog and your other writing.  Could you please tell us a little more about yourself - both personally and professionally?

Okay, this is a tougher question than it should be :-)

Personally, I'm married to a wonderful guy who can actually fix stuff around the house. When I "help," he always asks me if I've read step 2, because I tend to gloss over some of the early details in my impatience to get to the meat of the problem. We have 2 daughters, 19 and 15. I no longer know anything about small children, but I know a lot about teenagers :-)

Professionally, I have a BS in Computer Science and an MS in Systems Engineering. For you BAs, that MS is about how to develop systems, not how to write requirements. I started working professionally in 1977 as a developer. About 8 years later, I became a tester for a couple of  years, and then focused on project management, program management, and people management. I write code for fun still, but it's definitely not production quality code. I started my consulting business Labor Day of 1994. (For my non-US readers, that's early September, 1994.)

Q2 - Johanna Rothman

Q2: Could you tell us a little about each of your books?

• Hiring the Best Knowledge Workers,Techies & Nerds: The Secrets and Science of Hiring Technical People is about how to hire, from developing a hiring strategy to the first day and what to do if you can't find the right person. I wrote it because I've been fortunate enough to hire 100 or so people when I was a manager (and live with them in the organization). I also wrote it because I realized when I did assessments that a lot of the problems I saw in organizations was that nice people were in the wrong jobs.
• Behind Closed Doors: Secrets of Great Management I pair-wrote with Esther Derby. We saw managers failing at their jobs--not even completely failing, but missing significant portions of their jobs. If managers don't develop personal and trusting relationships with their staff (and their peers), they can't effectively manage. What a waste. So, we wrote the book.
• Manage It! Your Guide to Modern, Pragmatic Project Management is the book I've been writing for 10 or so years :-) I started writing articles about project management in 1997, and had been teaching project management since the late 80's, so it made sense to finally write the book. It's funny--some people have said, "It's a big book" but I couldn't write it any smaller. I'm very proud of this book, because I didn't include anything I haven't actually done on a project.

My next book is about managing the project portfolio. It turns out that a number of schedule games (I named 15 in Manage It!) arise from not managing the project portfolio, so I decided it was time to tackle that subject. I've been managing the portfolio in organizations and helping managers determine what they need to do as a consultant since the late 80's, so I have a lot of knowledge here too :-)

Q3 - Johanna Rothman

Q3: Of your books, which one is your personal favourite (for whatever reason)?

Come on! That's like asking which of your children you love more :-)

I'm very proud of Hiring the Best, because I learned how to write while writing that book. I'm thrilled with the pairing I did with Esther on Behind Closed Doors. And, I love Manage It! because although I prefer agile or incremental approaches, I can see when it makes sense to use other approaches, and I hope my readers will see that too.

Q4 - Johanna Rothman

Q4) Where would you send a seasoned PM who wants to know more about agile project management and software development?

 

First, I would suggest that the PM read _Manage It!_, because some seasoned PMs have funny ideas about Agile. Agile is the most disciplined approach to software, because the discipline arises from the team (with obstacle removal by the PM). But some seasoned PMs don't realize that the team's discipline is more important than the PM's discipline. With a disciplined team, the team can accomplish great things. But with just a disciplined PM, everyone is frustrated, and only the PM can solve problems. Not a scalable solution. Then, it depends on how the PM learns and the PM's experiences. A seasoned PM with an open mind might benefit from an Agile conference or a Scrum class. But in my experience, the PM will need to learn collaboration and team skills. The best place to do that is at the AYE conference (yes, I'm one of the hosts).

Q5 - Johanna Rothman - The Hudson Bay Story

Q5) I loved the Hudson Bay story in your book ... can you share that here?

Sure. Here's the quote from _Manage it!_ :"The Hudson Bay Start approach was originated by the Hudson Bay Company in the 1600--1700s in north eastern Canada. The Hudson Bay Company outfitted fur traders. To make sure the traders hadn't forgotten anything they needed, they left Hudson Bay and camped just a few miles away. By making camp just a few miles away, the traders ensured they hadn't forgotten any tools or supplies---before they abandoned civilization. With just a short start to their journey, they had a better idea about their ability to endure the winter.

On one notable project, the management team mandated a new computer, which necessitated a new compiler, a new build system, a new product with a new GUI, and most of a new team. The team had no idea how to make anything happen, so I suggested we use a Hudson Bay Start. I suggested we do "Hello world" just to see if we could. Of course, everyone laughed at me (sometimes that's the role of the PM), and it took us an entire week to write Hello world and nothing else to the screen. We celebrated at the end of the week, and now we had enough data to estimate what it might *really* take to do the work. Sure, we went a little faster as people learned how to make the infrastructure work, but the product was difficult to imagine and implement. Our estimates were surprisingly close for several months. That's the value of a short iteration. Learn by doing and use that data to predict the next iteration.

Q6 and Q7 - Johanna Rothman

Q6) What's the most fun you have in your job?

I love all of the consulting, teaching, and writing that's part of my job now. I love meeting all these people all over the world. I love doing assessments, and seeing how people are actually working. Writing about that so the leadership can choose to change--that's a challenge and I do enjoy it. I get a kick out of teaching and helping people learn new approaches. I really enjoy all the other consulting too: facilitating a retrospective or strategic planning or portfolio gathering and evaluation--helping people see what to do and helping them be successful--is such a blast. Writing is a different kind of fun. Most often, writing is solitary, so I enjoy it, but sometimes I wrestle with my brain to get the words out :-) When I do succeed, I feel great.

Q7) What's the least enjoyable part of your job?

It's sometimes difficult to help the accounting people at my clients live up to the terms we agreed upon at the beginning of the work.

Shane Warden - Q6

Q6.  How did you two work together when writing?  Are you geographically close?  Oh, and how did the two of you meet?  And how and when did you decide to write a book together?

We both live in the Portland, Oregon metro area, somewhere between 25 and 40 minutes by car, depending on traffic.  It was easy to get together in person if we needed to, but usually we ended up in the same place for other reasons (hanging out with the guys on Thursday nights, for example).

We spent a couple of weeks coming up with a really great outline as part of the book proposal process, and then did it again when it was clear that the book needed to change dramatically from our original vision.  That gave us several small sections for each chapter, and we divided those between us equally, where Jim would write the first draft and then I'd edit it, and vice versa.  Occasionally we spent an afternoon in person making notes and plans for more complex sessions.  For example, we took a couple of days to spread notecards around my living room to figure out the third section of the book when we realized again that our original approach wasn't quite right.

We originally met... well, Jim's girlfriend at the time (now wife) called my roommate (at the time) to invite him to Jim's 30th birthday party a few years ago.  I tagged along.  Somehow he found out that I was a programmer and I found out that he was a consultant and programmer and that was a few months before I started working on the XP Pocket Guide.  He seemed like a perfect candidate to review it.
A couple of years after that, he had the idea of revising the pocket guide.  I resisted for a while, but it eventually made sense, at least until we looked at the outline and said "This isn't a pocket guide anymore."

Shane Warden - Q7

Q7. If you could sum up Agile in 2 sentences ... what would it be?  What's the essence of agile to you?

Success comes from identifying and pursuing what's really valuable.  Agility means planning for and adapting to change.

Shane Warden - Q8

Q8: Are you planning on writing another book?

I'm halfway through my second novel, but there's no programming in it. There's talk around O'Reilly about working on a couple of other projects, but they're probably language-specific and not quite as abstract as software development.

Shane Warden - Q9

Q9: Do you have any further questions you'd like to ask yourself?

"Does that mean that there was programming in your first novel?"

It's still the only novel I've ever read with an action scene built around fixing a Unicode bug in an object-relational mapper while dodging robots and gun-toting librarians.  Take *that*, Dan Brown.

[Clarke: You should have called it "The DaVinci Coder"]

Jim Shore - Q5

Q5.  How did you two work together when writing?  Are you geographically close?  Oh, and how did the two of you meet?  And how and when did you decide to write a book together?

Shane and I have been friends for several years.  We both live in Portland and we usually see each other once or twice a week.  (A bunch of us like to get together and play German-style board games like Settlers of Catan.)  He also works for O'Reilly and has written several books, including the XP Pocket Guide, so naturally I thought of him when I wanted to write my first book.  Originally, my thought was that I could get my feet wet by updating his Pocket Guide for Kent Beck's 2nd edition XP book.  It would be a way for me to learn more about the world of book publishing without investing too much time or effort.  (How's that for an iterative approach?)

Of course, it didn't work out that way!  We ended up creating a completely different book, and it took a huge amount of time and effort.  I couldn't be more thrilled with how it's turned out, though.  The feedback has been almost universally positive.

In terms of how we worked together, we actually applied a lot of agile principles to working on the book as well.  I only wish we had done more.  We set up a Subversion repository for the book and wrote it using a markup language that we "built" using a Perl script.  We had a master chart that showed every section of the book, who was working on what, and how far along it was.  I kept a burn-down chart, too.  And of course we tried to involve as many "customer representatives" as we could to review the book and we posted sections for feedback as soon as they were completed, about once per week. 

Jim Shore - Q6

Q6: Are you planning on writing another book?

Probably, but I'm not sure what, yet.  I'm currently publishing bonus material for the book to the book's website (http://jamesshore.com/Agile-Book/) at one section per week.  That will probably take me a year to finish.  After that, who knows?  I have a lot of topics in mind, but nothing has asserted itself yet.  I actually thought my first book would be on evolutionary design, so imagine my surprise when we ended up with this book instead.

Jim Shore - Q7 and Q8

Q7.  If you could sum up Agile in 2 sentences ... what would it be? What's the essence of agile to you?

That's a tough question for me to answer.  "Agile" itself is such a nebulous term... outside of the manifesto, we don't really have a common definition of "agile."  In a way, that's good, because it lets everybody put their own interpretation on the term, and it lets all of these different methods (XP, Scrum, FDD, Crystal, DSDM, and so on) share the same tent. 

And, in a way, it's bad... because it lets everybody put their own interpretation on the term.  As "agile" gets more popular, I think we're going to see more and more big companies sell "agile" approaches that are just warmed-over versions of what they've always done.  The key word to run away from: "enterprise" agile, as in (insert cheesy announcer voice here): "Our pragmatic approach to agile takes into account the real world constraints of today's enterprise!  Use our enterprise agile toolset to manage your project plan and assign work to remote workers.  With our enterprise web testing tool, create tests in minutes, no programming required!"

Just now, I tried to write two sentences that defined agile, but I couldn't do it.  I can recognize an agile approach when I see it, but I don't think I could define it.  Really, I think, it's a philosophy... a way of looking at software development.  For me, it involves a rigorous yet informal approach.  A focus on delivering meaningful results. Respect for the individual, and appreciation for the fact that software development is about getting over-evolved chimps to cooperate with each other.

...you know what?  I'm going to punt.  In the third part of our book, we try to describe the essence of agile.  We encountered the same problem writing that part that I'm having now.  So rather than make up our own definition, we collected all of the philosophical statements (values and principles) that we could find in other agile books and we collated them into a giant affinity map.  We came up with fifteen principles that all agile methods seem to share.  I'll just list those out.  It's not two sentences, but it's the closest thing I've got to the essence of agile.

Improve the process: understand your project; tune and adapt; break the rules Rely on people: build effective relationships; let the right people do the right things; build the process for the people Eliminate waste: work in small, reversible steps; fail fast; maximize work not done; pursue throughput Deliver value: exploit your agility; only releasable code has value; deliver business results; deliver frequently Seek technical excellence

Q8: That was a rather serious answer!  To counterbalance it can you tell me something odd, unexpected or just plain fascinating?  Something your friends know ...

I mentioned earlier that my friends like to get together and play German-style board games. Well, one of those games is Settlers of Catan and we have most of the expansions for it--Seafarers of Catan, Cities and Knights, Das Buch, even the two-player card game.

We don't play it much any more, though, and that's because my wife always wins.  Every time.  No matter which variant of the game we play or who we're playing with.  She won't tell me how she does it, but I think I'm on to her.

It's mind control.  Yes, my wife is an alien.

Thanks to Shane and Jim - plus 2 copies of "The Art of Agile Development" to give away

Thanks to Shane and Jim.  As I said earlier I think that "The Art of Agile Development" is my favourite book on Agile.  I happily give away copies to my clients.  And now I feel like I know both of you just a little better.

You can, btw, read chapter 4 - Adopting XP - for free here.

But, if you would like to win a free copy of The Art of Agile Development then Shane and Jim have kindly offered up 2 free copies.  To win a copy email me now and tell me something which is vaguely interesting about something.  I'll pick the two winners at random at the end of March 2008.  Email me at clarke.ching@gmail.com.

Jared Richardson Q1

Jared Richadson is co-author (with Will Gwaltney) of one of my favourite software development books - Ship it!.  Just a couple of weeks ago I gave away a copy of Ship It to a client but - to be honest - I am hoping they don't read it.  If they do then they may not need to use me so much ...

Here's Q1 - Hey Jared, I thought Ship it! was a real keeper.  I've recommended it to many of my clients and friends. Tell me a bit about yourself - both your personal life (if you don't mind) and your professional life.  What makes you tick?

Let's see... big picture first. I'm married to a beautiful girl I met in high school, but couldn't land a date with her until I was in college. Not quite a high school sweetheart, but close. We have two daughters, ages 4 and 9. As Andy Hunt recently told me, it's not the time that kids take, it's the bandwidth. We live in Morrisville, North Carolina, which is in the middle of Research Triangle Park. We sit in the middle of the Raleigh-Durham-Chapel Hill area, which is to say, near the Duke Blue Devils, the University of North Carolina Tar Heels, and the North Carolina State University Wolfpack. Basketball country.

Both personally and professionally, I'm a tinkerer. I don't like being locked into a single task or job.

I've built water-cooled computers (from components, not kits), including an evaporative cooling tower. :) I'm running OS X, Kubuntu Linux, and Windows XP in my home office. I've got a MythTV box that's very close to pushing out the satellite dish. I have a canoe and a kayak that don't make it into the water very often. A few years ago I  wired my generator into my house's electrical circuit. I blew out a few appliances in the process, but that's another story. :) The lights over the kitchen table blowing out were spectacular!

After two years of consulting I joined a startup and I'm loving it. I get to tinker with architecture, write marketing literature, help with GUI redesigns, and sit in on sales calls. I've written Ruby code, set up Java application servers, and so on. I enjoy being at a small company where we all get to wear multiple hats.

What makes me tick? A problem I can immerse in for hours or days. When I'm working on this type of problem (right now I'm trying to find a completely different way to present the information our product collects and generates), I need to have dedicated time to dig into it, consider it, and bounce around different alternatives. I'll research alternatives on the web, scribble on white boards, mind map ideas on legal pads, and so on. I like to pull in other people to brainstorm and bounce ideas off of them... then I step away to another problem and let the first problem soak... (probably ferment!). But I keep coming back to the original problem. It always takes me longer than I want to solve these types of problems, which is frustrating, but when the problem is solved, it's very satisfying.

So I guess I'm task-oriented. I like a problem to solve.

I also like stepping back to see the bigger picture. I like to understand what the pattern is and why the problem occurred in the first place.

If we have to work overtime for weeks to get a release out the door, why was it that painful? We had no continuous integration system or test automation in place, so the product drifted into a brittle state. We spent too much time verifying our changes hadn't broken anything instead of adding new code.

Spotting (and fixing) these types of organizational patterns is what motivated Ship It! in the first place. It's really a book about these types of patterns you can use to map out your software shop. The startup I'm at (http://6sa.com) is gathering metrics about the way developers and teams work to spot some of these problems automatically. That's been an interesting challenge.

Jared Richardson Q2

Q2 - Tell me about the experience of writing Ship it!

Writing Ship It! was hard. Andy Hunt lives nearby and Will and I met him for lunch to pitch the idea. At the time I had a number of half- written articles on a Wiki that I was sure I could string together into a book in a month. Two months at the outside I told my wife. I think the book idea was the first one Dave and Andy accepted for the Pragmatic Publishing line (outside of their own books)... Mike Clark was right in there though. I'm not sure who actually signed papers first. But it took us a year and half to finish Ship It! My wife still reminds me that I thought I could finish it in a month.

The first problem we had is neither Will or myself had even written professionally before. We were developers. We thought we could write, but we couldn't. Andy invested a lot of time into us as not only an editor, but a writing coach. Being one of the first authors on the label has it's advantages. I doubt Andy would have time for authors like us today. But it paid off. Ship It! turned out very nicely and has been translated into Japanese, German, Korean, and even has an India sub-continent version.

The inspiration for the book is another nice story. I was having lunch with a friend and he was telling me about a project that was having problems. A short lunch stretched out well over an hour while I started scribbling out ideas on napkins for him. The topics ranged from the proper use of source code management to tracer bullet software development. Will and I had been lucky enough to work at two startups with very little process in place, so we (along with another developer named Jim Weiss) got to bring in the process. We had to gather (or generate) the ideas, sell them to the developers, then make them work. And this was the information I was sharing with my friend.

After lunch I dropped by Will's office and told him we had to write this stuff down. After a few brainstorming sessions we met with Andy, and the next year and half just sped by. :) Actually, there were times it sped by and times that an evening of writing felt like it took days.

The hardest part is not writing new material. The hardest part is going back over what you've written to tighten it up and make it worth reading. Rewriting so someone besides you understands the ideas, removing the 'extra' words, breaking up long passages with stories and humor. After you've rewritten and retooled a section a couple of dozen times, it starts to run together.

It was  great experience though. When I look at what I write now and what I wrote then, I'm embarrassed by what I thought was good writing. I've a few books "in my head" for the next book. I just have to carve out the time to write them. It was a great experience and I plan to write more books.

Jared Richardson Q3

Q3 - Am I right that you used to work for SAS?  If I aren't then ignore ... I've heard some fantastic things about them as a company ... what did you take away from your time there?

The take away from SAS is pretty straightforward. Your people are your greatest resource. If you work hard to keep your people happy and productive, then the software is a by-product. Dr. Goodnight makes it a priority to keep his teams happy and that shows. I worked at the corporate headquarters with 5,000 other people and they had (if memory serves) a 3% turnover. For a software company that's incredibly low. Most people just don't leave SAS. Look at your company's training budget (for new hires) and ask yourself what you could do with that money to keep your existing people to stay around. Avoid having to retrain so many people.

But what you've heard about SAS is true. There were M&M's in the break rooms. The walking trails in the woods are paved. There was an indoor pool (attached to the gym), a daycare (or three), doctors, hair stylists, massage therapists.... I could go on. They try to keep you on campus for things like haircuts and doctors visits. This is a huge convenience to the employees, but it also gets you back at work sooner. Everyone wins.

Q1 - Chris Fry Salesforce.com

Chris Fry is the instigator of one of the agile worlds must credible and newsworthy (in my opinion) Agile Success Stories ...

Q1: Hi Chris, We first met a couple of years ago when I reviewed/shepherded your submission to the Agile2007 conference. To be honest I don't think I added a lot to your submission - you had an excellent story and you told it well.  Before I ask about your story can you tell us a little about yourself - your personal and professional background?

I am a Senior Director at salesforce.com where I lead the platform development team.   I've worked at a startup and in enterprise software in various roles (architect, engineer, manager).  I have a Ph. D. in Cognitive Science from UCSD and have always enjoyed cross functional work in academics and industry.   I live in the Bay Area and surf on the weekends.

Q1 - Alistair Cockburn

I really need to introduce Alistair Cockburn do I.  You think you know him?  Wait 'til question 3 ...

Q1. Hi Alistair, Tell me about yourself, especially, how did you get to be so damned brilliant? I don't say that lightly but some people I know go all gooey eyed when talking about your work and I reckon your approach to writing Use Cases gives sliced bread a run for its money. What's your story? What got you here? And, what does your wife think of all this agile stuff?

That's one question? Can't wait to see the rest!

Many thanks for the kind words ... There are enough people around me who think I'm incredibly stupid that maybe they sort of offset each other – kind of like the man with his feet in the freezer and his head in the over but who is, on average, at a comfortable temperature.

One of the things fairly unique about my background is we moved around a lot – My parents were English and wanted to get out of England after WW II, their goal was (a) to get far away from England, and (b) to travel the world. They dropped kids, in succession, in London (in the London Zoo to be exact), Kansas, Colorado, and Bangladesh. I was one of the ones dropped in Colorado. We lived there for almost a year after that before moving somewhere else. (Some people ask me where I was born; when I say "Just outside Denver", they say, "Oh!" as though they understand something, as though that fact might actually have any meaning.)

My father was an outside-the-box thinker and pretty headstrong. He nearly didn't get his MD degree because he kept disagreeing with one of the professors about some theory about the origin of diseases. He was an early epidemiologist. One of his earliest theories in the late 1950s was that diseases and parasites evolved along with homo whichever, all the way from pre-apes to modern humans. He couldn't get people to write about that in the US in the late 1950s because of the Christian evolutionist lock on science (so recently!). His book Evolution and Eradication of Ancient Diseases was published in 1963. He started a paleopathology movement, arranging multi-disciplinary autopsies on mummies. That was while I was in college, so I'd come home to see him discussing these trial autopsies (see http://ambassadors.net/archives/issue13/selected_studies.htm for  more on him).

Between my mother's arts-literary nature, my father's outside-the-box research and our constant moving (Sri Lanka, Massachusetts, Bangladesh, Ohio, Sweden, Detroit, Scotland, Cleveland, Salt Lake City), I had to adjust to many cultures and never got to be believing that one could be right ... quite different from home-folk in most parts of the world. I suspect this shows in the way the Crystal family of methodologies is constructed, and I believe you'll see it leak out on almost every page if you're looking for it.

I graduated from a city high school in Detroit at age 15, and went to be an exchange student to Sweden (1969-70) to kill a year before college. That got me hooked on travel even more. I arranged my own junior year abroad at St. Andrews. When I got married, I checked first with my wife-to-be that it would be OK to move to Europe for a few years. A few turned to seven, at the IBM Research Lab just outside Zurich, where I worked on formal specification techniques for a while (an interesting precursor to methodologies, and one I dip back into every now and then)  ... and that's where our two oldest children were born. We moved back to the US for a while, but I got itchy feet and sent myself plus family to Norway in 1997, where I worked with the Central Bank of Norway. Along the way, I discovered I like learning languages, so now I speak some extent of French, German, Swiss German dialect, Swedish and Norwegian. I tried to learn Farsi back in the late 1970s, but then the Shah got overthrown and I've never been there to tamp it down (sigh).

My wife is an artist. While I used to work in computer graphics at Evans & Sutherland, we had a lot to talk about. We still argue over additive versus subtractive colors. She's an outside-the-box thinker herself, tolerant, smart and self-motivated --- she would have to be or we'd never have been able to put up with each other for 28 years! Thanks to her, I have sculptures of skeleton faces and bones in my living room. Thanks to me (and my father), we have a smallpox mask from India  – a two-foot high scary face with tongue sticking out and snakes coming out of the head – hanging in our front entry way.

Agile comes easy to her – all she has to do is change her mind at any moment!

Anyway, you get a picture of our home life ;-).

Our kids grew up thinking this was normal :-). They're teenagers by now, so they don't any more, of course.

And regarding the word "normal", when my middle son Sean was in calculus last year, we realized that the "normal to a curve" is perpendicular to the tangent at that point, i.e., the vector pointing farthest away from the rest of the curve at that point. So now my kids have fun calling me "normal."

That was a freaking long question (answer). Sorry about that.

[If you feel comfortable here Alistair, I suspect people would love to
hear a little about your home life]

Not too much to say. Since leaving IBM in 1994, I live in Salt Lake City and do most of my writing in coffee shops (for example, right now). I wrote Surviving Object-Oriented Projects and Writing Effective Use Cases at the Beans & Brew coffee shop just below the ski resorts; I wrote Agile Software Development largely at the Salt Lake Roasting Company, Crystal Clear and my doctoral thesis at the Salt Lake Coffee Break (open till 2 a.m. every day!), and these days sit mostly at the Two Creeks coffee (good for articles, but not for books – for that I still need the Salt Lake Coffee Break :-).

I keep travel down to 50% so that I have enough home life for our family to survive; and I write books to have a good reason to stay home. Lately, I haven't been writing a book, so my travel time's been going up again. Ugh.

I swim competitively, mostly every lunch time when I'm home to keep fit and to keep motivated. My goal over the last 4 years has been to break every Utah swim record (short course yards) in the 50-54 age group. That obliges me to change events every year. I have 14 of the 18, but only 5 meets left – I don't think I'll get the last two, but I hope to get two more before time runs out.

Swimming and traveling don't go well together, mostly because I don't have enough discipline to train on my own while on the road.

Q4 - Jared Richardson

Q4.  Tell me about your current venture?  A start-up ... that's a big change from SAS.

The startup I'm at has a very interesting idea. It's about creating an extreme level of transparency and then using that information to measure progress. We look at information from the team's source code management, issue tracker, feature tracker and even the IDE.

At the end of the day I can look at a team and see that they're spending too much time in the debugger, so I can talk about continuous integration and test automation as a way to keep people out of the debugger and in code. Another team might be reading code all day, trying to understand it. Then we talk about the peer code review or pair programming to get training levels up.

We automatically create burn up charts and then use them to teach managers why feature creep is bad. They can instantly see how the team can't catch up with an ever growing scope. Or how the team committed to this feature set but only delivered half of that. Then we pull up a "difference from estimation" report to see which features had bad estimates.

And at every step, because we automatically harvest real data in the background, we've got the actual time spent on each feature. Instead of guessing how much development time was spent over three days, we can say it took 12.74 hours to create that feature.

But this level of transparency scares a lot of people. Our industry has been very closed traditionally. Our managers generally don't understand what we do or how we do it. And as a result, most projects fail. Some from bad estimation, others from bad management. Sometimes it's because no-one realizes there's a problem until too late to help. The last numbers I recall seeing were a 75% failure rate for the industry. We've got to step back and see if we can't find a better way to build software. Many of the agile practices do just that. 6th Sense Analytics makes an amazing amount of information available and we're starting to see some pretty amazing insights come out of that data.

A tool like 6th Sense can be abused by a bad manager and it's a fundamental culture shift for many shops. But on the teams with the courage to make the leap, we've found both development and managers come away with a better understanding of the other does.

Why am I at a startup? If you saw my first question and it's answers, you know I like to tinker. And I get to do a lot of that here. We have working software, but there's still a wide variety of work to be done.

Feedback! Please ... :)

Hi Everyone,

Let me quickly introduce myself.  I'm Clarke Ching, I live in Scotland, I'm the one doing the interviewing. 

Can you spare a moment to send me a quick email to Clarke.Ching@gmail.com?

I've a few questions:

• What do you think of the interviews so far?
• Is there anyone else who you'd like to see interviewed?
• Are there any question's you'd like to ask the interviewees?
• Is there anything I could do differently that you think would improve the blog?
• Would you like to do some interviewing yourself?

Clarke Ching
www.ClarkeChing.com

Well done Johanna Rothman

... and to all the other winners in the 2008 Jolt Awards

Johanna won a productivity award for Manage It! Your Guide to Modern Pragmatic Project Management

Alistair Cockburn - Q2

Q2. I know a lot of folks who glaze over when I mention use cases.  They are kinda dull and they don't seem all that agile. I've never seen the point of drawing stick figures, for instance. But then I discovered your book and your particular way of writing them and I was wowed! A couple of weeks ago I took a client's management team (along with a few of their senior techies) through an example and they were also wowed. Can you tell us a little bit about the format and how you developed it to the point where you realized you had to write a book about it? Can you tell us about a real live agile team that's using them in iterative/incremental development?

You're right – by today's agile standards, use cases are long and dull. I probably wouldn't learn them today if I didn't already know how useful they are. Actually, this is a bit troublesome, because I am having difficulty finding new people to teach them – all the use case experts I know learned them in the 1990s.

What makes them hard is they require thinking and communicating – both tough sells at any time, and especially into the agile space, where the reflex is type-and-show. (It's a good thing I helped write the agile manifesto, or they'd have ripped up my agile membership card long ago for saying stuff like that!)

I got to use cases through my assignment at the IBM Consulting Group in 1991. They asked me to construct a methodology for their consultants to use on OO projects, with no constraints on what the answer was.

After some interviews, reading, and research, I settled pretty quickly on the Ward Cunningham / Kent Beck CRC cards technique - Rebecca Wirfs-Brock' "Responsibility Driven Design", and incremental development.

These left out something critical. As I wrote in "In Search of Methodology",
"Responsibilities reveal which object or subsystem will carry out a part of the required function, interaction diagrams show how the function is carried, but neither say why it is required in the first place. That is what scenarios and use cases provide."

It is similar to the "warp and weft" in weaving: one direction lays down the nouns or objects (or components) needed, and the other direction lays down the verbs or functions needed.

Trying domain modelling without knowing which subset of the domain is important is just too big and too hard. We saw in the 1990s companies spending /hundreds/ of work-years (I am not kidding) on domain analysis and domain modelling because they didn't know incremental development and how to bound their inquiry.

Anyway, in 1992 I read Ivar Jacobson's 1987 OOPSLA paper on use cases, and saw that they were just the right size and shape to fit into this hole that was left by all the OO techniques at the time.

I read the pre-pub draft of Jacobson's OOSE book, and didn't understand how to write them from that, so I flew over to Stockholm and took a class from his company on how to write them. After that I still didn't understand how to write them. Indeed, at that time, nobody knew how to write them, no matter how many words Ivar wrote on the subject. I think maybe they sit too close to his spinal cord for him to be able to see what he himself is doing.

So at that point, I had to work out how to break use case writing down into small enough pieces that I could write an instructional booklet for the IBM consultants to use. I sat at my favorite coffee shop for two days trying to break it down. It turned out to be a 3-Aha! problem.

* The first Aha! that hadn't been articulated to that point was that each use case collected scenarios based on a /goal/ - that's what held the various scenarios together.

* The second Aha! was that essential to the notion of goal is /goal-failure/. Specifically, every use case has not one but /two/ exits, success and failure. This goes against everything we had learned from Edsgar Dijkstra about single-entrance / single-exit subroutines, so I had to be quite careful that it really worked (David Parnas came up with an interesting semantics for multiple-exit functions back in the 1980s, so I knew there was some precedent). Anyway, the two exits turns out to be essential to understanding how they all fit together.

* The third Aha! was that goals are recursive – we have big goals, which we break into smaller sub-goals, which ... etc. We tackle each sub-goal in turn until we achieve the main goal. If we get blocked, then we look for an alternate sub-goal. All this is common sense (but not commonly articulated), and also a well known artificial intelligence technique. Therefore, there was good, safe theoretical and practical precedent for believing it would be sound.

From these three, I was able to construct a semi-formal theory that allows us to do what is easy and natural in writing and expressing ourselves, and which at the same time is mathematically sound ... and also, just by the way, produces use cases that look just like what Ivar Jacobson wrote! I think that was one of the happiest little problem-solving moments of my life, to be honest.

I wasn't allowed to publish this at the time, because the IBM Consulting Group thought all this methodology stuff was top-secret, mission-critical, revenue-critical information. Not true, of course, processes and techniques rarely are, but it meant I couldn't speak about it for some years.

A side note – I couldn't publish my first methodology, written in 1992-3, for the same reason. In fact that methodology never even got a name, more's the shame. It would be interesting to look at now, because it was one of the first agile methodologies contructed (after Tom Gilb's Evo). All it contained was use cases, incremental development, responsibility-driven design, and programming. I wrote a little article about it in Object Magazine just before leaving IBM (see http://alistair.cockburn.us/index.php/In_search_of_methodology).

Use cases turned out to be a 4-Aha! problem, because there was a little hole left over, which people would often get around to asking about: Why do we write down system internal actions (such as "check that the account has a high enough balance before giving out money"). The goal-subgoal business doesn't explain that. We used to answer, "Because these are requirements, that's why. If you don't write them, you'll get the wrong system."

The 4th Aha! came (no doubt at the coffee shop) a few years later, when I realized that a system's design enforces a contract across competing interests of various stakeholders, and that the use case describes the contract to be enforced. This introduced the stakeholders and interests model, which closed the gap. We now can say that every sentence in a use case describes a stakeholder's interest that must be satisfied or protected ... and now it is clear why we write all the validation checks in the use case.

I still wasn't going to write a book on the subject, because I thought my original article was sufficient, but around 1999, I got so scared by all the terrible book proposals I was having to review that I decided I had to publish a book just I wouldn't get stuck having to teach out of someone else's. That, combined with the construction of my class on the subject, and my just having coached a team at Fireman's Fund Insurance Company, indicated that I had enough information to write a "Shu"-level technique book.

You ask for examples of real use.

I'll give you two, my first and my most recent, which happen also to be my biggest and my smallest projects, the least agile of my "agile" projects, and possibly the most agile of my "agile" projects. The usage of use cases is quite different between the two. I'll also say how I recommend using use cases on modern agile projects.

The first project I call Winifred. It is written up in detail in "Surviving Object-Oriented Projects", and I reference it quite a lot because it was a very interesting and educational project, and I got to see it from initial bid until delivery and payment.

It was interesting for this discussion because I had just gotten done with my use case theory, and I got a call from the IBM Consulting Group field team to hurry up and go out to a project with 180 freshly written use cases (written in "casual" style, per my book). We used what I had plus what they had written to construct finally about 240 use cases, casual style, captured in Lotus Notes, to be used as the formal contract between the IBM Consulting Group and it's client.

It was a fixed-price contract, so we spent two months writing, examining and estimating those use cases, the interfaces, the estimated domain complexity, the people we could hire, etc. All of us were itchy-fingered programmers, and we were nearly gnawing on our fingertips from the frustration of not being allowed to start typing. However, since the contract was fixed-price, at about 15M$, we weren't allowed to start programming until the contract was signed on all sides.

Our incremental delivery cycle was 3 months; we did incremental and iterative development within each cycle, for about 9 weeks, followed by a 4-week debug-and-deliver cycle. More about this is written up in  Using both incremental and iterative development, so I won't repeat it all here.

My current project is the exact opposite. I'm trying to replace the Media Wiki engine running my website, because I think Media Wiki is crippled when it comes to any media besides text and images (but never mind that for now). What's interesting is that this is not a fixed-anything project, it consists entirely of one programmer and me. I pay for it out of my pocket, we pair-program whenever we're both available, and he/we work on it about one day a week.

So according to modern agile theory, we'd never write use cases for it, right? Wrong. I still need the basic advantages of use cases (see http://alistair.cockburn.us/index.php/Why_I_still_use_use_cases), like having some idea of where we're going, how big the project is, where we are, contextualizing our work.

Therefore, I wrote really really light use cases, and replaced the extensions with "work tokens" (I don't like either of the terms User Stories or Features for this level of granularity). My programmer friend and I peel off a work token to implement, then check how we're doing against the big picture given by the use cases. We're currently going through the use cases to see what's the mimimum we have to do before we can replace my Media Wiki engine with the new engine. Cross your fingers! (If you are reading this some months after I wrote it, with luck you'll see the new site. Compare it to Wikipedia).

These use cases are shown at http://alistair.cockburn.us/index.php/Examples_of_ultra-light_use_cases

What I recommend to a project team doing hard-core agile and using use cases is a little different from either of these. It is described in http://alistair.cockburn.us/index.php/Agile_Use_Cases.

Write the use cases as in "Writing Effective Use Cases." You probably won't have more than 30 of them.

Print out the one(s) you are working during this month or 3-month period, in 16 point font. Tape them to the wall of your work room (you are in a team room, aren't you?). Take a yellow highlighter and highlight in yellow the particular sentences you are working on this iteration or sprint. Color them green when they are done.

Now you can show what you are working on, what is complete, and what has been left out, at any moment in time. You'll get the benefits of use cases and user stories and agile and incremental and information radiators, etc.

P.s. typically, the first "story" or work token to implement in a use case is the first and last sentence of the main success scenario.

One last thing, since this is supposed to be an interview and not a tutorial on use cases (never mind that you asked for a bit of a tutorial).

Most researchers and writers write whatever's on their minds at the time, not really caring if it is right or wrong. I recall a professor telling me that this (some) particular author was famous for his first book – which turned out to be all wrong, but never mind that, because he got famous for it, and then got famous again for his later book which refuted his first book.

I recall at the time thinking, roughly, "Yuck! That's just bad research!"

Then I found that almost everyone does this. Booch wrote his methodology back in the 1980s, then spent half the 1990s recanting and cutting stuff out. I used to tell people to stick with him, he'd get to the right answer sooner or later ... but they could join me and I'd just tell them the right answer now ;-)

When I was given the assignment in 1991 to write a methodology, I made two rules for myself:

1. Always honor the programmers in their job activity, because if they leave, the company ships nothing, no matter who else is around; on the other hand, if everyone else leaves, the programmers can still ship useful software.

2. Try not to have to recant. This meant, for me, to be correct and err on the side of leaving good things out, rather than sticking bunches of stuff in and having to do like Grady and take stuff out.

I've been fairly successful in those two rules, particularly the second one.

My first methodology in 1992-3 only had incremental development, use cases, responsibilities, and code. I added design patterns in 1993-4. I added more stuff for the Project Winifred project, but only temporarily, for that project. Crystal Clear has some more stuff, but nothing I think I'll need to recant about. In general, my error, if any has been to underspecify than overspecify, and I'm still happy with that decision. We'll see how long it goes before I have to recant on something.

Q3 Alistair Cockburn

Q3. Have you ever worn a cowboy hat or flares?

Yes.

I had lime green bell-bottoms and blue platform shoes with about an inch of toe platform and two inches of heel platform. Really. Hopefully all photos have been destroyed.

Back in the late 1970s, I was one of those disco people. I pretty much stopped dancing disco after "Saturday Night Fever" came out in 1977 and John Travolta stole my moves :-). The movie did fire up disco popularity for a while, and I had a short gig as a professional disco dancer at a place where we were supposed to be the seed couple to get other couples up to dance. Really.

I got into trouble with the mountaineering ski crowd because I'd wear my black shiny plastic disco-style windbreaker to go back-country skiing, instead of the "just-rolled-out-of-the-garbage-can" polypropylene and wool outfits that constitute official uniform in that crowd.

When I moved to Utah (1975) I got myself a good cowboy hat, cowboy shirt, and cowboy boots (still have them, and they still fit). I scared the dickens out of my girlfriend when she came back from a summer vacation and I picked her up wearing all that. She nearly climbed out of my beat-up 1963 VW beetle when she turned on the radio and it came on with cowboy music. "Just kidding" plus a fancy dinner and lots more cleaned that up.

You can tell I'm culturally sensitive ;-)

Q4 - Alistair Cockburn - The Books

Q4 - Tell us about your books.  Which order should I read them in and what would I get out of each of them?  Which one are you most proud of?

Well, there's something like 8 books (depending on what the definition of 'is' is, and 'book' and 'your'). Here are some unpublished notes about them ...

Surviving Object-Oriented Projects was a summary of my investigations and project debriefings in the early 1990s. It contains snapshots of the projects I debriefed that led me to to recommend the kinds of things I do. Very little of it is specific to object technology, most of it is about what makes projects succeed and fail, how to staff for experimental versus production projects, project management strategies, and of course, how object oriented programming affects things. I still meet people today who tell me this is their favorite book of mine.

I succeeded quite well with it, with one exception. I wrote an appendix at the back called "Project Management Patterns". One of the reviewers (in 1996) complained bitterly about me getting on "the patterns bandwagon" and suggested strongly I drop the term "pattern".

Seeing that "pattern" is synonymous with "strategy", I did a global search-and-replace of "pattern" to "strategy" (and "patterns" to "strategies"). The surprise was that there were literally no further editing changes to clean up the wording change! When I submitted the appendix to PLOP 96, I changed the words back to "patterns", and after incorporating their recommendations, changed them back to "strategies." Try it – you'll find that most patterns are nothing more than strategies!

However, it turned out that my reviewer was completely wrong! and I was a fool to listen to him. Had I stuck with the word "patterns", I could have kicked off a project-management patterns movement, whereas as it now stands, few people have any idea that there are a dozen project management strategies sitting in the back of that book. Foo.

Nonetheless, I got permission from my wife to put all the proceeds from the book into a separate savings account for a little sports car. In 1999 I had $20,000 in the account, so I went to the local Mazda rep, showed her my screen saver image of a silver Miata, told her I had exactly $20,000 in the bank from my royalties, and lo and behold! She turned up with a silver 1999 Miata that, after taxes and registration came to $19,986. My son and I picked it up and bought ourselves (a small) dinner on the way home with the remains of the saving account.

– Note on creating a macro-structure –

I spent weeks drawing pie charts and mind maps of topics to make sure that it contained the full range of topics without getting stuck in any job speciality. I'm not really a mind-map person, but I tried it. What worked for me in the end was a three-layer pie chart, the inner pie showing % of space allocated for the major topics (chapters), surrounding circle showing % of space for major headings, some third-level breakdowns in a third surrounding circle. From these percentages I could calculate how many pages to spend on each topic (I knew I wanted a 300-page book, so I just did the math, then wrote that many pages on each topic.)

I can recommend this as one way to structure your book. Keeps it to size (and I know _many_ authors who could benefit from this discipline).

– Note on writing style –

I used to write really complicated sentences (still do, given half the chance). I ran across the book Revising Business Prose, a short, small easy-to-read, 80-page paperback, which taught me to write short, active sentences. I took one entire pass through the book splitting compound sentences, choosing shorter words and eliminating adjectives and adverbs. The book was much the better for the change. Try the book, I believe you'll like it. (p.s. I only used the advice in the first half of the book.)

– Book 2 –

Crystal Clear was the second book that I wrote (1999). However, XP was just on it's ascendancy at that time. I decided I couldn't compete with it's popularity, so I stuck the book draft on my web site and let it sit for five years, until friends pestered me to get it published just so some people would see that there is an alternative to XP.

Amazingly, a professor Chris Jones, then at Weber State University in Utah, found the draft on my web site, downloaded it and made his students use it for their semester projects - And he never even wrote to me! I found out years later when talking with his son Nate during an agile roundtable. Chris Jones later used my brand-new undergraduate software engineering text – while I was writing it, in real time! Talk about someone willing to experiment.

– Book 3 –

At that point I signed a contract to write a book series! Which meant I needed to write the zeroth book in the series, the anchor book. This was originally entitled Software Development as a Cooperative Game. I started drafting it in 1999.

– Book 4 –

But while I was drafting that, I was being deluged with catastrophic book proposals about use cases. I decided if I wasn't careful, I'd find myself having to teach out of them. Out of desperation, I scraped my hard drive for whatever I had on use cases and overnight submitted a 100-page book proposal for Writing Effective Use Cases. That was put on the front burner and came out late in 2000 – and has been paying my mortgage ever since.

– Note on macro-structure –

I'm famous for not finishing books. I generally run out of energy by half-way through the book (as, for example, Revising Business Prose, which is really short, but I still only read half of).

So for each of my books, I've worked hard to arrange that all the really important stuff happens in the first half of the book, preferably before page 100. The hard part, of course, is what to fill the second half with. I toyed with rubbish generators, but concluded that some readers would actually read the back half of the book and complain. So I fill the second half of the book with helpful-but-not-critical information. Dedicated readers can then mine the back half for extra goodies.

– Book 3, again –

I got back to The Cooperative Game, whose writing was redirected by the writing of the Agile Manifesto. The publishers renamed it Agile Software Development, even though the book is much more general than just agile.

What was fun about this book was that I got to design the page size, 2-column layout, typography, and even ran its production. We had a tight deadline to get it out by OOPSLA 2001, so I used all the advice in the book in doing the copyediting, illustrations, layout, etc. We shrank the time-to-production from 5 months to 5 weeks!

For example, at one point the copyeditor was resisting meeting me for a discussion. I started to explain why it would be useful to meet. She said, "I know all that – I'm editing that chapter!" :-)

The first edition of Agile Software Development was and still is my favorite book, because of the Introduction and first two chapters. It was the book I had long wanted permission to write, because in it I got to talk about the nature of communication and humans. Without the first two books behind me, I'm not sure I would have felt brave enough to write it.

– Note on macro-structure –

I had trouble getting the macro-structure of this book. The pie charts didn't work. What solved it was Writing For Story. Where Revising Business Prose teaches how to edit single sentences, this book teaches how to construct the macro-structure. Using it, I constructed the Agile book as a sort of mystery story, where every chapter resolves certain tensions but introduces new tensions to resolve. Take a look at Writing For Story and see.

I failed on my strategy of making the second half of the book "goodies", although I did get the most important information in before page 100 (my publisher wrote me just as the book was gong to press, "I'm just readng the book now – I'm up to page 80, and you still haven't gotten to the meat!" I wrote back – "That IS the meat!").

So this book is really two books in one – the first halves of two books, glued together.

– Book 5 –

All this time, Steve Adolph, Paul Bramble, Andy Pols and I had been trying to write a patterns version of Writing Effective Use Cases. I was also working on my doctoral thesis at the University of Oslo.

It was all too much, and I withdrew from the Patterns for Effective Use Cases after we worked out the pattern language. Steve and Paul invited my back onto the author list just before publication, but I thought that would take too much attention away from the work they had done, so Andy and I put ourselves as "contributors".

– Book 6 –

My doctoral thesis, which I completed late in 2002, was entitled People and Methodologies in Software Development. It was intended as the academic version of Agile Software Development, just as readable, but with the necessary attention to research method and full of researh citations. I gave my Agile book to the examining professors, and my thesis to Addison-Wesley. AW never got around to publishing it :-(.

Part of the point of writing the thesis was to let people who had read Surviving OO Projects and who had heard me rumble on for years about my project debriefings, get a closer look at what those debriefings and interviews looked like. It was also to answer the question, "Can methodologies ever converge?" (The answer is No, we'll need an ever-increasing number of them, which means they should be really really easy to construct).

– Note on macro-structure –

I rewrote this thing 3, 4, or 5 times. What cracked the problem was Lars Mathiassen cornering me and saying – "A thesis should answer three questions. You know those answers, don't you?" I said, "Yes." He said, "What are the three questions those answers answer?" I told him. He said, "Good – Start by saying what the three questions are, then answer them."

Egad! So easy! (after the fact). That's exactly what I did, and I still like the structure of the book: No windup, it just starts running at full speed. I wish more dissertations did that.

– Book not! –

My next book was supposed to be the project management patterns book, but the Agile Development Conference got in the way – I spent two years creating and babysitting that before the AgileAlliance took it over, and by that time I was already past the PM patterns topic – the book had "written itself" in my mind and my body didn't want me to take the time to write it onto paper. Shucks. I really miss that book.

– Book 2, finally! –

Finally it was time to dust off the Crystal Clear book and blow the cobwebs out of the words. It had been intended back in 1999 for people who didn't have any notion about small-team, incremental-iterative development. What a change in the intervening five years! By 2004, thanks to XP and agile, those topics didn't need any defending.

Crystal Clear: A Human-Powered Methodology for Small Team is currently my second-favorite book, because it covers pretty much the full set of topics needed by a team, with concrete examples of each artifact needed.

While writing it I discovered that project properties are much more powerful than methodology rules. Instead of saying, "Follow these rules, and good things will show up," I found it stronger to say, "Here are some good things to have show up – Ensure that they do!" And then let the people on the team find ways to get them to show up.

– Book 7 = Book 3, revised –

By 2006 it was time to update Agile Software Development, because five years had passed since the writing of the agile manifesto, and there were five years of discussion and misunderstanding to deal with.

After this revision, Agile Software Development: The Cooperative Game had become three books –  a 100-page book about people, a 100-page book about methodologies, and a 100-page book about agile development. If you read it, you might like to approach it in this way.

– Book 8, unpublished –

In fall 2006 I started on my next book, tentatively titled Software Engineering in the 21st Century, an undergraduate textbook for third-year undergraduates in a semester-long course. Neil Harrison and Chris Jones agreed to teach it in two separate courses while I wrote it in real time underneath them! Brave or foolish, we did it that fall term. I turned in 20 pages of book, powerpoint lecture slides, homeworks and class activities, twice each week, Monday and Wednesday. That was a wild ride (and it took me months to recover from the exertion, because I was consulting, teaching and traveling at the same time).

That book isn't out in paper form, because I don't have a publisher. I may publish it in pieces on my new web site.

I like this book, because I got to incorporate everything I knew and a bunch of stuff I didn't, all the latest and greatest ideas I could find about modern software development, cut down to the size suited for undergraduated. You can read about this book and course on my web site.

That leaves me with two books outstanding (PM strategies and the undergraduate SE book) plus a zillion more trying to get out. I'm literally limited by typing speed.

Which do I like best? Still the first edition of Agile Software Development. It's thinner than the second edition (I like the heft of the thinner book), and it contains that Introduction I enjoyed writing so much.

Which should you read first? Dunno – what's your reading style? Some people like Crystal Clear because it's the most concrete, some like Agile Software Development because it's the most theoretical. Some like Surviving OO Projects because it's contains the most project pragmatics. These three actually form a sort of trilogy, and I wish they could be bought as such.

Only read Writing Effective Use Cases or Patterns for Effective Use Cases if you're in the use case business, because they are really sharply focused to that specific topic.

Read People and Methodologies from my website if you're curious about my project debriefings or the argument that methodologies are transient and should be picked up, used and thrown away like tissues.

Go and get lost in the (currently) 800 content pages on my web site if you want more of anything. http://Alistair.Cockburn.us.

Q5 - Alistair Cockburn

Q5 - Tell me about your consulting work.  What do you like most about it?  Least?

 

This one should be short :).

 

What I like is that I get  control my own time. I can say 'No' once my travel schedule hits a certain threhhold (50%). That was actually the reason I quit IBM – they put me permanently in the field, and I started traveling 5 1/2 days a week, coming home Friday night and leaving Sunday afternoon. I lasted 3 weeks and then turned in my resignation. The primary immediate advantage I got was the ability to say No.

 

The second advantage, related, is that it's my own business, so when I work overtime, I know who's benefitting from it – and it's me! (Not some corporate stockholder or exec shooting for a bonus). Then, when I'm not actively working, I know what I'm dong and why – I'm changing baby diapers (OK, that was back in 1994), helping kids wth homework, going shopping, or going swimming. When I was an employee, that was all "the other me, the one outside work". Nowadays, it's all "just me", whether I'm filing papers, doing bills, shopping, consulting, teaching, writing, hiking, inventing flip-flops, whatever. There is "work me" and "home me", there's only Me. That's a really freeing feeling.

 

On the downside, my brain can never shut off. I call it "riding the fences" (rancher duty – making sure cattle can't go through a hole in the fence). It's a little process that runs all the time. I can't stop it. Invoices, clients, the work pipeline, the bills. I've met other small company owners who tell me the same story.

 

The other, and from my perspective, biggest downside, is that I don't have professional colleagues to trade notes with and learn from. It turns out that keeping up with all the new technology is done much better in an office situation – there's all this osmotic communication about the new browser APIs or Google's new products or something about the Eclipse interface or ... or ... or ...

 

Taking it all together, I'm much happier being on my own.

 

Q2 - Chris Fry

Q2: Did you have an agile "lightbulb" moment?  Can you describe what led to that momement?

The agile lightbulb went off for me when I was managing my first team at Weblogic (BEA Systems).   There was always a strong automation culture and engineering practices inside Weblogic.  I picked up Agile as a way to run my first team when a friend recommended a couple books on the topic.  Looking back I had a lot to learn but I remember thinking the Agile principles matched my personal philosophy about running a good software team based on automated testing and cross-functional collaboration.  The flexibility and responsiveness of the agile model provided a great framework for building great software.

Q3 - Chris Fry

Q3: Can you tell us about what led up to the decision to "go agile" at salesforce.com?

The decision to go agile at salesforce.com grew out of a desire to create shorter more predictable releases.  We had gone a year without a major release and wanted to get onto a more predictable release schedule that would deliver value at a consistent rate to our customers.  Salesforce.com is a young internet company and an agile methodology was a great fit our culture.   A big driver for us was to move away from a process that was predictive and move to an empirical model.  We also like to do big things quickly so we decided to go "all in" and change the entire product development team at the same time which was very aggressive. Steve Greene was the Scrum Master of the rollout team and I was the Product Owner.  The rollout team was a motivated, volunteer, cross-functional group of people that were aligned on the move to the agile model and principles.   The results have been extraordinary and we will be presenting them at the annual Scrum Gathering in Chicago this April.

Q4 - Chris Fry

q4 - How big was the rollout?

We moved thirty teams in three months.   It was basically the entire product R&D organization inside salesforce.com. You can read our agile 2007 experience report here http://www.cfry.net/docs/cfry-agile-2007-final.pdf

 

Q5 - Chris Fry

Q5: So, what approach did you take to the rollout?  Did you get help?  And how long did it take?

Initially we planned on doing a few teams, one at a time, and learning from our mistakes.   After talking it over with our R&D leaders we decided that the commitment to the change and the need were great enough to justify an "all in" approach and decided to move the entire organization to one methodology.  Mike Cohn has written about the "all in" pattern here http://www.agilejournal.com/articles/articles/patterns-of-agile-adoption.html

Some of the advantages of the all in pattern are: 1)  Management demonstrates commitment to the new model; 2) The transition is quick; 3) There is no time where you are supporting two styles of work, everyone is committed to the change; 4) You deliver value early and are able to demonstrate it to the organization and the customer base; 5) It reduces the risk of one team working in a waterfall approach; 6) You are able compare teams going through the transition at the same time.  There are some drawbacks but it’s a great model when it works, as it did for us.

We did get help.   Mike Cohn gave us great advice when he said the "all in" approach would cost us more.  We got a lot of help from Pete Behrens who organized agile coaches for us and gave us great help throughout the transition (see http://trailridgeconsulting.com/blog/?p=96 ).   We sent Scrum Masters to certification and had Mike Cohn run in-house training for our Product Owners.  All the training we invested in really helped smooth the bumps for the organization.   

It took us 3 months to transition the entire team to agile releases.   Change is never done, I would say we are still constantly improving, learning from our mistakes, and introducing new agile techniques each month.

Q5 - Jared Richardson

Q5.  I give your book away a lot.  Tell me about the books you give away or recommend.

Thanks for that! I can't tell you how many times I talk to someone at a conference who came to hear my talk because they were given a copy of Ship It! by a friend or co-worker.

I interact with a broad range of companies and technologies, so my book suggestions vary widely. Here's a quick sampling.

• If you've got someone trying to learn the Java stack, point them at Scott Davis' JBoss at Work.
• Learning Ruby?  Dave Thomas and Andy Hunt's Ruby Pickaxe book and Hal Fulton's The Ruby Way.
• Rails -  Dave Thomas's Agile Web Development with Rails
• Ajax?  Justin Gehtland's Pragmatic Ajax. And the Prag Press has new Prototype and Dojo books out as well.
• Groovy?  Check out Venkat Subramaniam's Programming Groovy
• Grails?  Jason Rudolph's Getting Started with Rails.
• Andy Hunt just released his "Refactoring Your Wetware" book. It's all about your soft skills... the stuff in your head. I think it's something everyone should read.

Q6 - Jared Richardson

Q6.  What surprised you most about other peoples reaction to Ship It!

Two things have surprised me the most. The first is the broad range of reactions. Early in the review process, I got three emails on the same day. The first said "I can't believe you suggest these practices. Everyone already does everything you talk about." The second mail said "The book is too theoretical. The practices are great ideas but no-one actually does these things." The third mail said "Wow! I knew most of these practices, but I've never seen them put together so well. And I needed to be reminded of the ones I knew anyway. Thanks!"

And that sums up people's reactions. Some love it. Some don't. And that's okay. No single book will reach every person. We've reached a lot of people with Ship It! and that feels great. It's now in German, Japanese, Korean, and there's even an India sub-continent version. I get email fairly regularly from people who've retooled some part of their shop's environment around Ship It! It's a great feeling to know you've had an impact on people you've never met.

The second thing that's surprised me is it's sustainability. It's been almost three years and we continue to sell more books every quarter. It's a slow climb, but it's still moving up. We worked hard to keep Ship It! language and technology neutral, and I think it's worked.

Q7 - Jared Richardson

Q7 - Tell me - or really, the folk who've not read your book - a little more about the ins and outs of Ship It!  What's in it for them for reading it?  Who will enjoy reading it?  Who won’t enjoy reading it?

Ship It is our take on the entire software ecosystem. Often a shop that does a great job in many different areas will have huge blind spots. Maybe a team has done a great job at following a process but ignores the infrastructure. Ship It! covers three major areas. Techniques, infrastructure, and process. You can see this in our poster of key practices http://media.pragprog.com/titles/prj/color_poster.pdf.  The book also has a chapter of Frequently Asked Questions that talk about common scenarios you'll run into and how to use the key practices to solve those problems.

But it's not a religious diatribe on "The Right Way" to work. We tried very hard to introduce you to the reasons and ideas around following a process, then we suggest one you can use. I'm more about getting teams to use the ideas than to work like Will or me. We also covered the "what happens when you ignore this area" too.

Who should buy Ship It? The book was written as a guide for developers who wants to make a difference on their team, but it's been very popular with project managers and tech leads as well. In fact, it's often categorized as a project management book. I that's because there aren't a lot of ecosystem books out their and no one knows how to categorize it. :) It's not Java or C#. Must be.... project management!

Clarke: That's my last question for Jared for now.  I've found Ship It! extremely useful and I thank Jared and William (his co-author) for their wonderful contribution.

Q1 - The Poppendiecks

Q1.  Hi Tom and Mary,  I suppose I should have started this blog with you two since it was your book (actually, an early draft of your book) that got me into this business, but I didn't, so please forgive me :) It must be about 5 years now since you started writing the book, but can you step back even further than that and tell me about your lives before that?  Several people have emailed me to complain that there's too much techie stuff in this blog and not enough romance, so how'd you two meet?  Was it love at first sight?  I'd also like to hear more about your "pre-LSD" lives - correct me if I'm wrong, but your resumes don't hint strongly that you were both headed towards gurudom ...

Mary replied:

Life before Lean Software Development

When the professor was calling role the first day of our college physics class, he asked some guy in the front:  “Are you the one who won the high school science fair a couple of years ago?”  At the end of the class he asked us to choose a lab partner, and since I didn’t know anyone in the class, I made a bee line toward the guy in front who was smart enough to enter and win a science fair. Meanwhile, that guy was looking around for a female lab partner, and so we met. It was certainly not love at first sight, it was all about getting through physics class with a decent lab partner. 

But by the end of the semester, we were serious about one another, and I had finally learned to spell “Poppendieck”.  The professor had graded all of our lab reports and put them in a pile for us to claim.  As Tom and I dug through the pile to retrieve our lab reports, time and again Tom would laugh and say – look at how you spelled Poppendieck this time!  Yes, there are many ways to spell the name, and when our kids were young, they discovered that there are also a great number of nicknames that can be derived from Poppendieck.  When they complained, I would tell them “Talk to your father.  He grew up with the name – ask him how he handled it.” 

I proceeded to get a masters degree in mathematics (before computer science programs were widely available), while Tom supported us by teaching physics in high school.  Then he went on to get a PhD in physics, while I supported us by working as a programmer in the physics department at the University of Wisconsin.  We lived in married student housing, where the apartments were so small that we refused to share our meager space with a TV.  Parents traded babysitting script for evenings out, and Tom took our daughter to a day care center on the back of his bike.   

My job was to teach one of those new things called mini-computers to digitize bubble chamber film and send a preliminary analysis to tape for later number crunching on the big Univac computer on campus.  Tom got his PhD by analyzing the (100) surface layer of silicon in ultra-high vacuum.  Eventually Tom ended up getting a job as assistant professor at Hamline University in St. Paul.  I asked my local DEC salesman in Madison:  “Who’s buying DEC mini-computers in the Minneapolis area?” and I quickly found a job in the engineering research department of 3M, exploring options for mini-computer control of processes.

After a few years as a physics professor, Tom found himself bringing the first computers into Hamline University.  He remembers crawling through the heating tunnels stringing cables, and eventually helping the school establish the policy that all students should have access to a word processor.  I was glad that Tom was a professor, because I was traveling quite a bit starting up process control systems in 3M plants.  Professors have free time to take kids to the doctor and dentist, and they can even stay home most of the day when kids are sick. 

But then our jobs changed.  Tom’s university decided to formalize the job he was doing and call it “Director of Academic Computing”.  He was asked to apply for the job.  I suggested that if they were going to have him apply for his own job, he might as well send out some resumes and see if anyone else would want to hire him.  Within two weeks, Tom had a job working for Honeywell designing a product data management system for a division which developed inertial laser gyroscope based navigation systems for commercial airplanes. 

At about the same time, I was offered a job as IT Manager in a nearby 3M plant where I had just overseen the installed a process monitoring system.  This was my first management job, and my first encounter with the quality movement and with just-in-time.  Tom encountered the total quality movement, activity based management, and just-in-time at Honeywell at about the same time.

After a few years, I returned to 3M headquarters and worked in new product development, while Tom went on to become a software consultant.  When 3M was spinning off Imation, I was offered an early retirement package I couldn’t refuse.  So in the late 90’s I abandoned Minneapolis for a year and moved to California, working for a start-up company specializing in high purity polymer that I had taken an equity position in while I was at 3M.  Meanwhile, Tom traveled extensively, trying to help companies as far away as Brazil use IBM’s “SanFrancisco” framework.  We ended up in the same city every weekend – sometimes home, sometimes California, sometimes Las Vegas, or wherever. We had two tandem bikes – one in California and one home in Minneapolis – and we usually went on a long bike ride together every weekend.  Our vacations were almost always week-long bike rides across some state or other.

Back home in Minneapolis, I was looking for some way to make a bit of money for a few years, after which my pension would kick in and pay the bills.  Tom said there was a great need for project managers, and I figured that I could get back into software development easily enough, having plenty of background in programming and IT management. I started up our business and got a job gathering requirements (using JAD) for a state government project.  JAD I understood, I had used it before.  This thing called “Waterfall,” however, was new to me; I couldn’t figure out how that could possibly work – and actually it didn’t.  I ended up as the third project manager of that project, trying to rescue it when it got into trouble, with mixed results.  When it was over I decided to write a book about how ideas from the quality movement and lean manufacturing offer a better way to develop software.

Part way through the book, an economic downturn caused Tom’s consulting firm to falter and thus Tom joined me as author of the book and as a partner in the business.  At first we worked separately, but one November I was invited to give a talk at XP Days in London.  Airfares to London were so cheap that I thought it would be nice if Tom joined me.  That trip turned into a whirlwind tour of Edinburgh and Malmo, and as we were struggling with the luggage and the logistics I said to myself:  “I was going to come here alone?  How dumb was THAT?”  And we have never traveled separately overseas again.

We have found that we love to travel and see new places, we love to meet people and talk about lean software development, and we are very fortunate that people want us to share our insights with them and give us more insights to share with others.

Tom Gilb - Q1

Tom Gilb is one of my Agile heroes.  He's been doing and teach agile for a looooong time, but many Agile newbies may not recognise his way as the agile way.

Q1: Hi Tom. I started University in 1988 ... which is the same year that your classic book "Prinicples of Software Engineering Management" was published.  According to my calculations that means your book has now been around for 20 years.  I only came across it a few years ago but it has influenced my thinking more than any other "software development" book.  I know of several of the better known Agile guru's who say similar things.  Can you tell us a little (or a lot) about what lead you to write the book and where your ideas came from?

Yes PoSEM is now in 20th Printing - I was asked by my publisher to make an 'evergreen' - and apparently it is!

 

I like to think my 2005 book, Competitive Engineering (CE), is a worthy successor. It treats the same subjects, but more deeply. It is not such an easy read, but we have other books to serve that purpose including Kai Gilb' s "Evo" book          http://www.gilb.com/community/tiki-download_file.php?fileId=27

It was important to me that the CE, which officially defines Planguage, was reasonably rigorous. Time will tell, but it is in Second Printing and is well reviewed, so there is hope for the future!

What led me to write the book?

Well, let me remind you of the book's predecessors:

1976 Software Metrics (coining the term, and the official foundation for IBMs CMM Level 4   [I don't take all blame for CMM!]). In USA 1977.]}

1974-5 Reliable EDP Application Design, and Data Engineering (neither in USA, but both in Sweden and UK).

These were the real beginnings - PoSEM was second generation!

You might say the centre of my technical universe is 'quality quantification'.

The driver there is -  the Lord.

 

 

OK!       Lord .....   Kelvin.

 

 

About 1965 the following appeared in a Norwegian newspaper: (I joined IBM Norway 1958, at 17)

"I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it;

but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind;..."

http://zapatopi.net/kelvin/quotes.html,  

 

 

 

That got me thinking about whether we could quantify critical quality properties like portability and security. I had no internet to search, and my international personal network from conferences was a good Who's Who (Dijkstra, Langefors, Naur, Nygård, Dahl)  - so I tried common sense and invented and published some ideas on how to quantify them. I also quickly found out that they worked in practice.

 

As I worked as a consultant, I got challenged with extending the vocabulary, for example extensive work with ICL UK in the early 1980's led to defining Adaptability, and from 1980 with IBM led to defining Usability. As late at 1995 at Ericsson (Erieye airplane) we extended Usability with Intuitiveness and Intelligibility. And these and others became stable, and became patterns for tailoring local variations. More recently my friend Lawrence Day of Boeing pointed out to me that my method is in the Bible, 1st Corinthians, defining Love by decomposition into several quantifiable factors ('endureth long'). And I realized that Rene Descartes was on to 'my' method centuries ago.

 

Another early inspiration was a UK Professor Christopher Strachey, Oxford University Computing Labs, in the 1969 NATO Science Council "Software Engineering Techniques"  Proceedings page 65)  http://homepages.cs.ncl.ac.uk/brian.randell/NATO/nato1969.PDF

 Christopher Strachey (Oxford) : “Computing science has been under some attack on the grounds that it isn’t software engineering. I propose to attack it on different grounds. I think we should seriously ask ourselves the question: is computing science? 

Recently I did a small survey as to whether computing is suitable as an undergraduate subject in an English university. I did this by grading all the topics I could think of under the headings of relevance and state of development. The premise is that it is clearly wrong to teach undergraduates the state of the art; one should teach them  things which will still be valid in 20 years time: the fundamental concepts and underlying principles. Anything else is dishonest.

The gradings for relevance ran from “clearly relevant and essential” to “part of another subject” (like numerical analysis) and those for state of development from “well developed with theorems, laws and text books” to “a fruitful field for research”. Note, incidentally, the importance of text books. They are designed to be taught from; they are quite different from treatises and even further from research papers. Now, it turned out that all those subjects which score highly for relevance score very low on state of development and vice versa. 

Until we have a sufficient body of topics which are important, relevant and well developed we cannot call the subject a “science”. I am quite convinced that in fact computing will become a very important science. But at the moment we are in a very primitive state of development; we don’t know the basic principles yet and we

must learn them first. If universities spend their time teaching the state of the art, they will not discover these principles and that, surely, is what academics should be doing. 

I do not for a moment underestimate the importance of the state of the art in engineering. Clearly it is essential and furthermore from engineering practice we must get our experience and material from which we develop theory. But, before teaching students we must get our basic principles right." [Strachey 1969]

Well, I found he was right. There were not many principles published or available (Langefors (Theoretical Analysis of Information Systems) and Naur had a handful).

 

So, I drafted a number of principles, based on my 'engineering experience'. About 124 principles were published in PoSEM (1988) and at least 100 Principles in CE (2005).

I like to think they "still be valid in 20 years time". In fact we can now test the 1988 Principles - would anyone like to argue they are not valid today?

 

My own arguments for  subjects  that will "still be valid in 20 years time". is in my paper Undergraduate basics http://www.gilb.com/community/tiki-download_file.php?fileId=98

The paper argues that principles, concepts and measures are primary tools to learn about.

 

A few academics have understood this, but not many, as far as I can see. I also see the consequent helplessness and waste in software projects that seem to me to be the consequence of the lack of fundamentals. We cannot seem to get decent quantified requirements at the top level for large projects - ever. We never seem to get past step One.

Tom Gilb - Q2

Q2: How did your life change after you published the "Principles" book?

Not much. My working life has been similar for almost 50 years, teaching, lecturing, consulting, writing, reading, networking, inventing, travelling - running my own company. So much fun - who would want to change it!

Tom Gilb - Q3

Q3.  Can you tell us a little about your personal life?

Yes.

 

Ohhh you mean will I detail it here now?

 

Well I can tell you it has been fun - and to protect innocent people I can't tell you all the details!

 

The essence starts when I was 17 and met this wonderful Norwegian girl of 18 in London. She is sitting next to me in my living room as I write this. True Love = Endureth Long! (Corinthians 1 again). I was in my first year of 6th Form (boring). So of course I got engaged, moved to Norway and got a job at IBM just by asking! All really great decisions. :)

I mean who knew that Norway was going to be the richest country with the best quality of life in the world?

 

I continued my studies at University of Oslo for about 10 years, on the side of full time job, and a family from 1964. Sociology, Economics, Political Science, Philosophy. What fun - and I did it for fun, not for a job. Yes it is all useful for me today. I realized that I would not be a good bureaucrat at a University - and also realized that 95% of University people dislike me or my ideas. That is probably a good sign. The key is who the other 5% who do!

 

Thank goodness I never had to do a Thesis approved by the 95% who dislike my ideas! So I decided my books would be my 'Thesis". My ideas are evaluated by the readers - and I have full respect for that - even from those who feel they do not like my ideas. At least they had a look at them!

 

We have 4 boys and 5 grandchildren. We live just outside Oslo in a new apartment, half hour away is our amazing beach-front cabin on the Oslo fjord, and we have a "town cabin" in Covent Garden - which is useful as we love to go out in London, and visit my London Family.

 

In January 2008 the Norwegian Government decided to pay me a nice sum (which I have apparently been forced to save for 50 working years) every month for the rest of my life. I still continue working with my son Kai - but I am even less focussed on earning money, and more focussed on fun, travel for me and my wife, playing grandfather, and writing creating and spreading the word. I like the model of my friend W. Edwards Deming (Statistical Process Control) who held his last lecture at 93. I like to tease my listeners that I will teach their grandchildren when they are retired.

 

I enjoy perfect health, which I know is a gift at my age. No, I never smoked.

 

I am Norwegian Chairman of the Board for The Art of Living Norway (artofliving.org) which has made available to  me a lot of wisdom, and contact with a younger generation.

I think I believe, but cannot measure or prove, that we had past lives and will have future lives. This explains some things to me - but I might be wrong - so I am doing the best I can in this life. If I can get back to you afterwards and confirm my hypothesis, I will. Watch this website. Failing that I hope we can discuss this 'afterwards'.

 

One consequence of this hypothesis is that maybe my ideas are not as 'original' as I might like to believe: maybe I either 'remember' ideas, or are fed them by spirit guides? A least I get to keep the income and royalties. And spirits don't sue for plagiarism! Hey with a bit of luck I will either remember, or find an old copy of, my books, when I am young in my next life, and can follow Newton's idea of standing on the shoulders of giants. Well, even if I can't - you can!

My philosophy, from 17 years old:       do what you love,        do it well,       don't let money dictate what you do -    you will survive.

 

Tom Gilb - Q4

Q4.  I first met you at XPDay2004.  You were one of the keynote speakers.  I loved your talk because it resonated with my own concerns about "Agile".  Can you elaborate on those concerns? Do you still have them?

You bet!

For readers who were not there, the slides are at

 Paper: http://www.gilb.com/community/tiki-download_file.php?fileId=50

 Slides http://www.gilb.com/community/tiki-download_file.php?fileId=170

 

The good news is that some people who have practiced Agile thoroughly, now see my point - and realize that if we are to deliver value to our customers, we need to add to the conventional Agile vocabulary. We need to be far more explicit about what value improvements our stakeholder need, and how we are going to deliver them.

 

'Stories' and 'Use Cases', Features and Functions - are nowhere near what we need to deal with. Quantification of Values (Qualities) is essential. The Agile boys got the message about small iterations OK, but they completely ignored the message about quantified quality requirements. I put that down to immaturity. We are all immature at some stage. It has to be put right in time. Rapid delivery of the wrong values is still wrong.

 

A great example of a Scrum Master who 'gets it' is a friend I met at the Agile Business Conference in London, Ryan Shriver:

 Ryan's home page www.dominiondigital.com

 Slides… http://www.gilb.com/community/tiki-download_file.php?fileId=148

 Goalkeeper Tool  http://goalkeeper.dominiondigital.com

 

I have a very simple message to all those failed projects for which we are so  famous, and will be more infamous:

 

1. Quantify the critical stakeholder values      (current Agile culture does not understand 4 of 5 of those words)

2. deliver those values early and frequently.     (delivering value to stakeholders is NOT the same as delivering code or a story)

 

The survivors will get it :)

 

My favourite theory of how to change our world: "no cure no pay"

 No Cure Paper http://www.gilb.com/community/tiki-download_file.php?fileId=38

 

 Slides No Cure http://www.gilb.com/community/tiki-download_file.php?fileId=85

 

I cannot believe there are so many managers, 99.999% of them, who prefer to pay for effort and failed results, when common sense says they should only pay for good results.

People called 'programmers' sometimes crowning themselves 'software engineers' , have learned to fleece these rich managers thoroughly. A fool and their money will soon be parted. Hang in there guys, it may soon be illegal!

 

Well, I'd better quit before I insult too many people, after all they have a right to use their company's money any way they want to?

 

Warm Wishes to You All

 

Tom

Tom Gilb - an example from Jens ...

Jens Egil Evensen has been using Tom and Kai Gilb's EVO approach with Scrum.  Here's a few questions and some very interesting answers with Jens.  I strong recommend you read on ...

A very

Q: Hi Jens, Tom said that you've been using Scrum combined with EVO.  Can you tell me a little background about the project and what problem EVO solved that Scrum didn't?

I’ve been studying and using Evo for the past four years. First as an employee in the Norwegian Product Register (governmental) after attending a course with Tom Gilb,, and later as a consultant in one of Norways largest IT corporations (EDB ASA (Avenir)). Avenir has a rather complex and “heavy” framework for managing large waterfall like projects called AvePro (“homemade” PMI inspired) that they have been working on and used for 20 years.

One of my first assignments in Avenir (in December 2006) was when Microsoft asked us to be an implementation partner for a Norwegian telecommunications company that had decided to implement Microsoft CRM 3.0 as their standard CRM platform. The requirement specifications that where presented to us where purely functional, and there where no references to what goals or business values they wanted to achieve by implement the system. I quickly concluded that our traditional methods of project management would not cope with such a project. One of the reasons where that I suspected that if we just developed a system based on the specification that we had received, the changes in requirements during the course of the project would turn the project into one giant contract nightmare, and that delivery of the system would be infinitely delayed. It would simply join the majority of all other software development projects in the basket of failure.

We agreed to do a one month test period with one consultant (me), kind of no cure no pay, to qualify us and the system for their needs. I told the customer that with the method I intended to use, we were quite sure to give at least some stakeholder some value, and that this value would be far beyond the cost of the initial implementation of the project.

To do this I had two simple requests for the customer. I wanted one workshop with the sales managers ad the director of sales, and one workshop with the employees from their internal IT department that where suppose to participate in the project.

I started the workshop with the management by asking two simple questions:
1: What is this company's vision?
2: If you don’t think about IT or software at all, what are your greatest problem today, that prevents you from fulfilling this vision or goal?

(BTW, I’m not of the very formal types, so I attended this meeting with the management in my usual work outfit; T-shirt and jeans – in Norway that is almost perfectly okay)

At question #1, they answered a bit staggering, and after a short while, they said with a little laugh that is was their vision to be a leading telecommunications company in Norway. Laughing as if they where embarrassed by this vision (I call this Norwegian modesty). I thought that was an excellent vision, and a good starting point for our project. No point in setting the goal to low. If you’re aiming for the moon you are more likely to reach the sky than if you only aim for the roof of the nearest building.

At question #2, the director of sales had a clearer view. He said, and I quote: “We are loosing xx million Norwegian kroner each year on contracts that are ending and lost, without even trying to win them back, or prolonging them. Nobody knows they even exist”.

I answered: Okay, so if I implement a function in your system that alerts the account manager of a customer, let’s say, 2 month’s in advance that a contract is ending, you will potentially earn xx million kroner more?”. He answered yes.

This was actually a function that they could have implemented in their existing system at any time, but the management of sales had never discussed this business related problem with the IT department, and the IT department where probably to busy surviving bugs in the existing systems to listen to the real stakeholders everyday problems.

I told them that I could implement such a function for them in only 14 days and where met with a short silence... Then the director of marketing said: I never thought I would hear such a thing a IT consultant”.

In the workshop with the resources from the IT department, we discussed all the practical tasks necessary to install and configure the CRM database and server, and some other technical stuff. Just enough for me to gain their respect, and to make them know that I really knew what I was talking about when it came to programming and technical issues.

I also told them that to be real heroes to the management, they had to deliver solutions that really made some increased business value for important stakeholders in the management. Cool features might be impressive, but they are short lived and soon forgotten. Business value is sustainable. They where very skilled resources, and had many years of experience, but they had never thought of it in that way.

I told them: When your boss ask you to do something for him, you must ask why. Only then will you know what his real goal is, and only then will you be able to really help him. The answer to his question is never yes or no, it is rather one or more design ideas with an explanation, a cost estimate, and an impact estimate on his goal and other relevant impacts (good or bad). This information will make your boss able to make a smart decision on whether the solution should be implemented or not, and it will make the IT guys real heroes in the eyes of the management.

After working in this way for a month, creating business value they really needed, and showing the potential of the method, they prolonged the contract, and wanted more resources from our company. The total number of hours for this project became about 5,300. In Norway, that is considered a mid size to large project. We kept delivering business value to the customer in small weekly (!) increments for 15 months.

When the number of people working on the project increased, organizing the work became challenging. Mostly because of the rapid increments in the project, and  that not all of the developers working on the project where experienced, and needed more follow up. Evo is a great method to break down top level goals into business value and end state requirements, but contains no mechanism to organize the developers, and how they work. Although design specifications where made in detail, we had trouble with developers that where developing outside the scope of the project, and struggling for days with problems that weren't important to solve at all, without telling anybody (also called thrashing?).

To get the necessary control of the production environment Scrum where partly implemented with one team, daily stand-up and a product backlog. That product backlog where populated with design specifications from the Evo cycles. This, I found, was an ideal combination. Now I had total control of the most important requirements, expressed as the business value they wanted to achieve. A breakdown into end state stakeholder requirements. Mapping between business value and designs that where implemented. The stakeholders received business value for their money (bang for bucks), and the programmers no longer wrote code; they created business value!! Their work had suddenly got meaning, and they loved it.

As far as my knowledge goes, no single method widely used today covers all aspects of a project, from Vision to values, values to requirements, requirements to design, and design to working systems.

All methods seems to cover the “project layer” where the author of the method works, and maybe one layer up and one layer down. To cover all layers you will have to use more than one method, and select methods that have compatible interfaces, such as Evo and Scrum (Evo design specifications into the Scrum product backlog).

Avenir has now added this methodology (in a modified form) as one of our preferred agile project delivery methods (Avegility). It’s built up as three parallel cycles named “Goals and requirements”, “Production (ie Scrum)” and “Delivery and measurement”.

Q: How did you learn enough about EVO to get started?  Was that easy?

As mentioned earlier, I first learned about Evo when I attended a course held by Tom Gilb, and I must admit that the first two days of the course, I did not understand where he was going with this at all. After the course I tried it in a project at work, and I was truly amazed with the result. Lifting my eyes from the code and the software, and focusing on the required results opened up a whole new world of possible solution for me. As a consultant for a software company I must admit that this sometimes is a problem; when looking at clients requirements and goals today, I far too often sees that the best solution seldom is buying or developing new software, but to begin with the existing software, or the way they work.

If you got that business gene, and a true passion for the results you create, living the Evo is easy. If code, technicalities and gadgets are all you live for, you will probably not get the point at all.
Some people never realize that the best system is the one delivering the best results for the business, and not the fastest, prettiest and the one with the perfect architecture (although we have to care about those things as well, but then as expressed quality and performance requirements related to business values and goals)

Q:  Did you have to change Scrum much?

I suppose I get a lot of people angry now, but in this particular project, because of the short cycles, we removed the Sprint term, and worked continuously with developing and delivering items from the back log, meeting every day in the daily stand-up. In the formal method Avegility Scrum is unchanged but for one thing: It’s not the product owner that is responsible for handling and prioritizing the product backlog, it’s the Evo cycle’s output in the form of design specifications. Cost/benefit ratio decides the priorities.