I've been hinting that software development and physical manufacturing processes are more closely related than we may have realized. To help illustrate this I'll describe a simplified cupcakes boutique and attempt to map that process to a software development shop.
In the cupcakes shop a customer such as say an accounting firm calls in with an order for a batch of custom cupcakes. They may order 20 or 60 or whatever. The customer chooses from a selection of icing colours and optional sprinkles topping. In this simplified shop orders come in complete and correct.
The order appears on the shop floor behind the walk in counter. A baker mixes the batch of batter into the cupcake trays and puts it in the oven. After it bakes the trays are moved to the icing station. For quality purposes a different baker checks the batch, then adds the icing and sprinkles. Every baker is trained on both the mixing and icing phases and works interchangeably on both.
From there the cupcakes go to the wrap station where another quality check is done on the work. the cellophane wrap is added and from there onto the pick up area. the customer is phoned to alert them their order is ready to be picked up.
at any stage if a quality error is detected then the batch is sent back to the previous station, or the batch may need to be discarded and redone if the error is found late and not easily fixed.
--
This maps to my development site as follows. customer orders represent change requests from product management and the executive, as well as defects found by testers.
The mixing phase represents design and coding. The icing station is code review. The wrapping is testing by the test team.
So far so good. Now with this physical model we can hopefully better analyze the more virtual world of software, identify waste and expose the real value stream.
Suppose work comes into development from a triage process. Suppose triage is only done once a day at 10 AM. hmmm that means that from the time a customer calls in the order, up to 23 hours may go by with no progress and nobody even looking at or starting on the order. Perhaps the team should move closer to continuous triage using the 80/20 rule as most of triage is routine and can be done by a senior developer. only a fraction of the issues would need the daily conference call among several principles
The baking and icing. Suppose for whatever reason the bakers in the kitchen enjoy the mixing and baking part, while they enjoy the icing station less. perhaps it's about what they see as "their" work vs. "others" work. perhaps management is much more diligent about monitoring and measuring and rewarding batches baked, while the icing phase is watched far less closely.
now in the bakery we have "visual tools" in that a foreman can see if things are getting messy and backed up at the icing station and cupcakes are going stale. he can order bakers to stop baking new batches until the situation in the icing is cleaned up.
in software over the years I've observed a tendency to really focus on coding and the assignment and delivery of code, while code review is generally tackled reluctantly, at the end, until ordered by management. I've never met a developer who had to be ordered to write code. so in software the "neglected code review" anti patten can be addressed by the use of visual tools, as well as management placing equal emphasis and attention on code review as they do on the writing of code.
onto the wrapping. suppose the baking and icing phase tends to send the batches through based on each order. now in wrapping they prefer to let a few baked orders accumulate and do a big batch wrap as that is more efficient for them. so again the work is done and just stalled in some queue making no progress and getting staler. from the value stream perspective the company only gets paid when the work completes and is delivered. so having cupcakes stuck in wrap and not moving is costing the company money in the short term and customers in the long term.
to the software developer, once the code is checked in they want it tested as soon as possible. that way if there is testing feedback the developer gets it while the matter is still fresh in his mind. also again the dev team as a whole including test only gets credit for delivery when it's all done. so in software there's an opportunity to again use visual tools to see if there are a large number of items stuck in test awaiting verification. also to measure the takt time from when some item is available from development into test, and when the testing completes. this takt time analysis is a powerful tool as it can also be used in code review to measure how long in clock time from when the review was created to when it completed.
Pulling it together, suppose your software development shop was instead a cupcakes boutique. A customer calls at 11 AM on a Monday with an order for 36 cupcakes with powder blue icing and red sprinkles. How long would it take your shop to call the customer to come pick up his finished order. later that same day? the next day? a week, usually stale, with 20% deviation rate from the order instructions? at least a month? would your software shop be able to survive in a type of business where swift delivery with high quality is required?
Monday, September 09, 2013
Monday, September 02, 2013
code review: runtime errors only
I've continued to think about code review and how it should fit into the software development process. By applying concepts like value stream and one piece flow we can determine what the useful outputs of code review are. From there we can determine how to better focus code reviews, and even if we should be spending the time and effort to have code review at all.
Probably the first question is if we should even be doing code review. Would we be better off to just skip code review and thus be able to move faster in development? I still believe that done right there is positive return in the time and effort spent on code review.
The value is in identifying and fixing runtime errors. As is well documented the cost of dealing with defects rises exponentially the further along the code is before the error is found. With code review we can still fix defects very quickly and inexpensively. The code isn't in a build yet and it hasn't gone to any testers or users outside development. This is probably the last point where real errors can be quickly fixed with a minimum of cost and drama.
That is the entire value of code review. Anything else is waste. I've seen perfectly good code that works rejected and sent back for rework with bogus subjective "defects" that are just noise. If it's not a runtime error then it's not a defect.
Code review can be guided by these two simple bullet points. There are two purposes to code review
thus rejecting code and sending back for rework should be avoided. it should only be done for runtime errors, where the cost of a code review iteration is much less than if the error is exposed at a later stage.
I know that some believe that code review is an opportunity for coaching or teaching. This is misguided. On the teams I've spent my career on everyone is always experienced. 2, 5, 10, 15+ years of software development experience. at this point if the developer hasn't figured out or isn't motivated to write good, clean, fast, maintainable code then he's just never going to. and there's nothing code review can do about it.
maybe if you're dealing with interns, entry level people in their first year, or people working with some new to them technology that's about the only time there's use in pointing out a better (in your subjective opinion) way to do something.
Probably the first question is if we should even be doing code review. Would we be better off to just skip code review and thus be able to move faster in development? I still believe that done right there is positive return in the time and effort spent on code review.
The value is in identifying and fixing runtime errors. As is well documented the cost of dealing with defects rises exponentially the further along the code is before the error is found. With code review we can still fix defects very quickly and inexpensively. The code isn't in a build yet and it hasn't gone to any testers or users outside development. This is probably the last point where real errors can be quickly fixed with a minimum of cost and drama.
That is the entire value of code review. Anything else is waste. I've seen perfectly good code that works rejected and sent back for rework with bogus subjective "defects" that are just noise. If it's not a runtime error then it's not a defect.
Code review can be guided by these two simple bullet points. There are two purposes to code review
- identify runtime errors
- move the unit of work along as quickly as possible
thus rejecting code and sending back for rework should be avoided. it should only be done for runtime errors, where the cost of a code review iteration is much less than if the error is exposed at a later stage.
I know that some believe that code review is an opportunity for coaching or teaching. This is misguided. On the teams I've spent my career on everyone is always experienced. 2, 5, 10, 15+ years of software development experience. at this point if the developer hasn't figured out or isn't motivated to write good, clean, fast, maintainable code then he's just never going to. and there's nothing code review can do about it.
maybe if you're dealing with interns, entry level people in their first year, or people working with some new to them technology that's about the only time there's use in pointing out a better (in your subjective opinion) way to do something.
Friday, August 23, 2013
The Toyota Way by Jeffrey K. Liker
I finished the second suggested reading from work. It was The Toyota Way by Jeffrey K. Liker. This is the second book I got from a former VP at work, after Scaling Lean & Agile Development.
It was a pretty good book. As advertised on the cover there are 14 Toyota Way principles. The author describes all fourteen. The author knows his stuff and demonstrates several of the Toyota principles in writing the book such as "go see", and "towering technical knowledge".
It comes in at around 300 pages. With 14 principles to cover in detail it was lot of ground to cover. The first 100 pages were pretty good. The next 150 or so dragged a bit and it seemed a bit repetitive and hard to follow all of these specific Japanese terms getting through all 14 principles. The author finishes pretty strong in the last 50 pages.
All in all it was a good book. For the first half or so I struggled a bit to see the linkage between automobile design and factory assembly, and software development. However on thinking about it some more I think I may finally be grasping what that executive may have been getting at when he commissioned a copy of these two books for his entire organization.
I'm starting to finally see the linkage between software development and physical manufacturing/assembly processes. There's a lot to be gained in software development by applying ideas from Toyota and modern manufacturing such as lean, low inventory, levelling, value streams, tracing and counting the number of "steps" from idea to production code in the field, one piece flow, finding and eliminating queuing and other waste.
I've got some more to say about it and I'm pulling some thoughts together and I'll have some more to post on it coming up.
It was a pretty good book. As advertised on the cover there are 14 Toyota Way principles. The author describes all fourteen. The author knows his stuff and demonstrates several of the Toyota principles in writing the book such as "go see", and "towering technical knowledge".
It comes in at around 300 pages. With 14 principles to cover in detail it was lot of ground to cover. The first 100 pages were pretty good. The next 150 or so dragged a bit and it seemed a bit repetitive and hard to follow all of these specific Japanese terms getting through all 14 principles. The author finishes pretty strong in the last 50 pages.
All in all it was a good book. For the first half or so I struggled a bit to see the linkage between automobile design and factory assembly, and software development. However on thinking about it some more I think I may finally be grasping what that executive may have been getting at when he commissioned a copy of these two books for his entire organization.
I'm starting to finally see the linkage between software development and physical manufacturing/assembly processes. There's a lot to be gained in software development by applying ideas from Toyota and modern manufacturing such as lean, low inventory, levelling, value streams, tracing and counting the number of "steps" from idea to production code in the field, one piece flow, finding and eliminating queuing and other waste.
I've got some more to say about it and I'm pulling some thoughts together and I'll have some more to post on it coming up.
Wednesday, April 03, 2013
coffee and a smoke
At work there's a new tenant in our building. Originally the site was built and fully occupied by my company. It's a five storey building. After several rounds of restructuring and such a decision was made to seek tenants to rent out some of the space.
It took a few months but IBM has now moved in and taken over the 5th floor entirely. Welcome IBM! We've seen them around the elevators and 1st floor cafeteria a bit. They seem like decent blokes. There were a few stragglers up on 5 and they were just absorbed into the lower floors a few months ago. There's still a good amount of space so it wasn't disruptive in my area.
Someone mentioned to me last week that the Halifax site is the only IBM facility in the world with free coffee. Apparently that is a global IBM policy. This is because the cafeteria has free coffee from the existing contract with my company. Don't be upset IBMers worldwide, the free coffee is awful, sludge, pretty much undrinkable. It's free and you get what you pay for.
Still something occurred to me and IBM may have already done this. What they could do is set up a jar on their floor and anytime someone gets a free coffee downstairs they have to put say 25 cents or whatever into the jar. Every few months or so the jar is counted up and donated to some local charity. This would then respect the spirit of the no free coffee rule while being a feel good initiative for the employees.
There was another change in the wake of IBM's arrival. The outside smoking area has been restored. There was an original smoking area about 100 feet outside the side doors. One day a couple of years ago or so HR announced that worldwide all staff smoking areas were ended and the employees had to go off-campus to smoke. But now with shared space IBM cares more about the tiny percentage of employees who are smokers and requires them to be accommodated on site. So the designated smoking area is back.
I'm glad the smoking area is back. I don't smoke myself so it might seem irrelevant to me one way or the other. I was concerned about the safety and security of my coworkers while at work. After all let's consider the unintended consequences of the HR decision to kick the smokers off campus.
Well first of all by going off campus they immediately lose the protection of the corporate security guard patrols. By definition security's realm is limited to the campus grounds. When employees step off campus they are on their own and thus exposed.
The other major issue with off campus smoking is that they were moved out of sight from the cubicles. With the on site smoking area we can see them from where we sit. Thus if there's some disturbance or security problem we can raise security from our desks or rush outside ourselves. With offsite smoking a bad actor can easily walk up to the smokers unseen from the building and create trouble. Again the smokers are exposed to additional risk.
So I'm glad IBM put the safety and security of their employees ahead of the agenda of certain anti-smoking busybodies in HR.
It took a few months but IBM has now moved in and taken over the 5th floor entirely. Welcome IBM! We've seen them around the elevators and 1st floor cafeteria a bit. They seem like decent blokes. There were a few stragglers up on 5 and they were just absorbed into the lower floors a few months ago. There's still a good amount of space so it wasn't disruptive in my area.
Someone mentioned to me last week that the Halifax site is the only IBM facility in the world with free coffee. Apparently that is a global IBM policy. This is because the cafeteria has free coffee from the existing contract with my company. Don't be upset IBMers worldwide, the free coffee is awful, sludge, pretty much undrinkable. It's free and you get what you pay for.
Still something occurred to me and IBM may have already done this. What they could do is set up a jar on their floor and anytime someone gets a free coffee downstairs they have to put say 25 cents or whatever into the jar. Every few months or so the jar is counted up and donated to some local charity. This would then respect the spirit of the no free coffee rule while being a feel good initiative for the employees.
There was another change in the wake of IBM's arrival. The outside smoking area has been restored. There was an original smoking area about 100 feet outside the side doors. One day a couple of years ago or so HR announced that worldwide all staff smoking areas were ended and the employees had to go off-campus to smoke. But now with shared space IBM cares more about the tiny percentage of employees who are smokers and requires them to be accommodated on site. So the designated smoking area is back.
I'm glad the smoking area is back. I don't smoke myself so it might seem irrelevant to me one way or the other. I was concerned about the safety and security of my coworkers while at work. After all let's consider the unintended consequences of the HR decision to kick the smokers off campus.
Well first of all by going off campus they immediately lose the protection of the corporate security guard patrols. By definition security's realm is limited to the campus grounds. When employees step off campus they are on their own and thus exposed.
The other major issue with off campus smoking is that they were moved out of sight from the cubicles. With the on site smoking area we can see them from where we sit. Thus if there's some disturbance or security problem we can raise security from our desks or rush outside ourselves. With offsite smoking a bad actor can easily walk up to the smokers unseen from the building and create trouble. Again the smokers are exposed to additional risk.
So I'm glad IBM put the safety and security of their employees ahead of the agenda of certain anti-smoking busybodies in HR.
Tuesday, April 02, 2013
don't have time
Once in a while something will come up. Perhaps a code review you need someone to do, or something like that. Now sometimes you'll find there's a snag that the person doesn't have time for it.
So I got thinking about that a bit "I don't have time". First of all the statement is obviously false. You have plenty of time. There's eight full hours available each working day and the task would take less than one hour. What the person is really saying is "I don't have priority" - this isn't important enough to bump some other task.
It might seem unimportant to make this distinction, but it is valuable to describe the situation accurately. Because by describing the problem properly then it can be a start toward some resolution. After all saying "don't have time" is absolute, final, intractable, unrecoverable. After all it's not possible to create time.
Now "don't have priority" is much more manageable. For example it could be possible to de-prioritize some other task. But more likely it allows an honest conversation. If your task isn't important to the other person, then great. Eliminate the dependency, get someone else to do the code review or whatever, and move on. Accept that roles have changed and someone has bigger fish to fry now. Realize that an error in your area is less catastrophic than an error in some other area, thus we assume the lower risk to focus resources on more critical things. In any case by breaking the dependency on your priority isn't my priority it allows everyone to move ahead.
So I got thinking about that a bit "I don't have time". First of all the statement is obviously false. You have plenty of time. There's eight full hours available each working day and the task would take less than one hour. What the person is really saying is "I don't have priority" - this isn't important enough to bump some other task.
It might seem unimportant to make this distinction, but it is valuable to describe the situation accurately. Because by describing the problem properly then it can be a start toward some resolution. After all saying "don't have time" is absolute, final, intractable, unrecoverable. After all it's not possible to create time.
Now "don't have priority" is much more manageable. For example it could be possible to de-prioritize some other task. But more likely it allows an honest conversation. If your task isn't important to the other person, then great. Eliminate the dependency, get someone else to do the code review or whatever, and move on. Accept that roles have changed and someone has bigger fish to fry now. Realize that an error in your area is less catastrophic than an error in some other area, thus we assume the lower risk to focus resources on more critical things. In any case by breaking the dependency on your priority isn't my priority it allows everyone to move ahead.
Monday, April 01, 2013
interviewing again
I attended an interview a little while back. No I'm not seeking new opportunities. I'm getting settled in with a new, better team since January and things are going pretty well so far.
The team is looking to add a new person to backfill for turnover. A candidate got through the first interviews with HR and the team lead. The TL asked the developers to do the next interview. It was a good interview I thought. The candidate was pretty solid and I thought he came across pretty well. Myself and the other developers had some probing questions. After the interview the team lead escorted the candidate out and we had a quick meeting to give the instant feedback and offer a thumbs up or thumbs down hiring recommendation.
It's a good experience to be on the other side of the desk. I didn't think I'd get to do any more interviews. In the first couple of years with the company I did a number of interviews for everything from co-op student through to team lead. I enjoyed doing them and I believe I was good at it. Then one day HR announced that in order to do interviews we had to go to some sensitivity / political correctness training class. I of course declined and that was the end of interviewing. I'm not sure what happened between three years ago and today, I didn't bring it up when the TL mentioned about us doing an interview.
--
I'm glad the disastrous HR policy is apparently no more. The closest I heard to a rationale at the time was something about what if a rejected candidate comes back with a lawsuit? yes let's talk about that. The company is large, with thousands of employees. So it's safe to say there have been around 100,000 interviews. Of these how many lawsuits have there been? Probably zero, none that I ever heard of. In the worst case even if some settlement ended up being paid to a troll lawyer then we can quantify the cost of these settlements over the population of interviews. And in the worst case the cost is probably about $1 an interview for these theoretical lawsuits.
Now let's compare that to the cost of a bad hire. Even for an entry level programmer or tester the cost of a single bad hire is staggering, well into the tens of thousands. And at the intermediate level and above it's certainly upwards of and into six figures. So what's more important financially, avoiding some imaginary theoretical small lawsuit that legal/HR would have to deal with, or avoiding a bad hire that development and the entire company would have to deal with.
Obviously the value of avoiding bad hires is orders of magnitude more important (and hazardous, as bad candidates are many and can often get quite deep into the recruiting process and even all the way on board). Yet HR/legal had the power they added this policy in order to make their lives theoretically easier. The term for this business antipattern is local optimization, where some department will put in some new rule or procedure that makes their workflow easier or faster while the overall cost to the business as a whole is worse off.
The team is looking to add a new person to backfill for turnover. A candidate got through the first interviews with HR and the team lead. The TL asked the developers to do the next interview. It was a good interview I thought. The candidate was pretty solid and I thought he came across pretty well. Myself and the other developers had some probing questions. After the interview the team lead escorted the candidate out and we had a quick meeting to give the instant feedback and offer a thumbs up or thumbs down hiring recommendation.
It's a good experience to be on the other side of the desk. I didn't think I'd get to do any more interviews. In the first couple of years with the company I did a number of interviews for everything from co-op student through to team lead. I enjoyed doing them and I believe I was good at it. Then one day HR announced that in order to do interviews we had to go to some sensitivity / political correctness training class. I of course declined and that was the end of interviewing. I'm not sure what happened between three years ago and today, I didn't bring it up when the TL mentioned about us doing an interview.
--
I'm glad the disastrous HR policy is apparently no more. The closest I heard to a rationale at the time was something about what if a rejected candidate comes back with a lawsuit? yes let's talk about that. The company is large, with thousands of employees. So it's safe to say there have been around 100,000 interviews. Of these how many lawsuits have there been? Probably zero, none that I ever heard of. In the worst case even if some settlement ended up being paid to a troll lawyer then we can quantify the cost of these settlements over the population of interviews. And in the worst case the cost is probably about $1 an interview for these theoretical lawsuits.
Now let's compare that to the cost of a bad hire. Even for an entry level programmer or tester the cost of a single bad hire is staggering, well into the tens of thousands. And at the intermediate level and above it's certainly upwards of and into six figures. So what's more important financially, avoiding some imaginary theoretical small lawsuit that legal/HR would have to deal with, or avoiding a bad hire that development and the entire company would have to deal with.
Obviously the value of avoiding bad hires is orders of magnitude more important (and hazardous, as bad candidates are many and can often get quite deep into the recruiting process and even all the way on board). Yet HR/legal had the power they added this policy in order to make their lives theoretically easier. The term for this business antipattern is local optimization, where some department will put in some new rule or procedure that makes their workflow easier or faster while the overall cost to the business as a whole is worse off.
Subscribe to:
Posts (Atom)