Python back on my desktop

This week I was doing some stuff with some log files. In the logs there were some periodic logs that appeared every 30 seconds which was distracting and annoying.

In UNIX this is of course swoosh, its

tail -f logfile | grep -v pattern >> outfile

Alas running on my Windows desktop Microsoft in its wisdom sees no need of such conveniences like tail and grep.

So what to do? Of course, what I did back at SupportSoft when I had to deal with this. Write a Python script to tail and grep. It was pretty easy since there was already stuff on the Internet for that - of course in Python it was only about three lines anyway. And just like that I was in business, no more annoying periodic log messages.

It was a bit strange though when I went to write the script. I was surprised to notice that I didn't even have Python installed on my desktop. That's weird, have I really gone that long without it? Maybe since I switched jobs close to a year ago it takes a while to properly get settled in and comfortable to the point of firing up Python to make your day more productive and enjoyable. Plus I've switched to a new team recently so maybe the dynamic here is somehow different in some more positive way that steered me back toward my correctness of the past.

Anyway I'm glad to have it. Since then I've had my interpreter up all day long and it's just pleasing to see it there on my taskbar, like an old friend. I even found some more uses for some quick one line things. So welcome back to my desktop Python.

Non functional requirements

This is kind of a strange one if you think about the name. I mean, why would you want the software to be non functional?

Of course we know what non functional requirements are. That is stuff that an end user or even tester cannot verify directly by using the system. Such as "all code is reviewed", "JSF will be used to generate the GUI web pages", "database access will be through stored procedures", "development will follow the S methodology". Stuff like that. But if you look at the name itself in isolation it's kind of funny.

Software development projects

I recall back in fall 2001 back at Core Networks, shortly after I joined Core. We were kicking off a project for the next version of the CoreOS flagship product. The project lead mentioned that we were going to be doing some tweaks to the software development process for this project.

At that time Andrew, one of the earliest Core Networks employees from the founding in 1998, had this comment. He said "except for emergency bug fixes, no two software development projects in the history of this company have ever been done the same way."

Which was true at the time. What's interesting is 7 years later in 2008 when I left SupportSoft (which had acquired Core in 2004). In 2008 Andrew's statement was just as true as in 2001. Except for emergency bug fixes, we still had never done two software development projects the same way.

At least with Core/SupportSoft there was always some desire to change the way we did software development projects. For whatever reason management, the architects, or the developers or whoever instead of standardizing on something that worked was always reinventing how software was to be developed.

Perhaps it's like software itself, which is perceived as infinitely flexible and changeable. The process of software development is perceived as infinitely adjustable and changeable. Some new fad or methodology. I would argue that the perception that changing software development process is cost free encourages excessive and unnecessary tinkering and adjustments.

I think most everyone would agree that the old style late and large delivery to the test team is not a good way to go. However once you get past that I don't think people recognize that the changing the software development process is disruptive and expensive. I suspect the point of diminishing returns is reached sooner than people would like to admint.


Some recent work on my street has made me think about it I guess. There are 6 apartment buildings on my street. They are all wood, 3 storey, around 30 years old, built around the mid 70s. Although the ownership of the buildings is mixed, within the last 8 months or so 4 of the 6
buildings have had their exterior renovated. Front shingles and siding replaced. Two of the buildings have also got new vinyl windows to replace the legacy metal frame windows.

For the companies doing the work, I'm sure this is a pretty standard project. Shingles and siding on a 30 something 3 storey wood 20 unit apartment building. I suspect that they followed the same methodology for each building project, and did not make material changes from building to building on the order of work done, number of people per building, starting on front vs. back, amount of cleanup done per day and at the end, what work is done by the most experienced journeymen vs. the less experienced apprentices, etc. It was just the same standard way that everyone understands, is predictable, and works well and everyone knows what to do and how and when.

Thinking about it I kind of envied them a little bit. Maybe there's something to that we could use in software.

swag

A little surprise to start off the work year. I guess while I was on vacation they gave some stuff to everyone for end of year. There was still some left so I went up to where HR had it and got mine.

It was a soccer theme with company logo gear from umbro. There was a carry bag with a soccer ball to inflate, a ball cap, and a long sleeve soccer shirt. It looks fairly nice.

In high tech the flow of swag is often a fairly good indicator of how the company is doing. At least it was pretty accurate back with SupportSoft and Core Networks. When the company is doing well the swag can flow pretty fast and thick with t-shirts, baseball caps, golf shirts, knickknacks, mouse pads, fridge magnets, pens, rugby shirts, jackets, free meals, etc. When the company is not doing well then you can tell because the flow of swag dries right up.

User Stories Applied

I just finished reading the book User Stories Applied. It was interesting. A co-worker lent me the book and suggested I read it. I can't remember the last time I read a work related book on my own. It's been a long time. I guess my interests outside of work are more, well, "interesting" than studying up further on what I do during the daytime to get paid.

It's good for me to get more of a grasp on what all this agile stuff I hear about all the time is about. I still don't really get it. As I read through User Stories Applied I kept thinking to myself did you actually build real software this way? Although the author is careful to consistently back up his talk with examples from real projects. So it apparently has been used on real projects and isn't just some academic thought experiment.

I guess it's hard to get comfortable with something that is such a departure from the projects I've always worked on. Still over the years people have attempted to introduce some of the concepts of agile into the software development project management. I think that's the thing with agile among established companies. We want to pick and choose the more interesting and useful aspects without going whole hog and having say 2 programmers to a workstation.

For example iterative development with feedback is good. We've all known from hard experience for a long time that the "late and large" integration at the very end of a software project is a bad way to go. So we try to build earlier and keep it in a usable working state where people outside the team can use it and give feedback. That said we were using milestones back at PRIOR on the CMS SB3 project in 1998.

One thing that really struck me was pretty much every chapter kept going on about this individual called the "customer". The customer writes the user stories (NOT the programmers!). The customer writes the acceptance tests (hmmm). The customer provides feedback between iterations and determines which stories are to be done in each sprint. This is a certainly a departure from the past with that level of customer involvement. In my experience I'm reading this and thinking, Who is this customer? I don't recall seeing him around very much in my 12 years in the cubes.

So that's something that's important and different. Agile isn't some internal fad or thing that the programmers do on their own that is opaque to everyone else. The customers have to be more involved. That involves organizational change. If the organizational change doesn't happen and it ends up being the programmers writing the stories and the acceptance tests and planning the sprints, etc. then agile probably won't be very successful.

It was a good book. I did learn a lot more about agile. It's part of the Kent Beck signature series. I think I've grasped enough for now and I'm not planning to read the rest of that Beck series. I've got some other books outside of work related lined up to read.


I remember back at TUNS in the software engineering course. The prof did a series on 4GL and code generators. He showed us this demo of some TI system called IAF or something like that. After you set up all of the screens and described the data and workflows it set about generating C code at a rate of about 100,000 lines and hour.

The prof made an interesting observation. IAF was very good at the traditional integrated order entry - inventory - shipping problem for mid to large manufacturing companies. That one specific scenario. However you wouldn't build a compiler with IAF. You wouldn't build IAF itself using IAF.

I think that would also apply to user stories as well. For example I wouldn't specify the emergency shutdown software for a nuclear power plant using user stories. I wouldn't design air traffic control software with user stories. I think user stories and agile may be useful in certain categories of software development; and inappropriate for others.

Software development is a large and diverse space. Internal IT department. Systems integrators. Shrink wrap software installed at a remote site. Web apps running off one big live website. Within this large space there are probably some situations where user stories and agile work well.

Colour printing in the office

We put in some new printers a little while ago at the office. Usually that's nothing special. Stuff doesn't work for a couple of hours while they get the IP and printer name and such set up then business as usual.

But it was a bit different this time. We now have colour printing in the office for the first time. And it's great. I like to print code and it's great being able to get the same printout as is on my screen. So I'm really enjoying it.

It makes sense to have colour printing. Our monitors are in colour, powerpoints are colour, so you would logically expect to print in the same colour as you get on the screen. Plus we've had colour printing in our cheap home printers as completely standard for like 10 years now.

So why has it taken so long for colour printing to arrive in the office. Some history might be useful here. Back around 1997-1998 at PRIOR Data there was this Y2K project for the military. PRIOR was providing office space and some services and staff to a group who were doing this huge Y2K readiness project for the military, checking the readiness of all kinds of stuff like elevator systems and whatever else they could think of that might fail.

They were mostly downstairs from the main PRIOR office on Spring Garden Road. As part of Y2K they were producing a lot of documentation of course. And there was a rumour they had this special printer called 'Howe' that was a big HP machine that cost $27,000. And it had double sided printing.

It made sense for them for their project and the amount of documents they had to produce. For the rest of us upstairs we asked the sysadmin once in a while about double sided printing. The response was usually something like "that's really expensive; we would need to install special hardware kits on our printers; we would need to install special drivers on everyone's PCs". There was a perception that double sided printing was an exotic thing that was too difficult and expensive to make widely available.

But by 2001 when I joined Core it was just there, completely standard, even on the cheap Lexmark office printers. Double sided printing was just something that came out on the next generation of printers and it didn't cost any more when getting new printers to get double sided [although you saved a fortune on printer paper] so that was that. And we've had double sided without thinking about it for years now.

For a long time colour printing was where double sided was around 1997-1998. People were suspicious that the cost per sheet was too high. It was perceived as unnecessary. Plus colour printing was a status symbol, a perq of the important and powerful who had access too it.

But I think going forward when businesses go to replace their existing workhorse office printers they will find that colour is no longer an exotic expensive option just for the executive level. It will just be standard and the cost per page will be the pretty much same as the legacy dinosaur black and white printers. The employees will be very happy with it. Colour printing at work is great.