Wednesday, December 28, 2005

F7 your way to a better career

I find typos in e-mail annoying. It’s a distraction that makes it hard to concentrate on the content of the message.

Some people don’t care about e-mail typos. They either don’t notice them or don’t mind them. Perhaps the majority are this way. It wouldn’t surprise me if that’s the case.

Should you care about typos in your e-mail? Perhaps you should, for a few reasons. While you may not mind typos in e-mail, some of the people reading your message might. If that person is your boss or someone else important then you’re making yourself look bad. Typos in my opinion just make you look sloppy and less smart. People who are perceived as sloppy and less bright tend to not do as well in terms of career and salary advancement and job security.

Also, e-mail has a way of being read by people other than the original recipients. So your message may end up being posted on a department Wiki, or forwarded to people you didn’t expect to see it. e-mails can be distributed much further than originally intended. If your e-mail ends up “broadcast” out then I’d think you would want your writing to look crisp and polished.

There is an easy solution. You can make yourself look good with virtually zero effort. Before you send that e-mail, just click F7 in Outlook to run a spell check. It’s fast, and Outlook does all the work. The image you project at work is important. This is a very modest investment in your time that could pay off well for your career.

Friday, December 23, 2005

Eclipse tips and tricks

I've been working in Java for around a year and a half now. One of the best things about Java is working with Eclipse. It is a very good product.

When I started out, I found the product to be complex and intimidating. That's no surprise. While it has a good user interface, it's a powerful product with a number of concepts to grasp. Some people like to jump right in with new technology. For myself, I don't like starting without knowing anything.

Before starting Java development, I wanted something to read for an overview of how to use Eclipse effectively. It turns out that Eclipse has such a document. Under Help, click Tips and Tricks. This will bring up the tips and tricks switchboard.

From the switchboard I selected the Java development option, then the platform option. Each option brought up an extensive overview of useful things a developer could use. I printed off both and read them over.

It took around 2+ hours to read both tips and tricks right through. It was time well spent. I gained a much better understanding of what was going on in Eclipse and how to use it effectively. Especially useful were some of the time saving features and shortcuts.

I'd say that the 2-3 hours I spent reading the tips and tricks that first morning paid off in time saved within around the first two weeks. After that it's been all gravy. The Ctrl+Shift+R trick alone that I didn't know about before has saved me more than the amount of time I spent reading the tips and tricks. There are a number of things I use all the time in there that I wouldn't have known about otherwise.

When a new version of Eclipse comes out I reread the tips and tricks. They are updated to cover the features of the new releases. Also I sometimes absorb stuff that I might have overlooked previous times I read it.

So if you're starting out in Elipse, or even of you've been using it for a while, I recommend you read the tips and tricks.

Tuesday, December 20, 2005

Eclipse Javadoc tips

Lately at work we've been emphasizing Javadoc more. I here's a couple of things that I noticed Eclipse will do to make it easier to create Javadoc.

- if you type /** above a function then press Enter, then Eclipse will generate a Javadoc skeleton for that function

- Eclipse will also auto format your Javadoc content after you type it in. Type in the content from the Javadoc skeleton. Then select the Javadoc comment block and press Ctrl+Shift+F and Eclipse will do a pretty good job formatting it. So you don't have to manually edit your Javadoc for indents, column width, and line wraps. Just type in the content and get Eclipse to format it.


--

I sent the above around to the developers at the office. Like my note before about Google desktop search, a few people appreciated it. I find it's good when you discover some tips about your work environment to send them around to your co-workers. You'd be surprised what people don't know and aren't using.

Friday, November 18, 2005

Split the team for code and test

My previous post about JUnit got me thinking some more about the subject.

Traditionally in projects I've worked on the developers write both the code and the JUnit tests. The developer will write some code, then write tests for the code he just wrote. This means that a developer will spend a portion of a project schedule writing the code, and a part of the schedule writing the JUnit tests.

I wonder if that approach can be improved. I'll use an example. Consider a software development project with six developers. One of the developers is also the team leader. The team leader both develops code himself and manages the other developers on the project. This example is somewhat based on a project earlier this year where I was the team leader with around six developers. Although this example refers to Java and JUnit, it applies to the general issue of code and formal code testing.

In the traditional model described above, all six of the developers would both write code and write JUnit tests for that code. I can immediately see drawbacks to that approach.

  • Many developers at the code. That means more communication links, more coordination, more management to do. With more developers the team leader has to spend a lot of time keeping track of who is working on what, and managing the dependencies, and making sure people who depend on one another understand what the other person is expecting.
  • When the team leader spends a lot of time managing, then he has little time for writing code. This is generally a bad thing. Typically a developer is made a team leader because he has shown technical ability in the past. If the better developers spend their time not programming and the lesser developers write most or all of the code that's not good. There's probably an antipattern around this. You want the most capable developers to write as much of the code as possible. It's hard to write code when you're managing people.
  • With split responsibility of both code and JUnit, the JUnit necessarily cuts into the time a developer can spend writing code and informal scratch testing. The code is the most important product of a development project. I'm suspicious of activities that compete with and distract from writing code.
  • Lack of visibility. If a developer writes some code, then writes some JUnit, and the code builds and the JUnit runs through successfully, well he's done right? I guess so, things seem to be done. However the developer has worked in isolation. You don't really know until you pull the pieces together if he did a good job with the code and the JUnit. After all if someone writes bad code, then the JUnit will likely also be low quality and fail to detect the problems in the bad code.
So the question is, is there a way to address these problems? The solution would need to produce a higher quality software product within a software project that runs more smoothly within the original schedule and headcount resources.

Well I think there may be a better approach than that described above. Instead of having everyone do both code and test, split the development team into two. Have a code team and a test team. The code team does all the code. They should do informal scratch and feature testing as they go, but not formal JUnit testing. The testing team does the formal JUnit tests on the code the code team writes.

Using the example above, the team of six would be a three person code team and a three person test team. The original team leader is the subteam leader code team and remains the overall team leader. One of the JUnit test team developers is appointed subteam leader of the JUnit team.

I can immediately see benefits to this.

  • There are now only three people at the code instead of six before. This is good because there is less communication and dependencies between developers. There will be less misunderstandings and disconnects. There is less communication and synchronization overhead. The code development should go more smoothly.
  • With less code developers, the team leader will spend less time managing and coordinating individuals. Thus he can spend more time writing code himself. This is good because typically the team leader is also one of the better programmers. You want your good programmers to spend as much time as possible programming.
  • With a separate JUnit team, the management burden can be partly offloaded from the overall team leader to the JUnit subteam leader. This frees up the team leader's time to focus on writing code himself and managing the more important code project.
  • There is more focus. The code people are not distracted with JUnit and are able to spend more time on the code development. The JUnit people are able to focus on doing a good job testing instead of just having it cut into more important code activity.
  • Most software development sites aren't like Google. Instead of programming superstars everywhere there's more of a mix of strong developers, reliable middle class type developers, and people who tend to require more supervision (you need to keep this third group to a minimum, max 10% of developer head count, or you're in big trouble). By splitting the team up you can put the most capable people on the code team. Thus assuring a higher quality of delivered code. The rest can do the JUnit. If they do a mediocre job it doesn't really matter since the code will be solid anyway.
  • Do you still need six people? With less synchronization and coordination among individuals, and the team leader able to spend more time programming, I don't think you do any more. You can now get the same job done with higher quality and the same schedule using only five people instead of six. Two JUnit people will be enough to do a good job. So that's a financial benefit to the company, being able to get the project done with five people instead of six in the original schedule.
  • Finally, this way actually forces a real budget to be allocated for the JUnit. I've been on projects where the schedule was overambitious, and we were just told to "fit in" the JUnit on top of doing the code, even though the schedule didn't really even have enough time to do a proper job of the code. You can guess how much JUnit got done on those projects and the quality of it. By allocating a proper budget and resources for JUnit, it is much more likely to actually be done and a better job will be done with it.

Sounds good to me so far. Of course there are some hazards to this type of plan. I can see some political type problems from this. Some people are suspicious of change in general so a departure from established ways may meet resistance just because it is a bit different.

The other issue may be in the assignment of developers between the code and JUnit teams. Some people might not like the idea of developers being on a software project but being "relegated" to unit testing and not writing production code. One way to deal with this is to mix it up over different projects and have developers be code team on some projects and JUnit team on others. Or just deal with it and use it as an incentive to the mediocre developers to improve themselves to get on the code teams.

I think it's worth trying.

Thursday, October 27, 2005

Cutdown day

Update August 2006, survived another one.

---

After a tough quarter, the expected layoff happened at the office. I'm glad that I was fortunate to survive this downsizing.

I've never actually been downsized in high tech. That is something of a rarity in this business. I've managed to survive quite a few though. Here they are that I can remember.

Date Company
======== =======
Jul 2000 xwave
Jul 2001 xwave
Oct 2001 Core Networks
Mar 2002 Core Networks - huge one
Sep 2002 Core Networks
Apr 2003 Core Networks
Aug 2004 Core Networks
Oct 2005 present job
Aug 2006 present job
Oct 2007 present job

I've been pretty fortunate to survive that many. For some reason I've wondered what it's like to be fired. What does the manager actually say to you. Then what happens? Still, I don't really want to find out first hand.

In my career when it happens it's been kind of like with Syme in 1984. People just kind of disappear during the day and are never seen again. They are generally spoken of little afterward, however they don't become full unpersons.

It was expected that this action was going to happen. We've preannounced twice in the last year, so something had to give.

When a layoff is imminent or has just happened, it makes me think of a story I heard about a guy who was a programmer for Corel. I lived in Ottawa around 1994, and at that time Corel was high flying. It seemed they could do wrong.

Well fortunes can change quickly in high tech. Corel made some strange forays into WordPerfect and Linux. Add in an aging main product line and tough competition from Microsoft. By the early 2000s Corel was struggling.

After another tough quarter, it was known that there was going to be a big layoff in the Spring for the company to survive. Now "Bill" was a programmer at Corel. He'd been there through the high flying times and was still around as the company struggled. For Bill, he'd had just about enough of life at Corel. The struggling, the abrupt changes in direction, the increasingly dysfunctional environment took their toll.

So with the big layoff coming Bill realized that he wanted to be fired. He figured he'd be better off after it happened.

  • If he was sacked he'd get severance. Severance cash in hand today was better to him than underwater stock options which might never get into the black in the future.
  • It was Springtime in Ottawa. Ottawa is nice in the Spring. After he got laid off, Bill could go on unemployment insurance (called UI in Canada) . That meant Bill would get to have a "UI Summer". In Canada the UI summer is an unusual and coveted thing. Most people get to do this never, or at most just once.
  • Bill picked up A list tech skills working at Corel. Plus he was a good worker and would have good references. He knew some people at other companies who would be willing to help him out. He wasn't concerned about being able to find another job after Corel.

Yup, Bill was looking forward to getting laid off. Severance, followed by a UI summer. Plus he wouldn't have to work at Corel any more. He was going to relax, hit the beach, not even think about going back to work until the NFL kicked off in September.

In the last three weeks Bill cut back from his regular 45-50 hours a week to a straight 40 hours a week. That was Bill's signal to managment that he wasn't going to fight to save his job and they should let him go.

Finally the expected cutdown day arrived. But then something unexpected happened. Bill survived the layoff. Well that was disappointing. Instead of severance and UI like many of his co-workers, Bill got left doing extra work to cover for those who were now gone. Plus the company was now struggling to survive so he had to work under all that uncertainty. The environment at Corel was even more unenjoyable after the layoff.

So that's where I came up with the term Corel'ed. That's when the big layoff is known to be coming. You're looking forward to getting fired because you think you'll be personally better off after you're gone, and you won't have to work at the crummy old company any more. But when the layoff does happen, you unexpectedly survive it. Then you're unhappy to survive it because you still have to work there and you miss out on the severance and other stuff you were looking forward to.

I don't know what happened to Bill after he got Corel'ed. Perhaps he's still working there.

Thursday, September 22, 2005

Google desktop search

If you don't have this then get it. It's great. No more wasting time on interminable Outlook "advanced" searches or Windows explorer file searches. Although in win explorer I was able to turn off the stupid dog, I couldn't fix the rest of that horrid user interface, nor make it any faster. I was surprised this week at work to learn that quite a few people aren't using this.

You can get it at http://desktop.google.com.

Also on Google, I've got a stack of Gmail invites if anyone is interested in getting Gmail. Go to my profile and click on the e-mail link and I'll give you a Gmail invite.

Sunday, August 28, 2005

The average user

Last night I went over to my neighbours house. He was at our house about a week ago, picking up his son who was visiting my 7 year old. Anyway he mentioned that he had saved some pictures from his digitial camera to his computer. He was trying to get them from his computer onto some blank CDs he'd bought but was having problems and getting errors. He'd heard that I know about computers and asked if I'd drop by his house and take a look.

So I went over yesterday evening. He'd been trying to burn using some Nero software or something. I stepped through the wizard with him and confirmed that it didn't work. All I did was show him how to burn his pictures to CD directly using the Windows Explorer. The burn stuff that comes free, built in with XP, is not at all bad with an acceptable enough wizard. And it worked fine and he was able to save his pictures to CD successfully. So he basically paid for this junk Nero thing that didn't work when XP already has what he needed. He'd had the computer for over two years and until last night had not been able to save to CD successfully.

So that wasn't too bad. It only took about a half hour to get his pictures burned onto CD. Most of that time was spent training him so he could do it himself the next time when I wasn't around.

All done right? Not quite. He mentioned that his computer was starting slower and slower and he was getting these annoying pop ups. Could I look at that too? Hmmm. So anyway it was not a good scene.

  • his IE start page had been changed on him to some Oemji crap
  • there was some garbage IE "search" toolbar
  • he was at XP SP1. He wasn't receiving automatic updates
  • his firewall was not turned on
  • he was getting billed every month on his credit card from classmates.com and wasn't able to cancel it
  • he was getting these popups every few minutes for some junk or other
The very first thing I did was download and install Firefox. Then I could at least have a usable browser that would bring up Google successfully without being redirected.

From there I was able to squash Oemji (there was all kinds of it), fix his IE start page, get rid of the unwanted junk toolbar, squash the periodic popups, stop the scam "spy scan" program that ran at startup. Then I put on the Google toolbar - he'd had all kinds of problems with popups in IE. I also put him on windows automatic update (not sure how it got turned off), so hopefully he'll get SP2 shortly, and turned on his firewall.

For the classmates.com thing I told him to just call his credit card company and cancel the preauthorization with them. Don't bother trying to figure out how to cancel with classmates.

So all this extra stuff took a couple of more hours, not the most enjoyable way to spend an evening. Anyway he's kind of handy so I'll have a favor to call him for where I can use some of his skill in the physical world in exchange for the big electronic favor I did him.

I strongly hinted to him that he should switch to Firefox. I've been using Firefox for over two years and during that time my start page has never been changed on me, no unwanted toolbars have ever appeared, and there's never been a popup. I can only hope he took the hint.

He showed me the bill of sale from the computer. It was from a prominent local dealer, the ones who advertise on the radio all the time. I was highly unimpressed by what I saw. They charged him over $2000 for a mediocre system. It was basically a white box clone, not tier one. Also I noticed his CD player was an HP. I doubt it was brand new since the dealer could surely have bought a clone CD player cheaper - does HP even sell CD/R direct to small shop vendors? So it was very likely a used player cannibablized off some other HP system.

Still one more thing. At one point he took his PC back to the dealer about the inability to write his pictures to CD using Nero. The dealer gave him some mumbo jumbo tech talk he didn't understand, charged him $200, and didn't even fix it. The only difference before and after taking it in for "repair" was that he was out the $200.

I wonder what would happen with my neighbour if he had a leaky faucet, then called a professional plumber to come over to fix it. I doubt he'd pay that plumber $200 if he left his house without actually fixing the faucet. Yet that's just what he did with his PC.

It's a shame because for what he paid he could have had a Mac or a nice Dell workstation. Instead he got a white box clone that for two years didn't even write his pictures to CD, which is what he bought it for. I felt a bid bad for him for what he got for what he spent. Maybe if they didn't have so many radio ads their prices could come down some.

I'm pretty sure that most computer users are around his skill level. I'm also pretty sure a person who has only basic tech skills and knows how to use Google could make an OK living doing what I did, coming into people's houses and fixing IE start pages. Not that I would want to do that for money, no thanks.

The problem with doing it for money would be that you would go in and fix all the stuff, then two months later the user would be calling you saying that their IE start page has been changed again, the popups are back, and they're getting redirected off Google. Except when they call you back they'll say it's your fault and want you to come and fix it again, this time for free. So no, I'll stick with professional programming.

One thing I found interesting was my neighbour could not understand the difference between the Google start page and the IE address bar. He couldn't understand what the address bar was for or how it was different than typing something into Google. I tried to explain that the address bar will go straight to the site you want. But he just wasn't grasping it, I guess it is a subtle thing to many users. When I started his IE last night, the address bar wasn't even turned on.

What he likes to do is use Google as his IE start page. From Google he just types what he wants, then clicks the first link in the results. So if he wanted to go to Ebay, he would type ebay.com in Google, then use that to get to Ebay.

I once read that people go to Google, then type amazon.com and click through from there to Amazon. A guy from Google said he couldn't understand why people do that. I've now explained here why. People either have their address bar in IE turned off or pushed to the right, or don't know what the address bar is or how to use it. So they use trusty Google as their default IE start page, then type in where they want to go then click through from the Google results page. I can't imagine how much money Google must be making off selling that top link in the results. I can tell you that probably most users don't realize it's a sponsored link.

Wednesday, August 24, 2005

More Python

We're doing some stuff at work where we need to get the Base64 encoding of the SHA-1 hash of a password. Here's how to obtain it in Python


import sha1
import base64

print base64.encodestring(sha.new('password').digest())


That's it. One line of code. No download of 3rd party. No worrying about licensing or purchasing something.

Try it in C or Java. In Java the JDK sees fit to provide neither SHA-1 or Base64. So write your own or try to download or purchase something.

In C there's RFC 3174 for SHA-1. You can try to get it to compile on your system and get it into your build. Have fun with that, I did a couple of years ago. Like Java, you'll have to thrash around the Internet for Base64 or write it yourself.

Compare all that hassle with C/Java, compared to one line in Python using built in components. Python is just better.

Saturday, August 20, 2005

Value of JUnit?

I just wrapped up my part in a project which has now gone into testing. It went pretty well. I didn't do any JUnit stuff. I haven't done any JUnit in more than a year.

I actually like JUnit. It's a good way to generate automated tests. It is pretty easy to learn and use. So if there was more time then I'd be motivated to use it.

What should you do if there isn't quite enough time for JUnit? Don't get me wrong, I test and the code I write tends to have a low defect rate.

JUnit is kind of the "formal" test for the code. This type of test is a permanent type of test involving planning, source control of test code, potential peer review, ongoing maintenance. So there's overhead involved.

For me, JUnit is the third level of testing. The first level is to print the code after I write it, then go over it carefully looking for errors. I find most errors through this static self inspection. For schedule reasons we don't do peer review, which is unfortunate because I am a fan of peer review.

The second level is informally running the code. Create a scratch J2SE client program that exercises the new code. Or click on the screen that will invoke the new code and check that it works as expected. The important thing is to exercise the new code in scenarios that are as close to possible to how it will be invoked in production. This will give you confidence that the code will be OK in black box integration testing.

The third level is formally testing the code within a controlled environment like JUnit, where the formal tests go into source control and are maintained along with the actual code.

So the JUnit level is basically a formal retest of code that you know works. My experience has been that the effort for levels 1 and 2 (static code inspection, informal scratch testing) is a fraction of that required to do JUnit. So it is possible to ship robust, tested code without requiring the overheads of JUnit. So focus on checking your code and scratch testing, and only do the JUnit at the end if there is time.

An issue I do have with JUnit is that it tests code in unrealistic scenarios that will not happen in production, while missing scenarios that will happen in production. Earlier this year, I was team lead on a project. This project involved a Tomcat WAR that processed a formatted URL and sent a message over JMS to a J2EE server that invoked an EJB call that did a bunch of stuff. The code was written and ready to test.

The developer responsible for that section wanted to set up JUnit for the different pieces. I said don't bother with JUnit, do an end to end test instead. Set up the components, invoke the Tomcat URL, and verify from the database and the logs that it works end to end. Well he did what I asked, then he spent several days sorting out deployment issues, permission issues and glitches between the components. But he got it working and it worked when we went to production. We never did do the JUnit for that, and I'm glad we didn't. The issues the developer found would only have been uncovered by doing an end to end test of the running system. If we'd done JUnit instead, we would have looked bad as the problems would have been found by the test team.

In my opinion the bulk of value in JUnit is for refactoring. It's also valuable for automated test of working code. The automated test is basically a form of refactoring test. You take something that is verified to work with JUnit. Then if someone inadvertantly breaks your code by changing something you call, then the automated test will catch it. But code has to exist and work before it can be refactored. And refactoring of new code generally only happens after the current project ends. Also let the person doing the refactoring absorb the cost of JUnit, since he is the one who benefits from it.

My observation is that there is no link at the individual level between use of JUnit and defect rate. Some mediocre developers are JUnit fans, yet it doesn't seem to improve the quality of code they produce. Some developers write lots of good, low defect code while spending little time on JUnit.

My theory is that strong developers will write good code and if they do JUnit then they'll do a good job of that too. But for the good developers the JUnit is not really necessary as the code is high quality regardless. On the other hand, weak developers will write mediocre code. The JUnit will be of similar lower quality and there will be more defects regardless of how much time they spend on JUnit.

Tuesday, July 05, 2005

How to write a Java Object to a database blob column

This snippet describes how to write a Java Object to a database BLOB column, then read it back. The Java Object must be Serializable so that it can be represented as a byte array for either writing or reading.

First, writing to a blob column. For this example, there is a table called blobtable. It contains a BLOB colunm called blobcolumn. The way to write it to the DB is to assign the Serializable to a byte array, then write that byte array to the DB.


public void saveObjectToBlob(
        Connection conn,
        Serializable saveObject)
throws IOException, SQLException {

    PreparedStatement stmt = null;
    String query =
        "INSERT " +
        "INTO blobtable (" +
        " blobcolumn) VALUES (" +
        " ?) ";

    stmt = conn.prepareStatement(query);

    // write the object content to a byte array
    ByteArrayOutputStream bos = new ByteArrayOutputStream();
    ObjectOutputStream oos = new ObjectOutputStream(bos);
    oos.writeObject(saveObject);
    oos.close();
    byte saveObjectBy[] = bos.toByteArray();
    stmt.setBytes(1, saveObjectBy);
    // Oracle converts it to BLOB automatically

    stmt.executeUpdate();
}


After writing it to the blob column, you can read it back. The way to read it back is to use the Blob interface for the DB blob column. Read it back as byte array, then cast that to Serializable. Then cast the Serializable to the original Object type.


public Serializable readObjectFromBlob(Connection conn)
throws SQLException, IOException, ClassNotFoundException {

PreparedStatement pstmt = null;
ResultSet rset = null;
String query =
"SELECT " +
" blobcolumn " +
"FROM " +
" blobtable ";

pstmt = conn.prepareStatement(query);
rset = pstmt.executeQuery();
Serializable result = null;
if (rset.next()) {
byte blobcolumnBy[];
Blob bl = rset.getBlob(1);
blobcolumnBy = bl.getBytes(1, (int) bl.length());

ByteArrayInputStream bis =
new ByteArrayInputStream(blobcolumnBy);
ObjectInputStream ois = new ObjectInputStream(bis);
Object obj = ois.readObject();
ois.close();
result = (Serializable) obj;
}

return result;
}

Monday, July 04, 2005

Brown, J2EE, tractor trailer

I recently was working on an application that was Servlet based in Tomcat. That was pretty good. It was nice to be able to roam around the Java language and libraries and do stuff with threads and such.

Before this most of my Java experience was EJB. I don't really like EJB. It seems annoying a lot of the time with its rules and restrictions. And it's so verbose and so much overhead getting things deployed and callable. We've had some success using XDoclet to auto-generate some of the verbiage, but it's still big. We're not even using entity beans, thankfully; that would be just so much more overheads.

We have to be quick to get things done on aggressive schedules and be able to modify existing systems for new requirements. Sometimes with J2EE it seems like we're a courier trying to deliver packages in the downtown business district using a tractor trailer. The big rig is scalable; you can load all kinds of heavy freight on it, but in the downtown during the daytime it is so bulky that it has trouble navigating the traffic, changing direction or parking. For the traffic congested downtown you'd be better with one of the UPS brown vans, which carry a big enough load, but can move around better in thick traffic and tight parking and can change direction easily based on how business unfolds that day.

Now if you're moving say 40 brand new large sofas from a warehouse in Montreal to a furniture store in Sydney then the tractor trailer is the best vehicle to use. You can fit them all in one load, and the rig manages the long highway miles easily.

Looking at it that way, it makes me sometimes get down a bit on J2EE. But thinking about it more, I think my gripe is more with EJB than with the full J2EE. After all I know I like the Servlets stuff. EJB just seems to suck up the joy of programming at times.

I know I'm not the only one who isn't excited about EJB. When people write entire books about J2EE without EJB then there are many others too who want to use J2EE but are turned off by EJB. Still with J2EE, especially EJB the specs are just eye glazing. Even a book like the 21 days that tries hard to be accessible and reader friendly is painful to read when they necessarily delve into the J2EE specs and rules. I wish the concepts and terminology could be somehow simplified.

I always thought that EJB was a necessary evil to obtain scalability. Turns out that's not so. walmart.com was built using just Apache and Tomcat servlets. Apparently no need for EJB to achieve performance, security, etc. I find that very interesting.

So if we don't need it to get scalability then maybe we'd be better off without EJB. Just use servlet - which I've found well thought through and agreeable to work with - plus a much more lightweight framework, or maybe no real extra framework. That idea appeals to me. I'll get to work with Tomcat and servlet for a bit longer on this project, I'm not real excited about going back to EJB.

Tuesday, June 14, 2005

SAS

I like to read ComputerWorld Canada, especially since I get it for free. Every few weeks or so they have an article where they mention SAS. They always seem to have something good to say about them.

Like this recent article about software quality where they use SAS as a positive example of a software company where quality is a priority. Go to ITWorldCanada, enter code 050562 in the quick link, regrettably there's a one-time free registration, then a couple more clicks to the article.

Based on the ComputerWorld Canada articles, it seems like SAS would be a great place to work as a developer: focus on quality led by the CEO, privately held so less of the quarterly shenanigans, hiring during a down period because there was good talent floating around after the dot com bust. They seem to show that you can do software "right" and still be successful.

If I was single and more free to relocate I think I'd want to apply to work there.

Tuesday, June 07, 2005

Oracle PL/SQL function to get current time in UTC

This snippet is an Oracle PL/SQL function that returns the current timestamp in the UTC time zone. It takes advantage of the EXTRACT Oracle built in returning its result in UTC.

FUNCTION getUTCDate RETURN DATE
AS
  utcYear NUMBER(4);
  utcMonth NUMBER(2);
  utcDay NUMBER(2);
  utcHour NUMBER(2);
  utcMinute NUMBER(2);
  utcSecond NUMBER(2);
  dateString VARCHAR2(30);
  ts TIMESTAMP WITH TIME ZONE;
BEGIN
  ts := SYSTIMESTAMP;
  utcYear := EXTRACT(YEAR FROM ts);
  utcMonth := EXTRACT(MONTH FROM ts);
  utcDay := EXTRACT(DAY FROM ts);
  utcHour := EXTRACT(HOUR FROM ts);
  utcMinute := EXTRACT(MINUTE FROM ts);
  utcSecond := TRUNC(EXTRACT(SECOND FROM ts));
  --
  dateString := TO_CHAR(utcYear) || ':' ||
                TO_CHAR(utcMonth) || ':' ||
                TO_CHAR(utcDay) || ':' ||
                TO_CHAR(utcHour) || ':' ||
                TO_CHAR(utcMinute) || ':' ||
                TO_CHAR(utcSecond);
  RETURN TO_DATE(dateString, 'yyyy:mm:dd:hh24:mi:ss');
END getUTCDate;

Monday, June 06, 2005

The source of your source code

In the recent death march project, I was responsible a large block of inherited code. I don't mean inherited in the object oriented sense. I mean that the code was written by a different office and transferred to our office for use in the project. The intention was that by reusing the original code we could save time against building the module ourselves and help to meet the project schedule.

We would be using the inherited code using a slightly different workflow than the original deployment. Also we were switching the environment from Windows server to Solaris. Since it's Java it didn't need to be modified for platform.

Unfortunately in testing there were a some problems in the inherited code. I had to pull some all-nighters to fix some issues. hint: with servlet you need to be really careful about thread safety. hint2: well placed source comments can greatly assist someone who is not the original developer who may be maintaining your code in the future.

So it goes. Hard experience gained. Going forward we now have a track record from that office and if a future situation like this arises then can know what to expect and can better plan for how to deal with it.

It may be that every line of code you write is a liability. However every line of code you inherit from outside sources is a risk. In general, deal with widely used sources that have a good reputation like Apache. Whether internal from your organization or external from the Internet, be careful of code from previously unused sources. Be aware of the risk that the code will not perform as expected. Integrate it early, to allow the most time for testing and to deal with problems which may occur.

Thursday, June 02, 2005

Tech support dilemma

A former co-worker once described tech support from a large, well-known software vendor like this: "I found it was better to stand in front of a mirror and describe the problem to myself than to try to get help from XYZ support."

Why is tech support generally bad? The problem may be that once people get to a point where they can be effective at it, then they are at a point where they can be better used elsewhere. Suppose individual X does tech support for a large, complex software product; say a GIS system.

X is good at it, with deep understanding of the underlying specifications, full knowledge of the intricacies and glitches of the product in several functional areas, ability to quickly read customers (generally incorrect) source code and fix problems, communicates well with customers, etc.

What then becomes of X? Probably X will be transferred out of tech support into the consulting, testing or programming groups. In the consulting group the XYZ company can charge more for X's skills. In testing or programming X can bring a greater good than in tech support. In support you can basically help one person at a time and a fixed limited number overall. In the programming or test teams, you advance the product, which benefits everyone who uses the product. Thus more people benefit from X's efforts which should be more profitable to XYZ corp.

X's employer could decide to keep X in tech support because she is very good at it and her customers are happy. However this probably won't last because X can take her skills to a promotion out of tech support at a different company. So if XYZ doesn't promote X out of tech support then another company will.

So this may be why tech support is often bad. Once people get good at it they can be reassigned to more profitable areas. Also because of the cost and learning curve around big, complex software systems, it is better to keep around those tech support who aren't very good because turnover is expensive and the new replacements you bring in generally won't be better than those they are replacing.

Tuesday, May 31, 2005

Sleeping on an office floor

We're finally finishing up a real death march project. Several times over the two week crunch period, I along with several others had to pull all-nighters. It was an unfamiliar experience for me. A different life/career experience, but not something I want to start doing every quarter.

One night I fixed a showstopper bug in some inherited code. More on that later. Anyway I was just on standby if anything broke in my module. At around 1:30 AM I could barely stay awake. So I decided to grab a nap during the lull. In our office there was once a lot more people, so there is an empty upstairs area where there used to be a manager's office that is now vacant. I went up there and layed down on the carpet floor. I rolled up my jacket underneath my head to lay on. It was comfortably warm upstairs.

The sysadmin has a work area just outside the vacant office. There is a server there. The server creates a pleasant droning sound. Listening to the server droning, I was able to fall asleep for a good hour.

I awoke around 2:30 feeling much better. I had a thought when I woke up to sleep another hour and check on the testers downstairs then. But virtue got the best of me and I went downstairs to check how things were going. They were just getting frazzled about some new error. Luckily this time it was a configuration problem in a simulator and not a software error. I pointed out what they needed to change and they fixed the configuration and continued testing. Good thing I went back downstairs when I did.

You'd be surprised when you're that tired how comfortable the carpet floor of a vacant office can be.

Monday, May 30, 2005

Oracle PL/SQL function to convert dotted-quad IP address to number

This snippet is an Oracle PL/SQL function that will convert an IP address from dotted-quad format (like 66.102.15.100) to the corresponding numeric value. This can be useful in a table where an IP address is entered by the end user in the familiar dotted quad format and a trigger is used to keep the numeric value of the IP address in synch. For the application, just use a regular expression to validate the IP address entered by the user.

FUNCTION dottedQuadToNumber (dottedQuad IN VARCHAR2)
RETURN NUMBER AS
BEGIN
  RETURN (
    (substr(dottedQuad,
            1,
            (instr(dottedQuad, '.', 1, 1) - 1))
            * 256 * 256 * 256
    ) +
    (substr(dottedQuad,
            instr(dottedQuad, '.', 1, 1) + 1,
            instr(dottedQuad, '.', 1, 2) -
            instr(dottedQuad, '.', 1, 1) - 1) * 256 * 256
    ) +
    (substr(dottedQuad,
            instr(dottedQuad, '.', 1, 2) + 1,
            instr(dottedQuad, '.', 1, 3) -
            instr(dottedQuad, '.', 1, 2) - 1) * 256
    ) +
    (substr(dottedQuad,
            instr(dottedQuad, '.', 1, 3) + 1)
    )
  );
END dottedQuadToNumber;

Oracle PL/SQL function to convert numeric IP address to dotted-quad

This snippet is an Oracle PL/SQL function that will convert an IP address from numeric format to the corresponding dotted-quad string.

FUNCTION numberToDottedQuad(dottedQuad IN NUMBER)
RETURN VARCHAR2 AS
    item NUMBER;
    part1 VARCHAR2(3);
    part2 VARCHAR2(3);
    part3 VARCHAR2(3);
    part4 VARCHAR2(3);
    divisor NUMBER;
BEGIN
    item := dottedQuad;

    divisor := 256*256*256;
    part1 := TRUNC(item / divisor);
    item := MOD(item, divisor);

    divisor := 256*256;
    part2 := TRUNC(item / divisor);
    item := MOD(item, divisor);

    divisor := 256;
    part3 := TRUNC(item / divisor);
    item := MOD(item, divisor);

    part4 := item;

    -- piece them together for the result
    RETURN part1 || '.' || part2 || '.' || part3 || '.' || part4;
END numberToDottedQuad;

Sunday, May 29, 2005

gosub and design patterns

I started programming in BASIC on the Commodore 64 back when I was in Jr. High. At the start I used the GOTO statement to control program flow.

After a few months I was excited because I found out about this useful command called GOSUB. The GOSUB command allowed me to reuse blocks of code and the system would conveniently return control back to the calling point when the GOSUB block was done.

Later on in university I took the intro to programming in Pascal. An early topic in the course was "subroutines", expressed in Pascal using procedure and function. When they were talking about subroutines I immediately recognized this as GOSUB.

So "subroutine" can be considered a design pattern. It's a generic way of doing things that can be reused in many contexts. With GOSUB, I was using the subroutine design pattern without knowing about the formal label for it.

I've never read Design Patterns. I don't really plan to. A number of people I work with have. I've never seen anything in their code that is startling that makes me think I'm missing something important. My theory is that it's like with GOSUB. I already use the stuff, I just don't have the special labels for it.

If I ever lose my job and am out of work I'll have to read it so that I can be considered "competent" by the shops where it's a big deal.

Casey

The genX'ers may remember Bill Casey from the 80's. He was the CIA director for Ronald Reagan. Casey was involved in some way in the Iran Contra scandal. Unfortunately shortly after the scandal broke Casey died suddenly.

After Casey's death, suddenly everyone was blaming Casey for everything about Iran contra. It was convenient to lay it on Casey. He was involved in it. And since he was dead he couldn't defend himself.

I've seen and heard about this Casey effect in some high tech offices. Basically there's someone who was at least somewhat important for some time. That someone may have made some mistakes along the way or been disliked by some. That person is eventually fired or quits and is no longer around.

After he's gone and can't defend himself, he then becomes a Casey. Every problem, failure, screwup and lost sale is blamed on him. This blaming can go on for a while. A friend of mine is an electronics engineer in BC. He said there was a Casey in his office. That guy was blamed for stuff that happened after he left that he wasn't involved with.

I suspect the Casey effect is not limited to just high tech.

Thursday, May 26, 2005

Fixing the UI EJB Disconnect

Where I work we use Java/J2EE with the "beans and screens" model. We use JSP/struts for the user interface, backed by business layer EJBs. The business layer EJBs are backed by a database abstraction layer. We don't use entity beans - regular business layer EJB is verbose and overhead enough, thank you. We do quick HTML mockups of the UI screens before formal JSP development begins.

The beans and screens model works pretty well. Business logic is well separated from presentation. We have some solid EJB developers and some solid UI developers. I haven't done any UI myself, and that doesn't really bother me.

What can happen with beans and screens is the “UI EJB disconnect” problem. What happens is the EJB developer goes off and implements some methods as he sees fit. The UI developer later on attempts to wire the UI screens to the EJB methods. What I've seen is that the EJB developer doesn’t seem to know what signatures he needs to create. The UI developer often doesn’t have the methods that are needed to build the screens. This disconnect causes lost time on the UI side thrashing around looking for methods to build screens. Time is lost on the EJB side creating methods that are not used, and scrambling to put methods in quickly for UI requirements.

This situation is not ideal and should be resolved. The UI people should not have to thrash around looking for methods to build a screen (since the screen layout is known from the mockup). The EJB people should not be blindsided by unexpected UI method requirements.

To me it is a communication issue. Instead of going off and implementing signatures as he sees fit (trying to guess what the UI will need), it would be better if the EJB developer was told what signatures he needs to implement. This would also be communicated to the UI developer so when he codes a screen it is known in advance what the backing EJB calls will be.

The use of metaprogramming can fix the UI EJB disconnect. This is a top down approach to EJB specification where a metaprogrammer brokers the conversation between the UI and EJB developers. When one developer will need an EJB call to be provided by another developer, the metaprogrammer specifies to both developers the call signature and describes how to implement it. It is the metaprogrammer's job to anticipate the EJB calls that will be needed in advance and specify them before the client programmer gets to the point of needing to call them.

I tried this technique on a project where I was the team leader with around six developers working for me. It worked pretty well. There were no instances of a client developer being stuck needing an EJB call from someone else that hadn’t been specified yet. The UI developers seemed to like it. One thing that's important is to have fast turnaround of stub methods on the EJB side. When the metaprogrammer gives an EJB developer a method signature to create, the EJB developer needs to be required to put in place the signature and a stub implementation right away. The actual full implementation can be done later. This will avoid a situation where the EJB developer is given a signature to create, but doesn't get around to it in time, and the client developer is ready to call the stub signature that hasn't been created yet - causing delays for the UI client developer.

You may be reading this and thinking "Why not just fully specify all the EJB signatures up front as part of the design phase? Then it will be ironed out before coding starts and you won't have this problem." That's a good question. In an ideal world this would be a nice way to do it. Where I work the schedules are compressed, and doing that type of full design up front would end up consuming the entire development cycle and not leave any time to implement the code.

Also some shops are not fans of "big design up front", realizing that the built system typically differs substantially from the original feature set and design. In a more agile environment you need to be able to move quickly with short development cycles focusing on releasing new working features. So you need to advance the product in a way that is technically sound, yet requires little overhead.

Thursday, May 05, 2005

Python: it's just better

I just did a utility in Python where I took the content from a large table from a technical manual in Word and displayed it on a GUI in a tree view. I just exported the Word table to a delimited text file and the Python script parsed the text file and built up the content for the tree view. It was not too difficult.

That's the thing with Python. Everything just seems easier and faster. With Python you'll be trying to do something, then you'll find the way to do it in Python is just ridiculously easy.

Python is better because it addresses the programmer impedance issue. When I work on a technical problem, I like to solve the problem in my mind, then just type up the solution using the editor. With other languages, programmer impedance occurs when I have to translate how I think about the solution in my mind to how I express that solution using a programming language. With Python it is generally one-to-one how the solution in my mind maps to statements in Python. So the impedance is eliminated or at least massively reduced.

Here's an example. I once had a situation where I needed to "select an item from the set at random".

In Python that can be expressed as random(set). No impedance.

In say, Java I have to translate. I have to do something like
get the size of the set
get a random integer x between 0 and set size - 1
get item x from the set
print the code and read it carefully checking for off by one errors

That's an example of the impedance. By the time I obtain the random item in Java I've lost my flow and have to spend time getting momentum back. In Python with no impedance I don't lose my train of thought.

I'm not picking on Java. I like Java. I've worked with Java and C/C++. I consider Java better than C/C++. I consider Python better than Java.

I've read some people who say that the programming language doesn't matter. What's important is that there be a vast standard library. This may be true when comparing Java to C/C++. For me what makes Java compelling compared to C/C++ is the vast standard library, built in GC, platform independance, built in threading.

However looking at the modern language landscape with Java, C# and Python, the things that gave Java an advantage over C/C++ are just commodities today. Any modern language will have the large standard library, OO, GC, multi platform, and threading out of the box.

So some other commentators got it wrong. It is about the language. And Python is just better. Besides fixing the impedance issue, Python also eliminates the compile cycle. That alone yields a significant developer productivity increase.

Monday, April 18, 2005

xwave on a roll

I noticed my former employer xwave has been doing well lately. They've posted more jobs in the last two months than in the previous three years.

I've sometimes wondered about leaving xwave. I wonder how things would have turned out if I'd stayed there. It was a good place to work and I was well treated there. I didn't leave for any real good reason. It was kind of a strange time, around mid 2001. Like most IT companies, xwave went from high flying to struggling. They did their first round of layoffs in the former PRIOR office on Spring Garden Road. For myself though, I didn't have anything substantial to complain about. I just had it in my mind to make a change.

I've done well with my current employer. It's a good job and I'm happy with it. I'm not looking. The work is interesting and the cubicles are nice. They bring in free lunch and snacks often. Still, the jury is still out on if the xwave jump was really a good move. Perhaps it always will be out. Now if my employer's stock price would rise a few ticks and get those options nicely into the black then that might bring about a decisive verdict.

If you're thinking about leaving your job then I'd suggest to consider this
  • Don't quit to make a lateral move. Don't go from programmer at company A to programmer at company B. Get ahead in more ways than salary.
  • In general, don't quit a good job.

Where I work, I know of people who quit to go somewhere else, then asked to come back after things didn't work out as hoped with the new company. You can always keep your job and be unhappy, at least you're gaining seniority and accumulating benefits.

Sunday, April 10, 2005

Programming

I like to program. I get paid to program. I like to talk about programming.

I started programming in grade 8 with the good old Commodore 64. Those were the best times for progamming.

After those C64 experiences, I went on to university. Eventually I finished a CS degree from TUNS. TUNS is now Daltech. I have all good to say about TUNS and the old TUNS CS program. I consider a TUNS degree prestigious and I am pleased with myself to have one.

After TUNS I joined PRIOR Data Sciences. I liked PRIOR. Like TUNS, I consider it prestigious to have been part of it.

PRIOR was taken over by xwave in Feb 2000. xwave didn't quite get untracked with me and I left there in July 2001 and joined Core Networks. In September 2004 Core was acquired by SupportSoft which is where I work now. If I write anything grouchy about work then to yourself think "he must be talking about Core"

Luckily for me I've never been out of work in high tech. I'm confident that if I was out of work I'd be able to get something comparable to what I currently have. However, I'm not eager to test that theory.

Right, back to programming. I just want to mention the different languages and stuff I've worked with over the years.

1997 - 1998: Ada 83
1999 - 2001: Oracle SQL, PL/SQL, Oracle Forms, Oracle Reports. MS-Access.
2001 - 2004: C, C++, PHP, HTML, Javascript, Oracle SQL, PL/SQL
2004 - 2005: Java, J2EE, Oracle SQL, PL/SQL

Also I've now converted to Python, and I use that for scripting. I still have to work with Java though most of the time.

I expect the list to continue to evolve. I'm not convinced that Java will be the last stop on this line.