I'm doing something that many people in high tech do. I'm leaving my job to join a new company. I've been with SupportSoft and its acquired predecessor Core Networks since 2001. I'm joining Research In Motion later this month.
For me a job switch is a big move. I've only done one before, going from xwave to Core Networks in 2001 after 3 years at xwave and its acquired predecessor PRIOR Data Sciences. So many acquisitions in tech. Some people jump all the time, around every two years on average. But that's not my style and I'm personally a little bit suspicious of serial jumpers.
It's pretty wacky how networking and chance events can lead to new things. Last Christmas break I went to an annual road hockey game. There I met a guy who I knew over the years and worked with at PRIOR in the late 1990s. He's director level now. He was finishing up at his job after several years and was looking for a new job. We agreed to keep in touch over LinkedIn when he found a new job in case he was hiring additional people. As it turns out he landed at RIM in the first part of the year and that lead to me getting interviewed. I don't normally go to road hockey games expecting a new job to come of it but it just shows what can happen.
I don't have anything bad to say about SupportSoft. The local job market has been pretty good since around 2004 so if I was unhappy then I could have chosen to leave at any time. This just looks like a promising opportunity which I believe is in my best interest to take.
Every job has its ups and downs. Good things and little annoyances. I think expatriates mostly look back well on previous employers; even if they grumbled a bit when they worked there. I look back kindly on my time with PRIOR and xwave and I'll look back well on my seven years with Core and SupportSoft.
Wednesday, April 16, 2008
Saturday, April 05, 2008
Java GUI coming of age
Somewhat quietly the Java JDK GUI has improved. Especially since Java 5.
I'd kind of given up on swing over the years. Everyone has bad memories of traditional Java GUIs. Battleship gray, clunky. Control-C didn't work on Windows, instead it used Unix semantics like Control-Insert for copy and paste. Select a piece of text and right click the mouse. Nothing happens.
If there was one thing which made Eclipse it was developing SWT as a much superior end user experience for Java GUI. With native widgets Eclipse was so much better. Suddenly Ctrl-C works properly on Windows. Select a piece of text and right click the mouse and the context menu comes up. Finally the GUI just looks and works the way we expect. At the time it came out SWT was the obvious and superior choice.
I have to give Sun credit. They didn't give up on JFC. They obviously worked hard to improve it. Now in Java 1.5 they pass the "10 foot test" for the first time. That is, standing 10 feet away from the computer, you can't tell that it's a Java GUI. It looks and acts much smoother now.
One of my favorite programs that I use most every day is Oracle SQL developer. This uses the JDK GUI and it is just fine to work with. It looks and feels great. The standard keyboard and mouse actions all work the way you expect. It's plenty fast.
For the first time developers can consider using swing for serious Java GUI applications.
I'd kind of given up on swing over the years. Everyone has bad memories of traditional Java GUIs. Battleship gray, clunky. Control-C didn't work on Windows, instead it used Unix semantics like Control-Insert for copy and paste. Select a piece of text and right click the mouse. Nothing happens.
If there was one thing which made Eclipse it was developing SWT as a much superior end user experience for Java GUI. With native widgets Eclipse was so much better. Suddenly Ctrl-C works properly on Windows. Select a piece of text and right click the mouse and the context menu comes up. Finally the GUI just looks and works the way we expect. At the time it came out SWT was the obvious and superior choice.
I have to give Sun credit. They didn't give up on JFC. They obviously worked hard to improve it. Now in Java 1.5 they pass the "10 foot test" for the first time. That is, standing 10 feet away from the computer, you can't tell that it's a Java GUI. It looks and acts much smoother now.
One of my favorite programs that I use most every day is Oracle SQL developer. This uses the JDK GUI and it is just fine to work with. It looks and feels great. The standard keyboard and mouse actions all work the way you expect. It's plenty fast.
For the first time developers can consider using swing for serious Java GUI applications.
Tuesday, March 18, 2008
The computer and the network
Social networks are interesting. They represent the evolution of the computer and network.
First there were PCs. Then came local networks with printer and file sharing and e-mail. Originally the networks were to extend the capabilities of the PC. They were originally called "computer networks", i.e. computers linked together. This changed over time and the network became more important. Instead of the network being useful to the PC, the PCs purpose was to enable a network.
This continued as the networks extended out to the Internet and Web by the late 1990s.
Today we are seeing the next step in the evolution. The underlying network is becoming less important. The social network is becoming more important. Instead of social networking being an application of the Internet network, the network's purpose is becoming to enable social networking.
Social networking like Facebook is interesting because the network on its own is just inanimate technology. Instead of connecting computers to other computers, social networks are about enabling people to connect with other people. It becomes less about the technology and more about people meeting and interacting.
First there were PCs. Then came local networks with printer and file sharing and e-mail. Originally the networks were to extend the capabilities of the PC. They were originally called "computer networks", i.e. computers linked together. This changed over time and the network became more important. Instead of the network being useful to the PC, the PCs purpose was to enable a network.
This continued as the networks extended out to the Internet and Web by the late 1990s.
Today we are seeing the next step in the evolution. The underlying network is becoming less important. The social network is becoming more important. Instead of social networking being an application of the Internet network, the network's purpose is becoming to enable social networking.
Social networking like Facebook is interesting because the network on its own is just inanimate technology. Instead of connecting computers to other computers, social networks are about enabling people to connect with other people. It becomes less about the technology and more about people meeting and interacting.
Sunday, March 02, 2008
Application performance testing and optimization
I've been assigned to do performance testing for an upcoming application release. We made some architectural and database changes so we want to be sure our hardware estimation process is still valid. Also we want to try it out on the newer Sun T Series servers.
Along with measuring performance I can identify areas for performance improvement and optimize where possible.
Application performance is a bit like navigating through water of unknown depth. With a canoe you can paddle happily along on a shallow river. With software the canoe corresponds to one developer or tester clicking along through screens with very small data sets.
Moderate load is like navigating a 40 foot yacht. With the yacht there's more draft so if the water is very shallow then you'll run aground. The 40 foot boat would be like around 5-10 developers using the application at the same time with a modest size data set.
Heavy load is like the aircraft carrier. You need to be very deep to be able to handle this very large boat. Heavy load is when you simulate large numbers of simultaneous users and have a large data set. The load and data set size should be the same as what you plan to use in production using the same hardware.
With software like with the waterway, you don't really know about how it performs until you test it under load. You can't tell by looking at it. Taking a canoe or a small pleasure craft through a harbour does not tell you if the water is deep enough for a massive freighter ship.
Optimizing performance is fairly straightforward especially at the start. It is an 80/20 situation. 80% of the resources are consumed by 20% of the features. So using tools like JProbe it is easy to find the hot spots. Typically optimizing the small number of trouble spots will dramatically improve performance and then you're done.
This can be frustrating to the programmer because the same optimization patterns can be applied throughout the code base but the other areas don't use enough resources to justify the investment required to refactor for performance.
Although after the first iterations when the biggest resource hogs are dealt with some of the other problem areas that were hidden by the original optimized modules now become part of the 80 in the 80/20 rule and they can then be optimized.
In most applications, the biggest performance issues are around the database. This can be caused by inefficient queries that the DBMS cannot execute quickly. It can also be caused by a poor indexing strategy (or no indexes!) on tables which have large data sets.
A good free tool for DB analysis is Oracle SQL developer. I find you can learn a lot by just copying and pasting application queries into SQL developer. In addition to seeing the execution time you can also get the explain plan in an excellent graphical view.
Any intermediate level or higher professional software developer should be aware of database execution plans and how to interpret them and optimize them. Even at the junior level a programmer should understand how indexes impact query performance in large datasets.
One of the many eternal performance headaches with EJB is around static data from the DB that is requested often but changes infrequently or never. If a trip to the DB is required every single time then this redundancy will consume a lot of resources and really slow down system performance and responsiveness.
While caching is an apparent answer to this, EJB and caching basically don't seem to go together. We've had good success though using ehcache to deal with this shortcoming of EJB. I recommend ehcache based on my experience with it in this project.
Along with measuring performance I can identify areas for performance improvement and optimize where possible.
Application performance is a bit like navigating through water of unknown depth. With a canoe you can paddle happily along on a shallow river. With software the canoe corresponds to one developer or tester clicking along through screens with very small data sets.
Moderate load is like navigating a 40 foot yacht. With the yacht there's more draft so if the water is very shallow then you'll run aground. The 40 foot boat would be like around 5-10 developers using the application at the same time with a modest size data set.
Heavy load is like the aircraft carrier. You need to be very deep to be able to handle this very large boat. Heavy load is when you simulate large numbers of simultaneous users and have a large data set. The load and data set size should be the same as what you plan to use in production using the same hardware.
With software like with the waterway, you don't really know about how it performs until you test it under load. You can't tell by looking at it. Taking a canoe or a small pleasure craft through a harbour does not tell you if the water is deep enough for a massive freighter ship.
Optimizing performance is fairly straightforward especially at the start. It is an 80/20 situation. 80% of the resources are consumed by 20% of the features. So using tools like JProbe it is easy to find the hot spots. Typically optimizing the small number of trouble spots will dramatically improve performance and then you're done.
This can be frustrating to the programmer because the same optimization patterns can be applied throughout the code base but the other areas don't use enough resources to justify the investment required to refactor for performance.
Although after the first iterations when the biggest resource hogs are dealt with some of the other problem areas that were hidden by the original optimized modules now become part of the 80 in the 80/20 rule and they can then be optimized.
In most applications, the biggest performance issues are around the database. This can be caused by inefficient queries that the DBMS cannot execute quickly. It can also be caused by a poor indexing strategy (or no indexes!) on tables which have large data sets.
A good free tool for DB analysis is Oracle SQL developer. I find you can learn a lot by just copying and pasting application queries into SQL developer. In addition to seeing the execution time you can also get the explain plan in an excellent graphical view.
Any intermediate level or higher professional software developer should be aware of database execution plans and how to interpret them and optimize them. Even at the junior level a programmer should understand how indexes impact query performance in large datasets.
One of the many eternal performance headaches with EJB is around static data from the DB that is requested often but changes infrequently or never. If a trip to the DB is required every single time then this redundancy will consume a lot of resources and really slow down system performance and responsiveness.
While caching is an apparent answer to this, EJB and caching basically don't seem to go together. We've had good success though using ehcache to deal with this shortcoming of EJB. I recommend ehcache based on my experience with it in this project.
Wednesday, February 20, 2008
End of the Line for ComputerWorld Canada?
I just got my ComputerWorld Canada last week. It seems like it's been a while since the previous one. I noticed they have gotten thinner the last few months.
I think January 2008 is the slimmest I've seen. It was only 18 pages. While the content is still good, the seeming gaps between issues and the slimming is a concern. I wonder if they will cease publication soon.
It would be too bad if that happens. I've been reading ComputerWorld Canada and its merged predecessors like InfoWorld Canada as long as I've been in tech full time. They are useful to get an idea what's going on at a high level outside of your own company and projects. Although ComputerWorld Canada has always been more focused on the corporate IT departments than on my roles in systems integration and development in an ISV.
I was getting electronic delivery for a while. Around the middle of last year I switched back to delivery of the printed issues and I've been reading them when they arrive.
The tech magazines have had it tough in recent years probably due to the rise of the Internet. It would be unfortunate if ComputerWorld Canada goes away but I guess I could find a site or feed with similar content easily enough on the Web.
I think January 2008 is the slimmest I've seen. It was only 18 pages. While the content is still good, the seeming gaps between issues and the slimming is a concern. I wonder if they will cease publication soon.
It would be too bad if that happens. I've been reading ComputerWorld Canada and its merged predecessors like InfoWorld Canada as long as I've been in tech full time. They are useful to get an idea what's going on at a high level outside of your own company and projects. Although ComputerWorld Canada has always been more focused on the corporate IT departments than on my roles in systems integration and development in an ISV.
I was getting electronic delivery for a while. Around the middle of last year I switched back to delivery of the printed issues and I've been reading them when they arrive.
The tech magazines have had it tough in recent years probably due to the rise of the Internet. It would be unfortunate if ComputerWorld Canada goes away but I guess I could find a site or feed with similar content easily enough on the Web.
Thursday, February 07, 2008
The cost of bugs in the field
It's definitely true that the cost of dealing with software bugs that make it into the field is orders of magnitude larger than finding and fixing them pre production.
After our recent reorg I'm now responsible for maintenance on earlier releases of the flagship product out of our office. On Monday I got a trouble ticket about a customer who was experiencing problems after upgrading between minor releases. They wisely trial the upgrades in their lab before going live.
The customer helpfully provided a detailed description of the symptoms, the log file excerpts, as well as a packet capture. It turns out the packet capture was particularly helpful. In the wire trace I could see we were unexpectedly doing an HTTP 500 response in a certain common valid configuration. Embedded in the HTTP 500 response was a stack trace generated by Tomcat. The stack trace pointed directly to the issue. It was a bug in our code introduced by the previous maintenance developer in the minor release.
Looking at the code diff from revision history against the stack trace, it was obvious what the error was. The code fix was just a couple of lines, add a null check the original developer missed and it would be good again. However getting that change "done" on a system installed on a customer site is a tremendous amount of effort.
First I had to set up my development environment for the earlier code base including Eclipse, the application server, database, Perforce, Tomcat and all of that. Task switching between releases is tedious and that burned pretty much a day. There was an extra requirement to reproduce the issue where there had to be more than one Tomcat instance so I couldn't just run everything off my own PC. It took a while to get a separate Tomcat up with the correct Tomcat version and maintenance version of the application source code. Altogether it took more than a day just to get set up and reproduce the issue.
After reproducing the issue the actual code change only took a few minutes to implement. Then I had to install the fixed code and verify it was now working properly.
All done right? With shipped code, far from it. Then I had to package up a new release using the official procedures. Then assemble a patch to upload to the customer along with patch install instructions I had to write. The code fix also has to be merged to other later releases that will need it. The Wiki sites tracking this all had to be updated.
In addition to my own time spent which was several days for about a 3 line code change, there was the support rep in my company who had to manage the ticket and communicate with the customer, as well he had to update his own running site for that release. Additionally the customer lost a lot of time diagnosing this issue and now they have to lose more time doing the upgrade.
All in all more than one person week has been consumed by a code error that was 5 minutes work to correct. That's what happens when code bugs go into the wild.
Compare this to the cost of finding and fixing it earlier. If it had been detected by the original developer or a peer during code review it would have been about 5 minutes to fix.
If the developer had found it in his testing it would have cost around an hour to do the fix, rebuild, redeploy and rerun the test.
If the test team had found it it would have been about half a day to do the fix, do another baseline build, update the ticket tracking system and the testers rerun their test, and close the ticket.
So at every stage it gets progressively more expensive to fix serious bugs. That's why it's so important to find the bugs before they get into production.
After our recent reorg I'm now responsible for maintenance on earlier releases of the flagship product out of our office. On Monday I got a trouble ticket about a customer who was experiencing problems after upgrading between minor releases. They wisely trial the upgrades in their lab before going live.
The customer helpfully provided a detailed description of the symptoms, the log file excerpts, as well as a packet capture. It turns out the packet capture was particularly helpful. In the wire trace I could see we were unexpectedly doing an HTTP 500 response in a certain common valid configuration. Embedded in the HTTP 500 response was a stack trace generated by Tomcat. The stack trace pointed directly to the issue. It was a bug in our code introduced by the previous maintenance developer in the minor release.
Looking at the code diff from revision history against the stack trace, it was obvious what the error was. The code fix was just a couple of lines, add a null check the original developer missed and it would be good again. However getting that change "done" on a system installed on a customer site is a tremendous amount of effort.
First I had to set up my development environment for the earlier code base including Eclipse, the application server, database, Perforce, Tomcat and all of that. Task switching between releases is tedious and that burned pretty much a day. There was an extra requirement to reproduce the issue where there had to be more than one Tomcat instance so I couldn't just run everything off my own PC. It took a while to get a separate Tomcat up with the correct Tomcat version and maintenance version of the application source code. Altogether it took more than a day just to get set up and reproduce the issue.
After reproducing the issue the actual code change only took a few minutes to implement. Then I had to install the fixed code and verify it was now working properly.
All done right? With shipped code, far from it. Then I had to package up a new release using the official procedures. Then assemble a patch to upload to the customer along with patch install instructions I had to write. The code fix also has to be merged to other later releases that will need it. The Wiki sites tracking this all had to be updated.
In addition to my own time spent which was several days for about a 3 line code change, there was the support rep in my company who had to manage the ticket and communicate with the customer, as well he had to update his own running site for that release. Additionally the customer lost a lot of time diagnosing this issue and now they have to lose more time doing the upgrade.
All in all more than one person week has been consumed by a code error that was 5 minutes work to correct. That's what happens when code bugs go into the wild.
Compare this to the cost of finding and fixing it earlier. If it had been detected by the original developer or a peer during code review it would have been about 5 minutes to fix.
If the developer had found it in his testing it would have cost around an hour to do the fix, rebuild, redeploy and rerun the test.
If the test team had found it it would have been about half a day to do the fix, do another baseline build, update the ticket tracking system and the testers rerun their test, and close the ticket.
So at every stage it gets progressively more expensive to fix serious bugs. That's why it's so important to find the bugs before they get into production.
Subscribe to:
Posts (Atom)