Archive for January, 2006

Waterfall is back!

Friday, January 27th, 2006

If you have the time, be sure to attend Waterfall 2006. Waterfall is bigger than ever! Forget Agile, forget XP, and go back to your roots. You’ll see speakers like Ken Schwaber, Kent Beck, Alistair Cockburn, Kent Back and the like. And for your listening pleasure, all sessions will be run sequentially. To hell with concurrency, I want to enjoy all of them! Mark your calendars for April, 1st. See you then.

Can’t wait to see Andy Hunt and Dave Thomas talk about Dogmatic Programming. I’ve heard so much about it, and it’s supposed to be awesome!

Via Approaching Normal and /\ndy’s Blog.

Technorati Tags:
, ,

Eclipse vs. IDEA - Making the Switch

Monday, January 16th, 2006

Okay, I admit it: I’m a switcher.

I’ve been using Eclipse as a Java IDE pretty much from the beginning of my Java developing days. Well, apart from the time I developed Java stored procedures for Oracle. It was and still is a good IDE. But I remember my boss constantly telling about how great IDEA is, and that it’s making the developer more productive with a lot of little details which are hard to describe but must be experienced. Being as stubborn as I am I didn’t really believe it. I gave IDEA a look a while ago and left it at that. Then finally I had some time to get over myself and have a decent and objective look at it and to really spend some time exploring its features. And soon I was hooked. Man, what a great IDE. After a while I could really see what my boss was trying to tell me. You can’t blame a particular feature, but you really have to consider those niceties that make developer live just a little bit more productive. Together they add up, so that in the end, it really can boost your productivity. You even tend to forget the argument about the huge amount of Eclipse plug-ins. For web and J2EE development, you won’t need any. Well, except for web services. Some stuff is very different from Eclipse, but you get used to it, because most of it turns out to make you slap on the forehead and ask yourself, why nobody thought of it that way before.

I wrote about open source and the IDE market a while ago, and maybe it’s time to think again. A colleague of mine put it that way: If a particular technology is to be pictured as an iceberg, then as a vendor you can only earn money, if you’re on top of the iceberg. From time to time an iceberg will melt, and it’s close to impossible to earn money with it. If you’re lucky, there’s a little ice-cube called niche market floating about, but that’s about it. An interesting metaphor which in my opinion fits perfectly into the IDE market. IDEA is definitely top-notch in its market. The free competition is fierce, but there’s always room for someone who likes to think about a specific task that needs to be done by most developers, and how to do it right, fast and without hassle. Eclipse picked up a lot of IDEA’s features, and they’re definitely on the run (being pushed by IBM and all), but we’ll see how it turns out in the end. And no, I won’t forget to mention NetBeans which has made some very good progress since I last gave it a horribly disappointing shot more than two years ago. With version 5.0 coming up, the Java IDE market is gaining some momentum. It’s gonna be interesting to see what’s coming for the Java IDEs.

If you’re one of these people who put free cost above time and money gained a small investment, you should give IDEA a try. It’s a great IDE made by people who care about getting things done. It didn’t win the Java Developer Journal Reader’s Choice award for best Java IDE for nothing.

Technorati Tags:
, , , ,

Avoiding monkey traps

Thursday, January 12th, 2006

In his latest book “My Job Went to India”, Chad Fowler talks about monkey traps. If you don’t know them: it’s a simple yet effective way used in South India to catch those darn monkeys that usually steal your food on the street or wherever you stand with something eatable with your hand. In short, they dig a deep whole that’s narrow on the top and gets wider to the bottom. They put some rice in there, and a monkey, food-loving as he is, would put his hand through the whole, grab as much rice as he can (making a big fist in the process), and would try to pull back his hand. To nobody’s surprise the fist is too big to fit through the whole. But the monkeys care more about the food in their hand then their life, so they’d rather die trying to get the food out of the whole than let go of it.

A nice metaphor compared to how things sometimes work with people’s opinions in software development/engineering. Well, it might be applicable to other areas of professions as well.

That made me think about my personal monkey traps (as it was supposed to, thanks Chad ;). I used to have some big ones, I’m with Chad on the Linux one. Has been a huge monkey trap for me. Since now I’m using Mac OS X, that changed, but probably not in the way it was supposed to. But now I don’t want to put Linux (or open source software in general) on everyone’s desktop anymore. Learned a valuable lesson here from my parents which, in the long term, would rather stick to Windows, and it was good. I also changed my mind about exclusively using open source software myself, and I started paying for stuff again. Even the my view on commercial IDEs has changed. When you finally see that there’s a reason way you could pay for software, though not for that ridiculously overpriced JBuilder, things change. Even if it’s for the benefit of improved productivity.

Another one used to be vi. It’s still one of my weapons of choice, and I still like to make jokes about Emacs, but who doesn’t? But since I delved into Java programming I got way more comfortable using a good IDE which can do stuff that vi (or Vim for that matter) can’t or takes hours and intimate knowledge to get it working. Somewhere along the way I lost the way to fiddle with all that stuff, and with Mac OS X I learned to love again, that is, the usefulness of a nice and clean UI. Well, it still helps, if there are lots of keyboard shortcuts to use. Only people who aren’t slackers use the mouse for everything.

But that’s just the one side of the story. I also used to think (well, who didn’t, because nobody told them in college or university) that a programmer just programs, and he does whatever he’s told to. To some extend I actually did think that. I didn’t stick up to that as a dogma, but it’s one of these misconceptions you can stumble over easily. Of course you do more than that as a developer, or at least you should, because otherwise you’re not offering enough value for your company to hold on to you. Some people get through that way, but it’s questionable, if that will work out or even satisfy you in the end. But I’m kind of stepping on the point of Chad’s whole book here.

I’m still thinking about my technological monkey traps. I’m not saying I don’t have any, that would be too good to be true. Everybody has some, be it CORBA, an object-oriented database, J2EE, the term enterprise application, .NET, unit tests, model-driven development, you name it. Maybe a key to avoid monkey traps in a technological manner is widen your own portfolio constantly. Learning new languages, playing with new frameworks, testing new tools to include in your toolbox, maybe these things can help to avoid common monkey traps. The more technologies you’re familiar with, the easier it is to have an objective and differentiated look at things. None of that “Ruby on Rails is crap, because someone told me so” crap. Or “Eclipse sucks, because it’s a product of IBM”, or even “Java is not proprietary, and Sun will love us all forever” (all statements might or might not reflect opinions heard or said by your humble author, ). Stuff like that, you get the idea. I for my part am a little resistant towards .NET, so I’ll probably opt for giving at least C# a try. Mono to the rescue. It’s been on my radar for a while, but maybe it’s time to see, where Java got its recent language update ideas from.

It’s also a way to get a perspective for always wanting to do things perfect. On the way to strive to write perfect code, have a perfect design, perfect team, perfect management, you name it, you tend to forget that it’s your own fist that holds you back. The perfectionist will be trapped by her own desire to be perfect. Embrace the The Good Enough Approach, and you can open your fist now, and let go of the rice.

Chad puts it right when he recommends that you give that technology that you hated since forever (or only since recently) a try. If you’re a J2EE evangelist, try doing something in .NET. Always hated unit testing? Write a test for your next class or bug fix. Having preconceptions about Ruby? Play with it. After all, either you’ll know why you hated it in the first place, or best of all, you have something to put into your toolbox. If you put into that really dark corner, or right into the bottom drawer of your favourite tools doesn’t matter. The important point is that you have the possibility to classify the tool as what it is: Another tool to do things in a more or less appropriate way for the task at hand. At least the monkey doesn’t (or didn’t) have that luxury. Although evolution should kick in someday, they’re still doing the same mistake over and over again. Something to think about.

Technorati Tags:
, , ,

Deleting a large number of objects with Hibernate (or not)

Monday, January 9th, 2006

A component of our product uses an algorithm whose calculation can result in several ten thousand objects at the worst, if not more. Creating these wasn’t the problem once it was done in small pieces. For those of you who didn’t already know: it’s not a very good idea to handle a large number of objects at a single point in time with Hibernate. The more objects you pulled out of the database with a single session, the longer it will take for them being dirty-checked. With several thousand objects you can just sit and wait, or get a cuppa coffee, and wait for the transaction timing out. Why’s that you ask?

We had a main class that was associated with several other classes. There were three associations (one with a potentially big association by itself) that could end up sticking several thousand objects together. Now, if you remove the first-class object, Hibernate will fetch the associated objects (if you configured the association to cascade on deletion, that is), thus putting them into the session cache, and start cascading them one by one. It gets worse, if you have event listeners installed. So even, if handling a single object costs about 10 milliseconds, this adds up to minutes for several thousand objects. Consider 10000 objects: 10000 * 10 milliseconds = 100000 milliseconds = 100 seconds. You do the math for more than 10000. Now that is way more time than you’d like to spend your application with deleting a single object and its children.

So what did we do? We turned to good old JDBC. We loosened the coupling between the objects concerning Hibernate and did the deletion handling by ourselves. The deletion listener for the parent object just removed all its children through simple SQL statements. That’s it. It worked, we padded ourselves on the back and went to lunch.

But what do you know, our QA ran some tests and it still took an awful lot of time to remove all those rows in the database. It was Oracle, we thought, it’s gotta do this stuff fast, doesn’t it? So it was time to roll up the sleeves and get that explain plan out. Didn’t look that weird, but then it hit me. The database made a full table scan on each child’s table. Whoopsie, that shouldn’t happen. Even though there was only one parent row in the database, and all the data in the children’s tables (around 20000 rows in each one) belonged to that one entry, it took minutes for Oracle to clean up. We forgot the simplest thing: to create an index on the foreign key column. Doh! One more time, repeat after me: “select” isn’t broken!

Hibernate 3.1 introduced a new concept called `StatelessSession`. Maybe there’s something that can be done in a similar way with it. Although it doesn’t seem to support cascading which was the primary problem we had. But otherwise it sounds quite interesting as a tool for batch processing.

So the lessons learned here deal primarily with handling a large number of objects with Hibernate. We had two solutions to this problem. The first one, during the calculation, was to split up the larger transactions into smaller bundles and calculate asynchronously in small steps. This had the nice side effect that the calculation can be clustered in the future via JMS. So try to use as small transactions as possible when dealing with huge amounts of data.

The cascaded delete was just too big to handle for Hibernate, so we used direct SQL via JDBC to handle it. That’s only one side of the story. If you don’t index your foreign key columns, you won’t have a benefit from this. So performance measures should not only include the application, but also the database if worse comes to worst.

__Update__: The Hibernate documentation actually has been a helpful fella and pointed me to some more information after having a deeper look. It seems, according to the section about performance, that it’s possible to cause a single delete by just dereferencing the collection. It’s not really applicable in the case where you have large associations again referencing large associations like the one described (you’d at least have to fetch the objects of the first level to clear their collections which results again in a huge session cache and aforementioned problems), but it’s a solution if you only have associations at a depth no deeper than one level. Thanks to Chris Conrad for pointing that out.

__Update 2__: As Gavin pointed out, there is (yet) another way. If you trust your database to do the cascading for you. The key (indeed) to this is as simple as it gets. Whenever you define an association, you simply add an attribute `on-delete`:

`


So if your database supports it and you trust it, your obviously better off letting it do the dirty work. Thanks Gavin.

Hibernate you versatile beast!

__Update 3__: Smile, you’re on JavaLobby!

Technorati Tags:
, , , , ,

Stuff to do in 2006

Sunday, January 8th, 2006

The new year is still young, so it’s still reasonable to find some things to be done this year, a.k.a. new years resolutions.

* Improve project management skills. Still a lot to do on this one, but I’m on my way. Improving communication, delegation, and stuff like that are top items. Stephen Seay’s list is a great basis for getting better at this stuff. Hard to come up with anything more. Well, maybe, but a year is only 365 days long.
* Take more notes of things learned. It’s easy to forget stuff, and reflecting oneself, the team, problems, successes and failures is an important part of improving.
* Get going with Ruby on Rails. I postponed reworking my personal homepage with Rails because after ten to twelve hours of work I didn’t really have the urge to think about other stuff during the little free time I had left. But Ruby is a beautiful language, and Rails is a wonderful framework, so that’s definitely something I want to do this year.
* Catch up on what I’ve missed out regarding CSS and HTML. Being a former web developer I spent two years ignoring the current trends in web development. I’m not exclusively speaking of Web 2.0 stuff here. I don’t want to deal with frames or deeply nested tables anymore, so it’s time to pick up where I left off. May the Zen of CSS be with me.
* Delve into the details of the JVM. Java’s popularity might be declining, but the concept of its JVM sure isn’t. And I’ll be using Java for a considerable amount of time in the future anyway. This one is inspired by Chad Fowler’s “My Job Went to India”.
* Spread out the vacation across the year. Up until now I only took larger vacations to travel to Australia or New Zealand. As a developer that wasn’t really a problem for me. The stress level was alright, and four to six weeks were enough to refresh and come back with an urge to get stuff done. But over the last months as a project manager I realized that it’s important to take time off more often, even if only for a weekend in a different city, the countryside, or whatever.
* Spend more time with friends. Well, this should be a no-brainer, but it’s frighteningly easy to loose track of friends, if you don’t spend time with them. Career is not everything. But nonetheless, a nice and long trip to Australia is up first.
* Read! It’s time to read some books dealing with projects in general again. “The Art of Project Management”, “Mythical Man-Month” and “Death March” are on top of the list. New books will include “Applied Software Project Management”, and something dealing with business.

Doesn’t seem like too much, but we’ll see how it goes.

Technorati Tags:
,

TextMate 1.5 is out

Friday, January 6th, 2006

TextMate 1.5 (of Ruby on Rails fame) is finally here. It took quite some time for this release, but looking at the new features it was all worth it. And if you’ve been one of those cutting edge dudes, you’ll have probably enjoyed the newest features through the application feed.

It now comes along with a complete manual and finally a working symbol popup. A lot happened under the hood, and it’s well worth checking it out.

If you don’t know TextMate yet, be sure to check it out. It’s *the* text editor that was missing from Mac OS X, and it is my editor of choice for some months now. I wrote about it a while ago, and it has only gotten better since then. You can see it in action with the screencasts available on the website.

(Via MacroMates Weblog)

Technorati Tags:
, ,

Project Management Advice

Thursday, January 5th, 2006

Stephen Seay (of ProjectSteps) published four posts on his blog recently with some very good advice concerning project management.

The first one deals with the paradoxes of project management. They’re actually from a book named “Liberation Management”, but it’s a good list of paradoxes a project manager has to handle.

16 Steps to Project Management Maturity offers a neat but nonetheless valuable list of steps you can take to mature your (or someone else’s) project management skills.

As kind of an add-on to that he posted a list of strategies for project management. A list of actions you can and should take during every-day project manager life.

He finishes with his goals for the year 2006. A good conclusion of the above posts and completely in line with my impressions of “Behind Closed Doors”.

Great stuff for the new year! Thanks for sharing, Stephen.

Technorati Tags: