Pentaho’s Matt Caster has just published a benchmarking exercise comparing Kettle and Talend. In it he admits he’s not a Talend expert and he advises that people should perform their own benchmarks where possible as requirements differ. Nevertheless, unlike most other benchmarks we’ve seen on the subject he publishes not just the results but the actual transformation “code” used in the tests.
For many people these benchmarks are of no real interest as long as the product does what is required within the time and resources available they’re content. But it would be a mistake to think that benchmarks don’t matter, they do; people have and will make that final decision based on them. Remember ETL is not life and death, the decision which tool (if any) to go with may not get the level of investigation that the developers behind such products expect of their potential clientele and this is particularly true of open source. Busy people will use such reports to direct them down a path or to confirm their existing prejudices. So I’m really glad to see Matt responding and in particular, responding in the manner he has.
Databases vendors have for years played the benchmarking game, setting and breaking records either via real technological advances or simply gaming the process. We as purchasers and users knew in many cases to take the results with a large dose of salt, but purchasing decisions where nevertheless made on the backs of these surveys.
Why not join me on Twitter at gobansaor?