Jack of all trades

One of the first things I noticed when applying for a job Down Under was that where in Europe they recruit specialists, down here, they primarily recruit the ‘jacks of all trade’

Probably because there is a huge lack of really good software engineers (they are here, but there’s not enough of them). Where in Europe, the roles ‘analyst’, ‘developer’ and ‘tester’ are strictly different roles.

Think of it like ¬†the ‘Trias Politica’, where you differ between the power that defines law (politicians), the power that upholds law (police) and the power that checks if the other two are operating within the law (judges).

The same actually does go for software development projects. You’ve got (business) analyst (they who describe what is to be made), the developers (they who make what is described) and testers (they who check if what has been described is actually made).

As a developer in Europe, I’ve fought many battles with the analysts. Mostly about the ‘why’. I’ve fought many battles with the testers, mostly about the ‘how’. Although, we all agreed on the ‘what’, in my¬†opinion, it’s great to work with a team that¬†separates¬†these powers.

Down here the rules seem to be completely different, if you are a ‘software specialist’, you are analyst, developer and tester in one. I can see the market value of a jack of all trades, but let me give you some examples why this isn’t working.

Analyst can go on forever. It’s in their job description to never stop wondering about the why, analyse it, break it down into measurable entities. These generally are people with a huge fear of failure. If they didn’t capture it, millions of lives will be dependant. That’s why analysts want to capture each and every aspect of anything, all the time.

Developers are inherently lazy, well, not all of them, but the better ones are. Developers love to cut corners, to do in one line what others do in 20, their laziness defines their efficiency. You do things once, maybe twice, but if you have to do them a third time? You write a program to do it for you. You never know when you have to do it a fourth time.

Testers are nitpicking¬†bastards, developers are confronted with each and every corner they cut, but the testers caught them. It’s an ever ongoing war between the ‘can do’ mind of the developer and the ‘but you missed this’ mindset of the tester. Actually, there are no more roles so cut out for each other than testers and developers. The tester has to sign off what the developer made. Developers hate testers, but good developers hate not having testers even more. They are their¬†conscience. I personally love testers who are capable of detecting my latest¬†mischievous¬†hack. And in the end, when developers age, with all their experience, they probably become the testers. But right now, they are probably too creative to even think about that fate.

As a developer, I hate analysts because they specify things I can’t build, I hate testers because they interpret the analysis in a different way. I love the analysts because they translate what the business needs, and I love the testers because they provide the security that I actually wrote what the users wants. And if it is a good team, we understand that we need each other. They probably hate me for seeing things too binary (‘it either is or isn’t damnit! there is no magic in computers!’)

Hence the Trias Politica, each working in a team, to obtain the same goals, but looking at the project with different angles. Where things start going wrong is where management thinks either of these roles can be performed by the other. Developers doing analysis or testing, or testers doing analysis. And beware of the day analysts start development.

Sadly enough, down here, it seems more common than I’ve been used to. There seems to be a common feeling that ‘everything with a computer’ can be done by ‘someone who does stuff with computers’. It’s like your parents asking you where the button for ‘bold’ is, because ‘you design chips, so you know everything about computers’

How do you tell them that knowing everything about bits and accumulators and registers doesn’t make you the same guy that knows how to underline or how to use¬†italics in their word processor? Some of us may know how to do 2048 bits encryption, but we just may lack the knowledge of how to add 1 and 1 in an excel sheet.

There simply is no Jack of all Trades, and there never will be. In a computer science world, that has been a station passed. I can only hope that one day I will convince the people down here that you really need the 3 of us and that there simply isn’t one that is all 3.

Writing your own maven plugin

As some of you may know, I’m currently working on the WCMS project at the Australian Broadcasting Corporation. This is a large, multi-year project to replace the old web content management system with a brand new (Java) CoreMedia system which will allow the ABC to grow it’s online presence even further.

As with most java based systems, we use maven to support building the project. And we use many of the plugins that are out there to do specific tasks, for instance the maven-config-processor-plugin to create the various configurations for each of the target platforms, but also the maven-shade-plugin to change some .class files in specific jars.

One of these days I needed a plugin which was able to combine a few javascript files into one big one. So I googled for a few minutes to see if there was any plugin that could do this trick. Sadly enough, I was out of luck. Every promising link resulted in using the maven-ant-plugin and writing an ant task. I’m not a fan of escaping maven and falling back on ant, so I decided to look into writing my own plugin, I mean, how hard could joining files be?

To create a maven plugin, you will need the maven-plugin-api, which you configure in the dependencies part of the pom.xml:

  1. <project xmlns="http://maven.apache.org/POM/4.0.0"
  2.   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  3.   xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  4.   <modelVersion>4.0.0</modelVersion>
  5.   <groupId>net.sirious.maven.examples.filejoiner</groupId>
  6.   <artifactId>maven-filejoiner-plugin</artifactId>
  7.   <packaging>maven-plugin</packaging>
  8.   <version>1.0</version>
  9.   <name>maven-filejoiner-plugin</name>
  10.   <url>http://sirious.net</url>
  12.   <dependencies>
  13.     <dependency>
  14.       <groupId>org.apache.maven</groupId>
  15.       <artifactId>maven-plugin-api</artifactId>
  16.       <version>2.0</version>
  17.     </dependency>
  18.   </dependencies>
  19.   <build>
  20.     <plugins>
  21.       <plugin>
  22.        <artifactId>maven-plugin-plugin</artifactId>
  23.        <version>2.3</version>
  24.        <configuration>
  25.          <goalPrefix>filejoiner</goalPrefix>
  26.        </configuration>
  27.      </plugin>
  28.      <plugin>
  29.        <groupId>org.apache.maven.plugins</groupId>
  30.        <artifactId>maven-compiler-plugin</artifactId>
  31.        <version>2.3.1</version>
  32.          <configuration>
  33.           <source>1.5</source>
  34.           <target>1.5</target>
  35.         </configuration>
  36.       </plugin>
  37.     </plugins>
  38.   </build>
  39. </project>

Note that instead of packaging this as a jar, it’s packaged as a maven-plugin. Also, because we’ll be using annotations in the actual java source, we’ll set the source and target to use java 5.

Next, we’ll have to write some code that actually knows how to join an array of Files to another File. Thankfully, the AbstractMojo already has a everything on board to create a plugin. Just create a class that extends the AbstractMojo and implement the execute() method. Note that the class is annotated in the javadoc using the @goal and @phase annotations. These define the goal (in this case, it will be filejoiner:join) and the phase for the plugin (I chose prepare-package as I needed to join the files only when we’re constructing the final jar)

As we’re joining a list of files to a single file, we need a targetFile and an array of sourceFiles. We mark these in the javadoc as @parameter and make sure they are required by marking them @required. When using our plugin in another maven project, it will now know the names of the parameters as well as inform you that you’ve forgotten to provide them. The rest of the source is pretty¬†straight forwarded, we open a file for writing/appending and we’ll read the array of source files and append them to the target file, we throw in some getters and setters and we’re done. I could have made the encoding configurable too, but I didn’t because I always use UTF-8 and I strongly think everybody should ūüėČ

  1. package net.sirious.maven.examples.filejoiner;
  2. import org.apache.maven.plugin.AbstractMojo;
  3. import org.apache.maven.plugin.MojoExecutionException;
  4. import java.io.*;
  5. import java.util.List;
  6. import java.util.Scanner;
  7. /**
  8.  * @author Taco Kemna
  9.  *
  10.  * @goal join
  11.  * @phase prepare-package
  12.  */
  13. public class FileJoinerMojo extends AbstractMojo {
  14.   private static final String DELIMITER = System.getProperty("line.separator");
  15.   private static final String CHARSET = "UTF-8";
  16.  /**
  17.   * Location of the target file
  18.   * @parameter
  19.   * ¬† ¬†property="targetFile"
  20.   * ¬† ¬†alias="targetFile"
  21.   *
  22.   * @required
  23.   */
  24.   private File targetFile;
  25.  /**
  26.   * source files
  27.   * @parameter
  28.   * ¬† ¬†property="sourceFiles"
  29.   * ¬† ¬†alias="sourceFiles"
  30.   *
  31.   * @required
  32.   */
  33.   private File[] sourceFiles;
  34.   public void execute() throws MojoExecutionException {
  35.     File target = targetFile;
  36.     StringBuilder output = new StringBuilder();
  37.     if (target.exists()) {
  38.       output.append(readFile(target));
  39.       getLog().info("loaded targetFile " + target.getAbsolutePath());
  40.     } else {
  41.       // make sure the folder we'll be writing to exists...
  42.       if (!target.getParentFile().exists()) {
  43.         boolean created = target.getParentFile().mkdirs();
  44.         if (created) {
  45.           getLog().info("created new folders " + target.getParentFile().getAbsolutePath());
  46.         } else {
  47.           getLog().error("unable to create target folders, exitting.");
  48.           return;
  49.         }
  50.       }
  51.     }
  52.     // append all the sourcefiles
  53.     for (File source : sourceFiles) {
  54.       if (source.exists()) {
  55.         getLog().info("appending source " + source.getAbsolutePath());
  56.         output.append("\n").append(readFile(source));
  57.       } else {
  58.         getLog().error("file did not exist: " + source.getAbsolutePath());
  59.       }
  60.     }
  61.     // and write the output
  62.     write(target, output);
  63.   }
  64.  /**
  65.   * Writes the String to a file
  66.   * @param target File target file to write to
  67.   * @param output StringBuilder containing the data to be written
  68.   */
  69.   private void write(File target, StringBuilder output) {
  70.     // write the entire buffer
  71.     Writer out = null;
  72.     try {
  73.       out = new OutputStreamWriter(new FileOutputStream(target), CHARSET);
  74.       out.write(output.toString());
  75.       out.close();
  76.     } catch (UnsupportedEncodingException e) {
  77.       getLog().error("unsupported encoding: " + CHARSET, e);
  78.     } catch (FileNotFoundException e) {
  79.       getLog().error("file not found: " + targetFile, e);
  80.     } catch (IOException e) {
  81.       getLog().error("i/o exception", e);
  82.     }
  83.   }
  84.  /**
  85.   * Reads a file into a String
  86.   * @param file File
  87.   * @return String
  88.   */
  89.   private String readFile(File file) {
  90.     StringBuilder sb = new StringBuilder();
  91.     Scanner scanner = null;
  92.     try {
  93.       scanner = new Scanner(file, CHARSET);
  94.       scanner.useDelimiter(DELIMITER);
  95.     } catch (FileNotFoundException e) {
  96.       getLog().error("file not found while trying to read file " + file.getAbsolutePath(), e);
  97.       return "";
  98.     }
  99.     while (scanner.hasNextLine()) {
  100.       sb.append(scanner.nextLine()).append(DELIMITER);
  101.     }
  102.     scanner.close();
  103.     return sb.toString();
  104.   }
  105.  /**
  106.   * @return File[]
  107.   */
  108.   public File[] getSourceFiles() {
  109.     return sourceFiles;
  110.   }
  111.  /**
  112.   * @param sourceFiles File[] to set.
  113.   */
  114.   public void setSourceFiles(File[] sourceFiles) {
  115.     this.sourceFiles = sourceFiles;
  116.   }
  117.  /**
  118.   * @return File
  119.   */
  120.   public File getTargetFile() {
  121.     return targetFile;
  122.   }
  123.  /**
  124.   * @param targetFile File to set.
  125.   */
  126.   public void setTargetFile(File targetFile) {
  127.     this.targetFile = targetFile;
  128.   }
  129. }

And that is all that there is to it. How maven knows to use this class? No magic there, just because you annotated it with @goal, it will be picked up by maven. So you can put multiple goals in one plugin, nicely¬†separated¬†into different classes. Of course, to deploy this plugin to your repository, you will need to add a ‘distributionManagement’ section to your pom.xml, but if you just have a local repository, running ‘mvn install’ will do.

Now, to use our plugin in another project, we’ll need to configure it as a plugin. We already decided that the execution phase for this plugin was to be ‘prepare-package’, so, we configure the ‘join’ task during that phase and we configure some sourceFiles and a targetFile:

  1. <plugin>
  2.   <groupId>net.sirious.maven.examples.filejoiner</groupId>
  3.   <artifactId>maven-filejoiner-plugin</artifactId>
  4.   <version>1.0</version>
  5.   <executions>
  6.     <execution>
  7.       <phase>prepare-package</phase>
  8.       <goals>
  9.         <goal>join</goal>
  10.       </goals>
  11.     </execution>
  12.   </executions>
  13.   <configuration>
  14.     <targetFile>${basedir}/target/combined.txt</targetFile>
  15.     <sourceFiles>
  16.       <sourceFile>${basedir}/src/main/resources/1.txt</sourceFile>
  17.       <sourceFile>${basedir}/src/main/resources/2.txt</sourceFile>
  18.     </sourceFiles>
  19.   </configuration>
  20. </plugin>

So, now, whenever we run the ‘prepare-package’ or during any later phase, our plugin will take 1.txt and 2.txt and append them to a new or existing combined.txt. Easy does it. I’m pretty sure that writing the plugin, distributing and configuring it in our project actually took less time than this blog post ūüôā

More reading:

The Meta-Question

I think I’ve been using Google as a search engine from the day Google emerged as a search engine. Which, of course, was there sole business when they did. It was the fastest, most accurate and the least gruesome of all search engines of it’s time. Just this textbox and two buttons (at least, I can’t remember the day the ‘I feel lucky’ button was introduced, to me, it seems like it’s always been there)

So there it was, this mostly white page, with nothing more than that colorful logo just shouting at you ‘let me find what you want!’. Back in those days, I didn’t even think of the Meta-Question. I just typed and found. Even back then, I wondered what good a Search Engine would be if it didn’t Find anything. To me, it was the ultimate Find Engine.

Many years have passed and still today, it’s my Find Engine of choice. If I can’t find in on Google, I’m sure as hell not going to Find it on Yahoo, Bing, Just Jeeves or Ask.com. If Google can’t find it, it simply becomes a question to which there is no answer. I know it told me the meaning of life.

Without Google, my IQ drops at least 20 points. I’ve always said that knowing where to find the answer is as valuable as actually knowing the answer. It no longer falls back on the answer, but how you state the Question. Just asking ‘I need pussy’, doesn’t really get you laid. It however, might get you a kitten though.

But I’ve never asked the engine the Meta-Question: ‘Where can I find it’. Today I asked the mother of all search engines that question. The number one pick as about a¬†celebrity¬†style website. Nah, that wasn’t what I was looking for. So I asked Yahoo. Guess you can find it at Totally Toddler. So I asked Bing. I seem to find it at Find. Except from advertising, I can’t enter what I’d like to find at all.

Concluding, none of the mayor ‘search engines’ actually find themselves. A simple question remains unanswered. Where can I find it leads to dead links and outskirts of the internet. There is simply no answer to that simple Meta-Question.

And if you ask ‘What can I find’, the biggest hit is the wayback machine. So, if you ask the internet what it is you can find, it will point you to the biggest archive of all stuff ever found but never searched for. The question where you can actually find stuff remains unanswered.

I guess that’s why they still call them ‘Search Engines’. It’s about time someone invented a ‘Find Engine’. I know I’m done Searching.


I’ve been thinking about this a long time. Years ago, I started a blog to document and sometimes contemplate my and my families move to Australia. You can find it at http://www.kemna.eu. In the beginning I’d just put up some new post about where in the process we were, how the kids were doing etc. It was a family blog.

However, more and more colleagues asked me to write about my work life as well. I resisted every attempt by pointing out that most of the people blogging do not really care about what they are saying, they care about how many people actually take notice of their exhibitionism.

I can remember, years ago, years before wordpress, blogger, facebook and others made it easy to blog, there was a guy who wrote a piece called ‘why I hate weblogs’. The original post is¬†long gone but¬†I tried to find it today and I‚Äôm pretty sure I found the correct content, but it’s just put into… a blog…

Actually, people have copied it to their blogs, threw it on facebook, it’s mentioned in a weblog about blogging, oh wait, this post is becoming just that. Stop!

Anyway, thankfully the internet has more memory than I have and the original can still be found at the Wayback Machine, the original post at October 13, 2002 up until it’s deletion in June 2008, and even the funny test can be taken, but the outcome will forever be gone…

I guess I’ll never know, and who’s going to read it anyway ūüôā


About Sirious’s Business


It seems you’ve reached yet another outskirt of the internet where someone felt the need to put content.

If you are looking for my families blog about our move Down Under, please proceed to www.kemna.eu.

Here I’ll just try to keep up with life, work and everything else that doesn’t really fit the family category, the little annoyances and joyful moments life brings, as well as the things I work with, work at, or maybe even things that work against me.

After all, what I do is just Sirious’s Business ūüôā

Digital Inhabitant

I¬†acknowledge¬†the fact that I’ve been born a tv-native (I’ve never wondered about a tv-less world) but my children are what they call ‘digital natives’. Think of it, the communicator we’d love to have from Star-Trek, is what they call an iPhone, blackberry or if you are really hip, an android phone.

We got the technology Star Trek dreamt about delivered at our doorstep. To my son, there is no difference between the tv-remote and my iPhone, just because I was that stupid to install some application that allowed me to control my tv through my iPhone. But when he grows up, there really isn’t a difference.

I’m called Gen-X, the generation next to the BabyBoomers, just before GenY, the digitally adaptees, and so far away from GenZ, the digitally natives. My children will grow up in a world where tv-screens are a thing of the past, where social interaction, the day after a programme, will be cut short in YouTube fragments.

What both the babyboomers and GenY seem to misunderstand, is that GenX made it happen. Where would Facebook be, without cheap data plans on your mobile? Without internet reaching every single home? We grew up with the fear that one day, we’d be out of ip4 internet addresses, but we’re also the generation that thought of IPv6. We’ve put our parents on the internet and we’re giving that same internet to our children.

GenX may be silent and silly to most of you, we made the world you’re experiencing right now. We did see the potential, we just didn’t want to make money of something that should have been available to everyone when we were young. So, not only did we get our parent online, we shared the online world with our children.

Gen Y blames us for not having it ready way before they came, but I happen to think that if Gen X had the digital platform we now have, we probably wouldn’t be bothered with producing Gen Y at all. I know I’m still waiting for a massive online Grand Theft Auto ūüėČ Oh shoot, I just gave away yet another idea some GenY’er will make a fortune…

But really, the best digital inhabitant today probably is the 30-40 year old. These are the people that had to live without and learned to live with all the opportunities offered. We’re probably the last generation that actually had to go somewhere in the flesh to meet a partner. We know how to operate those communicators, as we’ve invented them… But we also know what it means to interact with real people in real life.

hell, we’re probably the last generation that knows the meaning of the acronym IRL ūüėČ