Your arguments are invalid.

Because it’s enough to have just one. See, thanks to scope and something called currying, any multi-argument function can be rewritten as a unary chained function. A simple example is this Javascript function to add two arguments:

1
2
3
function add(x, y) {
   return x + y;
}

it’s easy to see what that function would do, you pass in two arguments and they get added:

1
var sum = add(2, 3); // 5

now, in a functional world, you want to have unary functions that take only one argument. How can you achieve a simple add function if you’re not allowed to take more than one argument? This can be done by currying (and it requires scope)

1
2
3
4
5
function add(x) {
   return function (y) {
      return x + y;
   }
}

now the call to this method would be different because it just takes one argument. Thanks to the scope, the inner function can see the x, so you call this method like so:

1
var sum = add(2)(3); // 5

And that’s how you can decompose a multi-argument function into many, nested unary functions.

I needed a TriFunction

Since Java 8 you can do such nice functional things, I love it. But, for some reason it has Functions and BiFunctions, but no TriFunctions! So, it was time to add the TriFunction interface. And yes, I’m very immature ūüėõ

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
import java.util.Objects;
import java.util.function.Function;
 
@FunctionalInterface
public interface TriFunction<S, U, C, K> {
 
  /**
   * Applies this function to the given arguments
   * @param s the first argument
   * @param u the second arguments
   * @param c the third argument
   * @return K
   */
   K apply(S s, U u, C c);
 
  /**
   * Returns a composed function that first applies this function to its input
   * and then applies the {@code after} function to the result.
   * 
   * If evaluation of either function throws an exception, it is relayed to
   * the caller of the composed function.
   * @param <T>
   * @param after
   * @return 
   */
   default <T> TriFunction<S, U, C, T> andThen(Function<? super K, ? extends T> after) {
       Objects.requireNonNull(after);
       return (S s, U u, C c) -> after.apply(apply(s, u, c));
   }
}

 

Now, you can build functions that take 3 arguments and do something with them to return one result, like:

1
2
3
4
5
 TriFunction<Boolean, Boolean, Boolean, Boolean> sucks = (s, u, c) 
     -> Stream.of(s, u, c)
          .allMatch(t -> Boolean.TRUE.equals(t));
 
 boolean f = sucks.apply(true, false, true); // f is false

Guess it doesn’t suck after all!

You are more likely to be killed by an asteroid than by a terrorist.

Though what happened in Berlin today is still ‘most likely’ an attack (by a lone wolf, not by an army of asylum seekers) we can’t generalise and blame this on asylum seekers in general. That’s just what our nationalistic leaders want, what the news papers want because it sells more papers.
 
By these rules of generalisation, all men are rapists, all fathers are incestuous, all mothers are whores, all white people trade slaves and basically all human beings are monsters. You cannot blame anything like this on entire groups of people. The fact is that only one person was driving that truck. Not a nation, not a group, not anyone else but that guy.
 
The chances that you are going to be killed in the next ‘terror attack’ are much smaller than the chances that you are getting killed by a drunk driver, by crossing a road, by actually driving a car and for some reason, in the USA, by having your toddler grab your gun from your purse.
 
The media loves to make these things bigger than they are, because this kind of fear makes their sales go up. The chances of you dying in a terrorist attack are pretty slim if you are living in a western society like most of Europe, the USA and here in Australia. It’s practically non-existent.
 
You are about 10 times more likely to die from SUICIDE than from murder. You are about 5 times more likely to die from DIABETES than from a traffic accident. You stand a better chance at winning the lottery than getting killed in a terror attack.
 
Let’s look at a quote from one of the sources below:
 
“The chances of being killed in a terrorist attack are about 1 in 20 million. A person is as likely to be killed by his or her own furniture, and more likely to die in a car accident, drown in a bathtub, or in a building fire than from a terrorist attack.”
 
ok, 1:20,000,000, should be able to handle that. Let’s compare it to something else:
 
“The chances a person will be killed by an asteroid are 1 in 200,000, which is much higher than the odds of being killed by hail, which is 1 in 734,400,000.”
 
Let that sink in people. You run a 100x (a hundred times) bigger chance to be hit by an asteroid than be killed in a terrorist attack!!! Why the fuck aren’t we walking around with titanium umbrellas!?
 
for crying out loud, let’s stop giving the media what they want: attention. Let’s stop giving away our privacy over this bloated sense of insecurity. And the great thing with ‘terror’ is, if you show a complete disregard of the effects, it’s not terrifying at all. They’re just what they really are: sorry excuses for living beings.
  
just some of the sources below.
 
 
 

Fixing ORD fields in Oracle

If you use Hibernate and ordered list, you’re probably familiar with having to add a column to retain the order. If you don’t want nulls in your list, you sometimes need to reorder this (say, for instance, if you manually remove an item from a list via the database).

Of course, you can do an update to “set ord = ord – 1 where ord = <first gap>” and continue doing so until you have the items nicely sequenced again. But if you have Oracle*, you can use the row_number() feature

Say you have a table ‘someTable’ and you removed a few rows from the list that is grouped by a foreign key (some_fk_id) for 123456 and you want to renumber the ORD field based on the order of inserted ID’s, you can do the following:

1
2
3
4
5
MERGE INTO someTable T USING (
       SELECT id, (ROW_NUMBER() OVER (ORDER BY id ASC) - 1) rn 
       FROM someTable WHERE some_fk_id = 123456
     ) R ON (T.id = R.id) 
     WHEN MATCHED THEN UPDATE SET T.ord = R.rn;

This will then reorder the list, starting with 0 for that group. I’m pretty sure you can write your cursors to renumber entire groups of lists

* I believe other databases have this feature as well, such as Microsoft SQL Server, however, I don’t use those, so don’t blame me when it doesn’t work there ūüėČ

 

 

Doing Enums with Jackson

Recently I’ve been working on a Jersey 2 application (a JAX-RS REST) using JSON. For the JSON serialization and deserialization I’ve chosen to use the Jackson framework. One of the reasons for this is that prior we’ve been using Freemarker templates, but if you use those, you can easily create invalid JSON which will eventually screw up your clients. Using Jackson & Jersey allow me to skip the entire Object graph to template to output chain and simply use POJOs and use annotations to model the JSON request and responses.

So basically, if I have a request/response that has a “number”, I can use those annotations to expose getter and setter methods, or choose to ignore Jackson adding a default. So in the next fragment, the getter getSomeArbitraryNumber() will be written to “number”, any incoming “number” will be mapped to setSomeArbitraryNumber and when serializing, Jackson will ignore the isNumberSet() method (without the JsonIgnore() it will expose “numberSet”):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
    ...
    @JsonProperty("number")
    public Long getSomeArbitraryNumber() {
        return number;
    }
 
    @JsonProperty("number")
    public Long setSomeArbitraryNumber(Long number) {
        this.number = number;
        this.numberSet = true;
    }
 
    @JsonIgnore()
    public boolean isNumberSet() {
       return numberSet;
    }
    ...

Anyway, long story short, I ran into issues with Enums, simply because, well, you can’t instantiate a new Enum. Exposing them as JSON objects wasn’t that hard, you can just add the @JsonFormat(shape=JsonFormat.Shape.OBJECT) annotation to the class and Jackson will serialize the Enum as if it was a simple POJO:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
package net.sirious.jackson.enum
 
import com.fasterxml.jackson.annotation.JsonFormat;
import com.fasterxml.jackson.annotation.JsonProperty;
 
@JsonFormat(shape=JsonFormat.Shape.OBJECT)
public enum MyEnumType {
 
    TYPE_ONE("one", "this is basically just type one"),
    TYPE_TWO("two", "and this is type two");
 
    private final String id;
    private final String description;
 
    private MyEnumType(final String id, final String description) {
        this.id = id;
        this.description = description;
    }
 
    @JsonProperty("id")
    public String getId() {
        return id;
    }
 
    @JsonProperty("description")
    public String getDescription() {
        return description;
    }
 
    /**
     * Gets a MyEnumType from id or <tt>null</tt> if the requested type doesn't exist.
     * @param id String
     * @return MyEnumType
     */
    public static MyEnumType fromId(final String id) {
        if (id != null) {
            for (MyEnumType type : MyEnumType.values()) {
                if (id.equalsIgnoreCase(type.id)) {
                    return type;
                }
            }
        }
        return null;
    }
 
}

So if I had a pojo with a getter like this:

1
2
3
4
5
6
7
    ...
    @JsonProperty("myenumtype")
    public MyEnumType getMyEnumType() {
        return myEnumType;
    }
 
}

then Jackson will serialize MyEnumType.TYPE_ONE as follows:

1
2
3
4
5
6
   ...
   "myenumtype": {
      "id":"one", 
      "description":"this is basically just type one"
   }
   ...

However, deserializing is an issue, as normally Jackson expects a no-arg constructor, but you can’t have that with Enums. So you need something that can do this for you, which is a JsonDeserializer:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
package package net.sirious.jackson.enum.deserializer;
 
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.JsonMappingException;
import net.sirious.jackson.enum.MyEnumType;
import java.io.IOException;
 
public class RecurrenceEndTypeDeserializer extends JsonDeserializer {
 
    @Override
    public MyEnumType deserialize(JsonParser jp, DeserializationContext dc) throws IOException, JsonProcessingException {
 
        MyEnumType type = MyEnumType.fromId(jp.getValueAsString());
        if (type != null) {
            return type;
        }
        throw new JsonMappingException("invalid value for type, must be 'one' or 'two'");
    }
 
}

Above deserializer now allows you to accept Strings, either “one” or “two” in the JSON:

1
2
3
4
5
{
   ...
   "myenumtype":"one" 
   ...
}

And then you can use this on your POJO as follows:

1
2
3
4
5
6
7
8
    ...
    @JsonProperty("myenumtype")
    @JsonDeserialize(using = MyEnumTypeDeserializer.class)
    public void setMyEnumType(MyEnumType value) {
        this.myEnumType = value;
    }
 
}

And that’s it! Jackson will automatically get the String “one”, use the deserializer to get a MyEnumType and then pass that on your POJO. You can use this method to customize any serialization/deserialization with Jackson.

Security and safety

Recently people who use Mac computers have asked me about security. Good question. I’d say they are pretty secure. They didn’t ask me about safety. That is a different question. I will try to explain why:

The only safe operating system is the one that is turned off, disconnected and preferably locked in a safe. And even then, it only takes one human to breach that security. As long as it is not turned on, or plugged into a network, it is still secure, but not safe.

Now, if you understand those rules, you can maneuver pretty safe and secure on the internet.

To do so, there are some simple rules you can follow:

Choose an operating system that is released as open source (why? because everything is transparent, every single bit of your operating system is out there, people know about it, good hackers and bad hackers the like, history has proven this, you are more vulnerable with a Microsoft (closed source) OS than with any obscure linux variant out there). Apple is turning towards the Microsoft side, but is at least based on a BSD unix variant, so, as long as you don’t give out your password to install the next .dmg you download, you are pretty safe (even if history has proven that Mac users are the least computer savvy users out there, you just got lucky that it’s in essence a unix machine)

Choose an operating system that is maintained (what does that mean? simply, the more people that are working on it, the more people who know a lot more than you do about operating systems are working on it) If more people are working on it, it does not mean it is not ready yet, it actually most often means that those good hackers are quite fond of it (pick any popular linux or freebsd version, it’s free and people are actively working on it to make it the most secure system they can think of)

Third, be a user when you can, only change into an administrator if you need to be. The way the unix (and alike) operating systems were designed was about giving users power to do things, but not to disrupt other users. I personally change into my root clothes when I think it’s time to delete a user account that has been compromised, or at least leans towards it or when I’m simply done with it. And since I have the power to become root, I can always create a new user.

Finally, and absolutely not the least, don’t be lazy, don’t reuse that password on website nr1 you used for website nr2, unless you don’t care about either of these (this is a rule I break often, I start out with using a weak password until I think I really want to keep that account).

Don’t use that service name in your password. Cryptology is only as smart as you are. You can have all your keys encoded in (who cares) 8 gigabyte keys, but if you use “-facebook” for facebook and “-twitter” for twitter, the other party already has 50% of your password decoded.

No matter how many bits you use to encrypt it, if they know half the message, it’s pretty useless because knowing what you are looking for makes it easy. (Really don’t want to go into details here, but you can understand that looking for “Hi Mom!” in a decrypt is easier if you are sure it contains “Hi Mom!” then if you don’t know that it should contain that)

A password like ‘IchHateMeinFuckingFriendDieZesBlocksDownWoont’ (normal words, but 3 different languages) is far more effective than the not so random ‘qwerty123’, especially if you, as a hacker, already know that the password is going to end in the plaintext ‘-facebook’. (and it is easy to remember). As with anything, but especially in cryptoanalysis, if you already know what you are looking for, it is easier to find it.

Coming back to the 3 language 10 words password, it is 46 characters, let’s assume we only use lowercase, that would be 26^42 combinations. Being able to use uppercase makes it 52^42, include numbers, 62^42, giving a resolution of 1.9074403212938070052188251342723e+75 possible combinations.

With this I’d also advice to look at how the password is used. Yes, the iphone number password is actually quite safe, BUT ONLY if you turned on the feature to wipe the phone after 10 misses. If you don’t, it’s just 10000 tries in which case brute force works. And quickly too. Even the hardware of the iphone you are trying to crack is much faster in 0-10000 than you are. Don’t forget, the computing power in your pocket supersedes in many¬†magnitudes¬†the computing power that was used to put a man on the moon.

Don’t underestimate brute force, as a rule, it takes only 1/2 to crack it. Either they get it right at ‘A’ or at the end at ‘ZZZZZZZZZZZ’, on average it is half and brute force is actually one of the ‘cracking algorithms’ that is 100% sure to crack any code, unless it is accompanied by a total wipeout, as the iphone offers. Use it. Objects that completely lock after X tries are much safer than object that use whatever amount of gigabytes for keys. Imagine what luck you need to crack a safe if you have the previously amount of options (1.9074403212938070052188251342723e+75) with only 10 tries. Brute force is completely useless against systems that lock.

But, it’s not completely useless. If it’s a digital safe? it can be copied. You may lock after 10 tries, but if I can make¬†1.9074403212938070052188251342723e+75 copies, I only need to try 1 combination per copy. So what do you learn from this? If someone wants to crack you, and if they have enough resources, they can and will. The safest place to keep really private information is still locked in your brain. You will never be secure if your brain is not safe.

As per today, 25/04/2012, they haven’t found a way to secure anything that was saved in your brain and as long as you keep it there, it can’t be protected by security

T is for Telecom, where communication goes wrong…

It’s a wonderful day in the world of telecommunications… First, the 3G service on my phone went dead (it still showed wonderful connection, but as soon as you tried anything it said I didn’t have cellular data). Since I recently moved from Vodafone to Virgin Mobile who uses Optus, I thought, how bad can it be. But hey, after a few tests with luckily another phone at hand, it obviously showed that there is probably one flag not set correctly at their end. We can only hope that the morning crew is able to set the flag.

So that one is still on-going. The other problem I have is with Telstra (the fourth telco to mention in this post, however, the biggest here). Anyway, I updated my subscription with them, because, well, hey, after 2 years, times change, you can probably get a better deal. So I first started to use the DIY way via the website.

Since that quickly enough proved not viable, due to the exact combination of switches I wanted, I followed the advice and started a chat session. I really had some fun with a great guy on the other hand and after about 30 minutes we arrived at a new package for the bundle (phone, internet, tv). I would be called the next morning to arrange for the installation guy to do the physical part of the deal (installing the HD box and moving the non-HD box to the second room).

So I waited about a week. Obviously, nobody called. But since there is a store for anything, there are also stores for Telstra. The next week (last week) I go to the T-store and apparently, my previous order could not be found. I think, well, while I’m here, lets do it again. This time however, I also get a T-hub (Telstra’s take on a tablet, connected to your base station etc) thrown in the deal. Who am I to say no to a better deal, so I go home, thinking to have the same order, just this time with some new gadgets for the home.

After installing the T-hub, the base-station and 2 handsets, I start to play around with the new toys and decide to enable sms. Why not. However, to enable that, I first had to enable something else, which I couldn’t enable since it was already enabled (let’s call it voicemail). So, as the device instructed me, I called the helpdesk. Again, easy-peasy, no problemo and the issue was quickly resolved. Except for the fact that the activation code was expired. But, there was a link in the activation email that would allow me to generate a new activation code at any time.

Today I tried to generate that activation code (as my evening was already ruined because of the trouble with the 3G on the mobile phone), I click that link to find that the website is not exactly working as it should. Actually, somewhere halfway, the process advised me to call the corresponding helpdesk (Telstra has many).

Calling the T-Hub helpdesk is not so bad, apparently, not many people do that, so I got connected quickly enough (around 8.45pm). During the conversation, I was asked to install LogMeIn and even though I do know better in most of the cases, I was easy going and a nice customer, so I let them control my Mac for a while, until they discovered that a certain part of the website was not working. (Yes, the same link to get a new activation code)

That kind of brought the nice lady at the other end of the line to the end of her flowchart and the incident was logged and over the next few days, someone will contact me to activate the sms service on the T-Hub (I hope, but I should know better). However, to log the incident, she needed the serial number on the T-hub, which is conveniently stored inside the battery compartment. (note to reader: store this in mind, it will come back later…). So I remove the batter, read the serial number (quite long, but hey, we got through it) and that was about it.

“Anything else I can help you with”, she asks. Sure, why not, while we’re here (9.15pm), I kind of ordered something with you guys about 2 weeks ago, and again a week ago and I was to be contacted within 24 hours, but hey, they kind of didn’t. But, since she was the T-hub helpdesk, no can do. I get put back in hold, this time for the Billing helpdesk.

Billing is a bigger problem as it took a tad bit longer to finally get put through. Yay! I explain my quest (2 weeks ago via chat, last week via the store, no confirmation, no contact, no email, no nothing). She starts up the system after I’ve identified myself (again) and she can see both orders. However, she sees differences in the packages created by both colleagues (who both came to the same final price though, so how that happens, you tell me).

Since I now clearly know what I want and what it will cost me, we can go quickly through the changes and she decides that it is probably best to cancel both previous orders (sure, but, wouldn’t it be nice to just let one go through and keep me, well, informed?) and we create a new one (again, exact same price, but totally different bundle with the same components, really…). Only one thing, she can’t finalise it as that is just something the Bundle Team can do. Incident report number 2, reference number whatever, I’ll get a call on Friday.

“Anything else I can do for you?”. (9.45pm) Well, apparently, when I removed the battery from the T-hub, as your colleague asked me to do, the touch screen on the T-hub has become unresponsive (I noticed this because, well, you have to do something during the time you are put on hold). “Let me just connect you to our T-hub helpdesk”

And so, an hour later, I was back full circle with the T-hub helpdesk. Again, the personal details, my phone number, name, age, first 4 digits of my pin and the ccv code, the works, I am identified (ok, that about the pin and ccv is not true, but come on, 3 times during 1 phone call from a landline supplied by Telstra themselves? really?)

Anyway, after first chewing away the legendary question: “Have you tried turning it off and on again” we again get to the point where I have to remove the battery and read out the serial number. (again, remember this!) Reinsert the battery, try again, the works. At some point during this conversation her flowchart also ended and the T-hub is going to be replaced.

To do so, they also need the serial number that is on the base station as well as on the handset that came with it. So I start with the number on the base station and after 3 digits I get this strange feeling that I know this number. With only one exception. Where the T-Hub had a T, the base station has a convenient B. Where the base station had the B, the handset had a C.

Curious as I am, I ask the nice lady: “Wouldn’t it have been easier for me, instead of removing the battery from the T-hub, simply to lift up the base station and read that serial number?”. “Yes, they are the same, except that T stands for T-hub, B for base station and C for Communicator”…

“So, basically, strictly hypothetical, since it is too late anyway, I could have told the first person the serial number, which is printed on the outside of the base station and she could have concluded the serial number of the T-hub without me having to remove the battery and consequently ruining the device’s touchscreen?”

“Yes, when we replace them, we have to replace all 3”. Anyway, I now have a 3rd incident number and they will send me a replacement set. (it’s now 10.15pm). But, since I got here, I’m not done yet…

“How about the contact list I have stored in the device? I just had all my contacts in there?” (this is not entirely true, I just had my mobile number in there just to test stuff, we didn’t get around to actually transfer any of our other contacts in the system, but hey, at this point, you got to ask).

“I can give you the number of our technical support team who will help you with that”. I thanked the lady. I’ll go a long way playing innocent customer, but I sure won’t spend another 30 minutes on the phone with the next person to explain to me that I can use the extra handset to store the numbers and transmit them to the new base station, as I had already read the manual.

Let’s just wait and see what Friday brings. It can only get better from here I guess…

 

Jack of all trades

One of the first things I noticed when applying for a job Down Under was that where in Europe they recruit specialists, down here, they primarily recruit the ‘jacks of all trade’

Probably because there is a huge lack of really good software engineers (they are here, but there’s not enough of them). Where in Europe, the roles ‘analyst’, ‘developer’ and ‘tester’ are strictly different roles.

Think of it like ¬†the ‘Trias Politica’, where you differ between the power that defines law (politicians), the power that upholds law (police) and the power that checks if the other two are operating within the law (judges).

The same actually does go for software development projects. You’ve got (business) analyst (they who describe what is to be made), the developers (they who make what is described) and testers (they who check if what has been described is actually made).

As a developer in Europe, I’ve fought many battles with the analysts. Mostly about the ‘why’. I’ve fought many battles with the testers, mostly about the ‘how’. Although, we all agreed on the ‘what’, in my¬†opinion, it’s great to work with a team that¬†separates¬†these powers.

Down here the rules seem to be completely different, if you are a ‘software specialist’, you are analyst, developer and tester in one. I can see the market value of a jack of all trades, but let me give you some examples why this isn’t working.

Analyst can go on forever. It’s in their job description to never stop wondering about the why, analyse it, break it down into measurable entities. These generally are people with a huge fear of failure. If they didn’t capture it, millions of lives will be dependant. That’s why analysts want to capture each and every aspect of anything, all the time.

Developers are inherently lazy, well, not all of them, but the better ones are. Developers love to cut corners, to do in one line what others do in 20, their laziness defines their efficiency. You do things once, maybe twice, but if you have to do them a third time? You write a program to do it for you. You never know when you have to do it a fourth time.

Testers are nitpicking¬†bastards, developers are confronted with each and every corner they cut, but the testers caught them. It’s an ever ongoing war between the ‘can do’ mind of the developer and the ‘but you missed this’ mindset of the tester. Actually, there are no more roles so cut out for each other than testers and developers. The tester has to sign off what the developer made. Developers hate testers, but good developers hate not having testers even more. They are their¬†conscience. I personally love testers who are capable of detecting my latest¬†mischievous¬†hack. And in the end, when developers age, with all their experience, they probably become the testers. But right now, they are probably too creative to even think about that fate.

As a developer, I hate analysts because they specify things I can’t build, I hate testers because they interpret the analysis in a different way. I love the analysts because they translate what the business needs, and I love the testers because they provide the security that I actually wrote what the users wants. And if it is a good team, we understand that we need each other. They probably hate me for seeing things too binary (‘it either is or isn’t damnit! there is no magic in computers!’)

Hence the Trias Politica, each working in a team, to obtain the same goals, but looking at the project with different angles. Where things start going wrong is where management thinks either of these roles can be performed by the other. Developers doing analysis or testing, or testers doing analysis. And beware of the day analysts start development.

Sadly enough, down here, it seems more common than I’ve been used to. There seems to be a common feeling that ‘everything with a computer’ can be done by ‘someone who does stuff with computers’. It’s like your parents asking you where the button for ‘bold’ is, because ‘you design chips, so you know everything about computers’

How do you tell them that knowing everything about bits and accumulators and registers doesn’t make you the same guy that knows how to underline or how to use¬†italics in their word processor? Some of us may know how to do 2048 bits encryption, but we just may lack the knowledge of how to add 1 and 1 in an excel sheet.

There simply is no Jack of all Trades, and there never will be. In a computer science world, that has been a station passed. I can only hope that one day I will convince the people down here that you really need the 3 of us and that there simply isn’t one that is all 3.

Writing your own maven plugin

As some of you may know, I’m currently working on the WCMS project at the Australian Broadcasting Corporation. This is a large, multi-year project to replace the old web content management system with a brand new (Java) CoreMedia system which will allow the ABC to grow it’s online presence even further.

As with most java based systems, we use maven to support building the project. And we use many of the plugins that are out there to do specific tasks, for instance the maven-config-processor-plugin to create the various configurations for each of the target platforms, but also the maven-shade-plugin to change some .class files in specific jars.

One of these days I needed a plugin which was able to combine a few javascript files into one big one. So I googled for a few minutes to see if there was any plugin that could do this trick. Sadly enough, I was out of luck. Every promising link resulted in using the maven-ant-plugin and writing an ant task. I’m not a fan of escaping maven and falling back on ant, so I decided to look into writing my own plugin, I mean, how hard could joining files be?

To create a maven plugin, you will need the maven-plugin-api, which you configure in the dependencies part of the pom.xml:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>net.sirious.maven.examples.filejoiner</groupId>
  <artifactId>maven-filejoiner-plugin</artifactId>
  <packaging>maven-plugin</packaging>
  <version>1.0</version>
  <name>maven-filejoiner-plugin</name>
  <url>http://sirious.net</url>
 
  <dependencies>
    <dependency>
      <groupId>org.apache.maven</groupId>
      <artifactId>maven-plugin-api</artifactId>
      <version>2.0</version>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
       <artifactId>maven-plugin-plugin</artifactId>
       <version>2.3</version>
       <configuration>
         <goalPrefix>filejoiner</goalPrefix>
       </configuration>
     </plugin>
     <plugin>
       <groupId>org.apache.maven.plugins</groupId>
       <artifactId>maven-compiler-plugin</artifactId>
       <version>2.3.1</version>
         <configuration>
          <source>1.5</source>
          <target>1.5</target>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

Note that instead of packaging this as a jar, it’s packaged as a maven-plugin. Also, because we’ll be using annotations in the actual java source, we’ll set the source and target to use java 5.

Next, we’ll have to write some code that actually knows how to join an array of Files to another File. Thankfully, the AbstractMojo already has a everything on board to create a plugin. Just create a class that extends the AbstractMojo and implement the execute() method. Note that the class is annotated in the javadoc using the @goal and @phase annotations. These define the goal (in this case, it will be filejoiner:join) and the phase for the plugin (I chose prepare-package as I needed to join the files only when we’re constructing the final jar)

As we’re joining a list of files to a single file, we need a targetFile and an array of sourceFiles. We mark these in the javadoc as @parameter and make sure they are required by marking them @required. When using our plugin in another maven project, it will now know the names of the parameters as well as inform you that you’ve forgotten to provide them. The rest of the source is pretty¬†straight forwarded, we open a file for writing/appending and we’ll read the array of source files and append them to the target file, we throw in some getters and setters and we’re done. I could have made the encoding configurable too, but I didn’t because I always use UTF-8 and I strongly think everybody should ūüėČ

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
package net.sirious.maven.examples.filejoiner;
import org.apache.maven.plugin.AbstractMojo;
import org.apache.maven.plugin.MojoExecutionException;
import java.io.*;
import java.util.List;
import java.util.Scanner;
/**
 * @author Taco Kemna
 *
 * @goal join
 * @phase prepare-package
 */
public class FileJoinerMojo extends AbstractMojo {
  private static final String DELIMITER = System.getProperty("line.separator");
  private static final String CHARSET = "UTF-8";
 /**
  * Location of the target file
  * @parameter
  *    property="targetFile"
  *    alias="targetFile"
  *
  * @required
  */
  private File targetFile;
 /**
  * source files
  * @parameter
  *    property="sourceFiles"
  *    alias="sourceFiles"
  *
  * @required
  */
  private File[] sourceFiles;
  public void execute() throws MojoExecutionException {
    File target = targetFile;
    StringBuilder output = new StringBuilder();
    if (target.exists()) {
      output.append(readFile(target));
      getLog().info("loaded targetFile " + target.getAbsolutePath());
    } else {
      // make sure the folder we'll be writing to exists...
      if (!target.getParentFile().exists()) {
        boolean created = target.getParentFile().mkdirs();
        if (created) {
          getLog().info("created new folders " + target.getParentFile().getAbsolutePath());
        } else {
          getLog().error("unable to create target folders, exitting.");
          return;
        }
      }
    }
    // append all the sourcefiles
    for (File source : sourceFiles) {
      if (source.exists()) {
        getLog().info("appending source " + source.getAbsolutePath());
        output.append("\n").append(readFile(source));
      } else {
        getLog().error("file did not exist: " + source.getAbsolutePath());
      }
    }
    // and write the output
    write(target, output);
  }
 /**
  * Writes the String to a file
  * @param target File target file to write to
  * @param output StringBuilder containing the data to be written
  */
  private void write(File target, StringBuilder output) {
    // write the entire buffer
    Writer out = null;
    try {
      out = new OutputStreamWriter(new FileOutputStream(target), CHARSET);
      out.write(output.toString());
      out.close();
    } catch (UnsupportedEncodingException e) {
      getLog().error("unsupported encoding: " + CHARSET, e);
    } catch (FileNotFoundException e) {
      getLog().error("file not found: " + targetFile, e);
    } catch (IOException e) {
      getLog().error("i/o exception", e);
    }
  }
 /**
  * Reads a file into a String
  * @param file File
  * @return String
  */
  private String readFile(File file) {
    StringBuilder sb = new StringBuilder();
    Scanner scanner = null;
    try {
      scanner = new Scanner(file, CHARSET);
      scanner.useDelimiter(DELIMITER);
    } catch (FileNotFoundException e) {
      getLog().error("file not found while trying to read file " + file.getAbsolutePath(), e);
      return "";
    }
    while (scanner.hasNextLine()) {
      sb.append(scanner.nextLine()).append(DELIMITER);
    }
    scanner.close();
    return sb.toString();
  }
 /**
  * @return File[]
  */
  public File[] getSourceFiles() {
    return sourceFiles;
  }
 /**
  * @param sourceFiles File[] to set.
  */
  public void setSourceFiles(File[] sourceFiles) {
    this.sourceFiles = sourceFiles;
  }
 /**
  * @return File
  */
  public File getTargetFile() {
    return targetFile;
  }
 /**
  * @param targetFile File to set.
  */
  public void setTargetFile(File targetFile) {
    this.targetFile = targetFile;
  }
}

And that is all that there is to it. How maven knows to use this class? No magic there, just because you annotated it with @goal, it will be picked up by maven. So you can put multiple goals in one plugin, nicely¬†separated¬†into different classes. Of course, to deploy this plugin to your repository, you will need to add a ‘distributionManagement’ section to your pom.xml, but if you just have a local repository, running ‘mvn install’ will do.

Now, to use our plugin in another project, we’ll need to configure it as a plugin. We already decided that the execution phase for this plugin was to be ‘prepare-package’, so, we configure the ‘join’ task during that phase and we configure some sourceFiles and a targetFile:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
<plugin>
  <groupId>net.sirious.maven.examples.filejoiner</groupId>
  <artifactId>maven-filejoiner-plugin</artifactId>
  <version>1.0</version>
  <executions>
    <execution>
      <phase>prepare-package</phase>
      <goals>
        <goal>join</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <targetFile>${basedir}/target/combined.txt</targetFile>
    <sourceFiles>
      <sourceFile>${basedir}/src/main/resources/1.txt</sourceFile>
      <sourceFile>${basedir}/src/main/resources/2.txt</sourceFile>
    </sourceFiles>
  </configuration>
</plugin>

So, now, whenever we run the ‘prepare-package’ or during any later phase, our plugin will take 1.txt and 2.txt and append them to a new or existing combined.txt. Easy does it. I’m pretty sure that writing the plugin, distributing and configuring it in our project actually took less time than this blog post ūüôā

More reading: