Monday, 9 June 2014

Winging it and the Peter Principle

This story from the Guardian came at me from multiple people who +1's it or tweeted it or whatever. It's good, feels true etc. Read it now, I'll wait.

Like I said, it feels so true and at one level it certainly is. I was sitting at my laptop struggling with writing some tricky code, trying to figure out why whatever ought to work wasn't and thinking 'yeah, I am totally winging it here' which was true except for the 'totally'.

We work on the edge of our expertise a lot of the time. For many of us that's where the fun is, the challenge. It makes us grow, or it tells us our limits or something. We don't even think about the things we do that are second nature.

This is related to the 'Peter Principle', the notion that we are all promoted to one level beyond our expertise, which is just another way of saying we work on the edge. The Peter Principle is about people being promoted until we find they are in a position they cannot perform in. They've been good up until now, but the latest rise up the corporate ladder was one too many. They stay there because the people who put them there are too gutless (or incompetent) to push them back down again.

Mostly this is rubbish. It assumes there is just one point where people switch from competence to incompetence and that that this cannot be changed. I suggest that every new promotion has its challenges and each one takes time to work through, but what is really true is that we work on the edge of our expertise all the time. And it is true even for those of us who don't work in a corporate environment where 'promotion' is a foreign concept. We take on challenges because they are fun.

But are we totally winging it? Of course not.

Take some basics. I code using a laptop, mouse, second monitor etc. Do I have the slightest difficultly operating the mouse, getting it to point where I want and so on? No. It's not a trivial skill. Try teaching it to someone over 80 who hasn't used a mouse before. The development environment I prefer is Eclipse and I use it to write Java and C++ code. Navigating the file structures in that environment is something I do without hardly a thought. And, of course, those two languages have syntax which is as natural to me as speaking English. This isn't blowing a trumpet. Anyone who programs in this environment works this way.

A little while ago I reorganised my projects to build with maven instead of ant/ivy. I didn't know maven and I hate reading manuals. I generally read a little and try it, fall over, do a little more reading (just enough, I'm lazy) and get past that problem and so on. I was winging it the whole time. Now I use maven every day, and I don't think about it.

So it is true that we wing it all the time, but that is because we like working on the edge, but we're only on that edge because we have a rump of expertise to be on the edge of. So winging it, sure, but not totally winging it.

Sunday, 1 June 2014

Madura Demos

I've re-deployed my demos. This is more interesting than it sounds, well I hope it is anyway. I've already posted about each of them. This is about the pain of getting them up there and running in the cloud.

Long ago I had them running on CloudFoundry and they worked fine, then the CloudFoundry business model changed and their free option went away. Since I make no money from this stuff the demo site has to be free, so I went to AppFrog who were running the same CloudFoundry software and they lived there for a while...until the same thing happened.

Recently, after completing the Workflow project, I had some free time so I thought I'd try and get them going on Google Application Engine. But I gave up on that yesterday (yes, it was only yesterday) and loaded them onto OpenShift (Red Hat's cloud offering). They worked first time. It was so easy. I was surprised.

The reason I was surprised was that it was so very hard to do Google Application Engine. There were two main issues.

First, GAE doesn't like Logback, which is a commonly used logging mechanism and I use it everywhere (because commonly used means it should work everywhere, right?) GAE only likes JUL and no one I know uses JUL. JUL is the native Java logging mechanism and it is the reason for products like Logback, ie people hate JUL and want something else.

But switching to JUL was not a big deal. I wouldn't want to have to work with it, but these demos are already debugged so that's mostly okay.

Next GAE needs lots of classes to be serializable so it can flick sessions between servers. I see the sense of this and I went off and made all my classes serializable. It didn't take very long, actually. Of course I tried it out using GAE's Eclipse plugin to make sure it was working and it was.

But when I deployed it for real it complained of more classes that needed to be serializable. However these aren't my classes, they're libraries from other people. Spring etc. I can't change them (well, not unless I want to maintain them ever after, and I don't). I suspect there is a solution to that particular problem but I'd noticed that the only way to find out any of this stuff is with the remote deploy. The local test told me nothing. This was true of the Logback issue too. So I realised this might go on forever and I bailed.

Openshift, in contrast, was quite happy with Logback and didn't care about Serialization. That may cost them something in efficiency, but it sure got up and running fast.

There seem to be two general approaches with Openshift: using Git and using scp. The Git option assumes you store your project source on their Git repository and they build and deploy from there when you tell them. Sounds fine, but I already have a public Git repository at GitHub, and my maven build works just fine. I don't feel a pressing need to learn their build syntax etc, though it is likely easy enough.

The scp option is basically 'I have this war file, upload it and deploy it'. Great! That's what I did. Worked first time. I suspect the running applications are not as fast as they might be but for a free service I can live with that. I'll even listen to arguments that GAE's serialization requirements really help Google deliver a faster application. But working trumps speed and I have it working.

Madura Pizza Order Demo

I'd meant to post an entry about putting my demos on line so people can play with them and not have to build the software. Not everyone, it seems, is actually a software engineer :-). Then I realised I had never actually posted anything about the basic pizza order demo. So here it is.

The purpose is to show off the Madura software I've been working on some time now. All of it can be found on my Github repositories, all open source and, with one exception, all of it free. The idea with Madura is that it makes use of rich business objects (Orders, Pizzas, Employees etc) to remove code from the application. These objects are very rich, in that they can include business rules that monitor what is going on. Most people do this with application logic and UI logic which is harder to write and maintain than what we have here.

The demo needs to be run from Firefox. It works on IE6 (I don't run Windows normally but I had an old version lying around). Don't use Chrome because that will show you the mobile UI, which is a different demo. I guess I should add don't run it on your mobile device either. It works, but it isn't the demo I'm about to describe.

We are supposedly ordering a pizza (but don't worry, no pizza will be delivered). There are several pizza related products but the most interesting is the Configured Pizza, which dynamically adjusts the choices to be compatible with what has already been picked. The premise is that only some combinations of topping, base and size of pizza are allowed and the Madura-driven UI only ever
presents valid choices, dynamically altering the fields as necessary.

Click on the URL and log in as admin/admin. You should see something like this:
We'll configure a pizza in a moment, but first click on the Customer button.

Notice the four buttons at the bottom and that one of them, Dynamic,
is disabled. Enter the name 'fred' (without the quotes) in the Name field and tab off it. The Dynamic button is now active. Change 'fred' to something else. The button is disabled again.

What is going on here? Do we have some custom javascript or custom java driving this? Actually no. That's what other people do.

There is a rule defined using Madura Rules which controls whether the button is disabled or not. This is the rule:

rule: Customer "dynamic" {
    if (name == "fred") {
        dynamic = true;


 Normally this kind of thing gets done in something like Java, and you'd write code to test if the name field was 'fred', and you'd also write code to test if it was not 'fred' and then set dynamic to false. One of the nice things about rules is that their converse is implied without having to write it, so this is all we need.

The next demo step is to enter something into the email, eg 'abc'. Now you see an orange '!' beside email. This is the usual way Vaadin indicates an error. Roll over it with the mouse and you will see

Failed email: label=email not a valid email address, attempted=abc 

which is the default error message for this kind of error. The email field
requires an '@' sign so this is a simple validation error. There actually isn't a proper rule for this because it is too simple. We just add an Email annotation on the field and the rest happens. Naturally you can customise that error message and you can add I18n alternatives (which means it can show up in French etc if you need it to).

Also notice that the Save button is now disabled. This happens automatically because there is an error. You don't even have to write a rule. When the error is cleared the Save button will enable.

Change the email field to something valid and click the Save button to get back to the home page. You have a list of products on the left hand side. Expand the Sides heading and click on Boneless Chicken.
There isn't any configuration for this product, just a price. But note
that the message about the shopping cart now says it contains one item. This message is controlled by the following rules:

rule: Order "shoppingcartsize" {
    if (count(orderItems) > 0) {
        shoppingCartStatus = format("shopping.cart.status",count(orderItems));

rule: Order "shoppingcartsize" {
    if (count(orderItems) == 0) {
        shoppingCartStatus = format("shopping.cart.status.empty",0);

We could make this simpler if we were happy with saying 'Shopping cart contains 0 items', then we would only need one rule. But 'Shopping cart is Empty' looks nicer. The 'shopping.cart.status...' references are to a properties file that holds the actual messages we need, including the error messages mentioned above. The French messages are in a similar file, so if your user was French it would say "Le panier est vide" when the shopping cart is empty. To be clear, the language thing is just ordinary Java stuff, though it is rarely applied this well.

Now press the Save to Order button to keep the product in the shopping cart. The Cancel button would remove it so don't press that. The next step is to expand Pizzas and pick Configured Pizza.
This is a more complex form. There is a date field, which like all the other fields, is a standard Vaadin widget. The amount at the bottom is zero, until we pick enough options for the rules to work out a price.

But the first thing to note, which you cannot see very well on the form, is that this form is generated using exactly the same Java as the Boneless Chicken one. The only difference is that object it maps to is different. This is part of the 'rich objects' thing. The objects themselves have enough information in them for the Java code to build us a form. So defining a new product doesn't involve writing more Java (necessarily). It just needs the right object and the right rules.

Now we start configuring our pizza. Change the default Base to 'Puff' and pick Size='Medium'.
There is a new field called testing now and it has a red asterisk beside it, which is standard Vaadin for indicating a required field. The Save to Order button is disabled because we now have a required field that is not filled in. Finally you have an amount calculated. This is the relevant rule:

rule: Pizza "p3" {
    if (size == "Medium") {


amount = 15;

activate(testing) makes the field visible and require(testing) makes it required. There are no rules to disable the Save to Order, that is handled in the same way as the error in the Email field, ie automatically.

But there is more. Check the options for Topping. They should be Italian and Spanish. Now clear the Size (and notice the testing field vanishes and the button re-enables). Check the options for Topping again. There are three others. The lists of valid values are being dynamically constrained depending on what else is picked. This is independent of the order they are picked, so if you pick Topping first then Size will be constrained and if you pick Size first then Topping will be constrained.

The rules behind this aren't convenient to express in the if/then format we've seen so far, and they are awful in an ordinary language like Java. Instead we use decision tables. The format of the table in this case is XML but it could be a database table or, if you're prepared to write some code, something else. I won't bore you with the XML here. Conceptually the table looks like this:

topping          size
Seafood            Small
Italian               Medium
Spanish             Medium
Hawaiian          Large
Greek                Large

Pick some valid combination, press Save to Order then press Checkout.
This just lists the resulting order items and their prices with a total. The total is
calculated using a rule:

formula: Order "sum" {
    amount = sum(orderItems.amount);

This is fired whenever we add or remove an item from the orderItems list, or if we change the amount of something already in there.

Part of the richness of the objects is that they include permissions on the fields. For example the admin user you are logged in as has all permissions. Another user operator/operator has slightly less capability. For example he doesn't have access to the address field of the customer so when he goes to that form it doesn't show up. There is no code to write to handle that, it is just part of the object definition. Similarly if you type 'fred' into the name field the Dynamic button doesn't enable. Operator doesn't have permission to use the Dynamic button, so regardless of the value of the name it won't enable even if the rule tries to enable it. Declared permissions trump rule permissions.

All of this, apart from making the application a whole lot simpler is taking us to an interesting question. Since the business objects contain all the logic how come the UI can know what is going on?

It does it by knowing only that the objects are rich, that it can query them for labels and currently valid picks (the pizza sizes and toppings) and if they are required or not etc. It does not ever know anything about the rules, including the simple validation things like the Email validation. And this actually makes it brilliant when we want to implement another user interface.

Usually when a business decides to implement a new interface, such as something that runs on a mobile platform, people discover they've implemented a lot of business logic in their UI and they have to rewrite it for the new system. Then they have to keep the two in sync, doubling the maintenance effort. But we don't have to do any of that. We just write the new UI and leave the business rules where they are.

To prove the concept I went ahead and wrote another Vaadin UI for the pizza order. If you browse to the same URL with your iPad or Android phone or a Chrome browser you'll see it. The login is a bit fiddly, you may need to press the user and password fields to activate them, depending on your device. They are over on the right.

This mobile demo is a bit simpler, it just does the configured pizza so there's no need for a blow-by-blow walk through. You know enough now to try picking different options and see what happens. It ought to be much the same as before.

We said it before but it is worth saying again. Although the UI is different all the rules and objects are the same. The only thing we had to change was the UI code. But we do mean the bare UI. We did not have to write code to restrict the choices based on earlier selections, nor did we have to write any code to track the total order or make the submit button (actually the Log Out button here) disabled until all the required fields are complete. All of those things are managed behind the UI with the same code as before. This shows the power of keeping the business logic clearly separated from the UI, and Madura is clearly an effective way to do that.