Wednesday, 19 March 2014

JMX, Tomcat and VisualVM

I've spent most of today wrestling with this and it ought to have been easier, but everywhere I looked for instructions had lots of steps I didn't need and almost all of them missed one vital step.

So what am I trying to do? Java has a feature called JMX which allows us to expose parts of our applications to the outside world for the purposes of monitoring and control. For example I have a small lock management system. Although it never goes wrong, and never will, of course, it seems prudent to expose a way for a sysadmin to go kill a lock that has been left in place by mistake. JMX exposes 'MBeans' which are essentially Java classes which have methods that JMX can let me call remotely.

The environment: Tomcat 7, JDK7, VisualVM (bundled with JDK)

Tomcat is my app server and it contains my application code. It has JMX services built in. VisualVM is my client, it gives me a UI that I can do the monitor/control stuff from. In addition I am using Spring 3.2.6 in my application because it has code to simplify exposing the MBeans.

Spring used to have a separate module for JMX called spring-jmx, but I noticed that has not been updated since version 2. They've rolled the JMX code into spring-context. I already have that library in my maven dependencies so that's fine.

I added the following code to my Spring configuration file:

<bean id="simpleLockerJMX" class="nz.co.senanque.locking.simple.SimpleLockerJMX" />

<bean class="org.springframework.jmx.export.MBeanExporter"
        lazy-init="false"
>

   <property name="beans">
     <map>
     
<entry key="bean:name=simpleLockerJMX" 

        value-ref="simpleLockerJMX" />
   
</map >
  
</property ></bean>



The first bean: simpleLockerJMX is the MBean I want to expose through JMX. The second one is the way to tell Spring this is what I want to do. They do make this very easy. The simpleLockerJMX bean doesn't need to know it is an MBean, it is just a simple Java class. There are many, many posts around to make this more complicated, including in the Spring docs, but all I need here is enough to prove the concept, and this works. It is, I think, limited to the local machine and has no security (other than being limited to the local machine, of course). Those options can be added if you want more complexity.

Now to tell Tomcat we want it to do JMX. This is done by adding these to the catalina.sh file:

 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8090 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.hostname=localhost

They get added to the CATALINA_OPTS argument. Adjust your port to one that is available on your system. If you're running Tomcat under Eclipse then add those entries to the VM Arguments in the arguments tab in the tomcat launcher. Also edit your tomcat-users.xml file to grant one of your users the role: manager-jmx

Now you can start Tomcat and it should be exposing the beans. You can check by logging into the Manager app in Tomcat (which is not there if you're running under Eclipse, so start it stand-alone for that step) and there's a way to display the exposed beans. This is using an internal view of things though, not quite the same as exposing them to the outside. Here is what you do:
  1. Browse to http://localhost:8080/manager/jmxproxy/
  2. log in as the user who has manager-jmx
You should see a fairly crude dump of the exposed MBeans, including the new one.

Next step is to start VisualVM. This is a utility that comes with Java. There used to be a similar utility called jconsole that VisualVM supersedes. Run it by typing jvisualvm on a command line.

The first thing you must do once you start VisualVM is install the MBean plugin. I wasted hours by missing this step. Go to Tools..Plugins then the Available Plugins Tab and check the VisualVM-MBeans plugin and follow the short install procedure.

Then you should see something like this:

There are more methods exposed on the class than I'd like but we are definitely seeing it in VisualVM so I call it working.

To reduce the excess exposure of the MBean I added Java attributes like this:

@ManagedResource(objectName = "nz.co.senanque.locking:name=simpleLocker", 
description = "manager for Simple Lock Factory")
public class SimpleLockerJMX {

    @Autowired private SimpleLockFactory m_simpleLocker;
    public SimpleLockFactory getSimpleLocker() {
        return m_simpleLocker;
    }
    public void SimpleLockerFactory(SimpleLockFactory simpleLocker) {
        m_simpleLocker = simpleLocker;
    }
    @ManagedAttribute(description="The current locks")
    public String getDisplayLocks() {
        return m_simpleLocker.toString();
    }
    public void setDisplayLocks(String s) {
        return ;
    }
    @ManagedOperation
    @ManagedOperationParameters ({
        @ManagedOperationParameter(

         description="Name of the lock to kill", name="lockName")  
    })
    public void killLock(String lockName){
        m_simpleLocker.unlock(lockName);
    }
    public void setSimpleLocker(SimpleLockFactory simpleLocker) {
        m_simpleLocker = simpleLocker;
    }
}

This bean just delegates to another bean, the SimpleLockerFactory, to do what we want. That bean is not an MBean, though it is a Spring bean, so it is not exposed through JMX. I added the @Managed... attributes to the SimpleLockerJMX I had before and changed the Spring configuration a little:
    <bean id="simpleLockerJMX"
     class="nz.co.senanque.locking.simple.SimpleLockerJMX" />
    <context:mbean-export/>

And that is all I need. The result in VisualVM is just getDisplayLocks and killLock are visible, which is what I want. If I annotate any other classes like this one they will be picked up automatically by Spring. It does mean I have Spring annotations in my Java, which means it is dependent on Spring, but I have Spring dependencies in other places already.

This was all just fine until I added another JMX bean. It looks more or less like the one above, but with a different delegation bean injected. And that doesn't work at all. The stack trace is confusing but it seems to be a problem with Spring's initialization. It is as if the MBean exporter requires all dependent beans to be initialized before the MBean can be completed. In this case they aren't, though it isn't clear to me why. I got around it by deferring the injection of the dependent bean. The code looks like this:
    @ManagedOperation
    public boolean isFrozen() {
        return getExecutor().isFrozen();
    }
    private Executor getExecutor() {
        if (m_executor == null) {
            m_executor = (Executor)m_beanFactory.getBean("executor");
        }
        return m_executor;
    }

This makes the class even more dependent on Spring, of course. But it does work.

Wednesday, 12 March 2014

Ash Wednesday in the South

Ash Wednesday has just come and gone again this year. It is one of those things that us Anglicans have a mixed commitment to. Some of us do the whole ashes on the forehead thing as shown, some of us let it go by, possibly some of us wonder what the point is.

For those of you who don't know, Ash Wednesday marks the beginning of Lent which is a time we're supposed to hold back on the material things of life and take some time to build up our spiritual side so we can more appreciate Easter which comes at the end of Lent. Anglicans don't have strict rules around what we do in Lent, so some of us get into fasting etc and some of us don't and no one minds. The ashes-on-the-forehead are supposed to come from the little palm crosses they pass around on Palm Sunday (Sunday before Easter) which we've kept all year and they get burned on Ash Wednesday. I knew this well as a kid, but I never did manage to keep track of my palm cross for a whole year. Somehow by next Ash Wednesday it had gone missing. Still, it is a nice idea if you can do it.

There's a point to it all as well, the ashes are a reminder that we are all going to die (ashes-to-ashes) so this life is all temporary. In pre-Christian times Roman generals while enjoying their triumphal march into Rome after victories in far off lands would have someone next to them whispering from time to time 'remember one day you will die'. It was to keep them from getting too up themselves. There may be a connection with the ashes, though maybe not.

In the old days in the North the subsistence farmers were nearing the end of winter and their stores were getting low. It was a really good time to cut back on food, but they knew spring was not far off and something to look forward to. Weaving this notion into the Christian story made good sense even though, it should be noted, the Christian story doesn't quite fit it. Jesus fasted for 40 days which is supposed to be the Lent period, but he did it before he began his ministry, not three years later just before he was crucified. But they worked with what they had and made an annual cycle out of it all. It was probably helpful to have spiritual leaders prompting people to eke out their stores for the last of the lean season. Mardi Gras, or Carnivale, which is just before Ash Wednesday may be connected with the idea that there would not be enough food for any excess livestock in the next couple of months so best to eat them now. Carnivale and carnivore are closely related words.

Here in the southern hemisphere the whole Lent thing is awkward. We're in our abundant season. Fruit is falling from our trees uneaten because we can't keep up with it. I feel the need to just go out there and munch down a few more pears and apples and plums and figs and... well it would stop them going to waste. Sure, we preserve stuff, and even give some away, but it is still hard to keep up with. Tightening my belt around now just doesn't seem right.

There are aspects of Lent that do not involve food, but I like to think I do my share of those the rest of the time, prayer and kindness and so on. So Lent does sort of pass me by usually. Possibly there would be value in doing something Lenten in six months time, around September, but in our climate that is when spring is well underway and we're getting the first asparagus. Possibly July, which is really winter, would make more sense.

So I don't really worry about Lent too much, and I enjoy Easter when it comes.