Wednesday, 19 March 2014

JMX, Tomcat and VisualVM

I've spent most of today wrestling with this and it ought to have been easier, but everywhere I looked for instructions had lots of steps I didn't need and almost all of them missed one vital step.

So what am I trying to do? Java has a feature called JMX which allows us to expose parts of our applications to the outside world for the purposes of monitoring and control. For example I have a small lock management system. Although it never goes wrong, and never will, of course, it seems prudent to expose a way for a sysadmin to go kill a lock that has been left in place by mistake. JMX exposes 'MBeans' which are essentially Java classes which have methods that JMX can let me call remotely.

The environment: Tomcat 7, JDK7, VisualVM (bundled with JDK)

Tomcat is my app server and it contains my application code. It has JMX services built in. VisualVM is my client, it gives me a UI that I can do the monitor/control stuff from. In addition I am using Spring 3.2.6 in my application because it has code to simplify exposing the MBeans.

Spring used to have a separate module for JMX called spring-jmx, but I noticed that has not been updated since version 2. They've rolled the JMX code into spring-context. I already have that library in my maven dependencies so that's fine.

I added the following code to my Spring configuration file:

<bean id="simpleLockerJMX" class="nz.co.senanque.locking.simple.SimpleLockerJMX" />

<bean class="org.springframework.jmx.export.MBeanExporter"
        lazy-init="false"
>

   <property name="beans">
     <map>
     
<entry key="bean:name=simpleLockerJMX" 

        value-ref="simpleLockerJMX" />
   
</map >
  
</property ></bean>



The first bean: simpleLockerJMX is the MBean I want to expose through JMX. The second one is the way to tell Spring this is what I want to do. They do make this very easy. The simpleLockerJMX bean doesn't need to know it is an MBean, it is just a simple Java class. There are many, many posts around to make this more complicated, including in the Spring docs, but all I need here is enough to prove the concept, and this works. It is, I think, limited to the local machine and has no security (other than being limited to the local machine, of course). Those options can be added if you want more complexity.

Now to tell Tomcat we want it to do JMX. This is done by adding these to the catalina.sh file:

 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8090 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.hostname=localhost

They get added to the CATALINA_OPTS argument. Adjust your port to one that is available on your system. If you're running Tomcat under Eclipse then add those entries to the VM Arguments in the arguments tab in the tomcat launcher. Also edit your tomcat-users.xml file to grant one of your users the role: manager-jmx

Now you can start Tomcat and it should be exposing the beans. You can check by logging into the Manager app in Tomcat (which is not there if you're running under Eclipse, so start it stand-alone for that step) and there's a way to display the exposed beans. This is using an internal view of things though, not quite the same as exposing them to the outside. Here is what you do:
  1. Browse to http://localhost:8080/manager/jmxproxy/
  2. log in as the user who has manager-jmx
You should see a fairly crude dump of the exposed MBeans, including the new one.

Next step is to start VisualVM. This is a utility that comes with Java. There used to be a similar utility called jconsole that VisualVM supersedes. Run it by typing jvisualvm on a command line.

The first thing you must do once you start VisualVM is install the MBean plugin. I wasted hours by missing this step. Go to Tools..Plugins then the Available Plugins Tab and check the VisualVM-MBeans plugin and follow the short install procedure.

Then you should see something like this:

There are more methods exposed on the class than I'd like but we are definitely seeing it in VisualVM so I call it working.

To reduce the excess exposure of the MBean I added Java attributes like this:

@ManagedResource(objectName = "nz.co.senanque.locking:name=simpleLocker", 
description = "manager for Simple Lock Factory")
public class SimpleLockerJMX {

    @Autowired private SimpleLockFactory m_simpleLocker;
    public SimpleLockFactory getSimpleLocker() {
        return m_simpleLocker;
    }
    public void SimpleLockerFactory(SimpleLockFactory simpleLocker) {
        m_simpleLocker = simpleLocker;
    }
    @ManagedAttribute(description="The current locks")
    public String getDisplayLocks() {
        return m_simpleLocker.toString();
    }
    public void setDisplayLocks(String s) {
        return ;
    }
    @ManagedOperation
    @ManagedOperationParameters ({
        @ManagedOperationParameter(

         description="Name of the lock to kill", name="lockName")  
    })
    public void killLock(String lockName){
        m_simpleLocker.unlock(lockName);
    }
    public void setSimpleLocker(SimpleLockFactory simpleLocker) {
        m_simpleLocker = simpleLocker;
    }
}

This bean just delegates to another bean, the SimpleLockerFactory, to do what we want. That bean is not an MBean, though it is a Spring bean, so it is not exposed through JMX. I added the @Managed... attributes to the SimpleLockerJMX I had before and changed the Spring configuration a little:
    <bean id="simpleLockerJMX"
     class="nz.co.senanque.locking.simple.SimpleLockerJMX" />
    <context:mbean-export/>

And that is all I need. The result in VisualVM is just getDisplayLocks and killLock are visible, which is what I want. If I annotate any other classes like this one they will be picked up automatically by Spring. It does mean I have Spring annotations in my Java, which means it is dependent on Spring, but I have Spring dependencies in other places already.

This was all just fine until I added another JMX bean. It looks more or less like the one above, but with a different delegation bean injected. And that doesn't work at all. The stack trace is confusing but it seems to be a problem with Spring's initialization. It is as if the MBean exporter requires all dependent beans to be initialized before the MBean can be completed. In this case they aren't, though it isn't clear to me why. I got around it by deferring the injection of the dependent bean. The code looks like this:
    @ManagedOperation
    public boolean isFrozen() {
        return getExecutor().isFrozen();
    }
    private Executor getExecutor() {
        if (m_executor == null) {
            m_executor = (Executor)m_beanFactory.getBean("executor");
        }
        return m_executor;
    }

This makes the class even more dependent on Spring, of course. But it does work.
Post a Comment