Posts Tagged Tomcat

Too many open files on Tomcat

The other day, one of my websites was not available anymore. Looking at the log files, I found the following exception:

Dec 7, 2011 1:22:39 AM org.apache.jk.common.ChannelSocket acceptConnections
WARNING: Exception executing accept
java.net.SocketException: Too many open files
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
	at java.net.ServerSocket.implAccept(ServerSocket.java:450)
	at java.net.ServerSocket.accept(ServerSocket.java:421)
	at org.apache.jk.common.ChannelSocket.accept(ChannelSocket.java:307)
	at org.apache.jk.common.ChannelSocket.acceptConnections(ChannelSocket.java:661)
	at org.apache.jk.common.ChannelSocket$SocketAcceptor.runIt(ChannelSocket.java:872)
	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
	at java.lang.Thread.run(Thread.java:595)

This was the first time I was getting this exception. What is even stranger is that I didn’t change anything on this application for quite a while!
Anyway, I first did what I usually do with Tomcat: restart it! This fixed the issue but only for a few hours before it crashed again.

After some investigation, it seems that Tomcat was reaching the limit of open file descriptors allowed in this machine (1024 in my case).
To get the maximum number of open file descriptors, simply type the following command:

ulimit -n

It is possible to increase this value by editing the file /etc/security/limits.conf and adding the new limit for the user running Tomcat. However, this is not recommended as 1024 should be sufficient.

The second thing I did was to check the list of open files used by the Tomcat process:

lsof -p 

What I found by running this command was a bit odd. It seems that Tomcat was having a multitude of opened connections to one of the web services used by the application. So it looks like the connections between my website and the web service were never closed!  😐
Because I didn’t change the code on my side, I asked the third party who owns the web service to check their code. I don’t know what was the root cause of the problem but they fixed it on their side and it is now working fine.

In conclusion, if you get the same exception, try to find where the problem is coming from before increasing the maximum number of open file descriptors. πŸ˜‰

, , , , , ,

No Comments

Deploy your app to the root context

This is an easy trick which I am sure most of you already know.

Let’s take a Java application called MyAddressBook. Its generated war file could be called myaddressbook.war.

By default, when you deploy this web application to Tomcat, the URL to access it will be http://localhost:8080/myaddressbook/. And if you point a domain name such as ‘addressbook.com’ to this server, the URL would be http://addressbook.com/myaddressbook/.

I don’t know for you but I don’t like to systematically have the subfolder ‘myaddressbook’ after my domain. But maybe I am too picky! πŸ˜‰

The idea is to deploy our application in the Tomcat root context.
You have two ways of doing this:

  • Define a ROOT.xml file in your conf/Catalina/localhost folder, or;
  • Rename your war file to ROOT.war.

Note that the case is important, it has to be ROOT in UPPERCASE! πŸ™‚

Once this is done, you will be able to call your application via the URL http://localhost:8080/
or http://addressbook.com/. Way better! 😎

, ,

No Comments

How to monitor your Java application

It is very good to have your application and your database running on a Linux or Windows server, but who will tell you if your website is down? Who said “the users”? 😯 No, you won’t look very professional if you wait for a user complaint.

What you need to do is to monitor your server. But which tools to use? There are so many on the market… πŸ™

I tested a few of them (Hyperic, Nagios, Zenoss, Cacti, Monitis) but the one I choose is Zabbix. What I like with Zabbix is its price (free 😎 ) and the fact that it is easy to configure and very flexible. Indeed, you can monitor anything you want on the machine (CPU, network, disk space, services, etc)! πŸ™‚

However, if you also want to monitor your Tomcat application server or even Hibernate Java library, you need some more work.

To monitor Tomcat, you first need to enable JMX Remote. To do so, you can have a look at the official documentation or simply add the following parameters to your Tomcat startup script:

export CATALINA_OPTS='-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8999 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false'

Then, I will recommend you to deploy Zapcat JMX Zabbix Bridge in your Tomcat web server. This tool has been developed to simplify the communication between the JMX management API inside Java applications and the Zabbix monitoring tool.

To read the installation instructions for Tomcat, please click on the following link:
http://www.kjkoster.org/zapcat/Tomcat_How_To.html

Finally, if you are using Hibernate in your Java application and would like to monitor it, you have to instantiate and configure a Hibernate MBean (org.jboss.hibernate.jmx.Hibernate) that will be responsible for constructing a Hibernate SessionFactory and exposing it to your application through JNDI.

Here are the lines you need to add to your context XML file:

<bean id="mbeanExporter" class="org.springframework.jmx.export.MBeanExporter" lazy-init="false">
    <property name="server" ref="mbeanServerFactory"/>
    <property name="beans">
        <map>
            <entry key="org.hibernate:type=statistics"  value-ref="hibernateMBean"/>
        </map>
    </property>
</bean>

<bean id="mbeanServerFactory" class="org.springframework.jmx.support.MBeanServerFactoryBean">
    <property name="locateExistingServerIfPossible" value="true"/>
</bean>

<bean id="hibernateMBean" class="org.hibernate.jmx.StatisticsService">
    <property name="sessionFactory" ref="sessionFactory"/>
    <property name="statisticsEnabled" value="true"/>
</bean>

, , , , , , ,

No Comments