Low Cost Grails / Java Web Hosting HowTo - vServer memory and java heap size trouble
Java Virtual Machines and Linux virtual Servers do not play well with each other all the time. Some tweaking and configuration will be necessary to get it working and optimize the use of the (memory) resources.
You want to use java on your vServer for example with a minimum of 256 mb, you type
# java -version
and to your unpleasant surprise you get
Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.
#
although there is enough free memory. Why does this happen with Java 5 on vServers and how do you fix it?
Memory system on a vServer
A vServer is a virtualization solution with a paravirtualized Linux Kernel. Every virtual Linux operation system instance on one server uses the same kernel. The total main memory of the server is shared by each OS instance (of which the hosting provider may run up to 768 an one server!). Since memory is the most scarce resource on a vServer, they are charged by the amount of it. And they use a trick to relieve the demand for more memory: every instance gets a minimum guaranteed amount of memory and a maximum amount of memory which is available only if the other instances on the same server don't use it. The later is called FlexRAM and is often used for marketing. For the rest of the how-to we shall call them hostMem, minMem and maxMem.
How much memory do/did I get?
If your provider did not specifiy how much memory your vServer instance got, you have to see for yourself, type:
# cat /proc/user_beancounters | grep privvmpages
and you'll see some integer values, for example
# privvmpages 103010 103030 131072 133120 13
At first sight you are probably startled like 'did I only got 132 kilobyte or 132 megabyte or what?'. The answer is that these are pages of 4kb each on a 32-Bit Linux system (and of 8kb on a 64-Bit one although I can't check it myself). You'll have to multiply by 4 to get the size in kilobyte, and you'll have to divide by 1024 to get the size in megabyte. So what do this numbers mean?
Your processes currently use (reserve) 103010 pages, at some point in time they used a maximum of 103030 pages (100 mb), the barrier of memory reservation request is set to a maximum of 131072 pages (512 mb). Under no conditions the limit of 133120 pages shall ever be eceeded. And the barrier was tried to exceed 13 times in the past ( since reboot).
Your 'java -version' calls will increment the last value with every call. Try it out!
But is this minMem or maxMem? Unfortunatly it is maxMem, therefore you may not rely on it. Here comes the minMem:
# cat /proc/user_beancounters | grep oomguarpages
which gives you for example
# oomguarpages 41914 41914 49152 2147483647
Only the third number is relevant. 49152 pages are only 192 mb of guaranteed memory. To be informed before you order your vServer there are dumps of the output of "cat /proc/user_beancounters" for every bif german vServer provider in the serversupportforum. They even have a script to print the memory statistics summary in a nice format named vpsmem.
But even 192 mb of memory should suffice to run 'java -version'. Something else must go wrong.
Automatic heap size dimensioning in Java 5
Java 1.4.2 had a default maximum heap size of 64 mb. With Java 5 this was changed. If no "-client" or "-server" or "-Xms" or "-Xmx" options are given at the java call the java virtual machine tries to calculate reasonable defaults for the computing environment. If the host has at least 2 cpus and at least 2 gb of memory option "-server" is assumed. This activates the hotspot engine and the size of several buffers and caches is increased. The VSS size of my benchmark grails app was 787948 with "-server -Xmx512m" and 743640 with "-client -Xmx512m". This is an astounishing difference of 43 mb which could decide wether your app does run or not under more restrained memory conditions.
The following is taken from official documentation from Sun.
The "-Xms" parameter is taken from the larger of two values: 1/64 of physical memory or a reasonable value depending on the platform (which is not stated, but I guess it's 32mb on 32-Bit Linux).
The "-Xmx" parameter is taken from the smaller of two values: 1/4 of physical memory or 1 gb.
Example
Your vServer host has a physical memory of 8 gb and a dual core cpu or two cpus. The Java virtual machines therefore selects the "-server" option if none stated and calculates the heap size as follows:
1/64 * 8 gb = 128 mb > 32mb
It selects the larger value of 128 mb as minimum heap size.
1/4 * 8 gb = 2 gb > 1gb
It selects the smaller value of 1 gb as the maximum heap size.
The resulting call is equivalent to
# java -server -Xms128m -Xmx1024m -version
If your maximum memory of your vServer is even under good circumstances limited to 512 mb? Your Java call will not get the memory and fail like
Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.
and it is right!
The solution
The solution becames simple. Just provide every Java call with the explicit maximum heap size parameter and if you want to save memory add the "-client" parameter too.
If you are starting your Grails application directly it hides its Java options in the $GRAILS_HOME/bin/startGrails shell script. Some lines under the startGrails() function you may change it to
JAVA_OPTS="-client -Xmx256M $JAVA_OPTS"
or you specify $JAVA_OPTS in some environmental init script like $HOME/.bashrc or /etc/profile.
If you are using a standalone Jetty server the Java default is found (Debian 4) under:
/etc/default/jetty
Add a line like the one above to the 'jetty' file.
Special Debian 4 problem
Most vServers are installed and configured as LAMP systems. Therefore Java is missing and has to be installed by your own. It is preferable to use the package manager of your Linux distribution to install the java virtual machine. You'll want a fairly current JVM so your distribution should be a current one like Debian 4. Many providers offer only Debian 3 installations and you may get no Sun JDK or Sun JDK 1.4 only. In the Debian 4 distribution Sun JDK 5 is installable as a paket, but it does fail on vServers in the post-install configuration phase. Your Sun JDK package will never be marked as installed because it is not configured. There is an official bug report #402067 on that issue, but it is not solved and closed.
Debian 4 Sun JDK Solution
In file
/var/lib/dpkg/info/sun-java5-bin.postinst
add a '#' to the beginning of the following line to make it a comment ( it's not need to work )
# $basedir/bin/java -client -Xshare:dump > /dev/null
Then reconfigure the package with dpkg, dselect or aptitude
# dpkg --configure sun-java5-bin
and Sun Java 5 will be correctly installed.
Start having fun with Java, Groovy and Grails in your vServer!
Thanks for that hint. It made my day. :)
Gesendet von Marco Bakera am March 09, 2008 at 05:03 PM CET #
Thanks a lot for the great information!
Gesendet von Sven am March 21, 2008 at 01:04 PM CET #
You are awesome, my friend!
Gesendet von Karl am August 25, 2008 at 03:48 AM CEST #
Great. Thank you. Saved my day too!
Gesendet von ReneS am October 28, 2008 at 08:05 AM CET #