Performance Graph

The Java is Faster than C++ and C++ Sucks Unbiased Benchmark

November 14, 2003
UPDATE July 2010:
Despite its obvious flaws, this benchmark continues drawn a lot of attention and e-mail. The reason I posted it nearly seven years ago was not to prove anything, but rather to spark conversation. Prior to my results being published on Slashdot and the like, there was little discussion in the public sphere about whether JIT runtimes could even compare to native compiled languages' performance. To that end, I think this project succeeded in bringing such a dialogue into the developer world, and while this benchmark may not be perfect, this was a starting point for more thorough treatments and analyses of the subject.

Introduction

I was sick of hearing people say Java was slow, when I know it's pretty fast, so I took the benchmark code for C++ and Java from the now outdated Great Computer Language Shootout and ran the tests myself.

I used G++ (GCC) 3.3.1 20030930 (with glibc 2.3.2-98) for the C++, with the -O2 flag (for both i386 and i686). I compiled the Java code normally with the Sun Java 1.4.2_01 compiler, and ran it with the Sun 1.4.2_01 JVM. I ran the tests on Red Hat Linux 9 / Fedora Test1 with the 2.4.20-20.9 kernel on a T30 laptop. The laptop has a Pentium 4 mobile chip, 512MB of memory, a sort of slow disk.

The results I got were that Java is significantly faster than optimized C++ in many cases.

The results I got show that no one should ever run the client JVM when given the choice. (Everyone has the choice. To run the server VM, see instructions in the Using the Server JVM section below.)

I will post here anyone else's results as long as they use Java 1.4.2 or higher and any version of GCC that produces faster or equivalent code than the 3.3.1 I used. I encourage you to download the source and/or the binaries and perform the tests yourself, with your favorite compiler and on your favorite platform. See the Notes section below for details.

Data and Results

Notes

JVM startup time was included in these results. That means even with JVM startup time, Java is still faster than C++ in many of these tests.

Some of the C++ tests would not compile. I've never been very good at decoding GCC's error messages, so if I couldn't fix a test with a trivial modification, I didn't include it in my benchmarks.

I modified one of the tests, the string concatenation test for Java. The test was creating a new StringBuffer in each iteration of the loop, which was just silly. I updated the code to use a single StringBuffer and appending to it inside the loop. (The updated tests at the original shootout use this new method.) Java lost this benchmark even with the modifications, so if you want to accuse me of biasing the results, you're going to have to try harder.

Several versions of some of the C++ tests (like matrix) were present in the original shootout source. I used the versions without numbers in them, like matrix.g++ instead of matrix.g++2. I don't know which of these were used in the original benchmarks, but from my quick experimenting, the numberless ones generally ran faster than their numbered counterparts. (Looking at them again, matrix.g++3 runs faster than the matrix.g++ that I use. However, it still runs slower than the Java version, so I don't plan to modify the graph/data unless someone asks me to, since getting that graph in the first place was sort of a pain.)

I've been told that the C++ code for the Method Call benchmark returns by value while the Java code returns by reference, and that modifying the C++ code to pass a pointer makes that benchmark faster. However, even with the modification, the C++ version still runs slower than the Java version.

I ran the tests many times before running the "official" recorded set of tests, so there was plenty of time for both Java and the C++ tests to "warm up" (both the Java and C++ tests got faster after I ran them a few times).

I've been told that these tests are invalid because they were run with GCC. I have seen both benchmarks that show GCC producing faster code than Visual Studio's VC++ compiler, and benchmarks showing the opposite. If I update the benchmarks with another compiler added, it will be the Intel C++ Compiler, which I'm pretty sure produces faster code than VC++.

I've been accused of biasing the results by using the -O2 option for GCC, supposedly because -O2 optimizes for space, thus slowing down the benchmark. This is not what -O2 does. According to the GCC -O documentation:

-O2: Optimize even more. GCC performs nearly all supported optimizations that do not involve a space-speed tradeoff. The compiler does not perform loop unrolling or function inlining when you specify -O2. As compared to -O, this option increases both compilation time and the performance of the generated code.
On the other hand, -O3 performs space-speed tradeoffs, and -O performs fewer optimizations. Thus, for these tests, I think -O2 was the best choice.

I don't have an automated means of building and benchmarking these things (and the scripts that came with the original shootout didn't run for me). I really do want you to test it on your own machine, but it's going to take some work, I guess. I compiled the C++ code with:

g++ [test].cpp -O2 -march=i386 -o [test]-386

g++ [test].cpp -O2 -march=i686 -o [test]-686

I compiled the Java code with:

javac [test].java

To see how I ran the binaries, see the run log. You can download the source code I used in .bz2, .zip format.

Using the Server JVM

Every form of Sun's Java runtime comes with both the "client VM" and the "server VM." Unfortunately, Java applications and applets run by default in the client VM. The Server VM is much faster than the Client VM, but it has the downside of taking around 10% longer to start up, and it uses more memory.

There are two ways to run Java applications with the server VM:

  1. When launching a Java application from the command line, use java -server [arguments...] instead of java [arguments...]. For example, use java -server -jar beanshell.jar.
  2. Modify the jvm.cfg file in your Java installation. (It's a text file, so you can use Notepad or Emacs to edit it.) This is located in C:\Program Files\Java\j2reXXX\lib\i386\ on Windows, /usr/java/j2reXXX/lib/i386/ on Linux. You will see two lines: 
    -client KNOWN
-server KNOWN
    You should change them to: 
    -server KNOWN
-client KNOWN
    This change will cause the server VM to be run for all applications, unless they are run with the -client argument.

Contact

I can be contacted at .

Links / Responses

A poem from Rokky05: 
Full nonsense did not see not when rapid program on Java, as server so client.


Yes you what, boys, Java-machine

together with the famous fitter

garbage is written on what ... at least

on C, possibly even with the use of classes, and this already

C++ (on what to be yet written her not on asme certainly),

But as C can work quick than itself????


About the author of this article:

All that a fool does,

all he does not so !!!!!


Not article, and some delirium! Request

to the author of this article:



Write what firm do you work on and who?

Such feeling, that you see a computer only then when you write the articles!
Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License.