Just An Application

November 17, 2009

Bootstrapping OpenJDK 7 On Snow Leopard

Filed under: Java, OpenJDK7, Snow Leopard — Tags: , , — Simon Lewis @ 10:37 am

If you want to experiment with OpenJDK 7 on MacOS X you currently have no choice but to build it from scratch. This requires JDK 6 which is OK if you are running Snow Leopard.

The source is available as a zip or via Mercurial from here.

.

[Updated: A clarification. To build on Snow Leopard you need the BSD port which is here

       http://hg.openjdk.java.net/bsd-port/bsd-port

]

The build process requires a few variables to be set, so, for convenience, I bundled them up in a script like this.

    
    make \
        ARCH_DATA_MODEL=64                                                          \
        LANG=C                                                                      \
        ALT_BOOTDIR=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home \
        ALT_FREETYPE_HEADERS_PATH=/usr/X11/include                                  \
        ALT_FREETYPE_LIB_PATH=/usr/X11/lib                                          \
        NO_DOCS=true
	

The most important is

    ALT_BOOTDIR

which identifies the home directory of JDK 6 that the build process is going to use to boot with.

This points at the version of JDK 6 which ships with Snow Leopard.

Running the script fairly quickly gets you this

    
    [javac] [...]/ClassReader.java:860: reference to Version is ambiguous, both  ... and  ... match
    [javac]         AttributeReader(Name name, Version version, Set<AttributeKind> kinds) {
    [javac]                                    ^
    [javac] [...]/ClassReader.java:873: reference to Version is ambiguous, both ... and ... match
    [javac]         final Version version;
    [javac]               ^
   

I’ve elided the full path names and the classes for clarity.

The file being compiled is

    $(JDK7_ROOT)/langtools/src/share/classes/com/sun/tools/javac/jvm/ClassReader.java

The two matching classes are

    com.sun.tools.javac.jvm.ClassFile.Version

and

    com.sun.tools.javac.util.Version

The class com.sun.tools.javac.util.Version is being picked up by a wildcard import.

    import com.sun.tools.javac.util.*;

and com.sun.tools.javac.jvm.ClassFile.Version appears in a static import

    import static com.sun.tools.javac.jvm.ClassFile.Version.*;

There is no source for com.sun.tools.javac.util.Version so it looks as though javac has found its own version. Possibly this is a function of the version of javac that ships with Snow Leopard ? Rather than get bogged down at this point I simply edited the source to disambiguate the two uses.

Restarting the build results in a successful make in the langtools directory, but then it falls over again, this time while attempting to build a native library somewhere in the corba directory.

The actual error message is

    ld: library not found for -ljvm

Looking at the offending link command shows this


    -L/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/jre/lib/amd64 -ljava

and


    -L/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/jre/lib/amd64/server -ljvm

twice for some reason. Be that as it may, the problem is that neither directory exists. In the Snow Leopard version of JDK 6 all the libraries are all kept somewhere else, in

   /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Libraries

in fact.

At this point you can choose to tangle with the makefiles down in the corba directory, mess around with the structure of Apple’s JDK 6 release, or do what I did and set up a new JDK 6 home just for building OpenJDK 7.

Initially the top level directory looks like this.


Macintosh:jdk6home simon$ ls -la
total 8
drwxr-xr-x  4 simon  simon  136 16 Nov 09:33 .
drwxr-xr-x  7 simon  simon  238 16 Nov 09:32 ..
lrwxr-xr-x  1 simon  simon   67 16 Nov 09:33 bin -> /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/bin
drwxr-xr-x  3 simon  simon  102 16 Nov 09:33 jre

jre/lib/amd64 like this


Macintosh:amd64 simon$ ls -la
total 8
drwxr-xr-x  4 simon  simon  136 16 Nov 15:26 .
drwxr-xr-x  4 simon  simon  136 16 Nov 09:38 ..
lrwxr-xr-x  1 simon  simon   83 16 Nov 09:38 libjava.dylib -> /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Libraries/libjava.jnilib
drwxr-xr-x  3 simon  simon  102 16 Nov 09:38 server

and jre/lib/amd64/server like this


Macintosh:server simon$ ls -la
total 8
drwxr-xr-x  3 simon  simon  102 16 Nov 09:38 .
drwxr-xr-x  4 simon  simon  136 16 Nov 15:26 ..
lrwxr-xr-x  1 simon  simon   81 16 Nov 09:33 libjvm.dylib -> /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Libraries/libjvm.dylib

Changing ALT_BOOTDIR to point at the new ersatz JDK 6 home and restarting the build results in the successful completion of the make in the

  • corba,
  • jaxp, and
  • jaxws

directories, followed, eventually, by this failure, while building in the hotspot directory.

    Missing [...]/jdk6home/lib/tools.jar file. Use 1.6.0 or later version of JDK

Apple JDK 6 doesn’t have a tools.jar but it does have a classes.jar in

    /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes

which seems to contain all of the com.sun.tools packages.

Creating a lib directory and adding a symbolic link to classes.jar as tools.jar seems to do the trick and the build gets a bit further before hitting another problem.

    [...]hotspot/agent/src/os/bsd/StubDebuggerLocal.c:26:17: error: jni.h: No such file or directory

As before symbolic linking is our friend. This time to

	/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Headers/

Having fixed this the hotspot build eventually, it takes a while, completes and then tries to run a sanity test


    All done.
    cd bsd_amd64_compiler2/product && ./test_gamma
    java full version "1.6.0_15-b03-219"
    There was an error trying to initialize the HPI library.
    Could not create the Java virtual machine.

It turns out that this problem, unlike the previous ones, cannot be solved by a sleight of directory so to speak.

     test_gamma

is a shell script generated during the build process that compiles a Java file and then runs something called

     gamma

on it.

gamma is built from launcher.c which simply includes the files

    java.c

and

    java_md.c

These are OS specific files. In this case they are the BSD versions and can be found in

    $(JDK7_ROOT)/hotspot/src/os/bsd/launcher

What gamma does is establish the correct environment for the JVM and then, as you can deduce from the error message, run it. Part of the JVM environment as implemented by Sun is the HPI library alluded to in the error message.

This is dynamically loaded by the method

    void hpi::initialize_get_interface(vm_calls_t *callbacks)

which is defined in the file

    $(JDK7_ROOT)/hotspot/src/os/bsd/vm/hpi_bsd.cpp

except that it is not loaded because it does not exist. Hence the error message.

HPI stands for Host Porting Interface. It abstracts out a number of low-level OS specific operations such as DLL, file, and socket access. It is effectively part of the canonical model of how the Sun JVM is implemented.

The problem is that the Apple version of JDK 6 does not have a library called

    libhpi.dylib

nor anything that looks like as though it is standalone replacement for it.

At this point the JVM, or at least a version of it has been built. Now an attempt is being made to test it by running it in conjunction with the rest of the VM runtime from the JDK identified by ALT_BOOTDIR. It uses the boot version because the rest of the correct runtime has not been built yet, because some of it is dependent on the JVM.

So what happens if we ignore the test by commenting out the contents of

    $(JDK7_ROOT)/build/bsd-amd64/hotspot/outputdir/bsd_amd64_compiler2/product/test_gamma

and carry on regardless ?

Eventually, and it really is quite a long eventually, we get this


...
>>>Making sec-files-win @ Tue Nov 17 08:04:08 GMT 2009 ...
>>>Making jgss-files @ Tue Nov 17 08:04:08 GMT 2009 ...
>>>Finished making images @ Tue Nov 17 08:04:08 GMT 2009 ...
########################################################################
##### Leaving  jdk for target  sanity all  images                  #####
########################################################################
########################################################################

Control bsd amd64 1.7.0-internal build_product_image build finished: 
Control bsd amd64 1.7.0-internal all_product_build build finished: 
Control bsd amd64 1.7.0-internal all build finished: 

Does the result of all this work ? Well, I don’t have access to the TCK or anything, so I cannot say with any certainty that it is a bona fide JDK 7, but a couple of basic sanity tests seem to show that javac and java are doing the right thing, and it can for example run the Equinox OSGI framework.


Copyright (c) 2009 By Simon Lewis. All Rights Reserved.

Advertisements

5 Comments »

  1. […] InvokeDynamic And MethodHandles On Snow Leopard Filed under: JSR292, Java, Java VM, MethodHandles, MethodTypes, OpenJDK7, Snow Leopard, invokedynamic — Tags: invokedynamic, Java, JavaVM, JSR292, MethodHandles, MethodTypes, OpenJDK7, SnowLeopard — Simon Lewis @ 6:32 pm The most interesting feature in JDK 7 for me, at least until the recent reappearance of closures, and probably even then, is the JSR 292 invokedynamic support. It is basically the reason why I started my attempt to build OpenJDK 7 on Snow Leopard. […]

    Pingback by InvokeDynamic And MethodHandles On Snow Leopard « Just An Application — November 22, 2009 @ 6:32 pm

  2. […] — Tags: Java, OpenJDK7, SnowLeopard — Simon Lewis @ 8:56 pm The process described here builds a 64-bit version of OpenJDK […]

    Pingback by Bootstrapping OpenJDK 7 On Snow Leopard: The 32-Bit Version « Just An Application — November 23, 2009 @ 8:56 pm

  3. […] Want to bootstrap OpenJDK 7 on Snow Leopard? Here is how thanks to the work of one smart guy.  Click here. […]

    Pingback by Want to bootstrap OpenJDK 7 on Snow Leopard? « Chicago Mac/PC Support — November 29, 2009 @ 5:28 pm

  4. You’ll have an easier time of it if you just use Soylatte to bootstrap, as it uses the standard VM layout and contents: http://landonf.bikemonkey.org/static/soylatte/

    Comment by Craig Mirkan — November 30, 2009 @ 7:01 pm

    • Hi Craig,

      Thanks for your comment.

      I had used Soylatte to build JDK7 on Leopard.

      I was just curious as to whether it was still necessary given that Snow Leopard shipped with a version of JKD6

      simon

      Comment by Simon Lewis — November 30, 2009 @ 7:29 pm


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: