Just An Application

September 30, 2014

Building The Android Runtime (ART) For Mac OS X: Part Eight — Testing, Testing

Having got the Android Runtime to build, eventually, does it actually work ?

Running art the immediate answer is no,

    $art
    E/JniInvocation(46765): Failed to dlopen libart.so: dlopen(libart.so, 2): image not found
    Failed to initialize JNI invocation API from libart.so

The art executable is a script which contains the following so it looks as though this problem is easily fixed.

    ...
    
    invoke_with=
    DALVIKVM=dalvikvm
    LIBART=libart.so
    
    ...

Trying again after modifying the art script appropriately we get

    $art
    art I 46796 66656 art/runtime/gc/space/image_space.cc:269] RelocateImage: \
        /Users/simon/Scratch/art/out/host/darwin-x86/bin/../bin/patchoat \
            --input-image-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.art \
            --output-image-file=/Users/simon/Scratch/xper/art/android-data46788/dalvik-cache/x86_64/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.art \
            --input-oat-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.oat \
            --output-oat-file=/Users/simon/Scratch/xper/art/android-data46788/dalvik-cache/x86_64/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.oat \
            --instruction-set=x86_64 --base-offset-delta=9420800
    patchoat W 46797 66657 art/runtime/gc/heap.cc:216] \
        Could not create image space with image file '/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.art'. \
        Attempting to fall back to imageless running. \
        Error was: Failed to load /system image '/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/x86_64/core.art': \
            Failed to mmap at expected address, mapped at 0x100b06000 instead of 0x60000000

and then it hangs/spins.

The address

    0x100b06000

in the error message from patchoat shows that it is a 64-bit executable, so it needs to be rebuilt as a 32-bit executable.

Modifying the file

    art/patchoat/Android.mk

in the same way as we did for dex2oat, rebuilding and trying again, remembering to specify that we want to use a 32-bit VM, we get

    $art --32
    art I 24835 34536 art/runtime/gc/space/image_space.cc:269] RelocateImage: \
        /Users/simon/Scratch/art/out/host/darwin-x86/bin/../bin/patchoat \
            --input-image-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.art \
            --output-image-file=/Users/simon/Scratch/xper/art/android-data24827/dalvik-cache/x86/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.art \
            --input-oat-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.oat \
            --output-oat-file=/Users/simon/Scratch/xper/art/android-data24827/dalvik-cache/x86/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.oat \
            --instruction-set=x86 \
            --base-offset-delta=-11399168
    Class name required

which is definitely an improvement.

Trying again with

   -help

to find out how to actually run something we get a long list of arguments we can use and an equally long list of arguments we cannot.

    $art --32 -help
    dalvikvm: [options] class [argument ...]

    The following standard options are supported:
      -classpath classpath (-cp classpath)
      -Dproperty=value
      -verbose:tag ('gc', 'jni', or 'class')
      -showversion
      -help
      -agentlib:jdwp=options

    The following extended options are supported:
      -Xrunjdwp:<options>
      -Xbootclasspath:bootclasspath
      -Xcheck:tag  (e.g. 'jni')
      -XmsN (min heap, must be multiple of 1K, >= 1MB)
      -XmxN (max heap, must be multiple of 1K, >= 2MB)
      -XssN (stack size)
      -Xint
    
    The following Dalvik options are supported:
      -Xzygote
      -Xjnitrace:substring (eg NativeClass or nativeMethod)
      -Xstacktracefile:<filename>
      -Xgc:[no]preverify
      -Xgc:[no]postverify
      -XX:HeapGrowthLimit=N
      -XX:HeapMinFree=N
      -XX:HeapMaxFree=N
      -XX:NonMovingSpaceCapacity=N
      -XX:HeapTargetUtilization=doublevalue
      -XX:ForegroundHeapGrowthMultiplier=doublevalue
      -XX:LowMemoryMode
      -Xprofile:{threadcpuclock,wallclock,dualclock}
        
    The following unique to ART options are supported:
      -Xgc:[no]preverify_rosalloc
      -Xgc:[no]postsweepingverify_rosalloc
      -Xgc:[no]postverify_rosalloc
      -Xgc:[no]presweepingverify
      -Ximage:filename
      -XX:+DisableExplicitGC
      -XX:ParallelGCThreads=integervalue
      -XX:ConcGCThreads=integervalue
      -XX:MaxSpinsBeforeThinLockInflation=integervalue
      -XX:LongPauseLogThreshold=integervalue
      -XX:LongGCLogThreshold=integervalue
      -XX:DumpGCPerformanceOnShutdown
      -XX:IgnoreMaxFootprint
      -XX:UseTLAB
      -XX:BackgroundGC=none
      -XX:LargeObjectSpace={disabled,map,freelist}
      -XX:LargeObjectThreshold=N
      -Xmethod-trace
      -Xmethod-trace-file:filename  -Xmethod-trace-file-size:integervalue
      -Xenable-profiler
      -Xprofile-filename:filename
      -Xprofile-period:integervalue
      -Xprofile-duration:integervalue
      -Xprofile-interval:integervalue
      -Xprofile-backoff:doublevalue
      -Xprofile-start-immediately
      -Xprofile-top-k-threshold:doublevalue
      -Xprofile-top-k-change-threshold:doublevalue
      -Xprofile-type:{method,stack}
      -Xprofile-max-stack-depth:integervalue
      -Xcompiler:filename
      -Xcompiler-option dex2oat-option
      -Ximage-compiler-option dex2oat-option
      -Xpatchoat:filename
      -X[no]relocate
      -X[no]dex2oat (Whether to invoke dex2oat on the application)
      -X[no]image-dex2oat (Whether to create and use a boot image)
        
    The following previously supported Dalvik options are ignored:
      -ea[:<package name>... |:<class name>]
      -da[:<package name>... |:<class name>]
      (-enableassertions, -disableassertions)
      -esa
      -dsa
      (-enablesystemassertions, -disablesystemassertions)
      -Xverify:{none,remote,all}
      -Xrs
      -Xint:portable, -Xint:fast, -Xint:jit
      -Xdexopt:{none,verified,all,full}
      -Xnoquithandler
      -Xjniopts:{warnonly,forcecopy}
      -Xjnigreflimit:integervalue
      -Xgc:[no]precise
      -Xgc:[no]verifycardtable
      -X[no]genregmap
      -Xverifyopt:[no]checkmon
      -Xcheckdexsum
      -Xincludeselectedop
      -Xjitop:hexopvalue[-endvalue][,hexopvalue[-endvalue]]*
      -Xincludeselectedmethod
      -Xjitthreshold:integervalue
      -Xjitcodecachesize:decimalvalueofkbytes
      -Xjitblocking
      -Xjitmethod:signature[,signature]* (eg Ljava/lang/String\;replace)
      -Xjitclass:classname[,classname]*
      -Xjitoffset:offset[,offset]
      -Xjitconfig:filename
      -Xjitcheckcg
      -Xjitverbose
      -Xjitprofile
      -Xjitdisableopt
      -Xjitsuspendpoll
      -XX:mainThreadStackSize=N

Writing a very simple ‘hello world’

    package xper.hw;
    
    public final class HelloWorldART
    {
        public static void main(String[] theArgs)
        {
            System.out.println("Hello World courtesy of the Android Runtime (ART)");
            System.exit(0);
        }
    }

and turning it to a .dex and trying again we get

    art --32 -classpath hw.dex xper.hw.HelloWorldART
    art I 24871 35716 art/runtime/gc/space/image_space.cc:269] RelocateImage: \
        /Users/simon/Scratch/art/out/host/darwin-x86/bin/../bin/patchoat \
            --input-image-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.art \
            --output-image-file=/Users/simon/Scratch/xper/art/android-data24863/dalvik-cache/x86/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.art\
            --input-oat-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.oat \
            --output-oat-file=/Users/simon/Scratch/xper/art/android-data24863/dalvik-cache/x86/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.oat --instruction-set=x86\
            --base-offset-delta=-4640768
    dex2oat I 24873 35749 art/dex2oat/dex2oat.cc:1266] \
        /Users/simon/Scratch/art/out/host/darwin-x86/bin/../bin/dex2oat \
            --runtime-arg -classpath \
            --runtime-arg hw.dex \
            --instruction-set=x86 \
            --instruction-set-features=default \
            --runtime-arg -Xrelocate
            --host \
            --boot-image=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.art \
            --dex-file=/Users/simon/Scratch/xper/art/hw.dex \
            --oat-fd=3 \
            --oat-location=/Users/simon/Scratch/xper/art/android-data24863/dalvik-cache/x86/Users@simon@Scratch@xper@art@hw.dex
    dex2oat I 24873 35749 art/dex2oat/dex2oat.cc:284] dex2oat took 246.341ms (threads: 8)
    Hello World courtesy of the Android Runtime (ART)
    art I 24871 35716 art/runtime/native/java_lang_Runtime.cc:41] System.exit called, status: 0

And there you have it. Possibly the longest and most convoluted way of getting a Java ‘hello world’ program to run short of writing your own Java runtime.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

September 17, 2011

Android Internals: Resources – Part Six: The Typespec Chunk

1.0 The Example

The Package chunk header does not explicitly specify the number of TypeSpec chunks present in the Package chunk body, but there is always one TypeSpec chunk for each Resource type. The typeStrings StringPool contains the names of these types, hence there will always be the same number of TypeSpec chunks as there are type name strings, which in this case is four.

    0000000 02 00 0c 00 64 04 00 00 01 00 00 00 01 00 1c 00
    0000010 d0 00 00 00 06 00 00 00 00 00 00 00 00 01 00 00
    0000020 34 00 00 00 00 00 00 00 00 00 00 00 1d 00 00 00
    0000030 3a 00 00 00 57 00 00 00 6d 00 00 00 8f 00 00 00
    0000040 1a 1a 72 65 73 2f 64 72 61 77 61 62 6c 65 2d 6c
    0000050 64 70 69 2f 69 63 6f 6e 2e 70 6e 67 00 1a 1a 72
    0000060 65 73 2f 64 72 61 77 61 62 6c 65 2d 6d 64 70 69
    0000070 2f 69 63 6f 6e 2e 70 6e 67 00 1a 1a 72 65 73 2f
    0000080 64 72 61 77 61 62 6c 65 2d 68 64 70 69 2f 69 63
    0000090 6f 6e 2e 70 6e 67 00 13 13 72 65 73 2f 6c 61 79
    00000a0 6f 75 74 2f 6d 61 69 6e 2e 78 6d 6c 00 1f 1f 48
    00000b0 65 6c 6c 6f 20 57 6f 72 6c 64 2c 20 50 65 6e 64
    00000c0 72 61 67 6f 6e 41 63 74 69 76 69 74 79 21 00 09
    00000d0 09 50 65 6e 64 72 61 67 6f 6e 00 00 00 02 1c 01
    00000e0 88 03 00 00 7f 00 00 00 78 00 70 00 65 00 72 00
    00000f0 2e 00 72 00 65 00 73 00 6f 00 75 00 72 00 63 00
    0000100 65 00 73 00 2e 00 70 00 65 00 6e 00 64 00 72 00
    0000110 61 00 67 00 6f 00 6e 00 00 00 00 00 00 00 00 00
    0000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    *
    00001e0 00 00 00 00 00 00 00 00 1c 01 00 00 04 00 00 00
    00001f0 6c 01 00 00 04 00 00 00 01 00 1c 00 50 00 00 00
    0000200 04 00 00 00 00 00 00 00 00 01 00 00 2c 00 00 00
    0000210 00 00 00 00 00 00 00 00 07 00 00 00 12 00 00 00
    0000220 1b 00 00 00 04 04 61 74 74 72 00 08 08 64 72 61
    0000230 77 61 62 6c 65 00 06 06 6c 61 79 6f 75 74 00 06
    0000240 06 73 74 72 69 6e 67 00 01 00 1c 00 50 00 00 00
    0000250 04 00 00 00 00 00 00 00 00 01 00 00 2c 00 00 00
    0000260 00 00 00 00 00 00 00 00 07 00 00 00 0e 00 00 00
    0000270 16 00 00 00 04 04 69 63 6f 6e 00 04 04 6d 61 69
    0000280 6e 00 05 05 68 65 6c 6c 6f 00 08 08 61 70 70 5f
    0000290 6e 61 6d 65 00 00 00 00 02 02 10 00 10 00 00 00
    00002a0 01 00 00 00 00 00 00 00 02 02 10 00 14 00 00 00
    00002b0 02 00 00 00 01 00 00 00 00 01 00 00 01 02 34 00
    00002c0 48 00 00 00 02 00 00 00 01 00 00 00 38 00 00 00
    00002d0 20 00 00 00 00 00 00 00 00 00 00 00 00 00 78 00
    00002e0 00 00 00 00 00 00 00 00 04 00 00 00 00 00 00 00
    00002f0 00 00 00 00 08 00 00 00 00 00 00 00 08 00 00 03
    0000300 00 00 00 00 01 02 34 00 48 00 00 00 02 00 00 00
    0000310 01 00 00 00 38 00 00 00 20 00 00 00 00 00 00 00
    0000320 00 00 00 00 00 00 a0 00 00 00 00 00 00 00 00 00
    0000330 04 00 00 00 00 00 00 00 00 00 00 00 08 00 00 00
    0000340 00 00 00 00 08 00 00 03 01 00 00 00 01 02 34 00
    0000350 48 00 00 00 02 00 00 00 01 00 00 00 38 00 00 00
    0000360 20 00 00 00 00 00 00 00 00 00 00 00 00 00 f0 00
    0000370 00 00 00 00 00 00 00 00 04 00 00 00 00 00 00 00
    0000380 00 00 00 00 08 00 00 00 00 00 00 00 08 00 00 03
    0000390 02 00 00 00 02 02 10 00 14 00 00 00 03 00 00 00
    00003a0 01 00 00 00 00 00 00 00 01 02 34 00 48 00 00 00
    00003b0 03 00 00 00 01 00 00 00 38 00 00 00 20 00 00 00
    00003c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    00003d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    00003e0 08 00 00 00 01 00 00 00 08 00 00 03 03 00 00 00
    00003f0 02 02 10 00 18 00 00 00 04 00 00 00 02 00 00 00
    0000400 00 00 00 00 00 00 00 00 01 02 34 00 5c 00 00 00
    0000410 04 00 00 00 02 00 00 00 3c 00 00 00 20 00 00 00
    0000420 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0000430 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0000440 10 00 00 00 08 00 00 00 02 00 00 00 08 00 00 03
    0000450 04 00 00 00 08 00 00 00 03 00 00 00 08 00 00 03
    0000460 05 00 00 00                                    
    0000464

The bytes in blue are the TypeSpec chunk headers, the bytes in green the TypeSpec chunk bodies.

Note that the first TypeSpec chunk starting at 0x0000298 does not have a body and is immediately followed by a second TypeSpec chunk starting at
0x00002a8.

2.0 The TypeSpec Chunk Header

The format of a TypeSpec chunk header is defined by another slightly misleadingly named C++ struct (see frameworks/base/include/ResourceTypes.h lines 1532-1553)

    struct ResTable_typeSpec
    {
        struct ResChunk_header header;

        // The type identifier this chunk is holding.  Type IDs start
        // at 1 (corresponding to the value of the type bits in a
        // resource identifier).  0 is invalid.
        uint8_t id;
    
        // Must be 0.
        uint8_t res0;
        // Must be 0.
        uint16_t res1;
    
        // Number of uint32_t entry configuration masks that follow.
        uint32_t entryCount;

        enum {
            // Additional flag indicating an entry is public.
            SPEC_PUBLIC = 0x40000000
        };
    };

2.1 header

The header field is a struct ResChunk_header instance.

The header.type field is always 0x0202 (RES_TABLE_TYPE_SPEC_TYPE).

The header.headerSize field is always 0x0010.

2.2 id

The id field specifies the numeric id of the Resource type represented by this chunk.

The id uniquely identifies the represented Resource type within the containing Package chunk.

The id also specifies the name of the Resource type. It is the string at index id - 1 in the typeStrings StringPool chunk in the containing Package chunk.

2.3 entryCount

The entryCount field specifies the number of entries in the body of this chunk.

3.0 The TypeSpec Chunk Body

The body of a TypeSpec chunk contains entryCount 32-bit integers, one for each Resource of this type that has been defined in the Package represented by the containing Package chunk.

The value for each Resource specifies zero or more bit-flags.

One possible flag is the value of the anonyous enum member SPEC_PUBLIC which appears in the definition of struct ResTable_typeSpec.

If this flag is present the associated Resource is not visible and hence cannot be referenced from other Packages.

The other flags which may be set are defined by the following anonymous C++ enum (see frameworks/base/include/ResourceTypes.h lines 1030-1044) which has been re-formatted slightly for clarity.

    enum {
        CONFIG_MCC             = ACONFIGURATION_MCC,
        CONFIG_MNC             = ACONFIGURATION_MCC,
        CONFIG_LOCALE          = ACONFIGURATION_LOCALE,
        CONFIG_TOUCHSCREEN     = ACONFIGURATION_TOUCHSCREEN,
        CONFIG_KEYBOARD        = ACONFIGURATION_KEYBOARD,
        CONFIG_KEYBOARD_HIDDEN = ACONFIGURATION_KEYBOARD_HIDDEN,
        CONFIG_NAVIGATION      = ACONFIGURATION_NAVIGATION,
        CONFIG_ORIENTATION     = ACONFIGURATION_ORIENTATION,
        CONFIG_DENSITY         = ACONFIGURATION_DENSITY,
        CONFIG_SCREEN_SIZE     = ACONFIGURATION_SCREEN_SIZE,
        CONFIG_VERSION         = ACONFIGURATION_VERSION,
        CONFIG_SCREEN_LAYOUT   = ACONFIGURATION_SCREEN_LAYOUT,
        CONFIG_UI_MODE         = ACONFIGURATION_UI_MODE
    };

The constants being referenced in this definition can be found in the file

    frameworks/base/native/include/android/configuration.h

Each member of the enum identifies a specific characteristic of a device which may take on distinct values and for which a single Resource may define correspondingly distinct values.

If one of the flags defined by the enum is set in the entry for a Resource in the Typespec chunk then that Resource has different values corresponding to possible different values of the specific aspect of the configuration identified by that flag.

The most familiar example is probably screen density, specified here by the constant CONFIG_DENSITY. By default an Android application is created with distinct icons for low, high and medium, density screens. These are three values of the single drawable Resource icon.

4.0 The Example Annotated

4.1 Typespec Chunk For A Single Resource With Multiple Values

    ...

    000002a8 02 02       // type [TYPE_SPEC]
    000002aa 10 00       // header size
    000002ac 14 00 00 00 // chunk size
    --------------------

    000002b0 02          // id
    000002b1 00          // 0
    000002b2 00 00       // id
    000002b4 01 00 00 00 // entryCount
    ++++++++++++++++++++

    000002b8 00 01 00 00 // entry_flags[0]
    ==================== [End of TYPE_SPEC]
	
    ...

The Typespec chunk above defines the entries for the drawable Resources in the example Package.

The chunk contains a single entry for the icon Resource.

The entry has the value 0x0100 since the CONFIG_DENSITY flag is set as expected.

4.1 Typespec Chunk For Resource Type With Multiple Resources

    ...

    000003f0 02 02       // type [TYPE_SPEC]
    000003f2 10 00       // header size
    000003f4 18 00 00 00 // chunk size
    --------------------

    000003f8 04          // id
    000003f9 00          // 0
    000003fa 00 00       // id
    000003fc 02 00 00 00 // entryCount
    ++++++++++++++++++++

    00000400 00 00 00 00 // entry_flags[0]
    00000404 00 00 00 00 // entry_flags[1]
    ==================== [End of TYPE_SPEC]
	
    ...

The Typespec chunk above defines the entries for the string Resources in the example Package.

The chunk contains entries for the app_name and hello Resources.

Both entries are zero since neither Resource is public and both Resources only have a value defined for the default configuration so none of the configuration related flags are set.


Copyright (c) 2011 By Simon Lewis. All Rights Reserved.

Create a free website or blog at WordPress.com.