Just An Application

September 30, 2014

Building The Android Runtime (ART) For Mac OS X: Part Eight — Testing, Testing

Having got the Android Runtime to build, eventually, does it actually work ?

Running art the immediate answer is no,

    $art
    E/JniInvocation(46765): Failed to dlopen libart.so: dlopen(libart.so, 2): image not found
    Failed to initialize JNI invocation API from libart.so

The art executable is a script which contains the following so it looks as though this problem is easily fixed.

    ...
    
    invoke_with=
    DALVIKVM=dalvikvm
    LIBART=libart.so
    
    ...

Trying again after modifying the art script appropriately we get

    $art
    art I 46796 66656 art/runtime/gc/space/image_space.cc:269] RelocateImage: \
        /Users/simon/Scratch/art/out/host/darwin-x86/bin/../bin/patchoat \
            --input-image-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.art \
            --output-image-file=/Users/simon/Scratch/xper/art/android-data46788/dalvik-cache/x86_64/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.art \
            --input-oat-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.oat \
            --output-oat-file=/Users/simon/Scratch/xper/art/android-data46788/dalvik-cache/x86_64/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.oat \
            --instruction-set=x86_64 --base-offset-delta=9420800
    patchoat W 46797 66657 art/runtime/gc/heap.cc:216] \
        Could not create image space with image file '/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.art'. \
        Attempting to fall back to imageless running. \
        Error was: Failed to load /system image '/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/x86_64/core.art': \
            Failed to mmap at expected address, mapped at 0x100b06000 instead of 0x60000000

and then it hangs/spins.

The address

    0x100b06000

in the error message from patchoat shows that it is a 64-bit executable, so it needs to be rebuilt as a 32-bit executable.

Modifying the file

    art/patchoat/Android.mk

in the same way as we did for dex2oat, rebuilding and trying again, remembering to specify that we want to use a 32-bit VM, we get

    $art --32
    art I 24835 34536 art/runtime/gc/space/image_space.cc:269] RelocateImage: \
        /Users/simon/Scratch/art/out/host/darwin-x86/bin/../bin/patchoat \
            --input-image-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.art \
            --output-image-file=/Users/simon/Scratch/xper/art/android-data24827/dalvik-cache/x86/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.art \
            --input-oat-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.oat \
            --output-oat-file=/Users/simon/Scratch/xper/art/android-data24827/dalvik-cache/x86/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.oat \
            --instruction-set=x86 \
            --base-offset-delta=-11399168
    Class name required

which is definitely an improvement.

Trying again with

   -help

to find out how to actually run something we get a long list of arguments we can use and an equally long list of arguments we cannot.

    $art --32 -help
    dalvikvm: [options] class [argument ...]

    The following standard options are supported:
      -classpath classpath (-cp classpath)
      -Dproperty=value
      -verbose:tag ('gc', 'jni', or 'class')
      -showversion
      -help
      -agentlib:jdwp=options

    The following extended options are supported:
      -Xrunjdwp:<options>
      -Xbootclasspath:bootclasspath
      -Xcheck:tag  (e.g. 'jni')
      -XmsN (min heap, must be multiple of 1K, >= 1MB)
      -XmxN (max heap, must be multiple of 1K, >= 2MB)
      -XssN (stack size)
      -Xint
    
    The following Dalvik options are supported:
      -Xzygote
      -Xjnitrace:substring (eg NativeClass or nativeMethod)
      -Xstacktracefile:<filename>
      -Xgc:[no]preverify
      -Xgc:[no]postverify
      -XX:HeapGrowthLimit=N
      -XX:HeapMinFree=N
      -XX:HeapMaxFree=N
      -XX:NonMovingSpaceCapacity=N
      -XX:HeapTargetUtilization=doublevalue
      -XX:ForegroundHeapGrowthMultiplier=doublevalue
      -XX:LowMemoryMode
      -Xprofile:{threadcpuclock,wallclock,dualclock}
        
    The following unique to ART options are supported:
      -Xgc:[no]preverify_rosalloc
      -Xgc:[no]postsweepingverify_rosalloc
      -Xgc:[no]postverify_rosalloc
      -Xgc:[no]presweepingverify
      -Ximage:filename
      -XX:+DisableExplicitGC
      -XX:ParallelGCThreads=integervalue
      -XX:ConcGCThreads=integervalue
      -XX:MaxSpinsBeforeThinLockInflation=integervalue
      -XX:LongPauseLogThreshold=integervalue
      -XX:LongGCLogThreshold=integervalue
      -XX:DumpGCPerformanceOnShutdown
      -XX:IgnoreMaxFootprint
      -XX:UseTLAB
      -XX:BackgroundGC=none
      -XX:LargeObjectSpace={disabled,map,freelist}
      -XX:LargeObjectThreshold=N
      -Xmethod-trace
      -Xmethod-trace-file:filename  -Xmethod-trace-file-size:integervalue
      -Xenable-profiler
      -Xprofile-filename:filename
      -Xprofile-period:integervalue
      -Xprofile-duration:integervalue
      -Xprofile-interval:integervalue
      -Xprofile-backoff:doublevalue
      -Xprofile-start-immediately
      -Xprofile-top-k-threshold:doublevalue
      -Xprofile-top-k-change-threshold:doublevalue
      -Xprofile-type:{method,stack}
      -Xprofile-max-stack-depth:integervalue
      -Xcompiler:filename
      -Xcompiler-option dex2oat-option
      -Ximage-compiler-option dex2oat-option
      -Xpatchoat:filename
      -X[no]relocate
      -X[no]dex2oat (Whether to invoke dex2oat on the application)
      -X[no]image-dex2oat (Whether to create and use a boot image)
        
    The following previously supported Dalvik options are ignored:
      -ea[:<package name>... |:<class name>]
      -da[:<package name>... |:<class name>]
      (-enableassertions, -disableassertions)
      -esa
      -dsa
      (-enablesystemassertions, -disablesystemassertions)
      -Xverify:{none,remote,all}
      -Xrs
      -Xint:portable, -Xint:fast, -Xint:jit
      -Xdexopt:{none,verified,all,full}
      -Xnoquithandler
      -Xjniopts:{warnonly,forcecopy}
      -Xjnigreflimit:integervalue
      -Xgc:[no]precise
      -Xgc:[no]verifycardtable
      -X[no]genregmap
      -Xverifyopt:[no]checkmon
      -Xcheckdexsum
      -Xincludeselectedop
      -Xjitop:hexopvalue[-endvalue][,hexopvalue[-endvalue]]*
      -Xincludeselectedmethod
      -Xjitthreshold:integervalue
      -Xjitcodecachesize:decimalvalueofkbytes
      -Xjitblocking
      -Xjitmethod:signature[,signature]* (eg Ljava/lang/String\;replace)
      -Xjitclass:classname[,classname]*
      -Xjitoffset:offset[,offset]
      -Xjitconfig:filename
      -Xjitcheckcg
      -Xjitverbose
      -Xjitprofile
      -Xjitdisableopt
      -Xjitsuspendpoll
      -XX:mainThreadStackSize=N

Writing a very simple ‘hello world’

    package xper.hw;
    
    public final class HelloWorldART
    {
        public static void main(String[] theArgs)
        {
            System.out.println("Hello World courtesy of the Android Runtime (ART)");
            System.exit(0);
        }
    }

and turning it to a .dex and trying again we get

    art --32 -classpath hw.dex xper.hw.HelloWorldART
    art I 24871 35716 art/runtime/gc/space/image_space.cc:269] RelocateImage: \
        /Users/simon/Scratch/art/out/host/darwin-x86/bin/../bin/patchoat \
            --input-image-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.art \
            --output-image-file=/Users/simon/Scratch/xper/art/android-data24863/dalvik-cache/x86/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.art\
            --input-oat-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.oat \
            --output-oat-file=/Users/simon/Scratch/xper/art/android-data24863/dalvik-cache/x86/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.oat --instruction-set=x86\
            --base-offset-delta=-4640768
    dex2oat I 24873 35749 art/dex2oat/dex2oat.cc:1266] \
        /Users/simon/Scratch/art/out/host/darwin-x86/bin/../bin/dex2oat \
            --runtime-arg -classpath \
            --runtime-arg hw.dex \
            --instruction-set=x86 \
            --instruction-set-features=default \
            --runtime-arg -Xrelocate
            --host \
            --boot-image=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.art \
            --dex-file=/Users/simon/Scratch/xper/art/hw.dex \
            --oat-fd=3 \
            --oat-location=/Users/simon/Scratch/xper/art/android-data24863/dalvik-cache/x86/Users@simon@Scratch@xper@art@hw.dex
    dex2oat I 24873 35749 art/dex2oat/dex2oat.cc:284] dex2oat took 246.341ms (threads: 8)
    Hello World courtesy of the Android Runtime (ART)
    art I 24871 35716 art/runtime/native/java_lang_Runtime.cc:41] System.exit called, status: 0

And there you have it. Possibly the longest and most convoluted way of getting a Java ‘hello world’ program to run short of writing your own Java runtime.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

Building The Android Runtime (ART) For Mac OS X: Part Seven — N’th Time Lucky

Obliterating the entire out directory and restarting the build for the n’th time results in the now familiar litany of compiler
warnings but eventually the final invocation of dex2oat completes

    ...
    
    
    host dex2oat: out/host/darwin-x86/framework/x86/core.art \
        (out/host/darwin-x86/framework/core-libart-hostdex.jar \
         out/host/darwin-x86/framework/conscrypt-hostdex.jar \
         out/host/darwin-x86/framework/okhttp-hostdex.jar \
         out/host/darwin-x86/framework/core-junit-hostdex.jar \
         out/host/darwin-x86/framework/bouncycastle-hostdex.jar \
         art/runtime/oat.cc \
         art/runtime/image.cc)
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1266] out/host/darwin-x86/bin/dex2oatd \
        --runtime-arg -Xms64m \
        --runtime-arg -Xmx64m \
        --image-classes=frameworks/base/preloaded-classes \
        --dex-file=out/host/common/obj/JAVA_LIBRARIES/core-libart-hostdex_intermediates/javalib.jar \
        --dex-file=out/host/common/obj/JAVA_LIBRARIES/conscrypt-hostdex_intermediates/javalib.jar \
        --dex-file=out/host/common/obj/JAVA_LIBRARIES/okhttp-hostdex_intermediates/javalib.jar \
        --dex-file=out/host/common/obj/JAVA_LIBRARIES/core-junit-hostdex_intermediates/javalib.jar \
        --dex-file=out/host/common/obj/JAVA_LIBRARIES/bouncycastle-hostdex_intermediates/javalib.jar \
        --dex-location=out/host/darwin-x86/framework/core-libart-hostdex.jar \
        --dex-location=out/host/darwin-x86/framework/conscrypt-hostdex.jar \
        --dex-location=out/host/darwin-x86/framework/okhttp-hostdex.jar \
        --dex-location=out/host/darwin-x86/framework/core-junit-hostdex.jar \
        --dex-location=out/host/darwin-x86/framework/bouncycastle-hostdex.jar \
        --oat-file=out/host/darwin-x86/framework/x86/core.oat \
        --oat-location= \
        --image=out/host/darwin-x86/framework/x86/core.art \
        --base=0x60000000 --instruction-set=x86 \
        --instruction-set-features= \
        --host \
        --android-root=out/host/darwin-x86 \
        --include-patch-information
    dex2oatd I 47906 155398 art/runtime/gc/heap.cc:2182] \
        Explicit concurrent mark sweep GC freed 15367(1476KB) AllocSpace objects, 0(0B) LOS objects, 43% free, 5MB/9MB, paused 83.421ms total 185.818ms
    dex2oatd I 47906 155398 art/runtime/gc/heap.cc:2182] \
        Explicit concurrent mark sweep GC freed 34415(2MB) AllocSpace objects, 0(0B) LOS objects, 56% free, 3MB/7MB, paused 62.147ms total 133.061ms
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511] compiler [Exclusive time] [Total time]
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]   0.579s/11.173s dex2oat Setup
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.238s LoadImageClasses
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.130s Resolve Types
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.029s Resolve MethodsAndFields
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.007s Resolve Types
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.008s Resolve MethodsAndFields
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.024s Resolve Types
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.002s Resolve MethodsAndFields
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.002s Resolve Types
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0s Resolve MethodsAndFields
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.081s Resolve Types
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.006s Resolve MethodsAndFields
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.385s Verify Dex File
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.047s Verify Dex File
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.039s Verify Dex File
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.004s Verify Dex File
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.185s Verify Dex File
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.224s InitializeNoClinit
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.022s InitializeNoClinit
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.001s InitializeNoClinit
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0s InitializeNoClinit
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.143s InitializeNoClinit
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.065s UpdateImageClasses
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     1.538s Compile Dex File
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.124s Compile Dex File
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.167s Compile Dex File
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.013s Compile Dex File
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0.654s Compile Dex File
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     0s/0.213s dex2oat OatWriter
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]       0s InitOatHeader
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]       0s InitOatDexFiles
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]       0s InitDexFiles
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]       0.022s InitOatClasses
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]       0.028s InitOatMaps
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]       0s InitOatCode
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]       0.162s InitOatCodeDexFiles
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     3.497s Writing ELF
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]     2.732s dex2oat ImageWriter
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511] compiler: end, 11.173s
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:1511]
    dex2oatd I 47906 155398 art/dex2oat/dex2oat.cc:284] dex2oat took 11.365s (threads: 8)
    
    \e[0;32m#### make completed successfully (01:16:12 (hh:mm:ss)) ####\e[00m

and that’s it.

Products

After all that what have we got ?

Directory: out/host/darwin-x86/bin

  • acp

  • art

  • dalvikvm -> dalvikvm64

  • dalvikvm32

  • dalvikvm64

  • dex2oat

  • dex2oatd

  • dx

  • oatdump

  • oatdumpd

  • patchoat

  • patchoatd

Directory: out/host/darwin-x86/framework/{x86,x86_64}

  • core.art

  • core.oat

Directory: out/host/darwin-x86/{lib,lib64}

  • libart-compiler.dylib

  • libart.dylib

  • llibartd-compiler.dylib

  • llibartd.dylib

  • libbacktrace_libc++.dylib

  • libc++.dylib

  • libcrypto-host.dylib

  • libexpat-host.dylib

  • libicui18n-host.dylib

  • libicuuc-host.dylib

  • libjavacore.dylib

  • liblog.dylib

  • libnativebridge.dylib

  • libnativehelper.dylib

  • libsigchain.dylib

  • libz-host.dylib


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

September 29, 2014

Building The Android Runtime (ART) For Mac OS X: Part Six — Once More With Feeling

Removing the dex2oat and dex2oatd executables from the directory

    out/host/darwin-x86/bin

and the object files from

    out/host/darwin-x86/obj/EXECUTABLES/dex2oat_intermediates

and

    out/host/darwin-x86/obj/EXECUTABLES/dex2oatd_intermediates

and restarting the build results in dex2oat getting rebuilt as a 32-bit executable.

As expected, at the point dex2oat is first invoked in the build it no longer spins, instead, and not as expected, it crashes.

    ...

    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299] *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299] Fatal signal 10 (SIGBUS), code 2 (BUS_ADRERR) fault addr 0x2fb0ed6
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299] OS: Darwin 13.4.0 (x86_64)
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299] Cmdline: out/host/darwin-x86/bin/dex2oatd ...
    
    ...
    
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299] Thread: 72220 "<unknown>"
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299] Registers:
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299]     eax: 0x7eadf38a    ebx: 0x7da400e4    ecx: 0x00000076    edx: 0x00000b10
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299]     edi: 0x00000012    esi: 0x02fb0ed6    ebp: 0xbff54578    esp: 0xbff5456c
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299]     eip: 0x002db0e1                    eflags: 0x00010202 [ IF ]
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299]      cs: 0x0000001b     ds: 0x00000023     es: 0x00000023     fs: 0x0000001f
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299]      gs: 0x0000000f     ss: 0x00000023
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299] Backtrace:
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299]
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:313] Fault message:
    make: *** [out/host/darwin-x86/framework/x86_64/core.art] Error 1

Following the way of the printf once more reveals that the crash occurs in this function

File: $(ANDROID_SRC)/art/runtime/mem_map.cc

    ...
    
    static bool ContainedWithinExistingMap(uintptr_t begin,
                                           uintptr_t end,
                                           std::string* error_msg) {
      std::unique_ptr<BacktraceMap> map(BacktraceMap::Create(getpid(), true));
      if (map.get() == nullptr) {
        *error_msg = StringPrintf("Failed to build process map");
        return false;
      }
      for (BacktraceMap::const_iterator it = map->begin(); it != map->end(); ++it) {
        if ((begin >= it->start && begin < it->end)  // start of new within old
            && (end > it->start && end <= it->end)) {  // end of new within old
          return true;
        }
      }
      std::string maps;
      ReadFileToString("/proc/self/maps", &maps);
      *error_msg = StringPrintf("Requested region 0x%08" PRIxPTR "-0x%08" PRIxPTR " does not overlap "
                                "any existing map:\n%s\n",
                                begin, end, maps.c_str());
      return false;
    }
    
    ...

While this is clearly a very Mac OS X unfriendly function, it crashes a bit before the point where it might attempt to read the non-existent ‘file’

    /proc/self/maps

In fact it crashes at the point at which the for loop initialization is doing

      BacktraceMap::const_iterator it = map->begin();

The problem looks as though it might involve death by some combination of temporary variable/assignment operator/copy constructor C++ voodoo and the compiler getting horribly confused or something.

Various attempts at re-factoring to induce some non-crashing combination of temporary variable/assignment operator/copy constructor C++ voodoo all failed, so I tried this

    ...
    
    static bool ContainedWithinExistingMap(uintptr_t begin,
                                           uintptr_t end,
                                           std::string* error_msg) {
      std::unique_ptr<BacktraceMap> map(BacktraceMap::Create(getpid(), true));
      if (map.get() == nullptr) {
        *error_msg = StringPrintf("Failed to build process map");
        return false;
      }
    
      const backtrace_map_t* entry = map->Find(begin);
    
      if ((entry != NULL) && (end <= entry->end)) {
        return true;
      }

      std::string maps;
      ReadFileToString("/proc/self/maps", &maps);
      *error_msg = StringPrintf("Requested region 0x%08" PRIxPTR "-0x%08" PRIxPTR " does not overlap "
                                "any existing map:\n%s\n",
                                begin, end, maps.c_str());
      return false;
    }
    
    ...

which works.

Quite why it works I don’t know, since the BacktraceMap::Find method is defined like this

File: $(ANDROID_SRC)/system/core/libbacktrace/BacktraceMap.cpp

    ...
    
    const backtrace_map_t* BacktraceMap::Find(uintptr_t addr) {
      for (BacktraceMap::const_iterator it = begin();
           it != end(); ++it) {
        if (addr >= it->start && addr < it->end) {
          return &*it;
        }
      }
      return NULL;
    }
    
    ...

It too is using an iterator in exactly the same way. It just doesn’t crash when it does so.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

September 28, 2014

Building The Android Runtime (ART) For Mac OS X: Part Five — 64 Bits Considered 32 Bits Too Many

As we have seen, if mmap is not passsed the

    MAP_FIXED

flag, then, if the request is for memory somewhere in the low 4GB of the address space although the call succeeds the resulting memory is not in the low 4GB at all.

Setting the MAP_FIXED flag does not help. The call to mmap simply fails.

In short, the problem is that Mac OS X will not allocate memory in the low 4GB of the address space of a 64-bit executable, irrespective of the start address.

The fact that the low_4gb argument to the MemMap::MapAnonymous method is hard-wired to true when called from the Heap::MapAnonymousPreferredAddress method implies that this is an absolute requirement in this case. A possible implication of this is that the some or all of the code is written on the assumption that addresses are necessarily 32 bits so relaxing the constraint is probably not a good idea.

A quick check shows that other calls MemMap::MapAnonymous also pass true as the low_4gb argument, so those calls would also have to be changed as well.

Fortunately there is a workaround of sorts.

One way to get the OS to allocate memory in the first 4GB of the address space of a process is to ensure that the process only has a 4GB address space, i.e, turn dex2oat into a 32-bit executable.

There is probably a ‘proper’ way to do this, but the brute force way is to change this

    ...
        
    ifeq ($(ART_BUILD_HOST_NDEBUG),true)
      $(eval $(call build-art-executable,dex2oat,$(DEX2OAT_SRC_FILES),libart-compiler,art/compiler,host,ndebug))
    endif
    ifeq ($(ART_BUILD_HOST_DEBUG),true)
      $(eval $(call build-art-executable,dex2oat,$(DEX2OAT_SRC_FILES),libartd-compiler,art/compiler,host,debug))
    endif
        
    ...

in the file

    $(ANDROID_SRC)/art/dex2oat/Android.mk

to this

    ...
        
    ifeq ($(ART_BUILD_HOST_NDEBUG),true)
      $(eval $(call build-art-executable,dex2oat,$(DEX2OAT_SRC_FILES),libart-compiler,art/compiler,host,ndebug, 32))
    endif
    ifeq ($(ART_BUILD_HOST_DEBUG),true)
      $(eval $(call build-art-executable,dex2oat,$(DEX2OAT_SRC_FILES),libartd-compiler,art/compiler,host,debug, 32))
    endif
        
    ...

Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

September 26, 2014

Building The Android Runtime (ART) For Mac OS X: Part Three — Let’s Build ! Again !

Restarting the build after the modification to $(ANDROID_SRC)/art/Android.mk results in another unhealthy number of compiler warnings followed, eventually, by this (reformatted for clarity)

    ...
    
    dex2oatd I  2426 823971 art/dex2oat/dex2oat.cc:1266] out/host/darwin-x86/bin/dex2oatd \
       --runtime-arg -Xms64m \
       --runtime-arg -Xmx64m \
       --image-classes=frameworks/base/preloaded-classes \
       --dex-file=out/host/common/obj/JAVA_LIBRARIES/core-libart-hostdex_intermediates/javalib.jar \
       --dex-file=out/host/common/obj/JAVA_LIBRARIES/conscrypt-hostdex_intermediates/javalib.jar \
       --dex-file=out/host/common/obj/JAVA_LIBRARIES/okhttp-hostdex_intermediates/javalib.jar \
       --dex-file=out/host/common/obj/JAVA_LIBRARIES/core-junit-hostdex_intermediates/javalib.jar \
       --dex-file=out/host/common/obj/JAVA_LIBRARIES/bouncycastle-hostdex_intermediates/javalib.jar \
       --dex-location=out/host/darwin-x86/framework/core-libart-hostdex.jar \
       --dex-location=out/host/darwin-x86/framework/conscrypt-hostdex.jar \
       --dex-location=out/host/darwin-x86/framework/okhttp-hostdex.jar \
       --dex-location=out/host/darwin-x86/framework/core-junit-hostdex.jar \
       --dex-location=out/host/darwin-x86/framework/bouncycastle-hostdex.jar \
       --oat-file=out/host/darwin-x86/framework/x86_64/core.oat \
       --oat-location= \
       --image=out/host/darwin-x86/framework/x86_64/core.art \
       --base=0x60000000 \
       --instruction-set=x86_64 \
       --instruction-set-features= \
       --host \
       --android-root=out/host/darwin-x86 \
       --include-patch-information

It all goes ominously quiet until you get this

    dex2oatd W 2426 823972 dex2oat did not finish after 300 seconds

and then, after an interminable wait, you get this

    dex2oatd F 2426 823972 dex2oat did not finish after 1800 seconds
    make: *** [out/host/darwin-x86/framework/x86_64/core.art] Error 1

and everything stops.

Obviously dex2oatd is very busy doing something or other and it never gets a chance to finish, except that if you look at the process stats during the interminable wait for something to happen, it isn’t. The CPU load is very light and nothing else is happening.

It looks in fact as though dex2oatd is simply spinning and doing no work at all.

Using the arcane debugging technique known to those of us who worked on the OS formerly known as S*****n as The Way Of The printf reveals that the dex2oatd process appears to be sat in the method

    MemMap::MapAnonymous

which has been called from the method

    Heap::MapAnonymousPreferredAddress

which in turn has been called from the constructor

    Heap::Heap

which in turn has been called from

    Runtime::Init

In short, dex2oatd hasn’t even started, its just spent 1800 seconds attempting to initialize its Runtime.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

Create a free website or blog at WordPress.com.

%d bloggers like this: