Just An Application

September 30, 2014

Building The Android Runtime (ART) For Mac OS X: Part Eight — Testing, Testing

Having got the Android Runtime to build, eventually, does it actually work ?

Running art the immediate answer is no,

    $art
    E/JniInvocation(46765): Failed to dlopen libart.so: dlopen(libart.so, 2): image not found
    Failed to initialize JNI invocation API from libart.so

The art executable is a script which contains the following so it looks as though this problem is easily fixed.

    ...
    
    invoke_with=
    DALVIKVM=dalvikvm
    LIBART=libart.so
    
    ...

Trying again after modifying the art script appropriately we get

    $art
    art I 46796 66656 art/runtime/gc/space/image_space.cc:269] RelocateImage: \
        /Users/simon/Scratch/art/out/host/darwin-x86/bin/../bin/patchoat \
            --input-image-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.art \
            --output-image-file=/Users/simon/Scratch/xper/art/android-data46788/dalvik-cache/x86_64/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.art \
            --input-oat-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.oat \
            --output-oat-file=/Users/simon/Scratch/xper/art/android-data46788/dalvik-cache/x86_64/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.oat \
            --instruction-set=x86_64 --base-offset-delta=9420800
    patchoat W 46797 66657 art/runtime/gc/heap.cc:216] \
        Could not create image space with image file '/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.art'. \
        Attempting to fall back to imageless running. \
        Error was: Failed to load /system image '/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/x86_64/core.art': \
            Failed to mmap at expected address, mapped at 0x100b06000 instead of 0x60000000

and then it hangs/spins.

The address

    0x100b06000

in the error message from patchoat shows that it is a 64-bit executable, so it needs to be rebuilt as a 32-bit executable.

Modifying the file

    art/patchoat/Android.mk

in the same way as we did for dex2oat, rebuilding and trying again, remembering to specify that we want to use a 32-bit VM, we get

    $art --32
    art I 24835 34536 art/runtime/gc/space/image_space.cc:269] RelocateImage: \
        /Users/simon/Scratch/art/out/host/darwin-x86/bin/../bin/patchoat \
            --input-image-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.art \
            --output-image-file=/Users/simon/Scratch/xper/art/android-data24827/dalvik-cache/x86/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.art \
            --input-oat-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.oat \
            --output-oat-file=/Users/simon/Scratch/xper/art/android-data24827/dalvik-cache/x86/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.oat \
            --instruction-set=x86 \
            --base-offset-delta=-11399168
    Class name required

which is definitely an improvement.

Trying again with

   -help

to find out how to actually run something we get a long list of arguments we can use and an equally long list of arguments we cannot.

    $art --32 -help
    dalvikvm: [options] class [argument ...]

    The following standard options are supported:
      -classpath classpath (-cp classpath)
      -Dproperty=value
      -verbose:tag ('gc', 'jni', or 'class')
      -showversion
      -help
      -agentlib:jdwp=options

    The following extended options are supported:
      -Xrunjdwp:<options>
      -Xbootclasspath:bootclasspath
      -Xcheck:tag  (e.g. 'jni')
      -XmsN (min heap, must be multiple of 1K, >= 1MB)
      -XmxN (max heap, must be multiple of 1K, >= 2MB)
      -XssN (stack size)
      -Xint
    
    The following Dalvik options are supported:
      -Xzygote
      -Xjnitrace:substring (eg NativeClass or nativeMethod)
      -Xstacktracefile:<filename>
      -Xgc:[no]preverify
      -Xgc:[no]postverify
      -XX:HeapGrowthLimit=N
      -XX:HeapMinFree=N
      -XX:HeapMaxFree=N
      -XX:NonMovingSpaceCapacity=N
      -XX:HeapTargetUtilization=doublevalue
      -XX:ForegroundHeapGrowthMultiplier=doublevalue
      -XX:LowMemoryMode
      -Xprofile:{threadcpuclock,wallclock,dualclock}
        
    The following unique to ART options are supported:
      -Xgc:[no]preverify_rosalloc
      -Xgc:[no]postsweepingverify_rosalloc
      -Xgc:[no]postverify_rosalloc
      -Xgc:[no]presweepingverify
      -Ximage:filename
      -XX:+DisableExplicitGC
      -XX:ParallelGCThreads=integervalue
      -XX:ConcGCThreads=integervalue
      -XX:MaxSpinsBeforeThinLockInflation=integervalue
      -XX:LongPauseLogThreshold=integervalue
      -XX:LongGCLogThreshold=integervalue
      -XX:DumpGCPerformanceOnShutdown
      -XX:IgnoreMaxFootprint
      -XX:UseTLAB
      -XX:BackgroundGC=none
      -XX:LargeObjectSpace={disabled,map,freelist}
      -XX:LargeObjectThreshold=N
      -Xmethod-trace
      -Xmethod-trace-file:filename  -Xmethod-trace-file-size:integervalue
      -Xenable-profiler
      -Xprofile-filename:filename
      -Xprofile-period:integervalue
      -Xprofile-duration:integervalue
      -Xprofile-interval:integervalue
      -Xprofile-backoff:doublevalue
      -Xprofile-start-immediately
      -Xprofile-top-k-threshold:doublevalue
      -Xprofile-top-k-change-threshold:doublevalue
      -Xprofile-type:{method,stack}
      -Xprofile-max-stack-depth:integervalue
      -Xcompiler:filename
      -Xcompiler-option dex2oat-option
      -Ximage-compiler-option dex2oat-option
      -Xpatchoat:filename
      -X[no]relocate
      -X[no]dex2oat (Whether to invoke dex2oat on the application)
      -X[no]image-dex2oat (Whether to create and use a boot image)
        
    The following previously supported Dalvik options are ignored:
      -ea[:<package name>... |:<class name>]
      -da[:<package name>... |:<class name>]
      (-enableassertions, -disableassertions)
      -esa
      -dsa
      (-enablesystemassertions, -disablesystemassertions)
      -Xverify:{none,remote,all}
      -Xrs
      -Xint:portable, -Xint:fast, -Xint:jit
      -Xdexopt:{none,verified,all,full}
      -Xnoquithandler
      -Xjniopts:{warnonly,forcecopy}
      -Xjnigreflimit:integervalue
      -Xgc:[no]precise
      -Xgc:[no]verifycardtable
      -X[no]genregmap
      -Xverifyopt:[no]checkmon
      -Xcheckdexsum
      -Xincludeselectedop
      -Xjitop:hexopvalue[-endvalue][,hexopvalue[-endvalue]]*
      -Xincludeselectedmethod
      -Xjitthreshold:integervalue
      -Xjitcodecachesize:decimalvalueofkbytes
      -Xjitblocking
      -Xjitmethod:signature[,signature]* (eg Ljava/lang/String\;replace)
      -Xjitclass:classname[,classname]*
      -Xjitoffset:offset[,offset]
      -Xjitconfig:filename
      -Xjitcheckcg
      -Xjitverbose
      -Xjitprofile
      -Xjitdisableopt
      -Xjitsuspendpoll
      -XX:mainThreadStackSize=N

Writing a very simple ‘hello world’

    package xper.hw;
    
    public final class HelloWorldART
    {
        public static void main(String[] theArgs)
        {
            System.out.println("Hello World courtesy of the Android Runtime (ART)");
            System.exit(0);
        }
    }

and turning it to a .dex and trying again we get

    art --32 -classpath hw.dex xper.hw.HelloWorldART
    art I 24871 35716 art/runtime/gc/space/image_space.cc:269] RelocateImage: \
        /Users/simon/Scratch/art/out/host/darwin-x86/bin/../bin/patchoat \
            --input-image-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.art \
            --output-image-file=/Users/simon/Scratch/xper/art/android-data24863/dalvik-cache/x86/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.art\
            --input-oat-location=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.oat \
            --output-oat-file=/Users/simon/Scratch/xper/art/android-data24863/dalvik-cache/x86/Users@simon@Scratch@art@out@host@darwin-x86@bin@..@framework@core.oat --instruction-set=x86\
            --base-offset-delta=-4640768
    dex2oat I 24873 35749 art/dex2oat/dex2oat.cc:1266] \
        /Users/simon/Scratch/art/out/host/darwin-x86/bin/../bin/dex2oat \
            --runtime-arg -classpath \
            --runtime-arg hw.dex \
            --instruction-set=x86 \
            --instruction-set-features=default \
            --runtime-arg -Xrelocate
            --host \
            --boot-image=/Users/simon/Scratch/art/out/host/darwin-x86/bin/../framework/core.art \
            --dex-file=/Users/simon/Scratch/xper/art/hw.dex \
            --oat-fd=3 \
            --oat-location=/Users/simon/Scratch/xper/art/android-data24863/dalvik-cache/x86/Users@simon@Scratch@xper@art@hw.dex
    dex2oat I 24873 35749 art/dex2oat/dex2oat.cc:284] dex2oat took 246.341ms (threads: 8)
    Hello World courtesy of the Android Runtime (ART)
    art I 24871 35716 art/runtime/native/java_lang_Runtime.cc:41] System.exit called, status: 0

And there you have it. Possibly the longest and most convoluted way of getting a Java ‘hello world’ program to run short of writing your own Java runtime.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

Advertisements

September 29, 2014

Building The Android Runtime (ART) For Mac OS X: Part Six — Once More With Feeling

Removing the dex2oat and dex2oatd executables from the directory

    out/host/darwin-x86/bin

and the object files from

    out/host/darwin-x86/obj/EXECUTABLES/dex2oat_intermediates

and

    out/host/darwin-x86/obj/EXECUTABLES/dex2oatd_intermediates

and restarting the build results in dex2oat getting rebuilt as a 32-bit executable.

As expected, at the point dex2oat is first invoked in the build it no longer spins, instead, and not as expected, it crashes.

    ...

    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299] *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299] Fatal signal 10 (SIGBUS), code 2 (BUS_ADRERR) fault addr 0x2fb0ed6
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299] OS: Darwin 13.4.0 (x86_64)
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299] Cmdline: out/host/darwin-x86/bin/dex2oatd ...
    
    ...
    
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299] Thread: 72220 "<unknown>"
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299] Registers:
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299]     eax: 0x7eadf38a    ebx: 0x7da400e4    ecx: 0x00000076    edx: 0x00000b10
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299]     edi: 0x00000012    esi: 0x02fb0ed6    ebp: 0xbff54578    esp: 0xbff5456c
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299]     eip: 0x002db0e1                    eflags: 0x00010202 [ IF ]
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299]      cs: 0x0000001b     ds: 0x00000023     es: 0x00000023     fs: 0x0000001f
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299]      gs: 0x0000000f     ss: 0x00000023
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299] Backtrace:
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:299]
    dex2oatd F 24874 72220 art/runtime/runtime_linux.cc:313] Fault message:
    make: *** [out/host/darwin-x86/framework/x86_64/core.art] Error 1

Following the way of the printf once more reveals that the crash occurs in this function

File: $(ANDROID_SRC)/art/runtime/mem_map.cc

    ...
    
    static bool ContainedWithinExistingMap(uintptr_t begin,
                                           uintptr_t end,
                                           std::string* error_msg) {
      std::unique_ptr<BacktraceMap> map(BacktraceMap::Create(getpid(), true));
      if (map.get() == nullptr) {
        *error_msg = StringPrintf("Failed to build process map");
        return false;
      }
      for (BacktraceMap::const_iterator it = map->begin(); it != map->end(); ++it) {
        if ((begin >= it->start && begin < it->end)  // start of new within old
            && (end > it->start && end <= it->end)) {  // end of new within old
          return true;
        }
      }
      std::string maps;
      ReadFileToString("/proc/self/maps", &maps);
      *error_msg = StringPrintf("Requested region 0x%08" PRIxPTR "-0x%08" PRIxPTR " does not overlap "
                                "any existing map:\n%s\n",
                                begin, end, maps.c_str());
      return false;
    }
    
    ...

While this is clearly a very Mac OS X unfriendly function, it crashes a bit before the point where it might attempt to read the non-existent ‘file’

    /proc/self/maps

In fact it crashes at the point at which the for loop initialization is doing

      BacktraceMap::const_iterator it = map->begin();

The problem looks as though it might involve death by some combination of temporary variable/assignment operator/copy constructor C++ voodoo and the compiler getting horribly confused or something.

Various attempts at re-factoring to induce some non-crashing combination of temporary variable/assignment operator/copy constructor C++ voodoo all failed, so I tried this

    ...
    
    static bool ContainedWithinExistingMap(uintptr_t begin,
                                           uintptr_t end,
                                           std::string* error_msg) {
      std::unique_ptr<BacktraceMap> map(BacktraceMap::Create(getpid(), true));
      if (map.get() == nullptr) {
        *error_msg = StringPrintf("Failed to build process map");
        return false;
      }
    
      const backtrace_map_t* entry = map->Find(begin);
    
      if ((entry != NULL) && (end <= entry->end)) {
        return true;
      }

      std::string maps;
      ReadFileToString("/proc/self/maps", &maps);
      *error_msg = StringPrintf("Requested region 0x%08" PRIxPTR "-0x%08" PRIxPTR " does not overlap "
                                "any existing map:\n%s\n",
                                begin, end, maps.c_str());
      return false;
    }
    
    ...

which works.

Quite why it works I don’t know, since the BacktraceMap::Find method is defined like this

File: $(ANDROID_SRC)/system/core/libbacktrace/BacktraceMap.cpp

    ...
    
    const backtrace_map_t* BacktraceMap::Find(uintptr_t addr) {
      for (BacktraceMap::const_iterator it = begin();
           it != end(); ++it) {
        if (addr >= it->start && addr < it->end) {
          return &*it;
        }
      }
      return NULL;
    }
    
    ...

It too is using an iterator in exactly the same way. It just doesn’t crash when it does so.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

September 27, 2014

Building The Android Runtime (ART) For Mac OS X: Part Four — MemMap::MapAnonymous

File: $(ANDROID_SRC)/art/runtime/mem_map.cc

Method: MemMap::MapAnonymous

    ...
    
    MemMap* MemMap::MapAnonymous(const char* name, byte* expected_ptr, size_t byte_count, int prot,
      bool low_4gb, std::string* error_msg) {
      if (byte_count == 0) {
        return new MemMap(name, nullptr, 0, nullptr, 0, prot, false);
      }
      size_t page_aligned_byte_count = RoundUp(byte_count, kPageSize);
    
      int flags = MAP_PRIVATE | MAP_ANONYMOUS;
      ScopedFd fd(-1);
    
    #ifdef USE_ASHMEM
    #ifdef HAVE_ANDROID_OS
      const bool use_ashmem = true;
    #else
      // When not on Android ashmem is faked using files in /tmp. Ensure that such files won't
      // fail due to ulimit restrictions. If they will then use a regular mmap.
      struct rlimit rlimit_fsize;
      CHECK_EQ(getrlimit(RLIMIT_FSIZE, &rlimit_fsize), 0);
      const bool use_ashmem = (rlimit_fsize.rlim_cur == RLIM_INFINITY) ||
         (page_aligned_byte_count < rlimit_fsize.rlim_cur);
    #endif
      if (use_ashmem) {
        // android_os_Debug.cpp read_mapinfo assumes all ashmem regions associated with the VM are
        // prefixed "dalvik-".
        std::string debug_friendly_name("dalvik-");
        debug_friendly_name += name;
        fd.reset(ashmem_create_region(debug_friendly_name.c_str(), page_aligned_byte_count));
        if (fd.get() == -1) {
          *error_msg = StringPrintf("ashmem_create_region failed for '%s': %s", name, strerror(errno));
          return nullptr;
        }
        flags = MAP_PRIVATE;
      }
    #endif
    
      // We need to store and potentially set an error number for pretty printing of errors
      int saved_errno = 0;
    
    #ifdef __LP64__
      // When requesting low_4g memory and having an expectation, the requested range should fit into
      // 4GB.
      if (low_4gb && (
          // Start out of bounds.
          (reinterpret_cast<uintptr_t>(expected_ptr) >> 32) != 0 ||
          // End out of bounds. For simplicity, this will fail for the last page of memory.
          (reinterpret_cast<uintptr_t>(expected_ptr + page_aligned_byte_count) >> 32) != 0)) {
        *error_msg = StringPrintf("The requested address space (%p, %p) cannot fit in low_4gb",
        expected_ptr, expected_ptr + page_aligned_byte_count);
        return nullptr;
      }
    #endif
    
      // TODO:
      // A page allocator would be a useful abstraction here, as
      // 1) It is doubtful that MAP_32BIT on x86_64 is doing the right job for us
      // 2) The linear scheme, even with simple saving of the last known position, is very crude
    #if USE_ART_LOW_4G_ALLOCATOR
      // MAP_32BIT only available on x86_64.
      void* actual = MAP_FAILED;
      if (low_4gb && expected_ptr == nullptr) {
        bool first_run = true;
    
        for (uintptr_t ptr = next_mem_pos_; ptr < 4 * GB; ptr += kPageSize) {
          if (4U * GB - ptr < page_aligned_byte_count) {
            // Not enough memory until 4GB.
            if (first_run) {
              // Try another time from the bottom;
              ptr = LOW_MEM_START - kPageSize;
              first_run = false;
              continue;
            } else {
              // Second try failed.
              break;
            }
          }
    
          uintptr_t tail_ptr;
    
          // Check pages are free.
          bool safe = true;
          for (tail_ptr = ptr; tail_ptr < ptr + page_aligned_byte_count; tail_ptr += kPageSize) {
            if (msync(reinterpret_cast<void*>(tail_ptr), kPageSize, 0) == 0) {
              safe = false;
              break;
            } else {
              DCHECK_EQ(errno, ENOMEM);
            }
          }
    
          next_mem_pos_ = tail_ptr;  // update early, as we break out when we found and mapped a region
    
          if (safe == true) {
            actual = mmap(reinterpret_cast<void*>(ptr), page_aligned_byte_count, prot, flags, fd.get(),
                          0);
            if (actual != MAP_FAILED) {
              // Since we didn't use MAP_FIXED the kernel may have mapped it somewhere not in the low
              // 4GB. If this is the case, unmap and retry.
              if (reinterpret_cast<uintptr_t>(actual) + page_aligned_byte_count < 4 * GB) {
                break;
              } else {
                munmap(actual, page_aligned_byte_count);
                actual = MAP_FAILED;
              }
            }
          } else {
            // Skip over last page.
            ptr = tail_ptr;
          }
        }
    
        if (actual == MAP_FAILED) {
          LOG(ERROR) << "Could not find contiguous low-memory space.";
          saved_errno = ENOMEM;
        }
      } else {
        actual = mmap(expected_ptr, page_aligned_byte_count, prot, flags, fd.get(), 0);
        saved_errno = errno;
      }
    
    #else
    #if defined(__LP64__)
      if (low_4gb && expected_ptr == nullptr) {
        flags |= MAP_32BIT;
      }
    #endif
    
      void* actual = mmap(expected_ptr, page_aligned_byte_count, prot, flags, fd.get(), 0);
      saved_errno = errno;
    #endif
    
      if (actual == MAP_FAILED) {
        std::string maps;
        ReadFileToString("/proc/self/maps", &maps);
    
        *error_msg = StringPrintf("Failed anonymous mmap(%p, %zd, 0x%x, 0x%x, %d, 0): %s\n%s",
                                  expected_ptr, page_aligned_byte_count, prot, flags, fd.get(),
                                  strerror(saved_errno), maps.c_str());
        return nullptr;
      }
      std::ostringstream check_map_request_error_msg;
      if (!CheckMapRequest(expected_ptr, actual, page_aligned_byte_count, error_msg)) {
        return nullptr;
      }
      return new MemMap(name, reinterpret_cast<byte*>(actual), byte_count, actual,
                        page_aligned_byte_count, prot, false);
    }
    
    ...

Although it is not entirely obvious, amongst the welter of #ifdefs and assorted detritus, there is a for loop hiding in the method.

It is only compiled in if

    USE_ART_LOW_4G_ALLOCATOR

is defined and not 0.

The definition, or otherwise, of USE_ART_LOW_4G_ALLOCATOR occurs in the file

    art/runtime/mem_map.h

and is itself conditional

    ...
    
    #if defined(__LP64__) && (!defined(__x86_64__) || defined(__APPLE__))
    #define USE_ART_LOW_4G_ALLOCATOR 1
    #else
    #define USE_ART_LOW_4G_ALLOCATOR 0
    #endif
    
    ...

so on a 64-bit Mac it will be defined as 1, so the loop is present,

The for loop is inside the if statement

      ...
    
      if (low_4gb && expected_ptr == nullptr) {
      
        ...
        
      }
    
      ...

where

    low_4gb

and

    expected_ptr

are both arguments to the method.

Looking at the caller in this case

File: $(ANDROID_SRC)/art/runtimegc/heap.cc

Method: Heap::MapAnonymousPreferredAddress

    ...
    
    MemMap* Heap::MapAnonymousPreferredAddress(const char* name, byte* request_begin, size_t capacity,
                                               int prot_flags, std::string* out_error_str) {
      while (true) {
        MemMap* map = MemMap::MapAnonymous(kMemMapSpaceName[0], request_begin, capacity,
                                           PROT_READ | PROT_WRITE, true, out_error_str);
        if (map != nullptr || request_begin == nullptr) {
          return map;
        }
        // Retry a  second time with no specified request begin.
        request_begin = nullptr;
      }
      return nullptr;
    }
    
    ...

we can see that the

    low_4gb

argument is always true, and the

    expected_ptr

argument is the value of the method’s

    request_begin

argument, or

    nullptr

so one way or another the MemMap::MapAnonymous method can definitely end up in the loop.

What the method is trying to do in the loop is get the OS to allocate a continuous area of memory of a given size somewhere in the low 4GB of the process address space, starting at the 64K mark and working its way upwards a ‘page’ at a time.

The worst case outcome is that the necessary memory is not available. In this case the loop will iterate over the entire address space, so although it will terminate it will not do so for a very long time.

Here is a simplified version of the loop in the form of a standalone program.


    static constexpr int kPageSize = 4096;

    static constexpr size_t KB = 1024;
    static constexpr size_t GB = KB * KB * KB;

    static constexpr uintptr_t LOW_MEM_START = 64 * KB;

    int main(int argc, const char * argv[])
    {
        void*     actual = MAP_FAILED;
        size_t    nBytes = 16 * kPageSize;

        for (uintptr_t ptr = LOW_MEM_START; ptr < 4 * GB; ptr += kPageSize)
        {
            uintptr_t tail_ptr;
    
            // Check pages are free.
            bool safe = true;
    
            for (tail_ptr = ptr; tail_ptr < ptr + nBytes; tail_ptr += kPageSize)
            {
                if (msync(reinterpret_cast<void*>(tail_ptr), kPageSize, 0) == 0)
                {
                    safe = false;
                    break;
                }
                else
                if (errno == ENOMEM)
                {
                    // not in use
                }
                else
                {
                    // some other error
                    return 1;
                }
            }
            if (safe == true)
            {
                actual = mmap(reinterpret_cast<void*>(ptr), nBytes, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANON, -1, 0);
                if (actual != MAP_FAILED)
                {
                    fprintf(stdout, "actual == %p\n", actual);
                    // Since we didn't use MAP_FIXED the kernel may have mapped it somewhere not in the low
                    // 4GB. If this is the case, unmap and retry.
                    if (reinterpret_cast<uintptr_t>(actual) + nBytes < 4 * GB)
                    {
                        break;
                    }
                    else
                    {
                        munmap(actual, nBytes);
                        actual = MAP_FAILED;
                    }
                }
            }
            else
            {
                // Skip over last page.
                ptr = tail_ptr;
            }
        }
        return 0;
    }

and here’s what it prints when it is run

    actual == 0x10cbf5000
    actual == 0x10cbf5000
    actual == 0x10cbf5000
    actual == 0x10cbf5000
    actual == 0x10cbf5000
    actual == 0x10cbf5000
    actual == 0x10cbf5000
    actual == 0x10cbf5000
    
    ... 

effectively forever, or certainly for a lot longer than I am prepared to wait for it to iterate approximately 4G/4K times then terminate.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

September 25, 2014

Building The Android Runtime (ART) For Mac OS X: Part Two — Let’s Build !

To do the build

One

Copy the necessary Android ‘source’ to a directory ($(ANDROID_SRC)) on the case-sensitive file system.

Two

Set the environment variable

    DEVELOPER_DIR

to the path to the Developer directory within the version of the Xcode.app to use to do the build, for example

    export DEVELOPER_DIR=/Users/simon/Dev/xcode/Xcode.app/Contents/Developer

Three

Set the environment variable JAVA_HOME to the Home directory of the version of JDK 7 to use to do the build, for example

    export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home

Four

Change to the directory containing the Android source and ‘source’ the file build/envsetup.sh to set up the Android build environment

    . build/envsetup.sh

Five

Start the build

    m build-art-host

What Happens Next ?

Eventually, after an unhealthy number of compiler warnings, this happens

    make: *** No rule to make target `out/host/darwin-x86/lib/libjavacore.so', needed by `build-art-host'.  Stop.

A quick check reveals that there is a

    libjavacore.dylib

in

    out/host/darwin-x86/lib

but not a

    libjavacore.so

which is what you would expect on Mac OS X.

Searching for references to libjavacore.so in the art build system turns up

    ...
    
    # ART_HOST_DEPENDENCIES depends on Android.executable.mk above for ART_HOST_EXECUTABLES
    ART_HOST_DEPENDENCIES := \
	    $(ART_HOST_EXECUTABLES) \
	    $(HOST_OUT_JAVA_LIBRARIES)/core-libart-hostdex.jar \
	    $(ART_HOST_OUT_SHARED_LIBRARIES)/libjavacore$(ART_HOST_SHLIB_EXTENSION)
    ART_TARGET_DEPENDENCIES := \
	    $(ART_TARGET_EXECUTABLES) \
	    $(TARGET_OUT_JAVA_LIBRARIES)/core-libart.jar \
	    $(TARGET_OUT_SHARED_LIBRARIES)/libjavacore.so
    ifdef TARGET_2ND_ARCH
    ART_TARGET_DEPENDENCIES += $(2ND_TARGET_OUT_SHARED_LIBRARIES)/libjavacore.so
    endif
    ifdef HOST_2ND_ARCH
    ART_HOST_DEPENDENCIES += $(2ND_HOST_OUT_SHARED_LIBRARIES)/libjavacore.so
    endif
    
    ...

in the file

    $(ANDROID_SRC)/art/Android.mk

It looks as though

    ...
    
    ifdef HOST_2ND_ARCH
    ART_HOST_DEPENDENCIES += $(2ND_HOST_OUT_SHARED_LIBRARIES)/libjavacore.so
    endif
    
    ...

should probably be

    ...
    
    ifdef HOST_2ND_ARCH
    ART_HOST_DEPENDENCIES += $(2ND_HOST_OUT_SHARED_LIBRARIES)/libjavacore$(ART_HOST_SHLIB_EXTENSION)
    endif
    
    ...

Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

Building The Android Runtime (ART) For Mac OS X: Part One — Prerequisites

Mac OS X

I am using Mac OS X 10.9.5.

I am not sure whether the exact version of the OS matters.

Xcode And Xcode Command Line Tools

To build am using Xcode 5.1 plus the corresponding command line tools.

JDK 7

To build I am using JDK 7. The version reported by java -version is

    1.7.0_45

It has to be JDK 7.

A Case Sensitive File System

You will need a case-sensitive file system which probably isn’t what you have by default, so you will need an external drive, a USB stick or possibly a .dmg that has been formatted appropriately.

I have successfully used an external drive and a USB stick, both formatted as

    Mac OS Extended(Case-sensitive, Journaled)

I haven’t tried the .dmg approach.

Some Android Source

I am using the the following specific subset of the Android ‘source’ all obtained within the last couple of days courtesy
of repo sync.

  • abi

  • art

  • bionic

  • build

  • dalvik

  • external/bouncycastle

  • external/conscrypt

  • external/expat

  • external/fdlibm

  • external/gtest

  • external/icu

  • external/junit

  • external/libcxx

  • external/libcxxabi

  • external/libphonenumber

  • external/llvm

  • external/okhttp

  • external/openssl

  • external/stlport

  • external/valgrind

  • external/vixl

  • external/zlib

  • frameworks/base

  • frameworks/compile/mclinker

  • frameworks/native

  • libcore

  • libnativehelper

  • prebuilts/clang/darwin-x86/host/3.5

  • prebuilts/misc

  • prebuilts/sdk

  • system/core

I think this is the absolute minimum subset you need to build with.

I assume it would also work if you simply have the entire source tree but I haven’t tried it.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

June 27, 2014

So Swift Then ? Or Wot No Brackets ?

So Swift is essentially Objective-C without the C and without the bracket message sending syntax and with some new stuff and some changed stuff.

Well the C is no loss but the brackets are the best bit so what new stuff do we get to compensate for their disappearance ?

In no particular order

  • tuples

  • a first class enum type

  • a first class struct type

  • operator functions and custom operators

  • range operators

  • subscript operators

  • optional values

  • generics

  • a super switch

  • a plethora of options for parameter lists

and

    much much more

as it always says on the posters.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

November 18, 2013

Service Discovery In Android And iOS: Part Seven – iOS Take Three: Down At C Level

For anyone who considers both Foundation and Core Foundation too rarified there is always a C function API for performing DNS/mDNS based service discovery and registration.

This is declared in the header file

    /usr/include/dns_sd.h

1.0 DNSServiceRefs And Connection Sharing

By default each function in the API which requires DNS functionality establishes a separate connection to the DNS service which it returns as a DNSServiceRef.

In this context the name DNSServiceRef can be a bit confusing.

A DNSServiceRef is a reference to the DNS Service, that is, the thing doing the search for services using DNS, not a reference to a service found using DNS.

There is an alternative to having a per function call connection and DNSServiceRef.

A connection to the DNS service can be established separately and then shared by passing it to each function that would otherwise create a new connection.

2.0 The Function Pattern

All the functions in the API which access the DNS service are declared using a common pattern

    DNSServiceErrorType <function-name>(
                            DNSServiceRef*           sdRef,
                            DNSServiceFlags          flags,
                            uint32_t                 interfaceIndex,
        
                            ... <function specific arguments> ...
    
                            <function specific type> callBack,
                            void*                    context);

2.1 The sdRef Argument

    DNSServiceRef*  sdRef

The sdRef argument is either

  • a pointer to an uninitalized DNSServiceRef which will be initialized to a valid DNSServiceRef if the call succeeds, or

  • a pointer to a shared DNSServiceRef which should be used by the function

2.2 The flags argument

    DNSServiceFlags flags

The flags argument unsurprisingly specifies zero or more flags. If an initialized DNSServiceRef is being passed via the sdRef argument then the flag

    kDNSServiceFlagsShareConnection

must be set.

2.3 The interfaceIndex Argument

    uint32_t   interfaceIndex

The index of the network interface to use to perform the requested DNS operation(s).

If the specific network interface is not important then when starting the search for services the constant

    kDNSServiceInterfaceIndexAny

can be used.

When a service is found the index of the network interface it is associated with is reported and subsequent calls can use this value.

2.4 The callBack Argument

The type of the callBack argument is specific to the function to which it is being passed but in each case it specifies the function to be invoked when a a result is available or an error occurs.

2.5 The info Argument

    void*  context

The value of the context argument will be passed as an argument to the callback function specified by
the callBack argument.

3.0 The Callback Function Pattern

All the callback function types are declared using a common pattern.

    typedef void (*<type-name>)(
                        DNSServiceRef       sdRef,
                        DNSServiceFlags     flags,
                        uint32_t            interfaceIndex,
                        DNSServiceErrorType errorCode,
    
                        ... <function specific arguments> ...

                        void*               context);

3.1 The sdRef Argument

The DNSServiceRef which was passed to the function which invoked this callback.

3.2 The flags Argument

The flags argument is used to pass general status information, e.g., if the

    kDNSServiceFlagsMoreComing

flag is set then this callback will be invoked again.

3.3 The interfaceIndex Argument

The index of the network interface on which the result was obtained.

3.4 The errorCode Argument

If the errorCode argument is not

    kDNSServiceErr_NoError

then an error has occurred.

3.5 The context Argument

The value of the context argument passed to the function which invoked this callback.

4.0 Searching For Services

4.1 DNSServiceBrowse

We can start the search for services of a given type by using the function DNSServiceBrowse
which is declared like this

    DNSServiceErrorType DNSServiceBrowse(
                            DNSServiceRef*        sdRef,
                            DNSServiceFlags       flags,
                            uint32_t              interfaceIndex,
                            const char*           regtype,
                            const char*           domain,
                            DNSServiceBrowseReply callBack,
                            void*                 context);

The regtype argument should be the domain relative type name, e.g.,

    "_ipp.tcp."

The domain argument should be the absolute name of the domain to search in, e.g.,

    "local."

4.2 The DNSServiceBrowse Callback Function

The function passed as the callBack argument to DNSServiceBrowse will be called once for each service of the given type that is found.

If the information about a service that was found becomes invalid, implying that it has ‘disappeared’, then the callback function will called again.’

4.3 The DNSServiceBrowseReply Function Type

The function type DNSServiceBrowseReply is declared like this

    typedef void (*DNSServiceBrowseReply)(
                       DNSServiceRef       sdRef,
                       DNSServiceFlags     flags,
                       uint32_t            interfaceIndex,
                       DNSServiceErrorType errorCode,
                       const char*         serviceName,
                       const char*         regtype,
                       const char*         replyDomain,
                       void*               context);

When a function of this type is invoked, then if the errorCode argument is kDNSServiceErr_NoError

  • serviceName is the type relative name of the service.

  • regtype is the domain relative name of the service type

  • replyDomainis the absolute name of the domain the service is registered in

The kDNSServiceFlagsAdd flag will be set in the flags argument if the service has been found, and clear if the service has been ‘lost’.

5.0 Resolving Services

Resolving a service involves two functions

    DNSServiceResolve

which obtains the service’s SRV and TXT records, and

and

    DNSServiceGetAddrInfo.

which obtains the address or addresses of the host on which the service is running

5.1 DNSServiceResolve

We can obtain the information contained in the SRV and TXT records associated with a given service by using the function
DNSServiceResolve which is declared like this

    DNSServiceErrorType DNSServiceResolve(
                            DNSServiceRef*         sdRef,
                            DNSServiceFlags        flags,
                            uint32_t               interfaceIndex,
                            const char*            name,
                            const char*            regtype,
                            const char*            domain,
                            DNSServiceResolveReply callBack,
                            void*                  context);

The name argument should be the type relative name of the service.

The regtype argument should be the domain relative name of the service type.

The domain argument should be the absolute name of the domain in which the service is registered.

5.1.1 The DNSServiceResolve Callback Function

The function passed as the callBack argument to DNSServiceResolve will be called once, either with the results of with an error.

5.1.1 The DNSServiceResolveReply Function Type

The function type DNSServiceResolveReply is declared like this

    typedef void (*DNSServiceResolveReply)(
                       DNSServiceRef        sdRef,
                       DNSServiceFlags      flags,
                       uint32_t             interfaceIndex,
                       DNSServiceErrorType  errorCode,
                       const char*          fullname,
                       const char*          hosttarget,
                       uint16_t             port,
                       uint16_t             txtLen,
                       const unsigned char* txtRecord,
                       void*                context);

When a function of this type is invoked, then, if the errorCode argument is kDNSServiceErr_NoError

  • fullname is the absolute name of the service, e.g., "ipp_server_1._ipp._tcp.local.".

  • hosttarget is the name of the host the service is running on.

  • port is the port, in network byte order, the service is listening on.

  • txtLen is is the length of the TXT record data.

  • txtRecord is a pointer to the TXT record data itself.

5.2 DNSServiceGetAddrInfo

We can obtain the address of a host using the function DNSServiceGetAddrInfo which is declared like this

    DNSServiceErrorType DNSServiceGetAddrInfo(
                            DNSServiceRef*             sdRef,
                            DNSServiceFlags            flags,
                            uint32_t                   interfaceIndex,
                            DNSServiceProtocol         protocol,
                            const char*                hostname,
                            DNSServiceGetAddrInfoReply callBack,
                            void*                      context);

If the

    kDNSServiceFlagsTimeout

is set in the flags argument then the operation may timeout after a system defined amount of time.

The protocol argument specifies the type of the address requested.

The value should be one of

  • 0

  • kDNSServiceProtocol_IPv4

  • kDNSServiceProtocol_IPv6

  • kDNSServiceProtocol_IPv4|kDNSServiceProtocol_IPv6

A value of 0 (zero) is usually equivalent to requesting both the IPv4 and IPv6 addresses.

The hostname argument should be the absolute name of the host.

5.2.1 The DNSServiceGetAddrInfo Callback Function

The function passed as the callBack argument to DNSServiceGetAddrInfo will be called once for each address type that was requested and is found

It will also be called if the address of the host becomes invalid, e.g., because the host has been turned off.

5.2.2 The DNSServiceGetAddrInfoReply Function Type

The function type DNSServiceGetAddrInfoReply is declared like this

    typedef void (*DNSServiceGetAddrInfoReply)(
                       DNSServiceRef          sdRef,
                       DNSServiceFlags        flags,
                       uint32_t               interfaceIndex,
                       DNSServiceErrorType    errorCode,
                       const char*            hostname,
                       const struct sockaddr* address,
                       uint32_t               ttl,
                       void*                  context);

When a function of this type is invoked, then, if the errorCode argument is kDNSServiceErr_NoError

  • hostname is the name of the host whose address this is

  • address is a pointer to its address, and

  • ttl is the time in seconds for which the given address is valid

The kDNSServiceFlagsAdd flag will be set in the flags argument if the address has been ‘found’, and clear if the address is no longer valid.

The kDNSServiceFlagsMoreComing will be set in the flags argument if there are more addresses, and will be clear if this is the last address.

The type of the address will of course depend upon what what specified as the protocol argument in the call to DNSServiceGetAddrInfo.

If both IPv4 and IPv6 addresses were requested then it will be necessary to examine the sa_family field of the sockaddr struct to find out which one it is.

6.0 The Care And Maintenance Of Your DNSServiceRef

Something that is not perhaps immediately apparent is that the shared DNSServiceRef or one created by a function like DNSServiceBrowse has to be actively handled on the client side.

There are two ways to do this, either

  • by obtaining the file descriptor associated with the connection by calling DNSServiceRefSockFD and doing it all yourself, or

  • by calling the function DNSServiceSetDispatchQueue which will result in the connection being handled ‘automatically’ on the dispatch queue of your choice.

6.1 DNSServiceRefSockFD

The function DNSServiceRefSockFD is declared like this

    int DNSSD_API DNSServiceRefSockFD(DNSServiceRef sdRef);

It takes a DNSServiceRef and returns the file descriptor of the underlying connection to the DNS service.

Once you have obtained your file descriptor you will need to determine when it is readable.

To do this you are going to need either

  • an fd_set and a system call, or

  • a pollfd struct and a different system call

6.1.1 Using select

To use the select system call and assuming sr is the DNSServiceRef you will need to do something like this.

    ...
    
    int fd = DNSServiceRefSockFD(sr);
    
    if (fd == -1)
    {
        fprintf(stderr, "fd == -1 !");
        return;
    }
    
    fd_set         readFDs;
    struct timeval tv;
    
    while (true)
    {
        FD_ZERO(&readFDs);
        FD_SET(fd, &readFDs);
    
        tv.tv_sec  = 1000000;
        tv.tv_usec = 0;
    
        int status = select(fd + 1, &readFDs, NULL, NULL, &tv);
    
        if (status == -1)
        {
            fprintf(stderr, "status == -1\n");
            break;
        }
        else
        if (status == 0)
        {
            fprintf(stderr, "status == 0\n");
        }
        else
        if (FD_ISSET(fd, &readFDs))
        {
            int error = DNSServiceProcessResult(sr);
    
            if (error != kDNSServiceErr_NoError)
            {
                fprintf(stderr, "DNSServiceProcessResult: error == %d\n", error);
                break;
            }
        }
    }

    ...
    

When the file descriptor is readable the function DNSServiceProcessResult is invoked to handle the input. It is this call that results in callback functions being invoked.

This assumes that you are sharing a single DNSServiceRef if not then you are going to end up knee deep in file descriptors and its all going to get very messy very fast.

6.1.2 Using poll

If the use of select is too retro for you you can always use the new-fangled poll system call.

The code looks very similar because poll is just select with unlimited [1] file descriptors.

    ...

    int fd = DNSServiceRefSockFD(sr);
    
    if (fd == -1)
    {
        NSLog(@"fd == -1 !");
        return;
    }
    
    struct pollfd   pollFD;
    
    while (true)
    {
        pollFD.fd = fd;
        pollFD.events = POLL_IN;
        
        int status = poll(&pollFD, 1 , 1000000);
    
        if (status == -1)
        {
            fprintf(stderr, "status == -1\n");
            break;
        }
        else
        if (status == 0)
        {
            fprintf(stderr, "status == 0\n");
        }
        else
        if ((pollFD.revents & POLL_IN) != 0)
        {
            int error = DNSServiceProcessResult(sr);

            if (error != kDNSServiceErr_NoError)
            {
                fprintf(stderr, "DNSServiceProcessResult: error == %d\n", error);
                break;
            }
        }
    }

    ...

6.2 DNSServiceSetDispatchQueue

The alternative to wrestling with file descriptors is the function DNSServiceSetDispatchQueue
which is declared like this

    DNSServiceErrorType DNSServiceSetDispatchQueue(
                            DNSServiceRef    service,
                            dispatch_queue_t queue);

The DNSServiceRef can be associated with a shared connection or with a per function connection.

See below for an example of its use.

7.0 Creating A DNSServiceRef For A Shared Connection

The only way to create a DNSServiceRef for a connection which can be shared is by using the function
DNSServiceCreateConnection which is declared like this

   DNSServiceErrorType DNSServiceCreateConnection(DNSServiceRef* sdRef);

A copy of the initialized DNSServiceRef that results should be passed to each function that is going to share the connection.

See below for an example of its use.

8.0 DNSServiceBrowse In Action

This is the start method of the third version of the FindServices class.

    - (BOOL)start
    {
        DNSServiceErrorType error = DNSServiceCreateConnection(&dnsServiceRef);
    
        if (error != kDNSServiceErr_NoError)
        {
            NSLog(@"Error: DNSServiceCreateConnection %d", error);
            return NO;
        }
        error = DNSServiceSetDispatchQueue(dnsServiceRef, dispatch_get_main_queue());
        if (error != kDNSServiceErr_NoError)
        {
            NSLog(@"Error: DNSServiceSetDispatchQueue %d", error);
            return NO;
        }
        browseRef = dnsServiceRef;
        error = DNSServiceBrowse(
                    &browseRef,
                    kDNSServiceFlagsShareConnection,
                    kDNSServiceInterfaceIndexAny,
                    [self.type UTF8String],
                    [self.domain UTF8String],
                    browseCallback,
                    (__bridge void*)self);
        if (error != kDNSServiceErr_NoError)
        {
            NSLog(@"Error: DNSServiceBrowse %d", error);
            return NO;
        }
        return YES;
    }
    static void browseCallback(
                    DNSServiceRef       sdRef,
                    DNSServiceFlags     theFlags,
                    uint32_t            theInterfaceIndex,
                    DNSServiceErrorType theErrorCode,
                    const char*         theName,
                    const char*         theType,
                    const char*         theDomain,
                    void*               theContext)
    {
        NSLog(@"browseCallback:  error == %d flags == %s", theErrorCode, flagsToString(theFlags));
        
        if (theErrorCode == kDNSServiceErr_NoError)
        {
            ServiceIdentifier si = { theName, theType, theDomain };
        
            if ((theFlags & kDNSServiceFlagsAdd) != 0)
            {
                [(__bridge FindServices*)theContext serviceFound:&si onInterface:theInterfaceIndex];
            }
            else
            {
                [(__bridge FindServices*)theContext serviceLost:&si];
            }
        }
        else
        {
            [(__bridge FindServices*)theContext browseFailed:theErrorCode];
        }
    }

9.0 DNSServiceResolve In Action

This is the resolve:onInterface: method of the ServiceResolver class

    - (void)resolve:(ServiceIdentifier*)theServiceId onInterface:(uint32_t)theInterfaceIndex
    {
        DNSServiceErrorType error;
        
        error = DNSServiceResolve(
                    &resolveRef,
                    kDNSServiceFlagsShareConnection,
                    theInterfaceIndex,
                    theServiceId->name,
                    theServiceId->type,
                    theServiceId->domain,
                    resolveCallback,
                    (__bridge void*)self);
        if (error != kDNSServiceErr_NoError)
        {
            NSLog(@"DNSServiceResolve: %d", error);
            [self.delegate serviceResolver:self didFail:error];
        }
    }

and this is the associated callback function.

    static void resolveCallback(
                    DNSServiceRef        theRef,
                    DNSServiceFlags      theFlags,
                    uint32_t             theInterfaceIndex,
                    DNSServiceErrorType  theErrorCode,
                    const char*          theFullName,
                    const char*          theTarget,
                    uint16_t             thePort,
                    uint16_t             theTXTRecordLength,
                    const unsigned char* theTXTRecord,
                    void*                theContext)
    {
        NSLog(@"resolveCallback: error == %d flags == %s", theErrorCode, flagsToString(theFlags));
        
        if (theErrorCode != kDNSServiceErr_NoError)
        {
            NSLog(@"resolveCallback: error !");
            [(__bridge ServiceResolver*)theContext resolveFailed:theErrorCode];
        }
        else
        if (theFlags == 0)
        {
            ServiceInfo si =
                {
                    theFullName,
                    theTarget,
                    ntohs(thePort),
                    theTXTRecordLength,
                    theTXTRecord
                };
        
            NSLog(@"%s %s %u", theFullName, theTarget, thePort);
        
            [(__bridge ServiceResolver*)theContext resolved:&si onInterface:theInterfaceIndex];
        }
        else
        {
            NSLog(@"resolveCallback: flags set !");
            [(__bridge ServiceResolver*)theContext internalError];
        }
    }

10.0 DNSServiceGetAddrInfo In Action

This is the resolved:onInterface: method of the ServiceResolver class

    - (void)resolved:(const ServiceInfo*)theServiceInfo onInterface:(uint32_t)theInterfaceIndex
    {
        DNSServiceErrorType error;
        
        error = DNSServiceGetAddrInfo(
                    &addressRef,
                    kDNSServiceFlagsShareConnection,
                    theInterfaceIndex,
                    kDNSServiceProtocol_IPv4|kDNSServiceProtocol_IPv6,
                    theServiceInfo->target,
                    addressInfoCallback,
                    (__bridge void*)self);
        if (error == kDNSServiceErr_NoError)
        {
            [self.builder serviceInfo:theServiceInfo];
        }
        else
        {
            NSLog(@"DNSServiceGetAddrInfo: %d", error);
            [self.delegate serviceResolver:self didFail:error];
        }
    }

and this is the associated callback function.

    static void addressInfoCallback(
                    DNSServiceRef          theServiceRef,
                    DNSServiceFlags        theFlags,
                    uint32_t               theInterfaceIndex,
                    DNSServiceErrorType    theErrorCode,
                    const char*            theHostname,
                    const struct sockaddr* theAddress,
                    uint32_t               theTTL,
                    void*                  theContext)
    {
        NSLog(@"addressInfoCallback: error == %d flags == %s ", theErrorCode, flagsToString(theFlags));
        
        if (theErrorCode != kDNSServiceErr_NoError)
        {
            NSLog(@"addressInfoCallback error");
            [(__bridge ServiceResolver*)theContext getAddrInfoFailed:theErrorCode];
        }
        else
        if ((theFlags & kDNSServiceFlagsAdd) != 0)
        {
            NSLog(@"theHostname == %s", theHostname);
            NSLog(@"theAddress->sa_family == %d", theAddress->sa_family);
        
            [(__bridge ServiceResolver*)theContext address:theAddress];
            if ((theFlags & kDNSServiceFlagsMoreComing) == 0)
            {
                [(__bridge ServiceResolver*)theContext done];
            }
        }
        else
        {
            NSLog(@"theHostname == %s", theHostname);
            NSLog(@"theAddress->sa_family == %d", theAddress->sa_family);
            // ignore
        }
    }

11.0 Examples

In each case FindServices is looking for services of type

    "_ipp._tcp."

in the domain

    "local."

In each case the log output is from FindServices and its delegate running on an iPad running iOS 7.0.

7.1 A Single IPPServer

A single instance of the CUPS test server IPPServer with the name ipp_server_1
running on a Mac and then being stopped.

In this case the addressInfoCallback function is called twice, first with the IPv6 address then with the IPv4 address.

    ...
        
    2013-11-18 14:53:20.304 XperTakeThree[334:60b] browseCallback:  error == 0 flags == kDNSServiceFlagsAdd
    2013-11-18 14:53:20.307 XperTakeThree[334:60b] serviceFound: ipp_server_1._ipp._tcp.local.
    2013-11-18 14:53:20.309 XperTakeThree[334:60b] resolveCallback: error == 0 flags == <none>
    2013-11-18 14:53:20.310 XperTakeThree[334:60b] ipp_server_1._ipp._tcp.local. Simons-Computer.local. 56088
    2013-11-18 14:53:20.312 XperTakeThree[334:60b] addressInfoCallback: error == 0 flags == kDNSServiceFlags{Add,MoreComing}
    2013-11-18 14:53:20.313 XperTakeThree[334:60b] theHostname == Simons-Computer.local.
    2013-11-18 14:53:20.314 XperTakeThree[334:60b] theAddress->sa_family == 30
    2013-11-18 14:53:20.315 XperTakeThree[334:60b] addressInfoCallback: error == 0 flags == kDNSServiceFlagsAdd
    2013-11-18 14:53:20.316 XperTakeThree[334:60b] theHostname == Simons-Computer.local.
    2013-11-18 14:53:20.317 XperTakeThree[334:60b] theAddress->sa_family == 2
    2013-11-18 14:54:18.545 XperTakeThree[334:60b] browseCallback:  error == 0 flags == <none>
    2013-11-18 14:54:18.547 XperTakeThree[334:60b] serviceLost: name == ipp_server_1._ipp._tcp.local.
        
    ...

11.2 A Single Printer

A printer being turned on and then turned off a couple of minutes later.

In this case we only get a single address, the IPV4 one, but we do get a second call to the function addressInfoCallback
function when the printer is turned off

    ...
        
    2013-11-18 14:55:55.137 XperFS_DNS_SD[351:60b] browseCallback:  error == 0 flags == kDNSServiceFlagsAdd
    2013-11-18 14:55:55.140 XperFS_DNS_SD[351:60b] serviceFound: Canon MG6200 series._ipp._tcp.local.
    2013-11-18 14:55:55.141 XperFS_DNS_SD[351:60b] resolveCallback: error == 0 flags == <none>
    2013-11-18 14:55:55.142 XperFS_DNS_SD[351:60b] Canon32MG620032series._ipp._tcp.local. 7D300C000000.local. 30466
    2013-11-18 14:55:55.144 XperFS_DNS_SD[351:60b] addressInfoCallback: error == 0 flags == kDNSServiceFlagsAdd
    2013-11-18 14:55:55.145 XperFS_DNS_SD[351:60b] theHostname == 7D300C000000.local.
    2013-11-18 14:55:55.146 XperFS_DNS_SD[351:60b] theAddress->sa_family == 2
    2013-11-18 15:02:18.835 XperFS_DNS_SD[351:60b] addressInfoCallback: error == 0 flags == <none>
    2013-11-18 15:02:18.837 XperFS_DNS_SD[351:60b] theHostname == 7D300C000000.local.
    2013-11-18 15:02:18.839 XperFS_DNS_SD[351:60b] theAddress->sa_family == 2
    2013-11-18 15:02:19.936 XperFS_DNS_SD[351:60b] browseCallback:  error == 0 flags == <none>
    2013-11-18 15:02:19.938 XperFS_DNS_SD[351:60b] serviceLost: name == Canon MG6200 series._ipp._tcp.local.

    ...

Notes

  • Subject to terms and conditions. The number of file descriptors may be subject to limits.

Copyright (c) 2013 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

May 28, 2013

Building Rust For Mac OS X: The Programming Language Not The Iron Oxide

Filed under: MacOSX, Programming Languages, Rust — Tags: , , , — Simon Lewis @ 6:11 pm

1.0 Prerequisites

To build Rust for Mac OS X you will need the Xcode development environment plus the lastest Rust tarball. At the time of writing that is version 0.6 which you can find here.

2.0 Building The Standard Way

Once you have downloaded the tarball and unpacked it, then on the command line cd to the root directory of the distribution

    cd $(WHEREVER_YOU_UNPACKED_THE_TARBALL)/rust-0.6

The instructions for building can be found in the file Readme.md in the root directory.

By default it is very simple.

On the command line, just do

    ./configure

followed by

    make

to build everything, followed by

    make install

which will install the various build products into the bin and lib subdirectories of

    /usr/local

Depending on who owns the various subdirectories of /usr/local you may need to run the last step as root

    sudo make install

3.0 Building The Non-Standard Way

I like to avoid ending up with a whole bunch of intermediate and final build products mixed up with the source code if I can help it, and I don’t like dumping everything and the kitchen sink into /usr/local so I did things a little differently.

If you do it this way the entire build is done in a completely separate directory tree, which is useful when you need to throw everything away and start again, or if you want to build multiple variants.

Once you have downloaded the tarball and unpacked it, decide where you want to do the build. The build directory can be anywhere you want.

Create the build directory if necessary then on the command line cd to it

   cd $(BUILD_DIR)

Now run the configure script which is in the root of the directory which was created when the tarball was created like this

    $(WHEREVER_YOU_UNPACKED_THE_TARBALL)/configure --prefix=$(WHEREVER_YOU_WANT_THINGS_TO_BE_INSTALLED)

In my case the root directory of the unpacked tarball was

    /Users/simon/Src/lang/rust-0.6

My build directory was

   /Users/simon/Scratch/rust/rust_0_6/macosx

and I wanted the binaries etc. installed in

   /Users/simon/Dev/tools/rust/macosx

so what I did was

  cd /Users/simon/Scratch/rust/rust_0_6/macosx

followed by

  /Users/simon/Src/lang/rust-0.6/configure --prefix=/Users/simon/Dev/tools/rust/macosx

If you don’t mind having everything dumped under /usr/local then omit the --prefix arg.

Once the configuration is complete just do

    make

on the command line.

If all goes well then this step is going to take a while since, amongst other things, it appears to build all of llvm from src.

Once it has finished do

    make install

on the command line to install the binaries and libraries that have been built.

5.0 Build Products

You should end up with the following in the directories under wherever you specified as the installation directory

  • in bin

    • rust

    • rustc

    • rustdoc

    • rusti

    • rustpkg

  • in lib

    • libcore-c3ca5d77d81b46c1-0.6.dylib

    • librust-39583f72884834e3-0.6.dylib

    • librustc-c84825241471686d-0.6.dylib

    • librusti-5047c7f210c7cac8-0.6.dylib

    • librustllvm.dylib

    • librustrt.dylib

    • libstd-4782a756585a81-0.6.dylib

    • libsyntax-84efebcb12c867a2-0.6.dylib

  • in lib/rustc/x86_64-apple-darwin/lib

    • libcore-c3ca5d77d81b46c1-0.6.dylib

    • libmorestack.a

    • librust-39583f72884834e3-0.6.dylib

    • librustc-c84825241471686d-0.6.dylib

    • librustdoc-1ac3c0763957e4b0-0.6.dylib

    • librusti-5047c7f210c7cac8-0.6.dylib

    • librustpkg-795073e423d025b-0.6.dylib

    • librustrt.dylib

    • libstd-4782a756585a81-0.6.dylib

    • libsyntax-84efebcb12c867a2-0.6.dylib


Copyright (c) 2013 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

October 29, 2009

An Extended NewAnnotation Wizard For Eclipse: Getting Started

Because I wanted to re-use the existing Eclipse NewAnnotation wizard code, if at all possible, I decided to create a fragment hosted by the bundle containing the code for the Eclipse Java type creation wizards, rather than a stand-alone bundle. Note that Eclipse uses the term ‘plug-in’, but an Eclipse ‘plug-in’ is an OSGi bundle with some added stuff, and I’m going to stick to the term bundle.

To do this select ‘New’ from the ‘File’ menu on the toolbar, ‘Project’ from the resulting sub-menu and ‘Fragment Project’ from the ‘Plug-in Development’ section of the resulting dialog

NewProject

Hit ‘Next’ and you will get the first page of the ‘New Fragment Project’ wizard

FragmentProject1

Fill in the Project Name and hit ‘Next’

FragmentProject2Blank

The Host Plug-in (bundle) section needs to be filled-in, so hit ‘Browse’ and you will get the ‘Plug-in Selection’ dialog show below

PluginSelectionBlank

We are actually interested in the code in the package

       org.eclipse.jdt.ui.wizards

but which bundle is it in ?

The mapping of packages to JDT bundles can be found in the JDT Plug-ins Map which shows that we are after the

       org.eclipse.jdt.ui

bundle.

Entering this is in the text field at the top gets us two matches

PluginSelection

Picking the top-most one and hitting ‘OK’ gets us back to the second page

FragmentProject2

That’s it. Hit ‘Finish’, respond to the resulting dialog about opening the associated perspective and its done.

Not that it does currently does anything but the resulting fragment can be ‘run’ by selecting the ‘run’ button from the toolbar and then selecting ‘Eclipse Application’ from the ‘Run As’ sub-menu, unless you are running Eclipse on Snow Leopard, that is, in which case you may get an error dialog instead, and a log containing the following


!ENTRY org.eclipse.osgi 4 0 2009-10-29 21:13:30.206
!MESSAGE Application error
!STACK 1
java.lang.UnsatisfiedLinkError: Cannot load 32-bit SWT libraries on 64-bit JVM
	at org.eclipse.swt.internal.Library.loadLibrary(Library.java:182)
	at org.eclipse.swt.internal.Library.loadLibrary(Library.java:159)
	at org.eclipse.swt.internal.C.(C.java:21)
	at org.eclipse.swt.internal.cocoa.NSThread.isMainThread(NSThread.java:33)
	at org.eclipse.swt.graphics.Device.(Device.java:116)
	at org.eclipse.swt.widgets.Display.(Display.java:628)
	at org.eclipse.swt.widgets.Display.(Display.java:619)
	at org.eclipse.ui.internal.Workbench.createDisplay(Workbench.java:532)
	at org.eclipse.ui.PlatformUI.createDisplay(PlatformUI.java:161)
	at org.eclipse.ui.internal.ide.application.IDEApplication.createDisplay(IDEApplication.java:143)
	at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:88)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:194)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:368)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:559)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:514)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1311)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1287)

which is a bit of a pain

Fortunately it is easily fixed.

Bring up the ‘Run Configurations’ dialog, using ‘Run Configurations …’ from the menu on the ‘run’ button for example, select ‘Eclipse Application’ then select the ‘Arguments’ tab.

In the ‘Program arguments’ text box at the top prepend the argument

       -d32

as shown below. This tells the VM not to mess about with all that 64-bit stuff, or something like that.

RunConfigurations

Try ‘running’ the fragment again and you should now get a second copy of Eclipse uncannily similar to the one you are already running.


Copyright (c) 2009 By Simon Lewis. All Rights Reserved

November 7, 2008

Dalvik Running Standalone On MacOSX

Filed under: Android, Dalvik, Java, Java VM, MacOSX — Tags: , , , , — Simon Lewis @ 5:35 pm

When the source for Google’s Dalvik Java VM became available as part of the Android open source release I thought I would see whether it was possible to build and run it standalone, i.e not as part of Android.

The answer is yes, and here’s a shot of it running in the debugger under Xcode on MacOSX to prove it

dalvik

Unfortunately the canonical HelloWorld application on its own doesn’t make for particularly interesting viewing

With verbose logging on the output is certainly a lot more voluminous.

Here’s the debugger window scrolled to different positions to show all of it

dalvik_verbose_a3

dalvik_verbose_b

dalvik_verbose_c2

Create a free website or blog at WordPress.com.

%d bloggers like this: