Just An Application

June 6, 2012

Adventures In Distributed Garbage Collection: Part Eighteen – The Great Service Record Mystery Revisited

Filed under: Android, Android Internals, Java, Mobile Java, Services — Tags: , , , , — Simon Lewis @ 9:31 am

1.0 Is The ServiceRecord Part Of A Binder Reference Cycle ?

As we have seen it is possible for a Java Binder object to be part of a Binder reference cycle which will prevent it from being garbage collected, and we know that a ServiceRecord is a Java Binder object, so does this explain what is happening in this case ?

To be sure we need to find a Binder reference cycle which starts and ends at the ServiceRecord object [0x406f89b8]

1.1 The ServiceRecord

We know that there is a JNI Global reference to the ServiceRecord object [0x406f89b8] which means that there exists at least one BinderProxy somewhere that references it.

1.2 The ServiceRecord BinderProxy

We know

  1. that the ServiceRecord object representing a Service is passed to the process running that Service via a remote procedure call to the scheduleCreateService() method of the ApplicationThread

  2. that during the the unmarshalling of the arguments to this method a BinderProxy object which references the ServiceRecord object is created

  3. that this BinderProxy object is passed to the Service.attach() method as the token argument

  4. that it is then assigned to the mToken instance variable of the Service Component object that implements the Service

  5. that in this case the Service Component object is the FastnetService object [0x0x40517518].

From this and in conjunction with the object graph we can identify the Java object in the Service process which references the ServiceRecord object [0x0x0x406f89b8] in the System process as the BinderProxy object [0x40513580].

Direct inspection of the Java heap dump confirms that the only effective reference to the BinderProxy object [0x40513580] is from the FastnetService object [0x0x40517518].

We also know that the process running the Service is the only one to which the ServiceRecord object representing that Service is sent, so this is the only BinderProxy which references the ServiceRecord object.

1.3 The FastnetService Object

We can see from this object graph that there are two references to the FastnetService object [0x40517518]

  • one from the ContextImpl object [0x40517548], and

  • one from the FastnetServiceBinder object [0x40517fc0].

We can ignore the reference from the ContextImpl object [0x40517548] as it in turn is only referenced by the FastnetService object [0x40517518] and an inner class object which only it in turn references.

The reference that is keeping the FastnetService object [0x40517518] in existence is the one from the FastnetServiceBinder object [0x40517fc0].

1.4 The FastnetServiceBinder Object

We know

  1. that the FastnetServiceBinder [0x40517fc0] object is a Java Binder object

  2. that there is a JNI Global reference to it

  3. that the JNI Global reference is the only effective reference to it, that is, it is the one which is preventing it from being garbage collected

  4. that it was returned from a call to the FastnetService implementation of the onBind() method

We can infer from the Service process runtime Binder information below

    binder proc state:
    proc 301
      thread 301: l 00
      thread 307: l 12
      thread 308: l 11
      node 4335: u0014c9a0 c00094f88 hs 1 hw 1 ls 0 lw 0 is 1 iw 1 proc 60
      node 4358: u001568a8 c00156888 hs 1 hw 1 ls 0 lw 0 is 1 iw 1 proc 60
      ref 4330: desc 0 node 1 s 1 w 1 d (null)
      ref 4333: desc 1 node 147 s 1 w 1 d (null)
      ref 4338: desc 2 node 209 s 1 w 1 d (null)
      ref 4339: desc 3 node 143 s 1 w 1 d (null)
      ref 4340: desc 4 node 196 s 1 w 1 d (null)
      ref 4343: desc 5 node 4342 s 1 w 1 d (null)
      ref 4350: desc 6 node 23 s 1 w 1 d (null)
      ref 4353: desc 7 node 44 s 1 w 1 d (null)

that there can be only one BinderProxy referencing it since there are only two Binders associated with the process and both of them are only referenced from one other process (60) which is in fact the System process.

1.5 The FastnetServiceBinder BinderProxy

We know

  1. that the Java Binder object returned from the call to a Service component’s implementation of the onBind() method is passed to the
    System process via a remote procedure call to the publishService() method of the ActivityManagerService.

  2. that during the unmarshalling of the arguments to this method a BinderProxy object is created which references the original Binder object

  3. that this BinderProxy object is then assigned to the binder instance variable of the IntentBindRecord associated with the ServiceRecord which represents the Service which created the original Binder object

From this and in conjunction with the object graph we can identify the Java object corresponding to the FastnetServiceBinder object [0x40517fc0] in the Service process as being BinderProxy object [0x4069ce20] which is referenced by the IntentBindRecord object [0x40704020].

Direct inspection of the Java heap dump confirms that this is the only reference.

1.6 The IntentBindRecord

We can see from the object graph that there is a single reference to the IntentBindRecord object [0x40704020] from the HashMap object [0x406f8a80].

Direct inspection of the Java heap dump confirms that this the only reference.

We can also see that there is a single reference to the HashMap object [0x406f8a80] from the ServiceRecord object [0x0x406f89b8].

Direct inspection of the Java heap dump confirms this the only reference.

1.7 The ServiceRecord

We are back at the ServiceRecord object [0x0x406f89b8] so there is indeed a Binder reference cycle and it looks like this

2.0 Does It Matter ?

Any Android Service implementation based on the example in the documentation can end up causing a Binder reference cycle.

If it does, then depending upon the exact implementation of the Service and its usage patterns it is possible that the process running the Service will crash when it runs out of memory.This is especially true if the Service component directly or indirectly uses a large amount of memory and a number of Service components are leaked as clients repeatedly bind and unbind.

Given enough different Service implementations all with the same problem then it is possible that the System process will crash as eventually it will run out of JNI Global references.

So it does matter, especially since if a given Service implementation does run into this problem it is not at all obvious what the underlying cause is.

3.0 Bug Fix

One of the problems associated with ServiceRecord reference cycles, namely the Service process runnning out of memory, was reported as an issue two years ago.

There is a fix for it in the Android code base but it is not clear when it actually turned up in shipping products. The submission date was just after the initial release of 2.3 so it probably did not start to appear until later revisions of 2.3 which may mean that there are still many devices on which this is potentially a problem.

The fix is specific to the ServiceRecord reference cycle. It is simply to break the reference cycle in the System process by clearing the HashMap referenced by the bindings instance variable of the ServiceRecord. This is done in the bringDownServiceLocked() method of the ActivityManagerService. Doing this breaks the reference between the ServiceRecord and the associated IntentBindRecord(s).

4.0 A Workaround

It is possble to work around the ServiceRecord reference cycle problem specifically by ensuring that the Java Binder object returned from a call to a Service component’s implementation of the onBind() method, only holds a weak reference to the Service component. This prevents the establishment of a Binder reference cycle.

The Java Binder object should either be an instance of a separate class, as in the Fastnet Service example, or a static inner class of the Service and it should use an instance of java.lang.WeakReference to hold the reference to the Service component.

For example, the Service class

    package xper.service.hebrides;

    import android.app.Service;
    import android.content.Intent;
    import android.os.IBinder;
    import android.util.Log;

    public final class HebridesService 
                       extends 
                           Service 
    {
        public HebridesService()
        {
        }
	
        @Override
        public void onCreate() 
        {
            super.onCreate();
            binder = new HebridesServiceBinder(this);
        }

        @Override
        public IBinder onBind(Intent intent) 
        {
            Log.d(TAG, "onBind(" + intent + ")");
            return binder;
        }
	
        //
	
        @Override
        public boolean onUnbind(Intent intent) 
        {
            Log.d(TAG, "onUnbind(" + intent + ")");
            return super.onUnbind(intent);
        }

        void doNothingInParticular()
        {
		
        }

        //
	
        private HebridesServiceBinder	binder;
	
        //
	
        private static final String TAG = "HebridesService";
    }

and the Binder class

    package xper.service.hebrides;

    import java.lang.ref.WeakReference;

    import android.os.RemoteException;
    import xper.service.hebrides.Hebrides.Stub;

    final class HebridesServiceBinder 
                extends 
                    Stub 
    {
        public void doNothingInParticular() 
                    throws 
                        RemoteException 
        {
            HebridesService hs = service.get();
		
            if (hs != null)
            {
                hs.doNothingInParticular();
            }
        }

        //
	
        HebridesServiceBinder(HebridesService theService)
        {
            service = new WeakReference<HebridesService>(theService);
        }
	
        //
	
        private WeakReference<HebridesService>	service;
    }

5.0 An Alternative Design Pattern

By convention, the documentation, implication, a Service Component is a sub-class of a class called Service, and it is declared in the Android manifest in an element called service, and probably, though not necessarily intention, the Service Component implements the functionality of the Service, that is the methods defined in the AIDL definition of that Service.

If the Service Component implements the functionality of the Service then a Binder created to enable access to that Service necessarily has a reference to the Service Component and it is that reference that results in a Binder reference cycle.

However, as a sub-class of the Service class the only method the Service Component has to implement is onBind(), that is, the only function it is required to perform is that of a Binder Factory.

As the example below shows it is possible to adopt a different design pattern where the Service Component does nothing other than create Binder objects when necessary. Another object, in the example, the Application, implements the Service functionality.

One consequence of this is that the Binder objects do not need to have a reference to the Service Component, but another is that the lifetime of the implementation of the Service is decoupled from that of the Service Component. This can be useful if the setting up and/or tearing down of the implementation of the Service functionality is computationally expensive.

5.1. Example

5.1.1 An Interface Defining The Service

    package xper.service.faeroes;

    public interface FaeroesService 
    {
        public void doNothingInParticular();
    }

5.1.2 The Application Class

    package xper.service.faeroes;

    import android.app.Application;

    public class FaeroesServiceApplication 
                 extends 
                     Application 
                 implements
                     FaeroesService
    {
        public void doNothingInParticular() 
        {		
        }

        //
	
        FaeroesService getService()
        {
            return this;
        }
    }

5.1.3 The Binder Factory Implementation

    package xper.service.faeroes;

    import android.app.Service;
    import android.content.Intent;
    import android.os.IBinder;
    import android.util.Log;

    public class FaeroesServiceBinderFactory 
                 extends 
                     Service 
    {
        @Override
        public void onCreate() 
        {
            super.onCreate();
		
            FaeroesServiceApplication app = (FaeroesServiceApplication)getApplication();
		
            binder = new FaeroesServiceBinder(app.getService());
        }

        //

        @Override
        public IBinder onBind(Intent intent) 
        {
            Log.d(TAG, "onBind(" + intent + ")");
            return binder;
        }

        @Override
        public boolean onUnbind(Intent intent) 
        {
            Log.d(TAG, "onUnind(" + intent + ")");
            return super.onUnbind(intent);
        }
	
        //
	
        private FaeroesServiceBinder binder;
	
        //
	
        private static final String TAG = "FaeroesServiceBinderFactory";
    }

5.1.4 The Binder implementation

    package xper.service.faeroes;

    import android.os.RemoteException;

    final class FaeroesServiceBinder 
                extends
                    Faeroes.Stub
    {
        @Override
        public void doNothingInParticular() 
                    throws 
                        RemoteException 
        {
            service.doNothingInParticular();
        }
	
        //
	
        FaeroesServiceBinder(FaeroesService theService)
        {
            service = theService;
        }

	    //
	
        private FaeroesService service;
    }

5.1.5 The Android Manifest

    <?xml version="1.0" encoding="utf-8"?>
    <manifest xmlns:android="http://schemas.android.com/apk/res/android"
        package="xper.service.faeroes"
        android:versionCode="1"
        android:versionName="1.0" >

        <uses-sdk android:minSdkVersion="9" />

        <application
            android:icon="@drawable/ic_launcher"
            android:label="@string/app_name" android:name="FaeroesServiceApplication">
            <service android:name="FaeroesServiceBinderFactory">
                <intent-filter>
                    <action android:name="xper.service.faeroes.ACTION_FAEROES"/>
                </intent-filter>
            </service>
        </application>
    </manifest>

Copyright (c) 2012 By Simon Lewis. All Rights Reserved.

June 1, 2012

Adventures In Distributed Garbage Collection: Part Seventeen – Binders And Garbage Collection

Filed under: Android, Android Internals, Java, Mobile Java, Services — Tags: , , , , — Simon Lewis @ 9:33 am

1.0 An Alternative Service implementation

Unlike the previous example the following Android Service implementation creates the Java Binder object returned from the onBind() method on demand and it does not retain a reference to it.

    package xper.service.malin;

    import android.app.Service;
    import android.content.Intent;
    import android.os.IBinder;
    import android.util.Log;

    public final class MalinService 
                       extends 
                           Service 
    {
        public MalinService()
        {
        }
	

        @Override
        public IBinder onBind(Intent intent) 
        {
            Log.d(TAG, "onBind(" + intent + ")");
            return new MalinServiceBinder(this);
        }
	
        //
	
        @Override
        public boolean onUnbind(Intent intent) 
        {
            Log.d(TAG, "onUnbind(" + intent + ")");
            return super.onUnbind(intent);
        }

        void doSomethingInParticular()
        {
        }

        //
	
        private static final String TAG = "MalinService";
    }

The MarlinServiceBinder object retains a reference to the MalinService object but that is all.

    package xper.service.malin;

    import android.os.RemoteException;
    import xper.service.malin.Malin.Stub;

    final class MalinServiceBinder 
                extends 
                    Stub 
    {
        public void doSomethingInParticular() 
                    throws 
                        RemoteException 
        {
            service.doSomethingInParticular();
        }

        //
	
        MalinServiceBinder(MalinService theService)
        {
            service = theService;
        }
	
        //
	
        private MalinService	service;
    }

Once it has been created there are no references to the MarlinServiceBinder object from any other Java objects.

We can show this using MAT and by examining the heap dump directly.

As in the ServiceRecord case MAT erroneously reports that the MalinServiceBinder object is referenced by a Native Stack GC Root but examination of the heap dump shows that, as expected, it is actually a JNI Global reference.

In this example, if the JNI Global reference did not exist then the MarlinServiceBinder object would be eligible for garbage collection. This is of course precisely why the Binder C++ code creates the JNI Global reference. It is to ensure that a Java Binder object that is otherwise unreferenced in the process in which it was created is not garbage collected whilest there are references to it from other processes.

2.0 Binder References

Explaining exactly how the Binder reference mechanism works more or less requires explaining exactly how Binders work full-stop, which would take a lot more space than is available in the margin of this post, so what follows is a somewhat simplified explanation.

2.2 Reference Counting

Binders are reference counted. Each BinderProxy which references a Binder constitutes a reference to that Binder.

The Binder reference counting mechanism is, necessarily, cross-process and entirely orthogonal to the per-process Java garbage collection mechanism.

2.2.1 Incrementing The Reference Count

Effectively each time a Binder is sent to another process and a BinderProxy is created for that Binder the reference count is incremented.

2.2.2 Decrementing The Reference Count

The finalize() method of a Java BinderProxy object, that is an instance of the class android.os.BinderProxy, calls the native method destroy() which decrements the reference count of the associated Binder object.

In other words, once a Java BinderProxy object has been created in a process then the reference from it to the associated Binder will exist until that Java BinderProxy object has both been garbage collected and finalized.

2.2 Interaction With Java GC

As we have seen the Binder reference mechanism creates a JNI Global reference to a Java Binder object to prevent it from being garbage collected whilest there remain BinderProxy objects in other processes that reference it.

The JNI Global reference is created the first time the Binder is referenced by a BinderProxy.

The JNI Global reference is destroyed when there are no longer any BinderProxies referencing the Binder as determined by the reference count.

That is the only interaction between the Binder reference mechanism and the Java garbage collection mechanism of the process in which the Binder object was created and it is completely opaque. That is the Binder reference mechanism creates a vanilla JNI Global reference that is indistinguishable from a JNI Global reference created by any other piece of native code.

2.4 Inspecting Per-Process Binder Related State At Runtime

It is possible to inspect the Binders and BinderProxies in use in a given process by looking at the contents of the file

    /proc/binder/proc/<PID>

where <PID> is the id of the process.

For example, entering

 cat /proc/binder/proc/368

in a shell running on the device after the client has bound to the Malin Service results in the following output

    binder proc state:
    proc 368
      thread 368: l 00
      thread 374: l 12
      thread 375: l 11
      node 3879: u0007ccf0 c00094f88 hs 1 hw 1 ls 0 lw 0 is 1 iw 1 proc 60
      node 3904: u001a5bb8 c001a5b98 hs 1 hw 1 ls 0 lw 0 is 2 iw 2 proc 357 60
      ref 3874: desc 0 node 1 s 1 w 1 d (null)
      ref 3877: desc 1 node 135 s 1 w 1 d (null)
      ref 3882: desc 2 node 196 s 1 w 1 d (null)
      ref 3883: desc 3 node 131 s 1 w 1 d (null)
      ref 3884: desc 4 node 183 s 1 w 1 d (null)
      ref 3887: desc 5 node 3886 s 1 w 1 d (null)
      ref 3896: desc 6 node 23 s 1 w 1 d (null)
      ref 3899: desc 7 node 44 s 1 w 1 d (null)

In this case 368 is the id of the process running the Malin Service, 357 is the id of the process running the client, and 60 is the id of the System process.

The corresponding output for the process running the client looks like this

    binder proc state:
    proc 357
      thread 357: l 00
      thread 366: l 12
      thread 367: l 11
      node 3830: u000a5168 c001ad3c0 hs 1 hw 1 ls 0 lw 0 is 1 iw 1 proc 60
      node 3754: u0014c9a0 c00094f88 hs 1 hw 1 ls 0 lw 0 is 1 iw 1 proc 60
      node 3785: u001ad4f0 c001ae468 hs 1 hw 1 ls 0 lw 0 is 2 iw 2 proc 113 60
      node 3783: u001ae450 c001ae430 hs 1 hw 1 ls 0 lw 0 is 1 iw 1 proc 60
      node 3791: u001ae6d0 c001ae6b0 hs 1 hw 1 ls 0 lw 0 is 1 iw 1 proc 60
      node 3870: u001b3ba8 c001b3b88 hs 1 hw 1 ls 0 lw 0 is 1 iw 1 proc 60
      ref 3749: desc 0 node 1 s 1 w 1 d (null)
      ref 3752: desc 1 node 135 s 1 w 1 d (null)
      ref 3757: desc 2 node 196 s 1 w 1 d (null)
      ref 3758: desc 3 node 131 s 1 w 1 d (null)
      ref 3759: desc 4 node 183 s 1 w 1 d (null)
      ref 3762: desc 5 node 3761 s 1 w 1 d (null)
      ref 3772: desc 6 node 23 s 1 w 1 d (null)
      ref 3775: desc 7 node 44 s 1 w 1 d (null)
      ref 3781: desc 8 node 219 s 1 w 1 d (null)
      ref 3789: desc 9 node 3788 s 1 w 1 d (null)
      ref 3799: desc 10 node 3798 s 1 w 1 d (null)
      ref 3803: desc 11 node 3802 s 1 w 1 d (null)
      ref 3807: desc 12 node 3806 s 1 w 1 d (null)
      ref 3828: desc 13 node 272 s 1 w 1 d (null)
      ref 3843: desc 14 node 3821 s 1 w 1 d (null)
      ref 3866: desc 15 node 765 s 1 w 1 d c5ff8940
      ref 3907: desc 16 node 3904 s 1 w 1 d c469be40

2.4.1 Binder Entries

The lines with the prefix node describe Binder objects that have been created in the process.

The numbers after the proc keyword are the ids of the processes which hold references to that Binder, that is, there
are BinderProxy objects in those processes which reference the Binder.

For example, the entry

      node 3904: u001a5bb8 c001a5b98 hs 1 hw 1 ls 0 lw 0 is 2 iw 2 proc 357 60

from the first example above from the Service process, 368, specifies that the System process (60) and the client process (357) both have BinderProxy objects which reference this Binder in the service process.

By inference this entry must be for the MalinServiceBinder Java Binder object.

2.4.2 BinderProxy Entries

The lines with the prefix ref describe the BinderProxy objects that have been created in the process

The number after the node keyword is the id of the Binder the BinderProxy references.

For example, the entry

      ref 3907: desc 16 node 3904 s 1 w 1 d c469be40

from the second example above from the client process, 357, specfies the BinderProxy which references the Binder with id 3904 which as we have seen is a Binder in the service process, 368.

3.0 A Binder Cycle Made For Two

The way the Binder reference mechanism works means that it possible to create cross-process reference cycles such that none of the objects in the cycle can be garbage collected if they are Java objects or freed if they are C++ objects despite the fact that none of them are in use. In short a memory leak.

The simplest cycle requires just two Java Binder objects in two separate processes each of which holds a Java BinderProxy object referencing the other, like so


Copyright (c) 2012 By Simon Lewis. All Rights Reserved.

Blog at WordPress.com.

%d bloggers like this: