Just An Application

August 23, 2014

Anatomy Of A PDF: Part Four — Beware Of Objects Bearing Gifts Or Containing Forms

Filed under: Document Format, PDF, PDF Objects — Tags: , , , — Simon Lewis @ 7:07 pm

The start of Object 1 looks like this.

    1 0 obj
    <</Filter [/Fl /Fl] /Length 13178 >>
    stream
    
    ...

and the end of it looks like this

    ...
    
    endstream
    endobj
    
    ...

and in between there are 13178 bytes of assorted binary sludge.

This is beacause the contents of the object have been filtered before being written to the file.

This is what the presence of the Filter entry in the object’s dictionary is telling us.

The associated value is an array which specifies the filters that have been applied.

    Fl

is an abbreviation for

    FlateDecode

There are two entries, so the FlateDecode filter has been applied twice, first to the original object, and then to the output produced by the first pass.

The FlateDecode filter produces data in the DEFLATE compressed data format.

To find out what is in Object 1 we are going to have to inflate it twice.

Seconds Out, Round One

Inflating it once gives us 408521 bytes of compressed XML up from 13178 so its obviously got a lot of zeroes in it or something.

Seconds Out, Round Two

Inflating our collection of 408521 newly minted bytes we end up with a decidedly non-trivial 91044256 bytes of XML which is a whole lot of angle brackets.

Its going to take forever to fill in all the fields in this thing.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

Anatomy Of A PDF: Part Three — Objects, Objects, Objects

According to the cross-reference table there are six objects in use starting with object number 1.

Starting from the root object as specified by the trailer.

Object 3

Object 3 starts at 13294 which 0x33EE.

    3 0 obj
    <<
    /Extensions <</ADBE <</ExtensionLevel 3 /BaseVersion /1.7 >> >>
    /AcroForm 2 0 R
    /Type /Catalog
    /Pages 4 0 R
    /NeedsRendering true
    >>
    endobj

The root object should be a Catalog and the Type entry shows that it is.

AcroForm

The presence of the AcroForm entry indicates that the PDF contains an interactive form.

The entry references Object 2 which should therefore be an interactive form dictionary specifying the form.

Pages

The Pages entry references Object 4 which should therefore be a page tree node.

NeedsRendering

The NeedsRendering entry is another clue that the document contains a form.

Object 2

Object 2 start at offset 13263 which is 0x33CF and it looks like this.

    2 0 obj
    <</XFA 1 0 R >>
    endobj

According to Object 3, the Document Catalog, this should be an interactive form dictionary and the presence of the XFA entry confirms that it is.

XFA is the XML Forms Architecture. Amongst other things it supports scripting, and the Adobe implementation the scripting support includes Javascript, which in this context seems highly significant.

The XFA entry identifies the object which contains the XML which describes the XFA form.

The referenced object is Object 1, so ten to one on there are scripts in Object 1 and they do something unpleasant.

Object 1

Object 1 starts at offset 15.

According to Object 2 it should contain XML specifying an XFA form.

It is the biggest object by far so it definitely contains something, and presumably something nasty at that, so we’ll save it for later.

Object 4

Object 4 starts at 13443 which is 0x3483 and it looks like this

    4 0 obj
    <<
        /Count 1
        /Kids [5 0 R]
        /Type /Pages
    >>
    endobj

According to Object 3, the Document Catalog, this should be a page tree node with a Type of Pages and it is.

The value of the Kids (sic) entry is an Array of Object References of length one.

This entry identifies the children of this object, which are either collections of pages or individual pages which make up the document.

Object 5

Object 5 starts at 13499 which is 0x34BB and it looks like this

    5 0 obj
    <<
        /Parent 4 0 R
        /Type /Page
        /Contents 6 0 R
        /Resources << /Font <&lt/F1 <</BaseFont /Helvetica /Subtype /Type1 /Name /F1 >> >> >>
    >>
    endobj

According to Object 4, the Pages Object, this should be either another page tree node or page object. The Type entry Page tells us that it is a page object.

The Parent entry is a reference to Object 4 which is indeed the parent of this object.

The Contents entry is a reference to Object 6.

The Resources entry is an example of an entry whose value is a dictionary, which is itself an example of a dictionary with entries whose values are dictionaries.

Object 6

Object 6 starts at 13644 which is 0x354c.

    6 0 obj
    <</Length 23 >>
    stream
    BT /F1 24 Tf 100 100 Td
    endstream
    endobj

We know that it is the contents of the page represented by Object 5.

It is in fact a Content Stream which comprises a series of graphic operators and their operands, with the operands preceding their operators (cough, Postscript, cough).

For what its worth

    BT

is the ‘begin text’ operator.

    Tf

is the ‘set text font and size’ operator.

It is preceded by its operands

    /F1 24

with F1 being the font as defined in Object 5 and 24 being the size.

    Td

is the ‘move text’ operator.

It is preceded by its operands

    100 100

And that’s it. Not a very exciting page it has to be said.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

Anatomy Of A PDF: Part Two — Its One Of Those Start At The End Of Formats …

The portable document format started life as one of those formats where you have to start at the end, like ZIPs. It seems to have been all the rage at one time, ‘tho not so much these days.

Times having changed, and document formats where you have to start at the end having gone out of fashion, it is now possible to produce PDFs which can be read from the front, but where’s the fun in that ?

Anyway, here’s the very end of my PDF file

   ...
    
    0003620   0   0   0   0   0       n      \n   t   r   a   i   l   e   r
    0003630  \n   <   <   /   R   o   o   t       3       0       R       /
    0003640   S   i   z   e       7       >   >  \n   s   t   a   r   t   x
    0003650   r   e   f  \n   1   3   7   1   6  \n   %   %   E   O   F
    000365f

You can tell its the end because it says

    %%EOF

at the end, which is helpful.

As you can see its mostly ASCII, except for the line endings, which aren’t.

For some reason the internal data in a PDF file is often human readable and the actual contents aren’t.

Formtting the ASCII bit according to the line endings gives us

    ...
    
    trailer
    <</Root 3 0 R /Size 7 >>
    startxref
    13716
    %%EOF

Dictionaries

A lot of the internal data in a PDF file is in the form of dictionaries which are collections of key/value pairs as you might expect.

Dictionaries are delimited by pairs of angle brackets like so

    << ... >>

In the example above you can see that on the line following the word trailer is a dictionary

    ...
    
    <</Root 3 0 R /Size 7 >>
    
    ...

with two entries

    Root

and

    Size

Dictionary keys are Names and Names are always prefixed with a '/'.

The corresponding values may be of any PDF data type including Names, Arrays, and Dictionaries.

Dictionaries often contain a Type entry which specifies the type of the dictionary from which it is possible to determine, via the PDF specification, what other entries the dictionary must, or can, contain.

Dictionaries do not have to be written on a single line, it just so happens that the one in the trailer is.

Objects And Object References

A PDF document comprises a number of objects.

Each object has a number and a generation number.

An object is referred to using both its object number and its generation number.

In the internal data of a PDF file object references are written like so

    object-number genration-number 'R'

The value of the Root entry in the dictionary shown above

    3 0 R

is an example of an object reference as it appears in internal data.

Objects in a PDF file always start with the object number and the generation number so when following references to objects for example, you can always work out whether the object you’ve got is the one you are expecting.

The Trailer

The trailer of a PDF file starts with the line trailer, followed by a dictionary, followed by the line startxref,
followed by an offset followed by the EOF marker.

The Root entry in the trailer dictionary specifies the root object of the document. From the root object you can find all the other objects in the file one way or another.

The Size entry in the trailer dictionary specifies the total number of entries in the document’s cross-reference table.

The offset following the word startxref line is the offset of the document’s cross-reference table within the file.

The Cross-Reference Table

The cross-reference table starts at 13716 which is 0x3594.

    ...
    
    0003590   o   b   j  \n   x   r   e   f  \n   0       7  \n   0   0   0
    00035a0   0   0   0   0   0   0   0       6   5   5   3   5       f
    00035b0  \n   0   0   0   0   0   0   0   0   1   5       0   0   0   0
    00035c0   0       n      \n   0   0   0   0   0   1   3   2   6   3
    00035d0   0   0   0   0   0       n      \n   0   0   0   0   0   1   3
    00035e0   2   9   4       0   0   0   0   0       n      \n   0   0   0
    00035f0   0   0   1   3   4   4   3       0   0   0   0   0       n
    0003600  \n   0   0   0   0   0   1   3   4   9   9       0   0   0   0
    0003610   0       n      \n   0   0   0   0   0   1   3   6   4   4
    0003620   0   0   0   0   0       n      \n   t   r   a   i   l   e   r
    
    ...

Formatting the bit starting at 0x3594 by obeying the line endings gives us

    ...
    
    xref
    0 7
    0000000000 65535 f
    0000000015 00000 n
    0000013263 00000 n
    0000013294 00000 n
    0000013443 00000 n
    0000013499 00000 n
    0000013644 00000 n
    
    ...

The second line of the cross-reference table

    0 7

specifies the number of the first object which has an entry in this table, and the number of entries.

In this case the number of the first object is 0 and there are seven entries.

The following seven lines are the entries for objects 0 to 6.

Each entry specifies the offset of the object within the file, the generation number of the object and whether it is in use (‘n’) or free (‘f’).

For example, the second entry

    0000000015 00000 n

tells us that object 1 is at offset 15 within the file, its generation number is 0, and it is in use.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

Anatomy Of A PDF: Part One — Enter A PDF Stage Left

Filed under: Document Format, PDF — Tags: , , — Simon Lewis @ 6:11 am

Some nice people have just sent me a PDF file.

I didn’t ask them to but they sent it nonetheless.

Its a shame they cannot spell the word ‘invoice’ properly but it’s the thought that counts isn’t it ?

Since I once made the mistake of finding out how PDFs ‘work’ so to speak, you really don’t want to know but I am going to tell you anyway, I was curious as to what exactly my unsolicited PDF file might contain instead of an ‘inovice’ and what would have happened had I been unwise enough to attempt to open it using, I would assume, acrobat reader running on windows.

Here’s the edited output of

    hexdump -c

run on my little, its only 14KB, PDF.


    0000000   %   P   D   F   -   1   .   5  \n   %  [.] [.] [.] [.] \n   1
    0000010       0       o   b   j  \n   <   <   /   F   i   l   t   e   r
    0000020       [   /   F   l       /   F   l   ]       /   L   e   n   g
    0000030   t   h       1   3   1   7   8       >   >  \n   s   t   r   e

    ...
    
    00033c0   d   s   t   r   e   a   m  \n   e   n   d   o   b   j  \n   2
    00033d0       0       o   b   j  \n   <   <   /   X   F   A       1
    00033e0   0       R       >   >  \n   e   n   d   o   b   j  \n   3
    00033f0   0       o   b   j  \n   <   <   /   E   x   t   e   n   s   i
    0003400   o   n   s       <   <   /   A   D   B   E       <   <   /   E
    0003410   x   t   e   n   s   i   o   n   L   e   v   e   l       3
    0003420   /   B   a   s   e   V   e   r   s   i   o   n       /   1   .
    0003430   7       >   >       >   >       /   A   c   r   o   F   o   r
    0003440   m       2       0       R       /   T   y   p   e       /   C
    0003450   a   t   a   l   o   g       /   P   a   g   e   s       4
    0003460   0       R       /   N   e   e   d   s   R   e   n   d   e   r
    0003470   i   n   g       t   r   u   e       >   >  \n   e   n   d   o
    0003480   b   j  \n   4       0       o   b   j  \n   <   <   /   C   o
    0003490   u   n   t       1       /   K   i   d   s       [   5       0
    00034a0       R   ]       /   T   y   p   e       /   P   a   g   e   s
    00034b0       >   >  \n   e   n   d   o   b   j  \n   5       0       o
    00034c0   b   j  \n   <   <   /   P   a   r   e   n   t       4       0
    00034d0       R       /   T   y   p   e       /   P   a   g   e       /
    00034e0   C   o   n   t   e   n   t   s       6       0       R       /
    00034f0   R   e   s   o   u   r   c   e   s       <   <   /   F   o   n
    0003500   t       <   <   /   F   1       <   <   /   B   a   s   e   F
    0003510   o   n   t       /   H   e   l   v   e   t   i   c   a       /
    0003520   S   u   b   t   y   p   e       /   T   y   p   e   1       /
    0003530   N   a   m   e       /   F   1       >   >       >   >       >
    0003540   >       >   >  \n   e   n   d   o   b   j  \n   6       0
    0003550   o   b   j  \n   <   <   /   L   e   n   g   t   h       2   3
    0003560       >   >  \n   s   t   r   e   a   m  \n   B   T       /   F
    0003570   1       2   4       T   f       1   0   0       1   0   0
    0003580   T   d  \n   e   n   d   s   t   r   e   a   m  \n   e   n   d
    0003590   o   b   j  \n   x   r   e   f  \n   0       7  \n   0   0   0
    00035a0   0   0   0   0   0   0   0       6   5   5   3   5       f
    00035b0  \n   0   0   0   0   0   0   0   0   1   5       0   0   0   0
    00035c0   0       n      \n   0   0   0   0   0   1   3   2   6   3
    00035d0   0   0   0   0   0       n      \n   0   0   0   0   0   1   3
    00035e0   2   9   4       0   0   0   0   0       n      \n   0   0   0
    00035f0   0   0   1   3   4   4   3       0   0   0   0   0       n
    0003600  \n   0   0   0   0   0   1   3   4   9   9       0   0   0   0
    0003610   0       n      \n   0   0   0   0   0   1   3   6   4   4
    0003620   0   0   0   0   0       n      \n   t   r   a   i   l   e   r
    0003630  \n   <   <   /   R   o   o   t       3       0       R       /
    0003640   S   i   z   e       7       >   >  \n   s   t   a   r   t   x
    0003650   r   e   f  \n   1   3   7   1   6  \n   %   %   E   O   F
    000365f


I’ve chopped out the bit in the middle because, as we will see, in its current format its not very interesting at all.

You can tell its a PDF because it says so at the front.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

August 22, 2014

And Another One: Part Twenty Two — The Actual Fix

And after all that, how has it actually been fixed ?

Like this apparently, at least in the short term.

    Add API to check certificate chain signatures
    
    Add hidden API to check certificate chain signatures when needed. The
    getCertificates implementation returns a list of all the certificates and
    chains and would expect any caller interested in verifying actual chains
    to call getCodeSigners instead.
    
    We add this hidden constructor as a stop-gap until we can switch callers
    over to getCodeSigners.

The implication of the above is that the long term fix is for the PackageParser loadCertificates method to call the JarEntry getCodeSigners method rather than the getCertificates method as it does now.

The getCodeSigners method is declared like this

    public CodeSigner[] getCodeSigners()

On its own this is not going to achieve anything since getCodeSigners does exactly the same amount of certificate chain verification as the original version of getCertificates, i.e., none.

What it does do is package the certificates up into instances of java.security.CodeSigner for you.

A CodeSigner contains a java.security.cert.CertPath.

Once you have one of those you can either validate the certificates in it yourself or hand the whole thing to a
java.security.cert.CertPathValidator who will do it for you, assuming you can work out how to set up the right CertPathParameters instance.

And after all that and assuming everything verifies you can collect all the constituent Certificates together and then turn them into Signatures just like before.

An alternative long term solution might be to come up with a proper Application Signature abstraction and a proper Security Policy abstraction.

Then, rather have random bits of code deciding to implement ad hoc security policies because they can get at the internals of Signatures, they would have to use the Security Policy !


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

And Another One: Part Twenty One — The Package Manager Revisited

File

    $(ANDROID_SRC)/frameworks/base/core/java/android/webkit/PluginManager.java

Source

    ...

    private static boolean containsPluginPermissionAndSignatures(PackageInfo pkgInfo) {
    
        // check if the plugin has the required permissions
        String permissions[] = pkgInfo.requestedPermissions;
        if (permissions == null) {
            return false;
        }
        boolean permissionOk = false;
        for (String permit : permissions) {
            if (PLUGIN_PERMISSION.equals(permit)) {
                permissionOk = true;
                break;
            }
        }
        if (!permissionOk) {
            return false;
        }
    
        // check to ensure the plugin is properly signed
        Signature signatures[] = pkgInfo.signatures;
        if (signatures == null) {
            return false;
        }
        if (SystemProperties.getBoolean("ro.secure", false)) {
            boolean signatureMatch = false;
            for (Signature signature : signatures) {
                for (int i = 0; i < SIGNATURES.length; i++) {
                    if (SIGNATURES[i].equals(signature)) {
                        signatureMatch = true;
                        break;
                    }
                }
            }
            if (!signatureMatch) {
                return false;
            }
        }
        
        return true;
    }
    
    ...

This is where we came in.

As an aside, the Signature handling code in this method bears an uncanny resemblance to the PackageManagerService checkSignaturesLP method code as it was in 2008 which may be a clue as to where it originated.

Anyway, given what we now know, how do we fix it ?

In what follows I will use the term signer certificate to mean the certificate which specifies the public key corresponding to the private key with which the plugin was signed.

Perhaps the easiest thing to do is start by seeing if we can find the hard-wired certificate at all before worrying whether someone is trying to pull a fast one.

        ...
    
            int index       = 0;
            int nSignatures = signatures.length;
    
            while (index < nSignatures)
            {
                if (signatures[index].equals(SIGNATURE_1))
                {
                    break;
                }
                ++index;
            }

If we don’t find it we return false.


            if (index == nSignatures)
            {
                // not found
                return false;
            }

If we find it and there is only one Signature then the hard-wired certificate is the signer certificate so we return true.


            else
            if (nSignatures == 1)
            {
                return true;
            }

If we find the hard-wired certificate at index 0 we know it is the first certificate of the first certificate chain so, irrespective of the number of Signatures, we know it is a signer certificate.

Whether we ought to know this is debateable, but given that we already know that Signatures are encoded certificates and are taking advantage of the fact, its a bit late in the day to worry about it.


            else
            if (index == 0)
            {
                return true;
            }

If none of the above are true then there is nothing for it. We are going to have to establish that the hard-wired certificate is genuinely part of a certificate chain.

We know that the hard-wired certificate is self-signed so it has to appear standalone, i.e., in a certificate chain of length one, or as the last certificate in a chain of length two or greater.

We can determine whether it is at the end of a certificate chain by checking to see who issued the preceding certificate.

Assuming that we have a method makeCertificate for making magically making X509Certificates, then


            X509Certificate ourCert    = makeCertificate(signatures[index--]);
            X509Certificate previous   = makeCertificate(signatures[index--]);
    

If the issuer of the previous certificate wasn’t the subject of the hard-wired certificate then the hard-wired certificate is a signer certificate so we can return true.


            if (!previous.getIssuerDN().equals(ourCert.getSubjectDN()))
            {
                return true;
            }

If the hard-wired certificate is at the end of a certificate chain then we should be able to use the public key from it to verify the previous one.


            try
            {
                previous.verify(ourCert.getPublicKey());
            }
            catch (Exception e)
            {
                return false;
            }

If the verification succeds we keep going until we reach the front of the certificate chain.


            X509Certificate current = previous;

            while (index > 0)
            {
                previous = makeCertificate(signatures[index--]);
                if (!previous.getIssuerDN().equals(current.getSubjectDN()))
                {
                    // we've successfully reached the front of the chain
                    break;
                }
                try
                {
                    previous.verify(current.getPublicKey());
                }
                catch (Exception e)
                {
                    return false;
                }
                current = previous;
            }

If we make it to the first certificate in the chain in one piece we have a valid certificate chain so we return true;

And there you have it.

Not only does it fix the problem but as a bonus it manages to tightly couple the PluginManager to a whole bunch of classes that do not even know it exists.

Note that it would be a lot simpler if there were some known invariants.

For example, if the hard-wired certificate must be the signer certificate and there cannot be multiple signers then the whole thing collapses down to

    return (signatures[length] == 0) && signatures[0].equals(SIGNATURE_1);

which is a lot less fragile.

Now all we need to do is find all the other code that is implementing ad-hoc security policies on the basis of knowing that Signatures are encoded certificates and fix that as well.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

August 21, 2014

And Another One: Part Twenty — Abstraction Failure Or Why Does The Class android.content.pm.Signature Exist ?

I have already quoted the class documentation comment for the android.content.pm.Signature class before but here it is again.

Opaque, immutable representation of a signature associated with an application package.

It is the comment which appeared in the first version of the class and despite subsequent changes it has never been updated.

Even in the original version it was not true as the class defined this method

    ...
    
    /**
     * @return the contents of this signature as a byte array.
     */
    public byte[] toByteArray() {
        byte[] bytes = new byte[mSignature.length];
        System.arraycopy(mSignature, 0, bytes, 0, mSignature.length);
        return bytes;
    }
    
    ...

Why should it be possible to turn the contents of something supposedly opaque into a byte array ?

It looks fairly innocuous but its a major abstraction failure all on its own so lets take the documentation at its word and get rid of it, and while we are at it we will get rid of

    public char[] toChars()

as well as

    public char[] toChars(char[] existingArray, int[] outLen)

not to mention

    public String toCharsString()

just to be on the safe side.

For an opaque object it defines a remarkable number of ways of finding out whats in it, and we’re not finished yet because its

    Parcelable

and you never know, if somebody got really desperate they might write it to a Parcel then go and read it out themselves as a byte array or something.

Best not to put temptation in their way, so we’ll get rid of all of that as well.

Now what can you do with a Signature as originally defined which is genuinely opaque and immutable ?

Not much to be honest.

What’s left are the equals and hashCode methods and as we have already seen all they do is sub-contract the heavy-lifting to the java.util.Arrays class.

To be fair in its modified form it hides the fact that its really just a wrapper around byte array pretty well but nothing about it realls shouts out Signature ! does it ?

Still its an improvement on its original form where it completely fails to hide the fact that its really just a wrapper around a byte array by giving you access to it.

So, to return to the original question, why does it exist at all in its current form, or to put it another why isn’t just a wrapper a Set of encoded certificates ?

The Past Is A Foreign Country, They Do Things Differently There

One possible explanation is that once upon a time Application signatures used to be defined in a different way and that the Signature class has been re-purposed or something.

Looking for the contemporary version of the PackageManegerService class we discover it in the package.

    com.android.server

That’s right, there was a time when the PackageManegerService class was not sufficiently important to warrant its own package ! Hard to believe isn’t it ?

Now lets see how they used to compare Signatures in those days.

Well they didn’t use compareSignatures thats for sure because its not defined.

So how did grantSignaturePermission work if there was no compareSignatures method ? Answer it didn’t because its not defined.

So where are signature permissions granted then ?

A ha, grantPermissionsLP and its calling checkSignaturesLP, should have guessed, and here it is

    ...
    
    int checkSignaturesLP(PackageParser.Package p1, PackageParser.Package p2) {
        if (p1.mSignatures == null) {
            return p2.mSignatures == null
                    ? PackageManager.SIGNATURE_NEITHER_SIGNED
                    : PackageManager.SIGNATURE_FIRST_NOT_SIGNED;
        }
        if (p2.mSignatures == null) {
            return PackageManager.SIGNATURE_SECOND_NOT_SIGNED;
        }
        final int N1 = p1.mSignatures.length;
        final int N2 = p2.mSignatures.length;
        for (int i=0; i<N1; i++) {
            boolean match = false;
            for (int j=0; j<N2; j++) {
                if (p1.mSignatures[i].equals(p2.mSignatures[j])) {
                    match = true;
                    break;
                }
            }
            if (!match) {
                return PackageManager.SIGNATURE_NO_MATCH;
            }
        }
        return PackageManager.SIGNATURE_MATCH;
    }

    ...

This looks familiar.

Thats all almost exactly the code structure same as the definition of the compareSignatures method now except that the semantics of Application signatures are completely different !

No Sets there, that’s a ‘member in common’ test !

Whoah, scary ! To be honest I wasn’t expecting that !

(Enough with the exclamation marks.)

Presumably Android Application Signature arrays were constructed somewhat differently in those days or that test does not make a whole lot of sense.

Nope, PackageParser.loadCertificates looks to be nigh on identical.

OK, so at least they used to verify the certificate chains properly surely. No they didn’t do that either. JarUtils.createChain is identical.

Oh dear.

The full implications of that are left as an exercise for the reader but bear in mind that quite possibly there is something somewhere doing something which means it is not as bad as it looks.

Code Rot

Given the way it seems Signature instances were used originally it turns out that the Signature class is not quite as pointless as it appears to be now.

It doesn’t explain why the abstraction was completely broken from day one but it does explain why, for example, the class represents a single encoded certificate rather than a set of them.

The combination of the class being public and the abstraction failure presumably meant that it was felt that it was not possible to substantially modify its behaviour and what it represented.

However, given the major change in semantics that occurred, changing the class substantively so that any code that relied on it broke would have had the benefit of revealing what if anything was relying on the assumption that it contained a certificate and that that certificate could be accessed.

Having said that it does not explain why at least internally a class could not have been defined that simply encapulated all the details of what constitutes an Application’s signature, how one is constructed and how they are compared.

Afterword

When I started writing this although I thought it possible that something about how Android Application signatures worked might have changed between the point at which the Signature class was originally defined and now I didn’t really didn’t know one way or another. It was just a possible explanation for the existence of a fairly pointless class.

The earliest version of the Android source I have immediately accessible is the version I changed to get running standalone, that is, on a standard JVM rather than Dalvik, which dates from late 2008.

Although I clearly changed the PackageManagerService class for some reason at that time I have no recollection of looking at this area at all, so I was genuinely surprised to discover

  1. that it really had changed a lot, and

  2. that at that point it seems it was completely broken

The earliest version of the PackageManagerService class I have locally that constructs sets of Signatures in order to perform a comparison is from March 2011 where it is being done by the checkSignaturesLP method.

At that point there is still no compareSignatures method defined.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

And Another One: Part Nineteen — Surely That Can’t Be The Only Thing The PackageManagerService Does With Signatures ?

It ought to be but then there is this which is just horrid

1.0 PackageManagerService.getUidForVerifier

File

    $(ANDROID_SRC)/frameworks/base/services/java/com/android/server/pm/PackageManagerService.java

Source

    ...
    
    private int getUidForVerifier(VerifierInfo verifierInfo) {
        synchronized (mPackages) {
            final PackageParser.Package pkg = mPackages.get(verifierInfo.packageName);
            if (pkg == null) {
                return -1;
            } else if (pkg.mSignatures.length != 1) {
                Slog.i(TAG, "Verifier package " + verifierInfo.packageName
                        + " has more than one signature; ignoring");
                return -1;
            }
    
            /*
             * If the public key of the package's signature does not match
             * our expected public key, then this is a different package and
             * we should skip.
             */
    
            final byte[] expectedPublicKey;
            try {
                final Signature verifierSig = pkg.mSignatures[0];
                final PublicKey publicKey = verifierSig.getPublicKey();
                expectedPublicKey = publicKey.getEncoded();
            } catch (CertificateException e) {
                return -1;
            }
    
            final byte[] actualPublicKey = verifierInfo.publicKey.getEncoded();
    
            if (!Arrays.equals(actualPublicKey, expectedPublicKey)) {
                Slog.i(TAG, "Verifier package " + verifierInfo.packageName
                        + " does not have the expected public key; ignoring");
                return -1;
            }
    
            return pkg.applicationInfo.uid;
        }
    }
    
    ... 

This method single-handedly drives a coach and horses through the idea that a Signature is a, to quote the class documentation

Opaque, immutable representation of a signature associated with an application package.

Immutable yes, opaque not so much.

1.1 Signature.getPublicKey

The Signature class did not have a getPublicKey method originally although it has always had a toByteArray method so it has never been as opaque as it ought to have been but know you can ask it to do this !

File

    $(ANDROID_SRC)/frameworks/base/core/java/android/content/pm/Signature.java

Source

    ...
    
    public PublicKey getPublicKey() throws CertificateException {
        final CertificateFactory certFactory = CertificateFactory.getInstance("X.509");
        final ByteArrayInputStream bais = new ByteArrayInputStream(mSignature);
        final Certificate cert = certFactory.generateCertificate(bais);
        return cert.getPublicKey();
    } ...

In fact you cannot ask it to do that because despite being a public method it has been hidden which is bizarre because you can always do the whole thing yourself.

Even more bizarrely it looks as though getUidForVerifier is the only method in the system that calls it.

Why add and a then hide a method that is only used by a single method in the PackageManagerService class ?

Why not do it locally ? The PackageManagerService class is almost 11000 lines long, 5 more aren’t going to make much difference.

1.2 What Does getUidForVerifier Do ?

What the method does is to look up the Package specified by the VerifierInfo‘s packageName instance variable.

If the Package exists the method then checks the length of the Package’s array of Signatures.

If the length is not one the method returns -1.

The length of an Application’s Signature array is going to be greater than one, if either

  • the Application has been signed more than once, or

  • there was more than one certificate in the signed signature file

No idea what is significant about either of these cases except that by excluding them it does mean that this method is unaffected by the failure to verify certificate chains.

If the length of an Application’s signature array is one then it necessarily contains the Signature constructed from the certificate which specifies the public key corresponding to the private key with which it was signed.

If the public key specified by the VerifierInfo‘s publicKey instance variable matches the public key extracted from the Signature then …

Then what exactly ?

Then you know you have the named Application and that it was signed by the private key corresponding to the specified public key.

1.4 Why A Public Key ?

If the VerifierInfo specified a name and an array of Signatures getUidForVerifier could have used the compareSignatures method rather than levering open a Signature, so why the public key ?

The VerifierInfo is obtained from the package-verifier element of an Android Application Manifest by this method

File

    $(ANDROID_SRC)/frameworks/base/core/java/android/content/pm/Signature.java

Source

    ...
    
    private static VerifierInfo parseVerifier(Resources res, XmlPullParser parser,
            AttributeSet attrs, int flags, String[] outError) throws XmlPullParserException,
            IOException {
        final TypedArray sa = res.obtainAttributes(attrs,
                com.android.internal.R.styleable.AndroidManifestPackageVerifier);
    
        final String packageName = sa.getNonResourceString(
                com.android.internal.R.styleable.AndroidManifestPackageVerifier_name);
    
        final String encodedPublicKey = sa.getNonResourceString(
                com.android.internal.R.styleable.AndroidManifestPackageVerifier_publicKey);
    
        sa.recycle();
    
        if (packageName == null || packageName.length() == 0) {
            Slog.i(TAG, "verifier package name was null; skipping");
            return null;
        } else if (encodedPublicKey == null) {
            Slog.i(TAG, "verifier " + packageName + " public key was null; skipping");
        }
    
        EncodedKeySpec keySpec;
        try {
            final byte[] encoded = Base64.decode(encodedPublicKey, Base64.DEFAULT);
            keySpec = new X509EncodedKeySpec(encoded);
        } catch (IllegalArgumentException e) {
            Slog.i(TAG, "Could not parse verifier " + packageName + " public key; invalid Base64");
            return null;
        }
    
        /* First try the key as an RSA key. */
        try {
            final KeyFactory keyFactory = KeyFactory.getInstance("RSA");
            final PublicKey publicKey = keyFactory.generatePublic(keySpec);
            return new VerifierInfo(packageName, publicKey);
        } catch (NoSuchAlgorithmException e) {
            Log.wtf(TAG, "Could not parse public key because RSA isn't included in build");
            return null;
        } catch (InvalidKeySpecException e) {
            // Not a RSA public key.
        }
    
        /* Now try it as a DSA key. */
        try {
            final KeyFactory keyFactory = KeyFactory.getInstance("DSA");
            final PublicKey publicKey = keyFactory.generatePublic(keySpec);
            return new VerifierInfo(packageName, publicKey);
        } catch (NoSuchAlgorithmException e) {
            Log.wtf(TAG, "Could not parse public key because DSA isn't included in build");
            return null;
        } catch (InvalidKeySpecException e) {
            // Not a DSA public key.
        }
    
        return null;
    }

    ...

As you can see the public key is created by Base64 decoding a String and then constructing an instance of

    java.security.spec.X509EncodedKeySpec

with the resulting bytes.

This is then turned into an instance of

    java.security.PublicKey

using brute force, which is interesting.

Given that the Android Application Manifest is an XML file this is perhaps not the best way to specify a public key.

Its not as though it hasn't been done before. See here for a different approach that even manages to include the information about what type of key it is.

There is also the question of the format of the data that needs to be specified.

The X509EncodedKeySpec constructor takes an encoded instance of one of these.

    SubjectPublicKeyInfo ::= SEQUENCE {
    algorithm AlgorithmIdentifier,
    subjectPublicKey BIT STRING }

which is defined in X.509

Now where would you get one of those ?

Given that

  1. it is defined in X.509 and

  2. that specifying a public keys is what a certificates is for

its not unreasonable to suppose that you might find one in an X.509 certificate and in fact you would, its actually the payload of the certificate.

So why a bit of a certificate rather than all of the certificate ?

If it was as certificate at least you could maintain the pseudo-opacity of the Signature class and you could use the compareSignatures method so no special-case code required.

The obvious advantages of specifying a public key are that it is smaller and it works with Signatures constructed with any certificate that specifies the public key rather than a specific certificate.

Given that Android package verifiers are something of an enigma there could be other reasons why it is necessary to specify a public key rather than a certificate or a certificate chain but they would need to be very compelling to justify the getUidForVerifier method in its current form.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog's author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

And Another One: Part Eighteen — What Are Android Application Signatures For ? (Continued)

1.0 What Can You Prove Using the compareSignatures Method ?

If you have two Applications A and B and you pass the Signature array of Application A and the Signature array of Application B to the method

   compareSignatures

and it returns

   PackageManager.SIGNATURE_MATCH

then you have proved that Application A and Application B were signed with the same private key(s).

2.0 Proof

Notation

C(puk) the Certificate specifying the public key puk
PrK(kp) the Private Key of the key pair kp
PuK(kp) the Public Key of the key pair kp
S(x) the set of Signatures constructed from the array of Signatures of an Application x
Sig(c) the Signature constructed using the encoding of a Certificate c

Assertion

if an Application X is signed with the private key PrK(KP) then the signature set S(X) must contain Sig(C(PuK(KP))).

Assume we have two signed Applications A and B and a key pair KP.

Let the Application A be signed using PrK(KP).

Let S(A) contain the single Signature Sig(C(PuK(KP))).

Call the compareSignatures method passing it the Signature array of Application A and the Signature array of Application B.

If the call returns PackageManager.SIGNATURE_MATCH then

    S(A) == S(B)

Therefore S(B) must contain the single Signature Sig(C(PuK(KP))).

There is only one Signature in S(B) which is Sig(C(PuK(KP))) so given our assertion Application B must have been signed with PrK(KP).

Alternatively, let S(A) contain Sig(C(PuK(KP))) plus N Signatures constructed from the certificates of the associated certificate chain.

If S(B) is equal to S(A), S(B) must contain the N Signatures that are not Sig(C(PuK(KP))) plus one other Signature.

Since the sets are equal the other Signature must be Sig(C(PuK(KP))).

Note that the converse is not true.

If the compareSignatures method returns PackageManager.SIGNATURE_NO_MATCH it does not prove that the Applications were not signed with the same private key.

3.0 And Exactly Why Is That Useful ?

It is useful because it is precisely what the PackageManagerService wants to know during Application installation, which is an astonishing coincidence, or perhaps not.

Those aspects of the Android Application security model which are embodied by the PackageManagerService are all based on establishing whether two applications are signed with the same private key(s).

3.1 PackageManagerService.grantSignaturePermission

As an example consider the grantSignaturePermission method which is used by the PackageManagerService when installing an Application.

A Signature permission is

A permission that the system grants only if the requesting application is signed with the same certificate as the application that declared the permission. If the certificates match, the system automatically grants the permission without notifying the user or asking for the user’s explicit approval.

Substituting signed with the same private key for the endlessly misleading

signed with the same certificate

formulation, you can see that the compareSignatures method is going to tell you exactly what you need to know.

File

    $(ANDROID_SRC)/frameworks/base/services/java/com/android/server/pm/PackageManagerService.java

Source

    ...
    
    private boolean grantSignaturePermission(String perm, PackageParser.Package pkg,
                                          BasePermission bp, HashSet<String> origPermissions) {
        boolean allowed;
        allowed = (compareSignatures(
                bp.packageSetting.signatures.mSignatures, pkg.mSignatures)
                        == PackageManager.SIGNATURE_MATCH)
                || (compareSignatures(mPlatformPackage.mSignatures, pkg.mSignatures)
                        == PackageManager.SIGNATURE_MATCH);
        if (!allowed && (bp.protectionLevel
                & PermissionInfo.PROTECTION_FLAG_SYSTEM) != 0) {
            if (isSystemApp(pkg)) {
                // For updated system applications, a system permission
                // is granted only if it had been defined by the original application.
                if (isUpdatedSystemApp(pkg)) {
                    final PackageSetting sysPs = mSettings
                            .getDisabledSystemPkgLPr(pkg.packageName);
                    final GrantedPermissions origGp = sysPs.sharedUser != null
                            ? sysPs.sharedUser : sysPs;
                    if (origGp.grantedPermissions.contains(perm)) {
                        allowed = true;
                    } else {
                        // The system apk may have been updated with an older
                        // version of the one on the data partition, but which
                        // granted a new system permission that it didn't have
                        // before.  In this case we do want to allow the app to
                        // now get the new permission, because it is allowed by
                        // the system image.
                        allowed = false;
                        if (sysPs.pkg != null) {
                            for (int j=0;
                                    j<sysPs.pkg.requestedPermissions.size(); j++) {
                                if (perm.equals(
                                        sysPs.pkg.requestedPermissions.get(j))) {
                                    allowed = true;
                                    break;
                                }
                            }
                        }
                    }
                } else {
                    allowed = true;
                }
            }
        }
        if (!allowed && (bp.protectionLevel
                & PermissionInfo.PROTECTION_FLAG_DEVELOPMENT) != 0) {
                // For development permissions, a development permission
                // is granted only if it was already granted.
                allowed = origPermissions.contains(perm);
        }
        return allowed;
    }
    
    ...

The grantSignaturePermission method works with both and signature and signatureOrSystem permissions.

The bulk of the method is some special case code for a systemOrSignature permission, but you can see that before that the method is indeed using the compareSignatures method to decide whether to grant a signature permission.

3.2 PackageManager.setInstallerPackageName

The setInstallerPackageName method is the back-end method which implements the functionality accesible via the PackageManager method of the same name.

The documentation for the PackageManager version is as follows

Change the installer associated with a given package. There are limitations
on how the installer package can be changed; in particular:

  • A SecurityException will be thrown if installerPackageName
    is not signed with the same certificate as the calling application.

  • A SecurityException will be thrown if targetPackage already
    has an installer package, and that installer package is not signed with
    the same certificate as the calling application.

The appearance, yet again, of the stock phrase

signed with the same certificate

leads us to expect that the method will use the compareSignatures method and indeed it does.

File

    $(ANDROID_SRC)/frameworks/base/services/java/com/android/server/pm/PackageManagerService.java

Source

    ...
    
    public void setInstallerPackageName(String targetPackage, String installerPackageName) {
        final int uid = Binder.getCallingUid();
        // writer
        synchronized (mPackages) {
            PackageSetting targetPackageSetting = mSettings.mPackages.get(targetPackage);
            if (targetPackageSetting == null) {
                throw new IllegalArgumentException("Unknown target package: " + targetPackage);
            }
    
            PackageSetting installerPackageSetting;
            if (installerPackageName != null) {
                installerPackageSetting = mSettings.mPackages.get(installerPackageName);
                if (installerPackageSetting == null) {
                    throw new IllegalArgumentException("Unknown installer package: "
                            + installerPackageName);
                }
            } else {
                installerPackageSetting = null;
            }
    
            Signature[] callerSignature;
            Object obj = mSettings.getUserIdLPr(uid);
            if (obj != null) {
                if (obj instanceof SharedUserSetting) {
                    callerSignature = ((SharedUserSetting)obj).signatures.mSignatures;
                } else if (obj instanceof PackageSetting) {
                    callerSignature = ((PackageSetting)obj).signatures.mSignatures;
                } else {
                    throw new SecurityException("Bad object " + obj + " for uid " + uid);
                }
            } else {
                throw new SecurityException("Unknown calling uid " + uid);
            }
    
            // Verify: can't set installerPackageName to a package that is
            // not signed with the same cert as the caller.
            if (installerPackageSetting != null) {
                if (compareSignatures(callerSignature,
                        installerPackageSetting.signatures.mSignatures)
                        != PackageManager.SIGNATURE_MATCH) {
                    throw new SecurityException(
                            "Caller does not have same cert as new installer package "
                            + installerPackageName);
                }
            }
    
            // Verify: if target already has an installer package, it must
            // be signed with the same cert as the caller.
            if (targetPackageSetting.installerPackageName != null) {
                PackageSetting setting = mSettings.mPackages.get(
                        targetPackageSetting.installerPackageName);
                // If the currently set package isn't valid, then it's always
                // okay to change it.
                if (setting != null) {
                    if (compareSignatures(callerSignature,
                            setting.signatures.mSignatures)
                            != PackageManager.SIGNATURE_MATCH) {
                        throw new SecurityException(
                            "Caller does not have same cert as old installer package "
                            + targetPackageSetting.installerPackageName);
                    }
                }
            }
    
            // Okay!
            targetPackageSetting.installerPackageName = installerPackageName;
            scheduleWriteSettingsLocked();
        }
    }
    
    ...

4.0 But What About The Certificate Chain Verification Issue ?

The compareSignatures method is immune to the certificate chain verification issue because it tests for the equality of sets of Signatures.

You cannot construct an Application X such that S(X) is equal to S(A) for some application A if you do not know the private key PrK(KPA) used to sign A.

The failure to verify the certificate chain means that until recently (?) you could, with a little effort, add Application A‘s entire certificate chain to the signed signature file of Application X.

The result would be that

    S(X)

would contain the Signatures corresponding to the certificates in Application A‘s certificate chain.

However, since you do not know

    PrK(KPA)

you must sign it with a different private key

    PrK(KPX)

As a result, as long as the assertion above holds then

    Sig(C(PuK(KPX)))

must be in

    S(X)

but it is not in

    S(A)

therefore the two sets are not equal.

August 17, 2014

And Another One: Part Seventeen — What Are Android Application Signatures For ?

Working out exactly why the PackageManagerService goes to all the trouble of acquiring an application’s signatures at installation time is easier said than done.

A top-down approach is rendered extremely difficult by the sheer size of the class,

The version of the file

    $(ANDROID_SRC)/frameworks/base/services/java/com/android/server/pm/PackageManagerService.java

which I am currently working with clocks in at

   10979

lines which is insane and makes the class effectively incomprehensible as a whole, not to mention unmaintainable.

How about bottom up ?

A search for the word

   signatures

comes up with a mere

   78

matches, one of which is

   compareSignatures

which looks as though it might be interesting.

Narrowing the search to

   compareSignatures

brings it down to

    12

Of these one is the method definition and the other eleven are calls to it so it looks as though it might be a good place to start at least.

1.0 PackageManagerService.compareSignatures

File

    $(ANDROID_SRC)/frameworks/base/services/java/com/android/server/pm/PackageManagerService.java

Source

    ...
    
    static int compareSignatures(Signature[] s1, Signature[] s2) {
        if (s1 == null) {
            return s2 == null
                    ? PackageManager.SIGNATURE_NEITHER_SIGNED
                    : PackageManager.SIGNATURE_FIRST_NOT_SIGNED;
        }
        if (s2 == null) {
            return PackageManager.SIGNATURE_SECOND_NOT_SIGNED;
        }
        HashSet<Signature> set1 = new HashSet<Signature>();
        for (Signature sig : s1) {
            set1.add(sig);
        }
        HashSet<Signature> set2 = new HashSet<Signature>();
        for (Signature sig : s2) {
            set2.add(sig);
        }
        // Make sure s2 contains all signatures in s1.
        if (set1.equals(set2)) {
            return PackageManager.SIGNATURE_MATCH;
        }
        return PackageManager.SIGNATURE_NO_MATCH;
    }

    ...

As written the method is pretty opaque.

Re-written like this it is hopefully clearer what is going on.

    ...
    
    static int compareSignatures(Signature[] s1, Signature[] s2)
    {
        if ((s1 == null) && (s2 == null))
        {
            return PackageManager.SIGNATURE_NEITHER_SIGNED;
        }
        else
        if (s1 == null)
        {
            return PackageManager.SIGNATURE_FIRST_NOT_SIGNED;
        }
        else
        if (s2 == null)
        {
             return PackageManager.SIGNATURE_SECOND_NOT_SIGNED;
        }
        else
        {
            Set<Signature>   set1 = makeSignatureSet(s1);
            Set<Signature>   set2 = makeSignatureSet(s2);
                
            return
                set1.equals(set2)
                        ?
                        PackageManager.SIGNATURE_MATCH
                        :
                        PackageManager.SIGNATURE_NO_MATCH;
        }
    }
                
    private static Set<Signature> makeSignatureSet(Signature[] theSignatures)
    {
        HashSet<Signature> set = new HashSet<Signature>();
                        
        for (Signature s : theSignatures)
        {
            set.add(s);
        }
        return set;
    }
    
    ...

The fourth case is the interesting one.

The two arrays of Signatures are converted to Sets of Signatures.

Only if the two Sets are equal are the two arrays of Signatures considered to match.

2.0 The Semantics Of The compareSignatures Method

At first glance the semantics of the compareSignatures method may look very straight forward but appearances can be deceptive.

The behaviour of the method is completely determined by the behaviour of the Sets that are constructed and compared.

To be sure what the method is doing it is necessary to understand how Sets of Signatures behave.

2.1 Set Construction

The documentation for the add method of the java.util.Set interface states

Adds the specified object to this set. The set is not modified if it already contains the object.

which means what exactly ?

The documentation for the add method of the java.util.HashSet class manages, if anything, to be even more vague

Adds the specified object to this HashSet if not already present.

so there is nothing for it but to look at the source

File

    $(ANDROID_SRC)/libcore/luni/src/main/java/java/util/HashSet.java

Source

    ...
    
    @Override
    public boolean add(E object) {
        return backingMap.put(object, this) == null;
    }

    ...

Oh great !

The instance variable backingMap is declared like this.

    ...
    
    transient HashMap<E, HashSet<E>> backingMap;
        
    ...

so its off to the HashMap source we go

File

    $(ANDROID_SRC)/libcore/luni/src/main/java/java/util/HashMap.java

Source

    ...
    
    @Override public V put(K key, V value) {
        if (key == null) {
            return putValueForNullKey(value);
        }

        int hash = secondaryHash(key);
        HashMapEntry<K, V>[] tab = table;
        int index = hash & (tab.length - 1);
        for (HashMapEntry<K, V> e = tab[index]; e != null; e = e.next) {
            if (e.hash == hash && key.equals(e.key)) {
                preModify(e);
                V oldValue = e.value;
                e.value = value;
                return oldValue;
            }
        }
        
        // No entry for (non-null) key is present; create one
        modCount++;
        if (size++ > threshold) {
            tab = doubleCapacity();
            index = hash & (tab.length - 1);
        }
        addNewEntry(key, value, hash, index);
        return null;
    }
    
    ...
    
    static int secondaryHash(Object key) {
        int hash = key.hashCode();
        hash ^= (hash >>> 20) ^ (hash >>> 12);
        hash ^= (hash >>> 7) ^ (hash >>> 4);
        return hash;
    }

    ...

On the basis of this we can conclude that an object is considered to be present in a HashSet on the basis of the values returned by that object's equals and hashCode methods.

Whilest this is what you might expect, as we have seen, in this code base what you might expect and what you actually get are not always the same thing.

2.2 Set Equality

The HashSet implementation inherits the equals method from java.util.AbstractSet where it is defined like this

File

    $(ANDROID_SRC)/libcore/luni/src/main/java/java/util/AbstractSet.java

Source

    ...

    @Override
    public boolean equals(Object object) {
        if (this == object) {
            return true;
        }
        if (object instanceof Set) {
            Set<?> s = (Set<?>) object;
    
            try {
                return size() == s.size() && containsAll(s);
            } catch (NullPointerException ignored) {
                return false;
            } catch (ClassCastException ignored) {
                return false;
            }
        }
        return false;
    }
    
    ...

AbstractSet in turn inherits its implementation of the containsAll method from java.util.AbstractCollection where it is defined like this

File

    $(ANDROID_SRC)/libcore/luni/src/main/java/java/util/AbstractCollection.java

Source

    ...
    
    public boolean containsAll(Collection<?> collection) {
        Iterator<?> it = collection.iterator();
        while (it.hasNext()) {
            if (!contains(it.next())) {
                return false;
            }
        }
        return true;
    }
            
    ... 

Neither HashSet nor AbstractSet define the method contains so the AbstractCollection implementation will be used.

It is defined like this

File

    $(ANDROID_SRC)/libcore/luni/src/main/java/java/util/AbstractCollection.java

Source

    ...
    
    public boolean contains(Object object) {
        Iterator<E> it = iterator();
        if (object != null) {
            while (it.hasNext()) {
                if (object.equals(it.next())) {
                    return true;
                }
            }
        } else {
            while (it.hasNext()) {
                if (it.next() == null) {
                    return true;
                }
            }
        }
        return false;
    }
    
    ... 

The contains method calls the equals of its argument if it is not null.

This is what you might expect but now we know for certain.

3.0 The Semantics Of Signatures

It turns out that the behaviour of the compareSignatures method is determined by the implementation, or lack of one, of the equals method for the Signature class.

The question is does the Signature class define an equals method and if so how is it defined.

3.1 Signature.equals

The answer is yes, and like this

File

    $(ANDROID_SRC)/frameworks/base/core/java/android/content/pm/Signature.java

Source

    ...
    
    @Override
    public boolean equals(Object obj) {
        try {
            if (obj != null) {
                Signature other = (Signature)obj;
                return this == other || Arrays.equals(mSignature, other.mSignature);
            }
        } catch (ClassCastException e) {
        }
        return false;
    }

    ...

Two Signatures are equal if their mSignature byte arrays are equal, which is to say, byte for byte identical.

The mSignature byte array is actually a byte encoded certificate so two Signatures are equal if they have been constructed with identical encodings of the same certificate.

3.2 Signature.hashCode

Since we are in the area we might as well have a look at the implementation of the hashCode method.

File

    $(ANDROID_SRC)/frameworks/base/core/java/android/content/pm/Signature.java

Source

3.3 Signature.hashCode

    ...

    @Override
    public int hashCode() {
        if (mHaveHashCode) {
            return mHashCode;
        }
        mHashCode = Arrays.hashCode(mSignature);
        mHaveHashCode = true;
        return mHashCode;
    }
    
    ...

The hash of a Signature is actually that of its encoded certificate.

4.0 The Semantics Of The compareSignatures Method Concluded

And there you have it, after all of that it turns out that the semantics of the compareSignatures method are very simple.

Two arrays of Signatures match, if and only if, after the elimination of duplicate Signatures the resulting sets are identical.

One of the rather strange implications of this is that if you want the signatures of different applications to match you had better make sure to include exactly
the same number of certificates in the signed certificate file of each one

If you include the legitimate cerificate chain

    A B C

where C is the issuer of B which is the issuer of A, and C is not self-signed, in the signed certificate file of one application, but for some reason you
include the legitimate certificate chain

    A B C D

where D is the issuer of C, in the signed certificate file of another application, then you end up with two applications whose signatures do not match despite
the fact that you actually signed both of them using the same private key, which is a bit odd really.

This applies equally to different versions of the same application.

It also implies that if one of the certificates in the chain you have included with an application expires you are must keep using it despite the fact that it has expired or, again, you end up with non-matching signatures even if the public keys involved do not change.

The simplest thing you can do is to include only the certificate specifying the public key corresponding to the private key with which you signed the signature file, although this does not help in the case of that certificate expiring.


Copyright (c) 2014 By Simon Lewis. All Rights Reserved.

Unauthorized use and/or duplication of this material without express and written permission from this blog's author and owner Simon Lewis is strictly prohibited.

Excerpts and links may be used, provided that full and clear credit is given to Simon Lewis and justanapplication.wordpress.com with appropriate and specific direction to the original content.

Older Posts »

The WordPress Classic Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: