Menu

Subscribe

Using Bytecode Manipulation to kill more Android library boilerplate

on Android, Annotation Processing, Bytecode Manipulation

TL;DR###

Annotation processing lets you create new java source files at build time. If you want to modify existing classes at build time you need a different technique called Java Bytecode Manipulation. I’ve setup a sample Android Studio library project that performs basic bytecode manipulation you can grab and start tinkering with.

Motivating Example: Memento###

The memento library helps you retain activity state in memory during rotation. The library has some rough edges, but it is an easy way to keep state that you only care about retaining on rotation. For example: a RxJava observable, so you can resubscribe to an ongoing network call after rotation.

This library doesn’t require much boilerplate because of its usage of annotation processing. In order to retain an Activity's fields, you do the following

  1. Annotate fields you wish to retain with @Retain
  2. Add Memento.retain(this) to the bottom of YourActivity#onCreate()
  3. Implement MementoCallbacks callback

Step #2 looks like unnecessary boilerplate. You’ve already annotated fields you want retained with @Retain. Therefore Memento already has awareness of which Activity's we care about. We shouldn’t need to declare anything inside YourActivity#onCreate().

Since annotation processing can not modify existing classes there is no way for Memento to hook together generated utility classes with YourActivity without the programmer implementing step #2.

A technique called “Bytecode Manipulation” can be used to modify existing classes. This technique can be used to perform arbitrarily complex transformations of existing classes or creation of new classes.

In order to avoid requiring step #2, the implementation of Memento could be entirely replaced with bytecode manipulation. Or supplemented with bytecode manipulation.

How to manipulate bytecode in Android###

In the current Android build system, .java files are converted into intermediate .class files (aka Java byte code) by the build system before being dexed and baked into apks. Android agnostic java libraries, such as javaassist or ASM, are able to operate on intermediate .class files. I prefer to use javaassist since it is higher level but still very powerful. Instead of manipulating bytes you manipulate classes, methods and parameters.

The Android specific part of this task is hooking up javaassist with your Android build. This is done by writing a gradle plugin that executes as a part of your apk’s build process after javac has run. Using morpheus makes writing this gradle plugin easier. Nonetheless, you should be comfortable writing gradle plugins before you attempt using this from scratch. Consider taking a look at this stackoverflow question or fork the accompanying github project I wrote.

Sample Project###

The sample Android Studio project does the minimal work possible to demonstrate the ability to fix the Memento example case. The project inserts the following arbitrary line of code into the end of a sample app's YourActivity#onCreate().

android.util.Log.d("MOO", "I am inserted code!!");

Sample App Module####

The sample_app module contains nothing more than the following Activity that uses the annotation @ExampleAnnotation.

@ExampleAnnotation
public class YourActivity extends Activity {
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.your_activity);
    }
}

This sample app module invokes the sample plugin by calling apply plugin: 'test-plugin' inside its build.gradle. And as a result, the plugin module performs a transformation on YourActivity.

Annotation Module####

This @ExampleAnnotation is defined inside the annotation module as the following. The purpose of this module is to act as an API for the plugin module.

@Target(value = TYPE)
public @interface ExampleAnnotation {}

Plugin Module####

All the interesting code belongs to the plugin module. This module generates a .jar file. The .jar file contains a definition for test-plugin inside the META-INF/gradle-plugins metadata that looks like the following.

implementation-class=com.brianattwell.plugin.ByteManipulationPlugin

The ByteManipulationPlugin is the plugin's entry point. It returns the custom javaassist transformer where the plugin does actual work. The transformer class looks roughly like the following.

public class ClassTransformer implements IClassTransformer {
    @Override
    public boolean shouldTransform(CtClass candidateClass) throws JavassistBuildException {
        try {
            return candidateClass.hasAnnotation(ExampleAnnotation.class) 
                    && isAnActivity(candidateClass);
        } catch (Exception e) {
            throw new JavassistBuildException(e);
        }
    }

    @Override
    public void applyTransformations(CtClass ctClass) throws JavassistBuildException {
        System.out.println(":plugin:applyTransformations on " + ctClass.getName());
        try {
            CtMethod[] methods = getOnCreateMethod(ctClass);
            for (CtMethod method : methods) {
                method.insertAfter("android.util.Log.d(\"MOO\", \"I am inserted code!\");");
            }
        } catch (Exception e) {
            throw new JavassistBuildException(e);
        }
    }

    ...
}

When sample_app applies the plugin to itself, the ClassTransformer is used to check every class inside sample_app. It decides to transform every Activity annotated with @ExampleAnnotation. It adds a java snippet to the end of these class's onCreate methods.

The println I added inside the transformer can be seen inside the gradle build messages when building the sample app.

gradle build messages image

After this transformation is performed in the build, you'll see a second copy of YourActivity.class inside /sample_app/build/ intermediates/transformations/. You can examine the bytecode changes that have been made by using javap -v filename.

The untransformed onCreate code looks like the following.

protected void onCreate(android.os.Bundle);
flags: ACC_PROTECTED
Code:
  stack=2, locals=2, args_size=2
     0: aload_0       
     1: aload_1       
     2: invokespecial #1    // Method android/app/Activity.onCreate:(Landroid/os/Bundle;)V
     5: aload_0       
     6: ldc           #2    // int 2130903040
     8: invokevirtual #3    // Method setContentView:(I)V
    11: return        
  LineNumberTable:
    line 14: 0
    line 15: 5
    line 16: 11
  LocalVariableTable:
    Start  Length  Slot  Name   Signature
           0      12     0  this   Lcom/brianattwell/github/PluginActivity;
           0      12     1 savedInstanceState   Landroid/os/Bundle;

The transformed onCreate code looks like the following.

protected void onCreate(android.os.Bundle);
flags: ACC_PROTECTED
Code:
  stack=5, locals=4, args_size=2
     0: aload_0       
     1: aload_1       
     2: invokespecial #2     // Method android/app/Activity.onCreate:(Landroid/os/Bundle;)V
     5: aload_0       
     6: ldc           #3     // int 2130903040
     8: invokevirtual #4     // Method setContentView:(I)V
    11: goto          14
    14: aconst_null   
    15: astore_3      
    16: ldc           #30    // String MOO
    18: ldc           #32    // String I am inserted code!!
    20: invokestatic  #38    // Method android/util/Log.d:(Ljava/lang/String;Ljava/lang/String;)I
    23: pop           
    24: return        
  LineNumberTable:
    line 13: 0
    line 14: 5
    line 15: 11
  LocalVariableTable:
    Start  Length  Slot  Name   Signature
           0      14     0  this   Lcom/brianattwell/github/PluginActivity;
           0      14     1 savedInstanceState   Landroid/os/Bundle;
  StackMapTable: number_of_entries = 1
       frame_type = 14 /* same */

FAQ###

Question: What if you’re using the Jack compiler?####

Answer The Jack compiler doesn’t output .class intermediate files. Therefore, bytecode manipulation APIs designed for .class files won’t easily work. For other reasons, the Jack compiler doesn’t even support regular annotation processing yet either. The Jack team states these isses are under development (see using gradle section). I suspect annotation processing will be supported before bytecode manipulation.

Question: Is this reliable?####

Answer The main risk is that bytecode manipulation makes your code harder to read and harder to debug. Otherwise, bytecode manipulation is safe as long you use well tested frameworks like javaassist/afterburner and understand its limitations. Ie, don’t expect annotation processors to be aware of your generated bytecode methods. Bytecode manipulation libraries like Retrolambda are used in millions of app installs.

Question: What about release builds? Can’t the javac optimizations cause problems?####

Answer Very few optimizations are performed during this step of the build process. So this isn’t much of a concern.

Question: How do I publish libraries to JCenter?####

Answer The same way you would publish an annotation processor with the bintray-release plugin.

Question: Why haven’t you updated the Memento GitHub project?####

Answer In this toy use case, I doubt the reduction of boilerplate justifies an API change and increase in library complexity. I’ll reach out to Matthias to get his opinion nonetheless.