Everyone is talking about modules these days. New languages try to incorporate them, and older languages try to retrofit them in. Which is great news, because modules are essential. Java 9 is around the corner, because it's supposed to come out next year, and the really big new feature is modularity, which it calls the Jigsaw project.

Ceylon is a language that featured modularity from the start, as part of the language and not as an afterthought requiring complex third-party tool integration. In fact, at the time we designed our Java JDK integration (at the time of Java 7), we went as far as using the Jigsaw modularity plans for the JDK (yes Jigsaw got delayed a few times) from the start, requiring JDK users to import Jigsaw modules as they were planned at the time, rather than import the whole JDK in one go. So perhaps we were the first ones with a modular JDK, in some sense :)

Java 9’s Jigsaw

Jigsaw is a very large project, which includes the following changes:

• Modularisation of the JDK into smaller units, such as java.base, java.xml that Ceylon users of the JDK are already familiar with.
• This modularisation means removal of rt.jar that contained every JDK class. In fact it's been replaced by a bootmodules.jimage file which is not a jar, but whose contents can be accessed by a virtual NIO FileSystem at jrt:/.
• You can write your own modules. To turn your Java code into a Java 9 module, you simply add a module descriptor in a file called module-info.java (much like Ceylon module descriptors, or Java package descriptors), which describes your module and the Java 9 compiler and jar tools will then generate a jar with a module-info.class descriptor at the root of the jar.
• That module descriptor allows you to specify the module name, the packages it exports, the name of the modules it imports and a few other things. But not versions, unfortunately, which are currently "out of scope" in Java 9.
• You can run your code as previously from the classpath, or as modules from the module path. The module path is just a folder in which you can place your modules and the JRE will look them up for you based on module name alone.

Ceylon and Jigsaw

Java 9 has two early-access (EA) downloads for users to try the module system. Only one of them includes user modules. Make sure you use that one if you want to try out Ceylon running on Java 9.

Over the past weeks I've worked on getting Ceylon compiling and running on Java 9. This involved (among other details) the following things:

• Generating module-info.class files from Ceylon module descriptors.
• Generating module-info.class files for the Ceylon distribution modules which are not written in Ceylon (like the compilers or runtime system).
• Making use of the Java 9 module descriptors for the shared packages information it contains (something supported by Ceylon since the beginning, but which was lacking for plain Java jars).
• Backporting Java 9 code that deals with modules to the javac fork we use to compile Java files and generate bytecode.
• Dealing with the removal of rt.jar and the boot classpath.
• Creating a new tool ceylon jigsaw which allows for the creation of a Java 9 module path.
• Making sure we can run Ceylon modules as Java 9 modules as an alternative to the four existing JVM runtimes which are the JBoss Modules, classpath, OSGi or Java EE.
• Make sure we can build and run on any of Java 7,8,9. This means that by default we do not generate Java 9 module descriptors, because several tools have problems dealing with them at this time.
• We have split some things out of the ceylon.language module so that it no longer depends on the compilers and type-checker, which means a lighter minimal runtime, which will be even further improved in the next weeks with more dependency removals :)

Just tell me how to try this!

I will spare you the many details of this work, but with help from the Java 9 team, this is how you can run your Ceylon modules on a Java 9 runtime:

• Download the Java 9 EA with Jigsaw.
• Get the Ceylon distribution code, and compile it with ant -Djigsaw=true clean dist to get the Java 9 module descriptors.
• Write your Ceylon module normally, but compile it with .../ceylon/dist/dist/bin/ceylon compile --generate-module-info to generate the Java 9 module descriptors.
• Create your Java 9 module path in an mlib folder with .../ceylon/dist/dist/bin/ceylon jigsaw create-mlib my.module/1.
• Run your Ceylon module on Java 9 with .../jdk1.9.0-jigsaw/bin/java -mp mlib -m ceylon.language my.module/1. At the moment, the ceylon.language module acts as main module and does the required setting up of the Ceylon runtime before loading and invoking your Ceylon module.

That's all there is to it!

Caveats

Java 9 is not complete yet, and our support for Java 9 is also not complete. There will be issues and bugs, and in fact we already know of several limitations, such as the following:

• While you can import a pure Java 9 module from Ceylon, we will respect its exported packages, but we will not respect its dependencies, because Java 9 modules do not include dependency versions. In fact, even the module's version is not stored in the source module descriptor, but added by an optional flag to the Java 9 jar tool. Ceylon requires module dependencies to describe a version, so we have to combine the Java 9 module descriptor with another descriptor such as an OSGi descriptor or a Maven pom.xml descriptor. This merging of information is not currently done.
• Java 9 does not currently support optional modules or module cycles. It is not clear if they will support them at this time, unfortunately.
• The ceylon import-jar tool may complain about module visibility artifacts. We intend to fix this in time, but for now you can use --force.
• The JDK module list we used in Ceylon has slightly changed in Java 9. This is what we get for being the first to support Jigsaw ;) For example, the javax.xml module has been renamed to java.xml. We have set up aliases so that it "just" works, but there are modules that have been merged, and packages that have changed module, so it will not always work.
• The Java 9 runtime has been tested, but not as thoroughly as the existing JBoss Modules, classpath, OSGi or Java EE runtimes. We expect a few issues in the Ceylon metamodel.

Note: edited on 16/5/2013 to add info about enum constructors as well.

About Java inner classes

Java allows member classes (classes that are defined inside other classes), local classes (classes that are defined inside statement blocks) and anonymous classes (classes with no names):

class Outer {
    Object anonymous = new Object(){}; // this is an anonymous class

    // anonymous initialisation block
    {
        // this is a local class
        class Local{}
        Local l = new Local();
    }

    Outer() {
        // this is a local named class in a constructor
        class Local{}
        Local l = new Local();
    }

    void method() {
        // this is a local named class in a method
        class Local{}
        Local l = new Local();
    }

    // this is a member class
    class Inner{}
    Inner i = new Inner();
}

The Java Language Specification classifies member, local and anonymous classes as inner classes.

Implementation “details”

What the Java Language or Virtual Machine specifications do not tell you is how they are implemented. Some of it is explained already in other articles, such as how the Java compiler generates synthetic methods to allow these members classes access to private fields, which would not be allowed by the JVM.

Another implementation detail of inner classes that is handy to know is that inner class constructors take extra synthetic parameters. It is relatively well-known that the first synthetic parameter of an inner class constructor will be its enclosing instance, which it will store in a this$X synthetic field. This is valid for all three kinds of inner classes: member, local and anonymous.

But it is generally not known that local classes who capture non-constant final variables will require all these variables to be passed as extra synthetic constructor parameters (captured constant final variables will be inlined and not generate extra synthetic constructor parameters):

class Outer {
    void method() {
        final String constant = "foo";
        final String nonConstant = "foo".toUpperCase();
        class Local{
            /* synthetic fields and constructor: 

            Outer this$0;
            String nonConstant;

            Local(Outer this$0, String nonConstant){
                this.this$0 = this$0;
                this.nonConstant = nonConstant;
            }
            */
        }
        Local l = new Local();
    }
}

Another example: Java enum classes

Java allows you to create enumeration classes, which is essentially little more than syntactic sugar to help you define a list of singleton values of a given type.

The following Java code:

enum Colours {
    RED, BLUE;
}

Is essentially equivalent to:

final class Colours extends java.lang.Enum {
    public final static Colours RED = new Colours("RED", 0);
    public final static Colours BLUE = new Colours("BLUE", 1);

    private final static values = new Colours[]{ RED, BLUE };

    private Colours(String name, int sequence){
        super(name, sequence);
    }

    public static Colours[] values(){
        return values;
    }

    public static Colours valueOf(String name){
        return (Colours)java.lang.Enum.valueOf(Colours.class, name);
    }
}

As you can see, it saves quite some code, but also adds synthetic fields, methods and constructor parameters. If you had defined your own constructor, with its own set of parameters, like this:

enum Colours {
    RED("rouge"), BLUE("bleu");

    public final String french;

    Colours(String french){
        this.french = french;
    }
}

You would have gotten the following Java code generated:

final class Colours extends java.lang.Enum {
    public final static Colours RED = new Colours("RED", 0, "rouge");
    public final static Colours BLUE = new Colours("BLUE", 1, "bleu");

    private final static values = new Colours[]{ RED, BLUE };

    public final String french;

    private Colours(String name, int sequence, String french){
        super(name, sequence);
        this.french = french;
    }

    public static Colours[] values(){
        return values;
    }

    public static Colours valueOf(String name){
        return (Colours)java.lang.Enum.valueOf(Colours.class, name);
    }
}

Luckily, enums can’t be inner classes, so they will not have an extra synthetic parameter inserted for the container instance to add to those two.

OK, but why should I care?

In most cases you don’t care, other than for your own curiosity. But if you’re doing Java reflection with inner or enum classes, there are a few things you should know, and because I haven’t found them listed or specified online, I thought it would be important to make a list of things to help others figure it out, because different compilers will produce different results in the Java reflection API.

The question is what happens when you use Java reflection to get a java.lang.reflect.Constructor instance for inner or enum class constructors? In particular, what happens with the methods that allow you to access the parameter types (pre-generics: getParameterTypes()), the generic parameter types (post-generics: getGenericParameterTypes()) and annotations (getParameterAnnotations()), and the answer is: it depends.

Suppose the following Java classes:

class Outer {
    class Inner {
        Inner(){}
        Inner(String param){}
        Inner(@Deprecated Integer param){}
    }
}
enum class Enum {
    ;// yes this is required
    Enum(){}
    Enum(String param){}
    Enum(@Deprecated Integer param){}
}

Here are the size of the arrays returned by these three reflection methods, on each of our constructor, and how they differ depending on the Java compiler used:

	Outer.Inner.class .getDeclaredConstructor()	Outer.Inner.class .getDeclaredConstructor( String.class)	Outer.Inner.class .getDeclaredConstructor( Integer.class)
getParameterTypes() .length	1	2	2
getGenericParameterTypes() .length compiled with Eclipse	1	2	2
getGenericParameterTypes() .length compiled with Javac	0	1	1
getParameterAnnotations() .length	1	2	1

And the results are consistent for our enum class:

	Enum.class .getDeclaredConstructor()	Enum.class .getDeclaredConstructor( String.class)	Enum.class .getDeclaredConstructor( Integer.class)
getParameterTypes() .length	2	3	3
getGenericParameterTypes() .length compiled with Eclipse	2	3	3
getGenericParameterTypes() .length compiled with Javac	0	1	1
getParameterAnnotations() .length	2	3	1

As you can see, the synthetic parameters are always included in getParameterTypes(), but are only included in getGenericParameterTypes() when compiled with Eclipse.

getParameterAnnotations() on the other hand, will always include synthetic parameters except when at least one of your constructor parameters are annotated.

With this info, you now understand the differences between the results of these methods, but so far I still haven’t found a way to determine which parameter is synthetic or not, because although you can make a good guess for the this$X synthetic parameter, which is required by every inner class, you have no way of knowing the number of non-constant captured variables that will end up as synthetic parameters to local class constructors.