Ceylon 1.2.2 was released back in March, and at the time it shipped with a new feature that did not make a lot of noise, but I think it's worth explaining it a bit more in this blog.

In general Ceylon developers have to download the Ceylon distribution in order to compile Ceylon code, and so do Ceylon users. When you check out a Ceylon project you have to make sure you're using the right version of the Ceylon distribution to compile it, and switching Ceylon distribution can be a bit too involved on some Operating Systems.

To solve that issue, Tako has implemented the ceylon bootstrap tool, very heavily inspired by the Gradle Wrapper, which allows you to generate a small script in your Ceylon project which your users and developpers can use to automatically download the right Ceylon distribution and cache it for future use.

Let's try it out by generating a new Ceylon project (on a machine with a Ceylon 1.2.2 distribution, this is a chicken and egg problem and we have to start somewhere):

$ ceylon new hello-world --ant=no --eclipse=no myproject
Enter module name [com.example.helloworld]: 
Enter module version [1.0.0]:

Now let's check that it compiles and runs:

$ cd myproject
$ ceylon compile,run com.example.helloworld
Note: Created module com.example.helloworld/1.0.0
Hello, World!

Excellent! Now let's add the Ceylon bootstrap tool:

$ ceylon bootstrap
$ ls ceylonb*
ceylonb  ceylonb.bat
$ ls .ceylon/bootstrap/
ceylon-bootstrap.jar  ceylon-bootstrap.properties

As you can see, we've created two ceylonb scripts (one for Unix, one for Windows), and a new configuration folder for those scripts containing a tiny jar (24k) and a settings file which points to the distribution to use:

$ cat .ceylon/bootstrap/ceylon-bootstrap.properties 
#Generated by 'ceylon bootstrap'
#Tue Aug 02 17:29:39 CEST 2016
distribution=https\://downloads.ceylon-lang.org/cli/ceylon-1.2.2.zip

These scripts and folders should be committed to your project code, so they can be used by others.

Now, whenever someone checks your project out, they can just use the provided ceylonb script to work rather than pre-install the Ceylon distribution:

$ ./ceylonb compile,run com.example.helloworld
Downloading Ceylon... 100%
Note: Created module com.example.helloworld/1.0.0
Hello, World!

The first time you run this, it will download the desired Ceylon distribution to ~/.ceylon/dists, but from then on, whenever you invoke ceylonb it will reuse that distribution without downloading anything. Even other projects using the same distribution will be able to use it.

As I said, Gradle Wrapper users are used to this, but still, it's a really good idea and there's no reason why programming languages couldn't do the same!

TL;DR: This article describes the modularity changes in the Ceylon run-time and distribution, in order to make them lighter at run-time. Skip to the Final runtime dependencies section if you just want the outcome.

Ceylon has featured a modular architecture from the start. Not just for Ceylon users who write modules, but also within the Ceylon distribution. Historically we used to have very few modules, that were directly related to separate Git projects. Adding a new module meant a new repository and lots of changes in the build. Naturally, as the project grew, each of those modules also grew, and got new third-party dependencies, and occasionally adding a feature in one module was made tremendously easier by just adding that "one more" dependency between distribution modules, resulting in a big spaghetti graph of distribution modules that is common in older/evolved systems.

As we initially expected most Ceylon users to run their code using the ceylon run command, we figured that since they have the Ceylon distribution installed, it does not matter if they depend on more modules from that distribution than strictly necessary. Those modules had to be there anyway, so it would not save any bandwidth to reduce those dependencies.

Naturally, we were wrong, and between the Ceylon Eclipse or IntelliJ IDEs, running Ceylon on OpenShift, WildFly, Vert.x or on Android, people started running Ceylon without the distribution installed, using just the standard java runner. It became soon apparent that we had to untangle those dependencies to make the runtime requirements lighter.

Historically we had the following modules:

• common module used by other modules
• typechecker (the shared compiler front-end)
• Java compiler back-end
• JavaScript compiler back-end
• module repository system
• JBoss modules runtime

The model module

When we implemented reified generics, we had to add subtyping to the runtime, so that we'd be able to figure out if is T bar was true or not. The easiest thing at the time was to "just" depend on the typechecker (compiler front-end) which dealt with the language model and subtyping, and the Java compiler back-end, which had infrastructure to load a language model from JVM information such as class files, or in this case reflection.

This essentially made the runtime depend on the compiler front-end and back-end, which we realised was not ideal, so during the Ceylon 1.2 development, we extracted all model description, loading and subtyping to a new ceylon-model module, but we did not have enough time to do more and so these dependencies remained due to other causes.

Supporting Java 9

During our work on supporting Java 9 / Jigsaw modules in Ceylon, it became clear that having kept our "fork" of javalang tools (that we use for javac) under its original package name would not work anymore, we renamed its package and used the opportunity to prune away parts of the java tools we did not use. We also extracted the class-file reader part to its own module so we could use it outside of the compiler to remove our dependency to jandex (a class-file scanner).

Finally, when we created the ceylon jigsaw tool (which populates a folder with the jar files required by a Ceylon module, to run it on a Java 9 VM) it became evident that the runtime still depended not just on the compiler front-end and Java back-end, but even on the JavaScript back-end, which frankly made little sense in most JVM executions.

These dependencies were due to the Ceylon Tool Provider API having snuck into the ceylon.language module as a convenience (at the time). Since that allowed you to compile and run Ceylon programmatically for both Java and JavaScript back-ends, it had to depend on the tools.

We decided to split the Ceylon Tool Provider into its own module and got rid of the final dependencies from the language module to the compilers and typechecker, but had no more time to get rid of further dependencies such as JBoss Modules and Aether in time for Ceylon 1.2.2.

Supporting Android

Initial work on running Ceylon on Android revealed that what passes for small dependencies on ordinary JVM executions, or even on Java EE deployments, was not an option on Android where every method counts.

At this point we had to bite the bullet and make every non-required transitive dependency go.

We noticed that the old common module had grew to include the Command-Line Tooling API that makes the ceylon command and its subcommands and plugins work. That in turn depended on a Markdown renderer used by ceylon doc. It was pretty trivial to extract it to its own module because this was never used in Ceylon user programs.

Next in line was our Shrinkwrap Resolver dependency, which our module repository system uses to interoperate with Maven repositories. This was a fat-jar with all its dependencies included, including some Apache Commons modules, and an outdated version of Eclipse Aether. That fat-jar had already been problematic in our Maven module, which already had its version of Aether, so getting rid of the fat-jar was a good idea. We also realised that some of its Apache Commons dependencies were already included outside the fat-jar in our distribution repository, so there was that duplication to fix too.

So what we did was remove the Shrinkwrap Resolver dependency and use Aether directly, by incorporating all its subcomponents into our distribution. It turns out that because the latest version of Aether requires Google Guava, our distribution grew in size rather than shrink (that jar is huge). But to offset that, we made the Aether dependency optional, and made sure it was possible to run Ceylon without it as long as there was some compilation step beforehand that provided all the Maven dependencies that you may use in interop. ceylon fat-jar or ceylon jigsaw would do that for you, for example.

Our module repository system also provided support for writing to WebDAV or Herd repositories, which required some dependencies on Apache Http Client or Sardine, and we made these dependencies optional as well, because at runtime your Ceylon program is very unlikely to write to HTTP repositories. This is something only the compiler and other tools do.

We also removed a dependency to JBoss Modules from the language module using abstraction, since that platform was optional and never used on Android or other flat-classpath runtimes.

Finally, the language module only had one dependency left on the (much slimmer) module repository system via the presence of the Main API in there, and we moved that class to its own module.

Final runtime dependencies

After all this pruning, the language module on the JVM is back down to requiring the following set of transitive dependencies:

• common (small and free of tooling and dependencies)
• model (which depends only on the class-file reader)
• class-file reader

So your Ceylon module will only depend on four jars (these three and the language module), the sum size of which is 2.4 Mb, which is much smaller than initially, and has dramatically less methods, at around 17148 methods. This is still too much, but can be brought down by tooling such as ProGuard to remove unused classes. Remember this includes a runtime for an entire language, so it's not that big, all things considered.

SDK changes

In order to be able to use Ceylon's HTTP client on Android, we also split up the ceylon.net module from the Ceylon SDK into client and server modules. Otherwise the HTTP server and its dependencies were too much drag for Android's method count.

Ceylon 1.2.3 is looming closer, and so we should start talking about some of the new features that it will contain. One of those new features is the new ceylon fat-jar command.

ceylon fat-jar my.module/1 lets you generate a jar which contains the my.module/1 module and every required dependency. Once you have that jar, you can execute it with java -jar my.module-1.jar and it will execute the module.

Naturally, you can customise the function or class to execute with the --run option, but the default is the run method in your module (my.module::run in our case).

The modules are executed in a "flat classpath" mode, without JBoss Modules or any sort of module isolation, and with the metamodel already set up statically when the fat jar was created, so it's a little bit different to the older alternative of running Ceylon modules using the Main API and ceylon classpath tool, which did not deal with packing all the dependencies. It is also different to the ceylon run --flat-classpath in that it does not require a module repository at run-time, since ceylon fat-jar packs it all together.

Try it out, it's pretty useful to distribute Ceylon programs and run them in places without having to install the Ceylon distribution.

In my last post, I explained how you can use Ceylon in Apache Cordova to write applications for every mobile platform, including iOS and Android. This time, with many apologies for writing it late (“next week” turned into next month), I will explain how to use the Ceylon IntelliJ plugin to write native Android applications in Ceylon in Android Studio.

Getting started with Ceylon on Android Studio

To start writing Ceylon applications in Android Studio, follow these steps:

• Download Android Studio
• Start it
• Create a new application by clicking on Start a new Android Studio Project
• You can use these values for Application name: CeylonDemo
• And for Company domain: android.example.com
• Next, select an Empty activity, with Activity Name: MainActivity
• Click Finish and wait for the project to be created

At this point you have an Android project open, but we still haven't had time to install the Ceylon plugin, so let's do this right now:

• Click on File > Settings > Plugins > Browse Repositories
• Then on Manage Repositories > +
• Add this repository: https://downloads.ceylon-lang.org/ide/intellij/development/updatePlugins.xml
• Now click on Install Ceylon IDE

You will likely need to restart Android Studio, so do that.

Next we're going to convert our Android project to a Ceylon Android project:

• In the Android view, Right-click on app > Configure Ceylon in this Module
• Click OK on the resulting configuration dialog

This will set up the Ceylon plugin, and will add most of what you need in your Gradle build to build Ceylon Android applications. You now have your Ceylon sources in app/src/main/ceylon and it includes a module descriptor and an empty activity:

At the moment, this requires a Ceylon 1.2.3 distribution to build, and since it's not released yet you're going to have to either build one yourself (just the Getting the source part), or download a nightly build. Once you have it, edit app/build.gradle near then end to add ceylon > ceylonLocation and make it point to where you installed your distribution (it needs to point to the Ceylon binary, not just the distribution root):

ceylon {
    // ...
    ceylonLocation ".../ceylon/dist/dist/bin/ceylon"
}

Make sure you click on Sync now to sync your Gradle build.

Now, there's a bug we're in the process of fixing which fails to detect the exact version of the Android SDK tooling and modules, and so depending on which version of the Android Tools you're using you may have to sync the imports of com.android.support:appcompat-v7:23.1.1 in app/build.gradle (in dependencies) and in the Ceylon module descriptor in module.ceylon. Make sure the Ceylon import version is the same as the Gradle import version, because the Gradle build is what makes it available to Ceylon, due to Android's peculiarities.

In order to finish the conversion, make sure you delete the Java activity (since we're going to keep the Ceylon one), in Project Files, delete app/src/main/java.

Due to another pending plugin fix, you may have to click on Tools > Ceylon > Reset Ceylon Model at this point so that the Ceylon plugin gets synchronised with all these past changes (don't worry we're fixing this at the moment).

The good news is we're already able to click on Run app and try this in the emulator, but we're going to make it a little more interesting.

Customising your Ceylon Android activity

We're going to be displaying a list of Ceylon modules published on Ceylon Herd, so we will make use of the Ceylon SDK, and in particular you will have to edit module.ceylon to add the following imports:

import ceylon.http.client "1.2.3";
import ceylon.uri "1.2.3";
import ceylon.json "1.2.3";
import ceylon.collection "1.2.3";
import ceylon.interop.java "1.2.3";

Next, we're going to turn our MainActivity into a ListActivity and run an asynchronous task to connect to the Herd REST endpoint, so edit MainActivity.ceylon with this:

import android.os { Bundle, AsyncTask }
import android.app { ListActivity }
import android.widget { ArrayAdapter, ListAdapter }
import android.support.v7.app { AppCompatActivity }
import ceylon.interop.java { createJavaStringArray }
import java.lang { JString = String }
import android { AndroidR = R }
import ceylon.language.meta { modules }
import ceylon.uri { parseUri = parse }
import ceylon.http.client { httpGet = get }
import ceylon.json { parseJson = parse, JsonObject = Object, JsonArray = Array }
import ceylon.collection { MutableList, LinkedList }

shared class MainActivity() extends ListActivity() {

    class LoadModules() extends AsyncTask<String, Nothing, List<String>>() {
        shared actual List<String> doInBackground(String?* uris){
            assert(exists uri = uris.first);
            value response = httpGet(parseUri(uri)).execute();
            value modules = LinkedList<String>();
            assert(is JsonObject json = parseJson(response.contents),
                    is JsonArray results = json["results"]);
            // Iterate modules
            for(res in results) {
                assert (is JsonObject res); // Get the list of versions
                assert (is String name = res["module"],
                        is JsonArray versions = res["versions"]);
                modules.add(name);
                print(name);
            }
            return modules;
        }
        shared actual void onPostExecute(List<String> result){
            print("Got result: ``result``");

            ListAdapter adapter = ArrayAdapter<JString>(outer, AndroidR.Layout.simple_list_item_1,
                createJavaStringArray(result));
            listAdapter = adapter;
        }
    }

    shared actual void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.Layout.activity_main);

        LoadModules().execute("https://modules.ceylon-lang.org/api/1/complete-modules?module=ceylon.");
    }
}

Now edit app/src/main/res/layout/activity_main.xml to change the activity type to a list activity:

<ListView
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:id="@android:id/list"/>

And lastly request the network permission for your app, since we're hitting a web service, by adding this to app/src/main/res/AndroidManifest.xml:

<uses-permission android:name="android.permission.INTERNET"/>

That's all you need, now just click on Run > Run app and watch your Ceylon application display the list of modules in the emulator:

Some technical info

The Ceylon IntelliJ plugin has not been released yet, but a preview is forthcoming really soon. You will see it's already quite advanced when you try this out. Don't hesitate to report any bugs, or better yet, contribute fixes :)

Most of it works well enough for Android, except the caveats noted above, and the fact that Android Studio does not yet recognize Ceylon classes, so they will be marked as errors in the .xml files that refer to them, and when you run your application it will report an error:

Could not identify launch activity: Default Activity not found. Error while Launching activity

It only means it could not start your application, you will have to click on it to start it in the emulator. But the deployment worked. We're fixing this at the moment, so it will only improve.

If you want to revert to the Ceylon Eclipse IDE to edit your Ceylon Android application, you can, it will work once you have your project set up with Android Studio. It's much easier to use it to set it up so all the Gradle config is just right. Once that is done, you can use Eclipse if you want, and use $ ./gradlew assembleDebug to build your APK.

This work depends on changes we've made in Ceylon 1.2.3 (to be released soon) which adds support for jars which provide alternate smaller JDKs (such as the Android jar), improvements in modularity so that the created applications depend on much fewer runtime Ceylon jars than before, fixes in the runtime metamodel to support Android runtimes, and several other tweaks. I will probably write an account of all that in a future blog entry.

This work also depends on the Ceylon Gradle plugin written by Renato Athaydes, and on a new Ceylon Gradle Android plugin which adds support for Ceylon in Android applications. This plugin is by no means finished, and in particular does not yet support incremental compilation (even though the Ceylon IDE and compiler do). It also does not yet support the latest Android Instant Run feature. Again, please report issues or better, contribute pull-requests :)

Ceylon already runs on the JVM, whether bare-bones, via JBoss Modules, Vert.x, Java EE Servlet containers such as WildFly, or OSGi containers, as well as on JavaScript VMs such as Node.js and the browser. But today we're going to explain how to run Ceylon on mobile devices, not just in the browser (though it does play a part in it), but as applications, via Apache Cordova.

Apache Cordova allows you to write applications for every mobile platform, including Android and iOS, using nothing but HTML, CSS and JavaScript. Since Ceylon compiles to JavaScript this is perfect as it allows us to run our Ceylon applications on iOS, via the JavaScript compiler backend.

Note that this article is using Ceylon 1.2.3 which is not yet released, because the JavaScript runtime in Cordova on Android had one peculiarity we had to work around in the language module JavaScript implementation. Luckily you can get nightly builds of Ceylon 1.2.3 and the Ceylon 1.2.3 SDK.

Writing your Ceylon Cordova application

Installing Apache Cordova

First, install Apache Cordova and add two platforms. I haven't been able to test the iOS platform since it requires an OSX platform to build and an iOS device to test, and I lack both, so I will explain how to package for Android and the browser, and let you guys try it out for iOS, but I have enough faith in Apache Cordova that it will Just Work™.

# Install npm, the Node.js package manager
$ sudo apt-get install npm
# Then install Apache Cordova
$ npm install cordova

Small note: for me this installed things in ./node_modules/cordova and the Apache Cordova command in ./node_modules/cordova/bin/cordova, so adapt your path as you must.

# Create your application
$ cordova create ceylon-cordova-demo
$ cd ceylon-cordova-demo
# Now add the browser and Android platforms
$ cordova platform add browser
$ cordova platform add android

At this point you have your application ready to be checked in your browser:

$ cordova platform run browser

Or in an Android emulator, provided you have downloaded the Android SDK already:

$ ANDROID_HOME=.../Android/Sdk cordova platform build android
$ ANDROID_HOME=.../Android/Sdk cordova platform run android

Getting a little side-tracked about styling

Writing an application using just HTML and CSS means you have to make some effort for it to look good, and instead I decided to delegate to use Polymer so that my application would have the look and feel of Android Material Design applications to feel even more like a native application on Android. No doubt a similar look and feel exists for iOS.

So let's download Polymer in our application's HTML sources:

$ npm install bower
$ cd www
$ bower init
# At this point just hit enter/Yes/No until it's set up 
$ bower install --save Polymer/polymer
$ bower install --save Polymerelements/paper-item

And now edit www/index.html to use Polymer:

<script src="bower_components/webcomponentsjs/webcomponents.js"></script>
<link rel="import" href="bower_components/paper-item/paper-item.html">
<link rel="import" href="bower_components/paper-item/paper-item-body.html">

You should also remove the default CSS:

<link rel="stylesheet" type="text/css" href="css/index.css">

Getting require.js and jQuery

Ceylon compiles to JavaScript modules by way of require.js, so we're going to have to download it too:

$ cd www/js
$ wget http://requirejs.org/docs/release/2.2.0/minified/require.js

Our Ceylon demo will use jQuery to add elements to the HTML page, so we also need it:

$ cd www
$ bower install --save jquery

Now edit www/index.html to use both:

<script type="text/javascript" src="js/require.js"></script>
<script src="bower_components/jquery/dist/jquery.js"></script>

Writing the Ceylon application

We're going to write a trivial application that queries Ceylon Herd for the list of modules, to display them in a list.

Let's start by creating a Ceylon module in source/cordova/demo/module.ceylon:

module cordova.demo "1.0.0" {
    import ceylon.json "1.2.3";
}

And our application's main method in source/cordova/demo/run.ceylon:

import ceylon.json { parseJson = parse, JsonObject = Object, JsonArray = Array }

shared void run() {
    dynamic {
        // The HTML element where we'll add our items
        dynamic target = jQuery("#target");
        // The function called when we get data from the server
        void success(dynamic data){
            // Parse the JSON
            assert(is JsonObject json = parseJson(data),
                   is JsonArray results = json["results"]);
            // Iterate modules
            for(res in results){
              assert(is JsonObject res);
              // Get the list of versions
              assert(is String name = res["module"],
                     is JsonArray versions = res["versions"]);
              // Join them
              value versionText = ", ".join(versions.narrow<String>());
              // Now add the HTML items
              dynamic item = jQuery("<paper-item/>");
              dynamic body = jQuery("<paper-item-body two-line/>").appendTo(item);
              jQuery("<div/>").text(name).appendTo(body);
              jQuery("<div secondary/>").text(versionText).appendTo(body);
              target.append(item);
            }
        }
        // Query Herd for the list of modules
        jQuery.get("https://modules.ceylon-lang.org/api/1/complete-modules?module=ceylon.", null, success, "text");
    }
}

Now, obviously using jQuery to add HTML is far from ideal, so I can't wait for someone to extend ceylon.html to allow Polymer web components!

We can now compile our application for JavaScript:

$ ceylon compile-js

And copy our compiled module and all its dependencies to where the Apache Cordova application will find them in www/modules:

$ ceylon copy --with-dependencies --js --out www/modules cordova.demo/1.0.0

Invoking the Ceylon module from the Cordova application

Because we're going to use require.js inline and connect to Ceylon Herd, we have to adjust the Apache Cordova permissions in www/index.html, so find that line and edit it as such:

<meta http-equiv="Content-Security-Policy" 
      content="default-src 'self' 'unsafe-inline' 
               https://fonts.googleapis.com
               https://fonts.gstatic.com;
               connect-src *">

We're left with just invoking our Ceylon function in www/index.html:

<script type="text/javascript">
  // tell require.js where our Ceylon modules are 
  require.config({
    baseUrl:'modules',
  });
  // when the document is ready
  jQuery(function(){
    // load our Ceylon module
    require(['cordova/demo/1.0.0/cordova.demo-1.0.0'], function(client) {
      // and call our run method
      client.run();
    });
});
</script>

And setting up the target HTML elements where we're going to add every loaded module (in the same file):

<body id="app" unresolved>
  <app-shell class="fit">
    <div id="target" role="listbox"></div>
  </app-shell>
</body>

Trying it

And that's it, try it out in your browser:

$ cordova platform run browser

Or in an Android emulator:

$ ANDROID_HOME=.../Android/Sdk cordova platform build android
$ ANDROID_HOME=.../Android/Sdk cordova platform run android

If you have OSX and iOS dev tools, please try this with the iOS Cordova platform and let me know how it works :)

In the future, we would benefit from having a type-safe API in front of the Cordova JavaScript API that lets you access native mobile APIs such as the camera, GPS, contacts, but even without it you can already use them using dynamic blocks.

And now for the teaser… this is only one method to run Ceylon on Android, because obviously it may be more desirable to use the JVM compiler backend and integrate with the Android Tools to run Ceylon on Android using only type-safe APIs. Don't worry, it's coming, and next week I will show you how :)