Modules, ah, modules. The albatross of Java. I frequently joke that modules are scheduled for Java N+1 where N moves forward with each release. I remember perfectly the first time I heard of Java getting modules at Devoxx, back when they were still planned for Java 7. I remember I heard the announcement and what I saw made a lot of sense, I couldn't wait to get them. I agreed completely, and still do, with the assertion that modules belong in the language, just like packages, and that they should not be delegated to third-party external systems which will never be able to have the level of integration that you can achieve by being an integral part of the language. And then, they pushed it to Java 8. And then to Java 9. It's a shame because it looks like a really well thought-out module system.

Meanwhile, Ceylon already supports modules, in a generally awesome way. But why do we need modules? Here are some easy answers:

• To serve as a larger container than packages. Everyone ships Java code into jars which are generally accepted to be modules: they represent a group of packages that implement a library or program and have a version.
• To express and resolve dependencies: your module (jar) is going to depend on other modules (jars), so we should know which they are and how to find them.
• To distribute modules: Linux distributions have been doing this forever. With a modular system where modules have names and versions, you can organise them in a standard hierarchy, which can then be used by tools to obtain those modules in a standard way. With dependencies, the tools can then download/upload the dependencies too, which means distribution becomes a lot simpler.
• To isolate modules at runtime. This is part of escaping the infamous classpath hell: if you isolate your modules at runtime, then you can have multiple programs that depend on different versions of the same module loaded at the same time.

So why is it better if we have modules in the language, rather than outside of it?

• It's standard and tools have to support it. You could view this as a downside, but really, what part of the Java language would you rather have left unspecified? Would you be ready to delegage the implementation of packages to third-party tools? Having a single way to deal with modules helps a lot, both users and implementors.
• You get rid of external tools, because suddenly javac, java and friends know how to publish, fetch and execute modules and their dependencies. No more plumbing and sub-par fittings.
• It gets integrated with reflection. Modules are visible at runtime, fully reified and hopefully dynamic too, just like classes and packages are.
• Dependencies and modules are separated from the build system. There's absolutely no good reason why the two should be confused with one another.

These are the reasons why I can't wait for Java N+1 to have modules, I think they'll be great. But if you need modules now, then you can use Ceylon :)

Ceylon support modules in the language, from the start, with:

• a super small ceylon.language base module, which is all you need to start using Ceylon
• a modular SDK
• a great module repository: Herd
• support for module repositories in all the tools, be it the command-line or the IDE. They know how to deal with dependencies, fetch or publish modules to/from local or remote repositories
• superb support for Herd repositories from the IDE, with the ability to search for modules, have auto-completion and all
• support for a modular JDK, using the same module map as the Jigsaw project (Java's planned modular SDK)
• and even interoperability with Maven repositories, so that you can use Maven modules as if they were Ceylon modules

Other existing third-party module systems

As I mentioned, we support using Maven repositories from Ceylon, and we will probably support OSGi too, in time. Those are the two main third-party module systems for Java. OSGi is used a lot in application servers or IDEs, but rarely by most Java programmers, which prefer Maven. Maven was the first system that provided both modularity and a build system for the JVM. Prior to that we had Ant, which only provided the build system. Maven essentially prevailed over Ant because it supported modularity, which meant that people no longer had to worry about how to get their dependencies. Even applying the modularity solution to itself, it became easier to distribute Maven modules than Ant modules.

Maven supports modularity and dependencies, in order to resolve and download them, but once they are downloaded, the dependencies are crammed into the classpath, so as far as the Java compiler or runner are concerned, modularity has been erased. There is no support for multiple versions of the same module at runtime, or any kind of validation of dependencies.

The undeclared transitive dependency problem

We recently had bug reports for Ceylon where our users has trouble using Maven modules from Ceylon, due to missing dependencies. Since we do use the dependencies provided by Maven, we found that a bit weird. After checking, it appears that we've been hit by the modularity erasure of Maven. Here's a simple example of something you can do with Maven modules:

• You write a module A, which uses module B and C, but you only declare a dependency on module B
• Module B depends on module C
• Module C does not depend on anything

In Ceylon, if you tried to compile module A, the compiler would not let you because you failed to depend on C. With Maven, it just works, because Maven fetches modules B and C and puts them in the classpath, which means all implicit transitive dependencies end up visible to your module. Modularity is erased. This may seem convenient, but it really means that dependencies are not checked and cannot be trusted.

Due to that, we can't rely on Maven modules to properly declare their dependencies, so we cannot run Maven modules in isolation.

The undeclared implicit dependency problem

There's something more subtly wrong with unchecked module systems: implicit dependencies. This is something you're allowed to do with Maven:

• Module A uses module B
• Module B depends on module C and declares it
• Module B uses types from module C in it public API, such as parameter types, method or field types, or even super classes

This is a variant of the first kind of problem, except that in this case nobody can use module B without also importing module C directly, because it's not possible to use the types of module B without seeing the types of modules C.

In Ceylon, if you tried to compile module B, the compiler would not let you unless you export your module C dependency. This way, when you depend on module B, you automatically also depend on module C, because you really need both to be visible to be able to use module B's API.

Another point in favour of integrated module systems

If we had an integrated module system from the start, in Java, with proper module isolation, we would not have the issues I just described with missing dependencies that are so widespread in Maven, because there are no tools to prevent you from making these mistakes. Compilers do not let you use packages unless you import them, there's no reason to expect that the same would not hold for modules.

I still think the modules project for Java will be a great leap forward, but since Java N+1 is still not here, and there's a huge library of Maven modules that we want to be able to use, we have to find a way to bypass the limitations of Maven's dependency declarations to let you use them in Ceylon. We have various ideas of how to do that, from automatic detection of dependencies through bytecode analysis, to storing Maven modules in a "flat classpath" container, or even via dependency overrides where users could "fix" Maven dependencies. We're still in the process of evaluating each of these solutions, but if you have other suggestions, feel free to pitch in :)

This year we are going to participate in the Google Summer of Code under the JBoss Community organisation.

We've put together a page with our proposals, so check it out, and if you're a student and would like to participate in the Ceylon project in a way that will really matter for our users, go apply now.

The way Ceylon handles null is one of the big attractions of the language. Ceylon features a typed null value. The object null is just an instance of the perfectly ordinary class Null from the point of view of the type system. But when the compiler backend transforms Ceylon code to Java bytecode or JavaScript, the class Null is eliminated and the value null is represented as a native Java or JavaScript null at the virtual machine level.

This is great for interoperation with native Java or JavaScript. However, when designing our Java interop, we had to take into account that Java's null isn't typed, and that the information about whether a Java method might return null, or whether a parameter of a Java method accepts null is "missing" and not available to the Ceylon typechecker. I think we've done the right thing here, but the approach we took has surprised a couple of people, so I guess it's well worth calling explicit attention to what we've done and why.

Consider the following useful method of java.lang.String:

 public String toUpperCase(Locale locale) { ... }

By checking the implementation of this method, I determined that:

1. toUpperCase() never returns null, and that
2. the argument to locale must not be null.

Unfortunately, this information isn't available to the Ceylon typechecker. So perhaps you might expect that Ceylon would take a very heavyhanded route, treating the return type of toUpperCase() as String?, and forcing me to explicitly narrow to String using an assertion:

assert (exists uppercased = javaString.toUpperCase(locale));

In fact, this is not what Ceylon does. This approach would make interoperation with Java a nightmare, resulting in code full of hundreds of useless assertions. You would need an assertion essentially every time you call a native method.

Worse, the "heavyhanded" approach would prevent me from passing the value null as an argument to a parameter of a Java method. What would it mean to "assert" that a method parameter accepts null? That's not an assertion that can be tested at runtime!

Nor would the "heavyhanded" approach in practice even protect me from NullPointerExceptions. As soon as I call native Java code, a NullPointerException can occur inside the Java code I call, and there's nothing Ceylon can do to protect me from that.

So instead, Ceylon takes a "lighthanded" approach at the boundary between Java and Ceylon, letting you write this:

String uppercased = javaString.toUpperCase(locale);

The typechecker knows this is a Java method, and that the information about null is missing, so it assumes that you know what you're doing and that toUpperCase() never returns null.

But what if you've got it wrong, and toUpperCase() does return null at runtime? We would not want that null value to propagate into Ceylon values declared to contain a not-null type!

What the Ceylon compiler does is insert a null check on the return value of toUpperCase(), and throws a NullPointerException to indicate the problem. If you ever see that NullPointerException, at runtime, you're supposed to change your code to this:

String? uppercased = javaString.toUpperCase(locale);

This is essentially the best Ceylon can do given the information available to it. The NPE is not a bug! It's not even really a "gotcha".

Still, the NullPointerException might look like a bug to novice user. (Indeed, Stef recently noticed someone describe it as a bug in a conference presentation.) So I'm asking myself how we can make this less surprising. There's a range of options, including:

• add a better error message to the NullPointerException,
• throw an AssertionException instead of a NullPointerException, or even, perhaps,
• following the approach we use for JavaScript interop, require you to surround "lighthanded" code in a special construct that suppresses the "heavyhanded" typechecks, dynamic Null { ... } or something.

I lean towards the first of these options, I suppose.

Whatever we do, we need to document this better. Right now, the documentation is kinda hidden.

Ceylon M5 “Nesa Pong” is now available for download, along with a simultaneous compatible release of Ceylon IDE. This is a huge release, with the following headline features:

• a fully-reified type system with generic type arguments available at runtime,
• direct interoperation with native JavaScript,
• tuples,
• several important syntax changes in response to community feedback and practical experience, and
• a datetime module and an HTTP server.

You can download the Ceylon command line distribution here:

http://ceylon-lang.org/download

Or you can install Ceylon IDE from our Eclipse update site.

Language features

M5 is an almost-complete implementation of the Ceylon language, including the following new features compared to M4:

• tuples
• direct interop with native JavaScript APIs via the new dynamic block
• the : operator
• verbatim strings
• fat arrow => abbreviation for defining single-expression functions
• syntactic sugar for iterables
• more powerful spread operator
• the late annotation
• binary and hexadecimal numeric literals
• defaulted type parameters
• reified generics
• many miscellaneous minor improvements to the syntax and typing rules of the language
• ceylon.time module
• ceylon.net.http.server package (a HTTP server)
• the compose() and curry() functions
• reworked interop with Java arrays
• more than 500 features and bug fixes

In addition, the language specification and documentation have been substantially revised and improved.

The following language features are not yet supported in M5:

• the type safe metamodel
• user-defined annotations
• serialization

This page provides a quick introduction to the language. The draft language specification is the complete definition.

Source code

The source code for Ceylon, its specification, and its website, is freely available from GitHub:

https://github.com/ceylon

Issues

Bugs and suggestions may be reported in GitHub's issue tracker.

Community

The Ceylon community site includes documentation, the current draft of the language specification, the roadmap, and information about getting involved.

http://ceylon-lang.org

SDK

The new platform modules are available in the shared community repository, Ceylon Herd.

https://herd.ceylon-lang.org

Acknowledgement

We're deeply indebted to the community volunteers who contributed a substantial part of the current Ceylon codebase, working in their own spare time. The following people have contributed to this release:

Gavin King, Stéphane Épardaud, Tako Schotanus, Emmanuel Bernard, Tom Bentley, Aleš Justin, David Festal, Flavio Oliveri, Max Rydahl Andersen, Mladen Turk, James Cobb, Tomáš Hradec, Michael Brackx, Ross Tate, Ivo Kasiuk, Enrique Zamudio, Roland Tepp, Diego Coronel, Brent Douglas, Corbin Uselton, Loic Rouchon, Lukas Eder, Markus Rydh, Matej Lazar, Julien Ponge, Julien Viet, Pete Muir, Nicolas Leroux, Brett Cannon, Geoffrey De Smet, Guillaume Lours, Gunnar Morling, Jeff Parsons, Jesse Sightler, Oleg Kulikov, Raimund Klein, Sergej Koščejev, Chris Marshall, Simon Thum, Maia Kozheva, Shelby, Aslak Knutsen, Fabien Meurisse, Paco Soberón, sjur, Xavier Coulon.

Last week, we met the Ceylon HTTP server, which we can use to serve dynamic (or even static) content over HTTP. But what about the client side? Well, today we're going to write a little HTTP client that runs in the browser, mainly as a way of showing off Ceylon's new dynamic blocks.

First, let's see the server, in a module named hello.server:

import ceylon.net.httpd { createServer, startsWith, 
                          AsynchronousEndpoint, 
                          Request, Response }
import ceylon.net.httpd.endpoints { serveStaticFile }

void run() {
    createServer{ 
        AsynchronousEndpoint { 
            path = startsWith("/sayHello"); 
            void service(Request request, Response response, 
                    void complete()) {
                response.writeString("Hello, Client!");
                complete(); //async response complete
            }
        },
        AsynchronousEndpoint {
            path = startsWith(""); 
            service = serveStaticFile("../client");
        } 
    }
    .start();
}

The server has two asynchronous endpoints, one of which simply serves up static content out of the directory ../client, and the other of which just responds to requests with the text Hello, Client!.

The entrypoint to our application is a HTML page.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
    <head>
        <meta http-equiv="Content-type" 
              content="text/html;charset=UTF-8"/>
        <title>Hello</title>
        <script type="text/javascript"
                src="script/require/1.0.8/require.js">
        </script>
        <script type="text/javascript">
            require.config({
                // location of our project's compiled modules
                baseUrl: 'modules',
                // locations of external dependencies here
                paths: {
                    // path to the Ceylon language module 
                    'ceylon/language': 
                        '/script/ceylon/language'
                }
            });
            //alias our hello module to 'h'
            require(['hello/client/1/hello.client-1'], 
                    function (hello) { h = hello; });
        </script>
    </head>
    <body>
        <div>
            <input type="submit" 
                   value="Say hello" 
                   onclick="h.hello()"/>
        </div>
        <div>
            <span id="greeting"/>
        </div>
    </body>
</html>

The page has a button which calls the hello() function of the Ceylon module hello.client.

We're using require.js to load our Ceylon modules. Unfortunately require.js doesn't really have a concept of module repositories like Ceylon does, which means configuring it to find all our compiled Ceylon is a bit fiddly. Nor does it feature very good error reporting which turns "fiddly" into "time-wasting". (The implementation of require() in node.js is a much better fit,
but unfortunately we don't have that in the browser.)

Finally, the module named hello.client has the following function:

shared void hello() {
    dynamic {
        value req = XMLHttpRequest();       
        void handleResponse() {
            if (req.readyState==4) {
                document.getElementById("greeting")
                        .innerHTML = req.status==200 
                                then req.responseText 
                                else "error";
            }
        }
        req.onreadystatechange = handleResponse;
        req.open("GET", "/sayHello", true);
        req.send();
    }
}

This function makes use of the native JavaScript API XMLHttpRequest to send an asynchronous request to our server, and then interacts with the browser's DOM. But how on earth can a statically typed language like Ceylon call a weakly typed language like JavaScript!?

Well, code that appears inside a dynamic block is optionally typed. That is, Ceylon will make use of typing information when it is available, but when it is not, it will just shut up and let you write almost whatever you like, as long as it's syntactically well-formed Ceylon code. Ceylon can't be certain that there's really a function called XMLHttpRequest, and it certainly can't be sure that it has a member named onreadystatechange, but it will just assume you know what you're doing.

Inside a dynamic block, you can even instantiate a "dynamic" object:

dynamic {
    void printGreeting(value obj) {
        console.log(obj.greeting + " " + obj.language);
    }
    value obj = value { greeting="Hello"; language="Ceylon"; };
    printGreeting(obj);
} 

Of course, inside a dynamic block, all kinds of typing errors can occur at runtime, things that the Ceylon compiler would usually detect at compile time. But the compiler at least protects you from having dynamic typing errors "leak" out of the dynamic block and into your regular Ceylon code by performing runtime checks when you assign a dynamically typed expression (that is, an expression of unknown type) to something with a well-defined static type.

Well, that's essentially all there is to our application, except for a couple of boring module and package descriptors. Here we can see a server written in Ceylon running in the JVM being called by a client written in Ceylon running in a web browser and interacting with the browser's native JavaScript APIs. I think that's pretty cool.

A note of caution: we're still working through some wrinkles in the semantics of dynamic, and dynamic is still not supported on the JVM (it will be, eventually). Nevertheless, this will be available as an experimental feature in Ceylon M5.