About the Ceylon command-line interface

We’re programmers, so let’s face it, we spend lots of time in those small obscure windows with tiny text, preferably with a green or black background, that Hollywood movies often display scrolling a lot, really fast, you know? Terminals.

While good IDEs are required, and the Ceylon IDE is a must-have for Ceylon developers, they can never replace a good terminal session with the proper command-line interface (CLI) tools.

Oh, I know Eclipse has good Git support and I know a few colleagues who use it for committing and branching and other daily, nay, hourly push/pull command. But I really love my CLI when it comes to using Git, I never use Eclipse for that, so I spend a lot of time in my terminal.

Also, it’s essential to be able to build a Ceylon project from the command-line, and not just from the IDE. Imagine trying to build your project in a remote machine using an IDE with a remote X session? No, that doesn’t cut it, sorry.

So from the start, Ceylon had a CLI. Actually, we had a CLI before we had an IDE, but that didn’t last long. And in the beginning we copied Java and had a few commands such as ceylonc to compile, and ceylon to run. Then we added the API generator tool called ceylondoc, and very soon after that we figured we were doing something horribly wrong.

Back when Java came out in 1995 it was sorta cute and useful that all its commands started with the java prefix, but after you got around to remembering what javac and javadoc stood for, you started to wonder what the hell javap or javah could possibly mean.

Then Git came around and revolutionised not only distributed version-control, it also revolutionised command-line interfaces. Suddenly you could start with remembering a single command called git and find out from here what the full list of subcommands would be. You add --help and it spits out a man page. You have completion out of the box. And everyone can write plugins very easily in just a few lines of shell script. Not only that, but most options can be stored and retrieved in a really straightforward config file, with useful and logical lookup in your project, then home.

It’s really a treasure trove of good ideas, and much more modern than the Java CLI we started copying. So we stopped and rewrote our CLI to copy the Git CLI, and I have to say I’m really glad we did, because we really have a great CLI now.

Discovery

One Command to rule them all, One Command to find them, One Command to bring them all and in the CLI bind them In the Land of Terminal where the Shell lie.

We have a single command called ceylon, and if you do ceylon help or ceylon --help you will get the following:

NAME

        'ceylon' - The top level Ceylon tool is used to execute other Ceylon tools


SYNOPSIS

        ceylon --version
        ceylon [--stacktraces] [--] <command> [<command‑options...>] [<command‑args...>]


DESCRIPTION

        If '--version' is present, print version information and exit. Otherwise '<tool-arguments>' should 
        begin with the name of a ceylon tool. The named tool is loaded and configured with the remaining
        command line arguments and control passes to that tool.

        * 'all' - Compiles ceylon modules for JVM and JS and documents them

        * 'compile' - Compiles Ceylon and Java source code and directly produces module and source archives 
                      in a module repository.

        * 'compile-js' - Compiles Ceylon source code to JavaScript and directly produces module and source 
                         archives in a module repository

        * 'config' - Manages Ceylon configuration files

        * 'doc' - Generates Ceylon API documentation from Ceylon source files

        * 'doc-tool' - Generates documentation about a tool

        * 'help' - Displays help information about other Ceylon tools

        * 'import-jar' - Imports a jar file into a Ceylon module repository

        * 'info' - Prints information about modules in repositories

        * 'new' - Generates a new Ceylon project

        * 'run' - Executes a Ceylon program

        * 'run-js' - Executes a Ceylon program

        * 'src' - Fetches source archives from a repository and extracts their contents into a source directory

        * 'test' - Executes tests

        * 'version' - Shows and updates version numbers in module descriptors

        See 'ceylon help <command>' for more information about a particular subcommand.


OPTIONS

        --stacktraces
            If an error propagates to the top level tool, print its stack trace.


        --version
            Print version information and exit, ignoring all other options and arguments.

Yes it’s a man page, this is what is useful and will help you find your way out of the many ceylon subcommands we have.

A similar thing will happen if you type ceylon compile --help or ceylon help compile.

Completion

We ship with completion support for bash:

$ . /usr/share/ceylon/1.0.0/contrib/scripts/ceylon-completion.bash 
$ ceylon [TAB]
all         compile-js  doc         help        info        run         src         version     
compile     config      doc-tool    import-jar  new         run-js      test        
$ ceylon compile[TAB]
compile     compile-js  
$ ceylon compile --re[TAB]
--rep\=       --resource\=  

As you can see, it’s quite useful, and again, that’s what you expect in this day and age.

Documentation

The tool system that we wrote (well, mostly that Tom Bentley wrote) even allows us to generate HTML, XML and text documentation automatically, for example, the tool documentation pages are entirely generated, and the man pages that we ship in the Ceylon CLI distribution are also generated.

Plugins, plugins

You may not have noticed it but if you type ceylon help on your system, it will likely not include that line:

[...]
DESCRIPTION

        [...]

        * 'all' - Compiles ceylon modules for JVM and JS and documents them

And that’s because, like Git, we support CLI plugins, and ceylon all is a plugin I wrote for myself, that aggregates a bunch of subcommands in a single one.

Ceylon CLI plugins are as straightforward as Git CLI plugins: any executable in your path which starts with ceylon- will be picked up to be a plugin. It will be listed and documented too. If you add documentation in your plugin it will be used.

Here’s my ceylon-all shell script for example:

#!/bin/sh

USAGE='[generic options] module...'
DESCRIPTION='Compiles ceylon modules for JVM and JS and documents them'
LONG_USAGE='ceylon-all allows you to build the specified ceylon modules for the
JVM and JS backends, and generates the API documentation in a single command.'

. $CEYLON_HOME/bin/ceylon-sh-setup

$CEYLON_HOME/bin/ceylon compile $@
$CEYLON_HOME/bin/ceylon compile-js $@
$CEYLON_HOME/bin/ceylon doc $@

As you can see, our CLI will use the USAGE, DESCRIPTION and LONG_USAGE environment variables for completion and documentation. You only have to source the provided $CEYLON_HOME/bin/ceylon-sh-setup shell script to benefit from this. Then you can run whatever you want.

Configuration

There are a lot of options the CLI and IDE have good defaults for, such as the source path, or the output module repository, or the list of module repositories. But when you need something different it gets tiring really fast to have to specify them for every command, so we support configuration files just like Git, in the form of INI files located in your project at .ceylon/config. If that file does not exist or a particular configuration option is not defined in it, we will also try the user’s config in $HOME/.ceylon/config as a fallback.

The syntax is straightforward, take for example the .ceylon/config for the Ceylon SDK:

[repositories]
output=./modules
lookup=./test-deps

[defaults]
encoding=UTF-8

It just specifies that the output repository is modules, the additional lookup repository is test-deps (for test dependencies) and the source encoding is UTF-8.

There’s a ceylon config command which allows you to edit this file if you don’t want to fire up vim for that, and the best part is that this config file is actually used and edited by the IDE too! So no more hassle to keep the CLI and IDE settings in sync.

Welcome to the future! :)

It's often argued that dynamic typing is more expressive, and I am, for the most part, willing to go along with that characterization. By nature, a language with dynamic typing places fewer constraints on me, the programmer, and lets me "express myself" more freely. I'm not going to, right now, re-argue the case for static typing, which is anyway quite clear to anyone who has ever written or maintained a decent-sized chunk of code using an IDE. However, I would like to point out a particular sense in which dynamic typing is much less expressive than static typing, and the consequences of that.

(Now, please bear with me for a bit, because I'm going to take a rather scenic route to getting to my real point.)

In a nutshell, dynamic typing is less expressive than static typing in that it doesn't fully express types.

Of course that's a rather silly tautology. But it's still worth saying. Why? Because types matter. Even in a dynamically-typed language like Python, Smalltalk, or Ruby, types still matter. To understand and maintain the code, I still need to know the types of things. Indeed, this is true even in weakly-typed JavaScript!

The consequence of this is that dynamically-typed code is far less self-documenting than statically typed code. Quick, what can I pass to the following function? What do I get back from it?

function split(string, separators)

Translated to Ceylon, this function signature is immediately far more understandable:

{String*} split(String string, {Character+} separators)

What this means, of course, is that dynamic typing places a much higher burden on the programmer to conscientiously comment and document things. And to maintain that documentation.

On the other hand, static typing forces me to maintain the correct type annotations on the split() function, even when its implementation changes, even when I'm in a hurry, and my IDE will even help me automatically refactor them. No IDE on Earth offers the same kind of help maintaining comments!

Now consider what else this extra expressiveness buys me. Suppose that Foo and Bar share no interesting common supertype. In any dynamic language on earth, I could write the following code:

class Super {
    function fun() { return Foo(); }
}

class Sub extends Super {
    function fun() { return Bar(); }
}

But if I did write such a thing, my team-mates would probably want to throttle me! There's simply too much potential here for a client calling fun() to only handle the case of Foo, and not realize that sometimes it returns a Bar instead. Generally, most programmers will avoid code like the above, and make sure that fun() always returns types closely related by inheritance.

As a second example, consider a well-known hole in the typesystem of Java: null. In Java, I could write:

class Super {
    public Foo fun() { return Foo(); }
}

class Sub extends Super {
    public Foo fun() { return null; }
}

Again, this is something Java developers often avoid, especially for public APIs. Since it's not part of the signature of fun() that it may return null, and so the caller might just obliviously not handle that case, resulting in an NPE somewhere further down the track, it's often a good practice to throw an exception instead of returning null from a method belonging to a public API.

Now let's consider Ceylon.

class Super() {
    shared default Foo? fun() => Foo();
}

class Sub() extends Super() {
    shared actual Null fun() => null;
}

Since the fact that fun() can return null is part of its signature, and since any caller is forced by the typesystem to account for the possibility of a null return value, there's absolutely nothing wrong with this code. So in Ceylon it is often a better practice to define functions or methods to just return null instead of throwing a defensive exception. Thus, we arrive at an apparent paradox:

By being more restrictive in how we handle null, we make null much more useful.

Now, since a "nullable type" in Ceylon is just a special case of a union type, we can generalize this observation to other union types. Consider:

class Super() {
    shared default Foo|Bar fun() => Foo();
}

class Sub() extends Super() {
    shared actual Bar fun() => Bar();
}

Again, there's absolutely nothing wrong with this code. Any client calling Super.fun() is forced to handle both possible concrete return types.

What I'm saying is that we achieved a nett gain in expressiveness by adding static types. Things that would have been dangerously error-prone without static typing have become totally safe and completely self-documenting.

Warning: the following post is for people who enjoy thinking about types, and want to understand how the Ceylon compiler reasons about them. This post is partly an attempt to demystify §3.7 of the spec. To understand the following, you first need to understand how variance works in Ceylon.

Snap quiz!

Consider:

MutableList<Topic> | MutableList<Queue> list;

1. What is the type of list.first?
2. What is the type of the parameter of list.add()?

Background

One of the key goals of Ceylon's type system is to make subtyping and generics play nicely together. Almost every object-oriented language since C++ and Eiffel has tried this, and more recent languages have been getting gradually closer to a really convincing solution, without, in my opinion, completely nailing it. Ceylon's contribution, of course, is to introduce first-class union and intersection types to the mix. In this post we'll see one of the ways in which they help.

Four useful identities

To begin to fully understand how the Ceylon typechecker reasons about generic types, you need to grasp the central importance of the following identities:

1. Given a covariant type List<Element>, the union List<String>|List<Integer> is a subtype of List<String|Integer>.
2. Given a contravariant type ListMutator<Element>, the union ListMutator<Queue>|ListMutator<Topic> is a subtype of ListMutator<Queue&Topic>.
3. Given a covariant type List<Element>, the intersection List<Queue>&List<Topic> is a subtype of List<Queue&Topic>.
4. Given a contravariant type ListMutator<Element>, the intersection ListMutator<String>&ListMutator<Integer> is a subtype of ListMutator<String|Integer>.

Note that in each identity, we take a union or intersection of two instantiations of the same generic type, and produce an instantiation of the generic type that is a supertype of the union or intersection.

These identities are actually quite intuitive. No? OK then, let's consider them one at a time, and convince ourselves that they make sense:

1. If we have a list that, when iterated, either produces Strings or produces Integers then clearly every object it produces is either a String or an Integer.
2. If we have a list in which we can either insert Queues or insert Topics, then we clearly have a list in which we can insert something that is both a Queue and a Topic.
3. If we have a list that, when iterated produces only Queues, and, when iterated, produces only Topics, then clearly every object it produces must be both a Queue and a Topic.
4. If we have a list in which we can both insert a String and insert an Integer, then we clearly have a list in which we can insert something that we know is either a String or an Integer.

Satisfied? Good.

Generic supertypes and principal instantiations

So, how are these identities useful? Well, imagine that we have a type T whose supertypes include multiple distinct instantiations of a generic type. For example, imagine that T has the supertypes List<String> and List<Integer>. And suppose we want to determine the type of one of the members inherited by T from the generic type, say List.first.

Then we would first need to form a principal instantiation of the generic type List. In this case the principal instantiation would be List<String|Integer>, according to identity 1. It's a "principal" instantiation because every other instantiation of List that is a supertype of T is also a supertype of List<String|Integer>. That it is even possible to form a principal instantiation like this is one of the things that makes Ceylon's type system special. Now we can determine the type of first by substituting String|Integer for the type parameter of List. For the record, the result is the type String|Integer|Null.

A type like T arises when we explicitly write down an intersection like List<Queue>&List<Topic>, or a union like List<String>|List<Integer>, but it also arises through the use of inheritance.

Principal instantiation inheritance

In Java, a type can only inherit a single instantiation of a supertype. A class can't inherit (either directly or indirectly) both List<Queue> and List<Topic> in Java. We call this inheritance model single instantiation inheritance. Ceylon features a more flexible model called principal instantiation inheritance¹, where this restriction does not apply. I'm allowed to write:

interface Queues satisfies List<Queue> { ... }
interface Topics satisfies List<Topic> { ... }

class Endpoints() satisfies Queues & Topics { ... }

In which case, Endpoints is a subtype of List<Queue>&List<Topic>. Then the typechecker uses identity 3 above to automatically infer that Endpoints is a List<Queue&Topic>. Other languages can't do this.

Problem: invariant types

Now notice something about the identities above: they don't say anything about invariant types. Unfortunately, I simply can't form a principal instantiation of MutableList that is a supertype of MutableList<Queue>&MutableList<Topic>. Or at least I can't within Ceylon's type system as it exists today.

Caveat: I'm not a fan of use-site variance. I've been too burned by it in Java. However, if we do ever decide to introduce use-site variance in Ceylon, which does remain a real possibility, what I've just said will no longer apply, since this instantiation:

MutableList<in Queue&Topic out Queue|Topic> 

would be a principal supertype for MutableList<Queue>&MutableList<Topic>. But who the hell wants to have to deal with nasty-looking types like that?

Conclusion

The significance of this is that we should, where reasonable, be careful with how we use invariant types in Ceylon. To be sure, we still need invariant types, especially for concrete implementations like the class Endpoints above, but we should try to declare public APIs on covariant and contravariant types.

Consider the case of MutableList. What I'm saying is that its interesting operations should all be inherited from the covariant List and the contravariant ListMutator. It doesn't matter much whether we declare MutableList as a mixin interface like this:

shared interface MutableList<Element> 
        satisfies List<Element> & 
                  ListMutator<Element> { ... }

Or just as a type alias, like this:

shared alias MutableList<Element> 
        => List<Element> & ListMutator<Element>;

In either case, the following code² is accepted by the typechecker, assuming the interesting operations first and add() are defined on List and ListMutator respectively:

void doSomeStuff(MutableList<Topic>|MutableList<Queue> list) {
    Topic|Queue|Null topicOrQueue = list.first;
    if (!topicOrQueue exists) {
        Topic&Queue topicAndQueue = .... ;
        list.add(topicAndQueue);
    }
    ...
}

Here we're calling both "producer" and "consumer" operations of the invariant type MutableList on an intersection of distinct instantiations of MutableList. The typechecker determined that the principal instantiation of the covariant supertype is List<Topic|Queue> and that the principal instantiation of the contravariant supertype is ListMutator<Topic&Queue>, and that therefore the code is sound.

¹The notion of principal instantiation inheritance is mainly based on Ross Tate's research. As far as I know, the idea of combining principal instantiation inheritance with union and intersection types is original to Ceylon.

²Thanks for Riener Zwitzerloot for this example and for inspiring this post.

Today, we're proud to announce the first production release of the Ceylon language specification, compiler, and IDE. Ceylon 1.0 is a modern, modular, statically typed programming language for the Java and JavaScript virtual machines.

Ceylon enables the development of cross-platform modules that execute portably in both virtual machine environments. Alternatively, a Ceylon module may target one or the other platform, in which case it may interoperate with native code written for the platform.

In the box

This release includes:

• a complete formal language specification that defines the syntax and semantics of Ceylon in language accessible to the professional developer,
• a command line toolset including compilers for Java and JavaScript, a documentation compiler, and support for executing modular programs on the JVM and Node.js,
• a powerful module architecture for code organization, dependency management, and module isolation at runtime,
• the language module, our minimal, cross-platform foundation of the Ceylon SDK, and
• a full-featured Eclipse-based integrated development environment.

Language

Ceylon is a highly understandable object-oriented language with static typing. The language features:

• an emphasis upon readability and a strong bias toward omission or elimination of potentially-harmful constructs,
• an extremely powerful type system combining subtype and parametric polymorphism with declaration-site variance, including first-class union and intersection types, and using principal types for local type inference and flow-sensitive typing,
• a unique treatment of function and tuple types, enabling powerful abstractions,
• first-class constructs for defining modules and dependencies between modules,
• a very flexible syntax including comprehensions and support for expressing tree-like structures, and
• fully-reified generic types, on both the JVM and JavaScript virtual machines, and a unique typesafe metamodel.

More information about these language features may be found in the feature list and quick introduction.

This release does not introduce new language features. However, a very large number of bugs have been fixed.

IDE

Screenshots of the IDE may be seen here.

This release of the IDE introduces the following new features:

• a type hierarchy view, to complement the popup type hierarchy,
• a documentation view, to complement the documentation hover,
• a new Ceylon Module properties page, and the ability to manage dependencies from this page or from the New Ceylon Module wizard,
• enhancements to the search results view,
• improvements to syntax highlighting in the hover,
• a much improved wizard for importing Java archives into Ceylon module repositories, and
• many bugfixes.

Community module repository

Ceylon Herd is now open to the public.

SDK

The platform modules, recompiled for 1.0.0, are available in the shared community repository, Ceylon Herd.

This release introduces the following new platform modules:

• ceylon.build, a framework for writing build scripts in Ceylon, and
• ceylon.html, a library for defining HTML content in Ceylon.

Community

The Ceylon community site, http://ceylon-lang.org, includes documentation, and information about getting involved.

Source code

The source code for Ceylon, its specification, and its website is freely available from GitHub.

Issues

Bugs and suggestions may be reported in GitHub's issue tracker.

Acknowledgement

We're deeply indebted to the community volunteers who contributed a substantial part of the current Ceylon codebase, working in their own spare time. The following people have contributed to this release:

Gavin King, Stéphane Épardaud, Tako Schotanus, Emmanuel Bernard, Tom Bentley, Aleš Justin, David Festal, Flavio Oliveri, Max Rydahl Andersen, Mladen Turk, James Cobb, Tomáš Hradec, Michael Brackx, Ross Tate, Ivo Kasiuk, Enrique Zamudio, Roland Tepp, Diego Coronel, Daniel Rochetti, Loic Rouchon, Matej Lazar, Corbin Uselton, Akber Choudhry, Lucas Werkmeister, Julien Viet, Brent Douglas, Lukas Eder, Markus Rydh, Julien Ponge, Pete Muir, Henning Burdack, Nicolas Leroux, Brett Cannon, Geoffrey De Smet, Guillaume Lours, Gunnar Morling, Jeff Parsons, Jesse Sightler, Oleg Kulikov, Raimund Klein, Sergej Koščejev, Chris Marshall, Simon Thum, Maia Kozheva, Shelby, Aslak Knutsen, Fabien Meurisse, Paco Soberón, Sjur Bakka, Xavier Coulon, Ari Kast, Dan Allen, Deniz Türkoglu, F. Meurisse, Jean-Charles Roger, Johannes Lehmann, Nikolay Tsankov.

With the release of 1.0 beta, we reached the end of our previous roadmap. This doesn't mean we've run out of work to do, so of course we needed a new one. Here's a sketch of our plan:

Ceylon 1.0 final

The focus of development right now is of course bug fixes to the compiler backends, language module, and IDE. The major new feature planned for 1.0 final is serialization, which, even though we think of it as a language feature, is really, strictly speaking, a feature of the language module.

In parallel, David Festal is working on some very-needed optimizations to build performance in Ceylon IDE, though that work won't be finished in 1.0 final. I found the time this weekend to squeeze a couple of cool new features into the IDE, but that's not supposed to be the priority for this release.

There's also some issues we need to work through with interop between our module system and Maven. I'm not sure whether we'll be able to get this all sorted out in 1.0, or whether this will slip (again) to 1.1.

Finally, we hope to have a couple of new platform modules ready for the 1.0 final release, including ceylon.transaction.

Ceylon 1.1

Ceylon 1.1 will be all about performance, including language performance, compiler performance, and David's ongoing work on IDE build performance. There are a whole bunch of really obvious optimizations we can make, that we simply havn't had time to work on yet.

A warning: we expect to break binary compatibility between 1.0 and 1.1. That's not something we do lightly, and it's not something we plan to make a habit of. Changes affecting binary compatibility should occur in major releases, not minor releases. Please forgive us if we break our own rule this one time.

At the same time, building out the Ceylon SDK will also be a top priority. This will probably be what I personally focus on for a while.

Ceylon 1.2

Ceylon 1.2 will introduce a number of new language features. We're not completely sure of precisely which new features will make the cut, though we do have a shortlist of several which are almost certain to make it in. I'm not talking about major new features of the type system or anything here. We're really very happy with what's possible in the language as it exists today. The priority will be additional syntax for dealing with situations that are a little uncomfortable right now, and removal of a handful of limitations introduced temporarily for 1.0.

Here's a list of some of the things under very serious consideration for 1.2. Don't expect all of them to make the cut. We want to make the language easy to use. That means fixing things that suck without bloating out the language with too many overlapping features and minor syntax variations.