Blog of Gavin King

Tuple and entry destructuring

The next release of Ceylon features an interesting range of new language features, including constructors, if and switch expression, let and object expressions, and destructuring of tuples and entries. In this post, I'm going to describe our new syntax for destructuring.

A destructuring statement looks a lot like a normal value declaration, except that where we would expect to see the value name, a pattern occurs instead.

An entry pattern is indicated using the skinny arrow -> we use to construct entries:

String->Integer entry = "one"->1;
value key->item = entry;    //destructure the Entry

A tuple pattern is indicated with brackets:

[String,Integer] pair = ["one",1];
value [first,second] = pair;    //destructure the Tuple

The pattern variables, key, item, first, and second are just regular local values.

We can nest tuple and entry patterns:

String->[String,Integer] entry = "one"->["one",1];
value key->[first,second] = entry;

A tuple pattern may have a tail variable, indicated with a *:

[String+] ints = 1..100;
value [first,*rest] = ints;    //destructure the Sequence

(This syntax resembles the spread operator.)

Patterns may optionally indicate an explicit type:

value String key->[String first, Integer second] = entry;

Pattern-based destructuring can occur in a number of other places in the language. A pattern can occur in a for loop:

for ([x, y] in points) { ... }

for (key->item in map) { ... }

Or in an exists or nonempty condition:

if (exists index->item = stream.indexed.first) { ... }

if (nonempty [first,*rest] = sequence) { ... }

Or in a let expression:

value dist = let ([x,y] = point) sqrt(x^2+y^2);

You might wonder why we decided to introduce this syntax, or at least, why we decided to do it now. Well, I suppose the simple answer is that it always felt a bit incomplete or unfinished to have a language with tuples but no convenient destructuring syntax for them. Especially when we did already have destructuring for entries, but only in for, as a special case.

But looking into the future, you could also choose to see this as us dipping our toes in the water of eventual support for pattern matching. I remain ambivalent about pattern matching, and it's certainly not something we find that the language is missing or needs, but lots of folks tell us they like it in other languages, so we're keeping our options open. Fortunately, the syntax described above will scale nicely to more complex patterns in a full pattern matching facility.

This functionality is already implemented and available in github.

Useless lying version ranges

A frequent request from the Ceylon community is support for version ranges in expressing module dependencies. There's no doubt that our current module system is too inflexible in terms of dependency resolution in the face of version conflicts, and I have some reasonable ideas about how to address that problem without needing version ranges. But I would like to document precisely why I think version ranges are strictly-speaking useless at best, and harmful at worst.

First, a philosophical point: version ranges encourage module authors to make untested or untestable claims about their modules, such as:

my.module is compatible with other.dependency version 2.x.

Yeah, right, 'cos you've actually tested my.module with every single minor version and point release of other.dependency, including all the versions that have not even been released yet! Sorry, but I simply don't believe you and I have to assume you're lying to me. Almost nobody tests their program or library with many different versions of its dependencies, and it's easy to see why they don't: as soon as we have a program with several dependencies, we face a factorial explosion of dependency version combinations.

OK, sure, you might argue, but version ranges are still better than nothing. Alright, alright. I'm not the kind of guy who much buys into the notion that something broken is better than nothing, but I realize I'm in the minority on that one, 'cos, y'know, worse is better, as the neckbeards keep telling me.

So let's see what we could do to make the most of version ranges. Let's consider the problem first from the point of view of two library authors assigning version ranges to their modules, and then from the point of view of the program or person assembling these modules.

Let's suppose my library x.y depends on org.hibernate. I tested and released x.y with the then-current version of org.hibernate, which was 4.1.3. What version range would I have chosen when declaring this dependency?

  • Well, for the lower bound I decided to lie and write 4.1.0. Typically, I hadn't actually tested x.y with versions 4.1.0, 4.1.1 and 4.1.2, but I had been using 4.1.2 in development at one stage and it seemed to work, and I didn't see any particular reason it wouldn't also work with 4.1.0 and 4.1.1. On the other hand, maybe I could have just picked 4.1.3. (It's not going to matter for the rest of this argument.)
  • For the upper bound, I had no clue. How could I possibly have known at the time which future unreleased version of org.hibernate would break my library? Assigning the upper bound of 4.1.3 would have seemed much too restrictive, so what I did was assume that org.hibernate is following the completely untestable semantic versioning standard, and that future versions of Hibernate 4.x would not have any bugs.

Thus, I arrived at the version range 4.[1.0-] using some imaginary syntax I just pulled out of my ahem, excuse me, invented for the sake of argument.

A critical thing to notice here is that, from the point of view of the library authors, there is no reasonable way to determine an accurate upper bound to the version range. This is utterly typical and normal and is the case for almost any library author!

Now, sometime later, you released your library a.b, which also depends on org.hibernate. At this point, the current release of org.hibernate was 4.2.0. Quite atypically, you actually do test your library with a previous version of its dependency, and so you know that it is actually compatible with 4.1.5 (the latest release of 4.1.x). Thus, you arrive at the version range 4.[1.5-].

Now suppose some poor soul wants to use both our libraries together in their program, thus taking advantage of the bugs in both of them. So now, when assembling the program, what version of org.hibernate should the module system choose. Let's consider the reasonable options:

  • Pick the latest release that fits the version ranges, that is, the latest release of 4.x. This approach means that a new release of org.hibernate can break our application. We're picking a release which hasn't been tested with either x.y or a.b. Not acceptable.
  • Pick the earliest release that fits the version ranges, in this case 4.1.5. This is better. At least there's a chance that one of the libraries (in this case, a.b) has actually been tested with this version. Still, according to this strategy the system could in general pick a dependency version that's earlier than all the current versions when the libraries were developed. That seems quite suboptimal.
  • Pick 4.2.0, since that was the current version of org.hibernate when one of the libraries was developed, so we know for a fact that it works with at least one of the libraries, and it's newer that the version that the other library was developed with. This seems to me like it's by far the most robust and natural strategy.

There are some variations on this scenario, which raise other possible choices, and I'm going to let you experiment with the variations yourself, and see how it affects the conclusion. But as far as I can tell, there's essentially no common scenario in which the third strategy isn't at least as good as any other possible strategy.

And now note that this third strategy doesn't actually use version ranges at all! We can write down this strategy without reference to version ranges. It just says: pick the latest version of the dependency among the versions with which the libraries were developed. Version ranges don't really add any useful additional information to that, especially in light of the fact that upper bounds are essentially impossible to determine, and even lower bounds often lie. Why inject additional inaccurate information into the mix when we already have an algorithm that produces a result without depending on guesses and lies and unverifiable assumptions? (I apologize for going all logical positivist on your arse.)

What do you think? Am I wrong? Is there some reasonably common scenario where the module system can be expected to produce a better outcome with the addition of version range information? Is there a scenario in which upper bound information doesn't lie? Is there a strategy involving version ranges that is unambiguously better than my admittedly unsophisticated "pick the latest version" approach?

New features coming soon

We've been spending time discussing the priorities for development of Ceylon 1.1.5 and 1.2, including soliciting community feedback. The plan is still suprisingly fluid right now, but there are a number of things that we've already started working on, or have decided to start working on, and so in the interest of transparency, I thought I would share them.

Warning: we're not committing to a timeframe or release version for most of these features. It's merely a summary of what we're working on now, or plan to start work on soon.

Serialization

As already announced, ceylon.language 1.1.5 will feature an API for Serialization. Note that this API does not itself specify a serialization format. Rather, it's a general-purpose and platform-neutral facility for marshalling objects to and from a serialized stream. Serialization libraries founded on this API may serialize to text-based formats like JSON or XML, to binary formats, or even to a database via ORM.

Work on this API is already well-advanced. Tom has already done the Java implementation, and Enrique has got it working in JavaScript.

Type argument inference for function references

In Ceylon 1.1, we made it possible to leave off the type of a parameter of an anonymous function that occurs in an argument list, letting the type be inferred by the compiler, for example:

{Float*} measurements = ... ;
Float product = measurements.fold(1.0)((x,y)=>x*y);

In Ceylon 1.1.5, I've extended this approach to cover references to generic functions. So now, instead of this:

{Float*} measurements = ... ;
Float product = measurements.fold(1.0)(times<Float>);

You can write this:

{Float*} measurements = ... ;
Float product = measurements.fold(1.0)(times);

This even works for static value references, so instead of this:

{[Float+]*} sequences = ... ;
{Float*} heads = sequences.map(Iterable<Float>.first);

You can write simply this:

{[Float+]*} sequences = ... ;
{Float*} heads = sequences.map(Iterable.first);

This is already implemented, and you can try it out in git. It will be released in Ceylon 1.1.5.

Named constructors

In Ceylon 1.1, there is only one "constructor" of a class, the body of the class itself. For the vast majority of classes this is far more elegant and convenient. But in a minority of cases, there is a true need to have multiple initialization paths, and so we've designed a new syntax to support that. It took us a while to come up with something elegant and regular that didn't break the block structure of the language or the rules about definite initialization, but I'm very happy with the final outcome.

Since Ceylon doesn't have overloading (except for Java interop), constructors have distinct names.

class Point {
    shared Float x;
    shared Float y;

    //the "default" constructor
    shared new Point(Float x, Float y) {
        this.x = x;
        this.y = y;
    }

    //an additional constructor
    shared new Diagonal(Float d) {
        x = (d^2/2)^0.5 * d.sign;
        y = x;
    }
}

Every constructor must initialize all members which are left uninitialized by the body of the class, and must delegate to a constructor of the superclass (in this case, they delegate to Basic() by default). Now we can create a Point in two different ways:

Point p1 = Point(2.0, 3.0);  //call the default constructor
Point p2 = Point.Diagonal(1.0);

The typechecker already supports constructors, and Tom has made good progress on implementing this feature for the Java backend. I'm not sure if this will make it into 1.1.5, but if it does then we might actually need to rename 1.1.5 to 1.2, given that this is a pretty significant enhancement to the language itself.

Extensions to the expression syntax

We're making several extensions to the expression syntax. These features are already supported in the typechecker, but not yet by the backends. Note that these features are especially useful when combined with certain other features of the language, like comprehensions, anonymous functions, named argument lists, and fat arrow function definitions.

Inline object expressions

An inline anonymous object expression is very similar to an anonymous class in Java, and is useful in essentially the same cases. For example:

printAll(object satisfies {Integer+} {
    iterator() =>
        object satisfies Iterator<Integer> {
            variable value current = 0;
            next() => current++;
        };
});

let expressions

A let expression allows the definition of new values within an expression. For example:

Float d = ... ;
value ptl = let (x = (d^2/2)^0.5 * d.sign) Point(x,x);

Inline if and switch expressions

Ceylon's then and else operators are nice, but they don't do anything special in terms of flow-sensitive typing, so we quite often run into cases where we're forced to use a whole if or switch statement in a block. To alleviate that minor source of discomfort, we're now going to let you use if and switch within expressions. For example:

String string(Object it)
        => if (is Person it) 
           then it.name 
           else it.string;

Or:

String name(Person|Org it)
        => switch (it) 
           case (is Org) it.tradingName 
           case (is Person) it.firstName + " " + it.lastName;

Cayla web framework

Frameworks for developing web applications are a top request from the community. After some discussion, we've decided to focus first on the server side, and come back later to the problem of client-side web frameworks. Note that there's no problem at all with using a native JS client-side web framework to call a Ceylon module compiled to JavaScript.

Julien is going to work on getting Cayla, a web framework for use on Vert.x, ready for release.

To showcase Cayla, Ceylon, and Vert.x, Julien is going to do a partial port of Ceylon Herd from Java/Play to Ceylon/Cayla. That should make for a great demo.

SDK modules ceylon.html and ceylon.promise

Cayla will offer a choice of templating technologies, but one of the options we obviously want to offer is templates written in Ceylon. In order to avoid the cost of rebuilding the template from scratch on each request, ceylon.html needs to be enhanced to support a mix of static nodes and nodes which are created or rendered dynamically.

Work on Cayla will also likely necessitate improvements to ceylon.promise, and, in particular, we need to make this module cross-platform (right now it is only available on the JVM).

Java EE integration

Toby Crawley has started work on integration with Java EE. The first order of business here is to make it easy to write a servlet in Ceylon and package it into a war archive. After that, we'll need to make sure Ceylon works well with CDI, JPA, JAX-RS, etc.

Improved debugging in Ceylon IDE

David is going to work on making Eclipse's debugger work better with Ceylon. This is now the only really major feature missing from Ceylon IDE, so when he's done with that, he's going to move onto the #1 requested feature from the Ceylon community, which is...

IntelliJ-based IDE for Ceylon

The IntelliJ plugin for Ceylon is still rudimentary, and not yet ready for release. But now that the Eclipse-based IDE is feature complete, we're going to refocus our tooling development efforts on IntelliJ.

Note that this doesn't really represent a change of direction for us; I'm an Eclipse user, I prefer Eclipse, and I see no good reason to change to IntelliJ. That's especially true since whenever I discover a nice feature of IntelliJ, I just go ahead and reimplement it in Ceylon IDE ;-) However, we recognize that there are plenty of folks on the other side of the fence, who, preferring IntelliJ, and likewise seeing no reason to change, deserve a great plugin for Ceylon. So I hereby promise that we will have absolutely awesome tooling for both these IDEs.

Source maps

To make it easy to debug Ceylon code running on a JavaScript virtual machine, Enrique is going to add support for source maps to ceylon compile-js.

Consume Typescript interface definitions

Microsoft's Typescript project (which recently took inspiration from Ceylon by adopting our approach to union types and flow sensitive typing) has put a whole lot of work into defining statically typed definitions of important APIs in the JavaScript world. Now that Ceylon 1.1 has dynamic interfaces it's at least in principle possible to have a well-defined transformation from a Typescript API definition to a Ceylon type. This could take the form of a mechanical source translator, or even a "model loader" for the Ceylon compiler. Stef is going to investigate this.

More

The above is an incomplete list. If the thing you're waiting for (Android!!) isn't on that list, that doesn't mean we don't want to work on it, it just means I don't yet have a concrete plan for actually starting work on it right now. Feel very welcome to bug us about it in comments or on the mailing list or IRC :-)

Planning the future of Ceylon 1.x

With the release of Ceylon 1.1, we've reached a point where we need to do some serious thinking about what are our priorities for the development of Ceylon 1.1.5, 1.2, and beyond. I definitely don't yet have a crystal clear vision of what is going to be in 1.2, so we're also looking for community feedback on this.

I do know of one item which is the top priority right now, and will be the main feature of Ceylon 1.1.5:

This was a feature that slipped from Ceylon 1.0, and which again narrowly missed out on inclusion in Ceylon 1.1. The concept behind serialization in Ceylon is to have an API responsible for assembling and disassembling objects that is agnostic as to the actual format of the serialized stream. Of course, this API also has to be platform neutral, in order to allow serialization between programs running on the JVM and programs running on a JavaScript VM. Tom Bentley already has a working prototype implementation. Once this feature is done, we can start working on serialization libraries supporting JSON and whatever else.

I also count the following as a high priority areas of work:

  • Java EE integration, and support for technologies like JPA and CDI.
  • Adding properties to the language, that is, a new syntax for attribute references, allowing easy MVC UI bindings.
  • Improving the Cayla web framework, and ceylon.html.

Beyond that, we're not sure where else we should concentrate development effort. Here are some things that stick out to me:

  • Addition of named constructors, allowing multiple ways to instantiate and initialize a class.
  • AST transformers—a system of compiler plugins, based around ceylon.ast, enabling advanced compile-time metaprogramming, which would form the foundation for LINQ-style queries, interceptors and proxies, and autogeneration of equals(), hash, and string, and more.
  • Addition of a syntax for expressing patterns in BNF.
  • The Ceylon plugin for IntelliJ IDEA.
  • Android support.
  • Assemblies—a facility for packaging multiple modules into a deployable "application".
  • New platform modules defining dynamic interfaces for typesafe interaction with JavaScript APIs such as the DOM, jQuery, etc.
  • Interoperation with dynamic languages on the JVM, via Ceylon's dynamic blocks and dynamic interfaces.
  • Enabling the use of Ceylon for scripting.

We can't do all of this in Ceylon 1.2. Therefore, we're looking for feedback from the community. Let us know, here in comments, or on the mailing list, what you feel is missing from Ceylon, either from the above list, or whatever else you think is important.

Typesafe APIs for the browser

A new feature in Ceylon 1.1, that I've not blogged about before, is dynamic interfaces. This was something that Enrique and I worked on together with Corbin Uselton, one of our GSoC students.

Ordinarily, when we interact with JavaScript objects, we do it from within a dynamic block, where Ceylon's usual scrupulous typechecking is suppressed. The problem with this approach is that if it's an API I use regularly, my IDE can't help me get remember the names and signatures of all the operations of the API.

Dynamic interfaces make it possible to ascribe static types to an untyped JavaScript API. For example, we could write a dynamic interface for the HTML 5 CanvasRenderingContext2D like this:

dynamic CanvasRenderingContext2D {
    shared formal variable String|CanvasGradient|CanvasPattern fillStyle;
    shared formal variable String font;

    shared formal void beginPath();
    shared formal void closePath();

    shared formal void moveTo(Integer x, Integer y);
    shared formal void lineTo(Integer x, Integer y);

    shared formal void fill();
    shared formal void stroke();

    shared formal void fillText(String text, Integer x, Integer y, Integer maxWidth=-1);

    shared formal void arc(Integer x, Integer y, Integer radius, Float startAngle, Float endAngle, Boolean anticlockwise);
    shared formal void arcTo(Integer x1, Integer y1, Integer x2, Float y2, Integer radius);

    shared formal void bezierCurveTo(Integer cp1x, Integer cp1y, Integer cp2x, Float cp2y, Integer x, Integer y);

    shared formal void strokeRect(Integer x, Integer y, Integer width, Integer height);
    shared formal void fillRect(Integer x, Integer y, Integer width, Integer height);
    shared formal void clearRect(Integer x, Integer y, Integer width, Integer height);

    shared formal CanvasGradient createLinearGradient(Integer x0, Integer y0, Integer x1, Integer y1);
    shared formal CanvasGradient createRadialGradient(Integer x0, Integer y0, Integer r0, Integer x1, Integer y1, Integer r1);
    shared formal CanvasPattern createPattern(dynamic image, String repetition);

    //TODO: more operations!!
}

dynamic CanvasGradient {
    shared formal void addColorStop(Integer offset, String color);
}

dynamic CanvasPattern {
    //todo
}

Now, if we assign an instance of JavaScript's CanvasRenderingContext2D to this interface type, we won't need to be inside a dynamic block when we call its methods. You can try it out in your own browser by clicking the "TRY ONLINE" button!

CanvasRenderingContext2D ctx;
dynamic {
    //get the CanvasRenderingContext2D from the 
    //canvas element using dynamically typed code
    ctx = ... ;
}

//typesafe code, checked at compile time 
ctx.fillStyle = "navy";
ctx.fillRect(50, 50, 235, 60);
ctx.beginPath();
ctx.moveTo(100,50);
ctx.lineTo(60,5);
ctx.lineTo(75,75);
ctx.fill();
ctx.fillStyle = "orange";
ctx.font = "40px PT Sans";
ctx.fillText("Hello world!", 60, 95);

Notice that we don't need to ascribe an explicit type to every operation of the interface. We can leave some methods, or even just some parameters of a method untyped, by declaring them dynamic. Such operations may only be called from within a dynamic block, however.

A word of caution: dynamic interfaces are a convenient fiction. They can help make it easier to work with an API in your IDE, but at runtime there is nothing Ceylon can do to ensure that the object you assign to the dynamic interface type actually implements the operations you've ascribed to it.