Blog tagged ceylon-js

Ceylon in the browser (again)

Bastien Jansen
15 February 2016
ceylon-js sdk interop

As you might (or might not) know, Ceylon is more than a JVM language. It has been possible to compile Ceylon code to JavaScript for a long time, but other platforms such as Dart or LLVM are around the corner.

Having a JS backend means that you can actually write Ceylon code that can be run in a web browser, giving the opportunity to share code between the server and the client. The web IDE is a very good example of this. Up until now, using Ceylon in a browser wasn't really straightforward though. The good news is, Ceylon 1.2.1 brought two major features that overcome this problem:

Let's see how these fit together.

Creating a new project

First things first, we need an empty project that will hold two modules:

  • com.acme.client is a native("js") module that imports ceylon.interop.browser:

    native("js") module com.acme.client "1.0.0" { import ceylon.interop.browser "1.2.1-1"; }

  • com.acme.server is a native("jvm") module that imports ceylon.net:

    native("jvm") module com.acme.server "1.0.0" { import ceylon.net "1.2.1"; }

Serving Ceylon modules

In order to run com.acme.client in a browser, we have to import it from an HTML file. The recommended way is to use RequireJS:

<!doctype html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <title>Hello from Ceylon!</title>
</head>
<body>
    <div id="container">
    </div>
    <script src="//requirejs.org/docs/release/2.1.22/minified/require.js"></script>
    <script type="text/javascript">
    require.config({
        baseUrl : 'modules'
    });
    require(
            [ 'com/acme/client/1.0.0/com.acme.client-1.0.0' ],
            function(app) {
                app.run();
            }
    );
    </script>
</body>
</html>

Here, we tell RequireJS to use the prefix modules when downloading artifacts from the server, which means we need something on a server that will listen on /modules, parse module names and serve the correct artifact.

Option 1: using a Ceylon server

Ceylon SDK 1.2.1 introduced a new endpoint named RepositoryEndpoint, that uses a RepositoryManager to look up module artifacts in one or more Ceylon repositories, like the compiler or Ceylon IDE do:

import ceylon.net.http.server.endpoints {
    RepositoryEndpoint
}
import ceylon.net.http.server {
    newServer
}

"Run the module `com.acme.server`."
shared void run() {
    value modulesEp = RepositoryEndpoint("/modules");
    value server = newServer { modulesEp };

    server.start();
}

By default, this endpoint will look for artifacts in your local Ceylon repository, but also in the compiler's output directory. This greatly simplifies our development workflow, because each time we modify files in com.acme.client, Ceylon IDE will rebuild the JS artifacts, which can then be immediately refreshed in the browser.

Finally, to serve static files (HTML, CSS, images etc), we need a second endpoint that uses serveStaticFile to look up files in the www folder, and serve index.html by default:

function mapper(Request req) 
        => req.path == "/" then "/index.html" else req.path;

value staticEp = AsynchronousEndpoint(
    startsWith("/"), 
    serveStaticFile("www", mapper),
    {get}
);

value server = newServer { modulesEp, staticEp };

If we start the server and open http://localhost:8080/, we can see in the web inspector that the modules are correctly loaded:

Option 2: using static HTTP servers

Option 1 is interesting if you already have a backend written in Ceylon. Otherwise, it might be a little too heavy because you're basically starting a Ceylon server just to serve static files. Luckily, there's a way to create a standard Ceylon repository containing a module and all its dependencies: ceylon copy.

ceylon copy --with-dependencies com.acme.client

This command will copy the module com.acme.client and all its dependencies to a given folder (by default ./modules), preserving a repository layout like the one RequireJs expects. This means we can start httpd or nginx and bind them directly on the project folder. Modules will be loaded from ./modules, we just have to configure the server to look for other files in the www directory.

Attention though, each time we modify dependencies of com.acme.client, we will have to run ceylon copy again to update the local repository.

Option 2 is clearly the way to go for client apps that don't require a backend. Like option 1, it doesn't force you to publish artifacts in ~/.ceylon/repo.

Of course, if you are running a local Ceylon JS application, and your browser allows you to include files directly from the filesystem, you can also avoid the HTTP server and load everything for the filesystem.

Using browser APIs

Now that we have bootstrapped a Ceylon application running in a browser, it's time to do actual things that leverage browser APIs. To do this, we'll use the brand new ceylon.interop.browser which was introduced in the Ceylon SDK 1.2.1 a few days ago. Basically, it's a set of dynamic interfaces that allow wrapping native JS objects returned by the browser in nice typed Ceylon instances. For example, this interface represents the browser's Document:

shared dynamic Document satisfies Node & GlobalEventHandlers {
    shared formal String \iURL;
    shared formal String documentURI;
    ...
    shared formal HTMLCollection getElementsByTagName(String localName);
    shared formal HTMLCollection getElementsByClassName(String classNames);
    ...
}

An instance of Document can be retrieved via the toplevel object document, just like in JavaScript:

shared Document document => window.document;

Note that window is also a toplevel instance of the dynamic interface Window.

ceylon.interop.browser contains lots of interfaces related to:

Making an AJAX call, retrieving the result and adding it to a <div> is now super easy in Ceylon:

import ceylon.interop.browser.dom {
    document,
    Event
}
import ceylon.interop.browser {
    newXMLHttpRequest
}

shared void run() {
    value req = newXMLHttpRequest();

    req.onload = void (Event evt) {
        if (exists container = document.getElementById("container")) {
            value title = document.createElement("h1");
            title.textContent = "Hello from Ceylon";
            container.appendChild(title);

            value content = document.createElement("p");
            content.innerHTML = req.responseText;
            container.appendChild(content);
        }
    };

    req.open("GET", "/msg.txt");
    req.send();
}

Going further

Dynamic interfaces are really nice when it comes to using JavaScript objects in Ceylon. They are somewhat similar to TypeScript's type definitions, which means in theory, it is possible to use any JavaScript framework directly from Ceylon, provided that someone writes dynamic interfaces for its API.

The Ceylon team is currently looking for ways to load TypeScript definitions and make them available to Ceylon modules, which would greatly simplify the process of adding support for a new framework/API.

The complete source code for this article is available on GitHub.

A live example is available on the Web IDE.

Comments

Ceylon in the browser

Gavin King
26 February 2013
httpd ceylon-js

Last week, we met the Ceylon HTTP server, which we can use to serve dynamic (or even static) content over HTTP. But what about the client side? Well, today we're going to write a little HTTP client that runs in the browser, mainly as a way of showing off Ceylon's new dynamic blocks.

First, let's see the server, in a module named hello.server:

import ceylon.net.httpd { createServer, startsWith, 
                          AsynchronousEndpoint, 
                          Request, Response }
import ceylon.net.httpd.endpoints { serveStaticFile }

void run() {
    createServer{ 
        AsynchronousEndpoint { 
            path = startsWith("/sayHello"); 
            void service(Request request, Response response, 
                    void complete()) {
                response.writeString("Hello, Client!");
                complete(); //async response complete
            }
        },
        AsynchronousEndpoint {
            path = startsWith(""); 
            service = serveStaticFile("../client");
        } 
    }
    .start();
}

The server has two asynchronous endpoints, one of which simply serves up static content out of the directory ../client, and the other of which just responds to requests with the text Hello, Client!.

The entrypoint to our application is a HTML page.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
    <head>
        <meta http-equiv="Content-type" 
              content="text/html;charset=UTF-8"/>
        <title>Hello</title>
        <script type="text/javascript"
                src="script/require/1.0.8/require.js">
        </script>
        <script type="text/javascript">
            require.config({
                // location of our project's compiled modules
                baseUrl: 'modules',
                // locations of external dependencies here
                paths: {
                    // path to the Ceylon language module 
                    'ceylon/language': 
                        '/script/ceylon/language'
                }
            });
            //alias our hello module to 'h'
            require(['hello/client/1/hello.client-1'], 
                    function (hello) { h = hello; });
        </script>
    </head>
    <body>
        <div>
            <input type="submit" 
                   value="Say hello" 
                   onclick="h.hello()"/>
        </div>
        <div>
            <span id="greeting"/>
        </div>
    </body>
</html>

The page has a button which calls the hello() function of the Ceylon module hello.client.

We're using require.js to load our Ceylon modules. Unfortunately require.js doesn't really have a concept of module repositories like Ceylon does, which means configuring it to find all our compiled Ceylon is a bit fiddly. Nor does it feature very good error reporting which turns "fiddly" into "time-wasting". (The implementation of require() in node.js is a much better fit,
but unfortunately we don't have that in the browser.)

Finally, the module named hello.client has the following function:

shared void hello() {
    dynamic {
        value req = XMLHttpRequest();       
        void handleResponse() {
            if (req.readyState==4) {
                document.getElementById("greeting")
                        .innerHTML = req.status==200 
                                then req.responseText 
                                else "error";
            }
        }
        req.onreadystatechange = handleResponse;
        req.open("GET", "/sayHello", true);
        req.send();
    }
}

This function makes use of the native JavaScript API XMLHttpRequest to send an asynchronous request to our server, and then interacts with the browser's DOM. But how on earth can a statically typed language like Ceylon call a weakly typed language like JavaScript!?

Well, code that appears inside a dynamic block is optionally typed. That is, Ceylon will make use of typing information when it is available, but when it is not, it will just shut up and let you write almost whatever you like, as long as it's syntactically well-formed Ceylon code. Ceylon can't be certain that there's really a function called XMLHttpRequest, and it certainly can't be sure that it has a member named onreadystatechange, but it will just assume you know what you're doing.

Inside a dynamic block, you can even instantiate a "dynamic" object:

dynamic {
    void printGreeting(value obj) {
        console.log(obj.greeting + " " + obj.language);
    }
    value obj = value { greeting="Hello"; language="Ceylon"; };
    printGreeting(obj);
}

Of course, inside a dynamic block, all kinds of typing errors can occur at runtime, things that the Ceylon compiler would usually detect at compile time. But the compiler at least protects you from having dynamic typing errors "leak" out of the dynamic block and into your regular Ceylon code by performing runtime checks when you assign a dynamically typed expression (that is, an expression of unknown type) to something with a well-defined static type.

Well, that's essentially all there is to our application, except for a couple of boring module and package descriptors. Here we can see a server written in Ceylon running in the JVM being called by a client written in Ceylon running in a web browser and interacting with the browser's native JavaScript APIs. I think that's pretty cool.

A note of caution: we're still working through some wrinkles in the semantics of dynamic, and dynamic is still not supported on the JVM (it will be, eventually). Nevertheless, this will be available as an experimental feature in Ceylon M5.

Comments

Prototypes vs lexical scope in the Ceylon JavaScript compiler

Gavin King
02 January 2012
ceylon-js

In the previous post I introduced the Ceylon JavaScript compiler project. One of the things I mentioned was that there was an extremely natural mapping from Ceylon to JavaScript making use of JavaScript's lexical scope to create nicely encapsulated JavaScript objects. For example, given the following Ceylon code:

shared class Counter(Integer initialCount=0) {
    variable value currentCount:=initialCount;
    shared Integer count {
        return currentCount;
    }
    shared void inc() {
        currentCount:=currentCount+1; 
    }
}

We produce the following JavaScript:

var $$$cl15=require('ceylon/language/0.1/ceylon.language');

//class Counter at members.ceylon (1:0-9:0)
this.Counter=function Counter(initialCount){
    var $$counter=new CeylonObject;

    //value currentCount at members.ceylon (2:4-2:45)
    var $currentCount=initialCount;
    function getCurrentCount(){
        return $currentCount;
    }
    function setCurrentCount(currentCount){
        $currentCount=currentCount;
    }

    //value count at members.ceylon (3:4-5:4)
    function getCount(){
        return getCurrentCount();
    }
    $$counter.getCount=getCount;

    //function inc at members.ceylon (6:4-8:4)
    function inc(){
        setCurrentCount(getCurrentCount().plus($$$cl15.Integer(1)));
    }
    $$counter.inc=inc;

    return $$counter;
}

Notice that this code is really quite readable and really not very different to the original Ceylon.

Let's load this module up in the node REPL, and play with the Counter.

> Counter = require('./node_modules/default/members').Counter
[Function: Counter]
> Integer = require('./runtime/ceylon/language/0.1/ceylon.language').Integer
[Function: Integer]
> c = Counter(Integer(0))
{ getCount: [Function: getCount], inc: [Function: inc] }

The Counter instance presents a nice clean API with getCount() and inc() functions:

> c.getCount().value
0
> c.inc()
> c.inc()
> c.getCount().value
2

Notice that the actual value of $$counter is completely hidden from the client JavaScript code. Another nice thing about this mapping is that it is completely free of JavaScript's well-known broken this. I can freely use the methods of c by reference:

> inc = c.inc
[Function: inc]
> count = c.getCount
[Function: getCount]
> inc()
> count().value
3

Now, an issue that was bugging me about this mapping - and bugging Ivo even more - is the performance cost of this mapping compared to statically binding the methods of a class to its prototype. Ivo did some tests and found that it's up to like 100 times slower to instantiate an object that defines its methods in lexical scope instead of using its prototype on V8. Well, that's not really acceptable in production, so I've added a switch that generates code that makes use of prototypes. With this switch enabled, then for the same Ceylon code, the compiler generates the following:

var $$$cl15=require('ceylon/language/0.1/ceylon.language');

//ClassDefinition Counter at members.ceylon (1:0-12:0)
function $Counter(){}

//AttributeDeclaration currentCount at members.ceylon (2:4-2:45)
$Counter.prototype.getCurrentCount=function getCurrentCount(){
    return this.currentCount;
}
$Counter.prototype.setCurrentCount=function setCurrentCount(currentCount){
    this.currentCount=currentCount;
}

//AttributeGetterDefinition count at members.ceylon (3:4-5:4)
$Counter.prototype.getCount=function getCount(){
    return this.getCurrentCount();
}

//MethodDefinition inc at members.ceylon (6:4-8:4)
$Counter.prototype.inc=function inc(){
    this.setCurrentCount(this.getCurrentCount().plus($$$cl15.Integer(1)));
}

this.Counter=function Counter(initialCount){
    var $$counter=new $Counter;
    $$counter.initialCount=initialCount;        
    return $$counter;
}

Clearly this code is a bit harder to understand than what we started with. It's also a lot uglier in the REPL:

> c = Counter(Integer(0))
{ initialCount: { value: 0, ... } }

Notice that the internal state of the object is now exposed to clients. And all its operations - held on the prototype - are also accessible, even the non-shared operations. Finally, JavaScript's this bug is back:

> inc = c.inc
[Function: inc]
> inc()
TypeError: Object #<error> has no method 'getCurrentCount'
    at inc (/Users/gavin/ceylon-js/build/test/node_modules/default/members.js:21:31)
    ...

We have to use the following ugly workaround:

> inc = function(){c.inc.apply(c,arguments)}
> inc()
> c.getCount().value
1

(Of course, the compiler automatically inserts these wrapper functions when you write a function reference at the Ceylon level.)

Personally, I don't really see why the JavaScript interpreter in V8 could not in principle internally optimize our original code to something more like our "optimized" code. I think it would make JavaScript a much more pleasant language to deal with if there wasn't such a big difference in performance there.

Anyway, if you're producing your JavaScript by writing Ceylon, this is now just a simple compiler switch :-)

Comments

Compiling Ceylon to JavaScript

Gavin King
31 December 2011
ceylon-js

We've always talked about Ceylon as a language for the JVM, since that's the platform we use every day and the one we think has the most to offer for server-side computing. But it's not the only VM out there, and there is very little in the definition of Ceylon that is JVM-specific. A couple of days ago I finally found time to work on a little project that's been nagging at me for a while: compiling Ceylon to JavaScript.

Why would anyone want to do that? Well, here's a few ideas:

  • to do client side development using a modern statically typed language,
  • to reuse server-side code on the client,
  • to run Ceylon code on node, or
  • for easy experimentation in a REPL.

I had anticipated that the language translation part of this would be a pretty easy task, and it turns out to be even easier than I had imagined. JavaScript isn't a very big language, so it took me two or three hours to re-learn it (or, more accurately, learn it properly for the first time), and I was ready to start generating code!

Some things that made this job especially easy:

  • Ceylon has well-defined semantics defined in a written specification. This is absolutely key for any kind of multi-platform-oriented language.
  • The Ceylon compiler has a layered architecture with a well-defined API between the parser/typechecker and the back end. Indeed, the two projects are developed by completely independent teams. Therefore, adding a new backend is an easy task.
  • JavaScript lacks a native type system, and all objects are essentially just associative arrays. This makes it an especially easy target for language translation. I had not fully appreciated just how much difference this makes!
  • Ceylon and JavaScript both view a "class" as just a special kind of function. Un-shared Ceylon declarations map naturally to JavaScript's lexical scope, and shared declarations map naturally to object members. JavaScript and Ceylon have similar treatment of closure and of first-class function references.
  • Neither Ceylon nor JavaScript has overloading. Indeed, the way you do overloading in Ceylon, using union types, is a totally natural match for how the problem is solved in dynamic languages!

On the other hand, there is one area where JavaScript is extremely, embarrassingly, inadequate: modularity. After a bit of googling around, I decided we should map Ceylon packages to Common JS modules. CommonJS is more of a server-side oriented solution, but apparently there are tools to repackage CommonJS modules for the browser. (The structure I've gone for is one script per package, grouped together in directories by module and module version.) We'll see how we go with this solution. It's certainly convenient for running Ceylon modules on node.

From the JavaScript side, you can load a Ceylon package named foo.bar.baz from version 1.0 of a module named foo.bar like this:

var foobarbaz=require('foo/bar/1.0/foo.bar.baz');

Now it's easy to instantiate a class named Counter and call its methods and attributes:

var counter = foobarbaz.Counter();
counter.inc();
console.log(counter.getCount());

I've pushed what I have so far to the new ceylon-js repository in GitHub. I've been focusing on the "hard bits" like multiple inheritance, nested classes, modularity, named argument invocations, encapsulation, etc., so what's there today is missing some of the more basic stuff like operators and control structures. And we need to reimplement the language module in JavaScript. Still, it's likely that the JavaScript backend will soon overtake the JVM backend in terms of feature completeness. Of course, I'm looking for contributors!

Comments
1