View source | Edit

Module overrides

Resolving module conflicts

Sometimes the information in a module descriptor is not correct, or must be customized.

  • This is especially common when dealing with modules obtained from a Maven repository.
  • It's also sometimes necessary when assembling a Ceylon application from third-party modules with conflicting dependencies.

Issues affecting Maven module dependencies

Maven module descriptors are notoriously unchecked, and many Maven module descriptors are missing information about direct dependencies. These modules just happen to compile and run by accident when using Maven, with its flat classpath containing all transitive module dependencies. By contrast, Ceylon has ClassLoader isolation, and therefore requires correct and complete dependency information.

Furthermore, Maven does not support the notion of sharing module imports, so if a module A makes types from its imported module B visible to the users of A, the import of B must be made shared. Sadly, Maven modules frequently bundle things that should not be made visible, such as other modules or tests, that you surely want to exclude, so making transitive dependencies shared by default would be inappropriate.

Finally, Maven supports module version conflict resolution by design, luck, or overrides, while Ceylon uses strict module version imports.

The overrides file

For all these reasons, we created an experimental measure that lets you override the dependency information in a Maven or Ceylon module descriptor. The overrides.xml file allows us to:

  • define constants and use them in interpolated XML attributes
  • set a module version (for all modules)
  • replace a module by another module (for all modules)
  • remove a module (for all modules)
  • add/remove module dependencies (per module)
  • edit a module dependency, for example making it shared (per module)
  • include/exclude parts of the jar (for example, to exclude certain packages from a jar)

The format of the overrides.xml file is defined by this XML schema.

Alternatives to the overrides file

The --use-flat-classpath and --auto-export-maven-dependencies options to the ceylon command sometimes allow us to avoid the need to specify an overrides file, or allow us to significantly simplify it.

You can also find these options in Ceylon IDE:

  • on the Ceylon Build > Module Repositories page of the Project > Properties for your Ceylon project in Ceylon IDE for Eclipse, and
  • on the Repositores tab of the Ceylon settings in the Project Structure in Ceylon IDE for IntelliJ.

Gotcha!

Note that --auto-export-maven-dependencies does not automatically make all transitive dependencies of any Maven module visible to Ceylon modules that import the Maven module! It only makes all transitive dependencies visible to Maven modules themselves.

Overrides file syntax

The overrides file must be an XML file named overrides.xml or maven-overrides.xml (the name is not significant), valid according to the overrides schema.

For example:

<overrides xmlns="http://www.ceylon-lang.org/xsd/overrides">
    <!-- Define a constant to be used in expressions -->
    <define name="restletVersion" value="2.0.10"/>
    <!-- Replace all versions of weld with version 1.1.4.Final -->
    <set groupId="org.jboss.weld" artifactId="weld-osgi-bundle" version="1.1.4.Final"/>
    <!-- Remove all uses of a module -->
    <remove groupId="org.jboss.weld" artifactId="weld-osgi-bundle"/>
    <!-- Add a module as a module root, to force it being loaded despite removal overrides in Maven -->
    <add groupId="com.fasterxml.jackson.dataformat"  artifactId="jackson-dataformat-xml" version="2.6.5"/>
    <!-- Edit dependencies of org.restlet.jse:org.restlet/2.0.10 -->
    <artifact groupId="org.restlet.jse" artifactId="org.restlet" version="${restletVersion}">
        <!-- Add/replace a dependency -->
        <add groupId="org.slf4j" artifactId="slf4j-api" version="1.6.1" shared="true"/>
        <!-- Remove a dependency -->
        <remove groupId="org.osgi" artifactId="org.osgi.core" version="4.0.0"/>
        <!-- Share a dependency -->
        <share groupId="org.slf4j" artifactId="slf4j-impl"/>
        <!-- Override the default classifier, if required -->
        <classifier>jar</classifier>
    </artifact>
    <!-- Replace all uses of org.apache.camel:camel-core/2.9.2 with version 2.10 -->
    <replace groupId="org.apache.camel" artifactId="camel-core" version="2.9.2">
        <with groupId="org.apache.camel" artifactId="camel-core" version="2.10"/>
    </replace>
    <!-- Edit dependencies of org.osgi:org.osgi.core/4.0.0 -->
    <artifact groupId="org.osgi" artifactId="org.osgi.core" version="4.0.0">
        <!-- Only include org/osgi/** and META-INF/** -->
        <filter>
            <!-- You can include or exclude and the matching is in sequence and stops at the first match -->
            <include path="org/osgi/**"/>
            <include path="META-INF/**"/>
            <exclude path="**"/>
        </filter>
    </artifact>
</overrides>

Most ceylon commands accept the --overrides (or -O) argument to specify this file.

ceylon compile --overrides=overrides.xml

Artifact coordinates or module names

Every element that works on modules accepts the expects XML attributes that identify the module or Maven artifact.

For a Ceylon module override:

  • module specifies the Ceylon module name
  • version optionally specifies the module version

For a Maven module override:

  • groupId specifies the Maven group id
  • artifactId specifies the Maven artifact id
  • classifier optionally specifies a Maven classifier

If version is missing, the override will match all versions of the module or Maven artifact.

Defining constants

Constants may be defined using the define element:

<define name="version" value="2.0.10"/>

A constant may be used in any subsequent XML attribute with the ${constantName} syntax:

<remove module="com.foo.bar" version="${version}"/>

Removing a module entirely

You can remove a module entirely from every import:

<remove module="com.foo.bar"/>

This element accepts the common module coordinate attributes.

Adding a module root

You can force a module being loaded as a module root, and prevent it from being removed by Maven overrides. This is useful to add modules to the classpath at run-time.

 <add groupId="com.fasterxml.jackson.dataformat"  artifactId="jackson-dataformat-xml" version="2.6.5"/>

This element accepts the common module coordinate attributes.

Overriding a module version globally

You can replace every import of a given module to use a specific version:

<set module="com.foo.bar" version="2"/>

This element accepts the common module coordinate attributes.

Replacing a module globally

You can replace every import of a given module to use another module:

<replace module="com.foo.bar" version="2">
    <with module="com.foo.gee" version="3"/>
</replace>

These elements accept the common module coordinate attributes.

Overriding a single module's dependencies

You can add/replace/remove or share dependencies of a single module:

<module module="com.foo.bar" version="2">
    <!-- this will add or replace existing dependencies -->
    <add module="com.foo.gee" version="3"/>
    <remove module="com.foo.baz"/>
    <share module="com.foo.dep"/>
</module>

Or for Maven artifacts:

<artifact groupId="com.foo" artifactId="bar" version="2">
    <!-- this will add or replace existing dependencies -->
    <add groupId="com.foo" artifactId="gee" version="3"/>
    <remove groupId="com.foo" artifactId="baz"/>
    <share groupId="com.foo" artifactId="dep"/>
</artifact>

These elements accept the common module coordinate attributes.

Overriding a single module's classifier

Some Maven modules require a custom classifier to be resolved properly:

<artifact groupId="org.wildfly.swarm" artifactId="swarmtool">
    <!-- This will download the "-standalone" jar instead of the normal jar --> 
    <classifier>standalone</classifier>
</artifact>

Overriding a single module's version

As an alternative to using <set/> to override a module version globally, you can override the version of a specific version of a module:

<module module="com.foo.bar" version="2">
    <!-- Use version 3 instead of 2 --> 
    <version>3</version>
</module>

Or of a Maven artifact:

<artifact groupId="com.foo" artifactId="bar" version="2">
    <!-- Use version 3 instead of 2 --> 
    <version>3</version>
</artifact>

Filtering a single module's contents

Sometimes modules include classes you do not want to be seen at runtime, in the metamodel for example, as they are debug/test classes with missing dependencies. You can also exclude them with this:

<artifact groupId="org.osgi" artifactId="org.osgi.core" version="4.0.0">
    <!-- Only include org/osgi/** and META-INF/** -->
    <filter>
        <!-- You can include or exclude and the matching is in sequence and stops at the first match -->
        <include path="org/osgi/**"/>
        <include path="META-INF/**"/>
        <exclude path="**"/>
    </filter>
</artifact>

Example (Google Guice)

Here's an overrides.xml file that lets you import Guice from Maven:

<overrides xmlns="http://www.ceylon-lang.org/xsd/overrides">
    <module groupId="com.google.inject"
         artifactId="guice"
            version="4.0">
        <share groupId="javax.inject"
            artifactId="javax.inject"/>
    </module>
</overrides>

You can now import Guice like this:

native("jvm")
module com.my.app "1.0.0" {
    import maven:com.google.inject:"guice" "4.0";
}

Note that you don't need this overrides.xml file at all if you use the --auto-export-maven-dependencies flag which is supported by the command line tools.

Example (Hibernate JPA solution 1)

Here's an overrides.xml file that lets you import Hibernate's JPA-compliant API from Maven:

<overrides xmlns="http://www.ceylon-lang.org/xsd/overrides">
    <module groupId="org.hibernate" 
         artifactId="hibernate-entitymanager">
        <share groupId="org.hibernate" 
            artifactId="hibernate-core"/>
        <share groupId="org.javassist" 
            artifactId="javassist"/>
        <share groupId="org.hibernate.javax.persistence" 
            artifactId="hibernate-jpa-2.1-api"/>
    </module>
    <module groupId="org.hibernate" 
         artifactId="hibernate-core">
        <add groupId="javax.transaction" 
          artifactId="jta" 
             version="1.1"
              shared="true"/>
    </module>
</overrides>

Now you can import Hibernate JPA like this:

native("jvm")
module com.my.app "1.0.0" {
    import maven:org.hibernate:"hibernate-entitymanager" "5.0.4.Final";
    import maven:org.hsqldb:"hsqldb" "2.3.1";
}

And define a persistence unit by placing this XML configuration in resources/com/my/module/ROOT/META-INF/persistence.xml where resources is your Ceylon resources directory:

<persistence xmlns="http://java.sun.com/xml/ns/persistence"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd"
           version="2.0">

    <persistence-unit name="sample">
        <class>com.my.app.Person</class>
        <properties>
            <property name="javax.persistence.jdbc.driver" 
                     value="org.hsqldb.jdbcDriver"/>
            <property name="javax.persistence.jdbc.url" 
                     value="jdbc:hsqldb:mem:testdb"/>
            <property name="javax.persistence.jdbc.user" 
                     value="sa"/>
            <property name="javax.persistence.jdbc.password" 
                     value=""/>
            <property name="hibernate.dialect" 
                     value="org.hibernate.dialect.HSQLDialect"/>
            <property name="hibernate.hbm2ddl.auto" 
                     value="update"/>
        </properties>
    </persistence-unit>

</persistence>

Example (Hibernate JPA solution 2)

Alternatively, instead of using <share/> in overrides.xml, we can do some of the work with the --auto-export-maven-dependencies flag which is supported by the command line tools.

With this flag enabled, we can use the following simplified overrides.xml file:

<overrides xmlns="http://www.ceylon-lang.org/xsd/overrides">
    <module groupId="org.hibernate" 
         artifactId="hibernate-core">
        <add groupId="javax.transaction" 
          artifactId="jta" 
             version="1.1"
              shared="true"/>
    </module>
</overrides>

However, with this solution, we must explicitly import the JPA API module, since the --auto-export-maven-dependencies flag only affects transitive dependencies.

native("jvm")
module com.my.app "1.0.0" {
    import maven:org.hibernate:"hibernate-entitymanager" "5.0.4.Final";
    import maven:org.hibernate.javax.persistence:"hibernate-jpa-2.1-api" "1.0.0.Final";
    import maven:org.hsqldb:"hsqldb" "2.3.1";
}

Example (Spark Framework)

Spark depends on Jetty.

This overrides.xml file allows Spark to be used from Maven:

<overrides xmlns="http://www.ceylon-lang.org/xsd/overrides">

    <module groupId="org.eclipse.jetty" 
         artifactId="jetty-server"
            version="9.3.2.v20150730">
        <share groupId="org.eclipse.jetty" 
          artifactId="jetty-io" 
             version="9.3.2.v20150730"/>
    </module>

    <module groupId="org.eclipse.jetty" 
         artifactId="jetty-io"
            version="9.3.2.v20150730">
        <share groupId="org.eclipse.jetty" 
          artifactId="jetty-util" 
             version="9.3.2.v20150730"/>
    </module>

    <module groupId="org.eclipse.jetty" 
         artifactId="jetty-util"
            version="9.3.2.v20150730">
        <share groupId="javax.servlet" 
            artifactId="javax.servlet-api"
               version="3.1.0"/>
    </module>

</overrides>

Now we can import Spark like this:

native("jvm") 
module sparky "1.0.0" {
    import maven:com.sparkjava:"spark-core" "2.3";
}

Alternatively, if we use the --auto-export-maven-dependencies flag, we don't need an overrides.xml file at all. That's probably a more robust solution in this case, given that Jetty comprises a number of internal modules with inter-dependencies that are not all captured in the Maven metadata.