Skip to content

Maven build system

Jump to bottom Edit New page
David Leoni edited this page Dec 18, 2015 · 40 revisions

Last changed: December 17th, 2015

Project build is mavenized and the output WAR file can be easily deployed to Tomcat.

[Build and run](Maven build system#build-and-run)

[Deploy](Maven build system#deploy)

[Project structure](Maven build system#project-structure)

[Dependencies](Maven build system#dependencies)

NOTE: Instructions here are for code staying in a private repo. When project is sufficiently stable, we'll publish it in this repository. Still, we keep public wiki updated, so you can get an idea of the product.

Introduction and prerequisites

Project is currently split into two development branches, the stable named branch-0.3 and the unstable in master, with series 0.4.x. They have the following prerequisites:

0.3.x branch

This branch holds versions of odr that somewhat work with a dozen of different types of CSV datasets.

JDK: >= 1.6 (either OpenJDK or Oracle JDK)

Maven: 3.3.9 (maybe versions >= 3.1.0 work as well)

IDE : ODR and dependencies are edited best with Netbeans 8.1, Eclipse might need some setup

master branch (0.4.x)

This branch holds a more modular version of ODR with several improvements addressing maintenability of the system. Currently may not even compile.

JDK: >= 1.7 (either OpenJDK or Oracle JDK)

Maven: 3.3.9 (maybe versions >= 3.1.0 work as well)

IDE: ODR and dependencies are edited best with Eclipse Mars, as with Netbeans we got problems editing dependencies with annotations, see tod-commons wiki for more info.

Build and run

To build/run the project:

  1. Clone and select desired branch:

1.1a) Clone from public repository (very old non-functional code):

git clone https://github.com/opendatatrentino/OpenDataRise.git

1.1b) Clone from private repository (latest code):

For the private one address, ask david.leoni at unitn.it

1.2) Checkout the branch you want to use, for es. branch-0.3:

git checkout branch-0.3
  1. Configure and build

Code is divided in 2 big modules: Refine main module in /main, and ODR web in /odrweb.

2.1) Set Maven settings.xml properties:

Your Maven USER-DIR/.m2/settings.xml folder must have some properties set in order for odr to compile and run properly. Here we put only a template to follow. For a list of testing services addresses to use you can check the private OpenDataRise documentation named disi-odr-docs.

<servers>
    <server>    
      <id>sweb-ssh</id>
      <username>USERNAME</username>
      <password>PASSWORD</password>
    </server>         
    <server>    
       <id>sweb-snapshots</id>
      <username>USERNAME</username>
      <password>PASSWORD</password>
    </server>   
    <server>    
      <id>sweb-internal</id>
      <username>USERNAME</username>
      <password>PASSWORD</password>
    </server>   
</servers>

2.2) Copy all files in /conf-template in /conf

2.3) build everything:

To build everything (note integration tests will be executed and they might fail, still the WAR is produced anyway in target/ folder):

cd OpenDataRise
mvn clean install

A WAR in /odrweb/target/ will be produced. Using right-click Build or Clean and build on odrweb module in Netbeans will only produce an exploded WAR under /odrweb/target/odrweb-VERSION.

  1. Run it to check everyhing works fine. There are five run modes: debug, debug-integration, desktop, server and demo-server. Currently, the only supported modes are debug, debug-integration, and server, which is the default mode. The mode in which ODR will run is indicated in file odrweb/src/main/webapp/WEB-INF/web.xml

To run ODR with an embedded tomcat server:

cd odrweb
mvn tomcat7:run-war

This will run an instance available at http://localhost:5555/odrweb. By default configuration for tomcat is taken from conf-template/deploy/tomcat/context.xml and uses demo services. See Configuration paragraph in Debugging and Testing on how to configure it.

The embedded tomcat server is intended to mimic a production setting, but building the WAR takes too long for iterative development. For this purpose, see next paragraph.

  1. Develop/Debug it

See [Debugging and Testing page](https://github.com/opendatatrentino/OpenDataRise/wiki/Debugging and Testing#debugging)

Deploy

For now deploying is manual, go to tomcat admin interface and upload the war. See Configuration paragraph in Debugging and Testing on how to configure it.

Before deploying, you might run

mvn animal-sniffer:check

to ensure code is using:

  • branch-0.3.x : only Java APIs <= 1.6
  • master branch (0.4.x) : only Java APIs <= 1.7

Project structure

Root pom named odr is of packaging 'pom' and it is split into main, extensions and odrweb modules.

Original Refine parts

  • Refine core lives in /main folder and is customized with OpenDataRise changes.
  • folder structure of /main is kept as for the original ant build. So directory 'build' is used as target. Folder /main/tests/server/ contains original Refine tests.
  • /main and refine /extensions/* folders have local repo folders for jars not found in maven central. Their groupid is org.openrefine.opendata.deps
  • lib folders must not contain jars for now, and anyway their content is gitignored.
  • java part of rdf extension is integrated into core, but javascript part remains in extensions/rdf-extension folder
  • /extensions folder groups original Refine extensions, and it has its own pom.xml.
  • Each extension also has its own pom.xml
  • Refine custom Jetty server once in folder `/server was deleted, we now use Tomcat 7 as our reference server
  • Refine useless Broker in /broker folder was deleted

New ODR code

New odr code lives in /odrweb

  • folder structure of /odrweb follows Maven standard. Odrweb contains odr additions to Refine and new tests for Odr specific parts.
  • code has source compatible with Java 1.6 and targets Java 1.6 JRE
  • the core (which resides in /main folder) is customized with OpenDataRise changes. Nonetheless, OpenDataRise java code and tests live in /odrweb/. Refine web framework Butterfly doesn't load any Java class anymore at runtime, all jars are loaded by web server. Butterfly is only used to manage OpenDataRise HTML assets such as html pages and javascript code, which are held in /odrweb/src/main/webapp/extensions/odrext/module/

Dependencies

Logging

Refine used old log4j 1.2 and slf4j. We now use Logback and route calls from log4j to it.

Deps not found in Maven Central:

  • ant-tools-1.8.0
  • arithcode-1.1
  • icu4j-4.2.1
  • butterfly-trunk.jar
  • jrdf-0.5.6
  • json-20100208
  • lessen-trunk-r8 is slightly different in size than http://repo.typesafe.com/typesafe/repo/com/metaweb/lessen/1.0/ Source code is here
  • marc4j-2.4
  • opencsv-2.4-SNAPSHOT there is only 2.3 in central
  • poi-3.8-20120326 I took org.apache.poi 3.8
  • poi-ooxml-20120326 I took org.apache.poi 3.8
  • poi-ooxml-schemas-20120326 I took org.apache.poi 3.8
  • secondstring-20100303 not present in maven, corresponds to this: http://secondstring.sourceforge.net/
  • vicino-1.1 mmm this should be metaweb stuff
  • serializer.jar, xml-apis.jar and resolver.jar should be dependencies of xercesImpl (which I found)
Add a custom footer

OpenDataRise - Users

OpenDataRise - Developers

OpenRefine

Clone this wiki locally