Maven build system

Project build is mavenized and can be easily run and debugged with maven. Netbeans editor is fully supported. Apparently Eclipse has some issue recognizing paths, because of Refine custom folder structure.

[Run/debug](Maven build system#rundebug)

[Deploy](Maven build system#deploy)

[Project structure](Maven build system#project-structure)

[Dependencies](Maven build system#dependencies)

[Todo](Maven build system#todo)

---

Run/debug

Project can only be run with maven >= 3 for now, as it used to find the classpath given the dependencies. You also need a JDK >= 1.6 (either OpenJDK or Oracle JDK).

To run the project, a runDesktop, a debug and a runServer maven profiles were created in the server project, plus a Netbeans nbactions.xml to bind netbeans run and debug to activate the profiles. To run the project:

1. Clone it in your computer:

git clone https://github.com/opendatatrentino/OpenDataRise.git

2. Configure and build

Code is divided in 3 big modules: Refine main module in /main, Refine server in /server, and Open Data Rise extension in /extensions/odrext. For now code that should stay in odrext is hold in main module (/main) due to this bug.

2.1) Set Maven settings.xml properties:

Your Maven USER-DIR/.m2/settings.xml folder must have some properties set in order for odr to compile and run properly. Here we put only a template to follow. For a list of testing services addresses to use (so to know what to put instead of CKANALYZE-WEB-URL) you can check the private OpenDataRise wiki

<servers>
    <server>    
      <id>sweb-ssh</id>
      <username>USERNAME</username>
      <password>PASSWORD</password>
    </server>         
    <server>    
       <id>sweb-snapshots</id>
      <username>USERNAME</username>
      <password>PASSWORD</password>
    </server>   
    <server>    
      <id>sweb-internal</id>
      <username>USERNAME</username>
      <password>PASSWORD</password>
    </server>   
</servers>
<profiles>
 	<profile>
 	 	<activation>
	 		<activeByDefault>true</activeByDefault>
	 	</activation>	 
        	 <properties>
	 		<odr.testCatalog> http://dati.trentino.it</odr.testCatalog>
	 		<odr.ckanalyze> CKANALYZE-WEB-URL </odr.ckanalyze>
	 	</properties>	
 	</profile>
<profiles>

2.2) add conf folder: Ask David or Ivan to give you the 'conf' folder and add it both to /extensions/odrext/ and to /server folder.

2.3) build everything:

To build everything (tests are skipped during build in Netbeans):

cd OpenDataRise
mvn clean install 

3. Test it:

Tests are in extensions/odrext/tests/server. They are divided in regular tests and integration tests which end with "ITCase.java".

Tests are written with the TestNG testing framework, which is fully compatible with Maven. To run regular tests:

mvn test

Integration tests are run with

mvn integration-test

or in Netbeans by right clicking on Odr extension module-> Custom-> Odr integration test

4. Run it. There are five run modes: debug, debug integration, desktop, server and demo server.

To run it in debug mode (it will use mock services serverside thought for impianti di risalita csv)

cd server
mvn -Pdebug validate

To run it in debug integration mode (with real disi services):

cd server
mvn -Pdebug -PdebugIntegration validate

for other modes use respectively -PrunDesktop, -PrunServer, -PrunDemoServer

Alternatively, in Netbeans you can open the project in the /server folder and either run it for desktop mode (hit F6) or debug it (hit Ctrl F5). For other running modalities, right click on Server module, select Custom and then choose one of the following: Run in server mode, Run in demo server mode or Debug integration

Deploy

For now deploying is manual. Before deploying, you might run

mvn animal-sniffer:check 

to ensure code is using only Java APIs <= 1.6

Project structure

• folder structure is kept as for the original ant build. So directory 'build' is used as target
• root pom is of packaging 'pom' and it is split into main, server and extensions modules
• Each extension has its own pom
• main and server folders have local repo folders for jars not found in maven central. Their groupid is org.openrefine.opendata.deps
• lib folders must not contain jars for now, and anyway their content is gitignored. In the future we will build dependencies inside them
• Couldn't find a way to run the project from root pom, to this end the project in server folder must be used.
• java part of rdf extension is integrated into core, but javascript part remains in extensions/rdf-extension folder
• code has source compatible with Java 1.6 and targets Java 1.6 JRE
• the core (which resides in main folder) is customized with OpenDataRise changes. Nonetheless, OpenDataRise specific javascript and java tests are held in extensions/odrext, in the hope OpenDataRise will become just an extension of Refine.

Dependencies

Main

couldn't find:

• ant-tools-1.8.0
• arithcode-1.1
• icu4j-4.2.1
• butterfly-trunk.jar
• jrdf-0.5.6
• json-20100208
• lessen-trunk-r8 is slightly different in size than http://repo.typesafe.com/typesafe/repo/com/metaweb/lessen/1.0/ Source code is here
• marc4j-2.4
• opencsv-2.4-SNAPSHOT there is only 2.3 in central
• poi-3.8-20120326 I took org.apache.poi 3.8
• poi-ooxml-20120326 I took org.apache.poi 3.8
• poi-ooxml-schemas-20120326 I took org.apache.poi 3.8
• secondstring-20100303 not present in maven, corresponds to this: http://secondstring.sourceforge.net/
• vicino-1.1 mmm this should be metaweb stuff
• serializer.jar, xml-apis.jar and resolver.jar should be dependencies of xercesImpl (which I found)

Server

the servlet-api-2.5 is for Jetty 6.1.11, I found it in Central:

Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Created-By: 1.5.0_13 (Apple Inc.)
Built-By: jesse
Build-Jdk: 1.5.0_13
implementation-version: 6.1.11
mode: development
package: org.mortbay

• jdatapath-alpha2.jar
  • this one has even a jdatapath.dll !!

Todo

Assemble dependencies in lib folders and make a deployable WAR