Skip to content

Commit

Permalink
first commit
Browse files Browse the repository at this point in the history
  • Loading branch information
@lmatteis
lmatteis committed Mar 14, 2012
0 parents commit e4fbf3f
Show file tree
Hide file tree
Showing 1,217 changed files with 49,471 additions and 0 deletions.
106 changes: 106 additions & 0 deletions README.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
Creating a WAR package
======================

To build a WAR package out of the entire SINGER site you need to
install Maven (http://maven.apache.org) and of course Java (JDK).

The root of the project is where the 'pom.xml' file is located. This
file defines all of the project settings and dependencies. From this
folder you need to type this in the command-line:

mvn clean package

It will create a 'target' directory with a WAR file in it. You can
now use this WAR file to deploy SINGER.



Using the WAR package
====================

Now that you have the entire SINGER site in a single WAR package you
must deploy it to your web server. With Tomcat you can simply copy the
WAR package to Tomcats "webapps" directory, and it will automatically
deploy.

Be sure to rename the directory it creates to "ROOT" otherwise the
site will only be accessible through 'http://site.com/singer-webapp'.
Renaming the folder to ROOT will make it so that SINGER can be
accessed through http://site.com/, and not by the sub-folder.



Configuring SINGER before starting the web server
================================================

SINGER's configuration files are located under
'src/main/webapp/WEB-INF/'

* singer.properties
* statements.properties

You must ONLY edit singer.properties with your local settings.
The important fields to edit are:

* FT_INDEX_ROOT - The full path of the Lucene indexes root folder
(we will discuss this in the next paragraph)

* SQL_STATEMENTS - The full path of the statements.properties file

* WEB_ROOT - The full path to where the ROOT directory is
located under your Web Server

* CACHE_ROOT - This is a folder where cached Lucene indexes
are stored

* JDBC_CONNECT_STRING - This hold information for accessing the
MySQL database



Database setup
==============

Setting up the database is very easy. Simply import the data from the
live servers and edit the JDBC_CONNECT_STRING attribute in
singer.properties.



Configuring the Web Server for SINGER
=====================================

SINGER needs lots of Java memory to run. This is because of Lucene.
The JVM needs to be configured to run with at least 1gb of memory. To
do this you must set the CATALINA_OPTS environment variable before
starting Tomcats web server (other webservers like JBOSS have other
methods of increasing the JVM memory).

Type this in a command line to set the environment variable:

export CATALINA_OPTS="-Xms1g -Xmx1g"

Figure out your Web Servers setting and how to increase Java's memory
with that particular server.



Indexing data with Lucene
=========================

The site gets most of the its data from Lucene indexes which are
located in the FT_INDEX_ROOT path.

To create these indexes you need to run a .sh script I've created.
This script is called 'index.sh' and it's located at the root of the
project.

You need to edit this file with the appropriate paths and then you can
run it doing:

sh index.sh

This usually takes a couple of hours to run.

Be sure to delete the CACHE_ROOT directory when indexing new data,
it might show old data which is cached.
7 changes: 7 additions & 0 deletions TODO.mkd
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
* Create interface for Elizabeth to edit data-dictionary
* Distribution data on SINGER w/ Milko
* Reference Sets with Federico
* Collecting missions accessions linkage with Hannes
* Video tutorial for accessing Coll Missions
* Give metadata of coll-missions to Massimo

13 changes: 13 additions & 0 deletions deploy.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# This deployment script generates the appropriate
# war container, that contains the entire SINGER site.
# It uses Maven to create such package.
export CATALINA_OPTS="-Xms1g -Xmx1g"
sh ~/tomcat/bin/shutdown.sh
#mvn -Dmaven.test.skip=true package
mvn package
rm -r ~/tomcat/webapps/ROOT
cp target/bioversity-singer.war ~/tomcat/webapps/ROOT.war
sh ~/tomcat/bin/startup.sh

# copy a .properties files, FIXME
#cp ~/singer-webapp/src/main/java/org/sgrp/singer/ApplicationResources.properties ~/tomcat/webapps/ROOT/WEB-INF/classes/org/sgrp/singer/
10 changes: 10 additions & 0 deletions index.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#-i "acc|coop|dist|donor|inst|misscoll|misscoop|missdist|olinks|rec|miss|keywords" \
java \
-Xms500M -Xmx500M \
-classpath "/Users/lucamatteis/tomcat/lib/*:\
/Users/lucamatteis/tomcat/webapps/ROOT/WEB-INF/lib/*:\
/Users/lucamatteis/tomcat/webapps/ROOT/WEB-INF/classes/" \
org.sgrp.singer.Main \
-u \
-i "acc|coop|dist|donor|inst|misscoll|misscoop|missdist|olinks|rec|miss|keywords" \
-properties "/Users/lucamatteis/bioversity-singer/trunk/src/main/webapp/WEB-INF/singer.properties"
33 changes: 33 additions & 0 deletions livedeploy.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/bin/sh

# This is just a script that will deploy my local war to the singer.cgiar.org live server.
# It can and will only run from Bioversity's network.

# Unpack my local war
mkdir livedeploy/singer
cd livedeploy/singer
jar -xvf ../../target/bioversity-singer.war

# let's go back to our main folder where this sh script is
cd ../../

# copy the singer.properties.LIVE file to my local version
cp livedeploy/singer.properties.LIVE livedeploy/singer/WEB-INF/singer.properties

# pack the singer folder
cd livedeploy/singer
jar -cvf ../singer.war *

# let's go back to our main folder where this sh script is
cd ../../

# now we have a deploy/singer.war that we can send to the live site
# send it to the home directory on the live server
scp -r livedeploy/singer.war [email protected]:/home/singer

# then move the file using ssh
#ssh [email protected] sudo sh /opt/tomcat/bin/shutdown.sh ; sudo rm -rf /opt/tomcat/webapps/singer ; sudo cp /home/singer/singer.war /opt/tomcat/webapps/ ; sudo sh /opt/tomcat/bin/startup.sh

# cleanup
rm -rf livedeploy/singer
rm -rf livedeploy/singer.war
22 changes: 22 additions & 0 deletions livedeploy/singer.properties.LIVE
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
LANGUAGES=en
FT_INDEX_ROOT=/var/lib/lucene/october_27
SQL_STATEMENTS=/opt/tomcat/webapps/singer/WEB-INF/statements.properties
WEB_ROOT=/opt/tomcat/webapps/singer
STORE_RESULTS_ABOVE=3000
ITEMS_PER_ORDER=250
CACHE_ROOT=/var/lib/lucene/october_27/cache
ENVDATA_ROOT=/var/lib/lucene/october_27/envdata
TEMPLATES=/opt/tomcat/webapps/singer/WEB-INF/templates
USER_DATA_ROOT=/work/singerUsersData
DESCRIPTORS_LINK=http://hq8.grinfo.net:8083
JDBC_DRIVER=com.mysql.jdbc.Driver
JDBC_CONNECT_STRING=jdbc:mysql://localhost:3306/singer_latest?user=f07dbmanager&password=2hsnKqCKAf7vTP&autocommit=false&zeroDateTimeBehavior=convertToNull
GOOGLE_MAP_KEY_SINGERDEMO=ABQIAAAA166IDnF-TusBuyEMxy4V8RTvQohs8ZEqlh6rfuoELAEEDMbcqhR1fC1YzhOsox9ctL4uXSL5Yq_o1g
GOOGLE_MAP_KEY=ABQIAAAA166IDnF-TusBuyEMxy4V8RR55X9vBA0rnpwLnM8SvEjW6y2QiRQx-b1qqQbdnBRZAsi2-bAP0kjcKA
GOOGLE_MAP_KEYsinger.grinfo.net=ABQIAAAA166IDnF-TusBuyEMxy4V8RR55X9vBA0rnpwLnM8SvEjW6y2QiRQx-b1qqQbdnBRZAsi2-bAP0kjcKA
GOOGLE_MAP_KEYsinger.cgiar.org=ABQIAAAA166IDnF-TusBuyEMxy4V8RRN5CxjYmRadsXrTOkiHN5lJm4TfxTk6WXqXb9anJ9nttgp0SB76HuWMA
GOOGLE_MAP_KEYwww.singer.cgiar.org=ABQIAAAA166IDnF-TusBuyEMxy4V8RQE4snXQsYtRCGAS8JSd6tEICkVPRRA1YhLmWKjnEpRFpR8BsqA3J823w
ORDER_MAIL_HOST=smtp.serv.cgnet.com
#[email protected]
[email protected]
[email protected]
6 changes: 6 additions & 0 deletions lucenetest.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
java \
-cp \
"/Users/lucamatteis/tomcat/webapps/ROOT/WEB-INF/classes/:\
/Users/lucamatteis/tomcat/webapps/ROOT/WEB-INF/lib/*" \
org.sgrp.singer.lucenetest.LuceneTest \
$1
192 changes: 192 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,192 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.bioversityinternational</groupId>
<artifactId>bioversity-singer</artifactId>
<packaging>war</packaging>
<version>1.0-SNAPSHOT</version>
<name>bioversity-singer Maven Webapp</name>
<url>http://maven.apache.org</url>
<repositories>
<repository>
<id>java.net</id>
<url>http://download.java.net/maven/1/</url>
<layout>legacy</layout>
</repository>
<repository>
<id>javacarver</id>
<url>http://repo.javacarver.com/</url>
</repository>
<repository>
<id>GCP</id>
<url>http://maven.generationcp.org/m2repo/</url>
</repository>
<repository>
<id>jboss</id>
<url>http://repository.jboss.org/maven2/</url>
</repository>
</repositories>
<properties>
<cxf.version>2.2.2</cxf.version>
<spring.version>2.5.5</spring.version>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>xerces</groupId>
<artifactId>xercesImpl</artifactId>
<version>2.8.1</version>
</dependency>

<dependency>
<artifactId>mysql-connector-java</artifactId>
<groupId>mysql</groupId>
<version>3.1.12</version>
</dependency>

<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-core</artifactId>
<version>2.4.0</version>
</dependency>

<dependency>
<groupId>struts</groupId>
<artifactId>struts</artifactId>
<version>1.2.4</version>
</dependency>

<dependency>
<groupId>jfree</groupId>
<artifactId>jfreechart</artifactId>
<version>1.0.12</version>
</dependency>

<dependency>
<groupId>com.thoughtworks.xstream</groupId>
<artifactId>xstream</artifactId>
<version>1.3.1</version>
</dependency>

<dependency>
<groupId>org.apache.xmlrpc</groupId>
<artifactId>xmlrpc-client</artifactId>
<version>3.1.3</version>
</dependency>

<dependency>
<groupId>ws-commons-util</groupId>
<artifactId>ws-commons-util</artifactId>
<version>1.0.1</version>
</dependency>

<dependency>
<groupId>javax.activation</groupId>
<artifactId>activation</artifactId>
<version>1.1</version>
</dependency>

<dependency>
<groupId>javax.mail</groupId>
<artifactId>mail</artifactId>
<version>1.4.1</version>
</dependency>

<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.15</version>
<type>jar</type>
<scope>runtime</scope>
<exclusions>
<exclusion>
<groupId>com.sun.jmx</groupId>
<artifactId>jmxri</artifactId>
</exclusion>
<exclusion>
<groupId>javax.jms</groupId>
<artifactId>jms</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jdmk</groupId>
<artifactId>jmxtools</artifactId>
</exclusion>
<exclusion>
<groupId>javax.mail</groupId>
<artifactId>mail</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.5</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax.servlet.jsp</groupId>
<artifactId>jsp-api</artifactId>
<version>2.1</version>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.8</version>
</dependency>

<dependency>
<groupId>com.googlecode.json-simple</groupId>
<artifactId>json-simple</artifactId>
<version>1.1</version>
</dependency>



</dependencies>
<build>
<finalName>bioversity-singer</finalName>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>maven-jetty-plugin</artifactId>
<version>6.1.8</version>
<configuration>
<contextPath>cxf-webservices</contextPath>
</configuration>
</plugin>
<!-- this if for the ApplicationResources.properties file that needs to be copied to the target war -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>2.3</version>
</plugin>

</plugins>
<!-- still for the ApplicationResources.properties file -->
<resources>
<resource>
<directory>src/main/java/org/sgrp/singer</directory>
<includes>
<!-- search for all .properties files (we only have ApplicationResources.properties for now -->
<include> **/*.properties</include>
</includes>
</resource>
</resources>
</build>
</project>
1 change: 1 addition & 0 deletions src/main/java/org/apache/xmlrpc/client/XmlRpcClient.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
user.agent=Apache XML RPC ${project.version}
Loading

0 comments on commit e4fbf3f

Please sign in to comment.