Commit 1465381a authored by Matija Obreza's avatar Matija Obreza

Initial documentation

parent a32e0be5
......@@ -16,7 +16,8 @@
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
>
<modelVersion>4.0.0</modelVersion>
<groupId>org.genesys-pgr</groupId>
......@@ -55,6 +56,7 @@
<jdk.target>1.8</jdk.target>
<jdk.source>1.8</jdk.source>
<show.deprecations>false</show.deprecations>
<snippetsDirectory>${project.build.directory}/generated-snippets</snippetsDirectory>
<junit.version>4.11</junit.version>
<jsonpath.version>0.8.1</jsonpath.version>
......@@ -469,23 +471,23 @@
<artifactId>pgr-ontology-model</artifactId>
<version>0.7.2-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>2.0.2-beta</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path-assert</artifactId>
<version>0.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.8.2</version>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>2.0.2-beta</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path-assert</artifactId>
<version>0.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.8.2</version>
</dependency>
</dependencies>
<build>
......@@ -626,8 +628,57 @@
<pushChanges>false</pushChanges>
</configuration>
</plugin>
<plugin>
<groupId>org.asciidoctor</groupId>
<artifactId>asciidoctor-maven-plugin</artifactId>
<version>1.5.2</version>
<executions>
<execution>
<id>output-html</id>
<phase>generate-resources</phase>
<goals>
<goal>process-asciidoc</goal>
</goals>
<configuration>
<backend>html5</backend>
<sourceHighlighter>coderay</sourceHighlighter>
<attributes>
<linkcss>true</linkcss>
<toc>left</toc>
<icons>font</icons>
<sectanchors>true</sectanchors>
<idprefix />
<idseparator>-</idseparator>
<docinfo1>true</docinfo1>
</attributes>
</configuration>
</execution>
<execution>
<id>output-docbook</id>
<phase>generate-resources</phase>
<goals>
<goal>process-asciidoc</goal>
</goals>
<configuration>
<backend>docbook</backend>
<doctype>book</doctype>
</configuration>
</execution>
</executions>
<configuration>
<sourceDirectory>src/main/asciidoc</sourceDirectory>
<preserveDirectories>true</preserveDirectories>
<headerFooter>true</headerFooter>
<numbered>true</numbered>
<!-- <imagesDir>images</imagesDir> -->
<attributes>
<buildNumber>${buildNumber}</buildNumber>
<projectArtifact>${project.artifactId}</projectArtifact>
<projectVersion>${project.version}</projectVersion>
</attributes>
</configuration>
</plugin>
</plugins>
<resources>
<resource>
<directory>src/main/resources</directory>
......
Genesys Backup and Disaster Recovery Manual
===========================================
December 2015: Documentation commit {buildNumber}
:revnumber: {projectVersion}
:doctype: book
:toc: left
:toclevels: 5
:icons: font
:numbered:
:source-highlighter: pygments
:pygments-css: class
:pygments-linenums-mode: table
[[intro]]
Introduction
------------
This manual provides all information required to properly backup Genesys application,
configuration and data for a safe and valid recovery.
[NOTE]
====
This document is written under the assumption you are familiar with basics of GNU/Linux
administration and shell scripting.
====
include::sections/backup.adoc[]
include::sections/recovery.adoc[]
[[chElasticsearch]]
== Elasticsearch
https://www.elastic.co/products/elasticsearch[Elasticsearch] is used
[[chProperties]]
== Genesys configuration
=== Configuration files
Genesys Server {projectVersion} loads the following configuration file in the order
specified:
. <<application-properties,application.properties>>
. <<spring-properties,spring/spring.properties>>
. <<genesys-properties,genesys.properties>>
The first two files are bundled with the project and provide sensible defaults to
ensure a functional configuration. They are located in the `WEB-INF/classes` folder.
Any modifications to the defaults settings must be added to the `genesys.properties` file.
==== application.properties
The `application.properties` file makes available the project version and build information.
Do not modify!
==== spring.properties
`spring.properties` contains the default configuration options for any Genesys Server instance.
You can access its contents directly at
https://bitbucket.org/genesys2/genesys2-server/src/{buildNumber}/src/main/resources/spring/spring.properties[spring.properties on Bitbucket.org]
==== genesys.properties
Configuration for your instance of the server belong to `genesys.properties` file. This file is not
required and does not come with any Genesys distribution (not even source code).
This file needs to be created in a location that is included on the webapplication's CLASSPATH.
[[genesys-configuration-options]]
=== Configuration options
==== HTTP, CDN and cookie configuration
[cols="1,1,3", options="header"]
.HTTP-related configuration
|===
|Name
|Default value
|Description
|base.host|localhost|The hostname part of the Genesys URL
|base.hostname|${base.host}:8080|maven and Jetty run on port 8080 by default
|base.url|http://${base.hostname}|Genesys happily operates with relative URLs. The `base.url` is used
in email message templates and any other case where the full URL is needed.
|base.cookie-domain||Set the domain of the HTTP cookies. This allows us to share
cookies between all .genesys-pgr.org domains.
|base.cookie-secure|false|maven and Jetty don't use HTTPS by default. Switch this to `true` if you
plan to run on HTTPS protocol.
|base.cookie-http-only|false|AJAX calls from the browser sometimes require session information for
user authentication. Keep this value `false`.
|cdn.server|${base.url}|Default configuration does not use a CDN and defaults to the `base.url`.
|cdn.base|${cdn.server}|
|cdn.flags.url|${cdn.base}/html/flags|
|robots.allow|false|Setting this to `true` will produce a `/robots.txt` response that allows robots to index
the site.
|===
==== Database configuration
Genesys requires a database to function. Feel free to choose from a number of databases supported by
https://docs.jboss.org/hibernate/orm/4.3/javadocs/org/hibernate/dialect/package-summary.html[Hibernate].
The production server https://www.genesys-pgr.org runs on mysql-server-5.6 with InnoDB. We have seen
the server run on PostgreSQL database, but that's about it. Contact helpdesk@genesys-pgr.org if you wish
to explore running the server on some other RDBMS.
[cols="1,1,3", options="header"]
.Database configuration options
|===
|Name
|Default value
|Description
|db.url|...|Default mysql connect string used by most developers
|db.driverClassName|...|Defaults to Mysql Connector/J JDBC driver class name `com.mysql.jdbc.Driver`
|db.username|root|Database username
|db.password||Password
|db.showSql|false|Do not log SQL statements
|db.hbm2ddl|true|Apply changes to RDMBS based on current the application data model
|hibernate.dialect|org.hibernate.dialect. MySQL5InnoDBDialect|The
https://docs.jboss.org/hibernate/orm/4.3/javadocs/org/hibernate/dialect/package-summary.html[SQL dialect] that Hibernate can use when talking to the database
|c3p0.acquireIncrement|1|Acquire increment for SQL connections. We use 2 in production.
|c3p0.minPoolSize|1|Minimum number of SQL connections in the c3p0 pool We use 5 in production.
|c3p0.maxPoolSize|5|Maximum number of SQL connections in the c3p0 pool. We use 20 in production.
|c3p0.maxIdleTime|10|We use 30 in production.
|===
===== Creating a mysql database
Connect to your mysql instance and create a new blank database *genesys2*:
[source,sql,linenums]
----
# Create database
CREATE DATABASE genesys2 DEFAULT CHARSET UTF8;
----
The `DEFAULT CHARSET UTF8` ensures that tables created in this database will use UTF8 character
encoding by default.
Create user *genesys* with password *pwd* that will access the database from *localhost*:
[source,sql,linenums]
----
GRANT ALL ON genesys2.* TO 'genesys'@'localhost' IDENTIFIED BY 'pwd';
----
===== Database connection settings
Apply the relevant database configuration settings in `genesys.properties` file:
[source,properties,linenums]
----
# genesys2 with mysql database
db.url=jdbc:mysql://localhost/genesys2?useUnicode=true&characterEncoding=UTF-8&useFastDateParsing=false
db.driverClassName=com.mysql.jdbc.Driver
db.username=genesys
db.password=pwd
hibernate.dialect=org.hibernate.dialect.MySQL5InnoDBDialect
----
==== External Services
Genesys uses a number of external service APIs to provide users with advanced functionality.
[cols="1,1,3", options="header"]
.External APIs
|===
|Name
|Default value
|Description
|google.consumerKey||Google+ sign in
|google.consumerSecret||Google+ sign in
|google.api.key||Google API key, register your application with the https://console.developers.google.com/home/dashboard[Developers Console]
|google.url.shortener|https://www.googleapis.com/urlshortener/v1/url|Google URL shortener API
|captcha.privateKey|...|reCAPTCHA API private key. https://www.google.com/recaptcha/admin[Google reCAPTCHA]
|captcha.siteKey|...|reCAPTCHA API public or site key. https://www.google.com/recaptcha/admin[Google reCAPTCHA]
|captcha.publicKey|...|Obsolete, use `captcha.siteKey` instead.
|itpgrfa.easysmta.url|https://mls.planttreaty.org/itt/index.php?r=extsys/userinfo|Easy-SMTA URL
|itpgrfa.easysmta.username|bar|Easy-SMTA account username
|itpgrfa.easysmta.password|foo|Easy-SMTA account password
|google.analytics.account||The Google Analytics account for all pages.
|transifex.project|genesys-dev|https://www.transifex.com[Transifex] project name
|transifex.username||Transifex username
|transifex.password||Transifex password
|transifex.base.api.url|https://www.transifex.com/api/2/|Transifex API URL
|transifex.hook.key|this-is-a-test-hook-key|If you want to receive automatic notifications from Transifex, this is the hook.
|transifex.min.translated|80|Threshold of acceptable percentage translated for automatic acceptance.
|===
==== Other stuff
[cols="1,1,3", options="header"]
.Other stuff
|===
|Name|Default value|Description
|download.files.dir|./data/|Directory where files get stored.
|auto.createContent|false|Should default content be created on application startup? Set this to true
|list.logger.pagination.size|50|??
|news.menu.abbreviate.length|22|The maximum length for abbreviated menu item titles.
|===
==== Email and SMTP configuration
[cols="1,1,3", options="header"]
.SMTP configuration
|===
|Name|Default value|Description
|mail.user.from|test@localhost|The email address used as the *From:* address
|mail.requests.to|test@localhost|Email address to which requests for material are always *CC'd*
|mail.host|localhost|SMTP hostname
|mail.port|25|SMTP port
|mail.smtp.auth|true|Use SMTP authentication
|mail.user.name||Username for SMTP authentication
|mail.user.password||Password for SMTP authentication
|mail.smtp.ssl.enable|true|Enable SSL
|mail.smtp.starttls.enable|true|Enable STARTTLS
|mail.transport.protocol|smtp|Or `smtps`
|mail.async|true|Emails should be sent in asynchronous mode.
|===
==== Clustering
[cols="1,1,3", options="header"]
.Clustering configuration
|===
|hazelcast.instanceName|hz-genesys|Hazelcast instance name
|hazelcast.name|genesys-hz-3.5|Hazelcast group name
|hazelcast.password|hazelcasts|Password to join the Hazelcast group
|hazelcast.port|5701|TCP Port
|hazelcast.aws.access-key||AWS Autodetection access key
|hazelcast.aws.secret-key||AWS Autodetection secret key
|hazelcast.aws.region|eu-west-1|Limit search for nodes to the specified AWS region
|hazelcast.aws.security-group|sg-hazelcast|Security group used to identify instances considered for Hazecast group membership
|elasticsearch.url|http://localhost:9200/|Connection to <<elasticsearch>>
|===
==== Scheduler and Executor pools
[cols="1,1,3", options="header"]
.Pools and Schedulers configuration
|===
|Name|Default value|Description
|executor.core.pool.size|4|Executor thread pool size
|executor.max.pool.size|16|Maximum thread pool size
|executor.queue.capacity|100|Capacity of the executor queue if all threads are busy
|scheduler.max.pool.size|16|Scheduler thread pool size
|scheduler.tokens.cleanup.hours|1|Frequency of tokens cleanup job execution
|===
=== Other stuff
# Theme
theme.defaultThemeName=dev
# TileServer CDN
#tileserver.cdn='https://s1'
#tileserver.cdn='https://s1.cdn.genesys-pgr.org','https://s2.cdn.genesys-pgr.org','https://s3.cdn.genesys-pgr.org','https://s4.cdn.genesys-pgr.org'
tileserver.cdn='${base.url}'
# TileServer Cache
cache.defaultCacheSize=5000
cache.tileserver.max-size=1000
cache.tileserver.time-to-live-seconds=300
cache.tileserver.max-idle-seconds=0
cache.tileserver.eviction-policy=LRU
Genesys Server Reference Manual
===============================
December 2015: Documentation commit {buildNumber}
:revnumber: {projectVersion}
:doctype: book
:toc: left
:toclevels: 5
:icons: font
:numbered:
:source-highlighter: pygments
:pygments-css: class
:pygments-linenums-mode: table
[[intro]]
Introduction
------------
Genesys PGR (Plant Genetic Resources) is a free online global portal accessible at
link:$$https://www.genesys-pgr.org$$[www.genesys-pgr.org]
that allows the exploration of the world’s crop diversity through a single website.
This manual contains information required to configure, upgrade, run and manage a Genesys Server.
////
NOTE: Note...
TIP: Pro tip...
IMPORTANT: Don't forget...
WARNING: Watch out for...
CAUTION: Ensure that...
////
[[download]]
=== Download
Binary *genesys2-server* distribution packages are available on request. Please contact helpdesk@genesys-pgr.org.
[[sources]]
=== Sources
*genesys2-server* is licensed under link:$$http://www.apache.org/licenses/LICENSE-2.0.html$$[the Apache V2 license].
The source code of *genesys2-server* can be found on
link:$$https://bitbucket.org/genesys2/genesys2-server$$[https://bitbucket.org/genesys2/genesys2-server]
include::genesys-properties.adoc[]
include::running-genesys.adoc[]
include::elasticsearch.adoc[]
Backup and Disaster Recovery
----------------------------
This section provides information required to properly backup Genesys application,
configuration and data for a safe and valid recovery.
:leveloffset: 1
include::sections/backup.adoc[]
include::sections/recovery.adoc[]
:leveloffset: 0
[[chRunning]]
== Running Genesys
Genesys requires a working connection to a RDMBS. See Genesys Server <<database-configuration>> for information how to select
and configure the most appropriate database.
Setting up and running a server involves the following steps:
. <<installJetty,Download and install Jetty>>
. Obtain or <<quickBuild,build>> genesys2-server distribution package
. <<configureJetty,Configure Jetty for genesys2-server>>
. <<daemontools,Run Jetty as a service>> with daemontools
[[installJetty]]
=== Download and install Jetty
Download a **stable-9** copy of http://download.eclipse.org/jetty/[jetty] and unpack it
to your favorite location.
In the next few lines, you will download, unpack and move a Jetty distribution to /opt folder.
File ownership and permissions need to be adjusted.
[source,bash,linenums]
----
$ cd /tmp
$ wget http://download.eclipse.org/jetty/stable-9/dist/jetty-distribution-9.3.6.v20151106.zip
# unzip the jetty distribution in the /tmp folder
$ unzip jetty-distribution-9.3.6.v20151106.zip
$ ls -la jetty-distribution-9.3.6.v20151106/
# as root, move the distribution to /opt
$ cd /opt
$ sudo mv /tmp/jetty-distribution-9.3.6.v20151106 /opt/jetty
# Adjust ownership
$ sudo chown -R root.nogroup /opt/jetty
# Adjust permissions
$ sudo chmod -R g-w /opt/jetty
$ sudo chmod -R o-rwx /opt/jetty
----
[[quickBuild]]
=== Building the project
At the moment of writing, the `{projectVersion}` is {projectVersion}.
Obtain a copy of `genesys2-server-{projectVersion}-jetty.zip` archive by
building the project. The zip can be found in target/ directory.
[[configureJetty]]
=== Install *genesys2-server* and configure Jetty
Unpack `genesys2-server-{projectVersion}-jetty.zip` archive and move its contents into the jetty directory,
next to existing `demo-base` and `webapps` directories.
[source,bash,linenums]
----
$ cd /opt/jetty
$ sudo unzip .../genesys2-server-{projectVersion}-jetty.zip
$ ls -la genesys2-server-{projectVersion}-jetty/
----
Your customized configuration options go to `genesys2-server-\\{projectVersion}-jetty/resources/genesys.properties`.
[source,bash,linenums]
----
$ sudo nano genesys2-server-{projectVersion}-jetty/resources/genesys.properties
----
Check <<genesys-configuration-options>> for the list of options.
Start jetty from the `genesys2-server-{projectVersion}-jetty` base
[source,bash,linenums]
----
$ cd genesys2-server-{projectVersion}-jetty/
$ java -jar ../start.jar
----
Standard output stream (stdout) is used for logging.
This configuration uses HSQL database and is intended for testing genesys2-server.
To change the database settings, edit `genesys2-server-{projectVersion}-jetty/resources/genesys.properties` file:
[source,properties,linenums]
----
# Genesys configuration with mysql database
db.url=jdbc:mysql://localhost/genesys?useUnicode=true&characterEncoding=UTF-8&useFastDateParsing=false
db.driverClassName=com.mysql.jdbc.Driver
db.username=root
db.password=mysql
hibernate.dialect=org.hibernate.dialect.MySQL5InnoDBDialect
----
All available configuration options and their default values are listed in <<genesys-configuration-options>>.
=== Building from source
Building and running the server requires https://docs.oracle.com/javase/8/docs/technotes/guides/install/install_overview.html[Java 8 SDK],
https://maven.apache.org/[maven] and http://compass-style.org/install/[compass].
. Clone https://bitbucket.org/genesys2/genesys2-server.git[genesys2-server] to your computer
. Create a blank mysql database <<creating-a-mysql-database>>
. Configure database connection settings <<database-configuration>> in `src/main/resources/genesys.properties`
. Start Jetty with `mvn -Dspring.profiles.active=dev jetty:run`
[[chBackup]]
== Application
Genesys is usually run in a http://download.eclipse.org/jetty/[jetty] instance as described
in <<installing-genesys.adoc>>. This results in a very simple structure on the filesystem
that is easy to backup and will include the <<configuration>>.
== Configuration
Best practice requires keeping all custom changes to the <<genesys-properties.adoc>> in the
separate `genesys.properties` file.
In addition to Genesys configuration, all Jetty settings are located in the same base dir and are
easy to backup.
== Data
=== Database backup
=== File backup
=== Elasticsearch backup
== Validating backups
[[chRecovery]]
== Restoring Genesys from backups
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment