... | ... | @@ -5,429 +5,5 @@ |
|
|
2. Designate a volume for GGCE file repository.
|
|
|
3. Update the Docker compose file and start up your instance of GGCE.
|
|
|
|
|
|
**NOTE: This guide is work-in-progress!** Contact [GGCE Support](mailto:gitlab+grin-global-support-178-issue-@croptrust.org) for assistance!
|
|
|
|
|
|
# GGCE Installation Guide
|
|
|
|
|
|
**GGCE** is implemented through two main components: the GGCE-Server and the grin-global-web client that provides a **web user interface** through which genebank staff interact with GGCE.
|
|
|
|
|
|
The **GGCE-Server** manages the application business logic, interacts with the **database** which stores the application data, and with a **file volume** to store user-uploaded files and documents.
|
|
|
|
|
|
The GGCE web user interface **grin-global-web** represents the most complete GGCE client application and covers most genebank operations. Other client applications such as mobile apps to support specific processes in the genebank (e.g. recording trait observations), the Curator Tool, and other applications may also interact with the GGCE JSON or SOAP API provided by the **server**.
|
|
|
|
|
|
GGCE components are packaged as **Docker images** and designed for container deployment on Windows, Linux and macOS hosts.
|
|
|
|
|
|
![production-environment](images/production-environment.png)
|
|
|
|
|
|
The GGCE Server and the web application services run as different containers and they must be able to communicate on the designated ports. The server must be able to access the RDBMS.
|
|
|
|
|
|
Both **GGCE Server** and **GGCE Web** containers expose only HTTP endpoints:
|
|
|
|
|
|
- Server: HTTP on TCP port `8080`
|
|
|
- Web: HTTP on TCP port `3000`
|
|
|
|
|
|
In a Production environment one would commonly use *[Traefik](https://traefik.io)* or an alternative proxy solution to handle incoming connections. In that case the TCP ports do not need to be exposed outside the container environment; *Traefik* is the only service exposed to the Internet. Traefik can automatically manage SSL certificates (e.g. utilizing [LetsEncrypt.org](https://letsencrypt.org)).
|
|
|
|
|
|
## Docker compose file
|
|
|
|
|
|
Minimal docker-compose file to deploy GGCE server and web application with labels for Traefik:
|
|
|
|
|
|
```yaml
|
|
|
version: '3'
|
|
|
|
|
|
services:
|
|
|
ggce-server:
|
|
|
image: dockerhub.croptrust.org/grin-global/grin-global-server:latest
|
|
|
environment:
|
|
|
# Database connection
|
|
|
- DB_URL=jdbc:sqlserver://35.221.232.153:1433;DatabaseName=gringlobal
|
|
|
- DB_USERNAME=SA
|
|
|
- DB_PASSWORD=*****************
|
|
|
# Host names and URLs
|
|
|
- BASE_URL=http://gserver.genebank.org
|
|
|
- FRONTEND_URL=http://gclient.genebank.org
|
|
|
restart: always
|
|
|
networks:
|
|
|
- web
|
|
|
logging:
|
|
|
options:
|
|
|
max-size: "100k"
|
|
|
max-file: "3"
|
|
|
labels:
|
|
|
- "traefik.docker.network=web"
|
|
|
- "traefik.enable=true"
|
|
|
- "traefik.basic.frontend.rule=Host:gserver.genebank.org"
|
|
|
- "traefik.basic.port=8080"
|
|
|
- "traefik.basic.protocol=http"
|
|
|
|
|
|
ggce-ui:
|
|
|
image: dockerhub.croptrust.org/grin-global/grin-global-ui/gg-ce-web:latest
|
|
|
environment:
|
|
|
- API_URL=http://gserver.genebank.org
|
|
|
- ORIGIN=http://gclient.genebank.org
|
|
|
restart: always
|
|
|
networks:
|
|
|
- web
|
|
|
logging:
|
|
|
options:
|
|
|
max-size: "100k"
|
|
|
max-file: "3"
|
|
|
labels:
|
|
|
- "traefik.docker.network=web"
|
|
|
- "traefik.enable=true"
|
|
|
- "traefik.basic.frontend.rule=Host:gclient.genebank.org"
|
|
|
- "traefik.basic.port=3000"
|
|
|
- "traefik.basic.protocol=http"
|
|
|
networks:
|
|
|
web:
|
|
|
external: true
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
# The Database
|
|
|
|
|
|
An RDBMS needs to be available to GGCE. You should use an existing database engine provided by your IT department, this ensures that your data is properly managed, secured, and backed up.
|
|
|
|
|
|
MSSQL Server is the prefered database engine for both GRIN-Global and GGCE. While GGCE allows you to use almost any RDBMS, if you want to use the **[Curator Tool](https://www.grin-global.org/download_ct.html)** with GGCE, you must use the MSSQL database engine for which up-to-date queries are available. SQL queries for other engines are not actively maintained.
|
|
|
|
|
|
The GGCE server will detect a blank database and will automatically initialize the database schema and populate it with initial data on first startup.
|
|
|
|
|
|
## Configuration notes
|
|
|
|
|
|
The important database options to note are:
|
|
|
|
|
|
1. The hostname (or IP address) and port of the RDBMS engine (e.g. `mssqlserver1:1433`)
|
|
|
2. The name of the GGCE database (e.g. `ggce-db`)
|
|
|
3. Credentials of a user with owner permissions to the GGCE database.
|
|
|
|
|
|
## **Preparing a new MSSQL database for GGCE**
|
|
|
|
|
|
Connect to your MSSQL database instance and create a new database `ggce-db`. Please make sure that you use `SQL_Latin1_General_CP1_CI_AS` collation.
|
|
|
|
|
|
*TODO: Add screenshots from MSSQL Management Studio*
|
|
|
|
|
|
```sql
|
|
|
CREATE DATABASE ggce COLLATE SQL_Latin1_General_CP1_CI_AS
|
|
|
GO
|
|
|
```
|
|
|
|
|
|
Create separate credentials `ggce` for GGCE to interact with the database:
|
|
|
|
|
|
```sql
|
|
|
USE [master]
|
|
|
GO
|
|
|
CREATE LOGIN [ggce] WITH PASSWORD = 'someRandomPassword';
|
|
|
GO
|
|
|
|
|
|
-- Grant owner permissions on ggce-db
|
|
|
USE [ggce-db]
|
|
|
GO
|
|
|
CREATE USER [ggce] FOR LOGIN [ggce]
|
|
|
GO
|
|
|
ALTER AUTHORIZATION ON SCHEMA::[db_owner] TO [ggce]
|
|
|
GO
|
|
|
ALTER ROLE [db_owner] ADD MEMBER [ggce]
|
|
|
GO
|
|
|
```
|
|
|
|
|
|
At startup GGCE will detect if the database is blank and will automatically create the database schema and populate it with the initial data, including the default administrator/administrator user account.
|
|
|
|
|
|
## Using an existing GRIN-Global database
|
|
|
|
|
|
GGCE is backwards compatible with the original GRIN-Global databases. You can upgrade your existing GG database and use it with GGCE.
|
|
|
|
|
|
1. Make a *backup* of your existing GG database.
|
|
|
2. *Restore* the backup as a *new database*.
|
|
|
3. *Configure* GGCE server to *use the restored database*.
|
|
|
4. GGCE *will upgrade the database model* to add support for GGCE functionality.
|
|
|
5. *You cannot run both GG and GGCE against the same database*.
|
|
|
|
|
|
*Note*: The GRIN-Global user credentials remain the same, make sure you have one GG user in the “ADMINS” group.
|
|
|
|
|
|
## Running MSSQL as a Docker container
|
|
|
|
|
|
For testing on macOS and Linux you can run MSSQL Express as a Docker container.
|
|
|
|
|
|
Note: use a Docker volume to persist the data between service restarts.
|
|
|
|
|
|
```bash
|
|
|
# Create a volume for MSSQL data
|
|
|
docker volume create mssql-ggce-data
|
|
|
|
|
|
# Start MSSQL Express
|
|
|
docker run --name 'mssql-ggce' \
|
|
|
-e 'ACCEPT_EULA=Y' \
|
|
|
-e 'SA_PASSWORD=YourStrong@Passw0rd' \
|
|
|
-e 'MSSQL_PID=Express' \
|
|
|
-e 'TZ=UTC' \
|
|
|
-v mssql-ggce-data:/var/opt/mssql \
|
|
|
mcr.microsoft.com/mssql/server:2019-latest
|
|
|
|
|
|
# Connect to the database engine
|
|
|
docker exec -it mssql-ggce \
|
|
|
/opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P 'YourStrong@Passw0rd'
|
|
|
```
|
|
|
|
|
|
Once connected to the MSSQL console, follow the instructions above to create a blank database for GGCE.
|
|
|
|
|
|
## GGCE database configuration options
|
|
|
|
|
|
Please make note of the following database configuration options:
|
|
|
|
|
|
- `DB_URL`
|
|
|
JDBC connect string to RDBMS engine
|
|
|
Example: `dbc:sqlserver://mssqlserver1:1433;DatabaseName=ggce-db`
|
|
|
- `DB_USERNAME`
|
|
|
RDMBS username with owner permissions on `ggce` database
|
|
|
Example: `ggce`
|
|
|
- `DB_PASSWORD`
|
|
|
Username password
|
|
|
Example: `someRandomPassword`
|
|
|
|
|
|
|
|
|
|
|
|
# GGCE Server
|
|
|
|
|
|
**GGCE Server** is available as a Docker image
|
|
|
|
|
|
```bash
|
|
|
dockerhub.croptrust.org/grin-global/grin-global-server:latest
|
|
|
```
|
|
|
|
|
|
Please check[ https://gitlab.croptrust.org/grin-global/grin-global-server/-/releases](https://gitlab.croptrust.org/grin-global/grin-global-server/-/releases) for information on new releases!
|
|
|
|
|
|
## Configuration notes
|
|
|
|
|
|
The **GGCE** server container requires a few configuration options. The options
|
|
|
are provided as environment variables. You need to note:
|
|
|
|
|
|
1. The *database configuration options*.
|
|
|
1. The *disk volume* that will hold *files uploaded by users*.
|
|
|
1. The *name* and the *TCP port* where **GGCE Server** will be accessible.
|
|
|
1. Optionally: The *frontend URL* of the **GGCE Web** service.
|
|
|
|
|
|
## GGCE Server configuration options
|
|
|
|
|
|
Please make note of the following GGCE server configuration options:
|
|
|
|
|
|
- `port`
|
|
|
The exposed TCP port of GGCE Server
|
|
|
With docker, the internal GGCE port 8080 is exposed using the -p flag. For example `-p 80:8080` exposes the internal port 8080 on port 80 of the docker host.
|
|
|
- `BASE_URL`
|
|
|
The URL where the GGCE server is accessible to clients. For example `BASE_URL=https://gserver.genebank.org`.
|
|
|
It is used to set HTTP cookie domain and determine if HTTP cookies are flagged as secure.
|
|
|
- `DATA_DIR`
|
|
|
The disk volume that will hold files uploaded by users.
|
|
|
A docker volume ensures that the data persists between container restarts.
|
|
|
Example: `-v ggce-server-data:/data/gringlobal`
|
|
|
- `FRONTEND_URL`
|
|
|
The frontend URL of the GGCE Web service is/will be accessible.
|
|
|
|
|
|
A **docker volume** ensures that the data persists between container restarts.
|
|
|
|
|
|
```bash
|
|
|
# Create a data volume
|
|
|
docker volume create ggce-server-data
|
|
|
```
|
|
|
|
|
|
Start **GGCE Server** as a Docker container:
|
|
|
|
|
|
```bash
|
|
|
# Pull latest image
|
|
|
docker image pull dockerhub.croptrust.org/grin-global/grin-global-server:latest
|
|
|
|
|
|
# Start GRIN-Global Server
|
|
|
# Make sure that address for MSSQL "jdbc:sqlserver://MSSQL-SERVER:1433" is
|
|
|
# equal to ip from `docker inspect mssql-ggce` output
|
|
|
docker run --name 'ggce-server' \
|
|
|
--memory 4g
|
|
|
-p 8081:8080 \
|
|
|
-e "DB_URL=jdbc:sqlserver://MSSQL-SERVER:1433;DatabaseName=ggce" \
|
|
|
-e "DB_USERNAME=sa" \
|
|
|
-e "DB_PASSWORD=YourStrong@Passw0rd" \
|
|
|
-e "BASE_URL=http://YOUR_DOCKER_HOST_IP:8081" \
|
|
|
-e "FRONTEND_URL=http://YOUR_DOCKER_HOST_IP:3000" \
|
|
|
-e "DATA_DIR=/data/gringlobal" \
|
|
|
-e "JAVA_OPTIONS=-XX:+HeapDumpOnOutOfMemoryError -XX:+UseParallelGC -Xms1500m -XX:+UnlockExperimentalVMOptions -Djava.awt.headless=true -server -Dnetworkaddress.cache.ttl=10 --add-modules java.se --add-exports java.base/jdk.internal.ref=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.nio=ALL-UNNAMED --add-opens java.base/sun.nio.ch=ALL-UNNAMED --add-opens java.management/sun.management=ALL-UNNAMED --add-opens jdk.management/com.sun.management.internal=ALL-UNNAMED" \
|
|
|
-v ggce-server-data:/data/gringlobal \
|
|
|
dockerhub.croptrust.org/grin-global/grin-global-server:latest
|
|
|
```
|
|
|
|
|
|
***Note*:** After hitting **Enter** You should immediately seeing log messages on the console. If not, then your command is not valid. Hit `Ctrl + C` and retry.
|
|
|
|
|
|
GGCE internally listens on port **8080**. The `-p 8081:8080` flag above instructs docker to map the host port **8081** to the internal port 8080 used by GGCE. This makes your GGCE server accessible at http://YOUR_DOCKER_HOST_IP:8081 and cooresponds with variable `BASE_URL` in the command above.
|
|
|
|
|
|
When you see the message `[INFO] Started Jetty Server` you can connect to your own GGCE instance on the IP of the host
|
|
|
and port **8081** (if you used the command above) at `http://YOUR_DOCKER_HOST_IP:8081`
|
|
|
|
|
|
|
|
|
### Upgrading to newer version
|
|
|
|
|
|
To get the latest version of the image you need to pull the latest *edge* image and recreate your container:
|
|
|
|
|
|
```bash
|
|
|
docker pull dockerhub.croptrust.org/grin-global/grin-global-server:latest
|
|
|
docker stop ggce-server
|
|
|
docker remove ggce-server # if the container name is still defined
|
|
|
docker run... # as above
|
|
|
```
|
|
|
|
|
|
|
|
|
# GGCE Web client
|
|
|
|
|
|
The **GRIN-Global Client** is a set of tools for genebank technicians, curators and managers to interact with the[GGCE Server](https://gitlab.croptrust.org/grin-global/grin-global-server) APIs.
|
|
|
|
|
|
Please check https://gitlab.croptrust.org/grin-global/grin-global-ui/-/releases for information on new releases!
|
|
|
|
|
|
|
|
|
## Running GGCE Web
|
|
|
|
|
|
**GGCE Web** is available as a Docker image `dockerhub.croptrust.org/grin-global/grin-global-ui/gg-ce-web:latest`. It works best with the latest GGCE Server image `dockerhub.croptrust.org/grin-global/grin-global-server:latest`.
|
|
|
|
|
|
|
|
|
The **GGCE Web** container requires a few configuration options. The options are provided as environment variables. You need to note:
|
|
|
|
|
|
1. `API_URL`: The URL to the **GGCE Server** (e.g. `http://YOUR_DOCKER_HOST_IP:8081`)
|
|
|
This should match the *host name* (including port, if not default) where *GGCE Server* is accessible.
|
|
|
1. `ORIGIN`: The frontend URL of the **GGCE Web** service (e.g. `http://YOUR_DOCKER_HOST_IP:3000`)
|
|
|
This should match the `FRONTEND_URL` variable of your *GGCE Server* container when using the default OAuth client. You can omit this if using a OAuth client you declared manually.
|
|
|
1. `CLIENT_ID` and `CLIENT_SECRET`: OAuth Client credentials (or skip to use default values)
|
|
|
OAuth client credentials of **GGCE Web**.
|
|
|
|
|
|
Start **GGCE Server** as a Docker container:
|
|
|
|
|
|
```bash
|
|
|
# Start GGCE Web
|
|
|
docker run --name 'ggce-ui' \
|
|
|
--memory 500m
|
|
|
-p 3000:3000 \
|
|
|
-e "API_URL=http://YOUR_DOCKER_HOST_IP:8081" \
|
|
|
-e "ORIGIN=http://YOUR_DOCKER_HOST_IP:3000" \
|
|
|
dockerhub.croptrust.org/grin-global/grin-global-ui/gg-ce-web:latest
|
|
|
```
|
|
|
|
|
|
**Note:** After hitting **Enter** You should immediately seeing log messages on the console. If not, then your command is not valid. Hit `Ctrl + C` and retry.
|
|
|
|
|
|
GGCE Web internally listens on port **3000**. The `-p 3000:3000` flag above instructs docker to map the host port **3000** to the internal container port **3000**. This makes the GGCE web interface accessible at `http://YOUR_DOCKER_HOST_IP:3000`
|
|
|
|
|
|
### Troubleshooting
|
|
|
|
|
|
The effecive configuration is printed out immediately when you start the **grin-global-web** container.
|
|
|
|
|
|
The *Application config* section (example below) displays the values in use. These need to be properly
|
|
|
configured with Docker environment variables.
|
|
|
|
|
|
```bash
|
|
|
Running Genesys UI from /var/www/GGCE with CLIENT_ID= and API at
|
|
|
Starting on TCP port (default 3000).
|
|
|
2020-07-15T15:29:04: PM2 log: Launching in no daemon mode
|
|
|
2020-07-15T15:29:04: PM2 log: App [GGCE:0] starting in -cluster mode-
|
|
|
2020-07-15T15:29:04: PM2 log: App [GGCE:0] online
|
|
|
{
|
|
|
_: [ 'start', 'GGCE.yml' ],
|
|
|
'origin': '',
|
|
|
'api-timeout': '',
|
|
|
'api-url': '',
|
|
|
'client-id': '',
|
|
|
'client-secret': '',
|
|
|
'allow-robots': ''
|
|
|
}
|
|
|
Application config {
|
|
|
origin: 'http://localhost:3000',
|
|
|
apiTimeout: 10000,
|
|
|
apiUrl: 'http://localhost:8080',
|
|
|
clientId: 'defaultclient@localhost',
|
|
|
clientSecret: 'changeme',
|
|
|
allowRobots: false
|
|
|
}
|
|
|
```
|
|
|
|
|
|
## Next release
|
|
|
|
|
|
The preview of the next release is available as `dockerhub.croptrust.org/grin-global/grin-global-ui/grin-global-web:edge`.
|
|
|
|
|
|
You should update your copy of the image before **restarting** your container:
|
|
|
|
|
|
```bash
|
|
|
docker image pull dockerhub.croptrust.org/grin-global/grin-global-ui/grin-global-web:edge
|
|
|
```
|
|
|
|
|
|
|
|
|
# GGCE Deployment
|
|
|
|
|
|
## Use Traefik to run the containers
|
|
|
|
|
|
Please note that you must configure `grin-global-server` docker container as documented at https://gitlab.croptrust.org/grin-global/grin-global-server#configuration.
|
|
|
|
|
|
1. The *name* or *IP address* of the MSSQL server
|
|
|
2. *Username* and *password* for the MSSQL
|
|
|
3. The *name* or *IP address* where **GGCE Server** will be accessible
|
|
|
4. The *frontend URL* of the **GGCE Web** service
|
|
|
|
|
|
#### `docker run`
|
|
|
|
|
|
Setting environment variables with `docker run` CLI command is done with the `-e "NAME=VALUE"` argument:
|
|
|
|
|
|
```plaintext
|
|
|
docker run [OTHER FLAGS] -e "DB_URL=jdbc:sqlserver://DATABASE_SERVER:1433;DatabaseName=DATABASE" -e "DB_USERNAME=ggce" -e "DB_PASSWORD=somepassword" [OTHER FLAGS] dockerhub.croptrust.org/grin-global/grin-global-server:v2021.8
|
|
|
```
|
|
|
|
|
|
### Compose file
|
|
|
|
|
|
Setting environment variables in the compose file:
|
|
|
|
|
|
```yaml
|
|
|
version: '3'
|
|
|
services:
|
|
|
ggce-server:
|
|
|
image: dockerhub.croptrust.org/grin-global/grin-global-server:latest
|
|
|
environment:
|
|
|
# Database connection
|
|
|
- DB_URL=jdbc:sqlserver://DATABASE_SERVER:1433;DatabaseName=DATABASE
|
|
|
- DB_USERNAME=USERNAME
|
|
|
- DB_PASSWORD=PASSWORD
|
|
|
# Host names and URLs
|
|
|
- HOST_NAME=ggceserver.mydomain
|
|
|
- BASE_URL=http://ggceserver.mydomain
|
|
|
- FRONTEND_URL=http://ggceweb.mydomain
|
|
|
...
|
|
|
```
|
|
|
|
|
|
Your Traefik labels are OK.
|
|
|
|
|
|
`grin-global-server` with HTTPS requires a few more tweaks:
|
|
|
|
|
|
```yaml
|
|
|
environment:
|
|
|
- HOST_NAME=ggceserver.mydomain
|
|
|
- BASE_URL=https://ggceserver.mydomain
|
|
|
- HOST_NAMEANDPORT=ggceserver.mydomain:443
|
|
|
- BASE_COOKIESECURE=true
|
|
|
```
|
|
|
|
|
|
Relevant `gg-ce-web` configuration is documented at https://gitlab.croptrust.org/grin-global/grin-global-ui#running-grin-global-web:
|
|
|
|
|
|
```yaml
|
|
|
ggce-ui:
|
|
|
image: dockerhub.croptrust.org/grin-global/grin-global-ui/gg-ce-web:latest
|
|
|
environment:
|
|
|
- API_URL=http://ggceserver.mydomain
|
|
|
- ORIGIN=http://ggceweb.mydomain
|
|
|
...
|
|
|
```
|
|
|
|
|
|
### All system options
|
|
|
|
|
|
**GGCE** accepts a number of system configuration options. The full list and their default values are listed in [application.properties](https://gitlab.croptrust.org/grin-global/grin-global-server/-/blob/main/server/src/main/resources/application.properties).
|
|
|
|
|
|
For example, the default database connection parameters are:
|
|
|
|
|
|
```properties
|
|
|
db.url=jdbc:sqlserver://localhost:1433;DatabaseName=gringlobal
|
|
|
db.username=sa
|
|
|
db.password=sa1
|
|
|
```
|
|
|
|
|
|
Any of the properties can be configured using **environment** variables. Please convert the property name to uppercase and replace **.** with **_**: `db.url` becomes `DB_URL`.
|
|
|
# See https://ggce.genesys-pgr.org/install/
|
|
|
|