GGCE Server issueshttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues2023-09-05T06:52:04+02:00https://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/44Reporting: Health2023-09-05T06:52:04+02:00Matija ObrezaReporting: HealthThere are innumerable potential pathogens that material can be tested against.
Assuming we only consider the **most recent test results** for each pathogen:
1. How do you declare which pathogens must **be tested for** and **not be pres...There are innumerable potential pathogens that material can be tested against.
Assuming we only consider the **most recent test results** for each pathogen:
1. How do you declare which pathogens must **be tested for** and **not be present** to consider the material *healthy*?Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/43Reporting: Availability indicator2023-09-05T06:51:56+02:00Matija ObrezaReporting: Availability indicator@ncastaneda the proposed indicator is composed of:
* health - conducted at inventory level.
* viability - conducted at inventory level.
* quantity sufficiency - conducted at inventory level.
Let's focus on **viability**. There are pote...@ncastaneda the proposed indicator is composed of:
* health - conducted at inventory level.
* viability - conducted at inventory level.
* quantity sufficiency - conducted at inventory level.
Let's focus on **viability**. There are potentially multiple viability tests (and their results) per inventory.
I assume you want only the results of the **last** test. Even so:
1. What is **sufficient viability threshold** so that the material considered viable?
1. If the test is 10 years old, how do you consider the **time since the test** in marking something as *sufficiently viable*?Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/32GG PW and GG-CE2023-09-05T06:51:22+02:00Matija ObrezaGG PW and GG-CE@gcapilit You're now running the GG PW at gringlobal.irri.org. This is running on the Windows server and probably a different database.
What level of interaction do you see between GG PW and GG-CE Server?
@jalarcon Do you run two datab...@gcapilit You're now running the GG PW at gringlobal.irri.org. This is running on the Windows server and probably a different database.
What level of interaction do you see between GG PW and GG-CE Server?
@jalarcon Do you run two databases and PWs (external and internal) at CIMMYT?
## `Web*` tables
These tables are generally updated by web users. What kinds of integration would you want to put in place?
My assumption is that `web_*` tables would somehow be copied directly from the public database to the internal database to make the records available for processing.
In the other direction, `accession_*` and `order_*` tables would be copied from the internal server to the public database. I could be wrong.https://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/35Processes in request handling2023-09-05T06:51:11+02:00Matija ObrezaProcesses in request handling@jalarcon @gcapilit GG allows for recording process steps as `*Actions`.
There is no specific *status* on the order itself, it is considered as closed when the `completedDate` is set. Everything else is recorded as order actions.
Cons...@jalarcon @gcapilit GG allows for recording process steps as `*Actions`.
There is no specific *status* on the order itself, it is considered as closed when the `completedDate` is set. Everything else is recorded as order actions.
Considering a generic request for material (think distribution), once the list of accessions and request details are filed in GG, the next steps are:
1. **Assign** source inventory to each requested item
1. If there is **no available** inventory, what can we do with that order item? Set SYSTEM inventory and `CANCEL` the item?
1. This leaves us with order items that have valid source inventory and can be packaged for distribution
1. Determine if SMTA acceptance is required
1. If SMTA required, initiate order item action `SMTAACCEPT` for all remaining items
1. If other MTA is required, initiate relevant item action.
1. Determine if IMPORT PERMIT is required
1. Initiate `IMPORT_PER` action on items
For each item in the request:
1. **Retrieve** source inventories from storage
1. **Pack** take material from source inventory and package for distribution. This updates the available `quantityOnHand` of the source inventory.
1. **Verify** that packets for distribution match (@jalarcon This is your `ORIA_VER_SHIP` action)
1. **Ship** the material
Each of these is a task that can be recorded as `OrderRequestItemAction` with its own `start` and `completed` date.
Are any steps missing?
## Relationship with order item `Status`
The status of each order item begins with `NEW` and ends in one of: `SHIPPED`, `CANCEL` or `SPLIT` states.
Intermediate states in GG are: `HOLD`, `PENDING`, `INSPECT` and `QUALITYTEST`. It is not immediately clear how these relate to the actions listed above.Grace CapilitGrace Capilithttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/49CT Genesys Wizard dataview2023-09-05T06:50:53+02:00Lilibeth SisonCT Genesys Wizard dataviewThis is the error while trying to open the `get_passport_data ` dataview using CT:
![image](/uploads/8dfb546e6156575970db4acd151c6829/image.png)This is the error while trying to open the `get_passport_data ` dataview using CT:
![image](/uploads/8dfb546e6156575970db4acd151c6829/image.png)https://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/51Configuring Genesys Wizard in CT2023-09-05T06:50:43+02:00Grace CapilitConfiguring Genesys Wizard in CT@jalarcon , @mobreza
There is an error encountered in configuring Genesys Wizard in CT for Genesys sandbox
Server Address: https://api.sandbox.genesys-pgr.org/
Authorization Code generated in genesys sandbox doesn't work in CT.
![i...@jalarcon , @mobreza
There is an error encountered in configuring Genesys Wizard in CT for Genesys sandbox
Server Address: https://api.sandbox.genesys-pgr.org/
Authorization Code generated in genesys sandbox doesn't work in CT.
![image](/uploads/4e024142b28705a2da67f0d3c6beb547/image.png)
![image](/uploads/9458cdf04a52a01f904f672b90256742/image.png)
Surprisingly, it works with the Genesys production.Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/68Import backup inventories from SGSV2023-09-05T06:50:14+02:00Matija ObrezaImport backup inventories from SGSVAdd an admin function to import information on backups in SGSV as new inventories of existing accessions.
Recycle code from our 2016 implementation of the `SGSVUpdate` at https://gitlab.croptrust.org/genesys-pgr/genesys-server/-/blob/ma...Add an admin function to import information on backups in SGSV as new inventories of existing accessions.
Recycle code from our 2016 implementation of the `SGSVUpdate` at https://gitlab.croptrust.org/genesys-pgr/genesys-server/-/blob/master/src/main/java/org/genesys2/server/service/worker/SGSVUpdate.java
The CSV file from SGSV contains:
| institute_code | deposit_box_number | collection_name | accession_number | full_scientific_name | country_of_collection_or_source | number_of_seeds | regeneration_month_and_year | other_accession_designations | provider_institute_code | accession_url | country_code | country_name | continent_name | seeds | genus | species_epithet | species | taxon_name | date_of_deposit | date_of_dataset | sgsv_template_id | box_id |
|----------------|--------------------|-----------------|------------------|----------------------|---------------------------------|-----------------|-----------------------------|------------------------------|-------------------------|---------------|--------------|--------------|----------------|-------|-------|-----------------|--------------|--------------|-----------------|-----------------|------------------|--------|
| PAK001 | 02-PK/GB | Genebank | 6536 | Oryza sativa | Pakistan (PAK) | 500 | 2006 | | PAK001 | | PAK | Pakistan | Asia | 500 | Oryza | sativa | Oryza sativa | Oryza sativa | 17/06/2009 | 21/05/2009 | | 1092 |
| PAK001 | 02-PK/GB | Genebank | 6538 | Oryza sativa | Pakistan (PAK) | 500 | 2006 | | PAK001 | | PAK | Pakistan | Asia | 500 | Oryza | sativa | Oryza sativa | Oryza sativa | 17/06/2009 | 21/05/2009 | | 1092 |
| PAK001 | 02-PK/GB | Genebank | 6553 | Oryza sativa | Pakistan (PAK) | 500 | 2006 | | PAK001 | | PAK | Pakistan | Asia | 500 | Oryza | sativa | Oryza sativa | Oryza sativa | 17/06/2009 | 21/05/2009 | | 1092 |
| PAK001 | 02-PK/GB | Genebank | 6582 | Oryza sativa | Pakistan (PAK) | 500 | 2006 | | PAK001 | | PAK | Pakistan | Asia | 500 | Oryza | sativa | Oryza sativa | Oryza sativa | 17/06/2009 | 21/05/2009 | | 1092 |
And some additional fields...
In GG-CE, the code has to:
1. Filter SGSV data by admin-provided `String instCode`.
1. Find the matching `Accession` record by
`accession_number === trim(a.accessionNumberPart1 + ' ')`
` + (a.accessionNumberPart2 == null ? '' : a.accessionNumberPart2)`
` + trim(' ' + a.accessionNumberPart3)`
1. If no match, skip.
1. Create an `Inventory` for the accession with
1. `inventoryNumberPart1` <-- `"SGSV"`
1. `inventoryNumberPart2` <-- `sgsv_id`
1. `inventoryNumberPart3` <-- `null`
1. `formTypeCode` <-- `"SD"`
1. Check if such inventory exists and skip if it does.
1. `site` <-- find `Site` by WIEWS CODE *NOR051*
1. `inventoryMaintenancePolicy` <-- find by name *Backup Svalbard*
1. `quantityOnHand` <- `number_of_seeds`
1. `quantityOnHandUnitCode` <- `"SD"`
1. `storageLocation1` <-- `year(date_of_deposit)`
1. `storageLocation2` <-- `deposit_box_number`
1. `???` <-- `date_of_deposit`Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/74Selecting material for viability testing2023-09-05T06:50:02+02:00Matija ObrezaSelecting material for viability testingWhat is the SOP that **initiates** the viability testing? Who (and how) decides which inventories need testing?
In this ticket we will explore how GG-CE can best support this process.
The support for actual testing is discussed in #78What is the SOP that **initiates** the viability testing? Who (and how) decides which inventories need testing?
In this ticket we will explore how GG-CE can best support this process.
The support for actual testing is discussed in #78Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/78Viability testing2023-09-05T06:49:47+02:00Matija ObrezaViability testingSelecting material for viability testing is discussed in grin-global/grin-global-server#74. In this ticket we will discuss the execution of a viability test (i.e. inventories to be tested are already selected) and how to handle the resul...Selecting material for viability testing is discussed in grin-global/grin-global-server#74. In this ticket we will discuss the execution of a viability test (i.e. inventories to be tested are already selected) and how to handle the resulting data.
The viability testing in the *Original GG* is initiated from a `OrderRequest` and resembles the regular distribution procedure. This is sensible as the material needs to be pulled out of storage and the quantities are reduced by the corresponding amounts.
The existing CT *Viability wizard* is described at https://www.grin-global.org/docs/gg_viability_wizard.pdf
- The inventories for viability testing are selected (see #74) and processed as a new `OrderRequest`
- The material is tested and results recordedMatija ObrezaMatija Obrezahttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/86Explaining GG-CE vs GRIN-Global2023-09-05T06:49:35+02:00Matija ObrezaExplaining GG-CE vs GRIN-GlobalGRIN-Global (GG) is an ASP.Net web application running on IIS on a Windows Server. GG uses a MSSQL database to persist data:
![image](/uploads/35188b0d750c2dc5e9fdf7219e634dbb/image.png)
As a web application, GRIN-Global handles incomi...GRIN-Global (GG) is an ASP.Net web application running on IIS on a Windows Server. GG uses a MSSQL database to persist data:
![image](/uploads/35188b0d750c2dc5e9fdf7219e634dbb/image.png)
As a web application, GRIN-Global handles incoming HTTP requests:
- *SOAP* requests to `GUI.asmx`
- *Public website* requests
- `*.aspx` dynamic web page requests
- requests for static resources (images, CSS, JS)
It uses shared components (the middle tier) and triggers to react to incoming data.
The user interactions with the ASP.Net application and its components in more detail:
![image](/uploads/f5fcb071bd491ea0d9acdbff4b396a7e/image.png)
## The SOAP interface
The SOAP interface exposes a handful of methods, two of which facilitate all data exchange with GG: `GetData` and `SaveData`.
Remaining SOAP methods handle file upload/download, changing user language and password, lookup table stats, permission management and search.
# Curator Tool
The Curator Tool interacts with GG using SOAP protocol (XML over HTTP). It requires a local installation of MSSQL express to keep a copy of *Lookup tables*.
# GG-CE Server
GG-CE Server implements the same SOAP interface, making in backward compatible with GG and allowing for use of the Curator Tool and other software that uses the SOAP protocol.
It adds documented JSON API endpoints that make interactions with GG-CE more intuitive and building additional client software (i.e. mobile and web apps) simpler.
`gg-ce-server` does not provide a *"Public" website*. https://gitlab.croptrust.org/grin-global/grin-global-ui implements a **web interface** for genebank technicians and managers.
![image](/uploads/7b4ada44241b068fcc127d745bfd4302/image.png)
# GG-CE Deployment
GG-CE is packaged as Docker images and designed for container deployment on Windows, Linux and macos hosts. This makes it easy for IT department to deploy, manage, backup and upgrade individual components automatically.
![image](/uploads/f3796312c534de54513683ae4047064a/image.png)
Containers can run on different hosts as long as they are able to communicate on designated ports. Only the HTTPS port is exposed to the outside world.Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/130Move @Id to the superclass2023-09-05T06:47:48+02:00Maxym BorodenkoMove @Id to the superclassAll entities in the system (about 150 classes) contain their own `id` property with a list of required annotations.
```java
@Id
@JsonProperty
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "crop_id")
private Long id...All entities in the system (about 150 classes) contain their own `id` property with a list of required annotations.
```java
@Id
@JsonProperty
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "crop_id")
private Long id;
```
The current code looks not very flexible. For example, if we need to change `GenerationType` then we have to update each class instead of a single superclass. We had to use that approach because each entity has a unique name of id column in the DB (`table_name_id`). To get rid of duplicate code we could use JPA's `@AttributeOverride` annotation.
> Used to override the mapping of a <code>Basic</code> (whether explicit or default) property or field or <code>Id</code> property or field.
> May be applied to an entity that extends a mapped superclass or to an embedded field or property to override a basic mapping or id mapping defined by the mapped superclass
We can move the `id` field to the `@MappedSuperclass` and override the mapping of id using `@AttributeOverride`
```java
@Entity
@Table(name = "crop")
@Cacheable
@AttributeOverride(name = "id", column = @Column(name = "crop_id"))
@JsonIdentityInfo(scope = Crop.class, generator = ObjectIdGenerators.PropertyGenerator.class, property = "id")
public class Crop extends CooperatorOwnedModel implements Copyable<Crop> {
...
}
```Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/212GG-CE API client2023-09-05T06:47:28+02:00Maxym BorodenkoGG-CE API clientIn https://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/197 old project layout was converted to a modular Maven project. In this ticket, we will introduce a **client** module. It's a Java client library for GG-CE API.
We...In https://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/197 old project layout was converted to a modular Maven project. In this ticket, we will introduce a **client** module. It's a Java client library for GG-CE API.
We use `springdoc-openapi` library to automating the generation of API documentation. So, explore if we can generate a Java client from that documentation.Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/266Bug: Android BrAPI authentication2023-09-05T06:40:38+02:00Matija ObrezaBug: Android BrAPI authenticationUsing https://play.google.com/store/apps/details?id=com.fieldbook.tracker with BrAPI V2 server https://demo.api.genebanks.org opens the login page, but after successful login in the embedded Firefox (Firefox is the default browser) view ...Using https://play.google.com/store/apps/details?id=com.fieldbook.tracker with BrAPI V2 server https://demo.api.genebanks.org opens the login page, but after successful login in the embedded Firefox (Firefox is the default browser) view GG-CE redirects to `/` page instead of the redirect URL of the OAuth client.https://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/275OOM generating inventoryNumber2023-09-05T06:38:34+02:00Matija ObrezaOOM generating inventoryNumberWith very large databases we get an OOM when generating `inventoryNumber` (less likely `accessionNumbers`) from the parts1..4.
Configure a low enough max memory for JVM and trigger the OOM. Analyze dump.With very large databases we get an OOM when generating `inventoryNumber` (less likely `accessionNumbers`) from the parts1..4.
Configure a low enough max memory for JVM and trigger the OOM. Analyze dump.Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/412DTO2023-09-05T06:37:43+02:00Matija ObrezaDTOThis issue is to test the performance and feasibility of using DTOs instead of entity classes in API.This issue is to test the performance and feasibility of using DTOs instead of entity classes in API.2023.6Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/419Duplicate key exception (1658154335099-2)2023-09-01T05:15:42+02:00Antonio RiveraDuplicate key exception (1658154335099-2)Recently, we have tried to perform a new installation with our more recent database
In the installation process we have been having the following exception:
```
Invocation of init method failed; nested exception is liquibase.exception...Recently, we have tried to perform a new installation with our more recent database
In the installation process we have been having the following exception:
```
Invocation of init method failed; nested exception is liquibase.exception.LiquibaseException: liquibase.exception.MigrationFailedException: Migration failed for changeset classpath:liquibase/liquibase-changeLog.yml::1658154335099-2::ahrybeniuk:
Reason: liquibase.exception.DatabaseException: The CREATE UNIQUE INDEX statement terminated because a duplicate key was found for the object name 'dbo.accession' and the index name 'ndx_uniq_accessionNumber'. The duplicate key value is (<NULL>). [Failed SQL: (1505) CREATE UNIQUE NONCLUSTERED INDEX ndx_uniq_accessionNumber ON accession(accession_number)]
15:59:05,197 main ERROR ContextLoader:313 - Context initialization failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'databaseMigration' defined in org.gringlobal.application.config.DatabaseConfig: Invocation of init method failed; nested exception is liquibase.exception.LiquibaseException: liquibase.exception.MigrationFailedException: Migration failed for changeset classpath:liquibase/liquibase-changeLog.yml::1658154335099-2::ahrybeniuk:
Reason: liquibase.exception.DatabaseException: The CREATE UNIQUE INDEX statement terminated because a duplicate key was found for the object name 'dbo.accession' and the index name 'ndx_uniq_accessionNumber'. The duplicate key value is (<NULL>).
```
We have searched for the error according to the given changeset (1658154335099-2) but it is related to the Inventory table not the Accession table as indicated by the exception.
![image](/uploads/9e00eae35d4dcbca744b12281e58ad61/image.png)
This problem had not occurred in our previous installations and it is present in our two instances: Wheat and Maize (recently updated).
Full exception is here: [1658154335099-2.log](/uploads/bf89fbbff18b16b0e4cdd79b017928b3/1658154335099-2.log)Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/397BrAPI: Upgrade to 2.12023-08-07T06:42:40+02:00Matija ObrezaBrAPI: Upgrade to 2.1This ticket is to track the implementation of BrAPI 2.1 in GGCE.
Please discuss the steps and components in the comments section.This ticket is to track the implementation of BrAPI 2.1 in GGCE.
Please discuss the steps and components in the comments section.Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/417ITPGRFA GLIS configuration2023-08-07T06:41:27+02:00Matija ObrezaITPGRFA GLIS configurationUpdate `application.properties` and documentation for the new ITPGRFA GLIS URL.
- Test server: https://glis.qa.fao.org (apply to `itpgrfa.glis.basepath=` in `application.properties`)
- Production: https://glis.fao.org
## Rename config ...Update `application.properties` and documentation for the new ITPGRFA GLIS URL.
- Test server: https://glis.qa.fao.org (apply to `itpgrfa.glis.basepath=` in `application.properties`)
- Production: https://glis.fao.org
## Rename config variables
Rename in code and in `xxx.properties`:
- `itpgrfa.glis.smta.basepath` → `itpgrfa.easysmta.basepath`
- `itpgrfa.glis.smta...` → `itpgrfa.easysmta...`2023.6Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/418Default sort order id descending2023-08-04T14:33:20+02:00Matija ObrezaDefault sort order id descendingI want to test out default sorting of pages in descending order -- to show latest entries first.I want to test out default sorting of pages in descending order -- to show latest entries first.2023.6Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/grin-global/grin-global-server/-/issues/406The regenerationMethod and inventoryMaintenancePolicy fields are not describe...2023-08-04T08:51:32+02:00Juan Carlos Alarcon MaldonadoThe regenerationMethod and inventoryMaintenancePolicy fields are not described in Swagger.Add the description for `regenerationMethod` and `inventoryMaintenancePolicy` fields in Swagger for these endpoints:
![image](/uploads/c0ad328e9f4df63b7adf4248a1997248/image.png)Add the description for `regenerationMethod` and `inventoryMaintenancePolicy` fields in Swagger for these endpoints:
![image](/uploads/c0ad328e9f4df63b7adf4248a1997248/image.png)Artem HrybeniukArtem Hrybeniuk