Genesys Backend issueshttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues2019-08-13T10:21:01+02:00https://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/56GLIS DOI support2019-08-13T10:21:01+02:00Matija ObrezaGLIS DOI supportGLIS will provide DOI (https://www.doi.org) registration service for genebanks and other PGRFA (perhaps also GRFA, PGR) holders.
The DOI assigned by GLIS will be in the regexp format `10\.18730/[A-Z0-9]+` (production GLIS) and `10\.0155...GLIS will provide DOI (https://www.doi.org) registration service for genebanks and other PGRFA (perhaps also GRFA, PGR) holders.
The DOI assigned by GLIS will be in the regexp format `10\.18730/[A-Z0-9]+` (production GLIS) and `10\.0155/[A-Z0-9]+` (test environment). It is possible (and likely) that individual organizations assign their own DOI to accessions (`10\.[0-9]+/.+` format).
In this ticket we add support for DOI only and ignore other PUID types uploaded to Genesys.
# GLIS DOI extras
Checksum: the last character of the GLIS DOI is a checksum and can be one of *, ~, $, =, U and `[A-Z0-9]`.
Characters I, L, O, U are not in the "DOI" part of the DOI.
http://www.crockford.com/wrmg/base32.html
# The "big picture"
![Genebank_Genesys_GLIS](/uploads/751baf8c6a9b6e3589e91395123c3ab8/Genebank_Genesys_GLIS.png)
# Key principles
1. For genebank material (accessions) there is one DOI assigned to one accession.
1. From the moment an accession has a DOI assigned, that record can not be deleted from Genesys (REST delete should return 403 Forbidden).
1. Any other URL (like the current `/acn/id/[0-9]+`) must permanently redirect to the DOI'zed accession URL in Genesys.
# Accession URL
For accessions with a DOI registered in Genesys, we will use the following URL template: `https://www.genesys-pgr.org/[DOI]` for example `https://www.genesys-pgr.org/10.18730/A3C9`. Every URL to an accession with DOI must use the DOI'zed URL.
# Managing accession passport data
We add a **unique** DOI field (`doi varchar(255) null`) to the accession record.
Genebanks obtain DOI for their accessions and register the DOIs with their accession passport data. On upload of passport data to Genesys, the DOI is included with the passport data (next to `INSTCODE` and `ACCENUMB`).
The **DOI** takes absolute priority when identifying accession records. We now use the combination of `INSTCODE`, `ACCENUMB` and depending on the institute also `GENUS` to identify the record in our system. With the DOI provided, we only need to look up the record by the unique DOI value.
1. When DOI does not exist Genesys, record lookup is based on current approach (`INSTCODE`, `ACCENUMB`, `GENUS`?).
1. When DOI exists in Genesys we are updating that one single accession record.
**There is a possibility in current GLIS that the same DOI would be propagated between PGRFA holders and new holders opt to reuse the DOI assigned by the source genebank.** This can only be resolved by resolving the DOI against GLIS and obtaining information on the registered holder. Non-GLIS DOI resolution may not return the same information or even use the same format. This also should be explored in the future.
# Holder validation
Only on the first registration of DOI in Genesys we would need to resolve the DOI and retrieve registration data from GLIS. We need to verify that the WIEWS code of the holding institute matches the provided data.
Validation is implemented in genesys-pgr/glis-client#21
2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/251Excel corrupted2018-04-24T19:04:23+02:00Matija ObrezaExcel corruptedLarge datasets (~70K rows) break Excel 2016, 2013 limit on number of hyperlinks.
https://support.office.com/en-us/article/excel-specifications-and-limits-1672b34d-7043-467e-8e27-269d656771c3
> Hyperlinks in a worksheet: 66,530 hyperlinksLarge datasets (~70K rows) break Excel 2016, 2013 limit on number of hyperlinks.
https://support.office.com/en-us/article/excel-specifications-and-limits-1672b34d-7043-467e-8e27-269d656771c3
> Hyperlinks in a worksheet: 66,530 hyperlinks2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/252Delete user account2018-04-28T13:12:52+02:00Matija ObrezaDelete user accountAllow users to delete their Genesys user account.
The User/SID is linked to many entities. It is best if we replace all personal information with:
1. `email`: Used for logging in and must be unique. Replace with `deleted@${timestamp}` ...Allow users to delete their Genesys user account.
The User/SID is linked to many entities. It is best if we replace all personal information with:
1. `email`: Used for logging in and must be unique. Replace with `deleted@${timestamp}` (e.g. `deleted@2134523311`)
1. `fullName`: Displayed in various places, replace with `Account removed`
1. `accountType`: Add `DELETED` option and use that.
1. `enabled`: Set to `false`.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/254FTP password2018-04-26T14:58:20+02:00Matija ObrezaFTP passwordUsers are not able to generate FTP password.Users are not able to generate FTP password.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/255Rename model2018-05-03T20:46:16+02:00Matija ObrezaRename modelThe **DS2** model `Descriptor` does not follow the usual `DS2Descriptor` naming. Rename model and database table.The **DS2** model `Descriptor` does not follow the usual `DS2Descriptor` naming. Rename model and database table.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/259Upgrade appblocks2018-05-18T00:35:40+02:00Matija ObrezaUpgrade appblocksUpgrade to latest app-blocksUpgrade to latest app-blocks2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/260Increased page size for API calls2018-06-12T18:21:44+02:00Matija ObrezaIncreased page size for API callsThe `pageSize` for API responses is limited to 50. Increase this to 1000 for API calls.The `pageSize` for API responses is limited to 50. Increase this to 1000 for API calls.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/269Invalid JSON2018-07-15T12:02:14+02:00Matija ObrezaInvalid JSONSome code is generating invalid JSON:
```
{"lastModifiedDate":["2014-7-23"],+"institute.code":["UKR028"]}
```
There is a space `+` where there should't be.Some code is generating invalid JSON:
```
{"lastModifiedDate":["2014-7-23"],+"institute.code":["UKR028"]}
```
There is a space `+` where there should't be.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/270Debug upsert2018-07-16T12:34:46+02:00Matija ObrezaDebug upsertPosting
```json
{"acceName":["ajda","","èrna"],"coll":{"collNumb":"Ivan Kreft","collDate":null,"collCode":null,"collName":null,"collSite":"Kranj","collInstAddress":null,"collMissId":null,"collSrc":50},"geo":{"latitude":46.2333,"longitu...Posting
```json
{"acceName":["ajda","","èrna"],"coll":{"collNumb":"Ivan Kreft","collDate":null,"collCode":null,"collName":null,"collSite":"Kranj","collInstAddress":null,"collMissId":null,"collSrc":50},"geo":{"latitude":46.2333,"longitude":14.3667,"elevation":357.0,"uncertainty":null,"datum":null,"method":null},"sampStat":500,"ancest":null,"donorCode":null,"donorNumb":"Dr. T. Tajnšek","donorName":null,"otherNumb":["RUS001:106","unknown:106"],"duplSite":null,"storage":null,"acceUrl":null,"remarks":null,"instituteCode":"SVN018","accessionNumber":"SRGB 2080","acquisitionDate":null,"breederCode":null,"origCty":"SVN","taxonomy":{"genus":"Fagopyrum","species":"esculentum","spAuthor":null,"subtaxa":null,"subtAuthor":null}}
```
fails with
```
16:27:59,579 qtp1156060786-409 WARN o.g.s.s.c.r.AccessionController:251 - Upsert failed due to: could not execute statement; SQL [n/a]; constraint [null]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement data=...
2018-07-15T16:27:59.591979430Z sandbox_server.1.i0duijjbjwbj@genesys3 | org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; constraint [null]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement
```2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/278Bug: Create new list functionality2018-07-21T10:54:28+02:00Matija ObrezaBug: Create new list functionalityCreating new list fails with 500 server error. Steps to reproduce:
- Go to the https://sandbox.genesys-pgr.org/welcome
- Login as admin;
- Go to the "Browse" page;
- Add some accessions to the list;
- Go to the "My List" page;
- Go to t...Creating new list fails with 500 server error. Steps to reproduce:
- Go to the https://sandbox.genesys-pgr.org/welcome
- Login as admin;
- Go to the "Browse" page;
- Add some accessions to the list;
- Go to the "My List" page;
- Go to the "Accession List" tab;
- Fill the fields and click on the "CREATE NEW LIST" button.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/279Bug: Latitude filter fails2018-07-20T22:33:07+02:00Matija ObrezaBug: Latitude filter fails- Go to the "Browse" page
- Open Latitude filter
- Input some value in the filter fields- Go to the "Browse" page
- Open Latitude filter
- Input some value in the filter fields2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/280Bug: CWR page 5002018-08-27T10:06:08+02:00Matija ObrezaBug: CWR page 500- Go to the https://sandbox.genesys-pgr.org/welcome
- Click on the CWR Project link;
## Overviews
The page is fixed in 78739c47, but the overviews do not show up. Fix the overviews.- Go to the https://sandbox.genesys-pgr.org/welcome
- Click on the CWR Project link;
## Overviews
The page is fixed in 78739c47, but the overviews do not show up. Fix the overviews.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/295Accession overview API2018-08-03T05:07:05+02:00Matija ObrezaAccession overview API2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/299OOM reindexing Accessions2018-08-21T17:45:53+02:00Matija ObrezaOOM reindexing Accessions2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/301Accession listIds not in ES2018-08-22T18:13:47+02:00Matija ObrezaAccession listIds not in ESOverviews for CWR project-related accessions show no data -- ES has no information on listIds?Overviews for CWR project-related accessions show no data -- ES has no information on listIds?2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/304FTP: uploading fails2018-09-02T19:21:04+02:00Matija ObrezaFTP: uploading failsIRRI is trying to upload images and documents to genesys sandbox but there's an error, please see below:
![image](/uploads/765b568bc7b71c10d5c4a667e21d9954/image.png)
The issue relates to permission checks on RepositoryService.IRRI is trying to upload images and documents to genesys sandbox but there's an error, please see below:
![image](/uploads/765b568bc7b71c10d5c4a667e21d9954/image.png)
The issue relates to permission checks on RepositoryService.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/305Overview data invalid2018-08-23T17:36:10+02:00Matija ObrezaOverview data invalidSee https://sandbox.genesys-pgr.org/explore/overview?filter=%7B%22institute.code%22%3A%5B%22NGA039%22%5D%7DSee https://sandbox.genesys-pgr.org/explore/overview?filter=%7B%22institute.code%22%3A%5B%22NGA039%22%5D%7D2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/306Repository: permissions2018-09-30T11:30:46+02:00Matija ObrezaRepository: permissionsAnonymous users cannot access https://sandbox.genesys-pgr.org/acn/id/538527Anonymous users cannot access https://sandbox.genesys-pgr.org/acn/id/5385272.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/307Support inverse filters2018-09-13T14:45:00+02:00Matija ObrezaSupport inverse filtersThe current implementation of Filter objects and logic in Genesys allows for specifying "matching" or "positive" filters (e.g. `"crop": [ "maize", "wheat" ]`). We're aiming to support "non-matching" or "negative" filters (e.g. crop is no...The current implementation of Filter objects and logic in Genesys allows for specifying "matching" or "positive" filters (e.g. `"crop": [ "maize", "wheat" ]`). We're aiming to support "non-matching" or "negative" filters (e.g. crop is not maize or wheat).2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/308ES query error on storage2018-08-29T10:17:35+02:00Matija ObrezaES query error on storageFiltering for `"storage": [ 10, 13 ]` results in an invalid ES query being generated:
```json
{ "query":
{
"bool" : {
"filter" : {
"bool" : {
"must" : {
"terms" : {
"storage" : [ [ 13, 30 ] ]
...Filtering for `"storage": [ 10, 13 ]` results in an invalid ES query being generated:
```json
{ "query":
{
"bool" : {
"filter" : {
"bool" : {
"must" : {
"terms" : {
"storage" : [ [ 13, 30 ] ]
}
}
}
}
}
} }
```
Error is: `[For input string: "["]; nested: NumberFormatException[For input string: "["];`2.4Matija ObrezaMatija Obreza