Genesys Backend issueshttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues2024-03-28T10:06:39+01:00https://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/698MaterialSubRequest: Additional states2024-03-28T10:06:39+01:00Artem HrybeniukMaterialSubRequest: Additional statesWe're using the following states:
* `NOTCONFIRMED`: Waiting for the genebank to click the link in the email that they received the request (or used GGCE)
* `CONFIRMED`: Means the genebank clicked the link and they saw the request
* `SEN...We're using the following states:
* `NOTCONFIRMED`: Waiting for the genebank to click the link in the email that they received the request (or used GGCE)
* `CONFIRMED`: Means the genebank clicked the link and they saw the request
* `SENT`: Genebank completed the request
We're missing states:
* `PROCESSING`: The genebank is processing the request.
* `REJECTED`: The genebank rejected the request completely.
Related to https://gitlab.croptrust.org/grin-global/grin-global-ui/-/issues/654.2024.1Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/694Editing descriptor in review2024-03-28T10:06:39+01:00Christelle RabilEditing descriptor in reviewYonas reported this message after pressing the edit buttuon on descriptors in review :
>Something went wrong
>
>A bug caused this rapid unscheduled disassembly of Genesys. Please refresh the page and retry your action. If that does not ...Yonas reported this message after pressing the edit buttuon on descriptors in review :
>Something went wrong
>
>A bug caused this rapid unscheduled disassembly of Genesys. Please refresh the page and retry your action. If that does not help, please contact Genesys support and let them know about the problem.
>
>Error: Cannot be un-published. More than 24 hours have passed since the publication.
>URL: https://www.genesys-pgr.org/descriptors/41d610d1-b1ba-4aae-8f1d-da38ee12f37f
(I also tried on my side, and after clearing cache but also same message as above)
cc: @mobreza2024.1Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/692Dataset: Check for unmapped descriptors API2024-03-28T10:06:38+01:00Artem HrybeniukDataset: Check for unmapped descriptors APIRelated to https://gitlab.croptrust.org/genesys-pgr/genesys-ui/-/issues/1032.
We need an API endpoint that:
1. Checks which descriptors are mapped for this dataset in Amphibian (gets their `uuid`s)
1. Removes descriptors from the Datas...Related to https://gitlab.croptrust.org/genesys-pgr/genesys-ui/-/issues/1032.
We need an API endpoint that:
1. Checks which descriptors are mapped for this dataset in Amphibian (gets their `uuid`s)
1. Removes descriptors from the Dataset that are not in Amphibian
1. Adds descriptors that are in Amphibian to the Dataset2024.1Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/684Citations and literature2024-03-28T10:05:35+01:00Matija ObrezaCitations and literatureThis ticket adds support for citations and literature to Genesys.
## What does this mean?
Researchers often use data from Genesys and refer to Genesys as the source of the data. Genebanks often maintain the list of citations referencin...This ticket adds support for citations and literature to Genesys.
## What does this mean?
Researchers often use data from Genesys and refer to Genesys as the source of the data. Genebanks often maintain the list of citations referencing specific accessions in their collections. Either way, there is a paper published somewhere (with a URL) and a **citation record**. It is these citation records that we wish to include in Genesys.
A `Citation` includes:
- title (text)
- author names (text)
- year of publication (number)
- journal (text)
- type (one of `enum`: Journal article, Book, Book section, Conference proceedings, Preprint, Report, Other)
- article DOI (text, doi, must be unique in the database)
- download URL, if available (text, URL)
- abstract (long text)
- source of citation (one of: `enum` Google Scholar, bioRxiv, genebank, user submission)
- relevance (one or more of: `enum` Genesys cited, Genesys used, Accession referenced, Genesys discussed, Genesys mentioned)
- owner (SID) - user or `Partner` (like in datasets?)
- status (one of `enum`: SUBMITTED, PUBLISHED, DUPLICATE, REJECTED)
- statusText (text) - explanation why submission is not published
- link to valid `Citation` - used for duplicates, otherwise `null`
#### Extra properties (next version)
Additionally we may consider including:
- author OrcIDs and countries/territories/islands of affiliation
- is paper peer reviewed?
- is paper open access?
- subject of research
- source of funding
- countries/territories/islands covered in research
## Detecting duplicates
We'll use our **Similarity** method to detect similar/duplicate submissions. We'll compare the title, journal, year of publication, author names and abstract.
## Linking to Citations
A `Citation` may refer to Genesys in general (Genesys used, discussed, mentioned) or link to specific Accession**s**, Dataset**s**, or Subset**s**. These "links" will then be rendered on corresponding details pages in Genesys.
Other text-based categories could include **crops** and **species names**.
## User submissions
Users (any role) will be able to use a form in Genesys to submit a new citation. Genesys will first search for duplicates and present the matches to the user. If the user still wants to submit their citation, they'll need to include `submitAnyway=true` in the API request.
New submissions will have `status = SUBMITTED` until reviewed and handled by the Genesys team.
Users will be able to see their submitted citations (and their status) in their Dashboard.
## Genebank submissions
In the future, genebanks will be able to push citations related to `Accessions` (with GGCE).
## Managing citations
Users with `CITATION_MANAGER` or `ADMINISTRATOR` role can modify and publish individual citations. The corresponding UI will be in their Dashboard.
They are able to add links to specific Accession**s**, Dataset**s**, or Subset**s**.
> Inspired by https://www.gbif.org/literature-trackingArtem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/699Reject test DOIs2024-03-11T14:37:42+01:00Matija ObrezaReject test DOIsgrin-global/grin-global-server#464 documents a case were a "test DOI" made it to Genesys and then the "real DOI" couldn't be assigned due to how Genesys treats the DOI.
We need to prevent "test DOIs" in Genesys by throwing an error (pro...grin-global/grin-global-server#464 documents a case were a "test DOI" made it to Genesys and then the "real DOI" couldn't be assigned due to how Genesys treats the DOI.
We need to prevent "test DOIs" in Genesys by throwing an error (probably in `@PrePersist/@PreUpdate`).
@mmarsella says we can safely reject DOIs with prefix 10.0155 (i.e. `/^10\.(?!0155)/`. Further advice on matching `/^10\.[1-9]/` pending.2024.1Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/696DTO with mapstruct2024-02-19T10:32:42+01:00Matija ObrezaDTO with mapstructIn preparation for Genesys API **v2** please prepare DTO classes for entities and use `mapstruct` to generate mappings from DTO to Entity classes.
Adopt the `FilteredCRUDController2`, `CRUDController2`, `CRUDService2`, etc. from GGCE as...In preparation for Genesys API **v2** please prepare DTO classes for entities and use `mapstruct` to generate mappings from DTO to Entity classes.
Adopt the `FilteredCRUDController2`, `CRUDController2`, `CRUDService2`, etc. from GGCE as the base for API v2.
We will add `*Lang` support to Genesys (genesys-pgr/genesys-server#697) so `Translated...` base classes and interfaces can also be adopted.2024.1Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/697Translated entity support2024-02-15T18:48:59+01:00Matija ObrezaTranslated entity supportWe wish to add translation support for `Subset`s and `Dataset`s. Adopt `Translated...` approach from GGCE.We wish to add translation support for `Subset`s and `Dataset`s. Adopt `Translated...` approach from GGCE.2024.1Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/693GLIS DOI: Update only when data is changed2024-01-30T09:37:40+01:00Matija ObrezaGLIS DOI: Update only when data is changed@mmarsella asks us to find a way for Genesys to send to GLIS only changes to passport data or, better, to the descriptors that GLIS captures. The reason is that we are sending a lot of `addTarget` requests to GLIS that result in no updat...@mmarsella asks us to find a way for Genesys to send to GLIS only changes to passport data or, better, to the descriptors that GLIS captures. The reason is that we are sending a lot of `addTarget` requests to GLIS that result in no update to GLIS DOIs.
`ScheduledGLISUpdater` is now using `where(QAccession.accession.lastModifiedDate.after(date).and(QAccession.accession.doi.isNotNull()))`. We force-update the `lastModifiedDate` to reflect that the data is as fresh as possible, even when there is no need to update the record since there are no modifications.
A possible approach is to add a new field `Accession#dataChangedDate` (liquibase can use `lastModifiedDate` value), but we need to be able to detect if there are any changes in the data. Is this a viable option or is there an alternative?
How about checking **auditLogs**? We don't add a new field and we add a check to `ScheduledGLISUpdater` if any fields (including in `AccessionId`, etc) were updated?2024.1Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/695API Token authentication2024-01-10T06:56:37+01:00Matija ObrezaAPI Token authenticationThis issue adds API Token authentication support to Genesys. It uses common filters, authentication providers from genesys-pgr/application-blocks!142.
- The filter needs to be registered on `/api/**`
- `app-blocks` does not implement a ...This issue adds API Token authentication support to Genesys. It uses common filters, authentication providers from genesys-pgr/application-blocks!142.
- The filter needs to be registered on `/api/**`
- `app-blocks` does not implement a service to load user/client details for the token string, that needs to happen here.
- `ApiTokenService` does not provide any methods to create/update/delete/expire tokens2024.1Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/690Upgrade JUnit2023-12-04T10:06:18+01:00Matija ObrezaUpgrade JUnitPlease upgrade JUnit on the `gradle` branch.Please upgrade JUnit on the `gradle` branch.2023.8Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/691Confirmation of validated subRequest2023-11-10T11:14:13+01:00Matija ObrezaConfirmation of validated subRequestWhen a genebank confirms receipt of the `MaterialSubRequest` and we set `state = CONFIRMED` send a new email notification to the user and to the genebank:
- Subject: "We received your request"
- To: `request.email`
- CC: `instMailToSett...When a genebank confirms receipt of the `MaterialSubRequest` and we set `state = CONFIRMED` send a new email notification to the user and to the genebank:
- Subject: "We received your request"
- To: `request.email`
- CC: `instMailToSetting`
- Body: Template `smtp-request-confirmed`
If template `smtp-request-confirmed` is not available, then email notification is skipped.
### `smtp-request-confirmed`
Based on `smtp-material-request` but with a message "This is to inform you that {institute.title} confirmed receipt of your request to Genesys! They will get in touch with you directly."2023.8Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/689Move to OpenSearch2023-10-30T20:10:57+01:00Matija ObrezaMove to OpenSearchhttps://opensearch.org/docs/latest/upgrade-to/upgrade-to/https://opensearch.org/docs/latest/upgrade-to/upgrade-to/Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/687Subset and Dataset accessionRefs2023-10-12T17:31:32+02:00Matija ObrezaSubset and Dataset accessionRefsWhen registering `accessionRefs` with subsets or datasets, there is a possibility that the incoming list contains duplicate entries:
> could not execute statement; SQL [n/a]; constraint [UK_dataset_accessions_1]; nested exception is org...When registering `accessionRefs` with subsets or datasets, there is a possibility that the incoming list contains duplicate entries:
> could not execute statement; SQL [n/a]; constraint [UK_dataset_accessions_1]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement\ncould not execute statement\n(conn=807235) **Duplicate entry '2493-TWN001-VI011991-Capsicum' for key `UK_dataset_accessions_1`**
If an entry is duplicated (either in the input or in the database) Genesys should ignore the duplicate and keep just one copy of the `accessionRef`.\2023.7Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/688OpenAPI 3 not generated correctly2023-10-11T12:24:16+02:00Matija ObrezaOpenAPI 3 not generated correctlyhttps://api.sandbox.genesys-pgr.org/v3/api-docs/BrAPI (and others) don't generate `application/json` but base64 encoded string:
```
base64 --decode
eyJvcGVuYXBpIjoiMy4wLjEiLCJpbmZvIjp7InRpdGxlIjoiR2VuZXN5cyBBUEkiLCJkZXNjcmlwdGlvbiI6IlRo...https://api.sandbox.genesys-pgr.org/v3/api-docs/BrAPI (and others) don't generate `application/json` but base64 encoded string:
```
base64 --decode
eyJvcGVuYXBpIjoiMy4wLjEiLCJpbmZvIjp7InRpdGxlIjoiR2VuZXN5cyBBUEkiLCJkZXNjcmlwdGlvbiI6IlRoaXMg
{"openapi":"3.0.1","info":{"title":"Genesys API","description":"This %
```2023.7Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/686Traits: Accession filter AND not considered2023-09-15T14:14:51+02:00Matija ObrezaTraits: Accession filter AND not consideredEmbedded Genesys uses `AND` filter and when loading datasets for accession filters, the `AND` part with `institute.code` is not considered in the query. This results in datasets outside the scope of the configured filters of Embedded Gen...Embedded Genesys uses `AND` filter and when loading datasets for accession filters, the `AND` part with `institute.code` is not considered in the query. This results in datasets outside the scope of the configured filters of Embedded Genesys.2023.6Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/556OAuth token repository2023-09-12T10:09:24+02:00Matija ObrezaOAuth token repositoryWhen we introduced JWT, we stopped using the `TokenRepository` and validating tokens on the server.
At the moment, an OAuth JWT token cannot be revoked -- the token is valid until it expires and the expiration date is encoded in the tok...When we introduced JWT, we stopped using the `TokenRepository` and validating tokens on the server.
At the moment, an OAuth JWT token cannot be revoked -- the token is valid until it expires and the expiration date is encoded in the token itself.
I think we need to be able to revoke OAuth tokens.
@mborodenko what do you think?Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/683Accession order in trait data2023-09-12T06:42:34+02:00Matija ObrezaAccession order in trait dataWe usually rely on `seqNo + id` for accession order in Genesys. When fetching trait data from Amphibian, we send the UUIDs of all accessions to consider, and Amphibian handles the pagination.
Browsing trait data across different dataset...We usually rely on `seqNo + id` for accession order in Genesys. When fetching trait data from Amphibian, we send the UUIDs of all accessions to consider, and Amphibian handles the pagination.
Browsing trait data across different datasets shows accessions are not ordered:
![image](/uploads/f67f80436856c8ba423407bcf4624938/image.png) ![image](/uploads/df2bb61edaa4e6bde6d34a4d2fbbf45f/image.png)
1. The accession UUIDs in `AccessionService#getAccessionUuids(AccessionFilter filters)` must be ordered `seqNo asc, id asc`
2. Amphibian needs to respect the order of incoming UUIDs (we now use `Set<>` which does not guarantee order)
3. I'm not sure if we can easily fix the order of accessions in [single datasets](https://www.genesys-pgr.org/datasets/567b14d8-7721-481b-9b90-e252cb747031/view) because we don't have `seqNo` nor `id` in Amphibian.2023.6Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/682app-blocks@52023-09-07T18:10:49+02:00Matija Obrezaapp-blocks@5Update models and filters for app-blocks@5.0.Update models and filters for app-blocks@5.0.2023.6Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/685Draft: Deduplicated AccessionRefs2023-09-04T16:41:16+02:00Matija ObrezaDraft: Deduplicated AccessionRefsWe use `SubsetAccessionRef` and `DatasetAccessionRef` to store references to accessions. Each reference may point to an actual `Accession` in Genesys, or has a `null` reference. Each subset and dataset has its own copy of accession refer...We use `SubsetAccessionRef` and `DatasetAccessionRef` to store references to accessions. Each reference may point to an actual `Accession` in Genesys, or has a `null` reference. Each subset and dataset has its own copy of accession references, meaning that when referring to the same accession (for example `inst=NGA039, acce=TMp-419, genu=Manihot, spec=esculenta, doi=...`) we have multiple entries every time the same combination of `instituteCode`, `accessionNumer`, `genus`, ... is referenced.
In this issue we extract `AccessionReference` to a new entity with `@Table("accession_ref")`. Each reference can appear only once in this table. Instead of using `null`, we will use a blank string `""` to enable unique constraint checks in MariaDB.Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/584Cleanup: Drop API v02023-09-04T12:30:44+02:00Matija ObrezaCleanup: Drop API v0I don't think we use API **/api/v0** calls anywhere, the only place could be our admin MVC.
I believe that can be dropped.I don't think we use API **/api/v0** calls anywhere, the only place could be our admin MVC.
I believe that can be dropped.Matija ObrezaMatija Obreza