Genesys Backend issueshttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues2024-03-11T14:37:42+01:00https://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/699Reject test DOIs2024-03-11T14:37:42+01:00Matija ObrezaReject test DOIsgrin-global/grin-global-server#464 documents a case were a "test DOI" made it to Genesys and then the "real DOI" couldn't be assigned due to how Genesys treats the DOI.
We need to prevent "test DOIs" in Genesys by throwing an error (pro...grin-global/grin-global-server#464 documents a case were a "test DOI" made it to Genesys and then the "real DOI" couldn't be assigned due to how Genesys treats the DOI.
We need to prevent "test DOIs" in Genesys by throwing an error (probably in `@PrePersist/@PreUpdate`).
@mmarsella says we can safely reject DOIs with prefix 10.0155 (i.e. `/^10\.(?!0155)/`. Further advice on matching `/^10\.[1-9]/` pending.2024.1Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/698MaterialSubRequest: Additional states2024-03-28T10:06:39+01:00Artem HrybeniukMaterialSubRequest: Additional statesWe're using the following states:
* `NOTCONFIRMED`: Waiting for the genebank to click the link in the email that they received the request (or used GGCE)
* `CONFIRMED`: Means the genebank clicked the link and they saw the request
* `SEN...We're using the following states:
* `NOTCONFIRMED`: Waiting for the genebank to click the link in the email that they received the request (or used GGCE)
* `CONFIRMED`: Means the genebank clicked the link and they saw the request
* `SENT`: Genebank completed the request
We're missing states:
* `PROCESSING`: The genebank is processing the request.
* `REJECTED`: The genebank rejected the request completely.
Related to https://gitlab.croptrust.org/grin-global/grin-global-ui/-/issues/654.2024.1Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/697Translated entity support2024-02-15T18:48:59+01:00Matija ObrezaTranslated entity supportWe wish to add translation support for `Subset`s and `Dataset`s. Adopt `Translated...` approach from GGCE.We wish to add translation support for `Subset`s and `Dataset`s. Adopt `Translated...` approach from GGCE.2024.1Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/696DTO with mapstruct2024-02-19T10:32:42+01:00Matija ObrezaDTO with mapstructIn preparation for Genesys API **v2** please prepare DTO classes for entities and use `mapstruct` to generate mappings from DTO to Entity classes.
Adopt the `FilteredCRUDController2`, `CRUDController2`, `CRUDService2`, etc. from GGCE as...In preparation for Genesys API **v2** please prepare DTO classes for entities and use `mapstruct` to generate mappings from DTO to Entity classes.
Adopt the `FilteredCRUDController2`, `CRUDController2`, `CRUDService2`, etc. from GGCE as the base for API v2.
We will add `*Lang` support to Genesys (genesys-pgr/genesys-server#697) so `Translated...` base classes and interfaces can also be adopted.2024.1Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/695API Token authentication2024-01-10T06:56:37+01:00Matija ObrezaAPI Token authenticationThis issue adds API Token authentication support to Genesys. It uses common filters, authentication providers from genesys-pgr/application-blocks!142.
- The filter needs to be registered on `/api/**`
- `app-blocks` does not implement a ...This issue adds API Token authentication support to Genesys. It uses common filters, authentication providers from genesys-pgr/application-blocks!142.
- The filter needs to be registered on `/api/**`
- `app-blocks` does not implement a service to load user/client details for the token string, that needs to happen here.
- `ApiTokenService` does not provide any methods to create/update/delete/expire tokens2024.1Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/694Editing descriptor in review2024-03-28T10:06:39+01:00Christelle RabilEditing descriptor in reviewYonas reported this message after pressing the edit buttuon on descriptors in review :
>Something went wrong
>
>A bug caused this rapid unscheduled disassembly of Genesys. Please refresh the page and retry your action. If that does not ...Yonas reported this message after pressing the edit buttuon on descriptors in review :
>Something went wrong
>
>A bug caused this rapid unscheduled disassembly of Genesys. Please refresh the page and retry your action. If that does not help, please contact Genesys support and let them know about the problem.
>
>Error: Cannot be un-published. More than 24 hours have passed since the publication.
>URL: https://www.genesys-pgr.org/descriptors/41d610d1-b1ba-4aae-8f1d-da38ee12f37f
(I also tried on my side, and after clearing cache but also same message as above)
cc: @mobreza2024.1Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/693GLIS DOI: Update only when data is changed2024-01-30T09:37:40+01:00Matija ObrezaGLIS DOI: Update only when data is changed@mmarsella asks us to find a way for Genesys to send to GLIS only changes to passport data or, better, to the descriptors that GLIS captures. The reason is that we are sending a lot of `addTarget` requests to GLIS that result in no updat...@mmarsella asks us to find a way for Genesys to send to GLIS only changes to passport data or, better, to the descriptors that GLIS captures. The reason is that we are sending a lot of `addTarget` requests to GLIS that result in no update to GLIS DOIs.
`ScheduledGLISUpdater` is now using `where(QAccession.accession.lastModifiedDate.after(date).and(QAccession.accession.doi.isNotNull()))`. We force-update the `lastModifiedDate` to reflect that the data is as fresh as possible, even when there is no need to update the record since there are no modifications.
A possible approach is to add a new field `Accession#dataChangedDate` (liquibase can use `lastModifiedDate` value), but we need to be able to detect if there are any changes in the data. Is this a viable option or is there an alternative?
How about checking **auditLogs**? We don't add a new field and we add a check to `ScheduledGLISUpdater` if any fields (including in `AccessionId`, etc) were updated?2024.1Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/692Dataset: Check for unmapped descriptors API2024-03-28T10:06:38+01:00Artem HrybeniukDataset: Check for unmapped descriptors APIRelated to https://gitlab.croptrust.org/genesys-pgr/genesys-ui/-/issues/1032.
We need an API endpoint that:
1. Checks which descriptors are mapped for this dataset in Amphibian (gets their `uuid`s)
1. Removes descriptors from the Datas...Related to https://gitlab.croptrust.org/genesys-pgr/genesys-ui/-/issues/1032.
We need an API endpoint that:
1. Checks which descriptors are mapped for this dataset in Amphibian (gets their `uuid`s)
1. Removes descriptors from the Dataset that are not in Amphibian
1. Adds descriptors that are in Amphibian to the Dataset2024.1Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/691Confirmation of validated subRequest2023-11-10T11:14:13+01:00Matija ObrezaConfirmation of validated subRequestWhen a genebank confirms receipt of the `MaterialSubRequest` and we set `state = CONFIRMED` send a new email notification to the user and to the genebank:
- Subject: "We received your request"
- To: `request.email`
- CC: `instMailToSett...When a genebank confirms receipt of the `MaterialSubRequest` and we set `state = CONFIRMED` send a new email notification to the user and to the genebank:
- Subject: "We received your request"
- To: `request.email`
- CC: `instMailToSetting`
- Body: Template `smtp-request-confirmed`
If template `smtp-request-confirmed` is not available, then email notification is skipped.
### `smtp-request-confirmed`
Based on `smtp-material-request` but with a message "This is to inform you that {institute.title} confirmed receipt of your request to Genesys! They will get in touch with you directly."2023.8Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/690Upgrade JUnit2023-12-04T10:06:18+01:00Matija ObrezaUpgrade JUnitPlease upgrade JUnit on the `gradle` branch.Please upgrade JUnit on the `gradle` branch.2023.8Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/688OpenAPI 3 not generated correctly2023-10-11T12:24:16+02:00Matija ObrezaOpenAPI 3 not generated correctlyhttps://api.sandbox.genesys-pgr.org/v3/api-docs/BrAPI (and others) don't generate `application/json` but base64 encoded string:
```
base64 --decode
eyJvcGVuYXBpIjoiMy4wLjEiLCJpbmZvIjp7InRpdGxlIjoiR2VuZXN5cyBBUEkiLCJkZXNjcmlwdGlvbiI6IlRo...https://api.sandbox.genesys-pgr.org/v3/api-docs/BrAPI (and others) don't generate `application/json` but base64 encoded string:
```
base64 --decode
eyJvcGVuYXBpIjoiMy4wLjEiLCJpbmZvIjp7InRpdGxlIjoiR2VuZXN5cyBBUEkiLCJkZXNjcmlwdGlvbiI6IlRoaXMg
{"openapi":"3.0.1","info":{"title":"Genesys API","description":"This %
```2023.7Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/687Subset and Dataset accessionRefs2023-10-12T17:31:32+02:00Matija ObrezaSubset and Dataset accessionRefsWhen registering `accessionRefs` with subsets or datasets, there is a possibility that the incoming list contains duplicate entries:
> could not execute statement; SQL [n/a]; constraint [UK_dataset_accessions_1]; nested exception is org...When registering `accessionRefs` with subsets or datasets, there is a possibility that the incoming list contains duplicate entries:
> could not execute statement; SQL [n/a]; constraint [UK_dataset_accessions_1]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement\ncould not execute statement\n(conn=807235) **Duplicate entry '2493-TWN001-VI011991-Capsicum' for key `UK_dataset_accessions_1`**
If an entry is duplicated (either in the input or in the database) Genesys should ignore the duplicate and keep just one copy of the `accessionRef`.\2023.7Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/686Traits: Accession filter AND not considered2023-09-15T14:14:51+02:00Matija ObrezaTraits: Accession filter AND not consideredEmbedded Genesys uses `AND` filter and when loading datasets for accession filters, the `AND` part with `institute.code` is not considered in the query. This results in datasets outside the scope of the configured filters of Embedded Gen...Embedded Genesys uses `AND` filter and when loading datasets for accession filters, the `AND` part with `institute.code` is not considered in the query. This results in datasets outside the scope of the configured filters of Embedded Genesys.2023.6Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/683Accession order in trait data2023-09-12T06:42:34+02:00Matija ObrezaAccession order in trait dataWe usually rely on `seqNo + id` for accession order in Genesys. When fetching trait data from Amphibian, we send the UUIDs of all accessions to consider, and Amphibian handles the pagination.
Browsing trait data across different dataset...We usually rely on `seqNo + id` for accession order in Genesys. When fetching trait data from Amphibian, we send the UUIDs of all accessions to consider, and Amphibian handles the pagination.
Browsing trait data across different datasets shows accessions are not ordered:
![image](/uploads/f67f80436856c8ba423407bcf4624938/image.png) ![image](/uploads/df2bb61edaa4e6bde6d34a4d2fbbf45f/image.png)
1. The accession UUIDs in `AccessionService#getAccessionUuids(AccessionFilter filters)` must be ordered `seqNo asc, id asc`
2. Amphibian needs to respect the order of incoming UUIDs (we now use `Set<>` which does not guarantee order)
3. I'm not sure if we can easily fix the order of accessions in [single datasets](https://www.genesys-pgr.org/datasets/567b14d8-7721-481b-9b90-e252cb747031/view) because we don't have `seqNo` nor `id` in Amphibian.2023.6Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/682app-blocks@52023-09-07T18:10:49+02:00Matija Obrezaapp-blocks@5Update models and filters for app-blocks@5.0.Update models and filters for app-blocks@5.0.2023.6Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/681OAuth upgrades2023-08-11T13:08:23+02:00Matija ObrezaOAuth upgradesBring the caching of OAuth tokens and updated token cleanup from GGCE.Bring the caching of OAuth tokens and updated token cleanup from GGCE.2023.5Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/680Lombok2023-08-30T07:55:55+02:00Matija ObrezaLombokAdd lombok and apply only to `***Filter` classes. This will allow us to easily construct filters in unit tests -- so update unit tests to use the new accessor methods.Add lombok and apply only to `***Filter` classes. This will allow us to easily construct filters in unit tests -- so update unit tests to use the new accessor methods.2023.5Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/679Trait data of selected accessions2023-08-19T09:06:10+02:00Matija ObrezaTrait data of selected accessionsEmbedded Genesys uses `AccessionFilter` to limit its scope, this is usually by `{ institute: { code: [ .. ] } }`, but can also be by `{ crop: [ .. ] }`.
## Listing datasets of accessions
There are two aspects to this issue: the first i...Embedded Genesys uses `AccessionFilter` to limit its scope, this is usually by `{ institute: { code: [ .. ] } }`, but can also be by `{ crop: [ .. ] }`.
## Listing datasets of accessions
There are two aspects to this issue: the first is to just find the `Set<UUID> datasets` for the accession filter. The list of published datasets where the accession is documented is stored in `accession.datasets`, but it should be further filtered to only return Amphibian datasets.
The service method needs to use caching for the one argument: `filter`.
The second is an API endpoint in `DatasetController` that returns `Page<Datasets>` using the `Set<UUID> datasets`.
## ~~Listing descriptors of accessions~~
~~Add an API endpoint in `DatasetController` that returns `Page<Descriptor>` of all datasets from `Set<UUID> datasets` for the `filter`.~~
## New endpoint to query Amphibian
We need to be able to query Amphibian to return observations for selected descriptors in selected datasets for selected accessions, while applying the filters for the selected traits.
- This is very similar to returning observations for one accession, just that we now need to return more rows
- Pagination will be an interesting challenge
- Observations from different datasets need to be merged under the same descriptor
This will allow us to find all datasets and all traits for the selected accessions.
From here, we can allow the user to:
1. (Optionally) De-select some of the datasets
1. Select the descriptors of interest
- This is a mandatory step
1. (Optionally) specify filters for the selected trait descriptors
2. Query Amphibian to return observations for selected descriptors in selected datasets for selected accessions, while applying the filters for the selected traits
3. With this we are able to data from across datasets and display it2023.5Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/678ITPGRFA GLIS configuration2023-08-03T19:40:26+02:00Matija ObrezaITPGRFA GLIS configurationUpdate `application.properties` and documentation for the change of ITPGRFA GLIS settings according to genesys-pgr/glis-client#35.
It may be sensible to use:
```
# Obsolete, please use itpgrfa.glis.url directly
itpgrfa.glis.basepath=ht...Update `application.properties` and documentation for the change of ITPGRFA GLIS settings according to genesys-pgr/glis-client#35.
It may be sensible to use:
```
# Obsolete, please use itpgrfa.glis.url directly
itpgrfa.glis.basepath=https:....
# ITPGRFA GLIS URL
itpgrfa.glis.url=${itpgrfa.glis.basepath}/glis/
```2023.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/677Serialized OAuth no longer readable2023-08-03T20:59:16+02:00Matija ObrezaSerialized OAuth no longer readableSince !794 the data in `oauthauthorization` table is no longer compatible with Genesys as it refers to old class names in its JSON.
```
2023-08-02 09:30:24,590 qtp1459794865-12 ERROR JpaOAuth2AuthorizationService:376 - Exception in des...Since !794 the data in `oauthauthorization` table is no longer compatible with Genesys as it refers to old class names in its JSON.
```
2023-08-02 09:30:24,590 qtp1459794865-12 ERROR JpaOAuth2AuthorizationService:376 - Exception in deserializing map
com.fasterxml.jackson.databind.exc.InvalidTypeIdException: Could not resolve type id 'org.genesys2.spring.security.service.JpaOAuth2AuthorizationService$CustomPrincipal' as a subtype of `java.lang.Object`: no such class found
at [Source: (String)"{"@class":"java.util.HashMap","org.springframework.security.oauth2.core.endpoint.OAuth2AuthorizationRequest":{"@class":"org.springframework.security.oauth2.core.endpoint.OAuth2AuthorizationRequest","authorizationUri":"https://api.sandbox.genesys-pgr.org/oauth/authorize","authorizationGrantType":{"value":"authorization_code"},"responseType":{"value":"code"},"clientId":"9rFBj.jctRgeoYRouR9ALfp22i@sandbox.genesys-pgr.org","redirectUri":"https://sandbox.genesys-pgr.org","scopes":["java.util.Collecti"[truncated 6904 chars]; line: 1, column: 1233] (through reference chain: java.util.HashMap["java.security.Principal"])
at com.fasterxml.jackson.databind.exc.InvalidTypeIdException.from(InvalidTypeIdException.java:43) ~[jackson-databind-2.14.2.jar:2.14.2]
...
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3660) ~[jackson-databind-2.14.2.jar:2.14.2]
at org.genesys.spring.security.service.JpaOAuth2AuthorizationService.parseMap(JpaOAuth2AuthorizationService.java:373) ~[classes/:?]
at org.genesys.spring.security.service.JpaOAuth2AuthorizationService.lambda$toObject$0(JpaOAuth2AuthorizationService.java:190) ~[classes/:?]
at org.springframework.security.oauth2.server.authorization.OAuth2Authorization$Builder.attributes(OAuth2Authorization.java:524) ~[spring-security-oauth2-authorization-server-0.4.1.jar:0.4.1]
at org.genesys.spring.security.service.JpaOAuth2AuthorizationService.toObject(JpaOAuth2AuthorizationService.java:190) ~[classes/:?]
at java.util.Optional.map(Optional.java:265) ~[?:?]
at org.genesys.spring.security.service.JpaOAuth2AuthorizationService.findByToken(JpaOAuth2AuthorizationService.java:167) ~[classes/:?]
at org.genesys.spring.security.InvalidatedAccessTokenFilter.doFilterInternal(InvalidatedAccessTokenFilter.java:53) ~[classes/:?]
```2023.5Matija ObrezaMatija Obreza