Genesys Backend issueshttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues2023-08-28T05:47:00+02:00https://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/471Scheduled repeat of email notifications to genebanks2023-08-28T05:47:00+02:00Matija ObrezaScheduled repeat of email notifications to genebanksThe `MaterialSubRequest` maintains the `state` (0 for receipt not confirmed and 1 for confirmed) and the `lastReminderDate`.
Implement a scheduled task (to run every 4 hours) that:
1. Finds sub requests with `state == 0` and `lastRemin...The `MaterialSubRequest` maintains the `state` (0 for receipt not confirmed and 1 for confirmed) and the `lastReminderDate`.
Implement a scheduled task (to run every 4 hours) that:
1. Finds sub requests with `state == 0` and `lastReminderDate > 5 days ago`.
1. Exclude requests with `lastReminderDate > 1 month`.
1. Call `relayRequest(MaterialSubRequest materialSubRequest)` in `RequestServiceImpl` to send out new email. This updates `lastReminderDate`.Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/466Upgrade project dependencies2019-10-03T11:13:00+02:00Matija ObrezaUpgrade project dependenciesPlease see which project dependencies can be updated to their most up-to-date versions without impact to the code.Please see which project dependencies can be updated to their most up-to-date versions without impact to the code.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/440Inconsistent API response2019-05-06T11:34:33+02:00Matija ObrezaInconsistent API responseReported by @osavran, the descriptors sort property from response has this structure:
![Screenshot_from_2019-05-03_09-16-59](/uploads/0bfa68e143c0f780eb545e3a54065963/Screenshot_from_2019-05-03_09-16-59.png)
and for other modules:
![S...Reported by @osavran, the descriptors sort property from response has this structure:
![Screenshot_from_2019-05-03_09-16-59](/uploads/0bfa68e143c0f780eb545e3a54065963/Screenshot_from_2019-05-03_09-16-59.png)
and for other modules:
![Screenshot_from_2019-05-03_09-12-37](/uploads/339d1a3b521f6a138330af494b8e364a/Screenshot_from_2019-05-03_09-12-37.png)
`PaginationComponent` throw error "`pageObj.sort[0]` is undefined", descriptors action `updateRoute` also won't work. I think the same problem is in other components.
## Notes
This may have to do with difference between ES and JPA `Page<?>` responses.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/432Descriptor lists: Overview2019-03-28T10:09:35+01:00Matija ObrezaDescriptor lists: OverviewSimilar to Accession pages, we would like to add **Overview** page for `DescriptorLists` (including when filtered). This requires a new API endpoint.
Term summaries for:
- Data provider
- CropSimilar to Accession pages, we would like to add **Overview** page for `DescriptorLists` (including when filtered). This requires a new API endpoint.
Term summaries for:
- Data provider
- Crop2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/427Remove PublishValidationInterface2019-03-07T14:01:05+01:00Matija ObrezaRemove PublishValidationInterfaceRemove `PublishValidationInterface` as we're using javax validations.
CC @vpavlovRemove `PublishValidationInterface` as we're using javax validations.
CC @vpavlov2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/424AccessionUploader: delete operation2019-02-28T14:43:18+01:00Matija ObrezaAccessionUploader: delete operationThe `delete` operation in the AccessionUploader should be doing something very similar to `removeAccessions` in `GenesysServiceImpl`: it should copy the current `Accession` to `AccessionHistoric`.
I guess this function was forgotten whe...The `delete` operation in the AccessionUploader should be doing something very similar to `removeAccessions` in `GenesysServiceImpl`: it should copy the current `Accession` to `AccessionHistoric`.
I guess this function was forgotten when the new Uploader was implemented.
Please add a unit test.2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/421Versioned Subsets2019-03-05T22:24:56+01:00Matija ObrezaVersioned SubsetsWe'd like to implement *versioned subsets*: subset may be updated by the data provider and (when published) both versions should be available through Genesys.
1. Obsolete subsets should not be listed by default
1. Obsolete subset must p...We'd like to implement *versioned subsets*: subset may be updated by the data provider and (when published) both versions should be available through Genesys.
1. Obsolete subsets should not be listed by default
1. Obsolete subset must point to a new version of the subset
## Simple approach
The very simple approach is to add `private Subset nextVersion` to the `Subset` entity and then we know that a subset has been replaced if `nextVersion != null`. And we can display a link to the new version.
**Problem:** This would only work if the new subset would be immediately published. But it has to go through review, and until the updated version is in `PUBLISHED` state we do not have any mechanism to say that the old version is "current" (the latest).
## Dataset approach
We have this implemented for `Dataset` in `DatasetVersions` (but we are not using it at the moment). Subsets are much simpler than datasets, so I suggest we try it here. We will need to introduce `SubsetVersions` (similar to `DatasetVersions`) and implement service methods that allow for the following:
1. The owner of a subset can create a new Subset based on an existing `PUBLISHED` Subset
1. If the existing subset is not published, reject the request.
1. A new `Subset` is created in `DRAFT` mode and **all data is copied** to the new version
1. The old and the new `Subset` both link to `SubsetVersions` entity
1. When the new `Subset` is **published** the `SubsetVersions#currentVersion` is changed to point to this version.
1. Queries for listing Subsets must only return latest versions:
1. `select s from Subset s where s.current = true and s.state = 'PUBLISHED'`.
1. Users can still access a subset that is **not current**, but it is published: `loadSubset` should only check that `state = 'PUBLISHED'`.
1. We cannot inner join `SubsetVersions` because we will not store that in the Elasticsearch.
1. The service methods must update `subset.current` when the new version is `PUBLISHED`.
1. Only one `Subset` of all versions of `SubsetVersions` can be current. This is best achieved with a unique key on `subset.versions + subset.current`, but we have to set `subset.current = null` when the subset is not the current version.
1. This means that `subset.current = true` means it is the currently published version and `subset.current = null` means it is not a current version.
### New Subsets
1. A new subset is created as it is created now, but a `SubsetVersions` is created for as well.
1. The new subset has `subset.current = null` and `SubsetVersions#currentVersion = null`.
1. When the subset is published, both these values are updated: `subset.current = true` and `SubsetVersions#currentVersion = ¤tSubset`.
## Un-publishing Subsets
If a `PUBLISHED` subset is unpublished (by an Administrator) we have the case when:
1. The subset is **not current** and it **must not** be unpublished. Reject.
1. The subset **is current**: we need to pick a subset that is published and that can replace the current version
1. If there is no other version of a subset (this is the first published version): set `subset.current = null` and `SubsetVersions#currentVersion = null`
1. If there are other published versions of the subset (in `SubsetVersions#allVersions`) then make the youngest (by `createdDate`) the currently published version.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/419sitemap-INSTCODE.xml2019-02-11T11:53:41+01:00Matija Obrezasitemap-INSTCODE.xmlWe now generate the `sitemap-wiews.xml` that generates a sitemap to institute pages:
```java
for (FaoInstitute institute : instituteService.listActive(new PageRequest(0, Integer.MAX_VALUE))) {
writePage(response, sb, new SitemapPage("/...We now generate the `sitemap-wiews.xml` that generates a sitemap to institute pages:
```java
for (FaoInstitute institute : instituteService.listActive(new PageRequest(0, Integer.MAX_VALUE))) {
writePage(response, sb, new SitemapPage("/wiews/" + institute.getCode().toUpperCase()));
}
```
Instead it should reference another sitemap `sitemap-{INSTCODE}.xml` with the links to pages:
- `new SitemapPage("/wiews/" + institute.getCode().toUpperCase())`
- a `new SitemapPage("/a/...."` for every accession of that institute (use `AccessionProcessor`).2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/418Invalid handling of collCode, collNumb2019-02-17T16:04:54+01:00Matija ObrezaInvalid handling of collCode, collNumbGenesys partners sometimes submit `collCode` as an MCPD array string `COOD003;BEN089` instead of `["COOD003", "BEN089"]` array. Same for `collName`.
In API **v0** please add to `upgradeToV2` method some logic that will ensure these are ...Genesys partners sometimes submit `collCode` as an MCPD array string `COOD003;BEN089` instead of `["COOD003", "BEN089"]` array. Same for `collName`.
In API **v0** please add to `upgradeToV2` method some logic that will ensure these are converted to `String[]`.
Update `liquibase-changeLog.yml` with SQL that converts current incorrect data for `collCode` (e.g. `COOD003;BEN089`) to two records.
*Not sure how valid this SQL is, but it seems to work*
```yml
- sql:
drop table if exists t_collCode;
create table if not exists t_collCode as
select
@num_e := 1 + LENGTH(collCode) - LENGTH(REPLACE(collCode, ';', '')) AS num_e,
collectId, collCode,
SUBSTRING_INDEX(collCode,';',1) as coll1,
IF(@num_e > 1, SUBSTRING_INDEX(SUBSTRING_INDEX(collCode,';',2),';',-1), null) coll2,
IF(@num_e > 2, SUBSTRING_INDEX(SUBSTRING_INDEX(collCode,';',3),';',-1), null) coll3,
IF(@num_e > 3, SUBSTRING_INDEX(SUBSTRING_INDEX(collCode,';',3),';',-1), null) coll4
from accession_collect_code where collCode is not null and (LENGTH(collCode) - LENGTH(REPLACE(collCode, ';', ''))) > 0;
delete from accession_collect_code where collectId in (select distinct collectId from t_collCode);
insert into accession_collect_code (collectId,collCode)
(select distinct collectId, coll1 from
(select collectId, coll1 from t_collCode where coll1 is not null
union all select collectId, coll2 from t_collCode where coll2 is not null
union all select collectId, coll3 from t_collCode where coll3 is not null
union all select collectId, coll4 from t_collCode where coll4 is not null) Q);
drop table t_collCode;
```
Apparently we don't have any such problems in `accession_collect_name` table.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/416Update of Accession Aliases2019-02-17T16:04:52+01:00Matija ObrezaUpdate of Accession AliasesThe current code in `AccessionUploader` doesn't handle some cases very well.
There must be only one entry of type `DONORNUMB` with `donorNumb` and `donorCode` sourced from the incoming accession *JSON*. The code should simply update the...The current code in `AccessionUploader` doesn't handle some cases very well.
There must be only one entry of type `DONORNUMB` with `donorNumb` and `donorCode` sourced from the incoming accession *JSON*. The code should simply update the existing `AccessionAlias` of type `DONORNUMB` with the provided values (`usedBy = donorCode`, `name = donorNumb`).
There can be multiple entries of type `BREDNUMB`, `ACCENAME` and others.2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/415Better exceptions2019-02-19T15:09:36+01:00Matija ObrezaBetter exceptionsWe are using both `org.genesys2.spring.ResourceNotFoundException` and the `NotFoundElement?Exception`. Please consolidate the code so that one of these is used (the `NotFound` one) because it already uses nice logging.
```
14:20:51,979 ...We are using both `org.genesys2.spring.ResourceNotFoundException` and the `NotFoundElement?Exception`. Please consolidate the code so that one of these is used (the `NotFound` one) because it already uses nice logging.
```
14:20:51,979 qtp2027961269-14 ERROR o.g.s.a.ApiExceptionHandler:211 - Wow! Such! Exception!
org.genesys2.spring.ResourceNotFoundException
at org.genesys2.server.service.impl.AccessionServiceImpl.getAccessionDetails(AccessionServiceImpl.java:162)
...
...
```
## Throw `InvalidApiUsageException` in:
```
java.lang.RuntimeException: Refusing to export more than 200000 entries
at org.genesys2.server.mvc.ExplorerController.kml(ExplorerController.java:775)
```
Check code where it makes sense to replace `RuntimeException` with `InvalidApiUsageException`, for example:
```
java.lang.RuntimeException: Could not render image
at org.genesys2.server.mvc.ExplorerController.tile(ExplorerController.java:830)
at org.genesys2.server.mvc.ExplorerController$$FastClassBySpringCGLIB$$827bbdf0.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
```
```
Caused by: java.lang.RuntimeException: Could not parse date yyyy-M-d from --
at org.genesys2.server.service.filter.AppliedFiltersConverter.parseDate(AppliedFiltersConverter.java:362)
at org.genesys2.server.service.filter.AppliedFiltersConverter.lambda$convertDateFilter$5(AppliedFiltersConverter.java:333)
at java.lang.Iterable.forEach(Iterable.java:75)
```
```
java.lang.RuntimeException: Could not convert filters
at org.genesys2.server.service.impl.FilterHandler$AppliedFilters.convert(FilterHandler.java:517)
at org.genesys2.server.mvc.ExplorerController.viewFiltered(ExplorerController.java:240)
at org.genesys2.server.mvc.ExplorerController$$FastClassBySpringCGLIB$$827bbdf0.invoke(<generated>)
```
## There's an NPE
```
13:00:51,218 qtp2027961269-2490 ERROR o.g.s.m.UserControllerAdvice:103 - null on GET http://www.genesys-pgr.org/expl
java.lang.NullPointerException
at org.genesys2.server.service.filter.AccessionFilter.buildQuery(AccessionFilter.java:286)
at org.genesys2.server.service.filter.AccessionFilter.buildPredicate(AccessionFilter.java:116)
```
## And this
```
Caused by:
javax.servlet.ServletException: java.lang.IllegalStateException: Page needs a session and none is available
at org.apache.jsp.WEB_002dINF.jsp.errors.error_jsp._jspService(error_jsp.java:235)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:443)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:386)
```2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/412Scheduled GLIS update2019-02-07T14:32:32+01:00Matija ObrezaScheduled GLIS updateAdd a scheduled task that runs on **1** cluster node every 6 hours and updates GLIS with `addtarget` with new accessions **with DOI** that were modified in the last **8 hours**.
Use code from `glisUpdater` in `AccessionListenersConfig` ...Add a scheduled task that runs on **1** cluster node every 6 hours and updates GLIS with `addtarget` with new accessions **with DOI** that were modified in the last **8 hours**.
Use code from `glisUpdater` in `AccessionListenersConfig` and remove that listener from `AccessionListenersConfig`.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/409ClimateDataController v12019-01-29T18:26:31+01:00Matija ObrezaClimateDataController v1Add a new `ClimateDataController` in API *v1* namespace at */api/v1/climate* as base URL, with the following methods:
```java
@GetMapping("/current/{latitude},{longitude}")
public TileClimate getCurrentClimate(@PathVariable("latitude") ...Add a new `ClimateDataController` in API *v1* namespace at */api/v1/climate* as base URL, with the following methods:
```java
@GetMapping("/current/{latitude},{longitude}")
public TileClimate getCurrentClimate(@PathVariable("latitude") double latitude, double longitude) {
Long tileIndex = WorldClimUtil.getWorldclim25Tile(longitude, latitude);
if (tileIndex == null) {
throw new NotFoundElement("No current climate data for specified (lat, lon)");
}
return climateDataService.climateForTile(tileIndex);
}
@GetMapping("/past/{latitude},{longitude}")
public TileClimate getPastClimate(@PathVariable("latitude") double latitude, double longitude) {
throw new NotFoundElement("Past climate data not available");
}
@GetMapping("/future/{latitude},{longitude}")
public TileClimate getPastClimate(@PathVariable("latitude") double latitude, double longitude) {
throw new NotFoundElement("Future climate data not available");
}
```2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/408KPI ExecutionRun totalValue2019-01-29T00:31:52+01:00Matija ObrezaKPI ExecutionRun totalValueAdd `private Double totalValue` to KPI `ExecutionRun` and fill it with the sum of all `observations.value` in `@PrePersist void prepersist() {}`.
Update liquibase:
- add column
- add SQL to fill `totalValue` with current sums for every...Add `private Double totalValue` to KPI `ExecutionRun` and fill it with the sum of all `observations.value` in `@PrePersist void prepersist() {}`.
Update liquibase:
- add column
- add SQL to fill `totalValue` with current sums for every execution run2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/401Download MCPD for subsets and datasets2019-01-17T21:06:59+01:00Matija ObrezaDownload MCPD for subsets and datasetsAdd API endpoints to generate and stream Excel with MCPD passport data for any:
- subset
- dataset
Add buttons to UI.Add API endpoints to generate and stream Excel with MCPD passport data for any:
- subset
- dataset
Add buttons to UI.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/400Remove AuditedModel class2019-01-11T18:28:58+01:00Matija ObrezaRemove AuditedModel classUpgrade entities that extend `AuditedModel` (`Article`, `ITPGRFAStatus`, `Organization`, `VerificationToken`) to use `AuditedVersionedModel`.
Remove `AuditedModel` from the code.
Update liquibase.Upgrade entities that extend `AuditedModel` (`Article`, `ITPGRFAStatus`, `Organization`, `VerificationToken`) to use `AuditedVersionedModel`.
Remove `AuditedModel` from the code.
Update liquibase.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/399Accession with RepositoryFolder2019-01-15T11:26:44+01:00Matija ObrezaAccession with RepositoryFolderWe currently store accession-related images and files in the file repository at a specific path `/wiews/${instCode}/acn/${acceNumb}`. When loading full accession information, we check if such folder exists (by convention). This makes it ...We currently store accession-related images and files in the file repository at a specific path `/wiews/${instCode}/acn/${acceNumb}`. When loading full accession information, we check if such folder exists (by convention). This makes it difficult when ACCENUMB contains `/`.
Update `AccessionId` and add properties:
- `private RepositoryFolder repositoryFolder;`
- `private int imageCount;`
The `GenesysImageGalleryAspects` needs to be extended and the `repositoryFolder` (of the `ImageGallery`) recorded with the `Accession`. The `imageCount` in the `AccessionId` should be updated after each image `save` or `delete` in the accession's image gallery.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/397AppliedFiltersConverter for sciName2019-01-31T12:07:33+01:00Matija ObrezaAppliedFiltersConverter for sciNameShould filter on full name in Taxonomy2.
```
Caused by: java.lang.RuntimeException: Unhandled Taxonomy filter property=sciName
at org.genesys2.server.service.filter.AppliedFiltersConverter.apply(AppliedFiltersConverter.java:405)
at...Should filter on full name in Taxonomy2.
```
Caused by: java.lang.RuntimeException: Unhandled Taxonomy filter property=sciName
at org.genesys2.server.service.filter.AppliedFiltersConverter.apply(AppliedFiltersConverter.java:405)
at org.genesys2.server.service.filter.AppliedFiltersConverter.convert(AppliedFiltersConverter.java:156)
at org.genesys2.server.service.filter.AccessionFilter.convert(AccessionFilter.java:195)
```2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/395Google Sign-in authentication system2019-01-31T12:07:33+01:00Matija ObrezaGoogle Sign-in authentication systemMigrate from the **Google+ Sign-In** feature to the more comprehensive **Google Sign-in authentication system**. The Google+ Sign-In will be deprecated in late January 2019.
This affects the user logins on Genesys. Please update the bac...Migrate from the **Google+ Sign-In** feature to the more comprehensive **Google Sign-in authentication system**. The Google+ Sign-In will be deprecated in late January 2019.
This affects the user logins on Genesys. Please update the backend code in `GoogleOAuthUtil`.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/394CMS Activity post API v12018-12-24T17:06:23+01:00Matija ObrezaCMS Activity post API v1Upgrade `ActivityPost` to `AuditedVersionedModel`. Update liquibase with `version = 1`.
Add `ActivityPostFilter` and add `listActivityPosts` method with filtering and pagination to CMS controller v1. The Filter can extend `AuditedVersio...Upgrade `ActivityPost` to `AuditedVersionedModel`. Update liquibase with `version = 1`.
Add `ActivityPostFilter` and add `listActivityPosts` method with filtering and pagination to CMS controller v1. The Filter can extend `AuditedVersionedModelFilter`. Filter with keyword search on title and body.
2.4Maxym BorodenkoMaxym Borodenko