Genesys Backend issueshttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues2021-01-28T13:11:23+01:00https://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/567Descriptor terms2021-01-28T13:11:23+01:00Matija ObrezaDescriptor termsI noticed that we somehow allow duplicate codes for one Descriptor:
![image](/uploads/92ac128c1547e1d9697f20dadc56da9f/image.png)
https://www.genesys-pgr.org/descriptors/d678ec0b-ea09-45b8-be90-682901b3656d
Duplicate codes **for one D...I noticed that we somehow allow duplicate codes for one Descriptor:
![image](/uploads/92ac128c1547e1d9697f20dadc56da9f/image.png)
https://www.genesys-pgr.org/descriptors/d678ec0b-ea09-45b8-be90-682901b3656d
Duplicate codes **for one Descriptor** must not be allowed.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/568MCPD Excel export2021-01-29T10:54:32+01:00Matija ObrezaMCPD Excel exportOrganize columns sensibly!
Add dataproviderid.Organize columns sensibly!
Add dataproviderid.2.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/569CMS content with attachments2021-02-03T16:25:43+01:00Matija ObrezaCMS content with attachments`Article`s and `ActivityPost`s now support HTML content and we can insert images by uploading directly to Repository.
I would like to be able to copy-paste an image or file directly in the Editor and it should be uploaded to `/content/(...`Article`s and `ActivityPost`s now support HTML content and we can insert images by uploading directly to Repository.
I would like to be able to copy-paste an image or file directly in the Editor and it should be uploaded to `/content/(articles|news)/{article.id}/` folder (image gallery supported).
If the article/post is removed, this folder and its contents can also be removed.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/570CMS content with publishDate2021-02-12T12:10:47+01:00Matija ObrezaCMS content with publishDateExtend `ActivityPost` and `Article` with new `Date publishDate` and `Date expirationDate`.
The `publishDate` controls the date and time when a CMS item starts to be publicly visible. If `publishDate == null`, the contents is not yet rea...Extend `ActivityPost` and `Article` with new `Date publishDate` and `Date expirationDate`.
The `publishDate` controls the date and time when a CMS item starts to be publicly visible. If `publishDate == null`, the contents is not yet ready for publication and must not show up in `list()` or `get()` calls. Content managers can still access them in the Dashboard.
The `expirationDate` controls the date and time when a **published** CMS item is no longer publicly accessible. If `expirationDate == null` the item has no expiration date. If `expirationDate < now()` the item must not be included in `list()` or `get()` calls.
The `publishDate` should be serialized in `JsonViews.Public`. `expirationDate` is only serialized with `JsonViews.Protected`.
## liquibase
`publishDate` for existing items should be set to `createdDate`. `expirationDate` must stay `null`.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/571Taxonomy2 taxonNameHtml2021-02-05T19:26:14+01:00Matija ObrezaTaxonomy2 taxonNameHtmlWhen displaying scientific names we use *{ genus } {species}* (`<i>Gossypium herbaceum</i> L.`).
If `species` is `'sp.'` or `'spp.'` it must not use `<i>`.
The UI component uses `Taxonomy2.taxonNameHtml`. Please update so that `taxonNa...When displaying scientific names we use *{ genus } {species}* (`<i>Gossypium herbaceum</i> L.`).
If `species` is `'sp.'` or `'spp.'` it must not use `<i>`.
The UI component uses `Taxonomy2.taxonNameHtml`. Please update so that `taxonNameHtml` will properly format these cases.
Need an Admin tool that will recalculate `taxonNameHtml`.
Sourced from genesys-pgr/genesys-ui#7882.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/572Find similar accessions2021-02-05T13:57:02+01:00Matija ObrezaFind similar accessionsImplement a new component with a method that returns a list of accessions "similar" to the input `Accession`:
```java
public List<Hit<Accession>> findSimilar(final Accession input)
```Implement a new component with a method that returns a list of accessions "similar" to the input `Accession`:
```java
public List<Hit<Accession>> findSimilar(final Accession input)
```2.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/573Spring 5.3.3 and CORS Headers2021-02-10T20:28:56+01:00Matija ObrezaSpring 5.3.3 and CORS HeadersWith Spring v5.3.3 in the API calls from UI stop working -- the CORS request fails. I figure our OAuthClient needs updating.
Reverting to 5.2.4.RELEASE does not have the same issue:
```xml
- <spring.framework.version>5.3.3</spring.fra...With Spring v5.3.3 in the API calls from UI stop working -- the CORS request fails. I figure our OAuthClient needs updating.
Reverting to 5.2.4.RELEASE does not have the same issue:
```xml
- <spring.framework.version>5.3.3</spring.framework.version>
+ <spring.framework.version>5.2.4.RELEASE</spring.framework.version>
```2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/574API Search2021-03-11T13:26:17+01:00Matija ObrezaAPI SearchPlease implement a */api/v1/search* endpoint that will do a full-text ES search for top 5 hits for each of the indexed collections (accession, activitypost, faoinstitute).
The response should be a `Map<String, ?>` with the classname as ...Please implement a */api/v1/search* endpoint that will do a full-text ES search for top 5 hits for each of the indexed collections (accession, activitypost, faoinstitute).
The response should be a `Map<String, ?>` with the classname as key and the value as a `Page` with the top 5 search results (`totalElements`, etc.)
Do not use or generate a `filterCode`.
Extra points if the search can be made with a single call to ES.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/575KPI: group by storage2021-03-05T13:40:47+01:00Matija ObrezaKPI: group by storageI need to add an execution that will summarize the number of active accessions by genebank by storage type.
|institute.code|storage|count|
|--|--|--|
|ALB001|10|300|
|ALB001|20|310|
|ALB002|30|30|
The issue I'm facing is how to make a ...I need to add an execution that will summarize the number of active accessions by genebank by storage type.
|institute.code|storage|count|
|--|--|--|
|ALB001|10|300|
|ALB001|20|310|
|ALB002|30|30|
The issue I'm facing is how to make a HQL select that groups by elements of a collection:
```sql
select ?, count(acce) from accession a inner join a.acce acce
group by a.institute.code, acce.storage.elements() ??
```2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/576Updates from GG-CE Server2021-03-11T16:27:15+01:00Matija ObrezaUpdates from GG-CE ServerBring the following updates from gg-ce-server:
1. Prevent reindexing 18814608 Maxym Borodenko 10 Mar 2021 at 09:40
1. ShortFilterService: throw proper exception a7fff53c 5 Mar 2021 at 14:10Bring the following updates from gg-ce-server:
1. Prevent reindexing 18814608 Maxym Borodenko 10 Mar 2021 at 09:40
1. ShortFilterService: throw proper exception a7fff53c 5 Mar 2021 at 14:102021.1Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/577Release versioning2021-03-12T08:07:24+01:00Matija ObrezaRelease versioningAdopt the same versioning as GG-CE.Adopt the same versioning as GG-CE.2021.0Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/578Dataset analyzer2021-03-31T11:12:13+02:00Matija ObrezaDataset analyzerOur work in genesys-pgr/genesys-ui#338 assumes that datasets are clean and ready for final mapping to descriptors. That's not usually the case and data commonly needs more cleaning.
In this ticket, we add API endpoints to `AmphibianCont...Our work in genesys-pgr/genesys-ui#338 assumes that datasets are clean and ready for final mapping to descriptors. That's not usually the case and data commonly needs more cleaning.
In this ticket, we add API endpoints to `AmphibianController` to upload and analyze any dataset.
*/api/v1/amphibian/upload* (requires VETTEDUSER, ADMINISTRATOR): Upload a file to file repository to path */amphibian/* and initiate Amphibian preview.
Add */api/v1/amphibian/stats/{uuid}/{sheet}* to return (for selected columns `fields` and offset `startRow`):
- count of values
- count of distinct values
- list of max 100 distinct values
## The UI
The UI will allow the user to upload a data file and will then render a UI similar to:
![image](/uploads/9b730c815f71828a58fd2271f41d11ab/image.png)2021.1Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/579Find similar descriptors2021-03-18T13:42:35+01:00Matija ObrezaFind similar descriptorsAdd a function that for selected input `Descriptor` (in `@RequestBody`) will return a ranked list of similar existing `Descriptor`s in the database.
Use a similar approach as for accession similarity, but:
1. `title`, `description` str...Add a function that for selected input `Descriptor` (in `@RequestBody`) will return a ranked list of similar existing `Descriptor`s in the database.
Use a similar approach as for accession similarity, but:
1. `title`, `description` string similarity
1. `columnName`, `uom`
1. `category`
1. `crop`
1. `minValue`, `maxValue` can be ignored
1. compare `descriptor.terms` by `code`, `title`, `description`
1. find the codes of descriptor **A** that match a code in descriptor **B**
1. each match adds points to the score2021.1Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/581Logging downloads2021-03-22T11:23:02+01:00Matija ObrezaLogging downloadsI'd like to add a KPI for the number of downloads from Genesys. Since KPI's are based on JPA, we need something that will allow us to track these:
```java
class DownloadLog implements AclAwareModel {
Date date;
String url;
// These m...I'd like to add a KPI for the number of downloads from Genesys. Since KPI's are based on JPA, we need something that will allow us to track these:
```java
class DownloadLog implements AclAwareModel {
Date date;
String url;
// These may be useful
String entity;
String filterCode;
String clientIp;
}
```
The KPI would count the downloads by `url`, `entity` and `filterCode`.
If the download is canceled, the `DownloadLog` record should not be added.2021.1Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/582JaCoCo and SpotBugs2021-04-12T10:41:57+02:00Matija ObrezaJaCoCo and SpotBugsAdd JaCoCo and SpotBugs to `pom.xml` and update the pipeline. Use GG-CE server and app-blocks projects as examples.Add JaCoCo and SpotBugs to `pom.xml` and update the pipeline. Use GG-CE server and app-blocks projects as examples.2021.2Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/583Release 2021.12021-04-09T07:39:41+02:00Matija ObrezaRelease 2021.12021.1Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/584Cleanup: Drop API v02023-09-04T12:30:44+02:00Matija ObrezaCleanup: Drop API v0I don't think we use API **/api/v0** calls anywhere, the only place could be our admin MVC.
I believe that can be dropped.I don't think we use API **/api/v0** calls anywhere, the only place could be our admin MVC.
I believe that can be dropped.Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/586Performance with Accessions /list as CSV2021-09-20T10:49:38+02:00Matija ObrezaPerformance with Accessions /list as CSVUsing the CSV format to retrieve large selections (pages) using the /list endpoint appear to be very slow (see https://gitlab.croptrust.org/genesys-pgr/genesysr/-/issues/2#note_53485).
We should explore two things:
1. Adding support to...Using the CSV format to retrieve large selections (pages) using the /list endpoint appear to be very slow (see https://gitlab.croptrust.org/genesys-pgr/genesysr/-/issues/2#note_53485).
We should explore two things:
1. Adding support to `exclude` fields (we now have `select` to include fields)
1. Not serializing things with `flatMap` that are not selected.
The response is now very broad and includes very deeply nested entities. We should explore what can be excluded.2021.2Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/587Download log with @CreatedBy2021-04-16T09:25:10+02:00Matija ObrezaDownload log with @CreatedByThe logging works nicely in production:
| id | controller | method | url | date | clientIp |
|----|----|----|----|...The logging works nicely in production:
| id | controller | method | url | date | clientIp |
|----|----|----|----|----|----|
| 1 | DatasetController | downloadDatasetFile | /api/v1/dataset/b9c8e4db-d6b1-44a8-b1a3-307c1f496daf/download/131077 | 2021-04-09 02:48:33.000000 | 181.xxxx |
| 2 | DatasetController | downloadDatasetFile | /api/v1/dataset/f6cd4e85-7e44-4429-b00d-9d9eb99ea0bb/download/3244033 | 2021-04-09 06:46:56.000000 | 154.xxxx |
| 3 | DatasetController | downloadDatasetFile | /api/v1/dataset/929a273d-7882-43eb-8b1a-86032cbeb892/download/3112961 | 2021-04-09 08:46:07.000000 | 154.xxxx |
It would be very useful if `createdBy` was also included in the database.2021.2Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/588Session cookies on /api2021-05-19T08:14:19+02:00Matija ObrezaSession cookies on /apiA normal HTTP GET request to */api* should not generate new cookies:
![image](/uploads/7eefa65cf4300f6ba907059f3a55eb48/image.png)
A request to */api/v1/geo/iso3166/decode?locale=en* is quite frequent from the UI and it includes `JSESS...A normal HTTP GET request to */api* should not generate new cookies:
![image](/uploads/7eefa65cf4300f6ba907059f3a55eb48/image.png)
A request to */api/v1/geo/iso3166/decode?locale=en* is quite frequent from the UI and it includes `JSESSIONID` and `hz-session-id` in the request:
```
Cookie: GENESYS_sandbox.genesys-pgr.org=eyJhbGciOixxxx.eyJleHAixxx.e9piZqWpIxxxx; G_ENABLED_IDPS=google; G_AUTHUSER_H=1; JSESSIONID=node063vg9xhbqpul1z20x2tr6ov683.node0; hz-session-id=HZ72872DD6A59743BCBF6E66EAD733E702
```
Both relevant cookies `JSESSIONID` and `hz-session-id` are set to `SameSite` (i.e. api.sandbox.genesys-pgr.org).
They should be completely ignored for requests to */api*. It's even stranger that there are **two `hz-session-id` cookies** in the response?2021.2Maxym BorodenkoMaxym Borodenko