Genesys Backend issueshttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues2021-03-05T13:40:47+01:00https://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/575KPI: group by storage2021-03-05T13:40:47+01:00Matija ObrezaKPI: group by storageI need to add an execution that will summarize the number of active accessions by genebank by storage type.
|institute.code|storage|count|
|--|--|--|
|ALB001|10|300|
|ALB001|20|310|
|ALB002|30|30|
The issue I'm facing is how to make a ...I need to add an execution that will summarize the number of active accessions by genebank by storage type.
|institute.code|storage|count|
|--|--|--|
|ALB001|10|300|
|ALB001|20|310|
|ALB002|30|30|
The issue I'm facing is how to make a HQL select that groups by elements of a collection:
```sql
select ?, count(acce) from accession a inner join a.acce acce
group by a.institute.code, acce.storage.elements() ??
```2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/574API Search2021-03-11T13:26:17+01:00Matija ObrezaAPI SearchPlease implement a */api/v1/search* endpoint that will do a full-text ES search for top 5 hits for each of the indexed collections (accession, activitypost, faoinstitute).
The response should be a `Map<String, ?>` with the classname as ...Please implement a */api/v1/search* endpoint that will do a full-text ES search for top 5 hits for each of the indexed collections (accession, activitypost, faoinstitute).
The response should be a `Map<String, ?>` with the classname as key and the value as a `Page` with the top 5 search results (`totalElements`, etc.)
Do not use or generate a `filterCode`.
Extra points if the search can be made with a single call to ES.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/573Spring 5.3.3 and CORS Headers2021-02-10T20:28:56+01:00Matija ObrezaSpring 5.3.3 and CORS HeadersWith Spring v5.3.3 in the API calls from UI stop working -- the CORS request fails. I figure our OAuthClient needs updating.
Reverting to 5.2.4.RELEASE does not have the same issue:
```xml
- <spring.framework.version>5.3.3</spring.fra...With Spring v5.3.3 in the API calls from UI stop working -- the CORS request fails. I figure our OAuthClient needs updating.
Reverting to 5.2.4.RELEASE does not have the same issue:
```xml
- <spring.framework.version>5.3.3</spring.framework.version>
+ <spring.framework.version>5.2.4.RELEASE</spring.framework.version>
```2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/572Find similar accessions2021-02-05T13:57:02+01:00Matija ObrezaFind similar accessionsImplement a new component with a method that returns a list of accessions "similar" to the input `Accession`:
```java
public List<Hit<Accession>> findSimilar(final Accession input)
```Implement a new component with a method that returns a list of accessions "similar" to the input `Accession`:
```java
public List<Hit<Accession>> findSimilar(final Accession input)
```2.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/571Taxonomy2 taxonNameHtml2021-02-05T19:26:14+01:00Matija ObrezaTaxonomy2 taxonNameHtmlWhen displaying scientific names we use *{ genus } {species}* (`<i>Gossypium herbaceum</i> L.`).
If `species` is `'sp.'` or `'spp.'` it must not use `<i>`.
The UI component uses `Taxonomy2.taxonNameHtml`. Please update so that `taxonNa...When displaying scientific names we use *{ genus } {species}* (`<i>Gossypium herbaceum</i> L.`).
If `species` is `'sp.'` or `'spp.'` it must not use `<i>`.
The UI component uses `Taxonomy2.taxonNameHtml`. Please update so that `taxonNameHtml` will properly format these cases.
Need an Admin tool that will recalculate `taxonNameHtml`.
Sourced from genesys-pgr/genesys-ui#7882.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/570CMS content with publishDate2021-02-12T12:10:47+01:00Matija ObrezaCMS content with publishDateExtend `ActivityPost` and `Article` with new `Date publishDate` and `Date expirationDate`.
The `publishDate` controls the date and time when a CMS item starts to be publicly visible. If `publishDate == null`, the contents is not yet rea...Extend `ActivityPost` and `Article` with new `Date publishDate` and `Date expirationDate`.
The `publishDate` controls the date and time when a CMS item starts to be publicly visible. If `publishDate == null`, the contents is not yet ready for publication and must not show up in `list()` or `get()` calls. Content managers can still access them in the Dashboard.
The `expirationDate` controls the date and time when a **published** CMS item is no longer publicly accessible. If `expirationDate == null` the item has no expiration date. If `expirationDate < now()` the item must not be included in `list()` or `get()` calls.
The `publishDate` should be serialized in `JsonViews.Public`. `expirationDate` is only serialized with `JsonViews.Protected`.
## liquibase
`publishDate` for existing items should be set to `createdDate`. `expirationDate` must stay `null`.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/569CMS content with attachments2021-02-03T16:25:43+01:00Matija ObrezaCMS content with attachments`Article`s and `ActivityPost`s now support HTML content and we can insert images by uploading directly to Repository.
I would like to be able to copy-paste an image or file directly in the Editor and it should be uploaded to `/content/(...`Article`s and `ActivityPost`s now support HTML content and we can insert images by uploading directly to Repository.
I would like to be able to copy-paste an image or file directly in the Editor and it should be uploaded to `/content/(articles|news)/{article.id}/` folder (image gallery supported).
If the article/post is removed, this folder and its contents can also be removed.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/568MCPD Excel export2021-01-29T10:54:32+01:00Matija ObrezaMCPD Excel exportOrganize columns sensibly!
Add dataproviderid.Organize columns sensibly!
Add dataproviderid.2.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/567Descriptor terms2021-01-28T13:11:23+01:00Matija ObrezaDescriptor termsI noticed that we somehow allow duplicate codes for one Descriptor:
![image](/uploads/92ac128c1547e1d9697f20dadc56da9f/image.png)
https://www.genesys-pgr.org/descriptors/d678ec0b-ea09-45b8-be90-682901b3656d
Duplicate codes **for one D...I noticed that we somehow allow duplicate codes for one Descriptor:
![image](/uploads/92ac128c1547e1d9697f20dadc56da9f/image.png)
https://www.genesys-pgr.org/descriptors/d678ec0b-ea09-45b8-be90-682901b3656d
Duplicate codes **for one Descriptor** must not be allowed.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/566ES aggregation by multiple terms2021-03-11T13:26:17+01:00Matija ObrezaES aggregation by multiple termsPlease add a Junit test that will query the ES for `count(species)`, aggregated by `genus`, `species` -- an aggregation of `species` within `genus`. See https://www.elastic.co/guide/en/elasticsearch/reference/6.8/search-aggregations-buck...Please add a Junit test that will query the ES for `count(species)`, aggregated by `genus`, `species` -- an aggregation of `species` within `genus`. See https://www.elastic.co/guide/en/elasticsearch/reference/6.8/search-aggregations-bucket-terms-aggregation.html
Use JUnit to print ES query and then apply it to your indexed ES.
```json
GET /_search
{
"aggs" : {
"genus" : {
"terms" : {
"field" : "genus",
// "order" : { "playback_stats.max" : "desc" }
},
"aggs" : {
// "playback_stats" : { "stats" : { "field" : "play_count" } }
"species": { "terms": { "field": "species" } },
"aggs" : {
// "playback_stats" : { "stats" : { "field" : "play_count" } }
"origin.iso3166": { "terms": { "field": "origin.iso3166" } }
}
}
}
}
}
```
The result should be something like:
|`genus`|`species`|count|
|-----|--|-----|
|Arabidopsis| Arabidopsis thaliana| 686,602 |
|Hordeum | Hordeum vulgare | 235,395|
|Hordeum | Hordeum Hordeum vulgare | 17,788|
|Hordeum | ... | . |
|Triticum | Triticum aestivum | 332,060 |
|Triticum | Triticum turgidum | 73,364 |
|Triticum | Triticum triticum aestivum |29,196|
How can we control the number of results?
For genesys-pgr/genesys-ui#7822.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/565Knowledge management backend2021-03-14T12:16:13+01:00Matija ObrezaKnowledge management backendFor the next phase of Genesys we need to better document the meaning of terms, concepts and how they relate.
We can (and will) use data and metadata that exists in the Genesys database schema.
We want to be able to document the concepts ...For the next phase of Genesys we need to better document the meaning of terms, concepts and how they relate.
We can (and will) use data and metadata that exists in the Genesys database schema.
We want to be able to document the concepts using [SKOS](https://www.w3.org/TR/skos-primer/). Also see https://www.obitko.com/tutorials/ontologies-semantic-web/ontologies.html.
While following SKOS concepts, we will initially record Genesys terms in the database not RDF store.
## Background
Multiple LOD (linked open data) resources contain data highly relevant to understanding and defining Genesys data.
Many of these resources provide not only definitions, but also translations.
## Data model
The data modelling of a `Concept` follows SKOS nomenclature.
```json
{
"id": "foobar1",
"prefLabel": {
"_": "Default label",
"en": "Something else",
"de": "German label"
},
"altLabel": { ... },
"definition": { ... },
"sameAs" : "https://...",
}
```
## RDF engine
**And RDF engine would allow for making inferences about the data, but it appears that we do not yet have a need for that.**
https://jena.apache.org appears to be our best match.
In production, Jena (and whatever is needed) will run as a separate Docker container (just like Amphibian) and will be responsible
for its data (volume). TBD will most likely be the most appropriate engine as we don't want to overload the
Genesys accession database with this data.
Genesys will provide API endpoints to query and interact with the RDF engine.
In this first phase, we want to be able to
1. Add a new record to the RDF store
1. List (all) records existing in the store
1. Retrieve the details from the storeMatija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/564taxonomy.grinCurrentName for isNull2021-01-21T09:40:51+01:00Matija Obrezataxonomy.grinCurrentName for isNullWhile exploring *Medicago sativa* Overviews I came across:
> `Internal Server Error Error accessing field taxonomy.grinCurrentName for isNull() in class org.genesys2.server.model.genesys.QAccession`
1. https://www.genesys-pgr.org/a/ove...While exploring *Medicago sativa* Overviews I came across:
> `Internal Server Error Error accessing field taxonomy.grinCurrentName for isNull() in class org.genesys2.server.model.genesys.QAccession`
1. https://www.genesys-pgr.org/a/overview/v20ROxZzprX
1. Click on "Not specified" in **Matched GRIN taxon** or **Current GRIN taxon**2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/563Upgrade app-blocks, file-repository to 3.02021-03-11T13:26:17+01:00Matija ObrezaUpgrade app-blocks, file-repository to 3.0Releases 3.0-SNAPSHOT are built with Java 11. Please update app-blocks and file-repo and test.Releases 3.0-SNAPSHOT are built with Java 11. Please update app-blocks and file-repo and test.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/562FTP Service not functional2020-12-18T11:38:52+01:00Matija ObrezaFTP Service not functionalUsers are reporting problems accessing the Genesys FTP service:
![image](/uploads/e3c37da09859489f877c2e72fd777676/image.png)
```
13:55:01 Status: Logged in
13:55:01 Trace: Measured latency of 86 ms
13:55:01 Status: ...Users are reporting problems accessing the Genesys FTP service:
![image](/uploads/e3c37da09859489f877c2e72fd777676/image.png)
```
13:55:01 Status: Logged in
13:55:01 Trace: Measured latency of 86 ms
13:55:01 Status: Retrieving directory listing...
13:55:01 Command: PWD
13:55:01 Response: 257 "/" is current directory.
13:55:01 Command: TYPE I
13:55:01 Response: 200 Command TYPE okay.
13:55:01 Command: PASV
13:55:01 Response: 227 Entering Passive Mode (52,58,138,164,178,66)
13:55:01 Trace: Binding data connection source IP to control connection source IP 192.168.14.23
13:55:01 Trace: Trying to resume existing TLS session.
13:55:01 Command: MLSD
13:55:01 Response: 150 File status okay; about to open data connection.
13:55:01 Trace: TLS Handshake successful
13:55:01 Trace: TLS Session resumed
13:55:01 Trace: Protocol: TLS1.3, Key exchange: unknown, Cipher: AES-256-GCM, MAC: AEAD
13:55:01 Error: Received TLS alert from the server: User canceled (90)
13:55:01 Error: Could not read from transfer socket: ECONNABORTED - Connection aborted
13:55:01 Response: 226 Closing data connection.
13:55:01 Error: Failed to retrieve directory listing
13:56:01 Status: Connection closed by server
```
The same happens when trying to connect to local Genesys:
```
# Enable FTP on port 8021
repository.ftp.port=8021
```
```
57:57 Command: PASV
13:57:57 Response: 227 Entering Passive Mode (127,0,0,1,9,7)
13:57:57 Trace: Binding data connection source IP to control connection source IP 127.0.0.1
13:57:57 Trace: Trying to resume existing TLS session.
13:57:57 Command: MLSD
13:57:57 Response: 150 File status okay; about to open data connection.
13:57:57 Trace: TLS Handshake successful
13:57:57 Trace: TLS Session resumed
13:57:57 Trace: Protocol: TLS1.3, Key exchange: unknown, Cipher: AES-256-GCM, MAC: AEAD
13:57:57 Error: Received TLS alert from the server: User canceled (90)
13:57:57 Error: Could not read from transfer socket: ECONNABORTED - Connection aborted
13:57:57 Response: 226 Closing data connection.
13:57:57 Error: Failed to retrieve directory listing
```2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/561SID autocomplete and display2020-12-17T16:29:35+01:00Matija ObrezaSID autocomplete and displayI hit an issue when using the *Permission Editor* in Genesys UI where a `Partner` and `OAuthClient` use the same "name".
Please also implement this change in grin-global-server.
## Autocomplete
The SID autocomplete method in `Permissi...I hit an issue when using the *Permission Editor* in Genesys UI where a `Partner` and `OAuthClient` use the same "name".
Please also implement this change in grin-global-server.
## Autocomplete
The SID autocomplete method in `PermissionController` (v1, v2?) uses `Map<String, Long>` to return "names" mapped to their `AclSid#id` field. This means that if two entries use the same "name" (`client.getTitle()`, `role.name()`, `partner.getShortName()`, ...) the only the last one will be included in the `Map`.
It would be better if the map returned was `Map<Long, String>`, mapping `AclSid#id` to a "name" since `id` is guaranteed to be unique.
A new UI ticket needs to be created to respect this change.
## Display
This is a potential information leak.
The method to list permissions at */permissions/{aclObjectIdentityId}* uses `@JsonView(JsonViews.Minimal.class)`, but `OAuthClient` does not declare `JsonViews.Protected` or `.Internal` for many of its fields and getters.
Please add `@JsonView(JsonViews.Protected.class)` to `registeredRedirectUri`, `authorizedGrantTypes`, `authorities`, `allowedOrigins`, `autoApproveScopes` and `publicRecaptchaKey`.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/560Hazelcast ClassLoader2020-12-08T13:14:54+01:00Matija ObrezaHazelcast ClassLoaderAmal reported an error when uploading accession refs to a dataset:
`com.hazelcast.nio.serialization.HazelcastSerializationException: java.lang.ClassNotFoundException: org.hibernate.cache.spi.support.AbstractReadWriteAccess$Item`
![imag...Amal reported an error when uploading accession refs to a dataset:
`com.hazelcast.nio.serialization.HazelcastSerializationException: java.lang.ClassNotFoundException: org.hibernate.cache.spi.support.AbstractReadWriteAccess$Item`
![image](/uploads/135aa640d89b7a80b5b88280b8c054d1/image.png)
I can see the class in question in `hibernate-core-5.4.14.Final.jar`.
Internet says this is related to classloader configuration.
We also have
```
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.springframework.util.ReflectionUtils (file:/Users/mobreza/.m2/repository/org/springframework/spring-core/5.2.4.RELEASE/spring-core-5.2.4.RELEASE.jar) to constructor java.util.Calendar()
WARNING: Please consider reporting this to the maintainers of org.springframework.util.ReflectionUtils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
```2.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/559ApiInfo: subset and dataset counts2020-12-07T15:31:32+01:00Matija ObrezaApiInfo: subset and dataset countsapi-info call should return the number of `current` and **PUBLISHED** datasets and subsets.
It now returns the count of all published records (including non-current).api-info call should return the number of `current` and **PUBLISHED** datasets and subsets.
It now returns the count of all published records (including non-current).2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/558ES: searching aliases2020-12-01T13:09:54+01:00Matija ObrezaES: searching aliaseshttps://www.genesys-pgr.org/10.18730/HTSH8 has **Nacional salvadoreƱo** (as `accessionName`) but it doesn't show up in full-text search: https://www.genesys-pgr.org/a/v2AVXO4xaPK
Maybe grin-global/grin-global-server!121 helps?https://www.genesys-pgr.org/10.18730/HTSH8 has **Nacional salvadoreƱo** (as `accessionName`) but it doesn't show up in full-text search: https://www.genesys-pgr.org/a/v2AVXO4xaPK
Maybe grin-global/grin-global-server!121 helps?2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/557ApiInfo: add frontendUrl2020-11-26T14:11:01+01:00Matija ObrezaApiInfo: add frontendUrlPlease add `publicWebsiteUrl` property to `ApiInfo` populated from `frontend.url` prop.
```properties
frontend.url=https://www.genesys-pgr.org
```
This will allow us to link from Embedded genesys to actual Genesys website.Please add `publicWebsiteUrl` property to `ApiInfo` populated from `frontend.url` prop.
```properties
frontend.url=https://www.genesys-pgr.org
```
This will allow us to link from Embedded genesys to actual Genesys website.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/556OAuth token repository2023-09-12T10:09:24+02:00Matija ObrezaOAuth token repositoryWhen we introduced JWT, we stopped using the `TokenRepository` and validating tokens on the server.
At the moment, an OAuth JWT token cannot be revoked -- the token is valid until it expires and the expiration date is encoded in the tok...When we introduced JWT, we stopped using the `TokenRepository` and validating tokens on the server.
At the moment, an OAuth JWT token cannot be revoked -- the token is valid until it expires and the expiration date is encoded in the token itself.
I think we need to be able to revoke OAuth tokens.
@mborodenko what do you think?Artem HrybeniukArtem Hrybeniuk