Genesys Backend issueshttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues2021-03-11T13:26:17+01:00https://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/574API Search2021-03-11T13:26:17+01:00Matija ObrezaAPI SearchPlease implement a */api/v1/search* endpoint that will do a full-text ES search for top 5 hits for each of the indexed collections (accession, activitypost, faoinstitute).
The response should be a `Map<String, ?>` with the classname as ...Please implement a */api/v1/search* endpoint that will do a full-text ES search for top 5 hits for each of the indexed collections (accession, activitypost, faoinstitute).
The response should be a `Map<String, ?>` with the classname as key and the value as a `Page` with the top 5 search results (`totalElements`, etc.)
Do not use or generate a `filterCode`.
Extra points if the search can be made with a single call to ES.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/566ES aggregation by multiple terms2021-03-11T13:26:17+01:00Matija ObrezaES aggregation by multiple termsPlease add a Junit test that will query the ES for `count(species)`, aggregated by `genus`, `species` -- an aggregation of `species` within `genus`. See https://www.elastic.co/guide/en/elasticsearch/reference/6.8/search-aggregations-buck...Please add a Junit test that will query the ES for `count(species)`, aggregated by `genus`, `species` -- an aggregation of `species` within `genus`. See https://www.elastic.co/guide/en/elasticsearch/reference/6.8/search-aggregations-bucket-terms-aggregation.html
Use JUnit to print ES query and then apply it to your indexed ES.
```json
GET /_search
{
"aggs" : {
"genus" : {
"terms" : {
"field" : "genus",
// "order" : { "playback_stats.max" : "desc" }
},
"aggs" : {
// "playback_stats" : { "stats" : { "field" : "play_count" } }
"species": { "terms": { "field": "species" } },
"aggs" : {
// "playback_stats" : { "stats" : { "field" : "play_count" } }
"origin.iso3166": { "terms": { "field": "origin.iso3166" } }
}
}
}
}
}
```
The result should be something like:
|`genus`|`species`|count|
|-----|--|-----|
|Arabidopsis| Arabidopsis thaliana| 686,602 |
|Hordeum | Hordeum vulgare | 235,395|
|Hordeum | Hordeum Hordeum vulgare | 17,788|
|Hordeum | ... | . |
|Triticum | Triticum aestivum | 332,060 |
|Triticum | Triticum turgidum | 73,364 |
|Triticum | Triticum triticum aestivum |29,196|
How can we control the number of results?
For genesys-pgr/genesys-ui#7822.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/563Upgrade app-blocks, file-repository to 3.02021-03-11T13:26:17+01:00Matija ObrezaUpgrade app-blocks, file-repository to 3.0Releases 3.0-SNAPSHOT are built with Java 11. Please update app-blocks and file-repo and test.Releases 3.0-SNAPSHOT are built with Java 11. Please update app-blocks and file-repo and test.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/522Number of subsets and datasets2021-03-11T13:26:17+01:00Matija ObrezaNumber of subsets and datasetsFor each `Accession` it would be useful to have quick access to the number of associated **published** subsets and datasets.
Extend `AccessionId` with two new properties `long subsets = 0, datasets = 0`.
When a subset/dataset is publis...For each `Accession` it would be useful to have quick access to the number of associated **published** subsets and datasets.
Extend `AccessionId` with two new properties `long subsets = 0, datasets = 0`.
When a subset/dataset is published, the count for accessions in the list is increased by 1. When it is unpublished, it is decreased by 1.
This must be a very fast SQL update.
Updating these counters should not touch the lastModifiedDate or version fields, but it should schedule a reindex of accessions affected.
Implement an admin function that will:
1. Reset the counters to 0.
1. Scan for subsets and increase the counts
1. Scan for datasets and increase the counts.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/268Add picture available column in accessions browser2021-03-11T13:26:17+01:00Nora Castaneda-AlvarezAdd picture available column in accessions browserPlease add a "Picture available" column when browsing for accessions (https://sandbox.genesys-pgr.org/explore).
Accessions with pictures should be flagged with a "yes" or a small icon that indicates that pictures are available. Use this...Please add a "Picture available" column when browsing for accessions (https://sandbox.genesys-pgr.org/explore).
Accessions with pictures should be flagged with a "yes" or a small icon that indicates that pictures are available. Use this as example: https://sandbox.genesys-pgr.org/10.18730/9X2HE2.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/575KPI: group by storage2021-03-05T13:40:47+01:00Matija ObrezaKPI: group by storageI need to add an execution that will summarize the number of active accessions by genebank by storage type.
|institute.code|storage|count|
|--|--|--|
|ALB001|10|300|
|ALB001|20|310|
|ALB002|30|30|
The issue I'm facing is how to make a ...I need to add an execution that will summarize the number of active accessions by genebank by storage type.
|institute.code|storage|count|
|--|--|--|
|ALB001|10|300|
|ALB001|20|310|
|ALB002|30|30|
The issue I'm facing is how to make a HQL select that groups by elements of a collection:
```sql
select ?, count(acce) from accession a inner join a.acce acce
group by a.institute.code, acce.storage.elements() ??
```2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/542Synonymy search2021-02-15T10:00:19+01:00Matija ObrezaSynonymy searchThe current TaxonomyFilter supports search by *current GRIN names*. Search for *Solanum lycopersicum* (a current name) includes *Solanum lycopersicum*, *Lycopersicon esculentum* and *Lycopersicon lycopersicum*.
Searching for *Lycopers...The current TaxonomyFilter supports search by *current GRIN names*. Search for *Solanum lycopersicum* (a current name) includes *Solanum lycopersicum*, *Lycopersicon esculentum* and *Lycopersicon lycopersicum*.
Searching for *Lycopersicon esculentum* on the other hand does not include *Solanum lycopersicum*.
Update `TaxonomyFilter` to extend the search:
1. Find all *Lycopersicon esculentum* in `Taxonomy2` and use their **currentTaxonomySpecies**
1. Find all *Lycopersicon esculentum* in GRIN taxonomy and use their **currentTaxonomySpecies**?2.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/492Detect accessions in Dataset file2021-02-15T09:59:23+01:00Matija ObrezaDetect accessions in Dataset fileData providers generally upload the Excel file with data in Step 2, and then upload accession references in step 3.
Can we add `detectAccessions` function that will use Amphibian to check for `DOI` and `ACCENUMB` headers and automatical...Data providers generally upload the Excel file with data in Step 2, and then upload accession references in step 3.
Can we add `detectAccessions` function that will use Amphibian to check for `DOI` and `ACCENUMB` headers and automatically register accession references?
The user would trigger this function in step 3. So they either copy-paste the list or autodetect it from the file.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/420Spatial data2021-02-15T09:56:52+01:00Matija ObrezaSpatial dataWe'd like to make spatial (polygon) queries for accessions in Genesys.
## Use case
https://www.genesys-pgr.org/explore/map has the **Select area** that allows user to select a rectangle (`min&max` for accession collecting site `latitu...We'd like to make spatial (polygon) queries for accessions in Genesys.
## Use case
https://www.genesys-pgr.org/explore/map has the **Select area** that allows user to select a rectangle (`min&max` for accession collecting site `latitude&longitude`). If we allow user to select a polygon (https://leafletjs.com/reference-1.4.0.html#polygon), can the API make a query to the database to find accessions where `accessionGeo` is **inside** the selected polygon?
- https://mariadb.com/kb/en/library/geographic-geometric-features/
- https://www.baeldung.com/hibernate-spatial
## Other cases
- `FaoInstitute` location has `lat&long`2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/370Case-sensitive Taxonomy2021-02-15T09:56:05+01:00Matija ObrezaCase-sensitive TaxonomyTaxonomy lookups and updates should be case sensitive:
> had the species authors in capital letters `((AUBL.) SW.)`. This was corrected in the updated data `((Aubl.) Sw.)`, but the author remains in capital letters in Genesys.Taxonomy lookups and updates should be case sensitive:
> had the species authors in capital letters `((AUBL.) SW.)`. This was corrected in the updated data `((Aubl.) Sw.)`, but the author remains in capital letters in Genesys.2.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/344Invalid incoming JSON2021-02-15T09:55:50+01:00Matija ObrezaInvalid incoming JSONCan we URLDecode the incoming string without breaking anything?
```java
2018-09-26T09:46:44.962667827Z 09:46:44,961 qtp485815673-3398 ERROR o.g.s.s.c.UserControllerAdvice:89 - Unexpected character ('+' (code 43)): was expecting double-q...Can we URLDecode the incoming string without breaking anything?
```java
2018-09-26T09:46:44.962667827Z 09:46:44,961 qtp485815673-3398 ERROR o.g.s.s.c.UserControllerAdvice:89 - Unexpected character ('+' (code 43)): was expecting double-quote to start field name
2018-09-26T09:46:44.962707242Z at [Source: {"taxonomy.genus":["Trifolium"],+"taxonomy.species":["sp."]}; line: 1, column: 34] on GET http://www.genesys-pgr.org/fr/explore
2018-09-26T09:46:44.962718755Z com.fasterxml.jackson.core.JsonParseException: Unexpected character ('+' (code 43)): was expecting double-quote to start field name
2018-09-26T09:46:44.962725480Z at [Source: {"taxonomy.genus":["Trifolium"],+"taxonomy.species":["sp."]}; line: 1, column: 34]
2018-09-26T09:46:44.962732556Z at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1581)
2018-09-26T09:46:44.962739545Z at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:533)
2018-09-26T09:46:44.962745777Z at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:462)
2018-09-26T09:46:44.962751962Z at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddName(ReaderBasedJsonParser.java:1502)
2018-09-26T09:46:44.962758374Z at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:624)
2018-09-26T09:46:44.962765522Z at org.genesys2.server.service.impl.FilterHandler$AppliedFilters$Deserializer.deserialize(FilterHandler.java:415)
2018-09-26T09:46:44.962772316Z at org.genesys2.server.service.impl.FilterHandler$AppliedFilters$Deserializer.deserialize(FilterHandler.java:244)
2018-09-26T09:46:44.962778623Z at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3736)
2018-09-26T09:46:44.962784635Z at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2726)
2018-09-26T09:46:44.962791177Z at org.genesys2.server.servlet.controller.ExplorerController.viewFiltered(ExplorerController.java:207)
```2.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/296Replace all @Query with DSL2021-02-15T09:55:34+01:00Matija ObrezaReplace all @Query with DSLThe "text" mode `@Query` syntax in JPA repositories is impossible to maintain as the data models are updated and changed.
Migrate all cases where `@Query` is used to QueryDSL. Use the following approach:
1. If the query is used from a ...The "text" mode `@Query` syntax in JPA repositories is impossible to maintain as the data models are updated and changed.
Migrate all cases where `@Query` is used to QueryDSL. Use the following approach:
1. If the query is used from a single service method, implement the query in the service method
1. If the query is used from multiple service methods, implement the query in a `...CustomImpl` and extend the repository with a `...Custom` interface2.5Vladyslava MokliakVladyslava Mokliakhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/271Test sandbox2021-02-15T09:54:45+01:00Matija ObrezaTest sandbox@akirichenko I ran your tester against the upgraded https://sandbox.genesys-pgr.org and things looked pretty much okay.
Because this was a major upgrade of the data model I'm not 100% convinced that everything works and behave as expect...@akirichenko I ran your tester against the upgraded https://sandbox.genesys-pgr.org and things looked pretty much okay.
Because this was a major upgrade of the data model I'm not 100% convinced that everything works and behave as expected.
Can you please test the Sandbox if things behave as they should?2.5Aleksandr KirichenkoAleksandr Kirichenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/265ORG page2021-02-15T09:54:10+01:00Matija ObrezaORG pagehttps://www.genesys-pgr.org/org/EURISCO is very slow, see if the query can be updated.https://www.genesys-pgr.org/org/EURISCO is very slow, see if the query can be updated.2.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/264Remove links from Overview2021-02-15T09:53:54+01:00Matija ObrezaRemove links from OverviewLinks to genera and countries in Overviews suggest that clicking the link would display the actual records.
For example https://www.genesys-pgr.org/project/CWR lists "Most represented Genera" and one would expect to see the 300 *Vicia* ...Links to genera and countries in Overviews suggest that clicking the link would display the actual records.
For example https://www.genesys-pgr.org/project/CWR lists "Most represented Genera" and one would expect to see the 300 *Vicia* accessions of the project when clicking the link.2.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/179Article templates2021-02-15T09:52:04+01:00Matija ObrezaArticle templatesSome existing **global** articles are actually templates in the system.
Allow for editing of `Article#template` flag only for `ContentService#getDefaultLanguage`. If the flag changes, update all articles with the same `slug` to the sta...Some existing **global** articles are actually templates in the system.
Allow for editing of `Article#template` flag only for `ContentService#getDefaultLanguage`. If the flag changes, update all articles with the same `slug` to the state as set for default language.
Indexing and deleting from ES index was taken care of in 50bc2ad69438f27ae876e587eb526005f10f296e.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/172"In Trust" descriptor2021-02-15T09:51:54+01:00Nora Castaneda-Alvarez"In Trust" descriptorDocument the `InTrust` descriptor, a Genesys extension to MCPD, at https://www.genesys-pgr.org/doc/0/basics#mcpd-genesysDocument the `InTrust` descriptor, a Genesys extension to MCPD, at https://www.genesys-pgr.org/doc/0/basics#mcpd-genesys2.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/165Document file management with FTP2021-02-15T09:51:31+01:00Matija ObrezaDocument file management with FTPAdd **asciidoc** on managing files with FTP.Add **asciidoc** on managing files with FTP.2.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/164Respect institute permissions2021-02-15T09:50:43+01:00Matija ObrezaRespect institute permissionsPermissions must be checked for navigation and files management through FTP.
Related to genesys-pgr/file-repository#22Permissions must be checked for navigation and files management through FTP.
Related to genesys-pgr/file-repository#222.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/570CMS content with publishDate2021-02-12T12:10:47+01:00Matija ObrezaCMS content with publishDateExtend `ActivityPost` and `Article` with new `Date publishDate` and `Date expirationDate`.
The `publishDate` controls the date and time when a CMS item starts to be publicly visible. If `publishDate == null`, the contents is not yet rea...Extend `ActivityPost` and `Article` with new `Date publishDate` and `Date expirationDate`.
The `publishDate` controls the date and time when a CMS item starts to be publicly visible. If `publishDate == null`, the contents is not yet ready for publication and must not show up in `list()` or `get()` calls. Content managers can still access them in the Dashboard.
The `expirationDate` controls the date and time when a **published** CMS item is no longer publicly accessible. If `expirationDate == null` the item has no expiration date. If `expirationDate < now()` the item must not be included in `list()` or `get()` calls.
The `publishDate` should be serialized in `JsonViews.Public`. `expirationDate` is only serialized with `JsonViews.Protected`.
## liquibase
`publishDate` for existing items should be set to `createdDate`. `expirationDate` must stay `null`.2.5Maxym BorodenkoMaxym Borodenko