Genesys Backend issueshttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues2019-09-16T20:38:44+02:00https://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/465Document migration from API v0 to v12019-09-16T20:38:44+02:00Matija ObrezaDocument migration from API v0 to v1......2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/461Geo Countries list2019-07-30T17:11:58+02:00Matija ObrezaGeo Countries listhttps://api.sandbox.genesys-pgr.org/api/v1/geo/countries returns a massive JSON. If we don't use `nameL` on the frontend, please don't include it in response to this call.https://api.sandbox.genesys-pgr.org/api/v1/geo/countries returns a massive JSON. If we don't use `nameL` on the frontend, please don't include it in response to this call.2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/458Requests for material without PID2019-08-02T10:51:04+02:00Matija ObrezaRequests for material without PIDGenesys now requires that all requests for material obtain the Easy-SMTA PID before relaying the request to the holding genebanks.
Requests for material can be "internal" (within the institute), meaning that no SMTA is required.
Add su...Genesys now requires that all requests for material obtain the Easy-SMTA PID before relaying the request to the holding genebanks.
Requests for material can be "internal" (within the institute), meaning that no SMTA is required.
Add support for **Internal requests** through Genesys; add an option "Internal request (no S/MTA required)" to Genesys and skip the Easy-SMTA checks that are currently required for the request to be relayed to the genebank.
2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/457FilterCode v22019-06-22T05:49:57+02:00Matija ObrezaFilterCode v2Filter encoding **version 1** uses a randomly generated UUID with `-` removed:
```java
private String generateShortNameV1() {
for (int i = 0; i < 10; i++) {
final String code = "v1" + UUID.randomUUID().toString().replace("-", "");
...Filter encoding **version 1** uses a randomly generated UUID with `-` removed:
```java
private String generateShortNameV1() {
for (int i = 0; i < 10; i++) {
final String code = "v1" + UUID.randomUUID().toString().replace("-", "");
// return shortName if unique
if (shortFilterRepository.findByCode(code) == null) {
return code;
}
}
throw new RuntimeException("Failed to generate a unique filters short name in several attempts");
}
```
This generates filter codes like `v1a37ad370d9ca4f62b3d9840094aafae9` and they are fairly long.
## Shorter filter codes
We can make use of https://hashids.org/java/ to generate shorter codes with a custom salt!
Initial idea is to use the current timestamp (long value, 4b), resulting in drastically smaller unique short filter codes.
```java
private String generateShortNameV2() {
Hashids hashids = new Hashids("this is my salt");
for (int i = 0; i < 10; i++) {
String hash = hashids.encode(System.currentTimeMillis());
final String code = "v2" + hash;
// return shortName if unique
if (shortFilterRepository.findByCode(code) == null) {
return code;
}
}
throw new RuntimeException("Failed to generate a unique filters short name in several attempts");
}
```
We could extend this to our use of UUIDs in URLs, and see if that makes a difference in URL `code` length.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/453Request failed when user try apply "Not specified" filter on the "Accession b...2019-07-14T21:12:35+02:00Kiril MorhalenkoRequest failed when user try apply "Not specified" filter on the "Accession browser" pageSteps to reproduce:
1. Open "https://sandbox.genesys-pgr.org/a/overview"
2. Apply "Not specified" filter from "Genus" block
Expected result:
Filter is applied and results are shown
Actual result:
> Error happened while processing requ...Steps to reproduce:
1. Open "https://sandbox.genesys-pgr.org/a/overview"
2. Apply "Not specified" filter from "Genus" block
Expected result:
Filter is applied and results are shown
Actual result:
> Error happened while processing request:
>
> "Request failed with status code 500 Error accessing field taxonomy.genus for isNull() in class org.genesys2.server.model.genesys.QAccession"
This bug reproduce with "Genys", "Type of Germplasm storage" and "Provenance of material" blocks
![Request_failed](/uploads/dfc0204d491abdbb29ca29a45790145c/Request_failed.gif)2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/452Descriptor image and language2019-06-18T08:13:31+02:00Matija ObrezaDescriptor image and languageAdd `private RepositoryImage image` and `@MaxLength(2) private String lang` to `Descriptor`. The image should be loaded in `lazyLoad(Descriptor)`.
Extend Descriptor API v1 with methods: `updateImage(Descriptor d, ...)` and `removeImage(...Add `private RepositoryImage image` and `@MaxLength(2) private String lang` to `Descriptor`. The image should be loaded in `lazyLoad(Descriptor)`.
Extend Descriptor API v1 with methods: `updateImage(Descriptor d, ...)` and `removeImage(Descriptor r)`. Ensure API supports `lang` property.
The descriptor image should be saved at repository path `/descriptor/{uuid.substring(0,3)}/{uuid}.{ext}`.
Ensure the `descriptor.image` is included in JSON response for Descriptor Lists.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/451Delete Metadata2019-06-05T04:47:54+02:00Matija ObrezaDelete MetadataAllow for removing old `org.genesys2.server.model.genesys.Metadata` (e.g. https://www.genesys-pgr.org/data/view/3673) and all related `ExperimentAccessionTrait`, `ExperimentTrait` for administrator in the JSP page.Allow for removing old `org.genesys2.server.model.genesys.Metadata` (e.g. https://www.genesys-pgr.org/data/view/3673) and all related `ExperimentAccessionTrait`, `ExperimentTrait` for administrator in the JSP page.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/448DownloadService with ES2019-05-21T11:06:17+02:00Matija ObrezaDownloadService with ESWhen using full text filtering with ES, the downloads are disabled because DownloadService (and AccessionProcessor) support JPA queries only. Refactor the code so that PccessionProcessor can use ES or JPA filters to generate downloads.When using full text filtering with ES, the downloads are disabled because DownloadService (and AccessionProcessor) support JPA queries only. Refactor the code so that PccessionProcessor can use ES or JPA filters to generate downloads.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/447ES reindexing using BasicFilter2019-05-22T18:46:16+02:00Matija ObrezaES reindexing using BasicFilterAt the moment Genesys allows for reindexing of selected records using JSON filters in admin pages of JSP.
Please add API v1 endpoint(s) to reindex selected records (as in JSP) and add it to UI.At the moment Genesys allows for reindexing of selected records using JSON filters in admin pages of JSP.
Please add API v1 endpoint(s) to reindex selected records (as in JSP) and add it to UI.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/446Better thumbnail quality2019-05-21T20:06:19+02:00Matija ObrezaBetter thumbnail qualityCurrent code to generate thumbnails does not produce good quality thumbs. See for example some of the crop images in the crops list page.
The issue is most visible with very high-quality source images, compare the generated thumbnail to...Current code to generate thumbnails does not produce good quality thumbs. See for example some of the crop images in the crops list page.
The issue is most visible with very high-quality source images, compare the generated thumbnail to the original image:
- https://beta.genesys-pgr.org/10.18730/H8F64
- https://beta.genesys-pgr.org/10.18730/H8PVV
- https://beta.genesys-pgr.org/10.18730/HTM8R2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/445Request API: remove request2019-05-15T23:56:17+02:00Matija ObrezaRequest API: remove requestRequest database contains records that are duplicates or test requests. Please add `remove` endpoint to Request API, only for ADMIN accounts.Request database contains records that are duplicates or test requests. Please add `remove` endpoint to Request API, only for ADMIN accounts.2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/444ACCENAME in AccessionData2019-05-21T09:38:33+02:00Matija ObrezaACCENAME in AccessionDataDifferent names of an accession are stored as `AccessionAlias` and one accession has a list of them.
See if we can add `getAccessionName` that returns the first alias of type `ACCENAME`. This, in theory, should force lazy loading the a...Different names of an accession are stored as `AccessionAlias` and one accession has a list of them.
See if we can add `getAccessionName` that returns the first alias of type `ACCENAME`. This, in theory, should force lazy loading the aliases list. But we don't need to change the database.
I think in the past we misused the `getAccessionName` to return `accessionNumber`. If we have any EL expressions in JSP that use `accessionName` they should be modified to `accessionNumber`.2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/443ES not containing all data2019-05-19T18:10:09+02:00Matija ObrezaES not containing all dataDatasets ES index does not list different data providers. See https://beta.genesys-pgr.org/datasets/overview.
Another issue is with updating only one part of the ES document: When the image count is updated to the database, we do not us...Datasets ES index does not list different data providers. See https://beta.genesys-pgr.org/datasets/overview.
Another issue is with updating only one part of the ES document: When the image count is updated to the database, we do not use the entity update so that the Auditor doesn't update `lastModifiedDate`. But this results in ES not being updated.
We uploaded images for 2 accessions at https://beta.genesys-pgr.org/a/overview/v16f1def008dbf46969dc325f668c676d1 and the Overview doesn't show data for these. Can we partially update an ES document with only `{ accessionCount: 2 }`?2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/440Inconsistent API response2019-05-06T11:34:33+02:00Matija ObrezaInconsistent API responseReported by @osavran, the descriptors sort property from response has this structure:
![Screenshot_from_2019-05-03_09-16-59](/uploads/0bfa68e143c0f780eb545e3a54065963/Screenshot_from_2019-05-03_09-16-59.png)
and for other modules:
![S...Reported by @osavran, the descriptors sort property from response has this structure:
![Screenshot_from_2019-05-03_09-16-59](/uploads/0bfa68e143c0f780eb545e3a54065963/Screenshot_from_2019-05-03_09-16-59.png)
and for other modules:
![Screenshot_from_2019-05-03_09-12-37](/uploads/339d1a3b521f6a138330af494b8e364a/Screenshot_from_2019-05-03_09-12-37.png)
`PaginationComponent` throw error "`pageObj.sort[0]` is undefined", descriptors action `updateRoute` also won't work. I think the same problem is in other components.
## Notes
This may have to do with difference between ES and JPA `Page<?>` responses.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/439Use ES for text queries2019-05-01T21:52:50+02:00Matija ObrezaUse ES for text queriesWhen user provides `_text` filter (keyword search) the backend should query ES for results (not the database).
We have `_text` filter in `DescriptorListFilter` for example. Add `_text` filter to `AccessionFilter`.
This requires adding ...When user provides `_text` filter (keyword search) the backend should query ES for results (not the database).
We have `_text` filter in `DescriptorListFilter` for example. Add `_text` filter to `AccessionFilter`.
This requires adding support for converting `_text` field to a full-text ES query (https://www.elastic.co/guide/en/elasticsearch/reference/2.3/full-text-queries.html) in `ElasticQueryBuilder`.
Explore if this is possible.
##
```java
public Page<Accession> list(AccessionFilter filter, Pageable page) {
if (StringUtils.isNotBlank(filter._text)) {
// FIXME Use ES to list matches
elasticsearchService.findAll(Accession.class, filter, page);
} else {
List<Accession> content = accessionRepository.findAll(filter, page);
}
long total = countAccessions(filter);
return new PageImpl<>(content, page, total);
}
```2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/437Sitemap update2019-04-16T20:10:28+02:00Matija ObrezaSitemap updateThe sitemaps need updating; the root `sitemap.xml` now includes the following sub-sitemaps:
```xml
<loc>http://localhost:3000/sitemap-content.xml</loc>
<loc>http://localhost:3000/sitemap-geo.xml</loc>
<loc>http://localhost:3000/sitemap-...The sitemaps need updating; the root `sitemap.xml` now includes the following sub-sitemaps:
```xml
<loc>http://localhost:3000/sitemap-content.xml</loc>
<loc>http://localhost:3000/sitemap-geo.xml</loc>
<loc>http://localhost:3000/sitemap-wiews.xml</loc>
<loc>http://localhost:3000/sitemap-crop.xml</loc>
```
## `sitemap-content.xml`
It now has:
```xml
<loc>http://localhost:3000/welcome</loc>
<loc>http://localhost:3000/content/about</loc>
<loc>http://localhost:3000/explore/overview</loc>
<loc>http://localhost:3000/wiews/active</loc>
<loc>http://localhost:3000/content/terms</loc>
<loc>http://localhost:3000/explore</loc>
<loc>http://localhost:3000/org/</loc>
```
Instead it should list:
- `/`
- `/content/about/about`
- `/content/legal/terms`
- `/content/legal/disclaimer`
- `/documentation/apis`
- `/documentation/brapi`
- List of all `Activity posts` with their links `/content/news/72/....`
## `sitemap-geo.xml`
Rename it to `sitemap-iso3166.xml`. It should include custom ISO3166 codes (the XXX codes).
## Add `sitemap-partner.xml`
List links to all Genesys data providers `/partner/...`.
## `sitemap-crop.xml`
Should link to the crop pages, not to `../data`.
## New sitemaps
Add `sitemap-XXX.xml` for PUBLISHED **subsets**, **datasets** and **descriptor lists**.
- `sitemap-subset.xml` lists links to all published Subsets
- ...2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/436Accession page URL2019-04-15T12:47:33+02:00Matija ObrezaAccession page URLThe new website uses `/a/{uuid}` URL to access individual accession data. The JSP version still uses `/acn/id/{id}`. Both versions support the `/10.xxx/xxxx` URL using accession DOI.
Replace `/acn/id/{id}` with `/a/{uuid}` in JSPs and m...The new website uses `/a/{uuid}` URL to access individual accession data. The JSP version still uses `/acn/id/{id}`. Both versions support the `/10.xxx/xxxx` URL using accession DOI.
Replace `/acn/id/{id}` with `/a/{uuid}` in JSPs and make a permanent redirect from `/acn/id/{id}` to `/a/{uuid}`.2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/435Accession filters autocomplete2019-04-12T11:19:24+02:00Matija ObrezaAccession filters autocompletehttp://localhost:3000/proxy/api/v1/acn/autocomplete/countryOfOrigin.code3?f=&term=slove does not find `SVN` for Slovenia.http://localhost:3000/proxy/api/v1/acn/autocomplete/countryOfOrigin.code3?f=&term=slove does not find `SVN` for Slovenia.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/434Filter by genus+species2019-04-10T16:13:47+02:00Matija ObrezaFilter by genus+speciesAllow for filtering on full species name, not just the specific epithet portion of the scientific name.
Add `genusSpecies` to `TaxonomyFilter` which will look up `Taxonomy2` records by `taxSpecies` identifier. The filter code must first...Allow for filtering on full species name, not just the specific epithet portion of the scientific name.
Add `genusSpecies` to `TaxonomyFilter` which will look up `Taxonomy2` records by `taxSpecies` identifier. The filter code must first lookup the ID and apply it directly.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/433Access token as Cookie2019-04-15T10:11:40+02:00Matija ObrezaAccess token as CookieThe `/proxy` handler in expressjs (https://gitlab.croptrust.org/genesys-pgr/genesys-ui/blob/master/server/middleware/httpProxy.ts#L18-25) checks request cookies for `access_token` and converts it to `Authorization: Bearer ...` HTTP heade...The `/proxy` handler in expressjs (https://gitlab.croptrust.org/genesys-pgr/genesys-ui/blob/master/server/middleware/httpProxy.ts#L18-25) checks request cookies for `access_token` and converts it to `Authorization: Bearer ...` HTTP header before forwarding the request to the API server.
## Servlet
We need the API server to respect the `access_token` cookie, not just the `Authorization` HTTP request header. Maybe Spring Security libraries allow for this. If not, we need a servlet that is processed **before** OAuth auth servlets in the API that converts the cookie to the `Authorization: Bearer ...` header.
When this is implemented, we can remove the `/proxy` handler from `genesys-new-ui` project and update all HTTP links that start with `/proxy/**` to point to API URL directly.2.4Maxym BorodenkoMaxym Borodenko