Genesys Backend issueshttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues2021-02-15T09:55:34+01:00https://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/296Replace all @Query with DSL2021-02-15T09:55:34+01:00Matija ObrezaReplace all @Query with DSLThe "text" mode `@Query` syntax in JPA repositories is impossible to maintain as the data models are updated and changed.
Migrate all cases where `@Query` is used to QueryDSL. Use the following approach:
1. If the query is used from a ...The "text" mode `@Query` syntax in JPA repositories is impossible to maintain as the data models are updated and changed.
Migrate all cases where `@Query` is used to QueryDSL. Use the following approach:
1. If the query is used from a single service method, implement the query in the service method
1. If the query is used from multiple service methods, implement the query in a `...CustomImpl` and extend the repository with a `...Custom` interface2.5Vladyslava MokliakVladyslava Mokliakhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/295Accession overview API2018-08-03T05:07:05+02:00Matija ObrezaAccession overview API2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/294InstituteController v12018-08-04T12:33:53+02:00Matija ObrezaInstituteController v1Add a new API controller `v1.InstituteController` with these methods:
## `/details`
Loads basic `FaoInstitute` data and if not `null` it adds:
- ES termStatistics on `crop.shortName`, `cropName`, `taxonomy.genus`, `taxonomy.genusSpeci...Add a new API controller `v1.InstituteController` with these methods:
## `/details`
Loads basic `FaoInstitute` data and if not `null` it adds:
- ES termStatistics on `crop.shortName`, `cropName`, `taxonomy.genus`, `taxonomy.genusSpecies`
- PDCI statistics for the institute
- Last updates of passport data for the institute
```java
@GetMapping("/{wiewsCode}")
public InstituteDetails details(...) {
InstituteDetails details = new InstituteDetails();
details.stats = elasticsearchService....
details.blurb = ...
...
return details;
}
// New class
public static class InsituteDetails {
public FaoInstitute institute;
public Article blurb;
public ? lastUpdates;
public ? stats; // for term statistics
}
}
```
Using InstituteDetails class will also help with API docs generation.2.4Alexander PrendetskiyAlexander Prendetskiyhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/293Dataset location and timing2018-08-03T15:25:21+02:00Matija ObrezaDataset location and timingThe `DatasetLocation` now holds data on **where** the plant material was evaluated, but does not tell us **when** that happened.
Add `String startDate` and `endDate` fields to this class. `@Column(length = 8)`
## Date format
We need t...The `DatasetLocation` now holds data on **where** the plant material was evaluated, but does not tell us **when** that happened.
Add `String startDate` and `endDate` fields to this class. `@Column(length = 8)`
## Date format
We need to keep this as `String` because the exact month or day of the dates may not be known. We will use the `YYYYMMDD` format and the setter should test with `MCPDUtils.isMcpdDate()` method if the value is valid.
## Update MCPDUtils
Update `MCPDUtils.isMcpdDate()` method to test that the values for **DD** and **MM** are:
- DD: -- or 00 or valid month value from 1 to 12
- MM: -- or 00 or valid day in the month of the year (if provided)
## Dataset
Update `Dataset` class and add `String startDate` and `endDate` fields to `Dataset`. `@Column(length = 8)`
In `@PrePersist/@PreUpdate` get the minimum and maximum start and end dates from all dataset locations.
```java
if (dataset.locations != null) {
this.startDate = dataset.locations.stream().map(ds -> ds.startDate).sorted(/* ascending */).findFirst().orElse(null);
this.endDate = ... /* descending */
}
```
You may need to update methods for saving locations to also trigger an update on the dataset.
## After merge
When this is done, file a ticket in the **catalog-ui** project to update the forms and display of DatasetLocations.2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/292Genesys is down HTTP ERROR 5032018-08-29T11:12:02+02:00Aleksandr KirichenkoGenesys is down HTTP ERROR 503Steps:
Go to the "https://sandbox.genesys-pgr.org/welcome";
![SCR_0074](/uploads/c6062b4bcce75d99c7ae34e90bc401df/SCR_0074.png)Steps:
Go to the "https://sandbox.genesys-pgr.org/welcome";
![SCR_0074](/uploads/c6062b4bcce75d99c7ae34e90bc401df/SCR_0074.png)Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/291Bit.ly shortener2019-11-20T17:25:50+01:00Matija ObrezaBit.ly shortenerAdd support to shorten URLs using bit.ly in addition to goo.gl.
http://dev.bitly.com/v4_documentation.html#operation/createBitlink
Keep both services in the code.Add support to shorten URLs using bit.ly in addition to goo.gl.
http://dev.bitly.com/v4_documentation.html#operation/createBitlink
Keep both services in the code.2.5Alexander PrendetskiyAlexander Prendetskiyhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/290Error after Download as HTML is selected2018-07-26T09:37:32+02:00Nora Castaneda-AlvarezError after Download as HTML is selectedI'm trying to display a published descriptor list as HTML and I get an error. Use this descriptor list as example: https://catalog.demo.genesys-pgr.org/descriptorlists/e98bdd33-5c6c-4bac-ae95-95fbbb40d68cI'm trying to display a published descriptor list as HTML and I get an error. Use this descriptor list as example: https://catalog.demo.genesys-pgr.org/descriptorlists/e98bdd33-5c6c-4bac-ae95-95fbbb40d68cMatija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/289Keep average PDCI in FaoInstitute2018-09-11T13:23:34+02:00Matija ObrezaKeep average PDCI in FaoInstituteMaintain average PDCI score in `FaoInsitute` entity.
Add `Double` fields `pdciMin`, `pdciMax`, `pdciAvg` to `FaoInstitute` (update liquibase).
These values should be updated in the `org.genesys2.server.service.worker.AccessionCounter` ...Maintain average PDCI score in `FaoInsitute` entity.
Add `Double` fields `pdciMin`, `pdciMax`, `pdciAvg` to `FaoInstitute` (update liquibase).
These values should be updated in the `org.genesys2.server.service.worker.AccessionCounter` method **process**:
```java
FaoInstitute institute = instituteService.findInstitute(forProcessing.getObj());
if (institute != null) {
LOG.info("Updating count for {}", institute.getCode());
genesysService.updateAccessionCount(institute);
// Add
genesysService.updatePDCI(institute);
}
```
Implement method `genesysService.updatePDCI(FaoInstitute)` based on `genesysService.statisticsPDCI(FaoInstitute)`.
Remove method `genesysService.statisticsPDCI(FaoInstitute)` and refactor code to use the numbers stored in `FaoInstitute` entity. This means fix the code in `genesysService.statisticsPDCI(Organization)` -- just use numbers from the entity, don't query the database. You will need to use `institute.accessionCount` instead of `(Long) overall[3]` (which contains the count).
Update `mvc.AccessionController` and remove the call to statisticsService, use values from `institute` in the JSP.
2.5Alexander PrendetskiyAlexander Prendetskiyhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/288Cache responses to API queries2018-07-26T08:31:52+02:00Matija ObrezaCache responses to API queriesAdd `@Cache` annotations to services responding to API queries. Make those responses short-lived.Add `@Cache` annotations to services responding to API queries. Make those responses short-lived.2.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/287Move tileIndex to AccessionData2018-08-20T18:02:52+02:00Matija ObrezaMove tileIndex to AccessionDataQuerying for `lat, lon` takes forever, current filtering by latitude takes 30s:
![image](/uploads/790cc4d6f613120540bcdc030bedd2e9/image.png)
By moving `tileIndex` property from `AccessionGeo` to `AccessionData` we will have better acc...Querying for `lat, lon` takes forever, current filtering by latitude takes 30s:
![image](/uploads/790cc4d6f613120540bcdc030bedd2e9/image.png)
By moving `tileIndex` property from `AccessionGeo` to `AccessionData` we will have better access to georeferenced accessions.
Update liquibase to add tileIndex, copy current data from `AccessionGeo` to `Accession` and `AccessionHistoric`, but also keep it in Geo.
Add database index on `tileIndex`.
Update tileIndex (prepersist) in `AccessionData`.
## Updated querying for lat/lon
In addition to querying by `latitude` and `longitude` ranges, add criteria on `Accession` to have `tileIndex not null`.
2.5Alexander PrendetskiyAlexander Prendetskiyhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/286Publishing things2018-09-07T11:00:30+02:00Matija ObrezaPublishing thingsIn the Catalog the users were allowed to directly publish their data and then had a 24-hrs window during which they could **un-publish**.
## Revised publishing process
Instead of directly publishing datasets to the Internet we will int...In the Catalog the users were allowed to directly publish their data and then had a 24-hrs window during which they could **un-publish**.
## Revised publishing process
Instead of directly publishing datasets to the Internet we will introduce a **Review** step: when a user clicks **Publish** that will lock the dataset for editing, but will not make it available to the public.
Dataset metadata in **Review state** will be reviewed by Genesys Team who will provide feedback on metadata, request for clarifications or modifications, etc.
The dataset owner (and others with permissions) can click **Edit** when the dataset is in **Review state**. This action returns the record to **Draft state** where it can be updated again.
**Accept** function is available to Genesys Team and with this the dataset goes public, but it can still be "Unpublished” in the 24-hour window.
## Updates to backend
This affects datasets, descriptor lists and descriptors:
1. Introduce `enum` named `PublishState` with options: `DRAFT(0)`, `REVIEWING(-1)` and `PUBLISHED(1)` with those integers attached.
1. Rename `private boolean published;` to `private PublishState state;`. Update liquibase.
1. Add `state` filter to Filter models of affected entities
1. Service methods listing **published** datasets, descriptor lists and descriptors must return data that has `state == PUBLISHED`
## API actions
1. Update API methods `publish***` to change state to `REVIEWING`. Available to admins and whoever has permissions.
1. Add API methods `approve***` to change state from `REVIEWING` to `PUBLISHED`. Available to `hasRole('ADMINISTRATOR')` only.
1. Update API methods `edit***` to change state from `REVIEWING` to `DRAFT`. Available to admins and whoever has permissions.
## Unit tests
Make sure the tests are updated for the 3-state publish status.2.5Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/284Dataset AccessionIdentifier2018-08-03T05:07:03+02:00Matija ObrezaDataset AccessionIdentifierA Catalog Dataset contains a list of embedded `org.genesys.catalog.model.dataset.AccessionIdentifier` records (doi, instCode, acceNumb, genus, species). These must be linked with Accession records in Genesys.
1. First, make `AccessionId...A Catalog Dataset contains a list of embedded `org.genesys.catalog.model.dataset.AccessionIdentifier` records (doi, instCode, acceNumb, genus, species). These must be linked with Accession records in Genesys.
1. First, make `AccessionIdentifier` implement `AccessionIdentifier3`.
1. Add `private AccessionId accession;` to AccessionIdentifier with a `@ManyToOne` mapping. The value can be null if matching Accession is not in Genesys.
When registering AccessionIdentifiers with a Dataset, the service should lookup matching Accessions using GenesysService.2.5Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/283JWT OAuth tokens2018-08-22T18:32:13+02:00Matija ObrezaJWT OAuth tokensBring in the JWT token converter from the catalog-server.
Test if Genesys Uploader (Anno) works with JWT.Bring in the JWT token converter from the catalog-server.
Test if Genesys Uploader (Anno) works with JWT.2.5Alexander PrendetskiyAlexander Prendetskiyhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/282Accession APIv1 /filter2018-07-24T23:34:42+02:00Matija ObrezaAccession APIv1 /filterAdd **/filter** endpoint to `org.genesys2.server.api.v1.AccessionController` (depends on #281).
Method (based on `DatasetController#list`:
```java
@PostMapping(value = "/filter")
public FilteredPage<Accession> filter(@RequestParam(name...Add **/filter** endpoint to `org.genesys2.server.api.v1.AccessionController` (depends on #281).
Method (based on `DatasetController#list`:
```java
@PostMapping(value = "/filter")
public FilteredPage<Accession> filter(@RequestParam(name = "p", required = false, defaultValue = "0") final int page,
@RequestParam(name = "l", required = false, defaultValue = "50") final int pageSize,
@RequestParam(name = "d", required = false, defaultValue = "ASC") final Sort.Direction direction,
@RequestParam(name = "s", required = false, defaultValue = "id") final String[] sort, @RequestBody final DatasetFilter filter) {
if (StringUtils.isNotBlank(filterCode)) {
filter = shortFilterService.filterByCode(filterCode, DatasetFilter.class);
} else {
filterCode = shortFilterService.getCode(filter);
}
return new FilteredPage<>(filterCode, filter, genesysService.filter(filter, new PageRequest(page, Integer.min(pageSize, 100), direction, sort)));
}
```2.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/281Model filters2018-08-03T14:51:44+02:00Matija ObrezaModel filtersIn the Catalog we developed a good approach to providing filter data to the backend.
Add model filters to `org.genesys2.server.model.filters` package:
- `AccessionFilter` for `Accession` (and `AccessionData` and `AccessionId`)
- `Insti...In the Catalog we developed a good approach to providing filter data to the backend.
Add model filters to `org.genesys2.server.model.filters` package:
- `AccessionFilter` for `Accession` (and `AccessionData` and `AccessionId`)
- `InstituteFilter` for `FaoInstitute`2.5Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/280Bug: CWR page 5002018-08-27T10:06:08+02:00Matija ObrezaBug: CWR page 500- Go to the https://sandbox.genesys-pgr.org/welcome
- Click on the CWR Project link;
## Overviews
The page is fixed in 78739c47, but the overviews do not show up. Fix the overviews.- Go to the https://sandbox.genesys-pgr.org/welcome
- Click on the CWR Project link;
## Overviews
The page is fixed in 78739c47, but the overviews do not show up. Fix the overviews.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/279Bug: Latitude filter fails2018-07-20T22:33:07+02:00Matija ObrezaBug: Latitude filter fails- Go to the "Browse" page
- Open Latitude filter
- Input some value in the filter fields- Go to the "Browse" page
- Open Latitude filter
- Input some value in the filter fields2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/278Bug: Create new list functionality2018-07-21T10:54:28+02:00Matija ObrezaBug: Create new list functionalityCreating new list fails with 500 server error. Steps to reproduce:
- Go to the https://sandbox.genesys-pgr.org/welcome
- Login as admin;
- Go to the "Browse" page;
- Add some accessions to the list;
- Go to the "My List" page;
- Go to t...Creating new list fails with 500 server error. Steps to reproduce:
- Go to the https://sandbox.genesys-pgr.org/welcome
- Login as admin;
- Go to the "Browse" page;
- Add some accessions to the list;
- Go to the "My List" page;
- Go to the "Accession List" tab;
- Fill the fields and click on the "CREATE NEW LIST" button.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/277Updating controlled vocabularies fails2018-10-26T17:00:47+02:00Matija ObrezaUpdating controlled vocabularies failsTrying to update vocabularies from
- https://catalog.demo.genesys-pgr.org/proxy/api/v0/wiews/update
- https://catalog.demo.genesys-pgr.org/proxy/api/v0/lang/update
- https://catalog.demo.genesys-pgr.org/proxy/api/v0/geo/update
Depends ...Trying to update vocabularies from
- https://catalog.demo.genesys-pgr.org/proxy/api/v0/wiews/update
- https://catalog.demo.genesys-pgr.org/proxy/api/v0/lang/update
- https://catalog.demo.genesys-pgr.org/proxy/api/v0/geo/update
Depends on #266 Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/275Extend Recent activity 22018-07-26T09:32:03+02:00Matija ObrezaExtend Recent activity 2Similar to #274, but include display of:
- Last 5 Datasets published (depends on #266)
Include a link to https://catalog.genesys-pgr.org at the bottom.
Update Controller and JSP.Similar to #274, but include display of:
- Last 5 Datasets published (depends on #266)
Include a link to https://catalog.genesys-pgr.org at the bottom.
Update Controller and JSP.2.5Maxym BorodenkoMaxym Borodenko