Genesys Backend issueshttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues2018-10-09T15:48:06+02:00https://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/358Linking subsets and datasets with accessions2018-10-09T15:48:06+02:00Matija ObrezaLinking subsets and datasets with accessionsDataset currently uses a list of `AccessionIdentifier` records to specify which accessions (by instcode, accenumb, genus **or** doi) are referenced in the dataset.
Subset, on the other hand, uses a list of `AccessionId` so that even if ...Dataset currently uses a list of `AccessionIdentifier` records to specify which accessions (by instcode, accenumb, genus **or** doi) are referenced in the dataset.
Subset, on the other hand, uses a list of `AccessionId` so that even if the accession is deleted from Genesys (and is removed from `Accession` and added to `AccessionHistoric`) the subset still maintains the reference to the original record. This was a good idea originally, but it fails in the case where an accession is **reuploaded** to Genesys -- Subset will not link to the correct record anymore.
## Properly referencing accessions
- Rename and move `org.genesys.catalog.model.dataset.AccessionIdentifier` to `org.genesys2.server.model.genesys.AccessionRef`.
- Change `private AccessionId accession` to `private Accession accession`
- We can now use `AccessionFilter` in `DatasetFilter`!
### Serialization
Serialize property `accession` of `AccessionRef` as `@JsonUnwrapped`. This will embed all key data from `Accession` into this object (hope that works).
### AspectJ
- When an accession is **deleted** from `Accession` (table `accession`) the code should update all `AccessionRef#accession` to `null`.
- When an accession is **saved** to `Accession` (table `accession`) the code should update `AccessionRef#accession` where a match is found.
## Subset update
Replace `private List<AccessionId> accessions` with `private List<AccessionRef> accessions`.
This allows us to have Subsets that "reference" an accession that does not have passport data in Genesys, but it will in the future. It also allows users to delete an accession (rarely) and re-upload it to the system.
## Liquibase
- ~~Rename `inverseJoinColumns = @JoinColumn(name = "acce_id")` to `a_id`.~~
- Migrate all existing data2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/353Unpublishing referenced descriptors2018-10-07T11:10:39+02:00Matija ObrezaUnpublishing referenced descriptorsThe system should not allow unpublishing a descriptor that is referenced by a published dataset or a published descriptor list.
Add unit tests.The system should not allow unpublishing a descriptor that is referenced by a published dataset or a published descriptor list.
Add unit tests.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/356Dataset export details2018-10-04T17:04:26+02:00Matija ObrezaDataset export detailsDataset metadata export as Excel:
- Add url to the Catalog to sheet **metadata* (row **Dataset_URL**).
- Don't display emails. phones, fax numbers in Sheet 'creators'
Dataset metadata export as Excel:
- Add url to the Catalog to sheet **metadata* (row **Dataset_URL**).
- Don't display emails. phones, fax numbers in Sheet 'creators'
2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/357Dataset model, service, controller updates2018-10-04T17:04:26+02:00Matija ObrezaDataset model, service, controller updates## Model
We need the country ISO3 code in Dataset `Location` model. Add `countryCode` field (String length 3, nullable).
## Services and controllers
Merge `DatasetCreatorServiceImpl` to `DatasetService`.
Merge `DatasetCreatorControll...## Model
We need the country ISO3 code in Dataset `Location` model. Add `countryCode` field (String length 3, nullable).
## Services and controllers
Merge `DatasetCreatorServiceImpl` to `DatasetService`.
Merge `DatasetCreatorController`, `DatasetFilesController` to `DatasetController` (catalog API v0).2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/354Repository API v12018-10-02T12:58:04+02:00Matija ObrezaRepository API v1Implement API to Repository serviceImplement API to Repository service2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/349Image gallery 5002018-10-01T15:51:11+02:00Matija ObrezaImage gallery 500https://sandbox.genesys-pgr.org/admin/r/g/crop/cowpeahttps://sandbox.genesys-pgr.org/admin/r/g/crop/cowpea2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/306Repository: permissions2018-09-30T11:30:46+02:00Matija ObrezaRepository: permissionsAnonymous users cannot access https://sandbox.genesys-pgr.org/acn/id/538527Anonymous users cannot access https://sandbox.genesys-pgr.org/acn/id/5385272.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/338AsciiDocController v12018-09-29T18:04:48+02:00Matija ObrezaAsciiDocController v1Copy `org.genesys2.server.mvc.AsciiDocController` to **server.api.v1** with **`APIv1_BASE + /cms/d`** base path.
- Add `produces=` content typesCopy `org.genesys2.server.mvc.AsciiDocController` to **server.api.v1** with **`APIv1_BASE + /cms/d`** base path.
- Add `produces=` content types2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/335Requests API v1: creating requests for material2018-09-28T22:32:18+02:00Matija ObrezaRequests API v1: creating requests for materialExtend the `RequestsController` V1 with methods for creating new requests for material.
Expose methods available in `RequestService`:
- `initiateRequest(RequestInfo requestInfo, Set<UUID> accessionUuids)` as a POST to **/initiate** (t...Extend the `RequestsController` V1 with methods for creating new requests for material.
Expose methods available in `RequestService`:
- `initiateRequest(RequestInfo requestInfo, Set<UUID> accessionUuids)` as a POST to **/initiate** (this method doesn't exist, please add it).
- `validateClientRequest(String tokenUuid, String key)` as a POST to **/validate**2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/346JSP: updating Overviews2018-09-27T21:05:20+02:00Matija ObrezaJSP: updating OverviewsIf you load http://localhost:8080/explore/overview?filter=%7B%22regionOrigin%22%3A%5B%22015%22%5D%7D the booleans are correctly translated:
![image](/uploads/82247293f19017ee934f6679ff7e0329/image.png)
But if hit "Apply" again, the Jav...If you load http://localhost:8080/explore/overview?filter=%7B%22regionOrigin%22%3A%5B%22015%22%5D%7D the booleans are correctly translated:
![image](/uploads/82247293f19017ee934f6679ff7e0329/image.png)
But if hit "Apply" again, the Javascript code does not handle the booleans properly:
![image](/uploads/5c594b7a1e8d8dee620ef0c485ed9899/image.png)
There is some change in ES handling of booleans, please update the Javascript.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/340Organization: view map2018-09-26T20:29:17+02:00Matija ObrezaOrganization: view mapThe Organization MVC controller at `/org/{slug}/map` does not show a map of institutes.
## Alternatively
... it should redirect to map of accessions `/explore/map?filter=.....`
1. https://sandbox.genesys-pgr.org/org/COGENT
1. Click "...The Organization MVC controller at `/org/{slug}/map` does not show a map of institutes.
## Alternatively
... it should redirect to map of accessions `/explore/map?filter=.....`
1. https://sandbox.genesys-pgr.org/org/COGENT
1. Click "View map"
1. **Expected:** the user is redirected to the standard accession map2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/342Blank duplSite2018-09-26T20:20:22+02:00Matija ObrezaBlank duplSiteAccession overviews shows empty string as the most common Site of safety duplication.Accession overviews shows empty string as the most common Site of safety duplication.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/341FiltersĀ conversion2018-09-26T19:47:21+02:00Matija ObrezaFiltersĀ conversionFiltering by `DEU` as "Country of holding institute" calls: https://sandbox.genesys-pgr.org/explore/listFilterSuggestions?filter=%7B%22institute.country.iso3%22%3A%5B%22DEU%22%5D%7D
The conversion of the old filter (`AppliedFilters`) fa...Filtering by `DEU` as "Country of holding institute" calls: https://sandbox.genesys-pgr.org/explore/listFilterSuggestions?filter=%7B%22institute.country.iso3%22%3A%5B%22DEU%22%5D%7D
The conversion of the old filter (`AppliedFilters`) fails with **Unknown base institute.country**:
```
2018-09-25T08:12:20.605602345Z Caused by: java.lang.RuntimeException: Unknown base institute.country
2018-09-25T08:12:20.605609064Z at org.genesys2.server.service.filter.AppliedFiltersConverter.convert(AppliedFiltersConverter.java:168)
2018-09-25T08:12:20.605616054Z at org.genesys2.server.service.filter.AccessionFilter.convert(AccessionFilter.java:166)
2018-09-25T08:12:20.605623037Z ... 116 more
```
The same happens for original filters names `regionHoldInst` and `regionOrigin`.
## Updates
Please extend `CountryFilter` with support for `Set<Long> regionId` and update `AppliedFiltersConverter.convert` to convert:
* `regionHoldInst` to `holder.country.regionId`
* `regionOrigin` to `origin.regionId`
Please add `StringFilter alias` to `AccessionFilter`. It should check if `accession.accessionId.aliases.any().name` matches the filter.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/343Err: Class is not indexed interface2018-09-26T11:45:54+02:00Matija ObrezaErr: Class is not indexed interfaceFrom the server log:
```java
2018-09-25T09:30:37.770907745Z 09:30:37,770 qtp485815673-364 ERROR o.g.s.m.UserControllerAdvice:89 - Class is not indexed interface org.genesys2.server.model.json.Api1Constants$Accession on GET http://sandbo...From the server log:
```java
2018-09-25T09:30:37.770907745Z 09:30:37,770 qtp485815673-364 ERROR o.g.s.m.UserControllerAdvice:89 - Class is not indexed interface org.genesys2.server.model.json.Api1Constants$Accession on GET http://sandbox.genesys-pgr.org/explore/charts/data/country-collection-size
2018-09-25T09:30:37.770942665Z java.lang.RuntimeException: Class is not indexed interface org.genesys2.server.model.json.Api1Constants$Accession
2018-09-25T09:30:37.770952287Z at org.genesys2.server.service.impl.ElasticsearchServiceImpl.termStatistics(ElasticsearchServiceImpl.java:643)
2018-09-25T09:30:37.770958736Z at org.genesys2.server.service.impl.ElasticsearchServiceImpl.termStatisticsAuto(ElasticsearchServiceImpl.java:713)
2018-09-25T09:30:37.770964728Z at org.genesys2.server.service.impl.ElasticsearchServiceImpl.termStatisticsAuto(ElasticsearchServiceImpl.java:707)
2018-09-25T09:30:37.770984744Z at sun.reflect.GeneratedMethodAccessor1153.invoke(Unknown Source)
2018-09-25T09:30:37.770990979Z at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2018-09-25T09:30:37.770996182Z at java.lang.reflect.Method.invoke(Method.java:498)
2018-09-25T09:30:37.771001051Z at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
2018-09-25T09:30:37.771007089Z at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
2018-09-25T09:30:37.771012936Z at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
2018-09-25T09:30:37.771018273Z at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
2018-09-25T09:30:37.771023239Z at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
2018-09-25T09:30:37.771028957Z at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
2018-09-25T09:30:37.771034562Z at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
2018-09-25T09:30:37.771039615Z at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
2018-09-25T09:30:37.771044651Z at com.sun.proxy.$Proxy214.termStatisticsAuto(Unknown Source)
2018-09-25T09:30:37.771049851Z at org.genesys2.server.mvc.ChartsController.accessionsCollection(ChartsController.java:92)
```2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/337CMS Controller2018-09-25T16:47:43+02:00Matija ObrezaCMS ControllerImplement a new `CMSController` with base **`APIv1_BASE + /cms`** in **server.api.v1** that exposes select methods (from `ContentService`):
```java
List<ActivityPost> lastNews();
Page<ActivityPost> allNews(int page);
Article getArticl...Implement a new `CMSController` with base **`APIv1_BASE + /cms`** in **server.api.v1** that exposes select methods (from `ContentService`):
```java
List<ActivityPost> lastNews();
Page<ActivityPost> allNews(int page);
Article getArticle(Class<?> clazz, Long id, String slug, Locale locale, boolean useDefault);
Article getArticleBySlugAndLang(String slug, String lang);
Article getGlobalArticle(String slug, Locale locale, boolean useDefault);
Page<Article> listArticles(Pageable pageable);
Page<Article> listArticlesByLang(String lang, Pageable pageable);
ActivityPost createActivityPost(String title, String body);
Article updateArticle(Class<?> clazz, Long id, String slug, String title, String body, String summary, Locale locale) throws CRMException;
Article updateArticle(long id, String slug, String title, String body, String summary);
Article createGlobalArticle(String slug, Locale locale, String title, String body, String summary, boolean isTemplate) throws CRMException;
Article updateGlobalArticle(String slug, Locale locale, String title, String body, String summary) throws CRMException;
ActivityPost getActivityPost(long id);
ActivityPost updateActivityPost(long id, String title, String body);
void deleteActivityPost(long id);
Locale getDefaultLocale();
String processTemplate(String body, Map<String, Object> root);
// Menus
Menu getMenu(String key);
Menu updateMenu(String key, List<MenuItem> menuItems);
MenuItem ensureMenuItem(String menuKey, String url, String text);
```
- Please use sensible `@RequestMapping(value=` names.
## Extras
Add `ArticleFilter` and use the filter for `listArticles(ArticleFilter filter, Pageable page)`.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/336User registration API2018-09-24T10:45:28+02:00Matija ObrezaUser registration APIAdd `UserRegistrationController` to **server.api.v1** with base at `API_BASE = ApiBaseController.APIv1_BASE + "/user";`
Copy most of the method `addUser` from `org.genesys2.server.mvc.UserRegistrationController` and expose it as API POS...Add `UserRegistrationController` to **server.api.v1** with base at `API_BASE = ApiBaseController.APIv1_BASE + "/user";`
Copy most of the method `addUser` from `org.genesys2.server.mvc.UserRegistrationController` and expose it as API POST to **/register**.
- Check captcha response!
- Add method `validateEMail(String tokenUuid, String key)` that calls `EmailValidationService` as POST to **/validate**.
2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/339FTP2018-09-24T08:48:29+02:00Matija ObrezaFTPUpgrade to file-repository-ftp:1.1-SNAPSHOT that introduces repository folders and ACL.Upgrade to file-repository-ftp:1.1-SNAPSHOT that introduces repository folders and ACL.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/330ApiInfoController v12018-09-19T13:57:17+02:00Matija ObrezaApiInfoController v1Implement a new `ApiInfoController` **v1** at `public static final String CONTROLLER_URL = ApiBaseController.APIv1_BASE + "/info";` in the **server.api.v1** package.
Copy info method from existing ApiInfoController (from Catalog)
## Ex...Implement a new `ApiInfoController` **v1** at `public static final String CONTROLLER_URL = ApiBaseController.APIv1_BASE + "/info";` in the **server.api.v1** package.
Copy info method from existing ApiInfoController (from Catalog)
## Extra data
Add additional properties to the returned object:
- `String[] cdnServers` loaded from `application.properties` (see genesys-pgr/genesys-server/issues/328)
- `long accessionCount` number of all accessions from `accessionService.count()`
- `long datasetCount` number of published datasets
- `long subsetCount` number of published subsets
- `long instituteCount` number of institutes with accessions in Genesys
- `long descriptorCount` number of published descriptors
- `long partnerCount` number of Partners
## Implementation
Please implement these "count" methods when they are missing and use this template (when possible):
```java
@Override
public long countAccessions(AccessionFilter filter) {
long total = elasticsearchService.count(Accession.class, filter);
if (total < 10000) {
// If total is below 10K, use actual count
total = accessionRepository.count(filter.buildQuery());
}
return total;
}
```
## Extras
Also expose:
- `captchaSiteKey` as `captchaSiteKey`
- `google.analytics.account` as `googleAnalyticsId`
- `itpgrfa.glis.basepath` as `glisUrl`
2.4Alexander PrendetskiyAlexander Prendetskiyhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/331Accession APIv1: wrap Overview response2018-09-19T13:55:31+02:00Matija ObrezaAccession APIv1: wrap Overview responseAccession `overview` method now returns only the ES results.
Wrap the response into:
```
private static class AccessionOverview {
public String filterCode;
public AccessionFilter filter;
Map<String, TermResult> overview;
}
```Accession `overview` method now returns only the ES results.
Wrap the response into:
```
private static class AccessionOverview {
public String filterCode;
public AccessionFilter filter;
Map<String, TermResult> overview;
}
```2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/333Tile server: caching2018-09-17T07:11:49+02:00Matija ObrezaTile server: cachingTile server should use longer TTL in caching headers, especially for zoom levels 0-6, as tiles on these levels scan a lot of accession records. On higher zoom levels, fewer records are required and the TTL can be lower.
Set max-age head...Tile server should use longer TTL in caching headers, especially for zoom levels 0-6, as tiles on these levels scan a lot of accession records. On higher zoom levels, fewer records are required and the TTL can be lower.
Set max-age header as follows:
1. 0-4: max-age 1day
1. 4-6: max-age 1hr
1. 7+: max-age 5min
2.4Matija ObrezaMatija Obreza