Genesys Backend issueshttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues2018-09-13T14:41:35+02:00https://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/318Include inherited permissions2018-09-13T14:41:35+02:00Matija ObrezaInclude inherited permissionsIn Permissions API v1, add a static private class `_AclObjectIdentity` extending AclObjectIdentity with
```java
public List<AclEntry> inherited;
```
In method `lazyLoadForJson`, return `_AclObjectIdentity` and fill the `inherited` lis...In Permissions API v1, add a static private class `_AclObjectIdentity` extending AclObjectIdentity with
```java
public List<AclEntry> inherited;
```
In method `lazyLoadForJson`, return `_AclObjectIdentity` and fill the `inherited` list recursively with ACL entries for all `parentObjects`:
```java
private void inhertied(AclObjectIdentity objectIdentity, List<AclEntry> aclEntries, Set<AclObjectIdentity> handled) {
if (objectIdentity == null || handled.contains(objectIdentity)) {
return aclEntries;
}
aclEntries.addAll(objectIdentity.getAclEntries());
handled.add(objectIdentity);
if (objectIdentity.getParentObject() != null) {
inherited(objectIdentity.getParentObject(), aclEntries, handled);
}
}
```
Also add `objectIdentity.inherited.forEach(entry -> entry.getAclSid().getId());`.2.4Alexander PrendetskiyAlexander Prendetskiyhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/316Export: Descriptor data as Excel2018-09-13T14:41:38+02:00Matija ObrezaExport: Descriptor data as ExcelExtend DescriptorController API v0 with a method that returns filtered results in Excel format using the format of the existing sheet for descriptors of the standard Catalog Excel template.
```java
@PostMapping(value = "/export", produ...Extend DescriptorController API v0 with a method that returns filtered results in Excel format using the format of the existing sheet for descriptors of the standard Catalog Excel template.
```java
@PostMapping(value = "/export", produces = { "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" } )
public void exportDescriptors(
@RequestParam(name = "f", required = false) String filterCode,
@RequestBody(required = false) DescriptorFilter filter, HttpServletResponse response) throws IOException {
if (filterCode != null) {
filter = shortFilterService.filterByCode(filterCode, DescriptorFilter.class);
} else {
filterCode = shortFilterService.getCode(filter);
}
response.setContentType("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet");
response.addHeader("Content-Disposition", String.format("attachment; filename=\"genesys-descriptors-" + filterCode + ".xlsx\""));
response.flushBuffer();
// Write XLSX to the stream.
final OutputStream outputStream = response.getOutputStream();
try {
descriptorService.exportDescriptors(filter, outputStream);
response.flushBuffer();
} catch (EOFException e) {
LOG.warn("Download was aborted", e);
}
}
```
The export should include the `UUID` and `version` as the first two columns of the updated Excel sheet.2.4Alexander PrendetskiyAlexander Prendetskiyhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/307Support inverse filters2018-09-13T14:45:00+02:00Matija ObrezaSupport inverse filtersThe current implementation of Filter objects and logic in Genesys allows for specifying "matching" or "positive" filters (e.g. `"crop": [ "maize", "wheat" ]`). We're aiming to support "non-matching" or "negative" filters (e.g. crop is no...The current implementation of Filter objects and logic in Genesys allows for specifying "matching" or "positive" filters (e.g. `"crop": [ "maize", "wheat" ]`). We're aiming to support "non-matching" or "negative" filters (e.g. crop is not maize or wheat).2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/326Tile server: Pretty dots2018-09-14T08:53:35+02:00Matija ObrezaTile server: Pretty dotsInstead of using squares as accession points use a prettier round shape.Instead of using squares as accession points use a prettier round shape.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/323Subsets: creators2018-09-14T09:45:33+02:00Matija ObrezaSubsets: creatorsExtend `Subset` model with a list of `SubsetCreator` records (similar to `Dataset#creators`):
```java
/** The creators. */
@OneToMany(cascade = { CascadeType.ALL }, mappedBy = "subset", orphanRemoval = true, fetch = FetchType.LAZY)
@Fie...Extend `Subset` model with a list of `SubsetCreator` records (similar to `Dataset#creators`):
```java
/** The creators. */
@OneToMany(cascade = { CascadeType.ALL }, mappedBy = "subset", orphanRemoval = true, fetch = FetchType.LAZY)
@Field(type = FieldType.Object)
@JsonView({ JsonViews.Public.class })
private List<SubsetCreator> creators;
```
Add API methods to `SubsetController` and `SubsetService`. You can copy most of the business logic from `DatasetCreatorServiceImpl`.2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/329Accession API v1: Download KML2018-09-14T17:57:49+02:00Matija ObrezaAccession API v1: Download KMLAdd API endpoint `/api/v1/acn/downloadKml` to AccessionController v1:
```java
@RequestMapping(value = "/downloadKml", produces = "application/vnd.google-earth.kml+xml", method = RequestMethod.POST)
@ResponseBody
public void kml(@Request...Add API endpoint `/api/v1/acn/downloadKml` to AccessionController v1:
```java
@RequestMapping(value = "/downloadKml", produces = "application/vnd.google-earth.kml+xml", method = RequestMethod.POST)
@ResponseBody
public void kml(@RequestParam(value="f") String filterCode, HttpServletResponse response) throws IOException {
// get AccessionFilter from filterCode
AccessionFilter filter = ...
final int countFiltered = accessionService.countAccessions(filter);
LOG.info("Attempting to download KML for {} accessions", countFiltered);
if (countFiltered > DOWNLOAD_LIMIT) {
throw new RuntimeException("Refusing to export more than " + DOWNLOAD_LIMIT + " entries");
}
response.setContentType("application/vnd.google-earth.kml+xml");
response.addHeader("Content-Disposition", String.format("attachment; filename=\"genesys-kml-" + filterCode + ".kml\""));
// Write KML to the stream.
final OutputStream outputStream = response.getOutputStream();
try {
mappingService.filteredKml(filter, outputStream);
response.flushBuffer();
} catch (EOFException e) {
LOG.warn("Download was aborted", e);
}
}
```2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/327Subsets model2018-09-15T13:40:45+02:00Matija ObrezaSubsets modelExtend `Subset` model.
## Crops
Add `crops` to Subset:
```java
/** The crops. */
@ElementCollection(fetch = FetchType.EAGER)
@CollectionTable(name = "subset_crops", joinColumns = @JoinColumn(name = "subsetId"),
// index
indexes = ...Extend `Subset` model.
## Crops
Add `crops` to Subset:
```java
/** The crops. */
@ElementCollection(fetch = FetchType.EAGER)
@CollectionTable(name = "subset_crops", joinColumns = @JoinColumn(name = "subsetId"),
// index
indexes = { @Index(columnList = "subsetId, crop") })
@Column(name = "crop", nullable = false, length = 20)
@JsonView({ JsonViews.Minimal.class })
private Set<String> crops;
```
1. Add `crop` to `SubsetFilter`
## Other properties
1. `String date`, a MCPD date (@Column(length = 8))
1. `String source`, generally a URL. (@Column(length = 200))2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/328Accession API v1: Geo info2018-09-16T12:41:20+02:00Matija ObrezaAccession API v1: Geo infoAdd endpoint to AccessionController **v1** API `/acn/geo` as `geoInfo(String filterCode, AccessionFilter filter)` that returns
```json
{
"bounds": [
[-4.8587000, 39.8772333],
[-6.4917667, 39.0945000]
],
"accessionCount": 1...Add endpoint to AccessionController **v1** API `/acn/geo` as `geoInfo(String filterCode, AccessionFilter filter)` that returns
```json
{
"bounds": [
[-4.8587000, 39.8772333],
[-6.4917667, 39.0945000]
],
"accessionCount": 123123,
"tileServers": [ "cdn1.sandbox.genesys-pgr.org", "cdn2.sandbox.genesys-pgr.org" ]
}
```
## Bounds
Add `getBounds(AccessionFilter filter)` to **AccessionServiceImpl**.
See https://leafletjs.com/reference-1.3.4.html#latlngbounds
It should select `min(latitude), max(longitude), max(latitude), min(longitude) from accession.accessionId.geo` and return:
```js
[
[ min(latitude), max(longitude) ],
[ max(latitude), min(longitude) ]
]
```
## Accession count
Add `public long count(AccessionFilter filter)` to **AccessionServiceImpl**:
```java
@Override
public Page<Accession> list(AccessionFilter filter) {
long total = elasticsearchService.count(Accession.class, filter);
if (total < 10000) {
// If total is below 10K, use actual count
total = accessionRepository.count(filter.buildQuery());
}
return total;
}
```
## Tile servers
Tileservers info is the `cdn.servers` setting from `application.properties` and it's accessible at:
```java
@Value("${cdn.servers}")
private String[] cdnServers;
```2.4Alexander PrendetskiyAlexander Prendetskiyhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/332CDN issue2018-09-16T13:13:26+02:00Matija ObrezaCDN issuehttps://cdn1.sandbox.genesys-pgr.org/acn/tile/3/4/1?f=v13fc6a432ab2b4d5e8ac8064f01d1fa03
```xml
<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>C4CF5C346A9FFD4A<...https://cdn1.sandbox.genesys-pgr.org/acn/tile/3/4/1?f=v13fc6a432ab2b4d5e8ac8064f01d1fa03
```xml
<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>C4CF5C346A9FFD4A</RequestId>
<HostId>ZEPMCu2JmjGtRSMjmOs84x6OMmDNd+oy3QAGu46H15RIQrxX1D2ZAU37GPx6bRRuNQFf6tkyMLc=</HostId>
</Error>
```2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/333Tile server: caching2018-09-17T07:11:49+02:00Matija ObrezaTile server: cachingTile server should use longer TTL in caching headers, especially for zoom levels 0-6, as tiles on these levels scan a lot of accession records. On higher zoom levels, fewer records are required and the TTL can be lower.
Set max-age head...Tile server should use longer TTL in caching headers, especially for zoom levels 0-6, as tiles on these levels scan a lot of accession records. On higher zoom levels, fewer records are required and the TTL can be lower.
Set max-age header as follows:
1. 0-4: max-age 1day
1. 4-6: max-age 1hr
1. 7+: max-age 5min
2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/331Accession APIv1: wrap Overview response2018-09-19T13:55:31+02:00Matija ObrezaAccession APIv1: wrap Overview responseAccession `overview` method now returns only the ES results.
Wrap the response into:
```
private static class AccessionOverview {
public String filterCode;
public AccessionFilter filter;
Map<String, TermResult> overview;
}
```Accession `overview` method now returns only the ES results.
Wrap the response into:
```
private static class AccessionOverview {
public String filterCode;
public AccessionFilter filter;
Map<String, TermResult> overview;
}
```2.4Viacheslav PavlovViacheslav Pavlovhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/330ApiInfoController v12018-09-19T13:57:17+02:00Matija ObrezaApiInfoController v1Implement a new `ApiInfoController` **v1** at `public static final String CONTROLLER_URL = ApiBaseController.APIv1_BASE + "/info";` in the **server.api.v1** package.
Copy info method from existing ApiInfoController (from Catalog)
## Ex...Implement a new `ApiInfoController` **v1** at `public static final String CONTROLLER_URL = ApiBaseController.APIv1_BASE + "/info";` in the **server.api.v1** package.
Copy info method from existing ApiInfoController (from Catalog)
## Extra data
Add additional properties to the returned object:
- `String[] cdnServers` loaded from `application.properties` (see genesys-pgr/genesys-server/issues/328)
- `long accessionCount` number of all accessions from `accessionService.count()`
- `long datasetCount` number of published datasets
- `long subsetCount` number of published subsets
- `long instituteCount` number of institutes with accessions in Genesys
- `long descriptorCount` number of published descriptors
- `long partnerCount` number of Partners
## Implementation
Please implement these "count" methods when they are missing and use this template (when possible):
```java
@Override
public long countAccessions(AccessionFilter filter) {
long total = elasticsearchService.count(Accession.class, filter);
if (total < 10000) {
// If total is below 10K, use actual count
total = accessionRepository.count(filter.buildQuery());
}
return total;
}
```
## Extras
Also expose:
- `captchaSiteKey` as `captchaSiteKey`
- `google.analytics.account` as `googleAnalyticsId`
- `itpgrfa.glis.basepath` as `glisUrl`
2.4Alexander PrendetskiyAlexander Prendetskiyhttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/339FTP2018-09-24T08:48:29+02:00Matija ObrezaFTPUpgrade to file-repository-ftp:1.1-SNAPSHOT that introduces repository folders and ACL.Upgrade to file-repository-ftp:1.1-SNAPSHOT that introduces repository folders and ACL.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/336User registration API2018-09-24T10:45:28+02:00Matija ObrezaUser registration APIAdd `UserRegistrationController` to **server.api.v1** with base at `API_BASE = ApiBaseController.APIv1_BASE + "/user";`
Copy most of the method `addUser` from `org.genesys2.server.mvc.UserRegistrationController` and expose it as API POS...Add `UserRegistrationController` to **server.api.v1** with base at `API_BASE = ApiBaseController.APIv1_BASE + "/user";`
Copy most of the method `addUser` from `org.genesys2.server.mvc.UserRegistrationController` and expose it as API POST to **/register**.
- Check captcha response!
- Add method `validateEMail(String tokenUuid, String key)` that calls `EmailValidationService` as POST to **/validate**.
2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/337CMS Controller2018-09-25T16:47:43+02:00Matija ObrezaCMS ControllerImplement a new `CMSController` with base **`APIv1_BASE + /cms`** in **server.api.v1** that exposes select methods (from `ContentService`):
```java
List<ActivityPost> lastNews();
Page<ActivityPost> allNews(int page);
Article getArticl...Implement a new `CMSController` with base **`APIv1_BASE + /cms`** in **server.api.v1** that exposes select methods (from `ContentService`):
```java
List<ActivityPost> lastNews();
Page<ActivityPost> allNews(int page);
Article getArticle(Class<?> clazz, Long id, String slug, Locale locale, boolean useDefault);
Article getArticleBySlugAndLang(String slug, String lang);
Article getGlobalArticle(String slug, Locale locale, boolean useDefault);
Page<Article> listArticles(Pageable pageable);
Page<Article> listArticlesByLang(String lang, Pageable pageable);
ActivityPost createActivityPost(String title, String body);
Article updateArticle(Class<?> clazz, Long id, String slug, String title, String body, String summary, Locale locale) throws CRMException;
Article updateArticle(long id, String slug, String title, String body, String summary);
Article createGlobalArticle(String slug, Locale locale, String title, String body, String summary, boolean isTemplate) throws CRMException;
Article updateGlobalArticle(String slug, Locale locale, String title, String body, String summary) throws CRMException;
ActivityPost getActivityPost(long id);
ActivityPost updateActivityPost(long id, String title, String body);
void deleteActivityPost(long id);
Locale getDefaultLocale();
String processTemplate(String body, Map<String, Object> root);
// Menus
Menu getMenu(String key);
Menu updateMenu(String key, List<MenuItem> menuItems);
MenuItem ensureMenuItem(String menuKey, String url, String text);
```
- Please use sensible `@RequestMapping(value=` names.
## Extras
Add `ArticleFilter` and use the filter for `listArticles(ArticleFilter filter, Pageable page)`.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/343Err: Class is not indexed interface2018-09-26T11:45:54+02:00Matija ObrezaErr: Class is not indexed interfaceFrom the server log:
```java
2018-09-25T09:30:37.770907745Z 09:30:37,770 qtp485815673-364 ERROR o.g.s.m.UserControllerAdvice:89 - Class is not indexed interface org.genesys2.server.model.json.Api1Constants$Accession on GET http://sandbo...From the server log:
```java
2018-09-25T09:30:37.770907745Z 09:30:37,770 qtp485815673-364 ERROR o.g.s.m.UserControllerAdvice:89 - Class is not indexed interface org.genesys2.server.model.json.Api1Constants$Accession on GET http://sandbox.genesys-pgr.org/explore/charts/data/country-collection-size
2018-09-25T09:30:37.770942665Z java.lang.RuntimeException: Class is not indexed interface org.genesys2.server.model.json.Api1Constants$Accession
2018-09-25T09:30:37.770952287Z at org.genesys2.server.service.impl.ElasticsearchServiceImpl.termStatistics(ElasticsearchServiceImpl.java:643)
2018-09-25T09:30:37.770958736Z at org.genesys2.server.service.impl.ElasticsearchServiceImpl.termStatisticsAuto(ElasticsearchServiceImpl.java:713)
2018-09-25T09:30:37.770964728Z at org.genesys2.server.service.impl.ElasticsearchServiceImpl.termStatisticsAuto(ElasticsearchServiceImpl.java:707)
2018-09-25T09:30:37.770984744Z at sun.reflect.GeneratedMethodAccessor1153.invoke(Unknown Source)
2018-09-25T09:30:37.770990979Z at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2018-09-25T09:30:37.770996182Z at java.lang.reflect.Method.invoke(Method.java:498)
2018-09-25T09:30:37.771001051Z at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
2018-09-25T09:30:37.771007089Z at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
2018-09-25T09:30:37.771012936Z at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
2018-09-25T09:30:37.771018273Z at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
2018-09-25T09:30:37.771023239Z at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
2018-09-25T09:30:37.771028957Z at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
2018-09-25T09:30:37.771034562Z at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
2018-09-25T09:30:37.771039615Z at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
2018-09-25T09:30:37.771044651Z at com.sun.proxy.$Proxy214.termStatisticsAuto(Unknown Source)
2018-09-25T09:30:37.771049851Z at org.genesys2.server.mvc.ChartsController.accessionsCollection(ChartsController.java:92)
```2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/341FiltersĀ conversion2018-09-26T19:47:21+02:00Matija ObrezaFiltersĀ conversionFiltering by `DEU` as "Country of holding institute" calls: https://sandbox.genesys-pgr.org/explore/listFilterSuggestions?filter=%7B%22institute.country.iso3%22%3A%5B%22DEU%22%5D%7D
The conversion of the old filter (`AppliedFilters`) fa...Filtering by `DEU` as "Country of holding institute" calls: https://sandbox.genesys-pgr.org/explore/listFilterSuggestions?filter=%7B%22institute.country.iso3%22%3A%5B%22DEU%22%5D%7D
The conversion of the old filter (`AppliedFilters`) fails with **Unknown base institute.country**:
```
2018-09-25T08:12:20.605602345Z Caused by: java.lang.RuntimeException: Unknown base institute.country
2018-09-25T08:12:20.605609064Z at org.genesys2.server.service.filter.AppliedFiltersConverter.convert(AppliedFiltersConverter.java:168)
2018-09-25T08:12:20.605616054Z at org.genesys2.server.service.filter.AccessionFilter.convert(AccessionFilter.java:166)
2018-09-25T08:12:20.605623037Z ... 116 more
```
The same happens for original filters names `regionHoldInst` and `regionOrigin`.
## Updates
Please extend `CountryFilter` with support for `Set<Long> regionId` and update `AppliedFiltersConverter.convert` to convert:
* `regionHoldInst` to `holder.country.regionId`
* `regionOrigin` to `origin.regionId`
Please add `StringFilter alias` to `AccessionFilter`. It should check if `accession.accessionId.aliases.any().name` matches the filter.2.4Maxym BorodenkoMaxym Borodenkohttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/342Blank duplSite2018-09-26T20:20:22+02:00Matija ObrezaBlank duplSiteAccession overviews shows empty string as the most common Site of safety duplication.Accession overviews shows empty string as the most common Site of safety duplication.2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/340Organization: view map2018-09-26T20:29:17+02:00Matija ObrezaOrganization: view mapThe Organization MVC controller at `/org/{slug}/map` does not show a map of institutes.
## Alternatively
... it should redirect to map of accessions `/explore/map?filter=.....`
1. https://sandbox.genesys-pgr.org/org/COGENT
1. Click "...The Organization MVC controller at `/org/{slug}/map` does not show a map of institutes.
## Alternatively
... it should redirect to map of accessions `/explore/map?filter=.....`
1. https://sandbox.genesys-pgr.org/org/COGENT
1. Click "View map"
1. **Expected:** the user is redirected to the standard accession map2.4Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/genesys-server/-/issues/346JSP: updating Overviews2018-09-27T21:05:20+02:00Matija ObrezaJSP: updating OverviewsIf you load http://localhost:8080/explore/overview?filter=%7B%22regionOrigin%22%3A%5B%22015%22%5D%7D the booleans are correctly translated:
![image](/uploads/82247293f19017ee934f6679ff7e0329/image.png)
But if hit "Apply" again, the Jav...If you load http://localhost:8080/explore/overview?filter=%7B%22regionOrigin%22%3A%5B%22015%22%5D%7D the booleans are correctly translated:
![image](/uploads/82247293f19017ee934f6679ff7e0329/image.png)
But if hit "Apply" again, the Javascript code does not handle the booleans properly:
![image](/uploads/5c594b7a1e8d8dee620ef0c485ed9899/image.png)
There is some change in ES handling of booleans, please update the Javascript.2.4Maxym BorodenkoMaxym Borodenko