validator.genesys-pgr.org issueshttps://gitlab.croptrust.org/genesys-pgr/validator/-/issues2022-07-22T12:48:13+02:00https://gitlab.croptrust.org/genesys-pgr/validator/-/issues/20Reload taxonomic data2022-07-22T12:48:13+02:00Matija ObrezaReload taxonomic dataAdd a scheduled function that runs every 5 days and:
1. Downloads the latest GRIN taxonomy database
2. Checks if the file is different from the last downloaded file (need to keep the file)
- If the same, keep one copy, exit.
3. If fi...Add a scheduled function that runs every 5 days and:
1. Downloads the latest GRIN taxonomy database
2. Checks if the file is different from the last downloaded file (need to keep the file)
- If the same, keep one copy, exit.
3. If file is different, then update the Taxonomy database with new data2.0Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/validator/-/issues/19Upgrade dependencies2022-07-21T11:54:46+02:00Matija ObrezaUpgrade dependenciesUse GGCE or Genesys as source of versions.
- Jetty 10.0.11 (mvn and Dockerfile)
- Spring, log4j2 & co. upgrade
- Remove gzipFilter and use jetty'sUse GGCE or Genesys as source of versions.
- Jetty 10.0.11 (mvn and Dockerfile)
- Spring, log4j2 & co. upgrade
- Remove gzipFilter and use jetty's2.0Artem HrybeniukArtem Hrybeniukhttps://gitlab.croptrust.org/genesys-pgr/validator/-/issues/13Content negotiation2018-04-17T21:04:34+02:00Matija ObrezaContent negotiationResults are now rendered as `text/html`. Add support for content negotiation for `text/csv` and return analysis results as CSV.Results are now rendered as `text/html`. Add support for content negotiation for `text/csv` and return analysis results as CSV.2.0Matija ObrezaMatija Obrezahttps://gitlab.croptrust.org/genesys-pgr/validator/-/issues/11Auto-detect CSV configuration2017-11-15T16:35:34+01:00Matija ObrezaAuto-detect CSV configurationAdd "Autodetect" button to the CSV configuration section of the form. A Javascript function inspects the text in the `csvText` text area and tries to automatically detect the
1. separator char
1. quote char
1. escape char
1. decimal mar...Add "Autodetect" button to the CSV configuration section of the form. A Javascript function inspects the text in the `csvText` text area and tries to automatically detect the
1. separator char
1. quote char
1. escape char
1. decimal mark
Sensible defaults to start from are `tab`, `"`, `\` and `.`.
## Separator char
Simple version: try `tab`, `,`, ` `, `|` in this order and find for which candidate char the first 10 rows (or less) return the same number of columns when splitting the row.
Text analysis version: built a map of character occurrences for each of the first few lines. It is unlikely that `[a-Z0-9]` would be used as separator, so those can be ignored. The character that has the most similar number of occurrences in all rows is most likely the separator.
## Quote char
After selecting the separator, extract all column values of the first 10 rows.
Simple version: test if `"` or `'` appear as the first and last character.
Text analysis version: build the character occurrence map of first and last characters (they must must match) of all column values. It is again unlikely that `[a-Z0-9]` would be used as the quote char. The character that appears most as the first and last character is likely the quote char.
## Escape char
After selecting the separator and quote char, extract all column values of the first 10 rows that start and end with the quote char. Build an occurrence map of the character immediately before any occurrence of quote char within the column values.
## Decimal mark
Note: decimal mark appears only once in a number.
Extract all column values of the first 10 rows that contain at least one digit. Build a character occurrence map of all non-digit characters, ignoring `+`, `-` and `[a-Z]`. Decimal mark is the character that appears at most once in all column values that contain digits.2.0Maxym BorodenkoMaxym Borodenko