Commit 1207f049 authored by Matija Obreza's avatar Matija Obreza

Merge branch '465-document-migration-from-api-v0-to-v1' into 'master'

Resolve "Document migration from API v0 to v1"

Closes #465

See merge request genesys-pgr/genesys-server!451
parents df889fee da6a5802
......@@ -4,7 +4,7 @@
== Managing passport data
Passport data is based on the FAO/Bioversity Multi-crop Passport Descriptors
(http://www.bioversityinternational.org/e-library/publications/detail/faobioversity-multi-crop-passport-descriptors-v2-mcpd-v2/[MCPD V.2]) format.
(https://www.bioversityinternational.org/e-library/publications/detail/faobioversity-multi-crop-passport-descriptors-v21-mcpd-v21/[MCPD V.2.1]) format.
Accession records are *upserted*, meaning that:
......@@ -12,73 +12,79 @@ Accession records are *upserted*, meaning that:
. when it does not exist, a new record will be created.
Accession data in the database will be updated with whatever data is provided in the
request JSON.
request JSON. When a piece of information is not provided in JSON, it will not be modified.
If it is set to `null` it will be removed from the Genesys database.
=== Accession identity
Prior to full adoption of Permanent Unique Identifiers for germplasm, accessions could be
identified by the holding institute code (INSTCODE) and the accession number (ACCENUMB).
However, genebanks maintaining two or more collections of crops sometimes use identical
accession numbers in different collections.
Prior to full adoption of DOIs for germplasm, accessions could be
identified by the holding institute code (`instituteCode`) and the accession number (`accessionNumber`).
However, genebanks maintaining two or more crop collections sometimes use identical
accession numbers in separate crop collections.
For this reason, Genesys uses the *INSTCODE*, *ACCENUMB* and *GENUS* triplet to uniquely identify an
accession. For accessions with a digital object identifier ( https://en.wikipedia.org/wiki/Digital_object_identifier[DOI]) assigned in the ITPGRFA Secretariat Global Information System (
http://www.fao.org/plant-treaty/areas-of-work/global-information-system/en/[GLIS]), it will
use the DOI to uniquely identify the record.
For this reason, Genesys uses the *instituteCode*, *accessionNumber* and *GENUS* triplet to uniquely identify an
accession. For accessions with a digital object identifier (https://en.wikipedia.org/wiki/Digital_object_identifier[DOI]) assigned in the ITPGRFA Secretariat Global Information System (http://www.fao.org/plant-treaty/areas-of-work/global-information-system/en/[GLIS]), it will
use the `DOI` to uniquely identify the record.
.Identifiers in an accession JSON object
[source,json,linenums]
----
{
"instCode": "NGA039", <1>
"doi": "10.1054/12A1~", <2>
"acceNumb": "TMp-123", <3>
"genus": "Musa", <4>
"doi": "10.1054/12A1~", <1>
"instituteCode": "NGA039", <2>
"accessionNumber": "TMp-123", <3>
"taxonomy": {
"genus": "Musa", <4>
},
...
}
----
<1> Holding institute code (INSTCODE)
<2> DOI of the accession as registered in GLIS
<3> Accession number (ACCENUMB)
<1> DOI of the accession as registered in GLIS
<2> Holding institute code (instituteCode)
<3> Accession number (accessionNumber)
<4> Genus (GENUS)
=== JSON data model
The JSON data model of accession passport data closely follows
http://www.bioversityinternational.org/e-library/publications/detail/faobioversity-multi-crop-passport-descriptors-v2-mcpd-v2/[MCPD] definitions.
http://www.bioversityinternational.org/e-library/publications/detail/faobioversity-multi-crop-passport-descriptors-v21-mcpd-v21/[MCPD] definitions, but uses
human readable property names. Collecting data is grouped under `coll` key, taxonomic data
under `taxonomy` and geographic data under `geo` key.
By default, institutes in Genesys are configured to "Use unique accession numbers within the institute".
The accession JSON object must provide two identifying elements: `instCode` and `acceNumb`.
The accession JSON object must provide two identifying elements: `instituteCode` and `accessionNumber`.
In cases where accession numbers are not unique within the institute, `genus` is used to identify
In cases where accession numbers are not unique within the institute, `taxonomy.genus` is used to identify
the unique accession within the institute. In such a case the Accession JSON object must always provide three
identifying elements: `instCode`, `acceNumb` and `genus`.
identifying elements: `instituteCode`, `accessionNumber` and `taxonomy.genus`.
All other fields are optional.
.Example fields in an accession JSON object
[source,json,linenums]
----
{
"instCode": "XYZ111",
"doi": "10.1231/14S41Q",
"acceNumb": "M12345",
"instituteCode": "XYZ111",
"accessionNumber": "M12345",
"cropName": "banana",
"genus": "Musa",
"species": "acuminata",
"spauthor": "Colla",
"subtaxa": "var. sumatrana",
"subtauthor": "(Becc.) Nasution",
"orgCty": "NGA",
"acqDate": "20010301",
"mlsStat": true,
"taxonomy": {
"genus": "Musa",
"species": "acuminata",
"spAuthor": "Colla",
"subtaxa": "var. sumatrana",
"subtAuthor": "(Becc.) Nasution"
},
"origCty": "NGA",
"acquisitionDate": "20010301",
"mlsStatus": true,
"inTrust": false,
"available": true,
"historic": false,
"storage": [10, 20],
"storage": [ 10, 20 ],
"sampStat": 200,
"duplSite": "BEL084",
"bredCode": "NGA039",
"duplSite": [ "BEL084" ],
"breederCode": [ "NGA039" ],
"ancest": "PED/IG*REE",
"remarks": [ "remark1", "remark2" ],
"acceUrl": "https://my-genebank.org/accession/1",
......@@ -104,9 +110,9 @@ The `geo` field in accession JSON captures geographic data.
"latitude": 13.12312,
"longitude": 42.2321,
"elevation": 123.2,
"coordUncert": 2000.0,
"coordDatum": "WGS84",
"georefMeth": "GPS"
"uncertainty": 2000.0,
"datum": "WGS84",
"method": "GPS"
}
----
......@@ -143,7 +149,7 @@ The `coll` field in accession JSON captures collecting data.
"collSite": "7km south of Curitiba in the state of Parana",
"collNumb": "C90-23",
"collSrc": 12,
"collCode": "PER001",
"collCode": [ "PER001" ],
"collName": null,
"collInstAddress": null,
"collMissId": "C90"
......@@ -185,26 +191,28 @@ The `coll` field in accession JSON captures collecting data.
To reset or clear an existing value in the accession passport data, it should be provided
as `null`. Not providing a field means the field in the database should not be modified.
.Clearing the country of origin from an accession by sending a `null` value
.Clearing the country of provenance from an accession record by sending a `null` value
[source,json,linenums]
----
{
"instCode": "NGA039",
"acceNumb": "TMp-123",
"genus": "Musa",
"orgCty": null
"instituteCode": "NGA039",
"accessionNumber": "TMp-123",
"taxonomy": {
"genus": "Musa"
},
"origCty": null
}
----
=== Insert or update accessions
REST endpoint URL `/api/v0/acn/{instCode}/upsert` allows for inserting new accessions
REST endpoint URL `/api/v1/acn/{instituteCode}/upsert` allows for inserting new accessions
or updating existing records in Genesys. It accepts a JSON array of Accession JSON objects.
The array provides for sending batches of 50 or 100 accessions in one call, reducing
the HTTP overhead and improving performance.
NOTE: Only the `instCode` and `acceNumb` are required (and in some cases the `genus`).
The server will return an error when the `instCode` of JSONs does not match the `instCode` in the URL!
WARNING: Only the `instituteCode` and `accessionNumber` are required (and in some cases the `taxonomy.genus`).
The server will return an error when the `instituteCode` of JSONs does not match the `instituteCode` in the URL!
=== Deleting accessions
......@@ -214,7 +222,7 @@ also introduced the *accession archive*. The archive holds passport data for acc
that have been deleted from the active database.
REST endpoint URL `/api/v0/acn/{instCode}/delete` accepts an array of `instCode`, `acceNumb`, `genus` triplets
REST endpoint URL `/api/v1/acn/{instituteCode}/delete` accepts an array of `instituteCode`, `accessionNumber`, `genus` triplets
and deletes the corresponding accession records from Genesys. The *DELETE* permission is required for this operation.
NOTE: The delete operation will fail if C&E data exists for any accessions listed.
......@@ -223,19 +231,106 @@ NOTE: The delete operation will fail if C&E data exists for any accessions liste
.Deleting three accessions from the active database
[source,http,linenums]
----
POST /api/v0/acn/SYR002/delete
POST /api/v1/acn/SYR002/delete
[{
"instCode": "SYR002",
"acceNumb": "12345",
"genus": "Vicia"
"instituteCode": "SYR002",
"accessionNumber": "12345",
"taxonomy": { "genus": "Vicia" }
}, {
"instCode": "SYR002",
"acceNumb": "12345",
"genus": "Vicia"
"instituteCode": "SYR002",
"accessionNumber": "12345",
"taxonomy": { "genus": "Vicia" }
}, {
"instCode": "SYR002",
"acceNumb": "IG 1",
"genus": "Vicia"
"instituteCode": "SYR002",
"accessionNumber": "IG 1",
"taxonomy": { "genus": "Vicia" }
}]
----
=== Migrating from API v0
The *v1* API uses a more self-explanatory JSON schema, adopting the internal Genesys property names
in JSON de-/serialization, reducing the number of lines of code requried to parse and handle JSON
input and output.
While the change made to the Java code are easier to read and maintain (for us), it requires data providers (you) to adopt the new naming scheme and update your software.
The primary change is the move from `/api/*v0*/...` endpoints to `/api/*v1*/...`. API *v1* accepts and returns slightly different JSON compared to API *v0*.
WARNING: Upgrade to *v1* as soon as possible. The *v0* API endpoints will be made obsolete in February 2020.
NOTE: Genesys requires that `breederCode`, `duplSite`, `collCode` or `donorCode` use FAO WIEWS codes. Updating data will fail if existing or new records use invalid format (must be **XXX000** or **XXX0000**).
==== Core accession properties
[cols="1,1,3", options="header"]
.Updated core properties
|===
|v0
|v1
|Notes
|`instCode`|`instituteCode`|Holder institute code. Renamed for clarity.
|`acceNumb`|`accessionNumber`|Accession number. Renamed for clarity.
|`acqDate`|`acquisitionDate`|Date of acquisition. Renamed for clarity.
|`bredCode`|`breederCode`|FAO WIEWS code of the institute that bred the material. Renamed for clarity.
Accepts multiple values: `[ "XXX000", "YYY000" ]`
|`bredName`|`breederName`|Name of the institute that bred the material. Renamed for clarity.
Accepts multiple values: `[ "IITA", "CIMMYT" ]`
|`mlsStat`|`mlsStatus`|MLS Status.
|`orgCty`|`origCty`|ISO3166 code of provenance of material.
|===
==== Taxonomic data
All taxonomic data was in the root JSON in *v0* and is now grouped under
`taxonomy` property.
[cols="1,1,3", options="header"]
.Updated properties for taxonomic data
|===
|v0
|v1
|Notes
|`genus`|`taxonomy.genus`|
|`species`|`taxonomy.species`|
|`spauthor`|`taxonomy.spAuthor`|Note the uppercase `A`.
|`subtaxa`|`taxonomy.subtaxa`|
|`subtauthor`|`taxonomy.subtAuthor`|Note the uppercase `A`.
|===
==== Geographic data
Properties for geographic data under `geo` key are renamed:
[cols="1,1,3", options="header"]
.Updated properties for `geo` data
|===
|v0
|v1
|Notes
|`geo.coordUncert`|`geo.uncertainty`|Distance of uncertainty of coordinates in **meters**.
|`geo.coordDatum`|`geo.datum`|`WGS84` is most commonly used.
|`geo.georefMeth`|`geo.method`|Method of georeferencing, text. Use values `GPS`, `Map` or similar.
|===
==== Collecting data
Properties for collecting data in `coll` property are upgraded to support multiple values:
[cols="1,1,3", options="header"]
.Updated properties for `coll` data
|===
|v0
|v1
|Notes
|`coll.collCode`|Same|Accepts multiple FAO WIEWS codes: `[ "XXX000", "ZZZ0000" ]`
|`coll.collName`|Same|Accepts multiple values: `[ "collName1", "collName2" ]`
|`coll.collInstAddress`|Same|Accepts multiple values: `[ "collInstAddress1", "collInstAddress2" ]`
|===
......@@ -82,7 +82,7 @@ To add images to an accession you must first upload the image to Genesys with a
[source,bash]
----
$ curl 'https://sandbox.genesys-pgr.org/api/v0/img/XXX001/acn/ACC001/?originalFilename=IMG0012.jpg' -i -X PUT -H 'Content-Type: image/jpeg' -d @path/to/IMG0012.jpg
$ curl 'https://api.sandbox.genesys-pgr.org/api/v0/img/XXX001/acn/ACC001/?originalFilename=IMG0012.jpg' -i -X PUT -H 'Content-Type: image/jpeg' -d @path/to/IMG0012.jpg
----
The request `Content-Type` header must be set to the image content type (e.g.
......
......@@ -53,7 +53,7 @@ initiated by opening the authorization URL in a browser (please substitute `CLIE
with valid data):
----
https://sandbox.genesys-pgr.org/oauth/authorize?client_id=CLIENTID&client_secret=SECRET&response_type=code&redirect_uri=oob&scope=write
https://api.sandbox.genesys-pgr.org/oauth/authorize?client_id=CLIENTID&client_secret=SECRET&response_type=code&redirect_uri=oob&scope=write
----
The server will prompt you to authorize the access to your protected resources on Genesys.
......@@ -61,7 +61,7 @@ After your confirmation, the server will present an authorization code (which wi
From a shell, execute the following `curl` command, with your authorization code in place of `THECODE`:
----
$ curl 'https://sandbox.genesys-pgr.org/oauth/token?grant_type=authorization_code&client_id=CLIENTID&client_secret=SECRET&redirect_uri=oob&code=THECODE'
$ curl 'https://api.sandbox.genesys-pgr.org/oauth/token?grant_type=authorization_code&client_id=CLIENTID&client_secret=SECRET&redirect_uri=oob&code=THECODE'
----
==== Username and password
......@@ -71,7 +71,7 @@ applications) where the user's password will not be captured or stored -- i.e.
it will be used once. A password grant can be used as follows:
----
$ curl 'https://sandbox.genesys-pgr.org/oauth/token' --data 'scope=write+read&grant_type=password&username=USERNAME&password=PASSWORD&client_id=CLIENT_ID&client_secret=CLIENT_SECRET'
$ curl 'https://api.sandbox.genesys-pgr.org/oauth/token' --data 'scope=write+read&grant_type=password&username=USERNAME&password=PASSWORD&client_id=CLIENT_ID&client_secret=CLIENT_SECRET'
----
==== Client credentials
......@@ -93,7 +93,7 @@ To obtain a valid (but short-lived) access token, a simple `POST` request to
Genesys is required:
----
$ curl -X POST 'https://sandbox.genesys-pgr.org/oauth/token?grant_type=client_credentials&client_id=CLIENTID&client_secret=SECRET&redirect_uri=oob'
$ curl -X POST 'https://api.sandbox.genesys-pgr.org/oauth/token?grant_type=client_credentials&client_id=CLIENTID&client_secret=SECRET&redirect_uri=oob'
----
This type of grant takes out the *user* authentication and authenticates the
......@@ -132,7 +132,7 @@ or by including it in the request URL as a query string parameter:
[source,bash]
----
$ curl 'https://sandbox.genesys-pgr.org/api/v0/me?access_token=OAUTH-ACCESS-TOKEN'
$ curl 'https://api.sandbox.genesys-pgr.org/api/v0/me?access_token=OAUTH-ACCESS-TOKEN'
----
=== Using the refresh token
......@@ -156,7 +156,7 @@ The refresh token must be securely persisted and can be used to request a new ac
when the previous access token expires:
----
$ curl 'https://sandbox.genesys-pgr.org/oauth/token?grant_type=refresh_token&client_id=CLIENTID&client_secret=SECRET&redirect_uri=oob&refresh_token=OAUTH-REFRESH-TOKEN'
$ curl 'https://api.sandbox.genesys-pgr.org/oauth/token?grant_type=refresh_token&client_id=CLIENTID&client_secret=SECRET&redirect_uri=oob&refresh_token=OAUTH-REFRESH-TOKEN'
----
......
......@@ -13,9 +13,9 @@ December 2015: Documentation commit {buildNumber}
All API access is over HTTPS, and accessed from the https://www.genesys-pgr.org domain or
through https://sandbox.genesys-pgr.org for testing purposes. All data is sent and received as JSON.
In this manual, all URLs are pointing to the Genesys sandbox environment at https://sandbox.genesys-pgr.org.
All API access is over HTTPS, and accessed from the https://api.genesys-pgr.org domain or
through https://api.sandbox.genesys-pgr.org for testing purposes. All data is sent and received as JSON.
In this manual, all URLs are pointing to the Genesys sandbox environment at https://api.sandbox.genesys-pgr.org.
include::api/security.adoc[]
......
......@@ -14,9 +14,9 @@ March 2017: Documentation commit {buildNumber}
Genesys implements basic <<brapiv1, BrAPI v1 specification>>
All API access is over HTTPS, and accessed from the https://www.genesys-pgr.org domain or
through https://sandbox.genesys-pgr.org for testing purposes. All data is sent and received as JSON.
In this manual, all URLs are pointing to the Genesys sandbox environment at https://sandbox.genesys-pgr.org.
All API access is over HTTPS, and accessed from the https://api.genesys-pgr.org domain or
through https://api.sandbox.genesys-pgr.org for testing purposes. All data is sent and received as JSON.
In this manual, all URLs are pointing to the Genesys sandbox environment at https://api.sandbox.genesys-pgr.org.
include::brapi/auth.adoc[]
......
......@@ -10,7 +10,8 @@ Authorization: Bearer <TOKEN>
=== Login
Genesys does not support the `/login` endpoint.
Genesys does not support the `/login` endpoint. You must obtain the OAuth access token as described
in Genesys API manual.
=== Logout
......
......@@ -34,7 +34,7 @@ Genesys returns the standard BrAPI Germplasm response:
},
"result": <1> [{
"defaultDisplayName": "DJUG 16",
"germplasmPUI": null,
"germplasmPUI": "10.0000/ABCD*",
"pedigree": null,
"seedSource": null,
"synonyms": ["DJUG 16"],
......
......@@ -46,6 +46,7 @@ public class BrAPIServiceImpl implements BrAPIService {
Germplasm g = new Germplasm();
g.setUuid(accession.getUuid());
g.setGermplasmPUI(accession.getDoi());
g.setAcceNumb(accession.getAccessionNumber());
g.setDefaultDisplayName(accession.getAccessionNumber());
g.setAcceName(accession.getAccessionNumber());
......
......@@ -138,7 +138,7 @@ public class AccessionControllerTest extends AbstractApiTest {
public void beforeTest() throws Exception {
super.beforeTest();
mockMvc = MockMvcBuilders.webAppContextSetup(webApplicationContext).apply(documentationConfiguration(this.restDocumentation).uris().withScheme("https").withHost(
"sandbox.genesys-pgr.org").withPort(443)).build();
"api.sandbox.genesys-pgr.org").withPort(443)).build();
institute = new FaoInstitute();
institute.setCode("INS001");
institute.setFullName("An institute");
......
......@@ -92,7 +92,7 @@ public class CropsControllerTest extends AbstractApiTest {
super.beforeTest();
mockMvc = MockMvcBuilders.webAppContextSetup(webApplicationContext).apply(documentationConfiguration(this.restDocumentation).uris()
.withScheme("https")
.withHost("sandbox.genesys-pgr.org")
.withHost("api.sandbox.genesys-pgr.org")
.withPort(443)).build();
crop = cropService.addCrop("maize", "Maize", "Crop description in EN", null);
......
......@@ -84,7 +84,7 @@ public class ApiCropsTest extends AbstractApiTest {
@Before
public void setUp() {
mockMvc = MockMvcBuilders.webAppContextSetup(webApplicationContext)
.apply(documentationConfiguration(this.restDocumentation).uris().withScheme("https").withHost("sandbox.genesys-pgr.org").withPort(443)).build();
.apply(documentationConfiguration(this.restDocumentation).uris().withScheme("https").withHost("api.sandbox.genesys-pgr.org").withPort(443)).build();
crop = cropService.addCrop("maize", "Maize", "Crop description in EN", null);
assertThat(crop.getId(), greaterThan(0l));
......
......@@ -117,7 +117,7 @@ public class ApiImagesDocsTest extends AbstractApiTest {
@Before
public void setUp() throws InterruptedException {
mockMvc = MockMvcBuilders.webAppContextSetup(webApplicationContext)
.apply(documentationConfiguration(this.restDocumentation).uris().withScheme("https").withHost("sandbox.genesys-pgr.org").withPort(443)).build();
.apply(documentationConfiguration(this.restDocumentation).uris().withScheme("https").withHost("api.sandbox.genesys-pgr.org").withPort(443)).build();
List<FaoInstitute> institutes = new ArrayList<>();
institute = new FaoInstitute();
......
......@@ -72,7 +72,7 @@ public class ApiRequestsDocsTest extends AbstractApiTest {
@Before
public void setUp() {
mockMvc = MockMvcBuilders.webAppContextSetup(webApplicationContext)
.apply(documentationConfiguration(this.restDocumentation).uris().withScheme("https").withHost("sandbox.genesys-pgr.org").withPort(443)).build();
.apply(documentationConfiguration(this.restDocumentation).uris().withScheme("https").withHost("api.sandbox.genesys-pgr.org").withPort(443)).build();
Collection<FaoInstitute> institutes = new ArrayList<>();
institute = new FaoInstitute();
......
......@@ -81,7 +81,7 @@ public class BrAPITest extends AbstractApiTest {
@Before
public void setUp() {
mockMvc = MockMvcBuilders.webAppContextSetup(webApplicationContext)
.apply(documentationConfiguration(this.restDocumentation).uris().withScheme("https").withHost("sandbox.genesys-pgr.org").withPort(443)).build();
.apply(documentationConfiguration(this.restDocumentation).uris().withScheme("https").withHost("api.sandbox.genesys-pgr.org").withPort(443)).build();
crop = cropService.addCrop("maize", "Maize", "Crop description in EN", null);
assertThat(crop.getId(), greaterThan(0l));
......
......@@ -135,7 +135,7 @@ public class AccessionControllerTest extends AbstractApiTest {
public void beforeTest() throws Exception {
super.beforeTest();
mockMvc = MockMvcBuilders.webAppContextSetup(webApplicationContext).apply(documentationConfiguration(this.restDocumentation).uris().withScheme("https").withHost(
"sandbox.genesys-pgr.org").withPort(443)).build();
"api.sandbox.genesys-pgr.org").withPort(443)).build();
institute = new FaoInstitute();
institute.setCode("INS001");
institute.setFullName("An institute");
......
......@@ -64,7 +64,7 @@ public class AsciiDocControllerTest extends AbstractApiTest {
/*@formatter:off*/
mockMvc = MockMvcBuilders.webAppContextSetup(webApplicationContext)
.apply(documentationConfiguration(this.restDocumentation)
.uris().withScheme("https").withHost("sandbox.genesys-pgr.org").withPort(443))
.uris().withScheme("https").withHost("api.sandbox.genesys-pgr.org").withPort(443))
.build();
/*@formatter:on*/
......
......@@ -107,7 +107,7 @@ public class CropsControllerTest extends AbstractApiTest {
super.beforeTest();
mockMvc = MockMvcBuilders.webAppContextSetup(webApplicationContext).apply(documentationConfiguration(this.restDocumentation).uris()
.withScheme("https")
.withHost("sandbox.genesys-pgr.org")
.withHost("api.sandbox.genesys-pgr.org")
.withPort(443)).build();
crop = cropService.addCrop("maize", "Maize", "Crop description in EN", null);
......
......@@ -78,7 +78,7 @@ public class UserRegistrationControllerTest extends AbstractApiTest {
public void beforeTest() throws Exception {
super.beforeTest();
mockMvc = MockMvcBuilders.webAppContextSetup(webApplicationContext).apply(documentationConfiguration(this.restDocumentation).uris().withScheme("https").withHost(
"sandbox.genesys-pgr.org").withPort(443)).build();
"api.sandbox.genesys-pgr.org").withPort(443)).build();
}
@Test
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment