Ingest: Searchable Amphibian dataset
With the Preview
fully mapped and annotated, a searchable Amphibian Table
for this source file can be created.
Depends on
Ingesting
-
Need dryRunFromPreview(uuid, mapping, Descriptor)
that scans over the column of aPreview
for compatibility with the selectedDescriptor
and mapping- This checks that all distinct values are OK
- Returns a list of anything that doesn't fit, so it can be remapped
-
ingestFromPreview(uuid, mapping, Descriptor)
- Will ingest the column data
- Needs
accessionUuid
, meaning it needs to come with thePreview
so we canupsert
the column data
-
Should add ref: { a: uuid, instCode: string, acceNumb: string, genus: string }
at some point to thePreview
itself- Basically at the time we create
accessionRefs
on the Dataset - Instead of calling
Dataset.addRefs()
and making use of the background processing to link it up, we add full refs -- when possible -- toPreview
and theDataset
. - But it needs to be on the exact row where INSTCODE, ACCENUMB, GENUS and DOI were found
- So: load batches of
rowId, cI, cA, cG, cD
, link em up and put them back. - ... so that during
ingest
we query forcXX, ref as accessionRef
and have the value and the key to upsert in the dataset
- Basically at the time we create
- We never put passport data in the dataset, only the
ref
.
Should consider maintaining all 5M accessionRef
in Amphibian? Tough sell.
Querying
- List of descriptors in the dataset: what's the link to what's actually ingested?
-
sources
is amap<descriptor, meta>
-
sources.keys()
lists descriptors in the dataset
-
-
Fetch Descriptors
forsources.keys()
-
Render filters