crate 0.19.0 (2026-11-17)
-
nge/habitat/fwa/v1/segment, aschema_onlyentry for link’s habitat export: one row per stream segment, link config and species, keyed by FWA position, with the habitat flags, the segment’s size and gradient, their ratios to the build’s thresholds, and the species’ access codeaccess(0, 1, 2, -9), which link adds to v1 as an optional column so a part published before it still conforms. The YAML is link’s, moved byte for byte (NewGraphEnvironment/link#333, NewGraphEnvironment/link#423); the reasoning is indecisions/nge/20261117_habitat_segment.md.
crate 0.18.0 (2026-11-16)
- The five fish passage form schemas list the versions crews captured records into from 2023 on, read from the sync service’s version history, so
crt_schema_variant()names a 2023–2025 record’s version andcrt_schema_conform()no longer warns that its fingerprint is unknown: 7 moreupstream_variants(form_pscis2,form_fiss_site3,form_monitoring2), each with a description of how its columns differ from the build’s. A column set counts if a file in that shape gained a record: under a new key, with content the version before it did not hold, so processed copies and duplicated rows are not listed. Columns only earlier versions carry are declared, typed, with avariantskey naming the versions that have them, whichcrt_schema_cols()returns as a new list column so a form builder can leave them out.form_monitoring’ssite_idis now a string, because a 2024 version holds site names there, andcapture-utm-columns-2025’sfirst_seenis the measured 2024-09. The method and counts are inresearch/form_versions.md(#36).
crate 0.17.0 (2026-11-16)
- Two
schema_onlyentries for photos, which a caller conforms to withcrt_schema_conform().nge/photo_fileshas one row per file, keyed by(checksum, path):checksumis the SHA-256 of the file’s bytes, the digest a receipt carries for each part, so a resized copy that keeps its original’s name is a row of its own, and itsderived_fromholds the original’s checksum.timeis a UTC instant only where the camera wrote an offset; the wall clock is intime_localand the offset intime_offset, so a local time is never read as UTC. Position isgps_lonandgps_lat.nge/photo_linkssays what each photo shows: a form record by its full key(project, form, source_version, record_id), or a site and view. The gradesidentifiedandrecord(crt_schema_grade()) say which rows are complete enough to use. The reasoning is indecisions/nge/20261116_photo_tables.md(#67).
crate 0.16.0 (2026-11-11)
-
crt_receipt_stac()maps a receipt to the fields a STAC item mirrors from it:receipt:recipe_id,processing:datetime,processing:software, and each asset’sfile:checksumandfile:size. It executes the receipt mapping’sSTAC itemrows, which until now nothing in crate ran. Mapping v1.1.0 (inst/extdata/receipt/mapping/v1.1.0/, published athttps://newgraphenvironment.github.io/crate/receipt/mapping/v1.1.0/mapping.csv) states what v1.0.0 left open. A migrated receipt leavesreceipt:recipe_idout, a missing one on any other receipt is refused, and a nullversionis left out ofprocessing:software. A part no asset names is not mirrored, and an asset whose href names no part is refused. The asset that links the receipt carriesfile:checksum, the digest of the published receipt’s bytes, and nofile:size. Each refusal is an error of classcrate_stac_refusedcarrying acode.stac.jsonbeside the mapping holds 142 cases, every path the mapping reads crossed with every state, each with the item fields it maps to or the code of its refusal, for every implementation to run (#59). -
Breaking:
crt_receipt_write()returns the SHA-256 of the bytes it wrote, invisibly, rather than the receipt. That digest is what an item’s receipt asset carries asfile:checksum, and nothing else can compute it. To read the receipt, usecrt_receipt_read(), which returns exactly what was written; the listcrt_receipt_write()returned heldNAwhere the file holdsnull(#60). - The note on
form_capture’sharvested_atcolumn (inst/extdata/schemas/nge/form_capture.yaml) describes an internal issue in words rather than naming it. No code change.
crate 0.15.1 (2026-11-09)
- The README, which is also the pkgdown home page, is now the opening, one example of each of crate’s jobs with its real output, and pointers: Learn more, Ecosystem, Citation and License. How a registered file is read (the runtime, datetimes as instants, and how a variant is matched) moved to a new
vignette("reading"), and how to register a new file or table toinst/extdata/schemas/README.md. The DESCRIPTIONTitleis now “Canonical Schemas for Changing Data and Provenance Receipts”. No code change (#54).
crate 0.15.0 (2026-11-09)
- The receipt register and mapping are versioned on their own, apart from the receipt’s form.
register.csvandmapping.csvmoved toinst/extdata/receipt/register/v1.0.0/andinst/extdata/receipt/mapping/v1.0.0/(rows unchanged), and are published athttps://newgraphenvironment.github.io/crate/receipt/register/v1.0.0/register.csvand.../receipt/mapping/v1.0.0/mapping.csv, with a changelog at.../receipt/CHANGELOG.mdstating the rules.compare.jsontests the register’s actions, so it moved fromvectors/v1.0.0/toregister/v1.0.0/, byte for byte. A receipt’sreceipt_versionstill names its form only, so a register or mapping change never moves it.crt_receipt_compare()now returnsregister, the version the verdict was judged under, andoverrides, the action a package’s declaration gave each field it changed;print()names both. A released contract file can no longer change: the pkgdown workflow compares each with the first release tag that holds it (#58).
crate 0.14.1 (2026-11-08)
- The README, which is also the pkgdown home page, now names both of crate’s jobs, canonical schemas for data that changes shape and provenance receipts for outputs, and has a Provenance receipts section that writes a receipt and compares two builds. The receipt vignette says a subclass of
DateorPOSIXct, such as data.table’sIDate, is spelled as its base class. The README and the vignette no longer say that stacs implements receipts, which is still planned (stacs#12). No code change (#52).
crate 0.14.0 (2026-11-06)
-
crt_table_fingerprint()fingerprints a subclass ofDateorPOSIXct, such as data.table’sIDate, as its base class, so a table read withdata.table::fread()gets the same fingerprint as the same table read any other way, incrt_receipt_write()and incrt_receipt_compare()’s re-read alike. The class must end in the base class, on numeric storage; a column carrying an extra attribute is still refused, and every plainDateandPOSIXctcolumn keeps its canonical form (#49).
crate 0.13.0 (2026-11-06)
-
crt_receipt_write(file =)names the receipt insidedir(defaultreceipt.json), so outputs that share a directory keep one receipt each. A name with a path separator, or not ending.json, is refused, and so is one the filesystem resolves to a file the receipt hashes (a part, a config file or the lockfile), which the write would otherwise replace (#47).
crate 0.12.1 (2026-11-06)
- The package title is now “Canonical Schemas and Data Governance for Changing Data”, saying what crate is for rather than whose it is (#45).
crate 0.12.0 (2026-11-06)
crate holds the provenance receipt: one receipt.json per output, in one shape any language can write, saying what the output is, how it was made, what came out, and where and when. Any rebuild can then be compared with its reference copy and answered identical, equivalent within a declared tolerance, or different, naming each field that differed (#43). See vignette("receipt").
- The contract: a JSON Schema published at
https://newgraphenvironment.github.io/crate/receipt/v1.0.0/schema.json;recipe_id, a SHA-256 over the RFC 8785 canonical JSON of the recipe; a table fingerprint over a declared canonical form; and test vectors ininst/extdata/receipt/vectors/v1.0.0/that every implementation runs (Python’s, in stacs, among them). -
crt_receipt_write()computes the ids, fingerprints and digests and writes the receipt;crt_receipt_read()andcrt_receipt_validate()read and check one;crt_receipt_keys()reads a package’s declaration of its recipe, tables and tolerances. -
crt_receipt_compare()gives the verdict, deciding what a difference in each field does from a per-field register (inst/extdata/receipt/register.csv). Supplied with both builds’ tables, it re-reads them, checks each against its receipt, and accepts cells within tolerance. -
crt_table_fingerprint()hashes a table’s values;crt_schema_fingerprint()still hashes its column names and types. -
crt_receipt_migrate()reads link’smanifest.jsonand floodplains’provenance.jsonas receipts through a mapping table that also lists STAC’s keys. - A
receipt:STAC field schema (receipt:recipe_id) is published besidelidarbc:andlas:.
crate 0.11.0 (2026-10-31)
nge/fish_data reads NGE individual fish data from any season to one set of 29 columns with the same types, so seasons stack and a PIT-tagged fish can be followed between them (#6). It is the first nge entry crt_ingest() reads: crt_ingest("nge", "fish_data", path) takes the 2023 individual fish files and the 2024 onward field sheet. Names follow the provincial FISS individual fish template and the field form (species, length_mm, weight_g, haul_number_pass_number), 2023 PIT tags written into comments are read into pit_tag_id, and date is an ISO day. See the decision entry.
crate 0.10.0 (2026-10-31)
- The schemas README now names the data consolidation umbrella (#38) as the home of the QC rules that will read
valuesandrange, in place of the issue it replaced. No code change.
crate 0.9.0 (2026-10-31)
The five fish passage field forms have canonical schemas: nge/form_pscis, form_fiss_site, form_monitoring_fish_passage, form_monitoring_general and form_monitoring (#13). Each says what every form column means: its type, units, allowed values, range, what a photo path is relative to, and what a null means. The meaning is taken from the field definitions the forms are built from, by data-raw/forms_derive-schemas.R. Each schema also lists the form’s known versions with the fingerprint a capture of each produces. See the decision entry.
-
crt_schema_fingerprint()is now the one definition of a capture’sschema_version. It hashes the joined string’s bytes rather than R’s serialization of it, so tables harvested before this release carry different values until they are harvested again. -
crt_schema_cols(),crt_schema_grade()andcrt_schema_variant()return a schema’s column meanings, which records meet each of its grades, and which known version each record was captured with. -
crt_schema_conform()warns, with classcrate_unknown_fingerprint, on a fingerprint the schema does not list. -
crt_schema_read()no longer drops part of a schema that contains non-ASCII text when R runs in a C locale. -
citation("crate")andCITATION.cffcarry the package’s Zenodo concept DOI, which resolves to every archived release.
crate 0.8.0 (2026-10-20)
nge/discharge/fwa/v1/segment_daily declares the columns of the daily discharge wet publishes beside segment_monthly (NewGraphEnvironment/wet#99): one row per FWA segment, source and day of a 365-day common year, keyed by doy in place of month. Its description says what the days are for each source, and how they relate to the published months (#33).
-
segment_monthly’speriodnote now says both sources are 1981-2010, as they have been since NewGraphEnvironment/wet#103.
crate 0.7.0 (2026-10-12)
nge/discharge/fwa/v1/segment_monthly and nge/discharge/fwa/v1/watershed_monthly declare the columns of the per-segment and per-watershed discharge that wet publishes (NewGraphEnvironment/wet#7). They are the first entries keyed by a dataset ID, <variable>/<network>/<version>/<unit>_<timebase>, the same string that names the data’s S3 prefix and Postgres table (#31).
- A registry row’s schema must now sit at
schemas/<source>/<file_name>.yaml; every existing row already did. See the decision entry.
crate 0.6.1 (2026-10-12)
Every PR and every push to main now runs R CMD check and the whole test suite (#11). Before this, nothing ran the suite in CI.
- The check fails on warnings. Every run prints its skipped tests, and a skip for a reason other than the six expected ones adds a warning to the run summary.
- Those six tests need the source tree, which
R CMD checkdoes not have, so a second pass runs the suite from the checkout, where any skip fails the run. -
CITATION.cffis no longer in the built package.
crate 0.6.0 (2026-10-11)
lidarbc v1.1.0 adds lidarbc:datetime_unknown, a boolean that is true when neither LidarBC’s file name nor its path carries a date, so the item’s datetime is a placeholder. stac-elevation-bc’s 2,288 albers items will carry it once their rewrite (NewGraphEnvironment/stac_dem_bc#2) publishes. The schema is at https://newgraphenvironment.github.io/crate/stac/lidarbc/v1.1.0/schema.json (#26).
- v1.0.0 is unchanged and stays published for the items that name it. This is the first version added under the never-edit rule.
crate 0.5.0 (2026-10-10)
crate now publishes JSON Schemas for the custom fields NGE’s STAC catalogues write, starting with lidarbc: (LidarBC’s objectstore paths and file names) and las: (the LAS format’s version and point format), at https://newgraphenvironment.github.io/crate/stac/<prefix>/v1.0.0/schema.json (#23).
- An item lists the URL in
stac_extensions, and pystac then checks each field’s type: a point format of 11 or an unknownlidarbc:field fails validation. - A released version is never edited. A change is a new version beside it, and the site build fails if a version already on
gh-pageswould change or disappear. See the decision entry.
crate 0.4.0 (2026-09-02)
nge/track_sessions declares the four columns a crew fills in when they stop recording — track_name, track_type, track_description, named_by — and nge/track_annotations becomes the table of overrides for them.
- The four are optional strings. An rfp-deployed tracking layer carries them (new upstream variant
mergin-tracking-rfp-2026-08); a layer the plugin created on its own, or a GPX import, does not, and still conforms. A layer carrying some but not all is drift the reader must abort on — crate sees a frame, not a variant. -
track_annotationsnow carriestrack_name,track_typeandtrack_description, same names and types as the captured columns, and dropsnotes. The rule, stated once: a non-key annotation column sharing a captured column’s name overrides it where non-NA,coalesce(annotation, captured). A reader that assigns instead of coalescing blanks every unannotated session, which is why the rule is written down rather than left to each consumer. -
track_typeis an open string; the value list is the capture tool’s convention, recorded in the variant description and not enforced. See the decision entry for why captured beats annotated and why the annotation table was reshaped rather than extended.
crate 0.3.0 (2026-09-01)
nge/form_capture gains source_version, the upstream version a record was captured at.
- Deliberately a column rather than a consumer-side manifest field, which is the opposite of the usual rule. That rule exists because stamping the upstream’s current version on every row churns a snapshot’s bytes for reasons unrelated to the data; a per-row capture version is fixed at capture and never moves. It is also what keeps
(project, form, source_version, record_id)unique once a rebuild has restartedrecord_idat 1. -
schema_versionnow documents how it is computed —<n>-<digest>over sortedname:typepairs of the capture’s own columns, hashing a joined string rather than a vector. Two producers computing it differently would give two genuinely different vintages the same version by accident, so the rule belongs with the declaration rather than with each caller.
crate 0.2.0 (2026-09-01)
New schema_only entry nge/form_capture: the envelope every captured field-form record carries — project, form, record_id, schema_version, captured_at — and nothing about what any observation means.
- Vintage is a column, not a filename. Two shapes of one form are one logical table, distinguished by
schema_version; a column absent from the older vintage is null there andschema_versionsays why, which is what separates “not asked” from “not answered”. - Per-form column meaning is still #13. The envelope leans on
crt_schema_conform()’s documented floor-not-ceiling behaviour, so a form’s own columns pass through untouched until #13 defines them. See the decision entry for why an envelope rather than waiting, and whyharvested_atis not a column.
crate 0.1.0 (2026-08-24)
crate can now declare a canonical shape for data it does not read, and can type a column as an instant.
-
crt_schema_conform()takes a data frame you already have and holds it to a registered canonical schema.crt_ingest()calls it rather than reimplementing validation and typing, so a file and a caller-supplied frame get identical treatment. - New canonical type
datetime. It never moves an instant: aPOSIXcthas its timezone attribute stamped to UTC rather than converted, because several readers return correct UTC instants carrying no timezone label and “correcting” the rendering injects a real whole-hour error into every join. - The registry gains a
kindcolumn.schema_onlyentries declare a shape crate does not read;crt_ingest()aborts on them namingcrt_schema_conform(). - First
schema_onlyentries:nge/track_sessions,nge/track_verticesandnge/track_annotations, the canonical shape of a GPS track. See the decision entry for why a track is two tables and why annotations are a third. -
test-crt_registry_integrity.Rwalks the registry and the schemas directory rather than a hand-written list, so a row without a YAML, a YAML without a row, or a declared type nothing implements fails the suite.
crate 0.0.2 (2026-04-29)
Schema-as-contract scope settled and Convention C naming locked in (see crate#4).
- Every function in the crate namespace now prefixed
crt_*, family-namespaced (crt_schema_*,crt_registry_*,crt_handler_*). - New
crt_schema_*family:crt_schema_read()(load YAML by relative path),crt_schema_apply()(canonical type enforcement),crt_schema_validate()(required-cols enforcement). -
crt_ingest()now validates required cols and enforces canonical types after handler dispatch — schema YAML is the single source of truth.
