Skip to contents

default’s presence table is bcfishpass’s copy plus an overlay, a bundle can exclude a record bcfishpass has not listed, and the presence groups are bundle data (#472, backlog move 4). No species cell moves, so no output token moves: default stays acc-v6 and bcfishpass acc-bcfp-5. The issues expected a new acc row; none is needed, because the loaded presence cells, the groups and the record rule are the same values.

  • The presence table is a synced copy (#431). default’s overrides/wsg_species_presence.csv is now bcfishpass’s own (smnorris/bcfishpass@3bd2824), which the weekly sync checks as it does the bcfishpass bundle’s. Its 122 departures are rows of overrides/wsg_species_presence_overlay.csv, each with its reason: the #402 review’s 110 added and 10 removed cells, UTRE CH and LARL GR kept absent, plus the four ko cells. A reason carries the review’s decision, its rule and the opening of the tracker’s story. data-raw/logs/presence_overlay_431/build_overlay.R made them and stops unless the species cells come back exactly; the loaded default and default_tuned tables digest identical before and after in every species column, order and type. Only notes moves to the copy’s text: link’s notes there had pointed at the tracker.
  • A record bcfishpass has not listed can be excluded (#436). default’s overrides/observation_exclusions_link.csv holds the three records the review found wrong (GUIC CH and MESI SK misplaced, KOTL ST above Bonnington Falls), in the copy’s columns. .lnk_observation_exclusion_keys() reads both tables, so the pipeline, the validator and the presence check exclude them alike, and the sync names a row once upstream lists it. Only GUIC’s moves a model (MESI SK and KOTL ST are absent cells there): built before and after, GUIC’s chinook turns from 2 to 1 on 227 segments (52.3 km), and nothing else moves. ADMS and BULK are identical in every persisted table (data-raw/logs/presence_overlay_431/).
  • The presence groups are bundle data (#292). presence_groups.csv in default and bcfishpass holds lnk_presence()’s two groups (salmon; CT, DV, RB), under config_hash, and every pipeline caller and the acc spec read it. A bundle with none, default_extrabreaks and default_rearbreaks among them, keeps lnk_presence()’s default.
  • The presence check after the overlay flags 233 cells, down from 351 (data-raw/logs/presence_overlay_431/presence_check/). Rainbow trout’s return to default, which its three pending rows wait on, is #497.
  • config_hash moves for default, default_tuned, default_minimal, default_parent and bcfishpass, so earlier builds read as another bundle in recompute_readers_check.R and the export until re-modelled; every group already owes one. RUNBOOK §5 and §7. The pre-run gate runs again before the next provincial run (R/ and inst/extdata/ changed).

The habitat export carries the access code, its contract is crate’s, PUBLISH takes only builds a consumer can name, and a one-group run refuses a downstream group it cannot see (#473, backlog move 5).

  • The export carries access (#423): the species’ access code from streams_access per segment, config and species (0 blocked below, 1 accessible, 2 accessible with the species recorded upstream), so a reader can map known against suspected fish. It is added to v1 as an optional column. lnk_db_load() reads each tree’s columns from its manifest, so the tree published on 2026-11-14 still loads (with access NULL), and a table loaded before gains the column. Each config in the manifest summarises its builds’ tokens (configs[].tokens), so a reader knows which acc rule made code 2.
  • The column contract is crate’s nge/habitat/fwa/v1/segment (#333), moved byte for byte. The export and its readers need crate 0.19.0 (they ask for it when called, so modelling on a host with an older crate is not stopped), and the manifest cites the file at its release tag. exported_by adds crate_version, and its contract_md5 is now an md5 over the columns’ names and types.
  • PUBLISH refuses builds a floodplain item could not name or rebuild (#334): a build with no run_uid, from a dirty link or fresh tree, or with no thresholds logged, a recompute with no run_uid or from a dirty tree, or a config exported from a dirty tree. LNK_PUBLISH_ALLOW="<reason>" publishes past them, and the manifest records the reason. PUBLISH also checks that the bucket’s versioning is on and writes each part’s S3 VersionId into the manifest. lnk_catalog() returns version_id, and lnk_db_load() fetches that version. Each run now logs a digest of its persisted streams (log.streams_md5), and an export stops where the rows it reads are not that build’s. Public read is #334’s one open item.
  • lnk_wsg_downstream_check() refuses a downstream group with no FWA streams (#464). Before, a database without the groups below the outlet found no dams there and passed. It now names those groups (wsgs_unloaded), and fails, warns or records an override as for an unmodelled dam. A closure that errors refuses rather than passing.
  • The check snaps only the dams near the downstream path (#461): CABD dams within 65 m of the downstream groups’ polygons. #461’s 371 s did not reproduce on m4, even on a six-group database built by the cloud sessions’ loader, where the check takes 0.2 s. Without a stream geometry index, though, the old check takes 273 s on HARR and the new one 8 s. On all 246 groups the results are identical before and after. Its result carries each step’s seconds (steps_s), and wsg_run_one.R and wsg_recompute_one.R print them.
  • RUNBOOK §6d and §8c. The pre-run gate runs again before the next provincial run (R/ changed); the next PUBLISH is the operator’s.

One rule says which fish records count, and each lift is published with the records behind it (#470, backlog move 2). Records now count the same way for the observation cuts, code 2 (“accessible, the species recorded upstream”) and the barrier lifts, and the rule is data in the bundle. default is cut-full-top-2 acc-v6, bcfishpass is cut-full-ssf-all-2 acc-bcfp-5; their predecessors retire at this release, and every group re-models at the next provincial run.

  • Alias codes count as their species (#398, #425). default’s species_pooling.csv lists them at a new province scope: SST and WST as ST, ADV as DV, ABT as BT, CCT, ACT and CT/RB as CT. BT/DV and DV/BT count as BT only where the tracker pools DV into BT, and RB/CT counts as nothing. Before, the cutthroat aliases cut streams but never made CT read 2 (the walk tested exact codes), and the steelhead and char aliases counted nowhere.
  • Which records count is one rule (#400). .lnk_observation_species_map() resolves it per group and the observation tables carry the model species, so the cuts, the walk and the lifts read it unchanged. Under default (pipeline: observation_record_date: lift_floor) salmon, steelhead, GR and KO records need 1990-01-01 or later for code 2 and the cuts as for a lift; resident species count every record, undated included. DV counts as BT only where the tracker pools it, for lifts too, so BT’s lift lists BT. The presence check and its measures read the same aliases.
  • The bcfishpass bundle counts as bcfishpass does. bcfishpass’s filter tests a record’s own code, so the cutthroat aliases never reach bcfishpass.observations (0 of 2,004 alias records in SQAM, LFRA and VICT appear in its upstream arrays); link now drops them too. Its salmon and steelhead lifts take records dated after 1990-01-01, as bcfishpass does (#426); measured on ten groups that moves no observation lift.
  • default lifts a natural barrier on 2 salmon records (#427): CH, CM, CO, PK, SK, ST, GR and KO need 2 records since 1990, not 5. On SQAM, LSKE and HARR that adds 4, 6 and 8 lifts per salmon species (SQAM steelhead 24), each on at least two records; 1 record would have added 55, 14 and 37, most on a single record (data-raw/logs/record_rule_470/lifts_427.csv).
  • Each lift is published with what it rests on (#434). lnk_barrier_overrides(evidence =) writes, and the pipeline persists, <persist>.barrier_override_evidence: one row per lifted barrier, species and record (its key, the code it is filed under, its date and its own group) with the rule as applied, or the habitat row behind a habitat lift. study_area_verify.sql holds it equal to barrier_overrides. A study scoring records against access can recount a lift without the record being scored.
  • Measured before and after on eight default groups and three bcfishpass groups (data-raw/logs/record_rule_470/). Where DV no longer counts as BT, BT loses lifts and access: LSKE 61 of 61 lifts and 216 km, KLUM 119 of 123 and 168 km, SQAM 147 of 160 and 254 km. Where pre-1990 salmon records stop counting, “recorded upstream” km fall (LSKE 1,172 km across six species, KLUM 424, HOMA 366); aliases add some (FINA BT 18 km, HOMA ST 10 km). bcfishpass access did not move, and its SQAM segments fall where the CT alias cuts go. Every changed piece is traced to a record one rule counts and the other does not.
  • The cut spec carries the record rule where observations cut, so a tracker row that moves which records cut is a new token. #236 is narrowed to several lift rules per species, which stays open.
  • RUNBOOK §7, “The record rule”; §5 has the lift floors and the evidence table. The pre-run gate runs again before the next provincial run (R/ changed).
  • The README’s Ecosystem table renders from the shared registry (#492), so the list of sibling packages stays in step with the other repos’ READMEs.

A stalled statement leaves a record of what it was waiting on (#487). frs_near_river_walk stalled on the dispatcher on four dates and each time the backend was cancelled at the statement bound before anyone read pg_stat_activity. No modelling output changes and no vocabulary token moves.

  • Each group is watched from a second connection. wsg_run_one.R starts data-raw/wsg_stall_sample.R beside the group. While the group’s backend has been on one statement longer than --stall-sample-after minutes (default 10, or half the statement bound when that is shorter), every 60 s or more often it writes one line per backend into the group’s log: the statement, its parallel workers and each session pg_blocking_pids() names, with their state, wait event, start time and query, the host’s load average and the database’s active backends. It ends with its group and cannot fail it.
  • The driver splits the lines into <TS>_stall_<WSG>.tsv (run_stalls), lists each file in the summary, and prints it beside the statement under statement bound hit; a cancelled group names its file on the line after its TIMEOUT line. --stall-sample-after reaches both legs as LNK_STALL_AFTER_MIN and is refused at or above --statement-timeout; unset, wsg_run_one.R derives it from the group’s own bound, so the pre-run gate and a printed re-model get the same default.
  • The pre-run gate watches data-raw/wsg_stall_sample.R, so this release, like any change to R/, needs the gate again before a provincial run. Mechanics: RUNBOOK §8d, “A stalled statement leaves samples”.

What a run stamps names what it read, the pre-run gate finds its own history, and primitive loads survive a flaky service and record what they loaded (#474, backlog move 6). No modelling output changes and no vocabulary token moves; the pre-run gate runs again before the next provincial run.

  • log.bcfishobs_sha is retired (#451). It was the commit of a local bcfishobs checkout, which no run reads, and a dirty checkout refused the launch. The driver no longer reads one; new rows write NULL and the column stays. study_area_verify.sql names the observations each group read by their loaded version in log_input, and fails a run whose hosts read different versions of the object-store primitives or different content of the API ones.
  • One rule picks the bcfishpass reference build, and log.bcfp_date_completed carries its build time (#446, #266). The ledger tier reads only snapshot rows that loaded the reference views, so a CSV-sync row no longer relabels a host’s builds; the database tier takes LINEAR builds by completion time. model_version is a code identity that weekly rebuilds of one commit share, so the time is what tells two builds apart.
  • The pre-run gate needs no flags or copied logs (#450). A bundle with no full history of its own is held to the nearest config it extends with builds on the database; with none, the gate picks on every input. Each group is bounded on its newest minutes from the log or the committed record data-raw/logs/build_minutes.csv (seeded with the two provincial runs), scaled to the gate host. On m1, default with no history picks the same 11 groups as the hand-seeded #454 gate.
  • RUNBOOK says a build is the log rows of one run_uid (#458); nothing clears log or log_recompute.
  • scripts/update_hosts.sh installs the fresh DESCRIPTION pins (#269), never fresh’s main. It installs by pak and checks RemoteSha (link without its dependencies, so pak cannot put fresh back at main’s pin), judges each remote step by a printed marker, keeps the tarball path for cypher, installs the host it runs on locally, and has --dry-run.
  • snapshot_bcfp.sh retries, row-checks and falls back (#484, #459). Each PSCIS load is held to its rows and the WFS’s own count, retried with backoff, then read from the WFS directly (paged on a sort, in bc2pg’s column shape). The observations go through pyarrow where GDAL lacks Parquet, and every --config takes two arguments for GDAL 3.8. Both fallbacks were measured on m1 against the primary path: the pyarrow load equals GDAL’s, and the WFS load equals bc2pg’s by value but not by content md5, so the pre-flight refuses a host that fell back.
  • Hosts reach one freshness decision before any spend (#276). The snapshot skip guard needs the upstream build’s completion time as well as its version (on 2026-11-12 upstream had rebuilt v0.7.15-51 beneath m1’s snapshot); the pre-flight compares every host’s loads before any write (persistent workers before the spin) and refuses a mixed run; a run with cloud hosts must refresh the dispatcher.
  • The vintage gate asks whether an object’s source changed (#273): a changed ETag is stale whatever the age, the same one current whatever the age, and a failed HEAD falls back to age.
  • data-raw/fwapg_load_capture.sh records what fwapg’s load.sh fetched as loader rows, from its set -x trace (#270). Not yet run against a real load.

A build can classify each full-network segment on its minimal parent’s gradient (#463). pipeline: gradient: source: parent sets each working segment’s gradient from <schema_full>.streams_parent between break and classify, so a habitat rule tuned on the minimal network reads, on the network link ships, the gradient it was tuned on. The new bundle default_parent (extends: default, schema fresh_default_parent) is cut-full-top acc-v5 grad-parent q-fwapg. Built on BULK, SETN, MORK and COTR beside a default control, it holds the control’s segments and access exactly. Every habitat difference sits on a segment whose gradient changed or in a connected cluster holding one. Rearing rises (CH +3.9 %, ST +3.3 %, CO +1.5 %, BT +1.1 %), and spawning moves within 0.33 % (data-raw/logs/grad_parent_463/).

  • It refuses rather than read a stale parent. Before the log row opens, the group’s newest full and minimal builds must be closed, clean, on one fresh, with the bundle’s tokens, and streams_parent must have been joined from exactly those two runs, a row per segment of the right length. Before gradient is set, the newest builds are read again, and each working segment must map one to one onto the full schema’s by line and measures within 1 mm. A refusal names the rebuild (data-raw/streams_parent_build.R).
  • The persisted gradient in a grad-parent schema is the parent’s, the value classify read; the segment’s own stays in fresh_default. A default_parent build runs only where fresh_default, fresh_default_minimal and streams_parent live (the dispatcher). No province-wide build has been made.
  • The near-river distance stays the segment’s own, and on the four groups that costs nothing (#462). CH’s near-river pair reads 1050/1150 lines only. Over all 1050/1150 segments a segment’s distance and its parent’s differ on 94, by up to 542 m, and none crosses 1,000 m, so no CH rearing is lost to it. Carrying the parent’s distance would need fresh’s classify to take a prepared near-river table.
  • grad-fwa names reading the Freshwater Atlas segment’s gradient (#460): whse_basemapping.fwa_stream_networks_sp.gradient on the segment’s linear_feature_id. No link segment crosses an FWA segment boundary (0 of BULK’s 42,458). No build writes it; a consumer’s archive files that setup under it.
  • Builds older than the vocabulary stay filed by run id, not given tokens (#420), and a consumer’s own discharge read is its input, not a q- token (research/output_vocabulary.md).

A one-group run on a machine that holds only some groups can read its downstream groups’ barriers from the habitat export (#465). A group’s access is walked downstream through every group’s persisted barriers and barrier_overrides, so the export now carries both, one part per group per config, with a map of which groups each group’s walk reads; lnk_downstream_files() lists what a target needs and lnk_downstream_load() loads it. Modelled after loading its 11 dependencies from the export, BULL equals the provincial build row for row in streams, streams_access and streams_mapping_code; without them 341 segments lose their downstream-dam flags.

  • The dependency map is the line rule, now .lnk_wsg_deps_line() (R/lnk_wsg_deps.R), shared with the recompute scheduler (recompute_deps.R’s output is byte-identical). The outlet closure misses readers. Every one of fresh_default’s 7.7 M barrier and 44 k override rows sits on a line its own group touches, so the rule covers what a walk reads. The export refuses to compute it from a database whose FWA lacks a group it holds.
  • CABD and PSCIS are recorded by content at load (provenance.source_artifact.content_md5, copied into log_input.artifact_content_md5): they have no version handle. The export snapshots them province-wide, keyed by that md5, for each build whose md5 the exporting database still holds, and says why not otherwise. The loader names inputs it could not compare. Observations and modelled crossings are cited by loader version id. Every build so far reads “no content identity at build” until hosts re-snapshot their primitives.
  • lnk_downstream_files() classes each dependency as exported, skipped, not held (no barrier or override rows in the exported schemas) or not exported, and refuses a target the map does not cover rather than read it as one with no dependencies.
  • lnk_downstream_load() refuses a dependency missing from the export, and rows the schema built itself or holds from no recorded load, and records each load in <schema>.log_downstream_load. It reports, without refusing, a dependency built under another bundle and inputs that differ from what the dependencies were built from.
  • bigint goes to Parquet as text: before duckdb 1.5.6, duckdb::duckdb_register() read an integer64 column’s bits as a double, so linear_feature_id came back as 3.5e-315 with no error. duckdb 1.5.6 fixed it; text is right on every version.
  • The run summary prints the commands that republish the export after a modelled run; habitat_publish.R PUBLISH puts the new parts, skips unchanged ones and deletes their stale keys. Nothing is published without the operator. Manifests gain group_parts, dependencies and a per-config scope; older readers see only the segment parts.

Each segment of the full network can read the gradient of the minimal segment it lies in (#444). A habitat rule tuned on the minimal network (cut-min) reads each minimal segment’s gradient; the new lnk_streams_parent() writes <full schema>.streams_parent, one row per full segment naming its minimal parent, the parent’s gradient and length, and how far the full segment straddles it, so the same rule can be applied on the network link ships.

  • The join is refused unless it is well defined: every minimal cut must have a full cut within 1.01 m, the pieces must cover both networks, and every full segment must get a row. A group’s rows are replaced in one transaction.
  • data-raw/streams_parent_build.R builds it for every group two schemas hold, after judging the pair as the pre-run gate judges a record: one modelling code, fresh and token set per schema, cut-min against a cut-full-* token, the same species per group. --verify names groups a re-model has left stale. Nothing on the modelling path creates, copies or clears the table, so a re-model of either network owes a rebuild (RUNBOOK §6b and §8d).
  • A new vocabulary token, grad-parent, names the reading. No build writes it; a downstream archive uses it to tell the parent-gradient setup from grad-seg on the same build.

No code change. The minimal provincial run (#441) built default_minimal into fresh_default_minimal for all 241 groups (run 20261102_203300-60ab009d), and its record lands: the launch, the checks and the first host-speed calibration (data-raw/logs/minimal_run_441/, research/study_area_run.md).

  • RUNBOOK §8d says a worker’s consolidate runs at the speed of its path from the dispatcher. Mid-copy in this run m4 reached m1 through a Tailscale relay and m1’s 49 groups took 47.6 min against about 13 for each cloud host’s 48, while two hours later the path was direct; probe it just before launch.

No code change. The records behind the minimal provincial run (#441) land: the minimal network’s proof, test builds of default and the bcfishpass parity re-score, the hosts on fresh 0.52.0 with fresh primitives, the pre-run gate’s pass on default_minimal and a rehearsal across four hosts.

  • RUNBOOK §6b says what the minimal network does to a build (#415): on four groups it keeps every minimal cut, accessible km and the length-weighted gradient, and moves code 2, rearing, the user habitat overlay and the crossing fields. Observation lifts still move its cuts, by 0 to 1.2 % of km (research/minimal_network.md).
  • A worker’s prep reads the clone’s configured origin (data-raw/worker_prep.sh), so a host whose git config rewrites GitHub URLs (url.<base>.insteadOf) no longer fails it. test-worker_prep.R covers it (#418).

The pre-run gate can run on a bundle without a full history of its own (#442). default_minimal has no groups, minutes or range of its own before its first run, so pretest_run.sh now borrows them from a config the bundle extends with --history-schema and --history-config. The record names what it borrowed, and the launch check holds the gate to it.

  • A bundle that extends another borrows until its own builds cover every group it models, so one build of its own does not end it. The borrowed history must hold every group too, in the pick and at launch. The gate picks only groups the history holds, so a 4-group history picked exactly those 4; against the full 241 it picks 11, across link’s code since the last provincial run.
  • data-raw/pretest_history_load.sh copies a host’s log into a schema of its own, with its key and indexes, and refuses a target holding anything else.
  • --pick-only refuses a history no range can cover, a build with no link_sha or from a dirty or unknown tree, because a gate on it could never clear a run. A range that merely falls short is still a warning there.
  • link now pins fresh 0.52.0 (from 0.48.0). Its frs_order_child() and frs_habitat_overlay(bridge =) match the watershed group as well as id_segment. How this moves default and bcfishpass is measured under #440. Persistent workers need --auto-install at the next run.

A second output of the default model, with far fewer cuts (#415). The new default_minimal bundle cuts streams only at gradient barriers where a modelled species’ access changes, falls and user-definite barriers, so no fish record or road crossing cuts a stream at its own position. It is meant for tuning habitat thresholds. Records still reach the cuts through observation lifts: a barrier lifted for a species is no longer that species’ cut, though another species’ barrier or a falls may still cut there, and positions above it can become cuts. default, default_tuned and bcfishpass do not change.

  • The output is named cut-min, with default’s other three tokens. Cut tokens are keyed cut-<source set>[-<qualifier>...], written down in the RUNBOOK.
  • pipeline: break_order: ~ in a bundle that extends another is now refused. It would have deleted the parent’s list and fallen back to the default sources.
  • Nothing has been modelled on it yet. data-raw/logs/minimal_breaks_415/ holds the proof scripts, which compare the two networks group by group. They have not been run on the proof groups yet. Giving each full-network segment its minimal parent’s id and gradient is a follow-up built from what that proof shows. # link 0.76.0

A bundle can pause a species without losing its rules, and default pauses kokanee and rainbow trout (#432). configs/default/species_modelled.csv lists every rules species t or f; a paused row carries its reason, date and issue, and turning a species back on is a one-cell edit. KO is paused because its presence column was never sourced, RB because field records do not separate rainbow trout from steelhead. default_tuned inherits the file; bcfishpass declares none and is unchanged.

  • A paused species adds no gradient breaks, and is not classified or given access or mapping codes (#433, “off means off”). Measured on six groups built both ways: segments fall 0.2 % (BULK) to 6.0 % (KETL), barrier rows and accessible km do not move, and habitat moves only in KOTL (BT rearing 20.9 km flips, +5.4 km net on 2,161) and KETL (0.4 km). 33 of KOTL’s 1,473 fish records sit on a segment that changed, with a median gradient shift of 0.012. Observation tops and known-habitat endpoints still cut streams. Write-up: research/species_paused.md.
  • A group where only paused species are present is still modelled, for its segments, barriers, access annotations and mapping-code rows, with no habitat, because other groups’ access reads its barriers (LEUT and LNRS are each read by 19 groups). Under default that is BRID, GRNL, LEUT, LNRS, MURT, PITR and UISR, so a provincial run still models 241 groups. lnk_pipeline_species(paused = TRUE) says whether a group is modelled at all.
  • A full build clears what it no longer classifies. Right after its first persist, lnk_pipeline_run() deletes the group’s rows from the habitat tables and near-river pairs of species it did not write this time, since they belong to the old segmentation. lnk_persist_init() keeps a paused species’ tables and columns, so every host builds one shape and a consolidate’s per-group replace reaches them; study_area_verify.sql fails a run that leaves such rows.
  • acc-v5 replaces acc-v4 for default and default_tuned, and their config_hash moves. Every default group re-models at the next provincial run. The validator refuses a paused species only for a schema whose builds never classified it, the variants harness refuses a variant of one, and the compare step skips a group with none modelled.

default’s species presence table carries the first, provisional review of its cells against the fish records (#402). 111 salmon and steelhead cells are added and 10 cells removed, in 80 groups; each changed row’s notes names the review, and each cell’s reason is in configs/default/wsg_species_presence_review.csv. Nothing here is final: a later review can reverse any cell.

  • The record rules the review set: a salmon or steelhead record does not count when the nearest barrier below that blocks it is a mapped falls or a hand-entered barrier with at most one other record of its kind above it. When that nearest barrier is a modelled gradient barrier the record still counts, because removing it would assume the 15 and 20 % access gradients the records could test. There is no elevation rule; the 19 records above 1,400 m were judged one by one.
  • Added where records sit at several sites, or at one site with a record since 1990, except 10 such single sites where the species is unusual that far inland. Those, single sites with nothing since 1990, salmon in the Kootenay, upper Columbia and Mackenzie, and species outside their native range stay unmarked, as do Okanagan and Similkameen steelhead and the four Yukon cells until the literature review, which may add them.
  • Removed: the six Upper Dean salmon and steelhead cells added as a test, and four cells resting only on stocked fish (CT in THOM and LNTH, DV in VICT, GR in BULL).
  • Not yet decided: 141 kokanee and rainbow cells wait on pausing those species (#432), and 13 cells wait on a look at their records or the literature.
  • acc-v4 replaces acc-v3 for default and default_tuned, and their config_hash moves. The 80 edited groups, and the groups downstream of an added cell whose code 2 and lifts read its records (12 more, LFRA, SETN and LSKE among them), re-model at the next provincial run, which models every group.
  • default and bcfishpass now disagree in 122 presence cells, so habitat_validate.R refuses to score one with the other’s species pooling for a species whose cells differ in the scored groups.
  • data-raw/wsg_species_presence_map.R maps the records each proposed rule would drop. The session’s log and the evidence for one misplaced chinook record are in data-raw/logs/presence_check_402/decision_session/.

Every file a bundle copies from bcfishpass is synced, and link’s changes to one sit beside it as an overlay (#429). The weekly sync used to check 7 files and missed four that had drifted; its dry run said “no drift”.

  • Copies stay byte for byte. A bundle file can declare an overlay:, a link-owned CSV of single-cell changes, each with its reason and issue, which lnk_load_overrides() applies when it reads the file. The bcfishpass bundle’s species presence table is bcfishpass’s again, and link’s kokanee column is overrides/wsg_species_presence_overlay.csv. Moving it changed no loaded table and no output token.
  • The sync covers every copy, per bundle. It now also checks cabd_additions, cabd_blkey_xref, cabd_exclusions, cabd_passability_status_updates, dfo_known_sockeye_lakes, wcrp_watersheds and observation_exclusions, writes a change only into the bundle that declares the file (default’s presence table is link’s own), and stops when a file cannot be fetched.
  • A change that moves an output token is held for review. The sync loads each bundle with the changes applied and compares its token specs. A moved spec, or a bundle that no longer loads, opens a token-review pull request that is not merged, naming the vocabulary row to add.
  • Pulled at smnorris/bcfishpass@3bd2824: CH present in UTRE and ST present in QUES in the bcfishpass bundle, so acc-bcfp-4 replaces acc-bcfp-3; the Keogh River and Adams rows in cabd_passability_status_updates.csv; three lines of wcrp_watersheds.csv; and cabd_blkey_xref.csv without the final newline link had added (the reader now accepts a short file without one).
  • The two dam rows change no output yet. Neither id is in the CABD dams table that link or bcfishpass’s own build reads, and Keogh Dam is still a barrier in both. Evidence: data-raw/logs/csv_sync_429/.
  • config_hash moves for bcfishpass, default and default_tuned. The bcfishpass schema’s UTRE and QUES re-model at the next provincial run.

Each fish record is measured before it counts toward a presence cell (#402). The presence check now gives every record whether its own 50k code runs through the group it is filed under, its elevation, the nearest natural barrier below that blocks it with the records of its species above that barrier, a rearing lake for kokanee, lake and main-stem placement, and a historical distribution zone. It rolls them up per cell with distinct sites, waterbodies and years. Nothing is judged: the rules come from these numbers in a review session. No output moves.

  • Two measures catch 46 of the 67 records already marked as errors: a blocking barrier below with at most 2 records of the class above, and elevation above 1,400 m. The other 21 show no geometric tell.
  • Many flagged cells are thin: 117 of the 327 cells not marked present but recorded rest on a single waterbody.
  • Each review story in configs/default/wsg_species_presence_review.csv for a cell with its own records carries the cell’s measures; data-raw/logs/presence_check_402/ adds measures.csv, recall.csv and the evidence it read.

default’s species presence table is checked against the fish records (#402). data-raw/wsg_species_presence_check.R judges every cell of wsg_species_presence.csv on the operator’s rule: one record dated 1950 or later supports a cell, with releases (tagged as bcfishpass tags them) and observation_exclusions.csv left out, cutthroat aliases folded and species_pooling.csv applied. The check never edits the presence table, and no output moves.

  • 351 of 3,198 cells disagree. 24 are marked present with no counting record (12 with no record at all, 8 with releases only, 4 undated only), and 327 are not marked though records exist (137 of them kokanee).
  • Each disagreement waits in a review tracker, configs/default/wsg_species_presence_review.csv, with decision pending and a story drafted from the evidence: counts, years, sources, which lifts read the cell, what promotion does and the group’s note. It is not declared under files:, so review edits do not move config_hash.
  • Promotion does less than it looks. lnk_presence() adds a promoted species’ barriers to where streams break, for species the bundle models; access, classification and mapping code still follow the cell as written. 114 of the 327 absent cells are promoted and still unmodelled.
  • One exclusion rule, .lnk_observation_exclusion_keys(), now serves prep, the validator and the check; prep’s SQL is unchanged.
  • Changing presence cells is a later release, with a new acc vocabulary row. Follow-ups: #425, #426, #427.

Code 2 counts every group’s fish records (#401). The access walk’s observation call reads <schema>.observations_province, every group’s records, so a segment reads access_<sp> 2 where the species is recorded upstream in any group. Before, a main stem read 1 up to its group boundary whatever was recorded just upstream.

  • A new key, pipeline: observation_upstream:. province (default, and default_tuned by inheritance) counts a record of the species in any group. group_gated (the value when absent, so the bcfishpass bundle) also needs a record of any species from the segment’s own group upstream, which is what bcfishpass computes. Checked when the manifest loads. Breaks still read the group’s own records.
  • One walk, cheaper than before. It asks fresh for each segment’s distinct species upstream (frs_network_features(result = "distinct")), so link now needs fresh 0.48.0. Over every group’s records it costs less than the old walk over the group’s own: LFRA 65 to 37 s, BULK 21 to 8 s.
  • The access tokens are new. acc-v2 and acc-bcfp-2 are retired for acc-v3 (default, default_tuned) and acc-bcfp-3 (bcfishpass).
  • Measured. On six default groups and three bcfishpass groups built before and after, only code 1 to 2 moved, each piece under a record of the species from another group: 25.98 km under default (LNTH bull trout 12.25 km the most) and 0.05 km under bcfishpass. link’s bcfishpass builds still match bcfishpass’s observed km on BULK, PARS and UPCE, steelhead excepted. Evidence: data-raw/logs/obs_province_401/.
  • The pre-run gate has an input for it, obs_upstream: records upstream of a group’s outlet, which picks LFRA.
  • Persisted groups keep the old reading until re-modelled: the recompute passes no observations.

A fish record at a segment’s start no longer counts as upstream of it (#399). The access walk’s observation call passes include_equivalents = FALSE, as bcfishpass’s own walk does, so fwapg’s same-line measure test counts a record as upstream of a segment only when it lies at least 1 m above the segment’s start. With each species’ top a break, access_<sp> code 2 now normally ends at the species’ highest record on a line.

  • Both configs move, and there is no bundle key: bcfishpass reads the record the same way (FWA_Upstream(…, False, 1)), so link’s earlier reading was a transcription error.
  • The access tokens are new. acc-v1 and acc-bcfp are retired for acc-v2 (default, default_tuned) and acc-bcfp-2 (bcfishpass), with the same specs: the outputs changed, the specs did not.
  • Measured. On five default groups and two bcfishpass groups built before and after, only code 2 to 1 moved, on segments starting at a record: 0 to 5.4 % of a species’ observed km. Segments, barrier flags, mapping codes and habitat did not move. link’s bcfishpass builds now match bcfishpass’s observed km on BULK and PARS, steelhead excepted.
  • What else keeps code 2 above a top (records on tributaries and fwapg’s side-channel clause, and a top within 1 m of an FWA line end, which does not cut) is in RUNBOOK.md §2c.
  • Persisted groups keep the old reading until re-modelled: the recompute passes no observations.

The group rollup splits accessible km by whether the species is recorded upstream (#403). lnk_rollup_wsg() reported accessible_km only, and nothing in link told the two accessible codes apart.

  • Two default columns. accessible_observed_km is access_<sp> = 2 and accessible_unobserved_km is access_<sp> = 1, summed over the rows accessible_km sums, so they add to it within 0.01 km and all three are NA together where access is not built. bcfishpass reports the same split as length_naturallyaccessible_obsrvd_<sp> / _model_<sp>. lnk_habitat_validate()’s summary carries both.
  • Measured. On #396’s five groups, 0.3 % (ELKR grayling) to 42 % (LKEL coho) of accessible km has the species recorded upstream. Against bcfishpass on BULK and PARS, accessible km agree within 1.3 km and link’s observed km run 1.9 to 29.7 km higher, all of it the segment above a record with no other record upstream (#399). Write-up: research/known_suspected_km.md. Evidence: data-raw/logs/known_suspected_403/.

A partial run recomputes, or names, the groups outside it that read it (#274, on #377). A run rebuilds its groups’ barrier rows, and a group the schema holds outside the run whose recompute walk reads one of them used to keep access, mapping codes and settled habitat computed against the old rows.

  • Which groups. After coverage, run_finish asks data-raw/recompute_readers_check.R for every held group outside the run that reads a group it rebuilt, by recompute_deps.R’s line rule. The map is now computed in --recompute-at-end and --recompute-only runs too.
  • Recomputed only when compatible. A reader joins the recompute when its newest build is closed and carries the config hash, fresh sha and version, crate version and fwapg checkout the recompute would stamp now, from a commit with no change on the recompute’s path, and its downstream check would pass. Anything unknown refuses. The compare stays on the run’s own groups.
  • The rest are named, not failed: <TS>_outside_readers.tsv and a summary WARN list each reader as refused, waiting (it reads a run group that did not finish) or unchecked, with the re-model. A recomputed reader that lifted segments the habitat gate had cleared is named too, and study_area_verify.sql reports it rather than raising.
  • Measured. A six-group North Thompson run has 65 readers (every Fraser group reads LFRA), the 34-group field scope 78. No local schema has a compatible reader today (fresh_default was built at 0.68.0), so until the next provincial run the change names them. Evidence: data-raw/logs/partial_readers_274/. Mechanics: RUNBOOK §8d, “Groups outside a partial run”.

Every build is named by what it is: how its streams were cut, which access rules decided where, how gradient was measured and where flow came from (#414). Config names move under a fixed name; the tokens do not.

  • inst/extdata/vocabulary.csv holds the tokens, one row each with its axis, a prose definition, the spec it matches, and the release that introduced (and retired) it. lnk_vocabulary_read() returns it for consumers; lnk_vocabulary_resolve(cfg) derives a config’s four tokens. Seven to start: cut-full-top and cut-full-ssf-all, acc-v1 and acc-bcfp, grad-seg, q-fwapg and q-wet-fill. default is cut-full-top acc-v1 grad-seg q-fwapg, default_tuned the same with q-wet-fill, and bcfishpass is cut-full-ssf-all acc-bcfp grad-seg q-fwapg.
  • Access is its own axis. Since #326, gradient positions cut a stream only where a modelled species’ access changes, so access gradients, observation lifts, the modelled species and the presence table move segments as much as the break sources do. A network is its cut and acc tokens together.
  • Thresholds and data vintage get no token. Habitat thresholds, rules and width against flow change flags, not segments. Observation records, the bcfishpass correction CSVs, observation_exclusions.csv and FWA identify one build, which config_hash and log_input already record.
  • Each build records its tokens in <schema>.log (token_cut, token_acc, token_grad, token_flow), and the habitat export carries them in its provenance. Rows from earlier builds are NULL.
  • A config with no live token is refused before the run opens, and study_area_run.sh’s local pre-flight asks once before any spend. default_extrabreaks and default_rearbreaks have no rows and now refuse, as does a bundle loaded by path that matches no named one. Adding a row is the fix: RUNBOOK §6b, “Output vocabulary”.
  • Write-up: research/output_vocabulary.md.

The recompute walks each barrier set once and builds arrays only where it reads them (#394, on #377). Pins fresh v0.47.0. On top of 0.69.6’s prefilter, the access step is a further 1.0 to 2.6 times faster, with the same output.

  • Existence walks. lnk_pipeline_access() asks for result = "exists" (fresh v0.47.0) on every walk it reads only as “is a barrier downstream”: the per-species views and pscis. anthropogenic + dams, remediations (with a crossings table) and the observation walk keep their arrays. fresh’s existence mode is a DISTINCT over the same join, so the pairs are still enumerated and it gains modestly (SETN 1.2×).
  • One walk per distinct barrier set. Species views that hold the same rows within the walk’s reach of the group share one walk, so CHWK, LFRA and SETN walk 7 tables instead of 11. A barriers_per_sp table now needs watershed_group_code, which the key reads (lnk_barriers_views() provides it).
  • Measured on SALR, CHWK, LFRA and SETN on the dispatcher’s 2026-10-23 build, access step only: 3.9 / 21.5 / 89.5 / 347.3 s at the prefilter, 4.0 / 9.3 / 35.3 / 136.5 s now (26.9 / 535.0 / 1,145.7 / 763.6 s at fresh 0.44.0). Every output digest is identical, and scratch models of ADMS and ELKR digest identical in every persisted table at each step.
  • Hosts need fresh 0.47.0. lnk_preflight_fresh() now asserts frs_network_features(result). Cloud hosts install it at prep from the Remotes: pin; a persistent worker still on 0.46.0 fails the run’s pre-flight until the run is launched with --auto-install. The pre-run gate (#372) has to pass again before the next provincial run.
  • Mechanics: RUNBOOK §3, “How the walks run”. Evidence: data-raw/logs/recompute_access_394/.

Pins fresh at v0.46.0, whose downstream walks read only the features on the group’s own lines or on its downstream watershed codes (#409, from #261, on #377). Since fresh v0.45.0 a downstream walk given aoi copies the group’s segments into an indexed session temp table and walks only features on one of the group’s blue lines or on an ancestor-or-equal watershed code, instead of every feature in the province; v0.46.0 makes fresh’s connectivity passes match watershed_group_code. link already passes aoi on every downstream walk, so no link code changed.

  • Outputs did not move. Five groups’ whole recompute on the dispatcher’s 2026-10-23 build (SALR, LKEL, LSKE, CHWK, SETN), run alternately under fresh v0.44.0 and v0.46.0, left every table it writes digest-identical, and every walk returned the same features per segment. ADMS, NATR and ELKR built twice at model time digest identical in the five table kinds compared. fresh 0.45.0 also breaks ties in each array’s order on the feature id; no two anthropogenic features share a position on that build, so dam_dnstr_ind, which reads an array’s first element, cannot move there.
  • Measured (each version’s fastest run, serial): CHWK 686 → 75 s, LSKE 580 → 178, LKEL 111 → 46, SALR 42 → 23, 77 % less over the four; SETN 1,424 → 804 s; 60 % less over all five. Every PSCIS and dam walk got slower, by about 1 s in SALR, LKEL and CHWK and up to about 4 s in SETN; what costs it is not measured here. Walking each barrier set once (#394) makes fewer calls.
  • A host on fresh below 0.46.0 now fails lnk_preflight_fresh(). Re-prep cloud hosts and workers with --auto-install before the next dispatch. A downstream walk needs a session that can create temp tables.
  • Tests: every downstream walk in lnk_pipeline_access() passes aoi, and the fresh floor equals the Remotes: tag. The pre-run gate’s map covers fresh v0.42.0..v0.46.0, and fresh’s reach from link’s path is unchanged; the pin voids the gate’s last record.
  • Evidence: data-raw/logs/prefilter_409/; write-up research/recompute_cost.md, “Measured since fresh#272”.

A scoped pre-run gate before every provincial run (#372). On 2026-10-23 the run was the first contact between new code and OKAN and KTSU, and the dispatcher lost about five hours while three cloud hosts idled.

  • data-raw/pretest_run.sh --config= --schema= reads the files changed since the last provincial run, in link and in fresh (between the two pins). It maps each to the inputs that drive its cost (data-raw/pretest_inputs.csv; one query per input in data-raw/pretest/: the near-river walk’s connector run, segments, steep segments, observations, PSCIS crossings, dams). It models each input’s top two groups into fresh_pretest_<config>, bounded at 3× their newest finished minutes with a wall-clock kill. An unmapped changed file refuses the gate.
  • study_area_run.sh refuses a run that no passing record clears (<schema>.log_pretest, data-raw/pretest_check.R). A record holds while nothing on the modelling path, the fresh pin or the gate’s own rules has changed since the SHA it modelled, so a version bump keeps it and an R/ edit voids it. --pretest-note="<why>" runs without one and records the reason.
  • Measured:
    • The 0.66.6..0.68.0 range picks QUES for the walk and OKAN for dams. No input picks KTSU.
    • The first real gate failed: TAKL and SAJR were killed at their bounds while another session ran six modelling workers on the database, so the gate now refuses to start beside other sessions (--allow-busy records an override).
    • On a quiet database all eight picks passed at 36 to 75 % of their 0.68.0 minutes.
  • Mechanics: RUNBOOK §8d, “The pre-run gate”. Evidence: data-raw/logs/pretest_372/.

The variants harness runs a variant’s groups in parallel (#406). data-raw/habitat_variants_build.R --jobs=N (1 to 16, default 1, which keeps the serial loop) runs each group’s pass on a PSOCK cluster, one connection per worker, longest groups first. Variants still run one after another, and only the build process writes built.csv and variant_timing.csv.

  • Measured on #331’s nine groups on m4, from #404’s base: at --jobs=4 a variant’s pass takes 2.10 min against 6.23 serial and the default pass 3.02 against 8.06, so #331’s 12-rung ladder drops from about 83 min to 28. All 45 digests are identical across widths 1, 2, 4 and 8. Eight jobs add only 10 to 13 %, because classify and connect contend in the database, as #374 found for the recompute.
  • variant_timing.csv gains a jobs column (older rows read 1) and a pass_wall row per variant, the wall minutes a width is judged on.
  • A build at --jobs above 1 that is killed, or loses a worker, leaves the other workers and their database backends writing. The log names their R and backend pids to stop before relaunching. --wsgs is now de-duplicated, so no two workers share a group.
  • Record: data-raw/logs/variants_parallel_406/.

Pins fresh at v0.44.0, so a tuning-cost measurement names one fresh rather than whichever each host resolved (#404, #269).

  • Output-neutral. fresh v0.44.0 changes R/frs_point_snap.R (an exact distance tie now breaks on the higher downstream_route_measure, not the lowest linear_feature_id) and adds R/frs_migration_trial.R. link calls neither: snapping goes through its own lnk_points_snap(), and nothing in R/ calls fresh::frs_point_snap(). Verified against the v0.43.0..v0.44.0 diff, which touches no other file in R/.
  • Adopting fresh’s tie rule in link is still owed and is #383, with its own sweep and live tests; this bump does not start it.
  • The variants harness times each of a pass’s seven steps into <out>/variant_timing.csv (classify, connect, the working digest, persist, the access copy with ANALYZE, the gate, the persisted digests), with the discharge join, lnk_persist_init() and the base run’s model and recompute per group as rows outside the total. Measured on #331’s nine groups on m4: a variant’s pass is 0.3 to 1.0 min per group, classify 36 %, connect 31 %, persist 30 % and the two verification digests 0.4 %, so the digests are ruled out as a target and the change named for #377 is running a variant’s groups in parallel (5.6 min to 1.6 over nine groups at four jobs). The build runs again at all: its thresholds self-check, ahead of every --step, had stopped on main since #364 because write_thresholds() quoted species_code and the edge-type columns by name and #364’s text column rear_river_measure broke the byte-for-byte comparison; it quotes every non-numeric column now. Record: data-raw/logs/variant_cost_404/.

default breaks streams at each species’ highest fish observation on each line, not at every observation (#396). Measured on LKEL, BULK, UPCE, PARS and ELKR built both ways: segments fall 0.7 to 3.6 % and minutes do not move, so the change buys accuracy, not speed. The bias in segment length at fish sites closes: at UPCE’s FISS sites, the median segment at sites with fish against sites without was 98 m against 195 m, and is now 411 m against 385 m.

  • pipeline: observation_breaks: (new bundle key, checked at manifest load). all breaks at every observation, as bcfishpass does, and is the value when absent; the bcfishpass bundle keeps it. top_per_species breaks at each species’ farthest-upstream record per blue_line_key; default sets it, and default_tuned inherits it.
  • Access moves only the way a segment’s start moved. Checked on every piece in all five groups: 1 → 2 on 6.1 km, all at species’ tops, and no change between blocked and open. bcfishpass builds of BULK and PARS are unchanged, and lnk_habitat_validate() scores the same records.
  • lnk_habitat_validate()’s docs now say an observation attaches to the segment starting within 1 m of it, else the one holding it, and a test pins the mid-segment case.
  • A species’ own top break never moves its own “observed upstream” boundary: the access walk counts a record at a segment’s start as upstream of it. Putting each boundary at the highest fish is #399; known-vs-suspected fish is #398 to #403.
  • Owed: every persisted default group re-segments at the next provincial run, stacked on #326’s owed re-model. Mechanics: RUNBOOK §2c. Record: research/observation_breaks.md. Evidence: data-raw/logs/observation_breaks_396/.

Hosts are packed on measured minutes, and host speeds are learned from each run’s legs (#259, folding in #163). Scored out of sample, fitted on 2026-10-17 and tested on 2026-10-23’s buckets with KTSU’s and OKAN’s stalls left out: measured minutes put each leg’s share of the mean leg within 3.6 points, and FWA segments 28.9. Equal speeds did as well as fitted ones, so the learned state starts equal.

  • study_area_buckets.R reads host speeds from data-raw/logs/host_speeds.tsv, under --host-speeds and ahead of equal. Speeds are keyed by --host-names aliases, with every cloud host one cloud class. It normalizes measured minutes by the host that measured them. --plan-out writes one row per group (it needs --host-names), and --minutes-exclude keeps a stalled group’s minutes out.
  • data-raw/host_speeds_calibrate.R (new) turns that plan and a run’s phases.tsv into one update, damped (0.7 old + 0.3 observed) and clamped, with an append-only history. The run’s summary prints the command.
    • It reads whole legs, never per-group means: a simulation shows those run away from the true ratio, while whole legs converge.
    • Each leg is re-projected at the stored factor, so the speed a run was packed at never leaks in.
    • It divides out the run-wide drift. The same groups took 13 to 16 % longer on 2026-10-23 than on 2026-10-17, which is why every leg ran 12 to 20 % over.
    • It never learns a driver role key, and it leaves out a leg holding a group measured by a host the state does not know.
  • Owed: pack the next provincial run with --minutes-schema --host-names --minutes-exclude=KTSU,OKAN --plan-out and calibrate it; #163’s three-run acceptance follows. Mechanics: RUNBOOK §8d, “Learned host speeds”. Record: research/study_area_run.md, “How hosts are packed”. Evidence: data-raw/logs/packing_259/.

Streams break only where a present species’ access changes, as bcfishpass breaks them (#326). Every persisted group re-segments under both configs, so the next provincial run re-models all 241 groups in fresh and fresh_default. Until then, groups within one schema sit on different segmentations.

  • The break set is each present species’ barriers after its lifts, reduced to the downstream-most position per flow path (.lnk_pipeline_prep_minimal(), per species, never on the union). The independent falls, user-definite and subsurface sources still break where they sit. #223 reduced before the lifts and then kept every position, which this replaces; #225’s rename is moot. On the bcfishpass config, 11 groups hold 0.9995 to 1.006 times bcfishpass’s segments, and every one of the 454 link-only breaks has a named source. default drops 2.1 to 2.7-fold (PCEA 106,079 → 39,566 segments). Parity is 120 of 122 pairs, and BT rearing in PCEA, FINA and LKEL now matches bcfishpass segment by segment to 1.2 km.
  • The bcfishpass config takes bcfishpass’s order-1 rearing exception for BT, CH, CO, ST and WCT on cw groups, with its tests: a known width up to the minimum and a gradient within the rearing maximum. It needs fresh 0.43.0 (frs_order_child(channel_width_max =, gradient_max =)). PARS BT rearing against bcfishpass goes from −24.8 to −8.1 km; the rest is fresh’s direct-child test (fresh#158). bcfishpass’s CH / CO / ST / WCT waterbody and edge filter is not applied (at most 2.7 km per species per group).
  • lnk_barriers_unify() persists every staged barrier row. A measure in FWA’s millimetre gap between segments got no point from FWA_LocateAlong(), and the row was dropped: 4 to 31 per group. The reduction still held those positions, so the barriers above them went unbroken. .lnk_sql_locate() falls back to the nearest segment on the line. The upstream report is drafted as NewGraphEnvironment/fwapg#45 (upstream-draft).
  • Owed: the provincial re-model, and the PARS vignette’s cross-config join, which the configs’ separate segmentations invalidate (its guard refuses; #390). Kept: link-only BT spawning at exactly 2.0 m (FINA 4.6, PCEA 1.8, PARS 5.5 km) is fresh’s >= against bcfishpass’s >; a bundle minimum 0.005 above bcfishpass’s would reproduce it and was declined. Mechanics: RUNBOOK §2b and §7. Record: research/bcfp_segmentation_parity.md. Evidence: data-raw/logs/segmentation_326/.

Each group is recomputed while the legs still run, once what it reads has landed (#374). study_area_run.sh starts a group’s recompute as soon as it and every group whose barrier rows its walk reads are on the dispatcher, instead of in one pool after the last leg and consolidate.

  • The scheduler (data-raw/study_area_recompute.sh) runs beside the legs. The readers are data-raw/recompute_deps.R’s line rule, not the outlet closure #374 proposed: the closure misses them in 150 of 241 groups, because a watershed group is not one drainage and gradient barriers are persisted under every group touching a line. --recompute-at-end keeps the old order.
  • A too-early recompute cannot be undone (the habitat gate only clears flags), so readiness fails closed. data-raw/recompute_audit.R is a run post-condition that compares each start with each landing on one clock. It fails on an early start, and on one it cannot prove. LNK_LOG=0 is refused in early mode.
  • lnk_persist_init() replaces streams_habitat_long_vw only when its definition changed. The unconditional CREATE OR REPLACE VIEW took an exclusive lock, once per dispatcher group, on a view every recompute job reads.
  • Measured. Replayed on 2026-10-23 without its stalls, with its load on the dispatcher counted (a full-width recompute doubles the dispatcher’s modelling time), the recompute ends 36.7 min sooner. On the dispatcher and a worker, ten groups built early and at the end match row for row. Mechanics: RUNBOOK §8d. Evidence: data-raw/logs/recompute_early_374/. The provincial measurement is still owed (#374 stays open for it).

study_area_run.sh --workers=<ssh target>,... adds persistent workers (#376): machines that exist and stay up, modelled beside the dispatcher and the cloud hosts, consolidated and released when their own leg ends, and never burned.

  • Every remote status is read from what the remote shell prints, for cloud hosts as well. Tailscale SSH to macOS returns 0 whatever ran. data-raw/study_area_remote.sh and consolidate_remote() judge each probe, prep, leg and consolidate call on a printed trailer or end marker. This closes a data-loss path: a per-table COPY that died part-way was loaded, and while the bucket’s total still rose, the source was deleted. A persist table’s load must now also take as many rows as were sent.
  • A worker is prepped by data-raw/worker_prep.sh, piped over ssh, from a clone the driver owns (~/Projects/repo/link_worker). The leg runs link by load_all, so the worker’s own checkout and installed link are never touched. Before any spend, a worker must answer, hold its own database and hold no rows for its bucket. Prep refuses what the worker sets for R that would beat the run’s exports, because a ~/.Renviron line wins over an exported variable.
  • Consolidate and the worker’s partial drop take link’s own tables only (consolidate_link_tables()). In fresh they also reached the snapshot primitives and the bcfp reference.
  • study_area_buckets.R --workers=N packs onto workers; the CY_* maps keyed by any remote host are now HOST_*. Mechanics: RUNBOOK §8d. Evidence: data-raw/logs/worker_376/, three runs on a real worker. The provincial-scale acceptance (#377’s runs row) is owed.

Each cloud host is consolidated and burned as soon as its own leg ends (#369), while the other legs keep modelling. Before this, every host waited for the slowest leg and one consolidate of all of them.

  • study_area_run.sh finishes each cypher as its leg ends (data-raw/study_area_legs.sh: account, consolidate that one source, release, burn). The dispatcher’s persist is initialised and backed up once, before any leg writes; the dump runs in the background through spin and prep. A host whose bucket overlaps the dispatcher’s waits for the dispatcher leg, and a consolidate with no outcome keeps its host and its groups in_flight. The driver now needs bash 5.1 or later.
  • lnk_persist_init() and the log-table setup skip CREATE INDEX IF NOT EXISTS / ADD COLUMN IF NOT EXISTS when the catalog already has the index or column. Both no-ops take their table lock before finding the work done, so each group’s init waited behind a running consolidate COPY.
  • schema_consolidate() judges a copy on the bucket’s rows, refuses a destination without link’s core persist tables before deleting anything, logs each table’s copy time, and backs up through schema_consolidate_backup(), which stops when pg_dump fails.
  • A dispatcher run from a git worktree no longer logs link_sha NULL. .lnk_read_git_head() now resolves the common dir, packed-refs and relative gitdir: pointers.
  • Three small runs on real droplets: both cyphers burned 6.5 and 12.8 min before the dispatcher leg ended, a refused host was kept with its burn command, and both verifiers pass. The provincial-scale measurement is still owed (#369 stays open for it). Mechanics: RUNBOOK §8d, “Each cloud host is consolidated when its own leg ends”. Evidence: data-raw/logs/consolidate_on_finish_369/.

A run records when each phase started and ended, per host (#375). Driver changes only; no modelling code moved.

  • study_area_run.sh writes <TS>_phases.tsv, one row per phase per host as it ends: run_uid phase host start_utc end_utc minutes groups status, over prep, refresh, model, account, consolidate, burn, coverage, views, recompute and compare. A phase that did not run gets a skipped row with blank times, and the EXIT trap closes the phase a run died in as failed. The summary prints the table, the critical path (first start to last end) and the longest model leg; --recompute-only writes its own file under the run’s uid. Rows overlap, so never sum them.
  • On a dispatcher-only run the model and recompute rows bracket the database’s own log and log_recompute times to the second. The provincial run that fills the speed tracker’s runs row (#377) is still owed.
  • Mechanics: RUNBOOK §8d, “Where a run’s time went”. Evidence: data-raw/logs/phases_375/.

A runaway modelling statement fails its own group (#371). Driver changes, plus one internal helper; no modelling code moved.

  • study_area_run.sh --statement-timeout=<minutes> (default 30, 1 to 1440) bounds every modelling statement, as PGOPTIONS='-c statement_timeout=<N>min' on both legs, with LNK_STATEMENT_TIMEOUT_MIN beside it. wsg_run_one.R prints each group’s bound and refuses a connection whose bound differs, so a leg that lost the setting fails its first group instead of running unbound. The recompute keeps its own 10 min per statement. The default comes from 2026-10-23, where the longest finished group took 16.3 min in total (SETN).
  • A cancelled statement is named. wsg_run_one.R writes TIMEOUT after <bound>: <statement>, from the stack in a calling handler, or from .lnk_db_execute()’s Statement: when a transaction re-threw the error. The INCOMPLETE summary lists the group under statement bound hit: with a re-model command carrying the run’s bound, and LNK_TEST_STALL_WSG stalls one group to exercise it.
  • Mechanics: RUNBOOK §8d, “A runaway statement fails its group”. Evidence: data-raw/logs/statement_timeout_371/.

The province-wide default run is built at 0.68.0, and a drainage component can be split across hosts (#361). Driver and verifier changes only; no modelling code moved.

  • fresh_default is one default build of all 241 groups at 0.68.0 on fresh 0.42.0 (run 20261023_074321-24d85e8c, 2026-10-23; the 2026-10-17 build is kept as fresh_default_20261017). The re-model #364 and #367 owed is done, and lnk_habitat_validate() accepts the schema under default again.
  • Hosts can take single groups. The Peace’s 26 groups modelled on two hosts in the worst order, consolidated and recomputed, digest identical to the one-host build in everything the modelling order can reach (data-raw/logs/provincial_361/split_verdict.md), so study_area_buckets.R --atom=group packs single modelable groups and prints the makespan under both atoms; study_area_wsgs.R --exact resolves a list as given; study_area_run.sh --buckets-exact takes those lists, makes an overlap fatal and records bucket_mode in <TS>_run_wsgs.tsv. At four hosts the atoms tie on this run’s minutes; group atoms pay from five.
  • study_area_buckets.R --minutes-schema=<schema> weights each group by its measured modelling minutes, and --host-speeds defaults to equal (the 2026-10-17 run measured the cloud hosts at the dispatcher’s rate).
  • study_area_verify.sql passes a run that took the #348 path. A failed attempt keeps its log row open by design; the verifier now notes an open row a later finished row of the same group supersedes and raises only on one nothing supersedes (negative cases 6f and 6g).
  • Two cautions measured on the way. A build is byte-identical on one host and differs by up to 3.7e-9 m across architectures (lengths, geometry, snapped barrier positions), so “identical across hosts” is a digest after rounding. And one m1 group sat over four hours in a single statement that re-runs in 59 s on the same table; the driver bounds no statement, which is drafted as a follow-up with a scoped pre-run gate.
  • Evidence: data-raw/logs/provincial_361/; record: research/study_area_run.md.

Two builds from the same inputs give the same output, and default sees the dams outside BC (#367: #277, #244, #345). The release the next provincial default run is built at.

  • The PSCIS pick is reproducible. bcfishpass keeps only the nearest instream modelled crossing per PSCIS crossing per stream before the pick; link had skipped that step, so two modelled crossings on one feature tied and the pick fell to row order (at 0.67.0, two builds minutes apart picked a different modelled crossing for 464 PSCIS crossings). The step is ported, with tie-breaks on the modelled crossing’s id. Picks move toward bcfishpass’s: on ADMS, ELKR and NATR accessible km are unchanged and spawning or rearing moves by at most 0.094 km per species.
  • Barrier ids are stable. Gradient, falls, user-definite and subsurface barriers are numbered by position, and both nearest-stream snaps break ties on linear_feature_id. ADMS and ELKR built twice are identical in every persisted table.
  • default treats the cabd_additions dams as barriers (pipeline: cabd_additions_barriers; Grand Coulee / Chief Joseph and Ross, where barrier_ind is true). Dams never block access, so this moves only the DAM mapping-code token: 19,065 segments in LARL, KETL and SKGT, all NONE to DAM, and segments upstream in other groups by the same rule. The bcfishpass bundle keeps bcfishpass’s behaviour. The downstream guard probes them through the same SQL, so under default a group upstream of one fails the guard until KETL, LARL and SKGT are re-modelled at this release.
  • Barrier lifts count each observation once. natural_barriers can hold one position twice (a point where two FWA segments overlap), and the lift count credited a record once per copy. It is now a distinct count; no lift changes in the 158 groups checked.
  • A gradient barrier where two FWA segments overlap takes the lower segment’s codes by name, as every build so far did by chance.
  • The golden dams test pins ADMS’s 7 dam ids; the eighth of the old pin left CABD upstream.
  • Evidence: data-raw/logs/provincial_prep_367/.

default rears a stretch that runs through a wetland only where it would rear as a stream: big enough, or close to a large river (#364). Needs fresh 0.42.0.

  • Two dimensions.csv columns.
    • rear_wetland_flow sets how FWA 1050 / 1150 lines rear: stream (the species’ gradient, size and near-river thresholds), gradient (the rear gradient only) or free (no test, the earlier rule).
    • rear_near_river sets which rear rules take the species’ near-river pair.
    • default takes stream for BT, CH, CO, ST, WCT, CT, DV and RB. The bcfishpass bundle gives CO gradient, as bcfishpass’s coho SQL does. Every river-polygon rule now carries near_river: false.
  • The near-river pair is a threshold. parameters_habitat_thresholds.csv carries ten <stage>_river_* columns that fresh inherits. default gives CH rearing a river of at least 15 m modelled width under cw (3 m³/s under mad) within 1 km, on its through-wetland lines only (rear_near_river = wetland_flow). Every other species is NA until #346.
  • Persisted, logged and validated. <schema>.streams_near_river keeps the walked distances. <schema>.log.near_river records each group’s pairs, and lnk_habitat_validate() scores against both. A schema built before this release has neither, so it cannot be validated under default until it is re-modelled (the next provincial run).
  • The variants harness varies rules as well as thresholds. It takes a dims column, structure variants scored against the base, near-river cells in set, and pair anchors. built.csv records each schema’s rules and dimensions.
  • Measured before default moved, on five held-out groups (ADMS, BULK, CHWK, CLWR, SQAM; 291 CH locations). The door alone removes 43.1 km of CH rearing and no captured location. The pair on every stream rule would add 275.6 km where CH are found at 5 % of the core rate, so default limits it to through-wetland lines (operator, 2026-10-21). Under bcfishpass, CO rearing moves −0.2 km on ADMS and −1.5 km on BULK. Write-up: research/habitat_thresholds.md, “Through-wetland lines and the near-river test”.
  • citation("link") carries the package’s Zenodo concept DOI, which resolves to every archived release. It is read from a new Config/DOI field in DESCRIPTION, so inst/CITATION holds no package-specific literal.
  • RUNBOOK.md names fresh#258 as the fresh-side tolerance follow-up, and words default’s RB observation rule as shaped like BT’s (one RB record, only RB counting for RB, no date floor).

The persisted habitat flags follow streams_access, and barrier lifts count every group’s observations (#349, #336).

  • lnk_habitat_gate() (new): where access_<sp> = 0, accessible, spawning, rearing, lake_rearing and wetland_rearing go FALSE in the persisted streams_habitat_<sp>; rows user_habitat_classification marks keep spawning and rearing. It runs in lnk_pipeline_run() after the second persist (new argument habitat_gate = TRUE; mapping codes are now built from persist after it), and after lnk_access(merge = TRUE) in the recompute. Counts land on log / log_recompute (habitat_gate_*).
  • The gate is one-way. A later recompute that lifts a settled segment cannot restore its habitat; it counts it (habitat_gate_lifted) and study_area_verify.sql fails on it. So study_area_run.sh models with LNK_HABITAT_GATE=0 and lets the recompute settle every group. study_area_verify.sql also fails on any accessible segment whose access_<sp> is 0.
  • Barrier lifts count observations from every group (<schema>.observations_province, each record kept by its own group’s species presence), as bcfishpass does. Each species’ rule (which species count, how many, from what date) is unchanged. default’s RB row gains an observation rule (one RB record, any date, no control veto), so default’s config_hash changes.
  • Existing schemas are not settled until rebuilt. fresh_default fails the new verify check until a full re-model; re-model before re-gating, never the reverse (RUNBOOK §3). Write-up: research/access_columns.md.

study_area_run.sh records which groups a run was running, and a refused consolidate source no longer stops the run (#358).

  • <TS>_run_wsgs.tsv names every group and host before any spend, and is rewritten after per-host accounting with each host’s SKIPs. A bare --recompute-only --run-uid=<uid> reads it together with the dispatcher’s log rows, so a group that failed on a cloud host, or before its first log row, is named. A run from before this change falls back to the log alone; --wsgs= still wins.
  • A refused or failed source costs only its own groups. When every source has a recorded outcome (<TS>_consolidate.tsv), the run carries on to coverage, the recompute and the compare with the hosts that consolidated. The failed host’s groups are a new coverage class, not_consolidated, kept out of the recompute; the host is kept while finished groups remain on it; the run exits non-zero naming them. If R died without recording outcomes, the run still exits after consolidate.
  • A crash mid-copy fails closed. <TS>_consolidate_state.tsv lists each source’s groups in_flight before the copy and ok or failed:<stage> after it. A later bare --recompute-only keeps any group not ok out of the recompute until --wsgs= names it. Mechanics: RUNBOOK §8d, “A missing group costs only itself”.

In study_area_run.sh, one failed group no longer costs every other group its post-consolidate recompute, and --recompute-only finishes a run once its gaps are re-modelled (#348).

  • Coverage splits the run’s groups instead of exiting. Groups fall into four classes, read on the dispatcher after consolidate and written to <TS>_coverage.tsv:
    • rows from a build this run finished, meaning the group’s newest <schema>.log row is the run_uid’s and closed;
    • no rows;
    • an earlier run’s or partial rows;
    • lost at consolidate.
    The recompute and the compare run over the first class only. Any other class makes the run exit non-zero, with the groups named and the recovery commands given. If the first class is empty, neither runs and the report says why.
  • --recompute-only --run-uid=<uid> [--wsgs=] runs only the end of a run: no droplet, pre-flight or consolidate. It refuses a --config that the run’s log rows do not name. Read on fresh_default, run 20261017_004619-3f4c713b puts all 241 groups in the recompute class.
  • The code lives in data-raw/study_area_finish.sh. It is tested on a scratch schema in tests/testthat/test-study_area_finish.R. Mechanics: RUNBOOK §8d, “A missing group costs only itself”. Follow-up: #358.

schema_consolidate.R keeps the log history of every group it copies, and fresh_default has the rows #246 lost back (#354).

  • The bucket DELETE reaches persist tables only. log, log_input and log_recompute are loaded in one transaction per table, replacing a row only where an inbound row has its primary key, so a group’s earlier builds survive a build copied home from a cloud host, and a re-consolidate changes nothing. The log tables are named once, in .lnk_log_specs().
  • A source is refused before anything is deleted when a group has a log row on the dispatcher and the source’s newest is older or missing (dest_log_newer), when the source’s log cannot be read, or when a log table’s key is not shared. The persist tables carry no run id, so a group’s newest log row is what names the build they hold. A group with persist rows and no log row is not covered (RUNBOOK §8d).
  • Restored: the earlier log rows of DUNC, KOTL, LARL, SLOC and PINE, and their 60 log_input rows, from the run’s pre-consolidate dump. study_area_verify.sql and verify_extra.sql still pass at 241 (data-raw/logs/consolidate_history_354/).

study_area_run.sh counts each host’s finished groups from its database, and never burns a cloud host that may still hold unconsolidated work (#347).

  • Finished is a <schema>.log row for the run_uid with date_end set, read over consolidate’s own route and framed by sentinels, so a host that does not answer is unknown, never “nothing”. The log’s done markers, now matched anywhere on a line, only add to it. In #246 a marker pushed mid-line by an R warning hid 18 finished groups.
  • A cypher is held from dispatch, on every exit path including abort. It is released only when its session ended 0, consolidate is not running, and its database holds no finished group after consolidate. What a kept host still holds is split into finished, partial and unknown groups. Kept hosts are written to <TS>_kept_hosts.tsv along with the commands to recover and burn each, and the run exits non-zero. Pre-flight fails on a droplet left up in this run’s workspaces. The rules are in RUNBOOK §8d, “Which host may be burned”; the host-accounting code is in data-raw/study_area_hosts.sh.
  • cabd_blkey_xref.csv ends with a newline in all four bundles, so no group’s log carries the incomplete final line warning. Each bundle’s config_hash moves; that is provenance, not a change in behaviour.

fresh_default is one provenanced default build of all 241 watershed groups default can model, and two pipeline bugs that failed groups in that run are fixed (#246).

  • PSCIS crossings matched through the xref keep one row. An xref row that sets both modelled_crossing_id and linear_feature_id was emitted by both branches of the insert, and the duplicate broke barriers_pkey at persist (KITR). Branch A is kept, Branch B fills in only where A found nothing.
  • A species present in a group with no barrier in it is accessible and keeps its mapping code. access_<sp> is 1 (or 2 where observed upstream), and has_barriers_<sp>_dnstr is FALSE; it had come out boolean (UPET failed at persist) and then NA, which blanked the mapping code. A species with no presence information keeps NA (“no data”).
  • The run: run_uid 20261017_004619-3f4c713b, m1 plus three cloud hosts; study_area_verify.sql and verify_extra.sql pass at 241; bcfishpass parity on ADMS, BULK, BABL and ELKR 48/50. Measurements and what went wrong: research/study_area_run.md, “The province-wide default run”. Follow-ups #347, #348, #351.

A bundle chooses where mean annual discharge is read from, and default_tuned now reads wet’s published product, with BT’s MAD minima re-set on it (#331).

  • pipeline: discharge_source: fwapg | wet. fwapg (the default, and what default and bcfishpass keep) reads whse_basemapping.fwa_stream_networks_discharge. wet reads the annual pick of wet’s discharge.fwa_v1_segment_monthly (historical, month 0), with 0 m³/s read as no value. wet values 4.50 M lines against fwapg’s 2.00 M. The fwapg SQL is unchanged.
  • Recorded everywhere the fill is: <schema>.log.discharge_source per WSG (a row from before reads fwapg); the validator warns when the log and the bundle disagree for a group the bundle reads discharge in; the export manifest names each WSG’s source, plus a discharge.wet_load block naming the wet build it read; a wet run’s log_input carries wet’s table and its comment.
  • New export lnk_preflight_discharge(). Prepare stops with the load command when a bundle’s source table is missing. data-raw/discharge_wet_load.R loads the annual pick of wet’s table, and skips when the host already holds the expected build (--expect-load). study_area_run.sh checks the dispatcher holds a wet load and has each cypher load the same one (cypher_prep.sh, CYPHER_PREP_WET_LOAD); wsgs_run_pipeline.sh and wsgs_dispatch.sh refuse a wet config and point to it. Suggests wet (>= 0.4.7).
  • default_tuned: BT spawn_mad_min and rear_mad_min 0.078 → 0.043 on wet’s discharge. Rearing was scored on held-out groups; spawning takes the same any-stage percentile, because the held-out groups hold too few spawning-staged locations to score it (3 in the core). Its config_hash changes. A default_tuned build now exports wet’s mad_m3s and *_ratio_mad. No habitat flag moves while every group is on cw (NATR: 0 of 189,200 rows). GR, KO and RB keep their fwapg-fitted values: #343.
  • At these values mad is the closer model for BT: on held-out groups the water only cw keeps is used at 0.18 (rearing) and 0.14 (spawning) of the size-matched rate, and the water only mad keeps at 1.64 and 0.69. mad keeps 19.8 % less BT rearing than cw (#305: 32 % at 0.078 on fwapg).
  • The values moved because wet covers nine Skeena groups fwapg does not, not because the sources disagree. On locations both value, they agree (P10 0.075 / 0.076). Write-up: research/habitat_thresholds.md, “MAD on wet’s discharge (#331)”.
  • Scoring harness: a variant may set discharge_source and discharge_fill (written into its bundle, recorded in built.csv); a set cell the base already carries is dropped, so #302’s ladders build at HEAD again. query_habitat_thresholds_mad.R and query_width_mad_equivalent.R take --discharge-source.

The README, DESCRIPTION, CITATION.cff and package help describe link as a tool for developing, testing and communicating intrinsic habitat models (#338). The point of it is flexibility: every rule and threshold is a row in a config.

  • A short example: a config’s thresholds, then a run’s per-species rollup, each shown with its real output.
  • A figure of real output: Arctic grayling in the Parsnip, made by data-raw/readme_figure.R from the cached vignette data.
  • Data and methods it builds on: a section acknowledging bcfishpass (its method and community correction data), bcfishobs and fwapg. The bcfishpass row in NOTICE now matches.
  • Removed: the roadmap, the “network-agnostic by design” claim, the feature list and the function references.
  • CITATION.cff and its update workflow are new. New Graph Environment Ltd. is credited as an author for Zenodo. The ORCID, which failed R’s checksum, is corrected, and the cffr DOI and repository fields that pointed to an unrelated CRAN package named link are dropped.

Habitat flags export per config as Parquet + manifest, keyed by FWA position, with a write-once provenance tree and a loader into PostgreSQL (#308). The pattern is NewGraphEnvironment/wet#7’s for discharge. Nothing is published yet: habitat/ on the bucket is private and PUBLISH waits on the operator.

  • New exports:
    • lnk_scenario_export() writes one row per segment × config × species, keyed (blue_line_key, downstream_route_measure, upstream_route_measure), with the flags, the width / discharge / gradient the build tested, and ratios to the thresholds the build logged.
    • lnk_catalog() reads the manifest from a local path, https, or s3:// (through the aws CLI).
    • lnk_db_load() loads parts into habitat.fwa_v1_segment, checking md5s first.
    • Driver: data-raw/habitat_publish.R, BUILD then LNK_PUBLISH=1. Mechanics: RUNBOOK §6d.
  • Only finished, logged builds are exported. Provenance is written once per build, per config hash and per recompute. Bundle files are published only when today’s bundle hashes as the build did.
  • One judgement, .lnk_manifest_regressions(), guards both BUILD and PUBLISH. It refuses a tree that would lose a live config, a group:species pair, a newer build or recompute, or a newer export. Override: replace = TRUE / LNK_PUBLISH_REPLACE=1.
  • has_barriers_dnstr ships beside accessible, because fresh’s flag and link’s access table disagree on the segment that starts at a barrier: #336.
  • Pilot on local fwapg:
    • Scope: default on 5 logged groups and bcfishpass on 39. 22 parts, 9.9 M rows, byte-identical across two builds.
    • Spawning km and barrier-based accessible km match lnk_rollup_wsg() exactly in 150 config × species × group rows.
    • Ratios are NA for builds before #282, which logged no thresholds.
  • duckdb joins Suggests.
  • Follow-ups: #331 (discharge from wet), #332 (GeoPackage), #333 (crate registration), #334 (publish gates, versioning, public read), #335 (webmap).

Each DB primitive now records which version of which object it was loaded from, and the vintage gate fails a host that has not recorded its loads (#265, absorbing #247). The bchamp object store is versioned but cannot be listed, so a version id not written down at load time can never be named afterwards.

  • provenance.source_artifact holds, per host and artifact, the object store’s version id, ETag and Last-Modified, with kind (object / api / derived) and recorded_by (loader / probe). It sits in its own schema because it describes the database, not a run; state_clean.sh drops fresh but not provenance.
  • snapshot_bcfp.sh writes loader rows and reads exactly the version it recorded (?versionId=). A snapshot that cannot record fails. PSCIS and CABD have no version handle: load time and row count only.
  • New exports: lnk_artifact_record(), lnk_artifact_probe() (HEADs the FWA artifacts after the fact; never overwrites a loader row) and lnk_artifact_read() (recorded / missing / table_absent).
  • Each run carries what it read in <schema>.log_input.artifact_*, so a cypher’s provenance comes home before it is burned. schema_consolidate.R aligns the destination’s log tables first.
  • Breaking for operators: lnk_preflight_vintage() fails an unstamped snapshot primitive even when statistics say it is fresh. Each host needs one bash data-raw/snapshot_bcfp.sh --with-bcfp-views --force after upgrading.
  • fwapg_sha documented as what it is: the checkout’s HEAD at run time, not the commit that loaded the data. Recording that commit is #270.

PCEA’s +1.87 % BT rearing against bcfishpass is explained: link breaks streams where bcfishpass does not (#324). No package code changes.

  • Where streams break. bcfishpass reduces each access model’s barriers to the downstream-most per flow path after its observation and habitat lifts, and builds a model only where its species are present. link breaks at every gradient position of every configured species. In groups with no 15 / 20 % species present (PCEA, FINA, PARS) that moves 71–85 km of BT rearing both ways, netting +26.2 / −13.0 / +9.8 km.
  • Two rule differences beside it: bcfishpass’s order-1 rearing exception (PARS −24.8 km; the bundle declines it) and its strict > on the spawning width minimum (1.8–5.3 km of link-only BT spawning at exactly 2.0 m).
  • Follow-ups: #326 (break only where a present species’ access changes, both configs) and #327 (default’s order-1 position).
  • Write-up: research/bcfp_segmentation_parity.md, which lines up bcfishpass’s five access scripts against link’s break set; RUNBOOK §2 points to it. Instrument: data-raw/habitat_diff_segments.R.

rearing_km leaves lake connection lines out of link’s WSG rollups (#319). Before this, lnk_rollup_wsg()’s default counted them and the compare rollups did not, so two rearing_km columns could differ by 55–95 % for SK.

  • lnk_rollup_wsg(). The default rearing_km no longer counts FWA 1450 lines in lake and reservoir polygons. A new default rearing_lake_connection_km (0 where there are none) carries them, so the two sum to the flag total.
  • lnk_habitat_validate(). Cost takes the split. Capture still reads the rearing flag, because about half of in-lake fish records sit on a connection line (table in the roxygen).
  • Parity scripts. parity_crosssection.R and wsg_vignette_data.R apply the same rule to the bcfishpass view. The cross-section passes 25/25 before and after, and pars_accessible.rds is regenerated.
  • Breaking for anyone reading rearing_km as the flag total. On the bcfishpass config, BT rearing cost falls by 2,503 km over #283’s 51 WSGs, so default now carries about 911 km more BT rearing than the bcfishpass config, not 1,592 km less. The #284 scores reproduce (habitat_change.csv 36 / 36).
  • Unchanged: the rules and flags, so segment-level replication of bcfishpass is untouched. bcfishpass’s own published rearing km (smnorris/bcfishpass@620dbe4) equal rearing_km + rearing_lake_connection_km. lnk_aggregate() still sums the flag.
  • Write-up: research/habitat_thresholds.md, “One meaning of rearing_km (#319)”.

The compare rollups report lake connectors apart from rearing km (#317). FWA 1450 lines (“Construction line, connection”) join each tributary mouth to a lake’s main-flow line, so lake km had grown with a lake’s tributary count.

  • Rollups.
    • lnk_compare_rollup() / lnk_compare_wsg() gain a 9th habitat type, rearing_lake_connection (km).
    • rearing and rearing_lake km leave 1450 lines in lake and reservoir polygons out, on the link side and the bcfishpass side alike. bcfishpass counts these lines too, in BT and SK rearing.
    • Stream + lake + wetland still sums to rearing.
    • 1400 (“other flow / inferred connection”) is a construction flow line, not a connector, and stays in the km.
  • The rules are unchanged. fresh’s rearing flag still carries the 1450 lines, so cluster_rearing still joins inlet rearing to the lake.
  • lnk_rollup_wsg() gains a connection alias. Its default rearing_km is still every line flagged rearing, which is what the validator’s cost and the parity scripts compare.
  • data-raw/compare_rollups.R refuses to compare a directory that mixes pre- and post-0.62.0 rollups.
  • What moves (ADMS, NATR; data-raw/logs/lake_connection_317/):
    • Against bcfishpass, ADMS CH goes from +91.1 % to +38.9 %, CO from +75.0 % to +28.7 % and BT from +16.4 % to +2.0 %. CH and CO stay high because bcfishpass has no CH or CO lake rearing on ADMS.
    • SK rearing km fall 55–95 %, identically on both sides, so SK parity is 0.0 % on six bcfishpass-config WSGs. KO falls 62 % on NATR.
    • Write-up: research/habitat_thresholds.md, “Lake connection lines”.
  • Follow-ups:
    • #319: one meaning of rearing_km everywhere.
    • #316: wetland construction flow lines.
    • #321: lnk_aggregate() counts only the crossing’s own line.

In default, lake and wetland rearing need same-species spawning within 10 km, and lake centrelines count in rearing (#310). The rollups now split rearing km by the polygon each line sits in. That renames two rollup labels and redefines a third.

  • Rules.
    • lnk_rules_build() reads rear_lake_connected_distance_max / rear_wetland_connected_distance_max and puts requires_connected: spawning on a species’ first rear L / W rule only, which is the rule fresh’s bucket reads (fresh#240).
    • The additive lake rule admits lake and reservoir lines (1000/1100/1200/1250/1300/1350/1400/1450/1475) with thresholds: false.
    • The builder refuses connected rearing, and an area_only spawning anchor, for species whose spawning requires rearing.
    • Breaking: the retired rear_requires_connected / rear_connected_distance_max columns stop the build when set.
  • default.
    • The distance is 10 km for BT, CH, CO, GR (lakes only), RB, ST and WCT. CT and DV carry it too but get no rules, because neither has a thresholds row.
    • CO has no lake or wetland size floor.
    • SK, KO and bcfishpass are unchanged.
  • Rollups (breaking).
    • rearing_lake_centerline / rearing_wetland_centerline become rearing_lake / rearing_wetland.
    • rearing_stream keeps its name but now means rearing outside lake, reservoir and wetland polygons. Lines are matched by waterbody_key against the polygon tables fresh reads, not by edge type.
    • Lake hectares include reservoirs, and stream + lake + wetland = rearing.
    • lnk_rollup_wsg() gains a waterbody alias; its default metrics are unchanged.
    • data-raw/compare_rollups.R refuses to compare a pre-0.61.0 directory with a later one.
  • What moves. Measured on ADMS and NATR on one segmentation (data-raw/logs/lake_connected_310/):
    • Spawning is unchanged, and SK / KO are row-identical.
    • Rearing rises 237–525 km per lake species, almost all of it lake lines. Adams Lake’s 1450 connection lines take ADMS CH to +91 % against bcfishpass, and that inclusion is under review.
    • Buckets move little (NATR BT wetland −2 %).
    • Write-up: research/habitat_thresholds.md, “Lake and wetland rearing connected to spawning”.
  • config_hash changes for every default* bundle.

link pins fresh v0.39.0, and a declared wetland floor bounds both wetland rear rules (#311). default* outputs move with no config change. Cyphers must be re-prepped before the next dispatch.

  • What fresh changes.
    • A W rule’s wetland_ha_min now gates rearing (fresh#237).
    • Lake and wetland buckets are sized by polygon area alone (fresh#240). On NATR BT the lake bucket goes from 310 to 521 km and the wetland bucket from 684 to 1,287 km.
  • What link changes. lnk_rules_build() puts a species’ rear_wetland_ha_min on the 1050/1150 wetland-flow rear rule as well, as a waterbody_type: W rule.
    • It goes after the polygon W rule, so fresh’s bucket rule and its requires_connected anchor are unchanged.
    • It applies only when the polygon rule is emitted.
    • It affects BT, CH, CO, RB, ST and WCT. bcfishpass is unchanged.
    • The stream rule still admits mainlines in smaller wetlands.
  • What moves. Measured on one segmentation per WSG (ADMS, BULK, NATR, PARS; data-raw/logs/wetland_floor_311/):
    • The floor removes 198.8 km of rearing. That is sub-floor wetland-flow rearing going to 0, plus rearing it disconnects from spawning.
    • It costs 3 of 2,265 observation species-locations.
    • It moves the departure from bcfishpass by at most 1.1 points.
    • Write-up: research/habitat_thresholds.md, “The wetland floor on rearing”.
  • config_hash changes for every default* bundle.

default gives BT, GR, KO and RB mean-annual-discharge ranges, converted from its own channel-width minima (#307). A default group put on mad now keeps stream habitat for them instead of none, so cw and mad inside default test one stream size two ways. default_tuned keeps #302’s observed ranges.

  • The values. Each width minimum maps to the median mad_m3s at that modelled width, floored to two significant figures.
    • Spawning 2 m → 0.041 m³/s, and GR’s 4 m → 0.20. Rearing 1.5 m → 0.021.
    • Maxima are open. KO rearing is lake-only and stays unranged.
    • data-raw/query_width_mad_equivalent.R produces them and reproduces #302’s medians exactly.
  • What moves. Nothing under cw: every digest matches on NATR and ADMS. Under mad, BT, GR, KO and RB stream habitat goes from 0 to, for example, NATR BT 1,487 / 2,525 km spawning / rearing. Within default, mad agrees with cw within about 5 %.
  • A known artifact, not fixed here. Under mad the lake_rearing / wetland_rearing buckets shrink.
    • fresh sizes a lake by the stream through it, and the buckets ignore spawning connectivity.
    • Fixes are in fresh#240 and #310; #310 also drops CO lake rearing.
  • config_hash changes for default and default_tuned. #302’s MAD ladders regenerate only at v0.58.0: on --base=default the variants harness now stops with “variant … equals default”.

Discharge is filled on river-polygon main stems, and cw against mad is re-scored with it (#305). #300 found that mad’s most-used loss was edge type 1250 water with no mad_m3s, not a MAD threshold.

  • The fill. pipeline: discharge_fill, set in default_tuned only. An edge 1250 line with no value takes the nearest valued line upstream on its blue_line_key. Failing that it takes the nearest valued line downstream, and failing that the largest value on any upstream line. It never takes the receiving river’s value. It runs only in groups the discharge table covers, and only where something reads discharge: a mad group or a rule-level mad. mad_m3s_source names where each value came from.
  • One rule, recorded. prepare, the run log (new discharge_fill column), lnk_habitat_validate() and the variants harness all decide the fill with the same rule. The validator scores each group on the fill prepare applied there, refuses a run logged with another state, and reports mad_m3s_source. prepare now writes mad_m3s as double precision itself.
  • What it changes. On #300’s held-out groups and segmentation, BT rearing’s cw-only band falls from 0.71 to 0.19 of the size-matched core rate. BT and RB spawning now read as the models differing only on little-used water. RB rearing still reads both ways. mad keeps −32 % BT and −76 % GR stream rearing against cw. Nothing is moved to mad; research/habitat_thresholds.md, “Filling discharge on river-polygon main stems”, has the numbers.

The discharge (mad) habitat model is scored against channel width (cw) on fish observations (#300). Package code is unchanged. The work is in the scoring harness, the research doc and RUNBOOK §7.

  • What mad costs. At default_tuned’s ranges, on held-out groups, mad keeps −36 % BT, −82 % GR and −4 % RB stream rearing against cw. This is the first build of the landed ranges; #302’s figures were lower bounds.
  • What drives it is a gap in the discharge data, not a threshold. Most of the used water mad drops is river-polygon main stems (edge type 1250), which carry no discharge value. Fish use those reaches at 2.7× the rate of habitat both models keep. Below its minimum, mad drops small water fish use at about a fifth of the rate of comparable small streams. Fill discharge on main stems before moving any group to mad. research/habitat_thresholds.md, “cw against mad”, has the per-species verdicts.
  • The variants harness compares models. --base=<bundle> sets the base bundle. A model-only variant changes no threshold. A size-adjusted score reads each model’s exclusive habitat within stream-order classes. --step=bundles writes bundles without touching the database. Re-scoring #284 and #302 reproduces their outputs.

default_tuned gives BT, GR, KO and RB a mean-annual-discharge range, scored against held-out fish observations (#302). Under the mad habitat model a species with no range loses every stream rule, and default gives these four none. The ranges are inert while every watershed group is on cw, which is every group today.

  • The values (m³/s, every maximum open): BT 0.078 for spawning and rearing, GR 0.96 spawning and 0.97 rearing, RB 0.011 spawning and 0.019 rearing, KO 0.57 spawning. KO’s is unscored, because it has no held-out group.
  • How they were set. Each candidate is the 5th percentile of discharge at fish observations in 46 calibration groups (data-raw/query_habitat_thresholds_mad.R). Each was then scored on held-out groups under a rule fixed before the build. Every loosening past the 10th percentile was refused except RB spawning.
  • Read before moving a group to mad. At these values a mad group keeps at least a third less BT stream rearing than cw, and at least four-fifths less GR, on the held-out groups. Stream size and sampling effort are not yet separated. research/habitat_thresholds.md, “MAD (discharge)”, has the numbers.
  • The threshold-variant harness now scores discharge ladders. It gains model and set columns, an anchor rung from cw to mad, downward _min steps and --floor. Re-scoring #284’s build reproduces its outputs.

lnk_habitat_validate() scores each watershed group on the habitat model it was classified on (#299). It reads the method table in the cfg it is given and resolves each group’s model with fresh’s own rule, which lnk_pipeline_classify() now shares.

  • On a mad group, predicates, size relaxation and width_null use mean annual discharge (mad_m3s), joined from fwa_stream_networks_discharge on linear_feature_id because the persist does not carry it. Before, they used channel width, which the classification never tested.
  • Reason labels are unchanged (fails_width means the group’s size dimension). observations gains model and mad_m3s, and summary gains model.
  • New reason no_mad_threshold: a mad miss that a MAD range alone would admit, for the species that have none (BT, GR, KO, RB). Adding those thresholds is #302.
  • cw output is unchanged apart from the new columns. A run classified with a different method table, or with lnk_pipeline_classify(method_csv =), is not detected, so swap the table on the cfg you pass. lnk_preflight_fresh() now also checks frs_habitat_predicates(model).

Each config can now choose channel width or discharge per watershed group (#286). Bundles declare parameters_habitat_method.csv (watershed_group_code, model = cw or mad), which lnk_pipeline_classify() passes to fresh as params_method; a new method_csv = argument overrides it.

  • Outputs are unchanged. Every shipped bundle is all cw, a frozen copy of bcfishpass example_newgraph. It is deliberately outside csv-sync, so moving a group to mad is a reviewed edit. ADMS and BULK classify byte-identically before and after.
  • Under mad a group classifies on mad_m3s, which is joined onto the working streams and never persisted, and it skips the stream-order rearing bypass. Species with no MAD thresholds (BT, GR, KO, RB) lose their stream habitat there. A group without discharge coverage loses all of it, with no error.
  • fresh >= 0.35.0 is required (pinned at v0.36.2). lnk_preflight_fresh() now checks for the params_method argument, so re-prep cypher images before the next run. fresh 0.36.0 also reversed frs_db_conn()’s env-var precedence; lnk_db_conn() is unchanged.
  • config_hash changes for every bundle, and log_input now fingerprints fwa_stream_networks_discharge.

The calibrated BT rearing gradient in default_tuned is scored against fish, and it holds (#284). New lnk_habitat_validate_band() measures observation locations per km on the segments a threshold step moves, against the core every step agrees on. It joins on the full key, and stops when two schemas’ segmentation differs or a schema holds no habitat for a watershed group.

  • The result. On eight watershed groups held out from its calibration, the steps from 0.1049 to 0.1349 add rearing that bull trout use at 0.60 and 0.71 of the core rate (0.75 and 0.89 at the same elevation). The step to 0.1449 does not (8 found where 25 were expected). default_tuned keeps 0.1349, which adds 1,105 km of BT rearing (+5.8 %) across the 12 groups scored.
  • Scored on one shared segmentation. data-raw/habitat_variants_build.R models a drainage closure once and re-classifies each threshold variant on it; data-raw/habitat_variants_score.R applies a rule fixed before the run and writes the gradient taper, an elevation split and the per-group habitat change. Inputs are data in data-raw/habitat_score/.
  • Elevation confounds gradient. Core bull trout rearing thins from 24.6 to 10.2 locations per 100 km between the low and high thirds of each group, and steep water sits high. Weighting habitat instead of cutting it continues in NewGraphEnvironment/knowledge#28.
  • CH is unscored: too few held-out Chinook records to decide any step. The method and every number are in research/habitat_thresholds.md, “Step 5”.

FISS absence taxa are now data (data-raw/fiss_absence_taxa.csv), and the observation-validation drivers share one input loader; the #283 outputs are unchanged.

Which observation species count as evidence for which model species is now data, not code (#290). New lnk_species_pooling() resolves, per watershed group, which observation records count for each model species.

  • The tracker. It reads a tracker in the bundle, species_pooling.csv: one dated, sourced row per decision, scoped to a region, sub-region or watershed group.
  • Groups. Either side of a row can be a species or a group from species_groups.csv, so pooling Dolly Varden into bull trout, or all salmon together, is a row rather than a code change.
  • Resolution. The most specific scope wins, anything unlisted is not pooled, and an optional obs_year_max limits a row to records dated in or before a year.
  • Regions come from the new package-level inst/extdata/wsg_regions.csv: the drainage each group’s outlet flows to, plus curated sub-regions.

default pools DV into BT in the Fraser, Mackenzie and Columbia (with the Kootenay), and in the Skeena only above Hazelton. - Why the split: in the interior, DV is a name the records stopped using in the 1990s. In the Skeena it is still the recorded name, 86 % or more of char records in every decade. - The effect on default_tuned: 852 lower-Skeena DV records leave the #284 bull trout evidence. The calibrated BT rearing gradient stays at 0.1349, but its margin is now 0.0009. - The write-up: research/species_pooling.md has the naming history, the maps and five pooling scenarios.

lnk_habitat_validate() accepts the per-watershed-group table as species_obs, applies year limits record by record, and uses observation_date when a source has one. data-raw/habitat_validate.R resolves pooling once and applies it to both bundles it compares. config_hash changes for default and default_tuned, which now declare the tracker and hash the region lookup; no model output changed.

Habitat can now be scored against fish, not only against bcfishpass (#283). New lnk_habitat_validate() scores a persisted run. For each watershed group, species and stage it reports:

  • capture: the share of observation locations on modelled accessible, spawning and rearing segments;
  • cost: km of habitat, so capture can’t be bought by calling everything habitat;
  • miss reasons: the bundle’s own habitat predicates re-evaluated with gradient and width relaxed, so fails_gradient means the gradient maximum is binding;
  • absences (optional), as a false-positive check.

Observations can come from any source: bcfishobs by default, another table, or a data frame located on the FWA network. The bcfishobs filters default on and fail loudly when a source lacks their column. data-raw/habitat_validate.R compares two bundles.

Read with care:

  • Chinook spawning capture in default is mostly the user_habitat_classification overlay: 237 of 245 spawn-staged locations sit in forced reaches. The *_outside_uhc shares report the rest.
  • BT rear_gradient_max 0.1349 was set from these same observations, so scoring it here is in-sample. Hold records out to score it fairly.

The baseline (default vs bcfishpass, 51 WSGs) and the method are in research/habitat_validation.md. schema is a required argument, because default’s pipeline.schema names the schema the bcfishpass bundle writes. The function stops if a schema’s run log records another config.

default_tuned carries its first calibrated value (#284): BT rear_gradient_max 0.1049 → 0.1349. The evidence is bull trout and Dolly Varden observations pooled (inland DV records are bull trout under the other name), n 4,764 on accessible segments, P95 0.135, and Isaak et al. (2015)’s 15 % natal-habitat envelope agrees. Every other CH and BT gradient maximum and channel-width minimum was examined and kept, as were spawn_gradient_min and cluster_bridge_gradient. So default_tuned differs from default in one cell and still inherits parameters_fresh.csv. The values are candidates, unscored until the observation validation (#283) runs. default and bcfishpass are unchanged.

The evidence is in research/habitat_thresholds.md, one verdict per threshold with its observation, FISS-site and literature support. Three rule changes were made after the numbers had been seen, and each is disclosed there with what it moved. The producers are data-raw/query_habitat_thresholds_{obs,fiss}.R, writing to data-raw/logs/habitat_thresholds_284/; BT-only results are recorded beside the pooled ones. research/ now has a README index.

Each config bundle now carries its own habitat thresholds (#282): per-species spawn/rear gradient max, channel-width min/max and lake-area floor, in parameters_habitat_thresholds.csv under files:. Before this, every bundle read fresh’s installed copy, so tuning a CH cutoff would also have moved the bcfishpass parity reference.

Where the numbers come from now. lnk_pipeline_classify() and lnk_pipeline_connect() default thresholds_csv to NULL and resolve it from cfg. A custom bundle that declares no CSV falls back to fresh’s copy with a message, and its config_hash covers that file. The four shipped bundles carry a byte-identical copy of ’s CSV (bcfishpass example_newgraph@4699d0f plus fresh’s edge-type columns). The bcfishpass copy is a frozen parity input: csv-sync does not touch it. Runs record the values themselves in a new <persist>.log_parameters_habitat_thresholds table, built from a new dictionary. The config snapshot now gates per table, so a hash that was logged before a table existed still gets that table filled.

default_tuned is a thin bundle (extends: default, schema fresh_default_tuned). It is where the calibrated CH and BT values from #284 will go. Today its CSV is identical to default’s.

extends: had never been used by a shipped bundle, and five parts of it were broken. Inherited provenance resolved against the child’s directory, so lnk_config_verify() reported every inherited file missing and every run was flagged config_drift. config_hash named inherited files by absolute path and sorted by LC_COLLATE (default hashed 008ed3a1… under en_US and 4669cd38… under C), so the same bundle hashed differently on different machines. It also never read the parent’s config.yaml, so an inherited break_order change left the child’s hash unchanged. A three-level chain failed to load, and a relative extends: resolved against the working directory. All five are fixed. Every bundle’s config_hash changes in this release; existing log rows keep their old hashes.

Two columns look live and are not. On link’s rules path, the CSV’s MAD bounds and spawn/rear_edge_types are carried but never applied, and rear_lake_ha_min is baked into rules.yaml, so it only takes effect after the rules are rebuilt. build_rules.R, regen_provenance.R and audit §2 now build each bundle’s rules from that bundle’s own thresholds. The dictionary’s consumed_by column and RUNBOOK §7 record which file and line reads each column. The stale spawn_gradient_min 0.0025 claim was corrected in six places; every bundle ships 0.

Verified on fixed segmentation, because full runs are not reproducible on main. Two identical ADMS runs on main differ by one segment: the PSCIS-to-modelled crossing pick has no tiebreak when two candidates share a linear_feature_id. That is a separate fix; until it lands, compare code versions by re-running classify and connect on one prepared schema. On that basis, output is byte-identical to 0.50.0 on ADMS (4 species) and BULK (6 species). A thin bundle with CH rear_gradient_max 0.0321 changes CH rearing on ADMS (1,470 → 1,280 segments) and nothing else. Record: data-raw/logs/habitat_thresholds_282/.

Suite: R CMD check 1939 pass (0.50.0: 1867); the only failures are 3 tunnel-dependent tests, the same as the baseline.

Closes the code-identity gaps in fresh.log (#264), the last provenance work before the 217-WSG run. Measured against the 39 rows in the table: fresh_sha filled on 15, fresh_dirty on 0, and bcfishobs — a model input — carrying no code pin at all.

The dispatcher’s fresh_sha was never missing. It was unread. The standing explanation, repeated in CLAUDE.md, RUNBOOK.md, research/study_area_run.md and data-raw/study_area_verify.sql, was that m1 installs fresh locally (RemoteType: local, no RemoteSha). Measured 2026-09-01: m1’s install carries RemoteType github, RemoteRef v0.33.0 and a RemoteSha byte-identical to the SHA every cypher records. .lnk_pkg_git_sha() read an env var, then walked for .git, then gave up — and an installed package has no .git. The tolerance in the verifier was a workaround for an unread field, not an absent one, and a false rationale invites the fix in the wrong direction.

One resolver, two wrappers. New .lnk_pkg_git_state() returns list(sha, dirty, source); .lnk_pkg_git_sha() and .lnk_pkg_git_dirty() delegate. Two functions reading the same DESCRIPTION and the same checkout are free to drift about one package, silently. sha and dirty still walk their own tier lists over that shared lookup, because cypher_prep.sh has always written FRESH_GIT_SHA and never FRESH_GIT_DIRTY — a design where one env var claims the whole state leaves dirty NA on exactly the host that set the SHA.

Two inferences that had to be bounded rather than taken. Tier 2’s dirty = FALSE is sound — RemoteSha is only written for an install from a published ref, which is not a working tree — but it is gated on identical(sha, remote_sha), since applied to any other SHA it certifies a commit the host never built. And the tier applies a hex shape guard: a RemoteType: standard install writes the package version into RemoteSha, and 278 of 300 packages in this library carry a non-hex value there. Unfiltered it would put fresh_sha = "0.33.0" in a column two consumers read as a commit.

bcfishobs is a model input, not a reference dataset. lnk_barrier_overrides() counts its observations upstream of each barrier to decide which barriers are skipped, so different bcfishobs code gives different barriers, different access, a different mapping_code. New bcfishobs_sha column and .lnk_bcfishobs_sha(), sharing one body with .lnk_fwapg_sha(). No bcfishobs_dirty column, against the issue’s own proposal: fwapg has none either, because preflight_local() refuses to run over a dirty checkout and a column FALSE on every row is #257 pointing the other way. New fresh_sha_source records which tier answered, mirroring bcfp_pin_source.

fresh_sha is now a pre-flight parity key. It was excluded alongside link_sha on the joint grounds that one can only ever fail and the other could only ever pass vacuously. Only the first half survives. It is the one field proving every host runs the same fresh build rather than the same version string — #246 exactly, where a cypher silently ran the image’s fresh.

Where the review paid. Three rounds found fourteen defects, and rounds 2 and 3 found them inside the previous round’s fixes. scripts/update_hosts.sh installs via R CMD INSTALL of a source tarball to route around r-lib/pak#658, which writes no Remote* fields — so the new assertions would have failed that host after droplets were paid for. Teaching it to record what it installed then introduced three fresh defects of its own: | tail -3 swallowed a failed install so the pin was stamped for a build that never happened; pinning LINK_GIT_DIRTY=false would have made link_dirty permanently FALSE on the load_all dispatcher, which is #257 stuck in the direction nobody can notice; and resolving the SHA per host raced a mid-run push. The verifier gained a fresh_dirty = TRUE arm it had lacked entirely — the first draft gave the weaker IS NULL state a NOTE while the state that makes the newly-mandatory fresh_sha a lie closed the run with PASS.

Verified against the database, not an exit code. A log table built with the pre-#264 column set gains both columns via ADD COLUMN, and a real write lands all four values on log and log_recompute. The verifier was swept across nine single-fault states on a synthetic two-host fixture — every one answering correctly, healthy passing before and after, since a mutation left unrestored would leave later cases printing ticks against damaged data. The first attempt at that sweep reported a correct FAIL that had actually errored on a column the fixture lacked, which is why the post-restore case exists.

Rows logged before this release keep their NULLs and will now fail the verifier. That is correct: they cannot say which fresh build produced them, and it is not retroactively fixable.

Suite 1894 pass / 0 fail, 16 warnings (0.49.0 baseline: 1825 / 0 / 16).

Closes the provenance gaps that cannot be fixed after a run (#262, #257). The 217-WSG provincial run is next, and #246 exists because 93 WSGs once accumulated with no log tables at all — a partial record at 217 repeats that at scale.

Two of the four reported gaps were wired and unfed, and a third named the wrong source. run_label has been threaded from lnk_pipeline_run() to the INSERT since #127; it was NULL because nothing ever set LNK_RUN_LABEL. .lnk_bcfp_log_current() has been called at run open just as long; it returns NULL because it queries bcfishpass.log and the local docker fwapg holds zero bcfishpass tables — the run is tunnel-free by design. Implementing what the issue literally asked for would have added code beside working code twice and recorded a wrong value the third time.

A dispatch is now one queryable unit. New run_uid — minted once per study_area_run.sh invocation, shared by every host and every WSG — replaces “a time window plus a host list”, which is fragile at 34 WSGs on three hosts and ambiguous the moment two runs overlap. lnk_log_read() gains run_uid (which overrides latest, since DISTINCT ON would hide a WSG re-run inside the same dispatch) and phase. The identifier reaches cyphers because it is exported on both legs — the local loop and the ssh command string; the local-only version is the LNK_GUARD_DOWNSTREAM trap of #227, and it fails silently rather than loudly.

The recompute now records what it did. data-raw/wsg_recompute_one.R rewrites streams_access and streams_mapping_code — the values that actually ship — and logged nothing, so fresh.log recorded when a WSG was modelled and said nothing about when its persisted access last changed. New <schema>.log_recompute, its own table rather than a phase column, because a recompute row in log would win lnk_log_read()’s DISTINCT ON and be returned as “what produced this network”. It carries watershed_group_code, so schema_consolidate.R discovers it with no list to maintain, and LNK_LOG=0 keeps benchmark passes (recompute_sweep.sh, recompute_parity.sh) out of it — otherwise the new verify check would pass on benchmark noise.

on.exit() at the top level of an Rscript never fires, which is how the first draft of that logging shipped a dead failure handler. It attaches to the global environment, which never exits — measured, on both stop() and quit(). The same file already carried two dead handlers for that reason, the likely source of the 49 orphaned zz_lnk_mc_scratch_* tables recorded in #246. The work now runs inside a function, where on.exit is real.

The bcfp reference is pinned from the snapshot it was actually computed against. Three tiers (LNK_BCFP_MODEL_VERSION, bcfishpass.log, then the local bcfp_baselines.csv ledger) with a new bcfp_pin_source recording which answered. The env tier is not redundant: the ledger is per host and a cypher has no row of its own, so without it the majority of a 217-WSG run lands unpinned — the same shape, and the same fix, as FWAPG_GIT_SHA. Querying a live bcfishpass.log at compare time, as the issue proposed, would name the build the tunnel is at now rather than the one the numbers came from. bcfp_model_run_id stays NULL on the tunnel-free path because log.json carries no such key; honest absence beats a fabricated id.

link_dirty means something again (#257). It was t on all 21 dispatcher rows of the first provenanced run while the tracked tree was byte-identical to origin — the run writes ~15 files into a tracked log directory, so it dirtied its own checkout by operating. The predicate now excludes data-raw/logs and nothing else: untracked files elsewhere still count, because a new uncommitted R/*.R is invisible to a cypher and that is exactly the drift being detected. :(exclude) in long form, since :! aborts and an aborted git status returns empty, which reads as clean. And shQuote(), because system2() shell-quotes the command but pastes arguments on raw — the parentheses in :(top,exclude) were parsed by the shell, the command never ran, and the predicate returned NA for every input. That one was caught by running it; it is invisible by reading.

data-raw/study_area_verify.sql can now actually fail. It keyed on a 6-hour window and a hardcoded 34-WSG VALUES list — a scope pinned to nothing, and wrong for 217 before it was attempted. It now takes -v run_uid= and asserts via RAISE, so the exit status means something. expected_n is supplied from outside on purpose: deriving the expected set entirely from fresh.log is circular, since a WSG that never produced a log row vanishes from “expected” and that is the failure being checked for. New data-raw/study_area_verify_negative.sh proves it fails when it should and passes when it should — the healthy case runs first, because a script that always exits non-zero “detects” everything.

Runs the post-consolidate recompute N-wide (#250). It was a serial loop over every WSG in the run — 48 min of a 124 min run, single-threaded on a 10-core host, with the cyphers already burned so it was pure dispatcher time. --recompute-jobs= (default 4, max 16) replaces it. #205 had already made the per-WSG cost cheap; what remained was entirely serialisation.

The issue’s own safety argument had a hole, and finding it is most of this release. It states that each WSG’s recompute reads shared tables and writes only its own rows, so loop order is irrelevant. True of everything except lnk_access(), which unconditionally rebuilt the schema-scoped barrier views — names every sibling WSG reads. That is two distinct failures: DROP VIEW and the following CREATE autocommit separately, leaving an interval in which the view does not exist; and CREATE OR REPLACE VIEW takes an AccessExclusiveLock whose queued request blocks every subsequent AccessShareLock, so one job’s DDL stalls every sibling’s read until lock_timeout kills someone — surfacing as a silent [WARN]. The fix is therefore both: hoist the build out of the loop (new lnk_access(build_views =), new data-raw/barriers_views_build.R) and drop the redundant DROP (v0.47.3). build_views = FALSE verifies the views exist rather than trusting the caller, because an absent view otherwise fails deep inside the network walk with a bare relation does not exist that reads as one WSG’s problem.

Recompute failures could not previously fail the run at all. || echo sat inside the loop, so the subshell always exited 0 and the success line was unconditional; RUN_INCOMPLETE is assigned before the block and nothing after raised it. A run in which all 50 recomputes failed exited 0 and wrote a compare CSV. This is the more consequential half of the release — an incomplete recompute is silently wrong output, not missing output: those WSGs keep their pre-consolidate access, so token1/token2 and ;DAM are wrong for them while the compare CSV looks complete. It now has its own message saying so, with the per-WSG re-run command, kept distinct from the bucket-incomplete message so an operator is not sent to the wrong host.

New exported lnk_fanout_judge() is what makes that possible. A parallel loop over an empty work list exits 0 exactly like a successful one, and a per-item loop reports the exit of whichever item ran last — neither is something shell answers reliably, and this repo has no shell test harness, so the predicate lives in R where testthat can prove it. Five statuses, and the distinctions are the point: none_ran is not all_failed (nothing recorded anything, which indicts the harness rather than the work), and a job that failed is not a job that never reported — calling the second a failure sends someone to read a log that does not exist. It takes expected job names, not a count, so it can say which are missing; expected has no default, following n_expected in lnk_preflight_parity().

The pool is hand-rolled rather than xargs -P. xargs exits 123 for anything between 1 and 125, collapsing “1 of 50 failed” and “50 of 50 failed” into one status — exactly the distinction being reported. It tracks only liveness (kill -0 plus wait <pid>), since wait -n is bash 4.3+ and /bin/bash on macOS is 3.2.57; exit status comes from a per-job .rc file written last, so a job killed partway is judged “never reported” rather than counted as fine, and the evidence survives a killed dispatcher. Per-job logs go in ${TS}_recompute.d/ rather than flat, because that log directory is tracked — flat files would add 217 committed files per provincial run to a public repo.

Measured, and it is not the number the issue projected. Speedup is 1.43x on 4 WSGs and 1.51x on 8 at -j4, not ~3.4x. The cause is neither the pool nor the database: a pool’s makespan is bounded below by its slowest single job, and on both sets one WSG dominated – CHWK at 226 s is 74% of the 4-WSG set and 52% of the 8-WSG set, so the ceilings were 1.53x and 1.92x and the pool reached 93% and 79% of them. Two confident diagnoses were wrong on the way there (a Postgres worker-pool theory, refuted by a PGOPTIONS sweep that made scaling worse; a Docker I/O theory, refuted by 269 pg_stat_activity samples that were 100% CPU(running)), and both came from theorising about the aggregate instead of reading the per-job distribution. Full record with the numbers in research/recompute_parallel_2026_09_01.md.

Recompute cost does not track segment count – SALR is 20% larger than CHWK and finishes 20x faster, because the cost is the downstream barrier walk. So the pool now orders its work list longest-first from prior recompute times rather than alphabetically, falling back to input order (loudly) when no times exist. Note the recompute is not bucketed – buckets distribute modelling across hosts, and that packing was already made speed-aware in #253. Separately, checked against the 104 WSGs of modelling times already on disk, segment count predicts modelling time at R^2 0.636 with a 5x spread in unit cost; whether comp_weight should move to measured times is filed on its own, since measured times are not always present and a silently-degrading fallback is a recorded past incident.

Proof rather than assertion, since lnk_access(merge = TRUE) reads its own prior output (WHEN t.access_<sp> = 2 THEN 2 preserves observed access) and so is not a pure function of untouched inputs. data-raw/recompute_parity.sh runs three passes from a snapshot — serial, serial again on top of it, then parallel and shuffled from the same pre-state — so a failure points at one thing: A≠B means the recompute is not idempotent (nothing to do with parallelism), A=B≠C means parallelism broke something. data-raw/pool_probe.sh exercises the shipped pool functions (extracted, not copied) under both bash 5.3.9 and 3.2.57.

Stops lnk_barriers_views() dropping each view before recreating it (#250). CREATE OR REPLACE VIEW is atomic — a concurrent reader sees the old definition or the new one — but DROP VIEW IF EXISTS followed by a separate CREATE is not, because each statement autocommits. That pair left a real interval in which the view did not exist, so a concurrent fresh::frs_network_features() walk could fail with relation ... does not exist. Per call the statement count goes 38 → 19.

The DROP was belt-and-braces from the function’s first commit rather than a fix for anything, and the view column list has been byte-identical since — so CREATE OR REPLACE’s “may not rename, retype, reorder or drop an output column” restriction is satisfied. It had already cost something in the sequential case: RUNBOOK.md §6 records an orphaned backend holding a lock on barriers_bt_access while every later DROP VIEW blocked indefinitely, which is why wsg_recompute_one.R sets lock_timeout at all.

New recreate = TRUE restores the old pair for a genuine column-shape change. A shape change under the default stops with an error naming that flag rather than falling back automatically — an automatic fallback would reintroduce the window at the worst moment, part-way through a parallel recompute, where a sibling’s relation does not exist surfaces as a per-WSG [WARN] and not as an operator error. The unrelated-error path is tested too, so a missing barriers table cannot be relabelled as a shape change and send someone to fix the wrong thing.

The test asserts the absence of DROP VIEW, not merely the lower count: a count alone would still pass if a DROP were reintroduced alongside a removed CREATE. Verified by restoring the defect and confirming three tests go red.

Stops shipping comms/ and research/ in the built package (#235). R CMD build includes every top-level directory not named in .Rbuildignore, so pak::pak("NewGraphEnvironment/link") was installing 15 files of cross-repo coordination notes and 22 of research working files into the user’s library. R CMD check reports this only as a NOTE and .gitignore does not cover it. Verified against the built tarball rather than the config, because the regex is easy to get subtly wrong: both directories go 15/22 → 0. The gap opens over time rather than at scaffold — planning, dev, .claude and CLAUDE.md were already excluded, and these two were added to the repo later.

Also packs distributed work by finish time rather than segment count (#253). The hosts are not interchangeable: measured on identical work, the dispatcher runs at 0.0391 min per 1000 persisted segments and a cypher at 0.0872 — 2.23× slower. Balancing raw segments therefore balances work and unbalances time, and the makespan ends up set by a host that was given less to do; on the provincial set the dispatcher finished at 95 min while every cypher ran to ~190. Assigning each component to the host that would complete it earliest gives 149/149/150/150, cutting the provincial modelling phase by 41 minutes (21%) and taking the end-to-end estimate from 5.0 h to 4.3 h. --host-speeds= overrides the measured default. The relabelling step is gone rather than adapted — it packed into anonymous equal-speed bins and renamed them by load, which is meaningless once host 1 is the dispatcher and cannot be swapped for a cypher. Drainage-closure and downstream-first assertions were re-verified against the new assignment by restoring the bug, not inherited from the old one.

Adds research/run_record_2026_08_31_cypher_pilots.md and data-raw/logs/README.md — the first record under an internal issue, covering the four cypher pilots that produced the rates above and the four defects they exposed, including the ones where a correct diagnosis was retracted on a bad inference and where two gates passed tests that could not fail.

Fixes two defects in the v0.47.0 pre-flight gates, both found by piloting them against a real cypher for the first time (#246). Both would have blocked the provincial run, and neither could have been caught by the tests as written.

The parity gate’s dirty-tree check fired on every real run. preflight_hosts() ran lnk_preflight_parity() with the default forbid_dirty = TRUE, but by the time it runs both hosts are dirty by their own normal operation: the dispatcher has written five run logs into the tracked data-raw/logs/study_area_run/, and snapshot_bcfp.sh has stamped data-raw/logs/bcfp_baselines.csv on each cypher. The check now runs only pre-spin in preflight_local(), which is the meaningful moment — repo_sha equality still proves both hosts are on the same commit, which is what parity actually asserts. It passed its tests only because the fixtures set dirty = "FALSE": a fixture that could not reach the failure mode, which is the rule the same PR had just written into its own findings.

The vintage gate could not tell “absent” from “no statistics yet”. It reported never loaded / absent: bcfishobs.observations for a table that existed and was healthy. Statistics are collected by (auto)analyze, so on a freshly restored database — exactly what a cypher has after snapshot_bcfp — a table has rows and no stats at all, and GREATEST(last_analyze, last_autoanalyze) is NULL. The query now carries table_exists separately and falls back to the relation file’s mtime, verified against a deliberately missing table (both pg_relation_filepath and pg_stat_file are strict, so absence yields NULL rather than an error). Absence and ignorance are now distinct states — absent and unknown — because both fail but they send an operator to different places.

Both fixes keep the property that made the gates worth having: absence is still not a pass, a short result is still not agreement, and turning the dirty check off does not turn the gate off (tested — a repo_sha or fresh_version divergence still fails with forbid_dirty = FALSE).

Also records what the pilots established, since it is what the next run depends on. cypher_up reports ready roughly 227 s before cypher@ actually works, because it polls for a marker baked into the snapshot image while runcmd — which copies root’s SSH keys to the cypher user — is still running. link’s prep now waits on a deadline for that specific user and fails loudly rather than falling through to scp: Connection closed (shipped in 0.47.0’s line of work as #251); the root cause is fixed upstream in an internal pull request.

Closes the three gaps that would have made a provincial rerun silently skip most of its work (#246), and absorbs the sibling-host parity hook (#183).

The root cause was narrower than it looked, and it was a declaration. The cypher image bakes fresh 0.31.0, which exports neither frs_wsg_drainage nor frs_wsg_outlets — frs_wsg_drainage arrived in 0.32.0 and frs_wsg_outlets only in 0.33.0, so the floor is v0.33.0 rather than “newer than 0.31.0”. DESCRIPTION already pinned NewGraphEnvironment/fresh@v0.33.0, but the pin was never resolved: fresh sat in Suggests, and pak::local_install() defaults to dependencies = NA, which is hard dependencies only. Meanwhile 20 files call fresh:: with no requireNamespace() guard, including a default argument on an exported function — so fresh is a hard runtime dependency that was declared optional, and library(link) loaded cleanly on a host whose fresh could not run the pipeline. Moving it to Imports makes the existing pin load-bearing with no change to the pak call at all: upgrade = FALSE suppresses gratuitous upgrades, not required ones, and 0.31.0 does not satisfy the floor. The failure it prevents was silent in the worst way — wsg_run_one.R catches the missing-symbol error and quit(status = 1)s, the bucket loop logs [WARN] and continues, and the host exits 0 with none of its watershed groups persisted.

New exported lnk_preflight_fresh() asserts symbols rather than a version string, because a version is a proxy that fails in both directions: it reads 0.33.0 on a partial install or against a shadowing library path, and reads “wrong” while every needed symbol is present. A drift guard walks link’s own parsed namespace rather than the R/ directory — a source scan finds nothing in an installed package and would report a clean check — so a new fresh:: call site cannot ship undeclared. Two independent methods agree the real call-site count is 12; a naive grep would have listed 21 and included frs_point_snap_knn, which fresh 0.33.0 does not export and link never calls.

study_area_run.sh gains pre-flight gates in two blocks, because there are two questions. A cypher’s software is predictable from the dispatcher before the cypher exists — its link from git reset --hard origin/$BRANCH, its fresh from DESCRIPTION — so preflight_local() validates what the workers are going to get, free and before the spin, and preflight_hosts() confirms after prep that they got it. Predict before spend; verify before write. The second cannot run earlier and does not need to: it precedes any write to the persist, and a failure trips the existing EXIT trap, bounding the loss at prep time.

The credential probe is the substantive fix. doctl and the tofu provider use different tokens; both were minted 2026-05-18 and both expired on the same day, and only the first was ever checked. tofu plan cannot serve as the second probe — against a workspace with no resources it returns Plan: N to add without contacting DigitalOcean at all. The tfvars token now goes to /v2/account directly, with 401, an unreachable API and any other status reported as distinct outcomes rather than collapsed into “not success”.

New exported lnk_preflight_vintage() and lnk_preflight_parity() / lnk_preflight_stamp() put the load-bearing predicates in R where testthat can prove them. Vintage: absent is not fresh, a NULL timestamp is not fresh, and an empty result set is not a pass. last_analyze alone turned out to be unusable — measured NULL on all ten input primitives — so the query takes GREATEST of it and last_autoanalyze. Parity keys on repo_sha, not link_sha: the latter is a real SHA on the load_all dispatcher and NA on every pak-installed cypher, so comparing it would fail every legitimate run, while fresh_sha is NA on both and would pass vacuously. repo_sha is read on each host from the checkout it installed from — evidence, rather than a restatement of what the dispatcher believes. For the same reason cypher_prep.sh writes each cypher’s own observed SHA into its ~/.Renviron, while only FWAPG_GIT_SHA crosses the ssh boundary, since cyphers have no fwapg checkout and would otherwise log NULL.

Two post-conditions, and a sentinel that was wrong in both directions. schema_consolidate DELETEs the destination bucket and then COPYs, so a host that produced nothing does not merely fail to add rows — it removes the rows already there and returns ok = TRUE. Under #246 the run was deleting data and reporting success. Every host must now account for its whole bucket before consolidate, and every run WSG must have rows in the persist afterwards. Separately, the prep gate grepped for snapshot_bcfp.sh: complete, which the snapshot emits before lnk_persist_init runs: a persist_init FATAL exited 1 with the sentinel already in the log, so the gate passed and watershed groups ran against a half-prepped cypher. The snapshot’s legitimate skip-if-current path, meanwhile, never emits it, so a skipped load read as FATAL. An anchored === READY is correct in all four measured cases.

New data-raw/study_area_buckets.R derives the host buckets instead of asserting them: union-find over per-WSG frs_wsg_drainage() closures, then greedy LPT over the resulting components, writing research/study_areas.md. Components rather than study areas because overlapping closures make consolidate last-writer-wins on the shared groups — whichever host finished last would silently decide the answer — so the script asserts disjointness rather than assuming it. It reproduces the issue’s numbers from first principles: 96 focal groups, 125 in closure, 119 modelable, 22 drainage-independent components, 39 on the dispatcher, and the same six groups dropped by species presence.

There is deliberately no global pre-flight bypass. --preflight-note="<why>" downgrades only vintage and parity, and only with a written justification, mirroring lnk_wsg_downstream_check(override=); --auto-install is remediation rather than a skip. New --preflight-only runs every local gate and exits, so the whole block is testable with one command and no spend. Every gate was exercised against both known answers on m1 before shipping — tables in the PR and in RUNBOOK.md §8d.

Enforces the DS-first precondition that data-raw/wsg_run_one.R has stated in its own header since link#175 and that nothing checked (#227). Accessibility is not computed from the focal watershed group alone — link reads the already-persisted barriers of the groups downstream — so modelling a WSG out of order writes streams_access / streams_mapping_code marking segments accessible that are in fact dammed off, prints done in N min, and exits 0. A wrong answer indistinguishable from a right one. New exported lnk_wsg_downstream_check() verifies it instead: find the blocking dams on the focal WSG’s downstream flow path, confirm each is already persisted as a barrier, and fail loud naming them when it is not.

The predicate is path, not membership, and that is the difference between a guard and a nuisance. The issue proposed flagging when a downstream group contains a blocking dam; measured against the live database that fires on BULK — whose closure holds 18 blocking dams across LSKE/KISP/KLUM, none of them below its outlet — which is the issue’s own motivating example. Operators would learn to reach for the override and the guard would stop meaning anything. Testing each dam with the measure-aware whse_basemapping.fwa_downstream() from the outlet shipped by fresh::frs_wsg_outlets() gives PARS its three real dams (Peace Canyon, Site C, W.A.C. Bennett), SLOC the Brilliant Dam, and BULK nothing. It is complete rather than merely cheaper: access walks downstream from every segment and every focal segment exits through the focal outlet, so the out-of-WSG barriers reachable from any focal segment are exactly those below it. ~0.5 s against a 5 s budget.

The guard applies link’s own filters by sharing the pipeline’s SQL, not copying it: the cabd and matched CTE bodies now live once and are consumed by both .lnk_pipeline_prep_dams() and the probe, parameterized on source so the pipeline reads its staged tables while the guard inlines (VALUES …) and writes nothing — the pattern fresh 0.33.0 used to retire public.wsg_outlet. A guard that snapped or filtered differently would flag dams the pipeline treats as passable, which is the issue’s stated make-or-break. The refactor is behaviour-preserving and was verified so, against a golden capture taken first: byte-identical for ADMS, KOTL and PARS.

Three tiers. Auto-pass is the common case. Failure names the dams, their passability codes and the DS-first order to model them in, and offers wsg_recompute_one.R as a correct escape that is not the override. The override requires a written justification — a bare TRUE is rejected, because the justification is the mechanism — and lands in <persist>.log.notes beside wsg_upstream (#127), so lnk_log_read() reports afterwards that a network was built on a stated assumption. study_area_run.sh exports LNK_GUARD_DOWNSTREAM=warn on both legs, since a downstream group can legitimately be mid-flight on another cypher; that is a deferral rather than a hole because wsg_recompute_one.R re-runs the guard in error mode after consolidate, when everything must be persisted. A hard pre-flight there would be worse than the bug: per-WSG failures soft-fail, so the WSG would be skipped, and lnk_access(merge = TRUE) cannot repair a group that was never modelled.

Also fixes a defect shipped in 0.45.0: .lnk_log_create_tables() built the run-log tables but never the schema, so a brand-new persist schema failed with schema does not exist. The log is opened before lnk_persist_init by design — the open row must predate any write so wsg_upstream reflects the state the run started from — and every schema tested until now already existed. Found by running the guard’s smoke tests into scratch schemas. New RUNBOOK.md §8c carries the membership-vs-path measurement so it is not “simplified” back, and records the guard’s known bound: it inherits frs_wsg_drainage()’s one-outlet-per-group model. Follow-up #244 records that cabd_additions dams carry barrier_ind = t yet can never become barriers.

Purges stale public.wsg_outlet references and documents the ltree-closure trap (#243). #227/v0.45.1 removed the table as a concept — fresh@v0.33.0 ships outlets and passes them as a VALUES list — but references survived in places that still steered behaviour, including one shipped in v0.45.0: R/lnk_log.R listed public.wsg_outlet as a pipeline input primitive, so the run-provenance log fingerprinted a table that should not exist (real numbers on a machine carrying the leftover, NULLs on a clean one). research/study_areas.md was worse, documenting the superseded technique as current — “closure and downstream-first order are derived from wscode_ltree ancestry (@>)” — and citing #227 as tracking the table’s reproducible build, when #227 had been re-scoped to delete it. data-raw/study_area_wsgs.R was already a shim around lnk_wsg_resolve(), so no live code path computed a wrong closure; the damage was documentation steering a reader into the trap, which is exactly what it did. New RUNBOOK.md §8b records the mechanism: two watershed groups on the same stream share an outlet code, so a.outlet @> b.outlet is true in both directions and reports each as downstream of the other — closure is measure-aware, not code-aware. The worked example is the Kootenay, where FWA carries the out-and-back through Montana and Idaho under one continuous wscode_ltree = 300.625474: an ltree test puts KOTR/SMAR/BULL below Kootenay Lake when they sit above it via the US loop, and the true closure of LARL/KOTL/SLOC is just those three, LARL being the terminal BC group. §8b also warns that a public.wsg_outlet table found in any database is a leftover that still answers queries, wrongly. Found by falling into it — a hand-rolled @> query invented a five-WSG “outstanding downstream closure” that does not exist.

Drops the gr (Arctic grayling) presence flag for LARL (Lower Arrow Lake) in the default config bundle (#242). Grayling is an Arctic-drainage species and LARL is Columbia; because wsg_species_presence gates whether a species is modelled at all, the flag was producing a full grayling habitat and mapping_code layer for the Lower Arrow Lake system. It was not fabricated — four LARL observations spanning 1954-1995, plus one each in BULL and ELKR — but six historical records across three Columbia groups, newest 1995, is a thin basis for modelling a species across a watershed group, and most are plausibly mid-century stocking that never established. Scoped to default only: bcfishpass carries gr = true for LARL upstream, so the bcfishpass bundle is left byte-identical to it rather than silently diverged, since that is the bundle used for parity. The two copies of this file were identical and now differ deliberately; they converge again if NewGraphEnvironment/bcfishpass#14 lands. No upstream PR was needed first because this file is hand-authored in link (source: link (hand-authored), no source: bcfp in the files: block), so it is off the weekly csv-sync path. Also carries the 2026-08-12 csv-sync of pscis_modelledcrossings_streams_xref.csv (#239).

Adopts fresh 0.33.0, which rebuilt frs_wsg_drainage() on per-group outlet points and the measure-aware whse_basemapping.fwa_downstream() (fresh#214). This closes the public.wsg_outlet half of #227 by removing the dependency rather than repairing it — outlets now ship with fresh and reach the database as a VALUES list, so there is no ad-hoc table to materialize and none to lose on a docker volume rebuild. test-lnk_wsg_resolve.R closure-mode tests, red since the table went missing, pass again.

Closures tighten, so lnk_wsg_resolve() expectations move with them: c("PARS", "BULK") resolves to 9 WSGs where it previously returned 15. The dropped six were all over-inclusions of the old single-ltree predicate — LKEL is a separate Skeena tributary, MSKE and USKE sit upstream of the Bulkley confluence, LBTN is a Peace sibling, MORR is upstream of BULK, and FINA is a separate Williston arm the Parsnip never flows through. Study-area buckets built before this will differ from ones built after; the earlier runs were over-inclusive rather than short, so their results stand, but bucket membership is not comparable across the change. Adds a companion test pinning the exclusions by name, and bumps the fresh floor to 0.33.0. # link 0.45.0

Run provenance on the persist schema (#127). A network in <persist_schema> previously carried no record of what produced it: given streams and streams_mapping_code you could not say which config was used, what that config actually said at the time, which vintage of the DB primitives it read, or when it was written. Four sidecar tables now answer that, mirroring bcfishpass.log and its log_parameters_* children — log (one row per lnk_pipeline_run(), opened before anything writes and closed on success), log_parameters_fresh and log_dimensions (the full CSV rows, so observation_species = BT;DV at threshold 1 with no date floor is recorded in the database rather than as a pointer to a file that will move on), and log_input (per-primitive fingerprints). Read them with the new lnk_log_read(). This is what makes scenario work tractable: #236 runs the DV-as-BT override with and without, and without provenance both results land in the same schema indistinguishable a week later. Three design points earned their keep. run_id is text, not a serial, because schema_consolidate.R COPYs literal values between hosts whose sequences would both start at 1 — a serial guarantees PK collisions and silent loss of run history. .lnk_log_align_columns() issues ALTER TABLE ... ADD COLUMN IF NOT EXISTS on every init, because CREATE TABLE IF NOT EXISTS silently no-ops on an existing table and therefore could never ship a new config column, which would reintroduce one layer up the exact silent drift the issue exists to prevent. And config_hash hashes the observed bytes of the resolved file set rather than the declared provenance: block, since config.yaml is absent from its own block — a declared-set hash would be blind to pipeline$schema, break_order and gradient_classes; config_drift is kept as the separate “did it match what it claimed” axis. Input provenance is read rather than recomputed, and is honest about its gaps: bcdata.log supplies authoritative download times but covers only bc2pg downloads, so not FWA, whose pg_stat_user_tables row is empty because it is bulk-restored — the stream network’s only real version is the fwapg checkout commit, captured as fwapg_sha. There is deliberately no count(*) anywhere (exact counts on a 4.9M-row, 9.8 GB table across a provincial pass would add hours; row counts are reltuples estimates flagged as such). Everything after the open row degrades to a warning, so provenance logging can never kill a modelling run, and on.exit rather than tryCatch preserves traceback() and the invisible(conn) contract — leaving a three-state completion signal where date_end NULL with notes means an R error and NULL without means SIGKILL. digest moves from Suggests to Imports. Not yet included: schema_consolidate.R support for the two config_hash-keyed tables, so they do not yet travel between hosts.

Data dictionaries for both config CSVs, and an end to re-deriving the fresh↔︎link column-ownership boundary (#233). inst/extdata/configs/dimensions_columns.csv is renamed dictionary_dimensions.csv (pure rename — no code read it; the only live reference was the #75 entry in CLAUDE.md), establishing a dictionary_<file> convention for its new sibling dictionary_parameters_fresh.csv. The new file documents all 19 parameters_fresh.csv columns — type, group, owner, consumed_by, default-when-absent, description — where previously the 9 cluster_* columns were undocumented outside fresh’s source and the rest lived in scattered prose. Every consumed_by is a machine-verified file:line (24/24 resolve), which corrected two things worth knowing: link does not read the cluster_* columns at all, it only passes the frame through (R/lnk_pipeline_connect.R:107) to fresh’s .frs_run_connectivity(), and rear_gradient_min is read by no code in either package — recorded as unused rather than dropped, since fresh owns that schema. The owner column encodes, rather than restates, the partition settled by fresh#129 (shipped fresh 0.12.7, which removed observation_* from fresh: “fish passage interpretation belongs in link, not the network engine”): 14 fresh-owned engine params, 5 link-owned observation_* interpretation params. data-raw/audit_configs.R §3b now reads that column instead of hardcoding grepl("^observation_", ...), so a new link-owned column is documented once rather than taught to a regex, and gains three guards — undocumented-column coverage, a reverse check (dictionary claims link-owned but fresh ships it = stale dictionary), and a missing-dictionary guard; negative-tested by dropping a row, which exits 1. Because data-raw/ is .Rbuildignored and never runs for an installed package, the durable guard is the new tests/testthat/test-dictionaries.R (+23 tests), which checks coverage in both directions against the union of bundle columns — bundles legitimately carry different subsets, bcfishpass’s dimensions.csv having 30 columns to the defaults’ 32. RUNBOOK.md §7 gains the ownership section so the boundary is findable without archaeology through two repos’ planning archives. Separately, audit_configs.R no longer setwd()s to a hardcoded /Users/airvine/... path — it derives the repo root from its own script location and resolves every path through a new repo_path() helper, so the audit runs from any working directory (verified from /tmp), works for anyone’s checkout, and mutates no global state when source()d.

Reader-facing prose cleanup on the PARS vignette (vignettes/pars-habitat-connectivity.Rmd), from review — no model, artifact, or numbers change. Dropped internal issue numbers and build-infrastructure (“tunnel”) references, and cut the per-segment mapping_code disagreement detail (the top-diff-pattern columns and the “remaining disagreements” paragraph) since bull-trout parity is ~99%. The accessible / spawning / rearing km table stays but is reframed around lnk_rollup_wsg() as a standalone habitat-totals roll-up — link’s own numbers, no reference required — that happens to match bcfishpass, rather than a proof-of-parity exercise. Removed the mapping_code symbology paragraph entirely: the colour semantics were easy to mis-state (e.g. red marks an accessible reach whose downstream PSCIS crossing is ranked a barrier, not merely “field-assessed”), and each map already carries its own colour legend.

Extend the PARS vignette to demonstrate accessible_km bcfishpass-equivalence (#226). vignettes/pars-habitat-connectivity.Rmd gains an Accessible habitat (km) section: a link-vs-bcfishpass roll-up table (bull-trout accessible / spawning / rearing, from the new cached pars_accessible.rds) plus computed prose — link models 6,822.5 km of accessible bull-trout habitat against bcfishpass’s 6,822.9 km, a −0.01% difference (the #223 access-segmentation fix; spawning/rearing agree within the 5% habitat band). data-raw/wsg_vignette_data.R grows the accessible roll-up (mirroring data-raw/parity_crosssection.R’s lnk_rollup_wsg() + fresh.streams_vw_bcfp IN (1,2) computation) and a segmentation-parity guard that refuses to build when fresh and fresh_default disagree by more than 1%. All cached artifacts were regenerated from the post-#223 state: this required re-modelling the default (Arctic grayling) config for PARS into fresh_default, which had been left on the pre-#223 segmentation (48,558 segments vs fresh’s 97,538) — a naive regeneration would have joined the grayling mapping_code onto mismatched geometry and corrupted the map. The re-model (data-raw/wsg_run_one.R + a merge = TRUE recompute for cross-WSG ;DAM) brings both configs onto the same segmentation; the previously hardcoded grayling map-caption counts are now computed inline from the loaded layer so they track the artifact. Bull-trout mapping_code parity refreshes to 98.91% (from 99.04%, on the denser post-#223 segmentation). Verified with a database-free knit.

Fix a BT/ST accessible_km over-credit and add the per-WSG accessible_km roll-up (#223, #221). The stream break source gradient_barriers_minimal was fed the fresh::frs_barriers_minimal() downstream-most reduction as a segmentation source, so a single stream segment straddled the accessibility frontier and its whole reach — including the blocked reach above the barrier — was credited accessible (accessible_km +40.4% PCEA, +23.6% FINA). The fix (R/lnk_pipeline_prepare.R) unions the raw per-model gradient + falls positions into gradient_barriers_minimal so streams break at every frontier — mirroring the orphan path already in the code and matching bcfishpass, which segments at every gradient barrier; fresh::frs_barriers_minimal() is now unused in link. lnk_compare_rollup() gains an accessible_km column and a reusable lnk_rollup_wsg() (tunnel-free vs fresh.streams_vw_bcfp, access_<sp> IN (1,2)). Proven with data-raw/parity_crosssection.R across 11 WSGs × 8 species (Peace / Fraser / Skeena / Columbia): accessible_km converges 44/44 within 0.05%, spawning/rearing hold — the only two over-tolerance being the documented parked BULK SK (fresh#190 dual-rearing-lake topology). The fix is habitat-neutral (byte-identical spawn/rear pre/post); segment count grows 2–3.5× (inherent to bcfp-matching, intersects #205). Follow-ups: #224 (bcfp dam_dnstr_ind reservoir-inflow propagation), #225 (rename gradient_barriers_minimal → gradient_barriers_break), #226 (PARS vignette accessible_km demo), #227 (public.wsg_outlet builder + single-WSG guard).

lnk_pipeline_run() now builds and persists <persist_schema>.streams_access (and access_<sp>) regardless of the mapping_code flag (#218). Access is foundational — mapping_code depends on it, not the reverse — so the access build (lnk_presence → pre-persist → lnk_barriers_views → lnk_pipeline_access) moves out of the if (isTRUE(mapping_code)) block to run unconditionally; only the mapping_code token assembly (lnk_mapping_code) stays gated. Habitat-only callers (mapping_code = FALSE — data-raw/wsg_pipeline_run.R, lnk_compare_wsg(mapping_code = FALSE)) now also emit streams_access. No persist change required: lnk_pipeline_persist() already probes for streams_access and streams_mapping_code independently and copies whichever working tables exist. The mapping_code = TRUE execution order (pre-persist → barriers_views → access → mapping_code → final persist) is byte-identical to v0.42.0, so the cached vignette parity (99.04% BT) is unchanged.

First package vignette: vignettes/pars-habitat-connectivity.Rmd — bull trout and Arctic grayling habitat and connectivity classification for the Parsnip River Watershed Group (PARS, FWCP Peace), rehearsed end-to-end so it can transfer into the Fish Passage Peace 2025 report appendix (#215). Two analyses: (1) parity — link’s bcfishpass config reproduces bcfishpass’s per-segment mapping_code for bull trout at 99.04%; (2) extension — link’s default config models Arctic grayling, which bcfishpass does not model at all. Map symbology reuses the bcfishpass symbology registry bundled in gq (gq::gq_reg_main() + gq_tmap_classes(), the same recipe fresh uses), so stream colours match a bcfishpass QGIS project exactly. The vignette is tunnel-free: the model run + comparison run once locally in data-raw/wsg_vignette_data.R, which caches artifacts to inst/vignette-data/ (pars.gpkg, pars_parity.rds); the vignette only loads those, so pkgdown CI builds it with no Postgres and no bcfishpass snapshot. New Suggests (bookdown, gq, knitr, rmarkdown) + VignetteBuilder: knitr + gq Remote.

data-raw/audit_configs.R is now a trustworthy pre-trifecta gate. The script grew an end-of-run rollup that aggregates every finding and exits non-zero when any fired (previously findings scrolled past inline and the script always exited 0, so the trifecta could not gate on a clean audit), plus a section comparing fresh’s canonical parameters_fresh.csv column set against each link bundle’s copy (flags engine params fresh added that link is missing; treats observation_* as expected link-only extensions — the parameters_fresh half of the fresh↔︎link config-drift gap, rules.yaml half tracked in #129). All 30 findings the audit had been emitting were audit-side defects, not config drift (lnk_config_verify reports 0 byte/shape drift in both bundles): §1 now calls the canonical lnk_config_verify() rather than a divergent homegrown checksum recipe; §2 regenerates rules.yaml with edge_types="explicit" to match how the committed copy is built; §3 splits species-axis mismatches into flagged defects vs informational expected asymmetries; §4 resolves declared paths against the bundle dir and compares full relative paths instead of basenames. Audit now reports “No findings — config layers aligned.” and exits 0.

lnk_wsg_resolve() gains an optional conn argument so callers can control the DB connection rather than relying on env-var-driven lnk_db_conn(). When conn = NULL (default), behaviour is unchanged — lnk_db_conn() is used as before. The change matters in environments where PG_*_SHARE env vars (or ~/.Renviron) point at a tunnel that isn’t reachable: data-raw/study_area_wsgs.R now opens a local docker fwapg connection explicitly and passes it through, matching every other driver script’s localhost:5432/fwapg postgres/postgres pattern (and the pre-#207 inline behaviour). Strict mode and province mode remain DB-free; conn is consulted only in closure mode (wsgs non-NULL + expand = TRUE). Latent on the v0.41.0 release; exposed by the v0.41.1 study-area run when the dispatcher’s ~/.Renviron was pinned to the dead db_newgraph tunnel.

data-raw/study_area_run.sh pre-flight bug fix exposed by v0.41.1’s --schema= flag. The pre-flight check for the bcfp reference view was looking in $SCHEMA.streams_vw_bcfp (the override-able persist schema), but the bcfp reference is hard-coded to fresh.streams_vw_bcfp in R/lnk_compare_mapping_code.R:78 — it’s a constant comparison reference, not a per-run output. The coincidence held while $SCHEMA was always "fresh"; the new flag exposed the latent bug. Now the pre-flight always checks fresh.streams_vw_bcfp, independent of $SCHEMA.

data-raw/study_area_run.sh gains a --schema=<persist-schema> flag for side-by-side bundle compares. Without the flag, behaviour is unchanged (config’s YAML pipeline$schema default). With the flag, the driver exports LNK_SCHEMA so all per-WSG R scripts (wsg_run_one.R, wsg_recompute_one.R, study_area_compare.R) override cfg$pipeline$schema at runtime. The propagation works through SSH to cyphers too (each remote shell exports LNK_SCHEMA before its WSG loop). Use case: --config=default --schema=fresh_default lets a default-config run land in fresh_default without clobbering an earlier --config=bcfishpass baseline in fresh. Empty --schema= value errors loudly rather than silently falling through to the YAML default. Live-validated on ADMS (2.2 min, 11 species habitat tables landed in fresh_default, fresh untouched).

New exported function lnk_wsg_resolve() — the bundle-aware “what WSGs should we model?” resolver (#207). Composes the FWA drainage closure (now a fresh primitive: fresh::frs_wsg_drainage(), NewGraphEnvironment/fresh#211 / fresh v0.32.0) with the bundle’s wsg_species_presence filter (link#157). Three call patterns dispatched by (wsgs, expand): province mode (wsgs = NULL → all bundle-species WSGs, sorted alphabetically), closure mode (wsgs = c(...), expand = TRUE → focal + drainage closure, DS-first preserved), strict mode (wsgs = c(...), expand = FALSE → species-filter input verbatim). Validation mirrors lnk_pipeline_species; closure mode opens its own DB conn via lnk_db_conn() with on.exit cleanup; closure + strict modes emit message() listing any species-less WSGs dropped from the result (parity with the previous inline diagnostic). New @family wsg — pre-stages a lnk_wsg_* family for follow-on topology helpers (e.g. cross-host DS-first bucketing).

data-raw/study_area_wsgs.R shrinks 76 → 33 lines — pure CLI shim now, delegating to lnk_wsg_resolve(). Stdout is byte-identical for the regression baseline (PARS,BULK → the exact 15-WSG closure KISP, KLUM, LKEL, LSKE, MSKE, USKE, BULK, FINA, LBTN, LPCE, MORR, PARA, PCEA, UPCE, PARS), so data-raw/study_area_run.sh and downstream consumers are unchanged. fresh dependency pin: Remotes: NewGraphEnvironment/fresh@v0.31.0 → @v0.32.0. 22 tests added (tests/testthat/test-lnk_wsg_resolve.R): arg validation, stub-based province/strict (stub deliberately non-alphabetical so sort() is load-bearing), live-DB closure + province (gated on skip_if_no_db()).

Tunnel-free per-segment mapping_code parity for the 3 FWCP study areas (#175) — 50 drainage-closed WSGs across Peace / Fraser / Skeena, authoritative median match 99.66% / mean 99.11% / 130 of 148 rows ≥99%. Built around a new lnk_access() export (#205) — the portable access builder that’s the missing twin of lnk_mapping_code. Its merge = TRUE mode is the cheap post-consolidate recompute: rebuild only access + mapping_code from persisted streams/habitat/barriers (no streams segmentation or habitat classification re-derived), ~8× faster than the full-pipeline path (FINA 11.9 s wall vs ~90 s, identical bcfp parity). Methodology is now correctness-regardless-of-bucketing — distribute (any bucketing) → consolidate → recompute → compare — with the recompute as the correctness guarantee, bucketing as a speed knob.

New runner data-raw/study_area_run.sh + helpers (study_area_wsgs.R, wsg_run_one.R, wsg_recompute_one.R, study_area_compare.R) drives the whole flow tunnel-free on M1 + N cyphers (no :63333, no M4). Bug classes addressed along the way: persist + consolidate are now host- and species-count-agnostic (#204 — shape-tolerant schema_consolidate + cypher_prep aligned to cfg$species); lnk_mapping_code’s access read no longer goes cartesian against persist when id_segment repeats across WSGs (#203 symptom narrowed — filter by watershed_group_code when present); persist streams + barriers get the wscode_ltree / localcode_ltree GIST/btree indexes frs_network_features traversal needs (matches fresh::utils.R pattern). New RUNBOOK.md §6 gotchas (orphaned frs_network_features backends + statement_timeout / lock_timeout, view-vs-table planner direction, the per-tenant cartesian) and a cross-repo soul code-check.md Postgres + R-client section.

Two recompute-stable divergences remain as taxonomy candidates, not regressions: SETN salmon ~94% (SK-geography class), UNRS BT 61.8% (Kenney reservoir / dam-override). Validated against bcfishpass@v0.7.15-14-ge12c1a5.

Reproduce bcfp’s per-species accessibility so dam-downstream segments emit the dam descriptor (#200). The mapping_code phase previously drove accessible from barriers_<sp>_unified (all barriers, including dams), so every segment below a dam read inaccessible and lost its ;DAM/;MODELLED/;ASSESSED second token — emitting a bare SPAWN/REAR where bcfp emits SPAWN;DAM. It now uses a new per-species barriers_<sp>_access view that reproduces bcfp’s barriers_<sp>: natural barriers only (gradient at the species threshold ∪ falls ∪ subsurface), minus the observation/habitat override, plus all user-definite barriers (override-exempt). Dams stay in barrier_sources and annotate token2 only.

All three access inputs are now persisted province-wide so the cross-WSG downstream walk is correct in every watershed group, not just the run’s own: natural barriers (already), user_barriers_definite (new USER_DEFINITE family in lnk_barriers_unify, ltree-resolved via the FWA join like falls), and the observation/habitat override (new <persist_schema>.barrier_overrides table). Validated against bcfishpass@v0.7.15: PARS BT 98.95%, LFRA BT 97.77% / CO 97.90% per-segment mapping_code match. See RUNBOOK.md §5.

Persist the per-source downstream-barrier flag columns in streams_access so lnk_pipeline_mapping_code’s second token (DAM/MODELLED/ASSESSED/REMEDIATED/NONE) populates from persisted state instead of defaulting to NONE. Three coupled fixes (#196): lnk_persist_init adds the six flag columns to the streams_access DDL; lnk_pipeline_run pre-persists barriers before the mapping_code phase for cross-WSG dam visibility (link#152); lnk_pipeline_persist projects the flag columns in the INSERT (the DDL/INSERT pair must match — the missing projection was the actual NONE-token bug).

Adds RUNBOOK.md — the durable mental model of the barrier → access → mapping_code machinery, including the authoritative bcfp access-set mechanism (read from smnorris/bcfishpass@v0.7.15). Note: the per-species accessibility set still carries dams (it should be natural-only + observation/habitat-overridden, per bcfp); dam-downstream segments therefore still emit a bare habitat token rather than SPAWN;DAM. Characterized in RUNBOOK §5 with a scoped fix (follow-up issue).

Hotfix for wide-table species-set evolution in v0.40.0/v0.40.1’s lnk_pipeline_run(mapping_code = TRUE) path. Closes #194.

v0.40.1 made the mapping_code phase use active_species (per-WSG subset of bundle species) for working schema’s streams_access columns — matching the persist DDL because lnk_persist_init was also passed active_species. But persist is province-wide. When WSG #2 has a different active subset than WSG #1, the persist table is locked to #1’s column set and #2’s INSERT projection fails:

ERROR: column "has_barriers_ch_dnstr" of relation "streams_access" does not exist

Live smoke 2026-05-19: PARS ran first (default config, active = bt/gr/ko/rb) → 4-column persist DDL. BULK next (default config, active = bt/ch/co/pk/sk/st/rb — BULK is salmon-bearing in the Skeena) → 7-column INSERT projection against 4-column table → fail.

Fix: lnk_pipeline_run passes cfg$species (full bundle, 11 species for default config) to lnk_persist_init instead of active_species. Persist DDL is bundle-sized; per-WSG INSERTs in lnk_pipeline_persist continue using active_species for projection so unused species’ columns get NULL. Per-species habitat tables (streams_habitat_<sp>) similarly created for the full bundle — extras stay empty until populated.

Verified live: PARS + BULK now coexist in fresh_default.streams_access / fresh_default.streams_mapping_code with their respective active subsets, NULL for non-active columns.

Migration: existing persist tables created with narrower DDL do NOT auto-grow. Drop <persist_schema>.streams_access, <persist_schema>.streams_mapping_code, and <persist_schema>.streams_habitat_long_vw, then re-run lnk_pipeline_run(mapping_code = TRUE) to recreate them with the bundle-wide DDL.

No regression in bcfishpass bundle: cfg$species = bcfp 8 = active for most bcfp-bundle WSGs → identical INSERT projection.

Hotfix for v0.40.0’s lnk_pipeline_run(mapping_code = TRUE) path on non-bcfp bundles. Closes #192.

v0.40.0’s mapping_code phase hardcoded sp_set <- c("bt","ch","cm","co","pk","sk","st","wct") (the bcfp 8 species) and called lnk_barriers_views without a species arg (uses the same bcfp 8 default). Working schema’s streams_access got bcfp-8-species columns, but persist streams_access was created by lnk_persist_init(species = active_species) with the bundle’s species — for the default config that’s bt/gr/ko/rb. The persist INSERT ... SELECT projects persist’s column list against working → fails with column a.has_barriers_ko_dnstr does not exist.

Effect: lnk_pipeline_run(mapping_code = TRUE) worked only for the bcfishpass bundle (full 8-species). Every other bundle (including the default operator-facing one) errored on persist.

Fix: pipeline_run’s mapping_code phase now uses active_species (bundle-driven) for both lnk_barriers_views(species = ...) and lnk_pipeline_access(barriers_per_sp = ...). Passes species_<role> to lnk_mapping_code filtered against active_species — species in active_species that don’t appear in any bcfp residence category (GR/KO/RB) fall through to species_resident (placeholder; the data-driven categorization lands via #189).

Caught by live smoke 2026-05-19 on PARS with default bundle. Pre-merge unit tests didn’t cover this path; #191 tracks the test catch-up.

Mapping_code tunnel decouple + portable lnk_mapping_code() build + <type>_<role> rename sweep. Closes #187. Major architectural shift in how access semantics flow through the parity diff. BC: parameter and CLI-flag renames (deprecation shims for one release; removal v0.41.0).

  • Persist streams_access + streams_mapping_code + long-form habitat view. lnk_persist_init() now creates two new per-WSG per-species persist tables (streams_access, streams_mapping_code) and one VIEW (streams_habitat_long_vw = UNION ALL across streams_habitat_<sp> tables, presents the per-species split as long-form for any consumer that prefers it). Per-species column generators (.lnk_cols_streams_access_per_sp(), .lnk_cols_streams_mapping_code_per_sp()) are species-driven — pass a different species set, get matching columns. lnk_pipeline_persist() extended with streams_access + streams_mapping_code write blocks, gated by presence of the working-side tables (skip cleanly when the mapping_code path didn’t run).

  • lnk_mapping_code() — new exported portable build entry point. Schema-aware wrapper around lnk_pipeline_mapping_code() (the pure data transform). Takes explicit table_<role> args (table_access, table_habitat, table_streams) — function works against working-schema tables (mid-pipeline) or persist-schema tables (ad-hoc rebuild). Caller can invoke it directly against persist data with the tunnel down to rebuild streams_mapping_code without re-running the full pipeline — the headline use case unblocking QGIS bcfp-shape symbology via data-raw/build_species_views.R --bcfp.

  • lnk_pipeline_run(..., mapping_code = TRUE) — tunnel-free mapping_code phase. New optional phase that runs lnk_barriers_views (over working <schema>.barriers, tunnel-free, link-canonical) + lnk_pipeline_access + lnk_mapping_code between lnk_barriers_unify and lnk_pipeline_persist. Persist phase copies both new tables to <persist_schema>. Methodology shift: ACCESS now uses link’s own per-species barriers (derived from <schema>.barriers’s blocks_species predicate per link#152) instead of bcfp’s barriers tables staged via the tunnel. Pre-#187 the only path that built streams_mapping_code was lnk_compare_wsg, and access there used bcfp-staged barriers — so link’s streams_mapping_code reflected link’s habitat + bcfp’s access. Post-#187 it reflects link’s habitat + link’s access. The parity diff vs bcfishpass.streams_mapping_code becomes more meaningful (surfaces real link-vs-bcfp divergence that was artificially suppressed before). Expect non-trivial parity-number deltas on the next provincial run vs pre-#187 baselines.

  • lnk_compare_wsg() refactored. Build delegated to lnk_pipeline_run(mapping_code = TRUE); only the diff stays in compare. .lnk_compare_wsg_mapping_code_diff() rewritten to read from <persist_schema>.streams_mapping_code instead of working schema. The orphan helpers (.lnk_compare_wsg_mapping_code, .lnk_compare_wsg_stage_reference_barriers) deleted — ~200 lines simpler.

  • BC: parameter rename with_mapping_code → mapping_code in lnk_compare_wsg() and lnk_pipeline_run(). Old name accepted with .Deprecated() warning for one release; removal in v0.41.0.

  • BC: parameter rename <role>_species → species_<role> in lnk_pipeline_mapping_code() (three params: resident_species → species_resident, anadromous_species → species_anadromous, spawn_only_species → species_spawn_only). Matches the documented <type>_<role> convention (col_<role>, table_<role>, exp_<role>, now species_<role>). Old names accepted with deprecation warning until v0.41.0.

  • BC: CLI flag rename --with-mapping-code → --mapping-code in wsgs_run_pipeline.sh, wsgs_dispatch.sh, wsgs_run_m4_offline.sh, trifecta_smoke.sh, wsgs_run_host.R. Old flag accepted with stderr deprecation warning until v0.41.0.

  • lnk_barriers_views() gains barriers_table arg. Default NULL preserves the existing <persist_schema>.barriers source. Pass a working-schema table to build views over a per-WSG working barriers table — used by the new mapping_code phase. Backward-compatible.

  • Follow-up filed: #189 — data-drive species residence categorization (species_resident / species_anadromous / species_spawn_only) from dimensions.csv. Today the defaults are hardcoded to bcfp’s species residence model; #189 moves them to bundle data so custom species (sea-run cutthroat, Dolly Varden, future mixes) work without monkey-patching function defaults.

Fail loud on transient cypher prep failures. Closes #182. Trip-mode hardening before M1 takes over cypher dispatch while the user is in Europe.

  • data-raw/cypher_prep.sh — replace set -e with set -euo pipefail; wrap three | tail -N pipelines with tempfile + exit-check pattern (bash snapshot_bcfp.sh, Rscript pak::local_install, Rscript lnk_persist_init). Before: tail’s exit 0 masked upstream failures, script printed === READY while cypher was half-prepped, umbrella’s downstream marker-grep caught it but the failure was opaque on the cypher itself. After: each failure mode dumps its full log to stderr and exits 1, ssh-back to the umbrella surfaces the non-zero exit, marker-grep continues to work as belt-and-suspenders. Hit twice in 2026-05-15 (Peace Tier 2 retry + post-#185 re-spin; transient bcdata openmaps WFS timeout in both cases). Sibling fix shipped in an internal pull request for the cypher orchestration scripts; this is the link-side complement covering the per-cypher prep script.

Additive multi-host runs + two coupled fixes to schema_consolidate.R. Closes #180 and #185. Validated end-to-end via Peace Tier 2 retry (2026-05-15): 16 Peace WSGs additively dispatched into an existing 13-WSG fresh_default, all 16 land with complete per-species habitat tables, M4 final state = 29 WSGs.

  • Additive Step 0 (BC). wsgs_run_pipeline.sh’s Step 0 (state_clean.sh --schemas=$SCHEMA wipe) now requires --reset-schema to fire. Default is additive — pipeline writes rely on lnk_pipeline_persist’s per-WSG DELETE-WHERE-WSG idempotency to replace cleanly without losing other WSGs in the schema. Enables adding a new WSG set (e.g. Peace 16) to an in-flight schema without rebuilding everything.

  • Bucket-filtered COPY-streaming (BC). schema_consolidate.R replaces pg_dump+scp+pg_restore with per-table ssh <host> 'docker exec psql -c "COPY (SELECT * FROM <t> WHERE wsg IN (bucket)) TO STDOUT"' > /tmp/<f> + local psql -c "COPY <t> FROM STDIN" < /tmp/<f>. Source-side row filter eliminates the over-fetch class where leftover WSGs outside the bucket collided with destination data. bucket= is now REQUIRED per source. DROP SCHEMA on source replaced with bucket-scoped DELETE so out-of-bucket WSGs on source are preserved.

  • Fix: dest_conn default routed to wrong DB. schema_consolidate(..., dest_conn = link::lnk_db_conn()) default routed verification queries to M4’s tunnel :63333/bcfishpass while the COPY shellouts hardcode local :5432/fwapg. wgc_tables returned 0 rows → silent skip of every source. Default now NULL; function constructs its own localhost:5432/fwapg connection internally to match the COPY hardcodes. Caught Peace Tier 2 first attempt — 12 of 16 Peace WSGs lost from consolidate (M1’s 5 recoverable post-fix; 7 from burned cyphers lost).

  • Fix: per-source wgc_tables enumeration (#185). Previously enumerated tables on destination only. When source’s habitat-table set was a strict subset (cyphers’ Peace bucket = BT/GR/RB; M4 destination carried streams_habitat_ch/sk/st residue from prior runs), the loop hit streams_habitat_ch on source → relation does not exist → break → silently dropped _gr / _rb data. Now enumerates wgc_tables on BOTH source AND destination via parallel information_schema queries; iterates the intersection. Per-table failures use next over break — one bad table no longer poisons the rest. Source-side post-COPY cleanup DELETEs only successfully-copied tables (errored tables stay intact for retry). Per-source result gains copied, errored, skipped_source_only, skipped_dest_only.

  • wsgs_run_pipeline.sh: --cy-workspaces=A,B,C passthrough for #178 Tier 1/2 cypher integration tests (was hardcoded for the full job1,job2,job3 set). CY_WS_ARR threaded through Steps 3/4/5/7/9 + trap-EXIT burn. Step 9 SOURCES_R built dynamically per-cypher. Tier 1 (1 cypher) validated live: 13/13 study-area WSGs, 22m wall, exit 0, cy1 burn clean.

Provincial-run autonomy CLI + 8 operational-script renames to noun_verb convention. Closes #172. Builds on v0.37.0’s #168 decouple — with PG-state resume in place, the autonomy surface stays thin and the renames stay mechanical.

  • Single-command autonomous run. wsgs_run_pipeline.sh (was province_run.sh) accepts --wsgs=A,B,C, --config=<name>, --schema=<name>, --no-cyphers, --force, forwards to wsgs_dispatch.sh (was trifecta_provincial.sh) which intersects the WSG subset in its LPT split. M4+M1-only baseline validated end-to-end: 16-WSG default-bundle dispatch lands 16/16 in fresh_default.streams on M4, ~20 min wall, no operator prompts.
  • Step 0 pre-clean. When --schema= is set, umbrella fires state_clean.sh --schemas=<schema> on every host before Step 1. Drops only the target schema (skips the canonical-fresh heuristic + snapshot reload). Eliminates a class of consolidate failures where stale leftover WSGs on a source host collided with destination data during pg_restore.
  • Scoped state_clean.sh (was province_clean.sh). New --schemas=A,B,C mode drops only the listed schemas. Empty --schemas= rejected loud to prevent dynamic-arg silent fall-through to the destructive default mode.
  • Phantom-cy + error-surface fixes in wsgs_dispatch.sh. R’s paste0("cy", integer(0)) returns "cy" length-1 (constant recycling) — would put a non-existent cypher in the host plan under --no-cyphers. Three-branched cy_host_keys. Empty CY_WORKSPACES init via explicit CY_WS_ARR=() (was read -r -a yielding single-empty-element). SPLIT_OUT=$(Rscript ...) wrapped with explicit || block so R-side stop() messages reach the operator (e.g. --wsgs=BOGUS surfaces the R error verbatim instead of silent abort).
  • 8 rename mapping (git mv preserves git log --follow). Names now describe scope honestly — these scripts work for any list of WSGs / any host count / any reference:
Old New
data-raw/province_run.sh data-raw/wsgs_run_pipeline.sh
data-raw/province_clean.sh data-raw/state_clean.sh
data-raw/province_progress.sh data-raw/progress_check.sh
data-raw/trifecta_provincial.sh data-raw/wsgs_dispatch.sh
data-raw/run_provincial_parity.R data-raw/wsgs_run_host.R
data-raw/consolidate_schema.R data-raw/schema_consolidate.R
data-raw/archive_provincial_runs.sh data-raw/runs_archive.sh
data-raw/balance_provincial_buckets.R data-raw/buckets_balance.R

The wsg_* (singular, per-WSG functions from #168) vs wsgs_* (plural, collection-level orchestrators) distinction is now load-bearing in the naming. compare_bcfishpass_wsg.R → wsg_compare.R was renamed in #168.

Filed-but-not-closed follow-ups: cypher integration testing (issue #172 Phase 2 + 3 acceptance — defer until M4+M1 baseline lands repeatably); LPT-fallback empty-bucket edge case when N_WSGs ≤ N_hosts without timing CSVs (pre-existing, not a #172 regression).

Decouple bcfp comparison from the modelling pipeline. Closes #168. The link package’s deliverable — the per-WSG model in <persist_schema>.streams + per-species habitat + barriers — now runs and is observable independently of any comparison framework. Comparison vs bcfishpass (or any future reference) is a diagnostic overlay that reads the persisted state and never gates whether the model itself ran.

  • New exported lnk_pipeline_run(conn, aoi, cfg, loaded, schema, dams, cleanup_working) — modelling-only umbrella over the 7 lnk_pipeline_* phases plus persist_init + barriers_unify + persist. Writes per-WSG segment data to PG. lnk_barriers_unify is promoted from gated-behind-with_mapping_code to always-on so <persist_schema>.barriers is canonical state for any future reader.
  • New exported lnk_compare_rollup(conn, aoi, cfg, reference, conn_ref, species) — reads persisted state + reference DB, returns the long-format rollup tibble. Reference-agnostic via the reference arg ("bcfishpass" today). Species auto-discovered from PG via information_schema probe.
  • lnk_compare_wsg() refactored as a thin wrapper over both new functions. Bundled behavior preserved for with_mapping_code = TRUE (mapping_code decoupling deferred — follow-up). Active-species set is now PG-state-derived (post-persist) rather than cfg$species ∩ wsg_species_presence (pre-persist); equivalent on a fresh single-call run, future-proofs callers against config drift.
  • data-raw/compare_bcfishpass_wsg.R split into data-raw/wsg_pipeline_run.R (modelling) + data-raw/wsg_compare.R (compare). 4 callers updated to the explicit two-call pattern (_targets.R, regress_dams_isolation.R, rule_flexibility_demo.R, run_provincial_parity.R).
  • data-raw/run_provincial_parity.R resume gate uses PG state as canonical: probes <persist_schema>.streams via internal .lnk_wsg_persisted(); RDS files are diagnostic side-artifacts that no longer silently mask an empty pipeline. Four-branch logic (force / fully-cached / compare-only / pipeline+compare). New --force CLI flag bypasses all caching. New helpers .is_error_stub (re-runs WSGs whose previous attempt failed) and .rollup_has_mapping_code (invalidates bare-rollup cache when the mapping_code lens is requested). Closes the 2026-05-14 incident where 4 of 16 WSGs were silently skipped due to stale error-stub RDS files.
  • Phase 7 smoke matrix validates against live DB on DEAD WSG: empty state (57s pipeline+compare) → pipeline-cached (9s compare-only, ~6× speedup) → fully cached (2s skip) → --force (56s re-fire). Confirms the resume gate value and the decoupled boundary.

Filed-but-not-closed follow-ups: lnk_compare_mapping_code as its own family member (promotes the with_mapping_code = TRUE flag to a stand-alone export), lnk_compare_wsg → lnk_compare_run family rename, persist family naming pass, the 8 data-raw/ script renames (stay in #172).

Operational hardening from the 2026-05-13 → 2026-05-14 provincial dispatch session. No R/ API changes — patches landed in data-raw/ operational tooling. Closes #171.

  • data-raw/trifecta_provincial.sh: M1 reverse-forward tunnel (ssh -R 63333:127.0.0.1:63333) — M1 no longer needs its own (passphrase-protected) db_newgraph identity to reach bcfp. M4 idempotent inline-tunnel block. LPT fallback uses host_speeds-weighted alphabetical split when no _per_wsg_times.csv exists (was equal-split, ignored host_speeds). HOST_SPEEDS recalibrated to time-multiplier semantics: m4=1.0,m1=0.79,cy=1.23 (larger=slower=fewer WSGs assigned). Calibrated from per-WSG medians on the 5-host 2026-05-13 dispatch.
  • New data-raw/province_run.sh — top-level 10-step wrapper (pre-flight, snapshot, spin, prep, archive, smoke, dispatch, acceptance, consolidate, burn) with trap-EXIT cypher burn that fires regardless of mid-flight failure. Drafted ready for a --smoke-only regression-test mode in a follow-up.
  • New data-raw/province_clean.sh — idempotent multi-host state wipe (kills R --no-echo + Rscript + run_provincial, drops fresh + working_* + fresh_<bundle>* schemas, reloads fresh.modelled_stream_crossings via snapshot_bcfp.sh --force). <5 min wall.
  • New data-raw/province_progress.sh — mtime-based per-host progress probe. Cross-host TZ-glob hell solved by using find -mmin -N and ls -t (newest by mtime, not filename) — cypher logs use UTC, M4/M1 use local; date-globbing across hosts broke at TZ rollover.
  • research/post_compact_provincial_handoff.md — tunnel architecture gotcha section (how each host reaches bcfp) + LPT fallback gotcha section.
  • planning/active/{task_plan,findings,progress}.md — full PWF capture: 12 distinct gotchas surfaced during the session, including pkill -f Rscript missing the R --no-echo subprocess (caused concurrent dispatches), RDS-cache-skip in run_provincial_parity.R, stale cypher snapshot fresh.* data, M1 SSH key passphrase + Keychain-only unlock, and M4 PG over-tuning. Wrapper test strategy documented.

Follow-up issues filed (not closed here): #167 tunnel autossh, #168 decouple bcfp compare from pipeline, #169 simplify lnk_persist_init after an internal issue, #170 S3-based consolidate. Plus an internal issue (rebuild cypher snapshot with fwa-dump tables ONLY) and fresh#199 (reopened — M4 PG over-tuning evidence).

Run result: 217-WSG BC stream network model in M4 fresh schema. Annotated parity CSV at data-raw/logs/provincial_parity/20260514_0622_*_annotated.csv — 91 UNEXPLAINED rows at |diff_pct| >= 2% (acceptance bar still not met; investigation queue for next session).

Closes #162. Lifts two scattered data-raw/ scripts (linear rollup parity + per-segment mapping_code parity) into one package-level lnk_compare_wsg(), adds an annotated CSV pipeline (lnk_parity_annotate() against a YAML divergence taxonomy), modernizes the multi-host orchestrator to 5-host (M4 + M1 + N cyphers via tofu workspaces), and hardens the spin-up + smoke flow so failures fail loud + fail fast. Full per-phase summary: planning/archive/2026-05-link162-lnk-compare-wsg-annotated-csv/README.md.

  • New exported lnk_compare_wsg(conn, aoi, cfg, loaded, reference, with_mapping_code, ...). Per-WSG convenience wrapper around the existing lnk_pipeline_* phases. Returns list(rollup, mapping_code). reference = "bcfishpass" only initially; with_mapping_code = TRUE adds the per-segment lens additive on top of the same network state (no double-pipeline). Defensive empty-merge handling for the 36 WSGs bcfp doesn’t model (warning + NA-filled tibble, not error). data-raw/compare_bcfishpass_wsg.R collapses from 432 → 77 lines as a thin wrapper; data-raw/compare_bcfp_mapping_code.R deleted.
  • New exported lnk_parity_annotate(rollup, taxonomy, to, tolerance). First-match-wins lookup against research/bcfp_divergence_taxonomy.yml. Tags each rollup row with taxonomy_id, class, mechanism, status, refs columns. Unmatched rows: class = UNEXPLAINED (|diff_pct| >= tolerance) | WITHIN_TOLERANCE | NOT_APPLICABLE. Accepts both ref_value and bcfishpass_value column names. Optional CSV write.
  • New research/bcfp_divergence_taxonomy.yml — single source of truth for known patterns. 11 entries covering Classes A (SETN stale), B (HORS fresh#158 bypass), C (SK new-geographies fresh#190/#191), D (BBAR + small 2026-05-11 residuals), MEASUREMENT_ASYMMETRY (lake/wetland centerline-vs-polygon).
  • data-raw/trifecta_provincial.sh extended for M4 + M1 + N cypher workspaces (--cy-workspaces=job1,job2,job3). Inline greedy LPT bucket allocation (reads prior _per_wsg_times.csv, uses CLI --host-speeds=m4=1.0,m1=0.83,cy=1.83 for projection + back-normalization — no feedback loop). Pre-flight version check across all hosts before dispatch. Post-pull aggregate annotation against the taxonomy. Empty-bucket guard. Cypher-side R log pull-back at run end so cross-repo log boundary doesn’t hide errors. Truth-in-headline reports OK vs error-stub RDS counts (was misleading N/N pulled).
  • data-raw/trifecta_smoke.sh rewritten as a 77-line shim over the production orchestrator — one small WSG per host (m4=DEAD, m1=ELKR, cyN=ADMS/BABL/BULL), passes --fail-fast automatically, asserts every smoke RDS is a successful tibble (not error stub) before declaring pass. Exits non-zero with clear message + pointer to the cypher R log when any WSG fails. “Smoke passed” now means every smoke WSG produced a valid tibble, not just “scripts exited 0”.
  • data-raw/archive_provincial_runs.sh — new helper. Moves prior-run _per_wsg_times.csv + *.rds + *_annotated.csv into archive/<TS>/ so the LPT planner uses the most recent run only.
  • data-raw/balance_provincial_buckets.R — dedup (wsg, host) and cross-host before LPT so multi-run CSV accumulation no longer double-assigns WSGs to buckets. Superseded for the N-host orchestrator (which computes LPT inline) but kept for standalone planning.
  • data-raw/consolidate_schema.R — bucket-aware destination cleanup (DELETE FROM <schema>.<table> WHERE watershed_group_code IN (<bucket>) on each watershed_group_code-bearing table before pg_restore — prevents duplicate-key violations on re-consolidation). Pre/post row-count delta verification: ok = TRUE requires count(*) post-restore > pre-restore (NOT pg_stat_user_tables.n_live_tup which lags asynchronously).
  • lnk_persist_init(force_recreate = FALSE) — new flag + DDL drift detection via .lnk_validate_persist_table(). Errors loud when an existing target table has unexpected GENERATED ALWAYS columns (catches cypher snapshots baked when fresh::frs_col_generate() had been run on fresh.streams). force_recreate = TRUE DROPs+recreates with correct DDL. 6 new tests cover detection, force-recreate, no-op, and arg validation.
  • data-raw/run_provincial_parity.R — --with-mapping-code flag passthrough; new --fail-fast flag (default FALSE preserves soft-fail for full provincial runs; smoke runner injects it automatically so WSG #1 failure on a host stops the loop instead of confirming the same failure 30 more times); post-loop annotation step writes <TS>_<host>_annotated.csv.
  • Updated research/provincial_run_runbook.md for the 5-host flow + smoke-first cadence + DDL drift handling. The runbook is now the operational source of truth; data-raw/README.md#provincial-dispatch is the CLI reference.
  • 2026-05-12 → 13 live provincial run results in research/provincial_parity_2026_05_12.md. Acceptance bar (zero UNEXPLAINED at |diff_pct| >= 2%) NOT YET MET (56 surviving UNEXPLAINED rows; 93 cypher WSGs lost to DDL drift now fixed by lnk_persist_init hardening — next provincial run should hit 217/217 OK and provide the full picture). Operational lessons documented in planning/archive/2026-05-link162-lnk-compare-wsg-annotated-csv/findings.md.

Filed follow-up: #163 — adaptive host_speeds learning from observed wall times (LPT refinement; currently uses static CLI defaults).

  • data-raw/snapshot_bcfp.sh: replace grep -qi parquet with grep -i parquet > /dev/null in the Parquet prereq check (#160). Under set -euo pipefail, grep -q closes the pipe on first match, ogr2ogr gets SIGPIPE (exit 141), pipefail propagates, ! flips it, and the script FATALs even though the Parquet driver IS present. Originally chased as a non-interactive ssh / conda env issue — that was a misdiagnosis; PATH from an internal issue/#123 was always correct.

Closes #152. New unified province-wide <persist_schema>.barriers table with a pre-computed blocks_species text[] predicate. Closes the cross-WSG dam_dnstr_ind defect — PARS BT mapping_code parity jumped from 60.64% → 98.63% (+38 pp) because dam barriers in upstream-of-PARS WSGs (Bennett in PCEA, Peace Canyon / Site C in UPCE) now resolve correctly via FWA-topology walks over the province-wide table. Other Phase A WSGs maintained ≥99% across all species (full 6-WSG matrix in research/bcfp_compare_mapping_code.md).

  • New exported lnk_barriers_unify(conn, aoi, cfg, loaded, schema). Consolidates four per-WSG barrier source families into <schema>.barriers: anthropogenic (PSCIS / CABD / MODELLED_CROSSINGS with barrier_status IN ('BARRIER','POTENTIAL')), gradient (per-class, blocks_species derived from parameters_fresh$access_gradient_max), falls, and opt-in subsurface_flow. Per-source id_barrier namespacing keeps rows unique within a WSG without coordinating sequence IDs (anthropogenic = aggregated_crossings_id; others get <SOURCE>-<rownum> text prefixes).
  • New exported lnk_barriers_views(conn, schema, cfg). Emits per-species (<schema>.barriers_<sp>_unified for the 8 mapping_code species) + per-source (<schema>.barriers_{anthropogenic,pscis,dams}_unified) CREATE OR REPLACE VIEWs over <persist_schema>.barriers. Each view re-exposes id_barrier AS barriers_<x>_unified_id so the existing lnk_pipeline_access feature_id_col = "<table>_id" convention works unchanged. _unified suffix avoids name collisions with the per-WSG tables .lnk_pipeline_prep_minimal + lnk_barriers_emit already build (those stay — they’re useful primitives).
  • lnk_persist_init() extended with cols_barriers DDL: 13 columns, PK on (id_barrier, watershed_group_code), GIN index on blocks_species, btree indexes on (watershed_group_code, barrier_source) and (blue_line_key, downstream_route_measure), GIST on geom.
  • lnk_pipeline_persist() extended with a <schema>.barriers → <persist_schema>.barriers DELETE-WHERE-WSG + INSERT branch (gated on staging-table presence so older orchestrators that don’t yet call lnk_barriers_unify keep working without behaviour change).
  • data-raw/compare_bcfp_mapping_code.R: barrier_sources$anthropogenic + barrier_sources$dams now point at the unified views. barriers_per_sp keeps the bcfp-tunnel staging fallback (the unified-table blocks_species predicate doesn’t encode per-species minimal-position semantics — that’s a separate scope expansion).

Closes #154. lnk_pipeline_crossings() now reproduces bcfp’s PSCIS-to-modelled auto-snap layer byte-identically via the fresh primitive composition (lnk_points_snap(num_features = 5L) + fresh::frs_candidates_pick() + bcfp-shape scoring/dedup SQL). Phase A mapping_code parity hits ≥99% on every in-WSG species across ADMS, BULK, WILL, PARS — BULK jumped ~80% → ~99.5%, WILL ~86% → ~99.7%. PARS BT 60% remains cross-WSG dam_dnstr territory (tracked under #152).

  • New private helper .lnk_pipeline_pscis_build(conn, aoi, schema, loaded, …) mirrors bcfp’s 02_pscis_streams_150m.sql + 04_pscis.sql at smnorris/bcfishpass@v0.7.14-125-g6e9cf1c. Five-step composition: multi-stream snap → enrich + score (name_score, width_order_score) → b-side modelled-collision dedup → per-PSCIS pick via frs_candidates_pick + AOI filter + DBSCAN 5m cluster + UNIQUE(blue_line_key, downstream_route_measure) dedup → xref-driven INSERT (two-branch UNION ALL: modelled_crossing_id lookup vs linear_feature_id lookup, mirroring referenced_modelled_xing + referenced_streams CTEs). lnk_pipeline_crossings() now calls this helper in place of the bare lnk_points_snap(); minimum snap_tolerance clamped to 150 m to match bcfp.
  • lnk_points_snap(): bug fix in the segment-offset downstream_route_measure formula. Previous form ST_LineLocatePoint * ST_Length(s.geom) computed position WITHIN the candidate segment, not the absolute drm on the blue line. Now adds + s.downstream_route_measure and uses s.length_metre with GREATEST/LEAST/FLOOR/CEIL clamping per bcfp’s pattern. New num_features = 1L arg (backwards-compatible) returns up to N candidate streams per input point for downstream scoring workflows.
  • .lnk_crossings_union: modelled branch now LEFT JOINs <schema>.crossing_fixes (staged user_modelled_crossing_fixes) and filters WHERE cf.structure IS NULL OR cf.structure = 'OBS' — bcfp parity with load_crossings.sql:634. Without this filter, 275 NONE-fixed modelled crossings leaked through in BULK / 103 in WILL, breaking per-segment mapping_code parity for non-wct species. PSCIS branch now reads from <schema>.pscis (the canonical output of .lnk_pipeline_pscis_build); modelled-branch xref exclusion sources from the same table.

Closes #148. Wednesday-morning sync chain shifted earlier so a fully-fresh local fwapg lands before workday-start, and data-raw/snapshot_bcfp.sh is now schedulable per host without manual install gymnastics.

  • .github/workflows/sync-bcfishpass-csvs.yml cron: Wed 6 AM PDT (13:00 UTC) → Wed 4 AM PDT (11:00 UTC). Runs 1 h after the upstream dump in an internal issue (which itself shifted to Wed 3 AM PDT).
  • New exported lnk_baseline_current(log, host, path) predicate. Returns TRUE when this host’s most-recent data-raw/logs/bcfp_baselines.csv row already stamps the upstream bcfp_model_version carried in log. Per-host scoped — M4 stamping a SHA must not gate M1 from snapshotting its own fwapg.
  • data-raw/snapshot_bcfp.sh updates: self-anchors to repo root via cd "$(dirname "$0")/.." (so cron-default $HOME cwd doesn’t break the relative ledger path); skip-guard runs FIRST via lnk_baseline_current() before any DB-credential resolution (a host with a stale env file can skip cleanly when this week’s ledger already matches); sources ~/.config/snapshot-bcfp.env for per-host DATABASE_URL / PG* vars; xtrace removed from set -euxo pipefail → set -euo pipefail to keep credentials out of ~/.local/state/snapshot-bcfp/*.log.
  • New data-raw/scheduler/ directory with launchd plist (com.newgraph.snapshot-bcfp.plist for M4 + M1, fires Wed 5 AM local), Linux crontab line (snapshot-bcfp.cron for cypher, 0 12 * * WED UTC), and README.md documenting per-host install + uninstall + env file format.

Post-merge /code-check follow-up on #138 (v0.32.0). Three fragility fixes (no behaviour change for valid inputs) plus a stashed snapshot-script fix:

  • .lnk_crossings_union: cast modelled_crossing_id to bigint before adding 1e9 so values past int4 max can’t overflow. Override path (.lnk_crossings_apply_overrides) already did this; the union branch matches now.
  • .lnk_crossings_union: switch CABD + modelled FWA joins from LEFT JOIN to INNER JOIN. Missing linear_feature_id (FWA refresh drift) previously NULL’d watershed_key and the row got silently dropped much later by barriers_emit’s blue_line_key = watershed_key filter — drop at the union step instead so the count discrepancy is observable upstream.
  • lnk_points_snap: pre-flight check on input columns. pts.* would otherwise produce a CREATE TABLE AS error from a column-name collision deep in a 100-line statement; now errors out with a clear list of colliding columns before any DDL runs.
  • data-raw/snapshot_bcfp.sh: bcdata bc2pg --refresh requires the target table to already exist — drop-then-load instead so first-time snapshots succeed.

Closes #138. New lnk_pipeline_crossings() builds <schema>.crossings + <schema>.barriers_* from public-source primitives (BCDC PSCIS via bcdata bc2pg, CABD via the public API, bchamp modelled_stream_crossings.gpkg.zip) — no tunnel, no bcfishpass.barriers_* reads. Phase B of the self-sufficiency roadmap (#117 csv-sync + #137 snapshot script were Phase A).

Four new exports — three are generic enough that they may relocate to a future pac package once that’s scaffolded:

  • lnk_inputs_verify(conn, required) — fail-loud existence check for <schema>.<table> preconditions. Single round-trip via information_schema.tables.
  • lnk_points_snap(conn, table_in, table_out, ...) — bulk lateral-KNN snap to FWA. Wraps the same CROSS JOIN LATERAL ... ORDER BY <-> ... LIMIT 1 pattern used by bcfp’s load_dams.sql and link’s existing CABD path. One SQL round-trip; scales province-wide. Handles MultiPoint inputs via ST_GeometryN(..., 1).
  • lnk_barriers_emit(conn, schema) — emits <schema>.crossings_lookup (slim id + statuses projection) plus four <schema>.barriers_* tables (anthropogenic, pscis, dams, remediations). Filters mirror bcfp’s model/01_access/sql/barriers_*.sql and remediations_barriers.sql.
  • lnk_pipeline_crossings(conn, aoi, cfg, loaded, schema, snap_tolerance, pscis_table, modelled_table, dams_table) — exported pipeline phase. Composes input verification + PSCIS snap + source-precedence union + override application + barriers emit. Source tables configurable via the *_table args.

Lean column set: only what lnk_barriers_emit() consumes — aggregated_crossings_id, crossing_source, crossing_feature_type, barrier_status, pscis_status, dam_name, network position columns, geom. Drops bcfp’s road tenure / FTEN / OGC / rail / UTM metadata that downstream non-barrier consumers need.

Live ADMS smoke against local fwapg loaded with data-raw/snapshot_bcfp.sh (link#137): 67 PSCIS + 3,584 modelled = 3,651 crossings unioned in <1s; barriers_emit produces 3,616 anthropogenic / 33 PSCIS / 5 remediations.

Tests: 94 new mocked unit-test expectations across the four exports + two internal helpers (.lnk_crossings_union, .lnk_crossings_apply_overrides). 903 PASS / 0 FAIL total.

Closes #137. New data-raw/snapshot_bcfp.sh shell script loads bcfp dependencies into a local Postgres from public sources only — no SSH tunnel, no DB pg_dump. Prepares the local fwapg for lnk_pipeline_crossings() (link#138, in flight) and parity comparisons.

  • BCDC PSCIS via Python bcdata bc2pg --refresh → whse_fish.pscis_* (4 tables).
  • CABD dams via ogr2ogr from CABD’s public GeoJSON API → cabd.dams.
  • bchamp modelled_stream_crossings.gpkg.zip via curl + ogr2ogr → fresh.modelled_stream_crossings.
  • bchamp observations.parquet via ogr2ogr /vsicurl/... → bcfishobs.observations (same artifact bcfp’s jobs/load_observations consumes).
  • Optional --with-bcfp-views: pulls Simon’s bcfp output views (crossings_vw, streams_vw) from s3://newgraph/ for parity comparison.
  • Stamps data-raw/logs/bcfp_baselines.csv with the bcfp build identifier from s3://fresh-bc/bcfishpass/log.json via lnk_baseline_append().

Documented in data-raw/README.md under a new ## Bootstrap section.

Closes #117. csv-sync flips from GitHub-API SHA-walking to reading from s3://fresh-bc/bcfishpass/ (populated weekly by NewGraphEnvironment/db_newgraph). Cadence drops from daily to weekly Wed afternoon. Eliminates the 1–7 day drift between bundle CSVs and the upstream tunnel rebuild SHA.

Four new exports support csv-sync + downstream parity drivers + future multi-build comparison (grayling / rainbow / ko / etc.):

  • lnk_bucket_get(name, prefix, to) — fetch any artifact from a versioned S3 prefix. Returns raw bytes by default (caller decodes — read.csv(), jsonlite::fromJSON(), arrow::read_parquet()); writes to disk when to is supplied. Default prefix is NGE’s bcfp dump. Format-agnostic.
  • lnk_bucket_log(prefix) — sugar for the most common read: parses <prefix>/log.json into a list with model_version, date_completed, head_sha. Validates required keys.
  • lnk_baseline_read(path) — read the run-tracking ledger (data-raw/logs/bcfp_baselines.csv by default) as a tibble. Validates cols_baseline shape on read.
  • lnk_baseline_append(log, run_label, ...) — append a stamped row from a lnk_bucket_log() result. Used by csv-sync to record which build each sync ran against; reusable by parity-run drivers.

data-raw/sync_bcfishpass_csvs.R rewritten to consume the new exports; integrates a crate::crt_schema_validate() gate for provenance entries with canonical_schema: declared (escalates drift_kind to "shape" on validation failure).

httr + jsonlite added to Imports.

Closes #135. lnk_pipeline_access() now computes dam_dnstr_ind and (optionally) remediated_dnstr_ind from the same primitives that drive the per-species access codes, eliminating the bcfp-merge-in step needed for full BT/WCT parity in 0.30.0. Both lnk_pipeline_access() and lnk_pipeline_mapping_code() consume the new lnk_presence() helper (v0.30.1) to short-circuit absent species cleanly.

  • dam_dnstr_ind is sequence-aware: TRUE iff the next-downstream anthropogenic barrier is also a dam. Mirrors bcfp’s array[barriers_anthropogenic_dnstr[1]] && barriers_dams_dnstr overlap check. Both barriers_anthropogenic and barriers_dams populate their primary key from crossings.aggregated_crossings_id, so the IDs returned by frs_network_features are in a shared space and %in% works directly. ADMS parity vs bcfishpass.streams_access.dam_dnstr_ind: 11803 FALSE / 3960 TRUE, zero off-diagonal differences.
  • lnk_pipeline_access() gains an optional crossings_table = NULL arg. When supplied alongside barrier_sources$remediations, computes remediated_dnstr_ind per the bcfp-intended logic — TRUE iff the next-downstream remediation is a crossing where pscis_status = 'REMEDIATED' AND barrier_status = 'PASSABLE'.
  • bcfp’s own streams_access.remediated_dnstr_ind is FALSE for every row in the build due to a 2-year-old contradictory clause in load_streams_access.sql (introduced by smnorris/bcfishpass#339 and inlined in v070 by smnorris/bcfishpass#690). link computes the bcfp-intended dual-column semantics so mapping_code_<bt|wct> may emit REMEDIATED tokens where bcfp’s current output emits DAM / MODELLED / ASSESSED. Upstream fix filed as smnorris/bcfishpass#891 + smnorris/bcfishpass#892.
  • lnk_pipeline_access() and lnk_pipeline_mapping_code() accept an optional presence arg (an lnk_presence object). When supplied, absent species short-circuit cleanly: lnk_pipeline_access skips the frs_network_features query and emits access_<sp> = -9; lnk_pipeline_mapping_code emits "". Eliminates the salmon-group-absent over-emission caught in the multi-WSG sweep on ELKR + HORS.
  • ADMS validation, no bcfp merge-in: mapping_code_bt 15733/15763 (30 REMEDIATED divergences, all the bcfp v070 regression), mapping_code_ch/cm/co/pk/sk 15761/15763 (2 each), mapping_code_st/wct 15763/15763. Stamped logs under data-raw/logs/<TS>_link135_parity_*.txt.

Closes #124. Reproduces bcfishpass’s three classification surfaces (crossings.barrier_status, streams_access, streams_mapping_code) as additive layers — link’s existing severity and 5-bucket mapping_code are unchanged.

  • lnk_pipeline_access(conn, segments, aoi, ...) — composes fresh::frs_network_features() (fresh 0.29.0+) calls across species + observations into a streams_access-shape wide tibble. Per-segment per-species access_<sp> integer codes (-9 / 0 / 1 / 2) for absent / blocked / modelled / observed. Caches per-table dnstr queries — 5 species pointing at one grouped barriers table run the SQL once. Auto-NA propagation when a barriers source has zero rows in the AOI mirrors bcfp’s barriers_<sp>_dnstr IS NULL semantics for absent species.
  • lnk_pipeline_mapping_code(access, habitat, feature_code, ...) — pure R derivation over the bcfp-shape access columns. Resident-flavor (BT, WCT) vs anadromous-flavor (CH/CM/CO/PK/SK/ST) handling for mapping_code_barrier. Spawn-only species (CM, PK) emit only ACCESS / SPAWN token1 (no REAR per bcfp). feature_code = "GA24850150" flags INTERMITTENT. Optional to= arg writes <schema>.streams_mapping_code for downstream views.
  • ADMS parity validation: 15762 / 15762 byte-identical to bcfishpass.streams_mapping_code for all 8 species (BT, CH, CM, CO, PK, SK, ST, WCT). Per-species access_<sp> ≥99% match (1-row totals diff + ~13-row obs/modelled drift attributable to bcfp’s life_stage / activity / point_type observation filters not yet applied in link).
  • barrier_status (Phase 1) — already populated correctly by lnk_pipeline_load via .lnk_pipeline_apply_fixes + .lnk_pipeline_apply_pscis. Roxygen note added distinguishing barrier_status (bcfp-parity, PSCIS-field + CSV override) from severity (link’s culvert-geometry scoring). Both can coexist on the same crossings row.
  • build_species_views.R --bcfp sibling view per species — streams_<sp>_bcfp_vw carries the bcfp-shape mapping_code_<sp> string for QGIS A/B comparison against the existing streams_<sp>_vw (link’s 5-bucket categories). Both views co-exist; symbology hint covers each.
  • scripts/update_hosts.sh — pak-bug-bypass updater for trifecta hosts. Uses R CMD INSTALL from a GitHub source tarball, sidesteps r-lib/pak#658 which mis-reports cypher’s permission-denied installs as “empty archive” when the user’s first .libPaths() entry isn’t writable.
  • data-raw/trifecta_provincial.sh — --rds-dir= pass-through arg for recovery runs that need to bypass the resume RDS cache (e.g. running cypher’s bucket on M4 after cypher destroy).
  • Caveat for full BT/WCT parity: mapping_code_<bt|wct> uses bcfp’s pre-computed dam_dnstr_ind / remediated_dnstr_ind via merge-in. Computing those from link primitives requires sequence-aware “next downstream barrier IS a dam” logic — tracked as a follow-up issue. Anadromous species + non-resident BT/WCT in non-overlap WSGs are byte-identical without the merge.

Closes #121. Auto-stamps the bcfp comparison reference (model_run_id + version SHA + completion timestamp) into data-raw/logs/bcfp_baselines.csv from inside data-raw/run_provincial_parity.R. Tuesday weekly bcfishpass.* rebuilds shift the comparison reference; un-stamped runs were ambiguous after the fact. Orchestration tooling only — no public R API changes.

  • New inline stamp_bcfp_baseline() helper in data-raw/run_provincial_parity.R, called once per invocation between the per-WSG-timings setup and the WSG loop. Same wiring covers single-host and trifecta-dispatched per-host runs.
  • data-raw/logs/bcfp_baselines.csv gains a host column between run_started_pdt and run_label. Three existing rows backfilled to host=m4 (single-host M4 runs). Trifecta runs now produce three rows per run, one per host, all with the same bcfp_model_run_id.
  • Host alias resolves via LNK_HOST_ALIAS env var (e.g. LNK_HOST_ALIAS=m4 in ~/.Renviron); falls back to Sys.info()[["nodename"]] when unset.
  • Tunnel-tolerant: connection failure or unset PG_PASS_SHARE logs a warning and the build proceeds (per-WSG comparisons further down would fail too if the tunnel were genuinely broken, so the stamp is not the actual blocker). Idempotent on (host, link_schema, bcfp_model_run_id, run_started_pdt) — same-minute re-runs (resume scenarios) skip silently rather than duplicate.

Closes #118. DB hygiene to prevent the disk-full incident that crashed cypher’s fresh-db container during the 2026-05-04 default_extrabreaks provincial trifecta. Two-tier orchestrator-level cleanup; no in-package API changes.

  • compare_bcfishpass_wsg() gains cleanup_working = TRUE parameter — drops working_<aoi> schema after the rollup tibble is built. Default-on; pass FALSE for interactive debug. Saves ~10–15 GB per provincial run on every host (60+ working schemas accumulated otherwise).
  • consolidate_schema() gains keep_source = FALSE parameter — drops source schema on each remote host after a successful pg_restore. Default-on; rc-guarded (failed restore leaves source for retry); warn-but-don’t-fail on drop rc != 0. Saves ~25–30 GB per consolidated bundle on M1 + cypher.
  • data-raw/README.md documents per-worker disk capacity: rough footprints (~30 GB single-bundle persistent + 10–15 GB per-WSG scratch + 30–40 GB fwapg base), 60 GB minimum free recommendation, 2026-05-04 cypher incident as cautionary tale.
  • Bit-identical bcfp parity by default. ADMS rollup tibble post-cleanup identical() to pre-cleanup baseline (RDS file metadata differs but deserialized object is identical).
  • Approach: orchestrator-level cleanup, NOT in-package — lnk_pipeline_persist stays scoped to one job; the rollup query reads working schema in long-form AFTER persist returns, so the natural lifecycle owner is the orchestrator script.

Orphan-class break source — fed-vector experiments now Just Work without a separate knob. When cfg$pipeline$gradient_classes (or the caller’s classes arg) contains thresholds below every modelled species’s access_gradient_max, those positions enter gradient_barriers_minimal as a barriers_orphan table — no per-species filter, no minimal reduction (every detected position splits the network for segmentation precision). Access semantics are unaffected: fresh’s per-species access label filter at classify time rejects any gradient_NNNN label below the species’s threshold, so orphan classes never block any species.

  • New experimental bundle inst/extdata/configs/default_extrabreaks/ extends default with pipeline.gradient_classes set to the union of access (0.15/0.20/0.25) + per-species spawn / rear gradient maxima from fresh’s parameters_habitat_thresholds.csv (0.0249–0.1049). Persists to fresh_default_extrabreaks schema for side-by-side compare against the fresh_default reference.
  • ADMS smoke test on the bundle: BT spawning +11.2 km (+3.1 %) vs default-bundle baseline; SK spawning +13.9 km (+6.4 %); RB spawning +8 km (+2.6 %). Rear shifts much smaller (±5 km). Effect is the “ceiling sub-segment” mechanism: when a generally-flat reach is broken at a low spawn/rear gradient threshold, the steep pocket separates and the remaining majority averages to a lower local gradient that newly passes the per-segment spawn predicate.
  • Provincial run: ./trifecta_provincial.sh --config=default_extrabreaks --schema=fresh_default_extrabreaks (~2.5h wall, same shape as the v0.26.0 default trifecta).
  • Bit-identical to v0.27.0 on bcfp + default config (no orphans — both default vectors live at-or-above each species’s access threshold). Suite: 735 PASS / 0 FAIL.

Closes #45. Two coupled hardcodes in R/lnk_pipeline_prepare.R — the bcfishpass gradient class break vector and the per-model class filter list — are now configurable. Unblocks alternative-methodology experiments that need different break thresholds (e.g. breaking the network at the union of unique per-species rearing/spawning/access gradient values, or finer 0.05-step bins) while preserving bit-identical bcfishpass parity by default.

  • lnk_pipeline_prepare() gains a classes argument — a named numeric vector of gradient class break thresholds. When NULL, falls back to cfg$pipeline$gradient_classes if set in the bundle, otherwise to the bcfishpass default c("1500" = 0.15, "2000" = 0.20, "2500" = 0.25, "3000" = 0.30). Optional pipeline.gradient_classes knob documented (commented-out) in bcfishpass/config.yaml and default/config.yaml.
  • .lnk_pipeline_prep_minimal() replaces the hardcoded per-model models list with per-species derivation: for each species in cfg$species (with loaded$parameters_fresh$species_code fallback), classes whose value is >= access_gradient_max form that species’s barrier filter. Per-species barrier tables become barriers_<sp> (lowercase species code, validated). Species with NA / zero / missing access_gradient_max are skipped.
  • Bit-identical bcfp parity verified on ADMS/HARR/BABL/BULK (same digests as pre-#45 baseline). Override mechanism end-to-end demonstration: dropping the 0.25 break on ADMS expands BT habitat ~30% ( loses its barrier filter when no class >= 0.25 exists), CH/CO/SK unchanged.
  • Empty species set (no presence-flagged species + no override) yields a structurally valid empty gradient_barriers_minimal table so downstream phases find the expected name. Defensive sp_amax[1L] handles the (unlikely) case of duplicate species_code rows in parameters_fresh.csv — would otherwise trip R 4.3+ length-1 enforcement on ||.
  • 5 new + 2 updated mocked tests (prep_gradient classes threading; prep_minimal per-species derivation, skip path, custom-vector path; .lnk_resolve_classes precedence; YAML→R round-trip through lnk_config()). Full suite: 728 PASS / 0 FAIL.

Closes #112. Per-WSG output now persists into province-wide <schema>.streams + <schema>.streams_habitat_<sp> tables, mirroring bcfp’s bcfishpass.streams + bcfishpass.habitat_linear_<sp> pattern. Queryable across WSGs for cartography, intrinsic-potential maps, per-crossing rollups, and methodology comparisons — no more re-running 232 WSGs to look at one.

  • New lnk_persist_init(conn, cfg, species) — idempotent CREATE SCHEMA IF NOT EXISTS + CREATE TABLE IF NOT EXISTS for the persistent tables. DDL driven by cols_streams (21 columns mirroring bcfp.streams + link’s id_segment) and cols_habitat (7 columns: id_segment + watershed_group_code + 5 booleans accessible/spawning/rearing/lake_rearing/wetland_rearing). geom geometry(MultiLineStringZM, 3005) — FWA streams are XYZM (X, Y, elevation, measure).
  • New lnk_pipeline_persist(conn, aoi, cfg, species, schema) — DELETE-WHERE-WSG + INSERT for streams + per-species streams_habitat_. Long→wide pivot: per-species INSERT filters working_<aoi>.streams_habitat WHERE species_code = '<sp>' and projects cols_habitat (drops species_code from SELECT). Idempotent re-runs replace cleanly.
  • Pipeline rewire: per-WSG segment-level data (streams, streams_habitat, streams_breaks) now lives in working_<aoi> (the per-WSG schema where every other staging table already lived) instead of the previously-shared fresh schema. ~12 hardcoded literals updated across lnk_pipeline_prepare/break/classify/connect + compare_bcfishpass_wsg.R.
  • New pipeline.schema config knob (REQUIRED, default fresh) — enables side-by-side bundle compare (schema: fresh_bcfp vs schema: fresh_default), within-host parallelism (schema: fresh_w1/fresh_w2), branch isolation, centralized vs distributed write target.
  • compare_bcfishpass_wsg.R orchestrator now calls lnk_persist_init + lnk_pipeline_persist after lnk_pipeline_connect.
  • Trifecta provincial run end-to-end (M4 + M1 + cypher, ~2h wall, pg_dump consolidation onto M4): 217 WSGs / 5.3M segments persistently in fresh.streams. 5/5 test WSG rollups byte-identical to pre-#112 baseline (LRDO/SETN/ADMS/BULK/HARR on SK spawn+rear+lake).
  • New tests: test-lnk_persist_init.R (28), test-lnk_pipeline_persist.R (4). Updated 3 stale literal-string assertions in test-lnk_pipeline_prepare.R + test-lnk_pipeline_classify.R. Full suite: 710 PASS / 0 FAIL.
  • Removed data-raw/run_nge.R — superseded by compare_bcfishpass_wsg(wsg, lnk_config("default")).

Pre-trifecta config homework — catches staleness in the config layer before the 3-host distributed run, so we’re not chasing ghosts later.

  • Both bundles’ rules.yaml regenerated via lnk_rules_build() (date-stamp diff only — semantically identical to what was committed).
  • provenance: checksums recomputed in both config.yaml for the 4 files modified across v0.21–v0.25 (rules.yaml, dimensions.csv, parameters_fresh.csv, overrides/wsg_species_presence.csv). lnk_config_verify now reports drifted = 0 / 12 for both bundles.
  • Closes #108 — compare_bcfishpass_wsg returns bcfishpass_value = NA (not 0) when bcfp doesn’t model a species. Distinguishes “real measured zero” from “not modelled by bcfp”; diff_pct cleanly resolves to NA. PARS run proves GR / KO / RB classify end-to-end on the default bundle (KO 377 ha lake-rearing, RB 1,839 ha lake + 7,796 ha wetland, GR 1,566 ha lake).
  • compare_bcfishpass_wsg adds a species filter parameter — pass c("BT","CH",…) to drop GR/KO/RB from the rollup entirely.
  • 4 stale tests in test-lnk_rules_build.R updated for the stream_order → stream_order_min/stream_order_max rename (fresh#198) and the per-species in_waterbody semantics. Full suite: 668 PASS / 0 FAIL.
  • New data-raw/audit_configs.R reports drift across all layers — re-runnable before any trifecta or provincial run.

Closes #106. Drops the hardcoded species-presence column list in lnk_pipeline_species + lnk_pipeline_break — both now derive the column list from the wsg_species_presence.csv header via the new .lnk_wsg_species_present() helper. Adding a new species column propagates to every callsite without a code edit.

  • Adds ko (Kokanee) column to both bundles’ wsg_species_presence.csv with sentinel t for PARS, KOTL, NATR, CARP — interim until upstream bcfishpass.wsg_species_presence ships authoritative coverage (NewGraphEnvironment/bcfishpass#12).
  • Adds GR + KO species rows to default/parameters_fresh.csv (already in default/dimensions.csv and rules.yaml).
  • New tests assert column-propagation for newly-added species and notes-column ignoring.

Closes #103. Ingests CABD dams as a parallel reporting dimension. .lnk_pipeline_prep_dams() replicates bcfishpass’s model/01_access/sql/load_dams.sql against the cabd.dams source over the db_newgraph tunnel and writes <schema>.dams mirroring bcfishpass.dams column-for-column. Both bcfishpass and default bundles ingest — the data is methodology-agnostic at the data layer.

  • New optional conn_tunnel = NULL arg to lnk_pipeline_prepare(). When NULL, prep_dams short-circuits to DROP TABLE IF EXISTS <schema>.dams — zero-cost opt-out for CI / non-reporting workflows.
  • Four CABD edit CSVs (cabd_exclusions, cabd_blkey_xref, cabd_passability_status_updates, cabd_additions) ship in both bundles’ overrides/ and are loaded through lnk_load_overrides() like any other override.
  • Habitat output is unchanged. <schema>.dams and <schema>.cabd_* are not consumed by any break / classify / connect phase. HARR dams-ON / dams-OFF rollup is byte-identical to fp precision; confirms the parallel-data invariant.
  • LFRA verification: 65 dams / 59 barriers / 15 named, with Stave Falls (26 m), Alouette (22.5 m), Ruskin (59.4 m), Coquitlam (30.5 m), Northwest Stave + Upper Stave variants all present at the same (blue_line_key, downstream_route_measure) as bcfishpass.dams within fp precision.
  • Per-species methodology — “should some dam classes block which species in the default bundle?” — is intentionally out of scope; tracked at #83.

Closes #96. falls added as a segmentation break source — the FWA stream network is now broken at every fall position. Previously the <schema>.falls table was loaded and used for access gating + obs/habitat lift but not for segmentation, so close-paired falls (no other break source between them) produced segments that spanned the second fall and incorrectly classified its upper portion as accessible.

  • New entry in R/lnk_pipeline_break.R’s source_tables and the default break_order. Both bundle configs (bcfishpass, default) opt in via pipeline.break_order.
  • Falls are NOT minimal-reduced — each fall is its own barrier (unlike gradient barriers which go through frs_barriers_minimal).
  • Closes the implementation drift from the docstring at R/lnk_pipeline_break.R:10-13 which already documented the bcfp order as observations → gradient_minimal → falls → barriers_definite → habitat_endpoints → crossings.
  • 4-WSG regression vs pre-fix baseline (HARR/HORS/LFRA/BABL): all four show small expected reductions (BT ~0.6–1.5 km on HARR/HORS; 7 species × ~0.43 km each on LFRA; 4 species × 0.94–1.59 km each on BABL). All deltas negative — segments above falls correctly become inaccessible. See research/bcfishpass_comparison.md § “falls in break_order (#96)”.
  • HORS BLK 356357296 evidence case: pre-fix segment 12671 (1447 m straddling the fall at DRM 67565) split into 12677 (17 m below) + 12678 (1429 m above, accessible=FALSE).
  • Map cache helper data-raw/maps/_lnk_map_compare.R hardened — stale 0-row caches (left when the pipeline runs for one WSG and the map is rendered for another) now refetch instead of erroring on missing CRS.

Wires fresh::frs_order_child into the pipeline as link methodology — small streams plugging directly into large rivers can be credited as rearing despite low/missing FWA channel-width estimates. Closes fresh#158 on the link side.

  • Four new per-species columns in dimensions.csv (both bundles), all opt-in via rear_stream_order_bypass: yes/no:
    • rear_stream_order_parent_min — min order at the trib BLK’s mouth confluence (default 5, matches bcfp)
    • rear_stream_order_child_min — lower bound on segment’s own stream_order (default 1)
    • rear_stream_order_child_max — upper bound on segment’s own stream_order (default 1)
    • rear_stream_order_distance_max — cap (m) on distance from trib mouth (empty = no cap)
  • lnk_rules_build emits the values into a channel_width_min_bypass: block on the rear stream-edge rule. lnk_pipeline_classify reads the block and calls frs_order_child per species post-classification, gated on rear_stream_order_bypass.
  • Both bundles ship bypass: no for all species — infrastructure is parametric and tested but disabled by default. Re-enable per species via dimensions.csv. The 4-WSG regression (HARR / HORS / LFRA / BABL) is byte-identical to the pre-#96 baseline with bypass=off, confirming the wiring is purely additive when disabled.
  • Updates inst/extdata/configs/dimensions_columns.csv xref doc with all four new columns and refreshes the rear_stream_order_bypass entry (was stale — said “currently inert”).
  • Bumps fresh dep to >= 0.27.5 for the renamed bypass YAML schema (stream_order → stream_order_min + stream_order_max).

Related: link#23 (CH spawning misread, closed not-a-bug). PWF for the wire-up at planning/active/.

Closes #87. Default-bundle SK upstream-spawn now credits any spawn-eligible segment upstream of and accessible from a qualifying rearing waterbody, dropping bcfishpass’s restrictive cluster + lake-adjacency gate. bcfishpass-bundle SK keeps the gate (parity preserved).

  • New spawn_connected_lake_adjacent column on both dimensions.csv schemas. SK row: yes (bcfishpass) / no (default). Empty for non-SK species — inherits fresh’s TRUE default.
  • lnk_rules_build emits <sp>.spawn_connected.lake_adjacent when the dimension is non-empty. Older rules.yaml files without the key remain valid.
  • Bumps fresh dep to >= 0.26.0 (knob lives there).

Closes #92. Per-AOI observations filter mirrors bcfp’s wsg_species_presence + observation_key exclusions.

  • New .lnk_pipeline_prep_observations() builds <schema>.observations per AOI, mirroring bcfp’s model/01_access/sql/load_observations.sql. Filters bcfishobs.observations by the WSG’s species set (only species marked present count) and applies QA exclusions (data_error / release_exclude rows removed, keyed on observation_key — was fish_observation_point_id, never present in the CSV; the empty intersect silently dropped all 1,182 exclusions).
  • Downstream consumers updated: prep_overrides reads <schema>.observations (no longer takes observations param); lnk_pipeline_break_obs simplified to a thin reader; lnk_barrier_overrides uses observation_key.
  • TWAC pre-flight: BT spawning/rearing/rearing_stream collapsed from +21–30% over-credit to 0.0% across the board. 15-WSG tar_make: HARR + LFRA BT tightened toward parity (LFRA BT rearing_stream -3.75% → -0.93%; HARR BT rearing_stream -4.19% → -1.29%); other 13 WSGs unchanged. HORS BT stays -7.68% (fresh#158 stream-order bypass — distinct mechanism).
  • Default bundle also tightens (6 rows on HARR/LFRA BT) — methodology correctness improvement, not a regression.

Closes #88. Subsurfaceflow folded into the natural-barrier set so per-species observation/habitat upstream lift fires on it.

  • .lnk_pipeline_prep_natural() now builds the full bcfishpass natural-barrier union (gradient + falls + opt-in subsurfaceflow). Subsurfaceflow positions land in <schema>.natural_barriers, which lnk_barrier_overrides() consumes — so per-species observation/habitat upstream lift applies to subsurfaceflow exactly as it does to falls and gradient.
  • .lnk_pipeline_prep_subsurfaceflow() deleted; its body absorbed into prep_natural. Six prep helpers → five.
  • Default-bundle off-switch unchanged: omit subsurfaceflow from cfg$pipeline$break_order and the entire code path skips. Verified bit-identical default rollup (0 of 581 rows changed).
  • bcfishpass-bundle parity: HARR CH/CO/ST rearing_stream gaps closed from -14.8/-13.3/-11.6% to within ±0.32%. LFRA CH/CO/ST closed to within ±0.6%. HARR blkey 356286055 BT credits 6.509 km (was 0).
  • Reproducibility: two consecutive 15-WSG tar_make runs produced byte-identical rollup (digest::digest(link_value) matches across runs).
  • HORS rearing_stream gap (~7% on BT/CH/CO) is unchanged by this fix — separate mechanism, follow-up.

Closes #82. Subsurface-flow access barriers + parity claim retraction.

Subsurface-flow as opt-in access barrier. Closes the largest single gap surfaced when expanding the bcfishpass-config rollup from 5 to 10 watershed groups: NATR BT spawning +15.2% → +1.5%, NATR BT rearing +13.0% → -0.6% (10-WSG tar_make log: data-raw/logs/20260429_02_tar_make_subsurf.txt).

  • New .lnk_pipeline_prep_subsurfaceflow() materialises <schema>.barriers_subsurfaceflow from whse_basemapping.fwa_stream_networks_sp filtered to edge_type IN (1410, 1425). Honours user_barriers_definite_control. Mirrors bcfishpass model/01_access/sql/barriers_subsurfaceflow.sql exactly.
  • New subsurfaceflow entry in lnk_pipeline_break.R source_tables map; conditional UNION ALL in lnk_pipeline_classify_build_breaks so the new break source emits 'blocked' into fresh.streams_breaks when the config opts in.
  • Inclusion is gated on cfg$pipeline$break_order containing 'subsurfaceflow' at every site (prepare, break, classify). Configs control the toggle, not code.
  • inst/extdata/configs/bcfishpass/config.yaml opts in (parity with bcfishpass). inst/extdata/configs/default/config.yaml does not opt in (NewGraph methodology decision pending).
  • ?lnk_pipeline_break gains a ## Break sources table covering every valid break_order entry — source table, role, classify-phase label. Both bundled config.yaml files carry an inline comment listing the available entries with one-line semantics so future-readers see the toggle without leaving the config file.

Parity claim retraction. Earlier framing (“all species within 5%”, “exact reproduction”) held only on a small set of pre-selected WSGs. The 10-WSG rollup surfaced systematic gaps. Vignette pulled, README and DESCRIPTION reframed as experimental.

  • vignettes/habitat-bcfishpass.Rmd removed; bundled vignette data in inst/extdata/vignette-data/ removed.
  • README.md rewritten as one-liner (“Experimental package — breaking all the time and loving the learning curve”) plus install + license.
  • DESCRIPTION Title and Description reframed; bookdown, knitr, mapgl, rmarkdown dropped from Suggests; VignetteBuilder removed.
  • data-raw/_targets.R extended to 10 WSGs (PARS, MORR, KISP, KOTL, NATR added).
  • research/bcfishpass_comparison.md retraction at top with the diagnosis tables and the natural-vs-anthropogenic two-tier classification reference; historical content preserved below.
  • CLAUDE.md Status block flags remaining gaps.

Remaining departures (per research/bcfishpass_comparison.md): 7 of 210 spawning/rearing/rearing_stream rows >5%, six of seven link < bcfishpass. Concentrated on MORR ST (cluster connectivity), MORR SK and KISP SK (new geographies for the existing fresh#147 SK lake-proximity logic). Tracked separately; not in this release.

Closes #78. Adds attribution for redistributed upstream data and refreshes the package Title + Description to reflect the package’s current scope.

  • LICENSE-bcfishpass at root — verbatim copy of upstream smnorris/bcfishpass LICENSE governing the redistributed override CSVs
  • NOTICE.md at root — source/license table, names redistributed files
  • inst/extdata/configs/{bcfishpass,default}/overrides/README.md — pointer files reachable via system.file()
  • README.md “Acknowledgements” section above License
  • Authors@R — Simon Norris added as [ctb]
  • Title — Habitat and Connectivity Interpretation for Stream Networks (was the v0.6-era Crossing Connectivity Interpretation)
  • Description — refactored to mirror the README’s “fresh answers what the habitat is, link answers what the features mean for the network” framing; names the three habitat axes (intrinsic potential, accessibility under connectivity, per-feature rollups)

CITATION file and mirror to NewGraphEnvironment/crate (which also ships bcfishpass fixtures via crt_ingest examples) deferred — to be filed as their own work.

Closes #65. Decompose the config bundle into a manifest layer and a data-ingest layer, and route registered files through crate for source-agnostic canonicalization.

lnk_config() is now manifest-only. It reads config.yaml and returns paths, file declarations, pipeline knobs, and provenance — no parsed CSVs. Cheap to call. lnk_config_verify() and lnk_stamp() no longer pay for CSV parsing they don’t need.

New: lnk_load_overrides(cfg) materializes the data files declared in cfg$files and returns a named list of canonical-shape tibbles. Entries with source + canonical_schema declarations dispatch through crate::crt_ingest() (currently bcfp/user_habitat_classification); others fall through to local reads dispatched on path extension. New source families plug in by config edit alone — no link R code change.

New config.yaml schema. Top-level rules: and dimensions: paths replace files.rules_yaml / files.dimensions_csv (format follows from the path’s extension, not the key name). The previous files: and overrides: maps merge into one flat files: map keyed by filename stem (e.g. user_barriers_definite, pscis_modelledcrossings_streams_xref). Each entry carries path: and optionally source: and canonical_schema:. Configs may declare extends: to inherit from another config; child entries override same-key parent entries.

Pipeline phase signatures gain loaded. Every lnk_pipeline_* phase that reads a data table now takes cfg and loaded together. Callers (the bundled targets pipeline, project scripts) call lnk_load_overrides(cfg) once and thread the result through phases. cfg$overrides$X and cfg$habitat_classification access points become loaded$X. See data-raw/_targets.R and data-raw/compare_bcfishpass_wsg.R for the pattern.

Verification. tar_make() on 5 WSGs × 2 configs reproduces the v0.17.0 baseline rollup bit-identically (sha256 a82de9928809b9751213e08916c476b4ee3f99286bc9ea2dc53f9659eeb92097). Refactor introduces no behaviour change.

Migration

Old New
cfg$rules_yaml cfg$rules
cfg$dimensions_csv cfg$dimensions
cfg$parameters_fresh (data frame) loaded$parameters_fresh
cfg$habitat_classification loaded$user_habitat_classification
cfg$observation_exclusions loaded$observation_exclusions
cfg$wsg_species loaded$wsg_species_presence
cfg$overrides$X loaded$X (e.g. loaded$user_barriers_definite)

Out of scope (follow-up issues):

  • crate schemas for the other 9 bcfp-sourced files (one issue per file as canonical-shape decisions concretize). Today they fall through to plain CSV read.
  • nge / local source families (when project-experimental configs need them).
  • Type-aware variant matching in crate (planned crate v0.1.x roadmap).

Ship the Modelling spawning and rearing habitat using bcfishpass defaults vignette (vignettes/habitat-bcfishpass.Rmd) on top of the post-phase-3 codebase. Regenerated bundled artifacts (inst/extdata/vignette-data/{rollup, sub_ch, sub_ch_bcfp}.rds) reflect the corrected emit semantics and tighter parity.

bcfishpass-bundle parity (5 WSGs × 5 species, spawn + rear):

  • 42 of 42 non-NA rows within ±5%
  • 35 of 42 within ±2%
  • median 1.1%; max 5.0%

Tighter than v0.13.1’s 100% within ±5% / median 1.5% claim because phase 1’s emit-semantics fix landed in main, and the regenerated rollup reflects it. Spawning rows that previously sat at +3-5% (BT/CH/CO/ST across multiple WSGs) are now at +0-2%.

The vignette text claim updated to match the new numbers. Cuts the v0.13.1 vignette’s residual-deltas paragraph that mentioned overlay-range-containment and stream-order-bypass — those were pre-phase-3 artifacts; with rule emission corrected, residual deltas are mostly segmentation-boundary rounding plus the documented stream-order bypass.

Phase 3 of #69 — proof artifact + emit-semantics fix.

Proof artifact: new research/rule_flexibility.md runs BABL × CO under three configs (use case 1, use case 2, bcfishpass) by swapping only dimensions.csv cells, with rules.yaml diffs side-by-side. Reproducible via data-raw/rule_flexibility_demo.R + data-raw/rule_flexibility_render.R. Demonstrates that every methodology dial is a CSV cell, no buried emission rules. The numbers prove the matrix:

  • Use case 1 (default bundle): rearing 1388.90 km, lake_rearing 54507.85 ha, wetland_rearing 5786.74 ha. Counts polygon-mainlines as linear AND rolls up polygon area.
  • Use case 2: rearing 1271.02 km, same area rollups. Excludes polygon-mainlines from linear via in_waterbody: false + area_only: true on L/W; areas still bucket via the polygon rules.
  • bcfishpass bundle: rearing 1271.02 km, no area rollup (no L/W polygon rules at all). Functionally identical rear predicate to use case 2 because area_only: true makes the L/W rules contribute to bucket flags only.

Emit-semantics fix in lnk_rules_build() (under #69 phase 1 banner — corrects a bug introduced in 0.14.0):

Previous behaviour: rear_stream_in_waterbody: yes emitted in_waterbody: true on the stream rule. fresh interprets that as “match segments inside polygons ONLY,” the opposite of the column’s intent (“include polygon-mainlines too”). The default bundle’s permissive rear was effectively only matching in-polygon segments — broken since 0.14.0 but never visible because the bcfishpass bundle (which set no for all species) was the only side tested for parity.

Corrected emit:

  • yes (or absent): omit the in_waterbody field. Rule matches segments inside AND outside polygons (today’s permissive default — polygon-mainlines count too).
  • no: emit in_waterbody: false. Rule matches outside polygons only (strict partition).

The third grammar state (in_waterbody: true = inside polygons only) has no biological use case for stream rules and is no longer emitted by lnk_rules_build().

bcfishpass bundle output unchanged: the bundle ships rear_stream_in_waterbody: no for all species, so the fixed emit produces byte-identical rules.yaml to 0.15.0. Default bundle output changes (now actually permissive — pass-through stream rule).

Tests updated (3 cases): yes (or absent) omits the field; no emits in_waterbody: false; default bundle smoke tests assert the rear stream rule has no in_waterbody field.

Phase 2 of #69. Adds dimensions-driven area_only emission + polygon-rule mainlines edge filter. Default bundle now ships use case 1 (linear includes mainlines through L/W polygons; area rolls up via bucket flags) with the new edge filter restricting polygon-rule contributions to mainlines only (1000/1100). bcfishpass bundle output unchanged.

New per-species columns in dimensions.csv:

  • rear_lake_area_only — yes/no — emit area_only: true on the L polygon rule. When yes, fresh derives the lake_rearing bucket flag from the rule but excludes it from the main rear predicate (linear). When no or absent, the rule contributes to both (today’s behaviour). Both bundles ship no for all species — default ships use case 1; bcfishpass ships parity-with-bcfp.
  • rear_wetland_area_only — yes/no — same shape on the W polygon rule. Both bundles ship no for all species.

Polygon-rule edge filter (edge_types_explicit: [1000, 1100] on L/W rules in the additive rear branch):

  • Restricts the L/W polygon rule’s match to mainlines (single-line main flow + secondary flow) when emitted under rear_lake: yes or rear_wetland: yes + rear_wetland_polygon: yes. Without the filter, polygon rules matched every segment in the polygon (shorelines 1700, banks 1800, island edges, construction lines), all crediting linear rearing. The bucket pred (lake_rearing / wetland_rearing) is unaffected — area still rolls up the polygon’s full area as long as any tagged segment exists in it.
  • The rear_lake_only branch (SK / KO) is intentionally not filtered — the L rule there IS the rear classification, must continue matching the whole lake polygon.

Default bundle methodology shift — use case 1: linear km includes mainlines through wetlands and lakes, with area rollups (lake_rearing_ha, wetland_rearing_ha) populating from the polygon footprint. rear_wetland_polygon flipped from no (v0.14.0) back to yes for rear_wetland=yes species. The 2026-04-27 cut to no was the right call given the v0.14.0 grammar (no edge filter; W rule would over-emit), but with the mainlines edge filter shipped here, polygon-mainlines are the right thing to count for linear AND area.

Required: fresh ≥ 0.24.0 (#182, fresh#184) — area_only predicate decouples bucket-flag derivation from the main rear predicate.

Tests — test-lnk_rules_build.R 130 PASS (was 124 in 0.14.0): 6 new tests covering area_only emission per the columns + polygon-edge-types filter present on L/W rules (additive branch only) + rear_lake_only branch left untouched. Full suite 554 PASS / 0 FAIL.

BABL parity (bcfishpass bundle): unchanged from 0.14.0 — 8 of 10 rows within ±2%, 10 of 10 within ±5%. The new knobs are inert when set to today’s defaults, so bcfp bundle output is byte-identical to v0.14.0.

Coordinates with #69 phase 3 — research/rule_flexibility.md proof artifact runs BABL × CO under three configs (use case 1, use case 2, bcfishpass) by swapping only dimensions.csv cells, with rules.yaml diffs side-by-side.

Dimensions-driven in_waterbody + bcfishpass-bundle methodology fixes that bring 5-species BABL parity to ±5% (8 of 10 rows within ±2%) on the bcfishpass bundle. The methodology dials are now visible in dimensions.csv cells per species — no buried emission rules.

New per-species columns (#69 phase 1):

  • spawn_stream_in_waterbody — yes/no — emit in_waterbody: <bool> on the stream-spawn rule. no excludes polygon-mainlines from spawn classification (the partition that pairs with waterbody_type: R/L/W polygon rules); yes is permissive and matches polygon-mainlines too. Both bundles ship with no for all species (biology — spawning happens in stream channels).
  • rear_stream_in_waterbody — yes/no — same shape on the stream-rear rule. bcfishpass bundle ships no (strict partition matches bcfishpass’s per-species access SQL); default bundle ships yes (NewGraph permissive — counts polygon-mainlines as rearing for species with rear_lake: yes etc., orthogonal to area rollups).
  • rear_wetland_polygon — yes/no — gate emission of the waterbody_type: W polygon rule. When no, only the 1050/1150 wetland-flow carve-out emits; when yes (or absent), the W polygon rule emits too (sets the wetland_rearing flag for area rollups). Both bundles ship no for all species — segments inside an FWA wetland polygon are wider than the fish-bearing channel and shouldn’t count as rearing habitat.

Methodology fixes carried in from earlier branch work (previously held in vignette-ship):

  • apply_habitat_overlay: false flag in pipeline: block of bcfishpass config.yaml. Comparison-scope choice, not a behavioural claim about bcfishpass. bcfishpass ships both layers: habitat_linear_<sp> (per-species rule output) and streams_habitat_linear (rule + known-habitat overlay blended). The bcfishpass bundle disables frs_habitat_overlay() so its output is rule-only and compares apples-to-apples against bcfishpass’s own rule layer (habitat_linear_<sp>). Comparing the rule slices in isolation keeps rule-emission drift from hiding behind known-habitat overlay drift; overlay parity is a separate question to revisit once rule parity is locked. Default bundle keeps overlay enabled (NewGraph methodology produces the blended output by default).
  • lnk_barrier_overrides() habitat-confirmation SQL updated for bcfishpass’s authoritative CSV shape (post-2026-04-26: species_code + spawning + rearing integer columns instead of the dropped habitat_ind column).
  • lnk_pipeline_prepare() empty-table fallback CREATE TABLE matches the new CSV shape.

Required: fresh ≥ 0.23.1 (#180, fresh#181, fresh#183) — adds the in_waterbody predicate to the rule grammar plus the validator hotfix.

Tests — test-lnk_rules_build.R 124 PASS (was 86): 6 new tests for in_waterbody emission across permutations + bundle-level smoke tests; 4 new tests for rear_wetland_polygon (yes/no/absent backward-compat). Full suite 516 PASS / 0 FAIL.

BABL parity (bcfishpass bundle): 8 of 10 spawning+rearing rows within ±2%; max 5.0%; max spawning drift 1.5% (was 4.8%). The remaining ±2-5% drift is a follow-up — phase 2 will add the area_only predicate (fresh#182) and edge_types_explicit: [1000, 1100] filter on polygon rules to support the use case 2 pattern (mainlines excluded from linear, area still rolls up).

Coordinates with #69 phase 2 — adds rear_lake_area_only / rear_wetland_area_only columns once fresh#182 lands. Phase 3 ships the proof artifact (research/rule_flexibility.md) running BABL × CO under three configs (use case 1, use case 2, bcfishpass) by swapping only dimensions.csv cells.

Shape fingerprint + halt auto-merge on shape drift (#64).

data-raw/sync_bcfishpass_csvs.R and the daily sync-bcfishpass-csvs.yml cron previously compared each bcfishpass-sourced CSV against a recorded sha256 byte checksum and auto-merged any drift. That worked for value drift (rows added/edited) but was blind to shape drift — bcfishpass’s 2026-04-26 long→wide reshape (with column type change) passed straight through and broke link’s pipeline downstream. This release adds a separate shape fingerprint alongside the byte checksum; the workflow auto-merges byte-only drift as before but halts shape drift for coordinated review.

  • New shape_checksum field in the provenance: block of each bundle’s config.yaml. Computed as sha256 of the file’s first line (whitespace-normalized). Catches column rename / add / remove / reshape — the dominant failure mode. Type changes within stable columns are out of scope (rarer; can extend later if needed).
  • data-raw/sync_bcfishpass_csvs.R computes shape fingerprint at sync time, classifies each file’s drift as byte or shape, writes the overall drift kind to /tmp/sync_drift_kind for the workflow to consume.
  • .github/workflows/sync-bcfishpass-csvs.yml reads the drift kind. Byte-only drift → auto-PR + auto-merge as today. Shape drift → auto-PR opens with schema-drift label, NOT auto-merged, workflow exits non-zero (red on Actions tab) so the change is visible. Coordinated review across link / fresh / crate is required before merging.
  • lnk_config_verify() extended with shape_drift column. Breaking (pre-1.0): old single drift column renamed to byte_drift; existing tibble shape now (file, byte_expected, byte_observed, byte_drift, shape_expected, shape_observed, shape_drift, missing).
  • lnk_stamp() markdown rendering surfaces both byte and shape drift counts in the provenance summary.
  • 15 new tests (468 total, was 453) — .lnk_shape_fingerprint() helper + shape-drift detection + missing-file handling + backward-compat path for bundles without shape_checksum: field.

Coordinates with crate’s adapter pattern (link#65, crate#2) — when shape drift fires, crate’s normalize handler is the right place to absorb the upstream change before link’s pipeline sees it.

Pick up fresh 0.22.0 overlay simplification — caller-side update for the canonical-shape contract.

  • lnk_pipeline_classify() now calls frs_habitat_overlay() with species_col = "species_code" + habitat_types = c("spawning", "rearing") instead of format = "long" + long_value_col = "habitat_ind". Matches the shape bcfishpass’s user_habitat_classification.csv adopted on 2026-04-26 (row-per-(segment × species), per-habitat indicator columns). Three-line caller-side diff; no link API change.
  • Suggests: fresh (>= 0.22.0). Coordinates with fresh#177.
  • Pipeline runs again. The vignette stays in dev/ until link#64 (sync workflow shape fingerprint) and link#65 (lnk_load_overrides() via crate::crt_ingest()) land.

bcfishpass vignette pulled out of pkgdown until tighter.

  • vignettes/reproducing-bcfishpass.Rmd → dev/habitat-bcfishpass.Rmd.draft. Same pattern as scoring-crossings — out of build path, preserved for resumption when content lands clean.
  • Content updates applied before move: title now “Modelling spawning and rearing habitat using bcfishpass defaults”; new scope paragraph describing what bcfishpass covers beyond linear classification; entrypoint replaced with explicit lnk_pipeline_* calls (was tar_make()); map section clarifies linear classification covers spawning/rearing/lake_rearing/wetland_rearing per species.
  • README.md: “Full pipeline (reproducing bcfishpass)” → “Full pipeline (linear habitat classification)”; broken pkgdown vignette link removed.
  • Open follow-ups: rollup-query retarget to streams_habitat_linear for apples-to-apples post-overlay comparison; range-containment relaxation in fresh::frs_habitat_overlay.

Vignette cleanup.

  • vignettes/scoring-crossings.Rmd moved to dev/scoring-crossings.Rmd.draft — out of build path until the scoring methodology lands.
  • vignettes/reproducing-bcfishpass.Rmd updated for the v0.9.0 overlay: added overlay step to the pipeline DAG, new “Known-habitat overlay” subsection, clarified rollup vs. map comparison.
  • data-raw/vignette_reproducing_bcfishpass.R: bcfishpass-side map query reads streams_habitat_linear (model + known) instead of habitat_linear_ch (model-only) for apples-to-apples comparison with link’s post-overlay output.
  • Regenerated bundled snapshots (inst/extdata/vignette-data/{rollup,sub_ch,sub_ch_bcfp}.rds) from v0.10.0 + overlay state.

Config-bundle provenance + run stamps — closes the drift attribution loop. Pipeline outputs that shift between runs on the same DB state can now be traced back to which input changed. Closes #40; supersedes the narrower scope of #24.

  • inst/extdata/configs/{bcfishpass,default}/config.yaml carry provenance: blocks with sha256 checksums for every tracked file. Externally sourced files (bcfishpass overrides) record source URL + upstream_sha (ea3c5d8, synced 2026-04-13) + path within source repo. Generated files (rules.yaml) record generated_from + generated_by + generator_sha. Hand-authored files record link’s git sha at edit time.
  • lnk_config() exposes parsed provenance as cfg$provenance (named list, one entry per tracked file). print(cfg) shows the count of tracked files.
  • New lnk_config_verify(cfg, strict) recomputes sha256 for every provenanced file and returns a tibble (file, expected, observed, drift, missing). Default warns on drift; strict = TRUE errors. digest added to Suggests.
  • New lnk_stamp(cfg, conn, aoi, db_snapshot) returns an lnk_stamp S3 list capturing the full set of inputs at run time: cfg provenance with current observed checksums, software versions and git SHAs (link, fresh, R), DB snapshot row counts (bcfishobs.observations, whse_basemapping.fwa_stream_networks_sp) when conn is provided, AOI + start_time. lnk_stamp_finish(stamp, result, end_time) finalizes; format(stamp, "markdown") renders for report appendix or run-log dump.
  • data-raw/compare_bcfishpass_wsg.R now emits a stamp markdown at the head of every WSG run, captured into data-raw/logs/*.txt via the standard stderr redirect.
  • Tests: 93 new — provenance parsing, drift detection (clean / mutated / missing / strict), bundled-config drift = 0 invariants, stamp shape + markdown rendering + finalization + db-snapshot opt-out.

Default config bundle now uses explicit FWA edge_type codes for spawn and rear-stream predicates, matching bcfishpass’s 20-year-validated convention.

  • data-raw/build_rules.R: switched both default rule-builder calls (inst/extdata/parameters_habitat_rules.yaml and inst/extdata/configs/default/rules.yaml) from edge_types = "categories" to edge_types = "explicit". Predicates now emit edge_types_explicit: [1000, 1100, 2000, 2300] in place of edge_types: [stream, canal] (which expanded to 1000/1050/1100/1150 + 2000/2100/2300).
  • Drops 1050/1150 (stream-thru-wetland) and 2100 (rare double-line canal) from spawn AND rear-stream rules. The dedicated wetland-rearing rule (edge_types_explicit: [1050, 1150] with thresholds: false) is unchanged — wetland_rearing flag still captures stream-thru-wetland segments for species with rear_wetland = yes. Net rearing flag (= rear_stream OR wetland_rearing OR rear_lake) is preserved for those species; species with rear_wetland = no (GR, KO) lose 1050/1150 from both spawn AND rearing.
  • ADMS preflight (M1, fresh 0.21.0): default-bundle spawning km drops 4-7% across all spawning species (BT 397→368, CH 296→279, CO 340→318, SK 98→94, RB 331→311). Rearing km essentially unchanged for rear_wetland = yes species. Full per-WSG numbers in research/default_vs_bcfishpass.md.
  • Default and bcfishpass bundles now emit structurally aligned spawn predicates — confirms bcfishpass’s edge-type convention is what link ships by default.
  • tests/testthat/test-lnk_rules_build.R: regression tests added — default rules.yaml has no 1050/1150/2100 in spawn or rear-stream predicates; the dedicated wetland-rear rule still carries [1050, 1150].

lnk_pipeline_classify() now overlays known habitat from user_habitat_classification.csv onto fresh.streams_habitat after rule-based classification. Closes #55.

  • After frs_habitat_classify() finishes, calls frs_habitat_overlay() (fresh ≥ 0.21.0) when the manifest declares habitat_classification. Loaded long-format table is overlaid via a 3-way bridge join through fresh.streams (range containment on [drm, urm]).
  • Closes the gap surfaced in research doc §5/§7: bcfishpass’s published streams_habitat_linear.spawning_sk > 0 blends model + observation-curated knowns; link’s pipeline previously only emitted the model side.
  • 5-WSG rerun (digest 0f00c713) shows BABL SK spawning under bcfishpass bundle rises from 57.6 → 85.2 km (+27.6 km from overlay). ADMS SK +5.14 km, BULK SK +0.8 km. Default bundle similar magnitudes.
  • Requires fresh ≥ 0.21.0 (overlay rename + bridge support; see fresh#175).

Default NewGraph habitat-classification config bundle ships alongside the bcfishpass reproduction bundle (#51).

  • New inst/extdata/configs/default/ bundle — intentional methodological departures from bcfishpass: intermittent streams included in rearing, wetland rearing added for resident species, lake rearing extended to species beyond SK/KO with per-species rear_lake_ha_min thresholds, river_skip_cw_min = yes. Loadable via link::lnk_config("default").
  • Per-species rear_lake_ha_min via a new column in configs/default/dimensions.csv. lnk_rules_build() prefers that value over the shared fresh::parameters_habitat_thresholds default when present, keeping bcfishpass bundle at its 200 ha threshold for SK/KO while letting default express species-specific biology (CO 2 ha, BT/WCT/RB/CT/DV 10 ha, GR 40 ha, ST 60 ha, CH 100 ha, SK/KO 200 ha). Non-numeric entries in the dimensions CSV fall through to the fresh fallback rather than silently disabling it.
  • Per-species rear_wetland_ha_min via a new column in configs/default/dimensions.csv. lnk_rules_build() now emits both edge_types: wetland (for rearing km) AND waterbody_type: W (drives wetland_rearing_ha rollup) rules when rear_wetland = yes. Thresholds: CO 0.5 ha (beaver complexes), BT/CH/CT/DV/RB/ST/WCT 1 ha.
  • SK + KO spawn_connected block — added five columns to configs/default/dimensions.csv (rear_stream_order_bypass, spawn_connected_direction, spawn_connected_gradient_max, spawn_connected_cw_min, spawn_connected_edge_types) so lnk_rules_build() emits the spawn_connected: block with direction: downstream for lake-obligate species. spawn_lake = no for SK/KO to prevent lake-centerline inflation (Babine Lake alone is 177 km).
  • data-raw/compare_bcfishpass_wsg() emits a compound rollup with 7 rows per species × WSG × config: spawning/rearing km, lake_rearing/wetland_rearing ha, plus three edge-type slice rows (rearing_stream, rearing_lake_centerline, rearing_wetland_centerline) for decomposing the rearing total. Reference side uses the same habitat_linear_<sp> + fwa_{lakes,wetlands}_poly methodology as link, so both sides are apples-to-apples.
  • data-raw/_targets.R runs both bundles side-by-side across all 5 validation WSGs (ADMS, BULK, BABL, ELKR, DEAD) — 10 comparison targets, unified rollup with a config identity column. Rollup digest e3eaf5f62df44d6713bfed32cd08fc5d (357 rows) on M1 with fresh 0.17.1.
  • New research doc research/default_vs_bcfishpass.md — methodology comparison, per-WSG per-species results, 9 observations covering the debugging journey (SK spawning over-inflation root causes, bcfishpass known-habitat overlay via streams_habitat_known, gradient-floor calibration, segment-averaging risk).
  • Three companion maps (data-raw/maps/sk_spawning_BABL*.R) — mapgl overlays of SK spawning BABL comparing bundle-vs-bundle and default-vs-bcfishpass-published (model + known); per-layer toggle, popups with id_segment / segmented_stream_id / plain-language edge_type / gradient / length.
  • Requires fresh >= 0.17.1 for waterbody_type: L/W rear-rule honouring + lake_ha_min / wetland_ha_min thresholds.
  • tests/testthat/test-lnk_rules_build.R — new suite with 56 tests covering lake + wetland rule emission (per-config ha_min, fresh fallback, rear_lake=no / rear_wetland=no), spawn rules (stream+canal vs explicit codes, spawn_lake, spawn_requires_connected, spawn_connected block), rear precedence (no_fw, lake_only, all_edges), river polygon + river_skip_cw_min, species skipping, rear_stream_order_bypass, non-numeric ha_min fallthrough.

user_barriers_definite no longer eligible for observation-based override (#48).

  • .lnk_pipeline_prep_natural() previously unioned barriers_definite into natural_barriers, which lnk_barrier_overrides() iterates over. Net effect: the 227 reviewer-added user-definite positions (EXCLUSION zones, MISC detections the model misses) could be re-opened by observations clearing the species threshold. Confirmed active on ELKR pre-fix — 4 override rows at Erickson Creek exclusion and Spillway MISC positions that bcfishpass keeps as permanent barriers.
  • bcfishpass’s model_access_*.sql builds the barriers CTE from gradient + falls + subsurfaceflow only and appends barriers_user_definite post-filter via UNION ALL. Observations and habitat filters never see user-definite rows, so they’re never overridable. link now matches this shape: natural_barriers is gradient + falls only; barriers_definite stays consumed separately as a break source in lnk_pipeline_break() and as a direct UNION ALL entry into fresh.streams_breaks via lnk_pipeline_classify().
  • ELKR rollup shifts toward bcfishpass: BT spawning +3.4% → +2.8%, WCT spawning +4.0% → +2.6%, WCT rearing +1.6% → +0.3%. Other four WSGs unchanged (ADMS/BABL/DEAD have empty barriers_definite; BULK has 87 rows but no observation-threshold matches to any of them).

Honour user_barriers_definite_control.csv at the observation-override step.

  • lnk_barrier_overrides() now excludes observations upstream of control-flagged positions from counting toward the override threshold, matching bcfishpass’s access SQL. Previously controlled positions (concrete dams, long impassable falls, diversions) could be re-opened by upstream historical observations (#44).
  • Gated per-species by a new observation_control_apply column in parameters_fresh.csv — TRUE for CH/CM/CO/PK/SK/ST; FALSE for BT/WCT; NA for CT/DV/RB. Residents routinely inhabit reaches upstream of anadromous-blocking falls (post-glacial headwater connectivity, no ocean-return requirement), so their observations still override. Matches bcfishpass’s per-model application.
  • Habitat-confirmation override path intentionally bypasses the control table — expert-confirmed habitat is higher-trust than observations, and bcfishpass’s hab_upstr CTE has no control join either.
  • .lnk_pipeline_prep_overrides now passes the control table to lnk_barrier_overrides() when the config manifest declares barriers_definite_control. Manifest key is the contract; no DB probe.
  • .lnk_pipeline_prep_load_aux now always creates a schema-valid (possibly empty) barriers_definite_control table when the manifest declares the key — fixes an asymmetric gating bug that would have raised “relation does not exist” on AOIs with zero control rows.
  • End-to-end validation WSG: DEAD (Deadman River) added to data-raw/_targets.R. It has a single barrier_ind = TRUE control row at FALLS (356361749, 45743) with six anadromous observations upstream and zero habitat coverage — the unique combination that actively exercises the filter. All four prior WSGs (ADMS/BULK/BABL/ELKR) were rescued by either the observation threshold or habitat path, making them parity checks rather than filter tests.

Documentation and narrative for the targets pipeline.

  • New vignette: “Reproducing bcfishpass with link + fresh” — three-line entrypoint, rollup interpretation, BULK chinook habitat map (mapgl), reproducibility framing. Data-prep script at data-raw/vignette_reproducing_bcfishpass.R generates inst/extdata/vignette-data/{rollup,bulk_ch}.rds from a real run; vignette loads the .rds so pkgdown builds don’t need fwapg access. Follows the CLAUDE.md convention for vignettes that need external resources (#38)
  • Research doc (research/bcfishpass_comparison.md) updated with bit-identical rollup numbers from 2026-04-22 and a new “Targets orchestration” section showing how _targets.R composes the per-WSG runs.
  • mapgl, sf added to DESCRIPTION Suggests.
  • Retired data-raw/compare_bcfishpass.R — data-raw/_targets.R + data-raw/compare_bcfishpass_wsg.R supersede it. Git history preserves the prior form.

Targets-driven comparison pipeline for all four validated watershed groups.

  • Add data-raw/_targets.R — tar_map(wsg = c("ADMS", "BULK", "BABL", "ELKR")) over a per-AOI target function, synchronous execution, dplyr::bind_rows rollup. fresh.streams is a shared schema so single-host parallelism would collide — runs serially today; distributed runs (M4 + M1) are a follow-up alongside a fresh upstream change for per-AOI output paths (#38)
  • Add data-raw/compare_bcfishpass_wsg(wsg, config) — per-AOI target function. Wraps the six lnk_pipeline_* phases, diffs the output against bcfishpass.habitat_linear_* reference on the tunnel DB, returns a ~10-row tibble (wsg × species × habitat_type × link_km × bcfishpass_km × diff_pct). KB-scale — safe to ship over SSH.
  • Promote .lnk_pipeline_classify_species to an exported lnk_pipeline_species(cfg, aoi) — canonical public API for “species this config classifies in this AOI.” Used by lnk_pipeline_classify and lnk_pipeline_connect internally and by the targets per-AOI function externally. Removes the duplicate private helper that was briefly inlined in data-raw/.
  • End-to-end verification (data-raw/logs/20260422_11_tar_make_final.txt) — 4 WSGs / 34 rows produced over 8.5 minutes wall clock (serial). Reproducibility: consecutive tar_make() invocations on the same DB state produce bit-identical rollup tibbles. Parity to bcfishpass (informational): all 34 diff_pct values within 5% of reference; research-doc drift (BT rearing: -0.7 → -1.1 pp) traces to env state between 2026-04-15 and today, not to pipeline non-determinism.

Pipeline phase helpers extract the bcfishpass comparison orchestration into composable building blocks. The 635-line data-raw/compare_bcfishpass.R is now 136 lines of sequenced helper calls.

  • Add lnk_pipeline_setup() — create the per-run working schema (#38)
  • Add lnk_pipeline_load() — load crossings and apply modelled-fix and PSCIS overrides
  • Add lnk_pipeline_prepare() — load falls / definite / control / habitat CSVs, detect gradient barriers, compute per-species barrier skip list, reduce to minimal set via fresh::frs_barriers_minimal(), load base segments
  • Add lnk_pipeline_break() — sequential frs_break_apply over observations / gradient / definite / habitat / crossings in config-defined order
  • Add lnk_pipeline_classify() — assemble access-gating breaks table and run fresh::frs_habitat_classify()
  • Add lnk_pipeline_connect() — per-species rearing-spawning clustering and connected-waterbody rules
  • Canonical signature (conn, aoi, cfg, schema) — aoi follows fresh convention (WSG code today; extends to ltree / sf polygons / mapsheets later), schema is the caller’s per-run namespace (working_<aoi> by convention) so parallel runs do not collide
  • cfg$species parsed from the rules YAML at lnk_config() load — intersects with cfg$wsg_species presence to pick per-AOI classify targets
  • Requires fresh 0.14.0 (for frs_barriers_minimal)

Config bundles for pipeline variants.

  • Add lnk_config(name_or_path) — load a config bundle (rules YAML, dimensions CSV, parameters_fresh, overrides, pipeline knobs) as one list object. Bundles live at inst/extdata/configs/<name>/ with a config.yaml manifest, or any directory containing config.yaml for custom variants (#37)
  • Relocate bcfishpass config files into inst/extdata/configs/bcfishpass/ (rules.yaml, dimensions.csv, parameters_fresh.csv, overrides/). All R scripts and data-raw/ references updated.

Initial release. Crossing connectivity interpretation layer — scores, overrides, and prioritizes crossings for fish passage using configurable severity thresholds and multi-source data integration.