
Load the downstream groups' barriers into a persist schema
Source:R/lnk_downstream.R
lnk_downstream_load.RdPuts the barriers and barrier_overrides rows that
lnk_downstream_files() lists into the persist schema cfg names
(cfg$pipeline$schema, set as the run will set it), so a
lnk_pipeline_run() of the target there walks downstream through the rows
the provincial run's walk read. Each part is fetched and checked for size
and md5 before the database is touched; every group is then replaced in one
transaction, with each part's row count checked, and recorded in
<schema>.log_downstream_load.
Usage
lnk_downstream_load(
conn,
files,
cfg,
replace = FALSE,
allow_missing = FALSE,
cache_dir = tools::R_user_dir("link", "cache")
)Arguments
- conn
DBI connection to the target database.
- files
The data frame
lnk_downstream_files()returns, for one target and config.- cfg
The
lnk_configthe target will be modelled with, itspipeline$schemaset to the persist schema the run writes.- replace
Replace a dependency group's rows even when this schema built them, or holds them from no recorded load.
- allow_missing
Load what the export has even when some dependencies' rows are not in it.
- cache_dir
Where downloaded parts are kept.
Value
Invisibly, a data frame per dependency loaded: wsg, run_id,
barriers, barrier_overrides (rows; NA for a table files did not
list), config_hash, config_same.
Details
It refuses, before writing, when a dependency's rows are not in the export
(skipped or not_exported, unless allow_missing), and when the schema
holds rows of a dependency group that did not come from an earlier load:
a build of the group here (a finished <schema>.log row), or rows of
unknown origin (unless replace).
It reports, without refusing, each dependency build whose config_hash
differs from cfg's bundle (the rows were built under other rules than the
target will be), and each input whose identity here differs from what the
dependencies were built from: CABD's and PSCIS's content md5, the
observations' and modelled crossings' loader version id (attribute
inputs_differ of the result, whose state is differs or not compared: an input the build recorded no identity for is named, not
passed over). The target's own inputs on this machine still decide its own
barriers and lifts.
See also
Other export:
lnk_catalog(),
lnk_db_load(),
lnk_downstream_files(),
lnk_scenario_export()
Examples
if (FALSE) { # \dontrun{
conn <- lnk_db_conn(dbname = "fwapg", host = "localhost", port = 5432L,
user = "postgres", password = "postgres")
cfg <- lnk_config("default")
cfg$pipeline$schema <- "fresh_default"
f <- lnk_downstream_files("s3://fresh-bc/habitat/fwa/v1/manifest.json",
wsg = "PARS", config = "default")
lnk_downstream_load(conn, f, cfg)
} # }