Skip to contents

Download Spatial Data to GeoPackage

Source: R/rfp_source.R
rfp_source.Rd

Unified entry point for downloading BC spatial data layers into a GeoPackage. The bcdata and osm types run in pure R; fwa/aws call shell scripts that use ogr2ogr, and aws additionally uses rio for the lateral-habitat raster. Run rfp_check_deps() to see what resolves on your machine.

Usage

rfp_source(
  type = c("bcdata", "fwa", "aws", "osm", "url", "stac"),
  watershed_groups,
  path_gpkg = NULL,
  layers = NULL,
  operation = c("generate", "update"),
  columns = NULL,
  dir_layers = NULL,
  file_gpkg = NULL,
  layout = NULL,
  s3_base = NULL,
  lookups = NULL
)

Arguments

type

Character. One of "bcdata", "fwa", "aws", "osm", "url", or "stac".

watershed_groups

Character vector of BC watershed group codes (e.g., c("BULK", "KLUM")).

path_gpkg

Character. Path to the output GeoPackage file. Created by the bcdata type; required to exist for fwa and aws types.

layers

Character vector. Specific layer names to download. If NULL (default), uses the default layer list from inst/lookups/. When provided for the fwa/aws types, a temporary layer list file is created and passed to the shell script.

operation

Character. One of "generate" (default) or "update". "generate" creates a new GeoPackage (the bcdata type errors if path_gpkg already exists). "update" refreshes layers in place in an existing GeoPackage and requires path_gpkg to exist. The fwa type always updates when the GeoPackage exists, regardless of operation.

columns

Optional data frame with columns source_layer,column (the column keep-list sidecar shape, see rfp_project_slim()). Keep-lists apply server-side for the bcdata type (WFS propertyName) and via ogr2ogr -select for the aws type, so slimmed layers never re-fatten on refresh. The fwa type ignores them (its layers are already lean).

dir_layers

Character. Directory holding one GeoPackage per layer under the split layout. NULL (default) resolves from getOption("rfp.dir_layers"). See rfp-layout.

file_gpkg

Character. File name for the single-GeoPackage (monolith) layout. NULL (default) resolves from getOption("rfp.file_gpkg"). See rfp-layout.

layout

Character. "split", "monolith", or NULL (default) to detect from what is already on disk. An explicit value skips detection, which is what project creation needs - nothing exists yet to detect.

s3_base

Character. Object-storage base the aws pass reads from, or NULL (default) to use the shipped host. Ignored by every other type. Only the pass-level base - a layer staged under its own prefix is described by inst/lookups/rfp_source_aws.csv, because a base alone cannot express a path (#268). rfp_project_update() supplies this from the project's own record, so a refresh cannot silently swap buckets.

lookups

Named list of override CSVs for the tiers whose parameters live in a lookup rather than in the manifest row (osm, url, stac), or NULL (default) for the shipped ones. Each is MERGED row-wise over the shipped rows, keyed on source_layer, override winning - a replacement file would make every shipped layer it omits unknown and abort the run. See rfp_source_lookup(), which resolves the same way and lets a caller validate an override before writing anything. Names outside those three are refused rather than ignored (rfp#285, rtj#299).

Value

Invisible character path to the GeoPackage.

Details

Six source types are available:

bcdata

BC Data Catalogue layers in pure R via the bcdata package (no Python CLI, no temp files in the project directory, per-layer failure isolation). Layers controlled by inst/lookups/rfp_source_bcdata.txt.

fwa

Freshwater Atlas layers from the Hillcrest Geo feature service. Layers controlled by inst/lookups/rfp_source_fwa.txt.

aws

S3-hosted FlatGeoBuf files, model parameters, and lateral habitat raster from bcfishpass. Layers controlled by inst/lookups/rfp_source_aws.txt. Requires an existing GeoPackage from a prior bcdata run.

osm

OpenStreetMap ways from the Overpass API, in pure R. Layers are declared in inst/lookups/rfp_source_osm.csv, which carries the tag selector, the geometry type and the declared column list for each - OSM's tag set varies with the AOI (17 distinct keys on one project area, 45 on another, neither containing the other), so a response-derived schema would make a layer's shape depend on where the project is. A tag absent from an AOI yields an all-NA column, never a missing one.

NOTE for anyone editing the block above: a BLANK LINE inside a \\describe{} \\item{} body silently deletes this ENTIRE details section under markdown = TRUE. Not a warning - roxygenise() succeeds, man/ round-trips byte-for-byte, and R CMD check passes with \\details{} empty. Isolated in a minimal package 2026-09-06: single-paragraph items render, one blank line inside an item drops everything. That is why the two paragraphs below sit outside the block rather than inside the osm item where they were written.

Two things differ about osm. Overpass rate-limits at 2 concurrent slots per IP, so requests are paced as well as retried (rfp.osm_min_gap) and fall back across endpoints (rfp.osm_endpoints); and a total failure warns rather than raising, so a refresh during an outage leaves the previous layer intact rather than aborting the project update.

osm licence: OSM is ODbL 1.0, where bcdata/fwa/aws are OGL-BC with no downstream conditions. Attribution is required and share-alike attaches to derived databases. rfp records this in three places - the lookup's licence/attribution columns, the layer's gpkg_contents.description (what QGIS shows in Layer Properties), and the rfp_tracking provenance row. Fine for a Mergin field project; a condition to honour when the geometry flows into a published report or a redistributed dataset.

The osm, url and stac lookups are defaults, not the only place their parameters can live. lookups takes a caller's own CSV per tier, merged row-wise over the shipped rows and keyed on source_layer, so changing a collection, an epoch or a selector is a commit in the repo that owns the config rather than an rfp release and a reinstall on every host (rfp#285, rtj#299). See rfp_source_lookup().

For downloading a single BC Data Catalogue layer in pure R (no Python CLI needed), see rfp_source_bcdata().

Examples

if (FALSE) { # \dontrun{
rfp_source("bcdata", c("ADMS"))
rfp_source("fwa", c("ADMS"))
rfp_source("aws", c("ADMS"))
rfp_source("osm", c("ADMS"))
} # }