Skip to contents

rfp — Reproducible Field Projects

GIS project assembly and digital field form management. Pull provincial datasets from the BC Data Catalogue, Freshwater Atlas, and cloud-hosted layers, clip to any set of watershed groups, and assemble a fully styled QGIS project with digital field forms — ready to deploy to mobile devices for collaborative offline field collection.

We build and maintain forms for fish passage assessment, habitat confirmation, effectiveness monitoring, restoration site investigation, eDNA sampling, and benthic invertebrate collection — with more added as programs evolve. Data collected in some forms is structured for direct submission to provincial database systems (FISS, PSCIS, CABIN), while other data lands within central database systems we maintain and share with partners. Photos are automatically renamed and organized into site directories. Multiple team members contribute within the same shared project, with changes syncing between field and office.

Installation 

pak::pkg_install("NewGraphEnvironment/rfp")

System dependencies

rfp shell scripts require GDAL and several Python CLI tools. Install with:

# GDAL (also needed by R's sf package)
brew install gdal        # macOS
# apt install gdal-bin   # Linux

# Python CLI tools (mergin-client, rasterio)
pip install uv
uv tool install mergin-client
uv tool install rasterio

rfp finds these in a uv tool install, on your PATH, in the package’s own venv (a source checkout), or wherever options(rfp.python_venv = ...) points.

Verify everything is available:

Quick start 

library(rfp)

# Create a field-ready QGIS project
rfp_project_create(
  name = "elk_river_2026",
  watershed_groups = c("ELKR"),
  template = "bcfishpass_mobile",
  forms = c("pscis", "fiss_site")
)

# Carry the project's own styling into the layers, so QGIS and Mergin
# render them without the .qgs
rfp_styles_apply(
  "~/Projects/gis/elk_river_2026/background_layers",
  qgs = "~/Projects/gis/elk_river_2026/elk_river_2026.qgs"
)

# Push to Mergin Maps for mobile field collection
rfp_mergin_create("newgraph/elk_river_2026", "~/Projects/gis/elk_river_2026")
rfp_mergin_share("newgraph/elk_river_2026", "field_tech_1", "writer")

# Audit who changed what, when
rfp_mergin_collaborators("newgraph/elk_river_2026")
rfp_mergin_history("newgraph/elk_river_2026", since = "2026-01-01")

# Which rows in a layer changed between two points in time?
# e.g. watershed groups added to the Fraser project since 2025-03-31
rfp_mergin_diff(
  project = "newgraph/sern_fraser_2024",
  path    = "background_layers.gpkg",
  layer   = "whse_basemapping.fwa_watershed_groups_poly",
  from    = "2025-03-31",
  key     = "watershed_group_code"
)

See the package documentation for vignettes and function reference.

How the pieces fit together

rfp is built around a layered workflow — each layer is independently usable, and they compose into a field-and-web pipeline. The same QGIS project is the unit of work for mobile field collection (Mergin Maps) and for web map deployment (QWC2):

Layer Functions / artefacts Role
Layer sources rfp_source(); spacehakr::spk_source_bcdata(), spacehakr::spk_source_url() Pluggable abstraction for where a layer comes from: BC Data Catalogue, CSVs on disk, cloud URLs, test fixtures. Project assembly reads the registry. The generic fetchers moved to spacehakr so public data sourcing is reproducible without this package; rfp_source_bcdata(), rfp_source_csv() and rfp_source_url() remain here as deprecated shims.
Project assembly rfp_project_create(), rfp_project_layers(), rfp_project_subset(), rfp_project_templates(); inst/templates/{bcfishpass_mobile,bcrestoration_mobile}.qgs Stitch a styled QGIS project together from background layers + forms + templates + methods PDFs. rfp_project_subset() derives a focused child project from a heavy parent (e.g. a one-watershed mobile deploy or a browser-renderable subset from a multi-region parent).
Forms rfp_form_types(), rfp_form_create(), rfp_form_build(); inst/lookups/rfp_form_types.csv, inst/extdata/forms/schema/*.csv Registry of field-form GeoPackages (PSCIS, FISS, eDNA, habitat confirmation, restoration, CABIN). Add a form to a project, deploy with the project, sync back as data. Forms not derived from an upstream spreadsheet are defined in a git-diffable schema CSV; rfp_form_build() generates their GeoPackage and QGIS form/symbology from it, so a new form or a restyle is a data edit.
QGIS authoring rfp_qgs_rename(), rfp_qgs_themes(), rfp_qgs_theme_names(), rfp_qgs_theme_set(), rfp_qgs_theme_rm() xml2-based transformations on .qgs files: WMS-safe layer naming, and map themes (visibility presets) as read/set/rm. Porting a theme between projects is read-then-set with no intermediate file; a theme can also be built from nothing but a data frame of layer names. Group state travels by layer-tree path rooted at a project-specific group name, so paths are resolved against the destination tree rather than copied — the reason a ported theme kept its group state only by accident before.
Rasters rfp_qgs_raster_add(), rfp_raster_styles(), rfp_raster_style_path(); inst/extdata/styles/raster/*.qml Put a DEM, orthophoto or air photo into a project without a QGIS Desktop round trip — surgical xml2 edits that leave the project’s version stamp alone. Styles are data: a .qml plus a registry row, so adding one is not a code change. The colour stretch is computed from the raster or set explicitly, because two DEMs of the same terrain must share a range or they render discontinuously where they meet.
Cartography rfp_qgs_style_export(), rfp_styles_apply(); inst/pyqgis/apply_styles.py, render_map.py Put a project’s styling inside the data, so a layer arrives correctly styled wherever it is added or read without a project — QGIS’s Add Layer, QGIS Server/QWC2, rfp_map_render(). A .qml is a project’s <maplayer> node minus its source binding, so styles are lifted as XML and written to layer_styles in pure R — symbology, labelling and all. This provisions a layer rather than rendering one: a saved .qgs draws from its own style block and never re-queries the data. Styles for layers no project covers can still be rebuilt from the gq registry via headless PyQGIS, which reports what that lossy path drops. gq remains the source of truth for tmap/mapgl rendering.
Field deployment (Mergin Maps) rfp_mergin_create/share/pull/push/sync/diff/history/... (12 fns) Deploy projects to Mergin, share with collaborators by role, sync changes between field and office, audit history with row-level diffs.
Web deployment (QWC2) rfp_qwc_config_create() → qwc_config.csv; consumed by rtj’s qwc_project-add.sh Generate the QWC2 themesConfig + visibility-preset entry for a project. Theme keep-list, initial theme, locked layers, bbox override, project title, URL suffix — all driven from the .qgs map themes. WMS-spec name compliance is handled by rfp_qgs_rename() so QWC2’s flattened group hierarchy doesn’t collide.
Field artefacts rfp_photo_metadata_rm(), rfp_gpx_import(), rfp_gpkg_to_geojson() Photo metadata scrubbing, GPS track import, format conversion.
Provincial metadata rfp_meta_bcd_xref(), rfp_meta_bcd_xref_col_comments() Cross-reference layers to BC Data Catalogue records for documentation + lineage; surfaces WMS endpoints for downstream consumption.

The mobile-and-web dual deployment is the design intent: one QGIS project, two delivery surfaces. rfp_project_subset() is the data-thinning primitive that makes the same project legible in a browser (full provincial layers won’t render in QWC2).

Roadmap

  • Form lifecycle deepening — registry expansion (more form types, partner-defined types) and tighter coupling to downstream database submission for FISS / PSCIS / CABIN. The CSV-driven form builder now generates a form’s GeoPackage and QML from a schema CSV (rfp_form_build()); the remaining step is reading that schema from the shared crate data dictionary rather than an rfp-local CSV (#24), plus line and polygon form geometry (#82).
  • Style-registry tightening — connect QGIS layer_styles tables to the central gq style registry so a style edit propagates to every project that references it (#15, #17).
  • QGIS version support — bump templates to QGIS 4.0 once stable (#68); multi-version schema fixtures for qgs/subset tests (#69).
  • Layer-source plugins — extend rfp_source_*() to additional cloud-hosted sources beyond BC Data Catalogue, direct URLs, and CSV.

Issues and feature requests live in GitHub Issues.