Skip to contents

Generate a markdown report from a pipeline log.

log_report.Rd

Parses a piplog object produced by pipfun::log_filter() and writes a structured markdown document summarising errors, informational messages, and affected surveys.

Usage

log_report(
  log = NULL,
  path = NULL,
  title = "Pipeline Log Report",
  overwrite = FALSE
)

Arguments

log

A piplog object (inherits from data.table). Default NULL, in which case it is loaded internally via pipfun::log_filter(name = "pipdata_log").

path

Character scalar. File path for the output .md file. If NULL (default), the report is returned as a character vector and not written to disk.

title

Character scalar. Title for the report (default: "Pipeline Log Report").

overwrite

Logical. Overwrite path if it already exists (default: FALSE).

Value

Invisibly, the report as a character vector (one element per line). If path is non-NULL, the file is written as a side-effect.

Details

The report contains:

  • Running metadata (time window, total entries, success/fail counts).

  • Stage-aware warnings for DLW-only, pipeline-only, no-op, and incomplete runs.

  • DLW acquisition summary, including attempted, successful, and failed survey counts and failure details from the latest attempt.

  • DLW validation summary, separating valid, invalid, execution-failed, and workflow outcomes from the latest attempt.

  • Processing summary: total, cleaned, and failed counts (from process_summary_inf log entry).

  • Deflation summary: candidates, successes, failures, and failing surveys (from deflate_summary_inf log entry).

  • Auxiliary file changes: which measures changed and how many surveys were affected (from aux_changes_inf log entry).

  • Summary table by error / info type.

  • Country-level breakdown of errors.

  • Inventory verification: confirmed vs missing surveys (from inv_update_inf log entry).

  • Surveys skipped during data processing or metadata creation (skipped_svys_data / skipped_svys_metadata entries), with reasons.

  • List of surveys that failed processing (null_svys_inf entry).

Acquisition and validation are segmented independently from their latest attempt_start entry. An exact completion entry is preferred; logs produced before completion entries existed use a fallback confined to that latest segment. All DLW acquisition, validation, and wrapper discriminators are excluded from generic type and country sections, so dedicated DLW sections own those entries without historical leakage or double counting. Other sections that rely on a specific logmeta entry are silently omitted when that entry is absent.

See also

Other pd_process_data pipeline: add_attr(), aux_hash_candidates(), build_pip_inventory(), create_attr(), data_to_dt(), filter_aux_data(), filter_aux_inv(), fix_year_var(), get_aux_hashes(), inv_dlw_load(), inv_to_process(), pd_aux_attr(), pd_deflation(), pd_run_pipeline(), resolve_force_surveys(), save_pip_data(), survey_id_to_attr(), valid_dlw_load()

Examples

if (FALSE) { # \dontrun{
# Return as character vector (log defaults to the "pipdata_log")
report <- log_report()
# Write to file
log_report(path = "log_report.md", overwrite = TRUE)
} # }