Package index • pipdata

Pipeline entry points

Top-level functions for DLW ingestion, incremental processing, compatible stage execution, and read-only planning (see the “Pipeline” articles for the narrative walkthrough).

pipdata_dlw_process(): Acquire and validate DLW data
pipdata_get_gmd(): Acquire GMD catalog datasets and reconcile the local inventory
pipdata_validate_gmd(): Validate available GMD data and commit completed validation state
pd_run_pipeline(): Run the staged PIP data pipeline incrementally
pd_process_data(): Process DLW inventory and create cleaned pip data
pd_deflate_pipeline(): Batch-deflate every survey in the PIP master inventory
pd_change_report(): Report staged dependency changes without writing artifacts

Deflation

Deflates a cleaned survey’s welfare values; run as a separate, post-hoc step after pd_process_data().

pd_deflation(): Deflation of welfare using auxiliary data
deflation(): Deflation of welfare using auxiliary data (lower level)

Logging

Summarizes the unified "pipdata_log" across DLW acquisition, validation, survey cleaning, and deflation with stage-aware warnings.

log_report(): Generate a markdown report from a pipeline log.

Other functions

add_area(): Recode urban to area (lower level, S3 methods)
add_area(<pipgd>): Recode urban to area for group data
add_area(<pipmd>): Recode urban to area for micro data
add_dist_type(): Add distribution type (lower level, S3 methods)
add_dist_type(<pipgd>): Add distribution type group
add_dist_type(<pipmd>): Add distribution type micro
apply_recode_spec(): Apply recode specification to a data.table
build_pip_inventory(): Build the PIP master and release inventories from stamp catalogs
check_directory(): Check whether the working folder exists and abort if it does not
cln_changes(): Clean output from compare_aux_releases and compare_aux_vintages
cpfw_merge(): Merge country/survey PFW with dataliweb survey data
deflation(<pipgd>): Deflation of welfare for group data
deflation(<pipmd>): Deflation of welfare for micro data
diff_recode_spec(): Compare two recode_spec versions (or one version vs. package YAML)
dlw_clean(): Clean data from datalibweb structure (lower level, S3 methods)
dlw_clean(<pipgd>): Clean group data from Datalibweb original file
dlw_clean(<pipmd>): Clean micro data from Datalibweb original file
dlw_gmd_list(): Retrieve the GMD server catalog and initialize an acquisition inventory
dlw_gmd_match(): Match available local GMD rows to the current server catalog
dlw_gmd_new(): Compare the local GMD inventory with the current server catalog
dlw_gmd_unvalidated(): Get current available GMD datasets without completed validation
dlw_validation_engine(): DLW Validation Engine
export_recode_spec_yaml(): Export recode spec from stamp to YAML
get_country_pfw(): Get Country Price framework data based on PFW and DLW data info
get_data_status(): Get a simple frequency that shows number of valid and invalid datasets
get_validation_ctry(): List of validation result by country and module type
get_validation_list(): List of validation result by survey ID and module type
get_validation_report(): Get the validation report data
gmd_to_validate(): Get datasets list that needs to be validated
gmd_validated(): Return Validated GMD Records
inv_dlw_load(): Load DLW survey data and prepare it for the PIP pipeline
list_recode_spec_versions(): List recode_spec versions from stamp catalog
num_vars_to_attr(): Create a named vector of attributes
pd_aux_attr(): Build auxiliary metadata attributes for cleaned survey data
pd_cpfw_merge(): Merge country/survey PFW info with dataliweb survey data
pd_dlw_clean(): Clean data from datalibweb structure (High level)
pd_split_alt_welfare(): Split data based on alternative welfare
pd_wbpip_clean(): Clean data for wbpip compatibility (high level)
pipdata_int(): Get path to pipdata original files
ppp_to_wide(): Convert PPP data from pipload to wide format
print(<pipdata_pipeline_result>): Print a compact pipeline run result
print(<pipdata_stage_result>): Print a compact pipeline stage result
print(<pipeline_context>): Print compact pipeline context identity
process_data(): Process datalibweb data: merge PFW data and clean variables
save_pip_data(): Save cleaned PIP data or metadata to versioned storage
uniq_vars_to_attr(): convert variables with unique values along the data set to attributes and then remove those unique variables
unq_obs_dt(): Find unique values in PFW according to some key variables
valid_aux_load(): Retrieve the inventory of aux files that changed from previous release or vintage
valid_dlw_load(): Determine which DLW surveys need processing
vars_to_attr(): Make vars as attributes
vars_to_list(): Return a named list with unique values of variables
wbpip_clean(): Clean data to meet wbpip requirements and formats
wbpip_clean(<pipgd>): wbpip_clean method for pipgd class (group data)
wbpip_clean(<pipmd>): wbpip_clean method for pipmd class (microdata)