Package index • pipdata

Pipeline wrappers

The three top-level wrapper functions that make up the pipeline (see the “Pipeline” articles for the narrative walkthrough).

pipdata_dlw_process(): Process DLW Data
pipdata_get_gmd(): Retrieve and Save GMD Catalog Datasets to a Local Directory
pipdata_validate_gmd(): Validate GMD data and generate inventory report data
pd_process_data(): Process DLW inventory and create cleaned pip data

Deflation

Deflates a cleaned survey’s welfare values; run as a separate, post-hoc step after pd_process_data().

pd_deflation(): Deflation of welfare using auxiliary data
deflation(): Deflation of welfare using auxiliary data (lower level)

Logging

Summarizes the "pipdata_log" written during pd_process_data().

log_report(): Generate a markdown report from a pipeline log

Other functions

add_area(): Recode urban to area (lower level, S3 methods)
add_area(<pipgd>): Recode urban to area for group data
add_area(<pipmd>): Recode urban to area for micro data
add_dist_type(): Add distribution type (lower level, S3 methods)
add_dist_type(<pipgd>): Add distribution type group
add_dist_type(<pipmd>): Add distribution type micro
apply_recode_spec(): Apply recode specification to a data.table
build_pip_inventory(): Build the PIP master and release inventories from stamp catalogs
check_directory(): Check whether the working folder exists and abort if it does not
cln_changes(): Clean output from compare_aux_releases and compare_aux_vintages
copy_dlw_metadata(): Copy DLW Metadata Between Release Folders
cpfw_merge(): Merge country/survey PFW with dataliweb survey data
deflation(<pipgd>): Deflation of welfare for group data
deflation(<pipmd>): Deflation of welfare for micro data
diff_recode_spec(): Compare two recode_spec versions (or one version vs. package YAML)
dlw_clean(): Clean data from datalibweb structure (lower level, S3 methods)
dlw_clean(<pipgd>): Clean group data from Datalibweb original file
dlw_clean(<pipmd>): Clean micro data from Datalibweb original file
dlw_gmd_list(): Retrieve a List of GMD datasets from the Server and save it in the local dlw inventory folder.
dlw_gmd_match(): Get the list of current GMD datasets that match the local inventory
dlw_gmd_new(): Compare the local GMD dataset list with the server version to identify new entries.
dlw_gmd_unvalidated(): Get un-validated datasets list
export_recode_spec_yaml(): Export recode spec from stamp to YAML
get_country_pfw(): Get Country Price framework data based on PFW and DLW data info
get_data_status(): Get a simple frequency that shows number of valid and invalid datasets
get_validation_ctry(): List of validation result by country and module type
get_validation_list(): List of validation result by survey ID and module type
get_validation_report(): Get the validation report data
gmd_to_validate(): Get datasets list that needs to be validated
gmd_validated(): Return Validated GMD Records
inv_dlw_load(): Load DLW survey data and prepare it for the PIP pipeline
list_recode_spec_versions(): List recode_spec versions from stamp catalog
num_vars_to_attr(): Create a named vector of attributes
pd_aux_attr(): Build auxiliary metadata attributes for cleaned survey data
pd_cpfw_merge(): Merge country/survey PFW info with dataliweb survey data
pd_dlw_clean(): Clean data from datalibweb structure (High level)
pd_split_alt_welfare(): Split data based on alternative welfare
pd_wbpip_clean(): Clean data for wbpip compatibility (high level)
pipdata_int(): Get path to pipdata original files
ppp_to_wide(): Convert PPP data from pipload to wide format
process_data(): Process datalibweb data: merge PFW data and clean variables
save_pip_data(): Save cleaned PIP data or metadata to versioned storage
uniq_vars_to_attr(): convert variables with unique values along the data set to attributes and then remove those unique variables
unq_obs_dt(): Find unique values in PFW according to some key variables
valid_aux_load(): Retrieve the inventory of aux files that changed from previous release or vintage
valid_dlw_load(): Determine which DLW surveys need processing
vars_to_attr(): Make vars as attributes
vars_to_list(): Return a named list with unique values of variables
wbpip_clean(): Clean data to meet wbpip requirements and formats
wbpip_clean(<pipgd>): wbpip_clean method for pipgd class (group data)
wbpip_clean(<pipmd>): wbpip_clean method for pipmd class (microdata)