API Reference

The Python API is still early, but the objects on this page are the intended public library surface for scripts, notebooks, and reusable research workflows. For beginner workflow examples, start with Getting Started With the Beginner API; for module-level examples, see Advanced Library Use. This page is the generated member reference.

This page is generated with Sphinx autodoc, which imports modules and renders their docstrings and public members. See the Sphinx autodoc documentation for the underlying mechanism.

The lower-level sections are intentionally curated. They include objects that are useful for notebooks, scripts, reusable research workflows, and contributor extensions. CLI callbacks, formatting helpers, web-app internals, benchmark helpers, and other implementation details are left out even when they are public-by-name in the source tree.

Package

The top-level package exposes a small beginner-friendly API for concise notebooks and examples. The lower-level generated reference below is organized by the module that owns each concept.

Canadian synthetic population tooling.

The top-level package intentionally exposes a small beginner-friendly API for notebooks and short scripts. Import from modules such as synthpopcan.ipf or synthpopcan.tree when you need lower-level research and maintainer tools.

Beginner API

Beginner-friendly public API for notebooks and short scripts.

This module is the small, stable Python surface intended for people who want to use SynthPopCan without learning the internal command modules first. It favours plain file paths, lists of row dictionaries, and a small number of workflow functions that map directly to the two main beginner tasks:

  • fit seed rows to margin/control totals with IPF;

  • generate linked household/person rows from a prepared model package.

  • calibrate generated linked rows to small-area household controls.

Most users should import the top-level package and call these functions from there:

import synthpopcan as spc

controls = spc.read_controls("controls.csv")
fit = spc.fit_ipf("seed.csv", controls)
spc.write_weights(fit, "synthetic-weights.csv")

package = spc.read_model_package("model-package.json")
population = spc.generate_from_model(package, households=100)
spc.write_population(population, "synthetic-population/")
class synthpopcan.api.LinkedPopulation(households, persons)

Bases: object

Household and person rows generated from a linked model package.

A linked model package generates two related tables: one row per synthetic household, and one or more person rows inside those households. This object keeps those tables together so that downstream code does not accidentally lose the household/person relationship.

Parameters:

  • households (list[dict[str, str]]) – Synthetic household rows. The exact columns depend on the model package, but household identifiers are preserved when the package provides them.

  • persons (list[dict[str, str]]) – Synthetic person rows. Person rows are generated inside the synthetic households and may include household identifiers or household-level context columns.

Notes

Pass a LinkedPopulation to write_population() to write a directory containing households.csv and persons.csv.

synthpopcan.api.read_seed(path)

Read a seed CSV as plain row dictionaries.

Seed rows are the starting records for IPF. Each row should already contain the columns used by the margin/control table, for example age and sex in a small age/sex example.

Parameters:

path (str | Path) – Path to a CSV file with a header row.

Returns:

One dictionary per CSV row, using the CSV header values as keys.

Return type:

list[dict[str, str]]

Examples

>>> seed = read_seed("seed.csv")
>>> seed[0]["age"]
'18-64'
synthpopcan.api.read_controls(path)

Read a normalized SynthPopCan control CSV.

A normalized control CSV contains margin totals in the format used by SynthPopCan. These files can come from the CLI, the local web app, or a hand-prepared CSV.

Parameters:

path (str | Path) – Path to a normalized margin/control CSV.

Returns:

A parsed control table that can be passed directly to fit_ipf().

Return type:

synthpopcan.controls.ControlTable

See also

fit_ipf

Fit seed rows to the controls returned by this function.

synthpopcan.api.fit_ipf(seed, controls, *, weight_field=None, max_iterations=100, tolerance=1e-06)

Fit seed records to controls with iterative proportional fitting.

IPF adjusts weights on existing seed rows so that weighted totals match a set of margin/control totals. It does not create new categories or new variables. Every control dimension should already exist as a column in the seed rows.

Parameters:

  • seed (str | Path | Sequence[Mapping[str, Any]]) – Either a path to a seed CSV or an in-memory sequence of row mappings. Each row should include the dimensions named in the controls.

  • controls (str | Path | ControlTable | Sequence[IPFMargin]) – A path to a normalized control CSV, a synthpopcan.controls.ControlTable, or a sequence of lower-level synthpopcan.ipf.IPFMargin objects.

  • weight_field (str | None) – Optional seed column containing starting weights. If omitted, every seed row starts with weight 1.

  • max_iterations (int) – Maximum number of IPF passes through the controls.

  • tolerance (float) – Stop once the maximum absolute control error is at or below this value.

Returns:

The fitted records, fitted weights, convergence status, iteration count, and final maximum absolute error.

Return type:

synthpopcan.ipf.IPFResult

Raises:

ValueError – Raised when controls refer to missing seed columns or when a control cell cannot be represented by the available seed rows.

Examples

>>> fit = fit_ipf("seed.csv", "controls.csv")
>>> fit.converged
True
>>> write_weights(fit, "synthetic-weights.csv")
synthpopcan.api.expand_population(result, *, id_field='id')

Expand fitted IPF weights into full synthetic rows.

Weighted output is usually the practical default for browser and notebook work. Expanded rows are useful when another tool expects one row per synthetic person, household, or record, but they can become very large.

Parameters:

  • result (IPFResult) – A fitted IPF result from fit_ipf().

  • id_field (str) – Source seed-record identifier column to copy into the seed_id field of expanded rows. Each expanded row also receives a new synthetic_id.

Returns:

A full synthetic dataset where each row represents one expanded record.

Return type:

list[dict[str, str]]

Notes

Expansion integerizes fitted weights before repeating records. If the fitted weights represent a large population, the returned list can use substantial memory.

synthpopcan.api.write_weights(result, path, *, weight_column=None)

Write fitted seed records with a fitted-weight column.

This is the recommended way to save IPF output for most workflows. It keeps one row per seed record and adds a column containing the fitted synthetic population weight.

Parameters:

  • result (IPFResult) – A fitted IPF result from fit_ipf().

  • path (str | Path) – Destination CSV path.

  • weight_column (str | None) – Optional output column name for fitted weights. When omitted, the function uses weight unless that column already exists, in which case it uses fitted_weight.

Raises:

ValueError – Raised when the IPF result has no records to write.

Return type:

None

synthpopcan.api.read_model_package(path)

Read a linked household/person model package JSON.

Model packages are prepared artifacts created by SynthPopCan tooling. They can represent linked household/person generation without exposing raw microdata rows.

Parameters:

path (str | Path) – Path to a linked model package JSON file.

Returns:

The parsed package payload. Pass it to generate_from_model().

Return type:

dict[str, Any]

Raises:

ValueError – Raised when the file is not valid JSON, is not a JSON object, or uses an unsupported package schema.

synthpopcan.api.generate_from_model(package, *, households, conditions=None, random_seed=None, household_size_column=None, require_publishable=True)

Generate linked household and person rows from a prepared package.

This is the beginner-facing entry point for using an existing model package. It creates household rows first, then generates person rows inside those households using the package’s linked household/person model design.

Parameters:

  • package (str | Path | Mapping[str, object]) – A path to a linked model package JSON file, or an already-loaded package mapping returned by read_model_package().

  • households (int) – Number of synthetic households to generate.

  • conditions (Mapping[str, str] | None) – Optional fixed values for package condition columns. For example, {"geo": "Demo North"} asks the household model to generate rows for that geography when the package supports geo as a condition.

  • random_seed (int | None) – Optional random seed for reproducible generated rows.

  • household_size_column (str | None) – Optional override for the column that controls how many person rows are generated in each household. When omitted, the package setting is used, falling back to household_size.

  • require_publishable (bool) – When True, reject packages that are not marked as publishable candidates. Keep this enabled for normal use; set it to False only while inspecting trusted local development packages.

Returns:

Household and person rows generated from the package.

Return type:

LinkedPopulation

Raises:

ValueError – Raised when the package schema is invalid, required models are missing, an unsupported model type is encountered, or require_publishable is enabled for a package that is not marked as publishable.

Examples

>>> package = read_model_package("demo-linked-package.json")
>>> population = generate_from_model(package, households=25, random_seed=13)
>>> len(population.households)
25
synthpopcan.api.write_population(population, path)

Write generated population rows to CSV.

The output shape depends on the kind of population passed in:

  • a LinkedPopulation is written to a directory containing households.csv and persons.csv;

  • a flat list of row dictionaries is written to a single CSV file.

Parameters:

  • population (LinkedPopulation | list[dict[str, str]]) – Either a linked household/person population from generate_from_model(), or a flat list of row dictionaries such as the result from expand_population().

  • path (str | Path) – Destination directory for linked output, or destination CSV path for flat output.

Raises:

ValueError – Raised when there are no rows to write.

Return type:

None

Examples

>>> fit = fit_ipf("seed.csv", "controls.csv")
>>> rows = expand_population(fit)
>>> write_population(rows, "expanded.csv")
synthpopcan.api.calibrate_small_area_linked(*, households, persons, controls, geography_dimension, geography_column, households_out, persons_out, report_out=None, weights_out=None, household_id_column='synthetic_household_id', person_id_column='synthetic_person_id', weight_field=None, max_iterations=100, tolerance=1e-06, pool_size=None)

Calibrate linked household/person candidates to small-area controls.

This is the beginner-friendly Python entry point for the small-area workflow. It takes linked household and person candidate CSVs, assigns household rows to a target geography such as census tract (ct) or aggregate dissemination area (ada), and writes new household/person CSVs that preserve the household/person links.

The first implementation calibrates household-level controls. Person rows inherit the geography of their assigned household. Use linked-output validation afterward when person-level geography totals matter.

Parameters:

  • households (str | Path) – Candidate household CSV, usually generated from a prepared linked model package.

  • persons (str | Path) – Candidate person CSV linked to households by the household ID column.

  • controls (str | Path) – Normalized control CSV with one dimension naming the target geography and one or more household dimensions available in households.

  • geography_dimension (str) – Name of the geography dimension in the control CSV, for example ct or ada.

  • geography_column (str) – Output column name to write on assigned household and person rows.

  • households_out (str | Path) – Destination CSV for assigned household rows.

  • persons_out (str | Path) – Destination CSV for assigned person rows.

  • report_out (str | Path | None) – Optional JSON report path containing convergence and assigned-row counts by geography.

  • weights_out (str | Path | None) – Optional CSV path for fitted household weights. This can be very large for many small areas, so it is usually omitted.

  • household_id_column (str) – Household ID column shared by the candidate household and person CSVs.

  • person_id_column (str) – Person ID column in the candidate person CSV.

  • weight_field (str | None) – Optional candidate-household starting weight column.

  • max_iterations (int) – Maximum IPF iterations for each geography-specific household fit.

  • tolerance (float) – Convergence tolerance for each geography-specific household fit.

  • pool_size (int | None) – Maximum candidate households to use. Values of 5 000–10 000 reproduce aggregate statistics with near-identical accuracy to the full pool and run ~10× faster. Pass None (default) to use the full pool, which is needed when individual-household uniqueness matters.

Returns:

A summary report with total assigned household/person counts, geography counts, and per-geography fit diagnostics.

Return type:

dict[str, Any]

Examples

>>> summary = calibrate_small_area_linked(
...     households="candidate-households.csv",
...     persons="candidate-persons.csv",
...     controls="ct-tenure-controls.csv",
...     geography_dimension="ct",
...     geography_column="ct",
...     households_out="synthetic-households.csv",
...     persons_out="synthetic-persons.csv",
...     report_out="small-area-report.json",
... )
>>> summary["assigned_households"] > 0
True

Controls

Normalized control table parsing.

class synthpopcan.controls.ControlCell(categories, count)

Bases: object

One target count in a normalized control table.

Parameters:

  • categories (dict[str, str]) – Mapping from dimension names to canonical category values, such as {"age": "age_025_029", "sex": "female"}.

  • count (float) – Target population count for that exact combination of categories.

class synthpopcan.controls.ControlMargin(name, dimensions, cells)

Bases: object

A named collection of target cells over the same dimensions.

A margin represents one table of constraints, such as age by sex or household size by tenure. Every cell in the margin must use the same ordered dimensions so it can be converted into an IPF margin.

Parameters:

  • name (str) – Human-readable margin label used in reports.

  • dimensions (tuple[str, ...]) – Ordered dimension names that define each category tuple.

  • cells (tuple[synthpopcan.controls.ControlCell, ...]) – Target cells belonging to this margin.

to_ipf_margin()

Convert this control margin into the IPF margin representation.

Return type:

IPFMargin

class synthpopcan.controls.ControlTable(margins, dimensions)

Bases: object

A normalized set of one or more control margins.

Control tables are the bridge between source-specific files, such as Statistics Canada downloads, and the generic IPF engine. They preserve the margins as interpretable objects and can be converted to IPF margins when fitting weights.

Parameters:

  • margins (tuple[synthpopcan.controls.ControlMargin, ...]) – Margins that should be applied to the same synthetic-population workflow.

  • dimensions (tuple[str, ...]) – Union of the category dimensions used by the margins, in file order.

to_ipf_margins()

Convert all margins in this table into IPF margins.

Return type:

list[IPFMargin]

synthpopcan.controls.build_wds_category_mapping_template(path, *, dimensions, preset='blank')

Build a source-to-canonical category mapping template for a WDS ZIP.

The returned mapping has the shape {dimension: {source_label: canonical_label}}. The canonical preset fills known age and sex Census Profile labels where possible; blank leaves every target empty for manual review.

Parameters:

  • path (Path)

  • dimensions (tuple[str, ...])

  • preset (str)

Return type:

dict[str, dict[str, str]]

synthpopcan.controls.census_profile_template(name, *, geography_column='GEO_CODE', geography_dimension='geo', characteristic_column='CHARACTERISTIC_NAME', count_column='C1_COUNT_TOTAL')

Return a starter mapping template for common Census Profile margins.

Supported template names are "age5" and "sex". The returned object can be written as JSON, reviewed, and passed to read_census_profile_control_table().

Parameters:

  • name (str)

  • geography_column (str)

  • geography_dimension (str)

  • characteristic_column (str)

  • count_column (str)

Return type:

dict

synthpopcan.controls.inspect_census_profile_characteristics(path, *, characteristic_column='CHARACTERISTIC_NAME', count_column='C1_COUNT_TOTAL', search=None, limit=25)

List characteristic labels from a Census Profile CSV.

This helps users discover the exact source labels needed in a mapping file. Each returned dictionary includes the characteristic label, an example count, and the number of source rows with that label.

Parameters:

  • path (Path)

  • characteristic_column (str)

  • count_column (str)

  • search (str | None)

  • limit (int)

Return type:

list[dict[str, str]]

synthpopcan.controls.inspect_wds_zip(path, *, sample_rows=5)

Inspect a WDS ZIP and return columns, sample rows, and command hints.

The result is a plain dictionary suitable for JSON output. It includes the selected CSV member, row count, candidate count columns, candidate dimension columns, sample rows, and a suggested command-line normalization command.

Parameters:

  • path (Path)

  • sample_rows (int)

Return type:

dict[str, Any]

synthpopcan.controls.read_category_mapping(path)

Read and validate a WDS category mapping JSON file.

The mapping must be a JSON object whose values are objects mapping source labels to canonical category values.

Parameters:

path (Path)

Return type:

dict[str, dict[str, str]]

synthpopcan.controls.read_census_profile_control_table(path, mapping_path)

Normalize a Census Profile CSV using a JSON mapping file.

The mapping file identifies the geography column, characteristic column, count column, and one or more margins whose characteristic labels should be converted into canonical categories.

Parameters:

  • path (Path)

  • mapping_path (Path)

Return type:

ControlTable

synthpopcan.controls.read_census_profile_mapping(path)

Read and validate a Census Profile control mapping JSON file.

The function validates the high-level structure and raises ValueError for missing keys before any source CSV is processed.

Parameters:

path (Path)

Return type:

dict

synthpopcan.controls.read_control_margins(path)

Read a normalized controls CSV and return IPF-ready margins.

This is a convenience wrapper around read_control_table() for callers that only need the IPF representation.

Parameters:

path (Path)

Return type:

list[IPFMargin]

synthpopcan.controls.read_control_table(path)

Read a normalized controls CSV into a ControlTable.

The CSV must include margin, dimensions, category columns, and count. Rows are grouped into margins by the margin value. The dimensions column is a comma-separated list naming the category columns that define a cell.

Raises:

ValueError – If required columns are missing, counts are not numeric, a margin mixes dimensions, or a target cell is duplicated.

Parameters:

path (Path)

Return type:

ControlTable

synthpopcan.controls.read_wds_control_table(path, *, dimensions, count_column, margin_name, category_mapping=None)

Read a Statistics Canada WDS ZIP as a normalized control table.

dimensions and count_column name columns inside the ZIP’s CSV file. category_mapping can translate source labels into seed-data category codes before the controls are passed to IPF. The returned table contains one margin named by margin_name.

Raises:

ValueError – If the ZIP cannot be interpreted as a WDS table, required columns are missing, counts are invalid, or the same target cell appears twice.

Parameters:

  • path (Path)

  • dimensions (tuple[str, ...])

  • count_column (str)

  • margin_name (str)

  • category_mapping (dict[str, dict[str, str]] | None)

Return type:

ControlTable

synthpopcan.controls.write_control_table(path, table)

Write a ControlTable to the normalized controls CSV format.

The output can be read back with read_control_table() or used by the command-line IPF workflows.

Parameters:
Return type:

None

IPF

Iterative proportional fitting for seed records and margin tables.

class synthpopcan.ipf.IPFMargin(dimensions, targets)

Bases: object

A target margin used to calibrate seed records with IPF.

Parameters:

  • dimensions (tuple[str, ...]) – Ordered column names that identify the category cells in this margin.

  • targets (collections.abc.Mapping[tuple[str, ...], float]) – Mapping from category tuples to the desired weighted count for each cell. Each key must have the same length and order as dimensions.

class synthpopcan.ipf.IPFResult(records, weights, converged, iterations, max_abs_error)

Bases: object

Result returned by fit_ipf().

The result keeps the original records alongside calibrated weights so callers can inspect convergence, validate totals, or expand the weighted seed records into integer synthetic rows.

Parameters:

  • records (collections.abc.Sequence[collections.abc.Mapping[str, Any]]) – Original seed records passed to fit_ipf().

  • weights (list[float]) – Calibrated weight for each seed record, in the same order as records.

  • converged (bool) – Whether the fit reached the requested tolerance before exhausting the iteration limit.

  • iterations (int) – Number of full IPF adjustment passes completed.

  • max_abs_error (float) – Largest absolute residual between a target cell and the fitted weighted total when fitting stopped.

margin_totals(dimensions)

Calculate weighted totals for one set of dimensions.

Parameters:

dimensions (tuple[str, ...])

Return type:

dict[tuple[str, …], float]

synthpopcan.ipf.calculate_max_abs_error(records, weights, margins)

Return the largest absolute residual across all margin cells.

This helper is useful when validating saved weights or comparing an IPF result after additional filtering or expansion.

Parameters:

  • records (Sequence[Mapping[str, Any]])

  • weights (Sequence[float])

  • margins (Sequence[IPFMargin])

Return type:

float

synthpopcan.ipf.expand_records(records, weights, *, id_field='id')

Expand weighted seed records into integer synthetic records.

Fractional weights are rounded with integerize_weights(). The output includes synthetic_id and seed_id columns so generated rows can be traced back to their seed-record source without exposing private data.

Parameters:

  • records (Sequence[Mapping[str, Any]]) – Seed records whose calibrated weights should be expanded.

  • weights (Sequence[float]) – Non-negative weights in the same order as records.

  • id_field (str) – Optional seed-record column to copy into seed_id. If the column is missing, the one-based record index is used.

Return type:

list[dict[str, str]]

synthpopcan.ipf.fit_ipf(records, margins, *, weight_field=None, max_iterations=100, tolerance=1e-06)

Fit record weights so seed records match supplied control margins.

Parameters:

  • records (Sequence[Mapping[str, Any]]) – Seed records containing the categorical columns named by each margin.

  • margins (Sequence[IPFMargin]) – Target totals that the weighted records should match.

  • weight_field (str | None) – Optional seed-record column containing starting weights. If omitted, every seed record starts with weight 1.0.

  • max_iterations (int) – Maximum number of full adjustment passes over all margins.

  • tolerance (float) – Stop once the largest absolute residual is at or below this value.

Raises:

ValueError – If records or margins are empty, if weights are invalid, or if a positive target cell has no seed records that can represent it.

Return type:

IPFResult

Notes

IPF can only adjust weights for category combinations represented in the seed records. A missing positive target cell is a structural problem in the seed/control design, not a convergence problem.

synthpopcan.ipf.integerize_weights(weights)

Convert non-negative fractional weights to integer replication counts.

Uses systematic sampling: the cumulative weight axis is divided into N equal-width intervals and one sample is drawn from each interval starting at the mid-point of the first. This is deterministic and preserves proportional distributions regardless of weight magnitude.

The largest-remainder method (the prior implementation) degenerates when all weights are less than 1 — every floor is 0 and the N remainder bumps all land on whichever group has the highest per-candidate weight, discarding the margin structure entirely. This arises in small-area synthesis when a large candidate pool (e.g. 50 000 records) is fitted to a small per-CT target (e.g. 1 400 households), giving every candidate a weight of ~0.03.

Raises:

ValueError – If any weight is negative.

Parameters:

weights (Sequence[float])

Return type:

list[int]

synthpopcan.ipf.validate_margin_coverage(records, margins)

Check that every positive target cell is represented by seed records.

Use this before fitting when you want to report unsupported controls as an input-design problem. The function returns None when all positive target cells have at least one matching seed record and raises ValueError for the first unsupported target.

Parameters:

  • records (Sequence[Mapping[str, Any]])

  • margins (Sequence[IPFMargin])

Return type:

None

synthpopcan.ipf.weighted_totals(records, weights, dimensions)

Aggregate weighted records by a tuple of categorical dimensions.

The keys in the returned dictionary are ordered tuples of string category values matching dimensions. This is the same representation used by IPFMargin.

Parameters:

  • records (Sequence[Mapping[str, Any]])

  • weights (Sequence[float])

  • dimensions (tuple[str, ...])

Return type:

dict[tuple[str, …], float]

Microdata

Census microdata seed sample contracts and adapters.

class synthpopcan.microdata.SeedSample(level, source_format, records, columns, weight_column, geography_columns, id_columns, metadata=<factory>)

Bases: object

A loaded seed microdata sample with normalized metadata.

Seed samples keep raw CSV rows plus the column roles needed by IPF and tree workflows, such as identifier, geography, and weight columns.

Parameters:

  • level (Literal['household', 'person']) – Whether records describe households or people.

  • source_format (str) – Adapter name identifying how the source file was interpreted.

  • records (tuple[dict[str, str], ...]) – Source rows represented as dictionaries of strings.

  • columns (tuple[str, ...]) – Column names available in records.

  • weight_column (str | None) – Optional column containing source weights.

  • geography_columns (tuple[str, ...]) – Columns that locate records geographically.

  • id_columns (tuple[str, ...]) – Columns that identify source records within the sample.

  • metadata (dict[str, Any]) – Additional source-specific summary values.

as_summary()

Return a compact, JSON-serializable summary of the sample.

Return type:

dict[str, Any]

class synthpopcan.microdata.TreeColumnBlockSpec(name, level, target_columns, conditioning_columns)

Bases: object

A named set of target and conditioning columns for tree workflows.

Blocks are teaching and workflow aids: they group plausible target and conditioning columns for supported source formats without hiding the final modelling decision from the caller.

Parameters:

  • name (str)

  • level (Literal['household', 'person'])

  • target_columns (tuple[str, ...])

  • conditioning_columns (tuple[str, ...])

class synthpopcan.microdata.TreeColumnSuggestionProfile(source_format, geography_columns, identifier_columns, weight_columns, replicate_weight_prefixes, derived_columns, blocks)

Bases: object

Column-role suggestions for a known source microdata format.

A profile records known geography, identifier, weight, replicate-weight, derived, and block columns for a source adapter. It is used to produce reviewable suggestions rather than silently choosing a model design.

Parameters:

  • source_format (str)

  • geography_columns (tuple[str, ...])

  • identifier_columns (tuple[str, ...])

  • weight_columns (tuple[str, ...])

  • replicate_weight_prefixes (tuple[str, ...])

  • derived_columns (tuple[str, ...])

  • blocks (tuple[TreeColumnBlockSpec, ...])

synthpopcan.microdata.build_tree_geography_feasibility_report(sample, *, geography_column, household_block='household_core', person_block='person_demographics', likely_person_rows=10000, likely_household_rows=4000, borderline_person_rows=2500, borderline_household_rows=1000, min_support=50, max_purity=0.95)

Assess which geographies have enough support for tree modelling.

The report compares person and household row counts, condition-group support, and dominant-outcome purity so users can avoid modelling very sparse or identifying subsets.

Parameters:

  • sample (SeedSample)

  • geography_column (str)

  • household_block (str)

  • person_block (str)

  • likely_person_rows (int)

  • likely_household_rows (int)

  • borderline_person_rows (int)

  • borderline_household_rows (int)

  • min_support (float)

  • max_purity (float)

Return type:

dict[str, Any]

synthpopcan.microdata.check_statcan_2016_household_seed_columns(sample, *, columns)

Check whether selected columns are constant within each household.

The report is JSON-serializable and includes one check per requested column, plus checks for WEIGHT and derived household_size.

Parameters:
Return type:

dict[str, Any]

synthpopcan.microdata.derive_statcan_2016_household_seed_sample(sample, *, columns)

Derive household seed records from person-level hierarchical rows.

One output record is produced per HH_ID. Selected household columns and WEIGHT must be constant within the household; household_size is derived from the number of person rows.

Parameters:
Return type:

SeedSample

synthpopcan.microdata.export_seed_rows(sample, *, columns)

Select seed columns for IPF or simple synthetic-population examples.

The returned tuple contains output rows and a JSON-serializable manifest. Identifier, geography, derived household-size, and weight columns are kept when they are known on the input sample.

Parameters:
Return type:

tuple[list[dict[str, str]], dict[str, Any]]

synthpopcan.microdata.export_statcan_2016_household_training_rows(sample, *, target_columns, conditioning_columns)

Export one household-level training row per household ID.

Household-level source columns must be constant within each household. Use check_statcan_2016_household_seed_columns() before exporting when preparing a model design interactively.

Parameters:

  • sample (SeedSample)

  • target_columns (tuple[str, ...])

  • conditioning_columns (tuple[str, ...])

Return type:

tuple[list[dict[str, str]], dict[str, Any]]

synthpopcan.microdata.export_statcan_2016_person_training_rows(sample, *, target_columns, conditioning_columns)

Export person-level training rows from 2016 hierarchical microdata.

household_size may be requested even though it is not a source column; it is derived from the number of persons sharing each HH_ID.

Parameters:

  • sample (SeedSample)

  • target_columns (tuple[str, ...])

  • conditioning_columns (tuple[str, ...])

Return type:

tuple[list[dict[str, str]], dict[str, Any]]

synthpopcan.microdata.export_training_rows(sample, *, level, target_columns, conditioning_columns)

Export a tree-training view from a supported seed sample.

This dispatches to the supported source-specific household or person export routine. It returns rows plus a manifest describing source format, level, selected columns, identifiers, and weight column.

Parameters:

  • sample (SeedSample)

  • level (Literal['household', 'person'])

  • target_columns (tuple[str, ...])

  • conditioning_columns (tuple[str, ...])

Return type:

tuple[list[dict[str, str]], dict[str, Any]]

synthpopcan.microdata.read_fixture_seed_sample(path, *, level, weight_column, geography_columns, id_columns)

Read a small fixture CSV as a seed sample.

This adapter is mainly for examples, tests, and teaching workflows where column roles are supplied directly by the caller. It performs only column validation; it does not infer census-specific roles.

Parameters:

  • path (Path)

  • level (Literal['household', 'person'])

  • weight_column (str | None)

  • geography_columns (tuple[str, ...])

  • id_columns (tuple[str, ...])

Return type:

SeedSample

synthpopcan.microdata.read_statcan_2016_hierarchical_seed_sample(path)

Read a Statistics Canada 2016 hierarchical microdata extract.

The returned sample is person-level and records known identifier columns, the WEIGHT column, household counts, person counts, and a basic duplicate-person-ID check in its metadata.

Parameters:

path (Path)

Return type:

SeedSample

synthpopcan.microdata.resolve_tree_column_block_pair(sample, *, household_block, person_block)

Resolve named household and person column blocks into column tuples.

Returns household targets, household conditioning columns, person targets, person conditioning columns, and a design report. The report highlights feasibility issues that should be reviewed before training linked models.

Parameters:

  • sample (SeedSample)

  • household_block (str)

  • person_block (str)

Return type:

tuple[tuple[str, …], tuple[str, …], tuple[str, …], tuple[str, …], dict[str, Any]]

synthpopcan.microdata.suggest_tree_column_blocks(sample)

Suggest tree-model column blocks for a known source format.

Suggestions are source-aware but not authoritative. They are intended to start a model-design review by naming available blocks, missing columns, and excluded identifiers or replicate weights.

Parameters:

sample (SeedSample)

Return type:

dict[str, Any]

Statistics Canada

Statistics Canada download helpers.

class synthpopcan.statcan.CensusProfileDownload(geo_level, label, filetype, geono)

Bases: object

Metadata for one supported 2016 Census Profile bulk download.

Instances describe the geography level, display label, download parameters, and local filename for a supported Census Profile CSV.

Parameters:

  • geo_level (str)

  • label (str)

  • filetype (str)

  • geono (str)

property url: str

Return the Statistics Canada download URL for this entry.

property filename: str

Return the local filename used for this download.

class synthpopcan.statcan.WDSTableSearchResult(product_id, cansim_id, title_en, start_date, end_date)

Bases: object

One table match from the Statistics Canada WDS cube list.

Search results contain just enough metadata for a user or script to choose a candidate product ID and then request metadata or a full-table download.

Parameters:

  • product_id (str)

  • cansim_id (str)

  • title_en (str)

  • start_date (str)

  • end_date (str)

as_dict()

Return a JSON-serializable representation for CLI output.

Return type:

dict[str, str]

synthpopcan.statcan.classify_wds_ipf_suitability(dimensions)

Classify whether WDS dimensions look useful for simple IPF controls.

This is a heuristic based only on dimension names. It can suggest that age and sex controls look plausible, but it cannot verify category compatibility with a seed sample.

Parameters:

dimensions (list[str])

Return type:

dict[str, Any]

synthpopcan.statcan.extract_wds_dimension_names(metadata)

Extract English dimension names from WDS metadata.

The helper tolerates small shape differences in WDS metadata payloads and returns an empty list when dimension names cannot be found.

Parameters:

metadata (dict[str, Any])

Return type:

list[str]

synthpopcan.statcan.extract_wds_dimension_previews(metadata, *, member_limit=3)

Extract short member previews for each WDS dimension.

Each preview includes the dimension name, total member count, a short member sample, and a truncated flag.

Parameters:

  • metadata (dict[str, Any])

  • member_limit (int)

Return type:

list[dict[str, Any]]

synthpopcan.statcan.fetch_census_profile_2016(geo_level, out_dir)

Download a supported 2016 Census Profile CSV and manifest.

geo_level must be one of the supported keys in _CENSUS_PROFILE_2016_DOWNLOADS. The function returns the CSV path and writes a JSON provenance manifest beside it.

Parameters:

  • geo_level (str)

  • out_dir (Path)

Return type:

Path

synthpopcan.statcan.fetch_wds_metadata(product_id)

Fetch metadata for a WDS product ID.

Raises ValueError when Statistics Canada returns no usable metadata for the requested product. Network and service errors from urllib are not wrapped so callers can decide how to retry or report them.

Parameters:

product_id (str)

Return type:

dict[str, Any]

synthpopcan.statcan.fetch_wds_table(product_id, out_dir, lang='en')

Download a WDS full-table ZIP and write a small provenance manifest.

Returns the path to the downloaded ZIP. A JSON manifest is written beside the ZIP with source, product ID, language, service URL, final download URL, and local path.

Parameters:

  • product_id (str)

  • out_dir (Path)

  • lang (str)

Return type:

Path

synthpopcan.statcan.normalize_language(lang)

Normalize a WDS language code to en or fr.

Parameters:

lang (str)

Return type:

str

synthpopcan.statcan.normalize_product_id(product_id)

Validate and normalize a Statistics Canada product ID.

Product IDs must contain only digits. Leading and trailing whitespace is stripped, but the digit string is otherwise preserved.

Parameters:

product_id (str)

Return type:

str

synthpopcan.statcan.search_wds_tables(query, limit=10)

Search the WDS cube list for tables matching all query terms.

The search is a simple case-insensitive term match over product ID, CANSIM ID, English and French titles, and date fields. It is useful for discovery, not as a ranked search engine.

Parameters:

  • query (str)

  • limit (int)

Return type:

list[WDSTableSearchResult]

synthpopcan.statcan.summarize_wds_metadata(metadata)

Summarize WDS metadata into dimensions, suitability hints, and commands.

The summary is designed for display and JSON output. It includes dimension names, short member previews, a rough IPF suitability classification, and suggested next commands for downloading and normalizing controls.

Parameters:

metadata (dict[str, Any])

Return type:

dict[str, Any]

synthpopcan.statcan.wds_all_cubes_lite_url()

Return the WDS endpoint URL for the all-cubes lite index.

Return type:

str

synthpopcan.statcan.wds_download_url(product_id, lang='en')

Build the WDS endpoint URL that returns a full-table download link.

This returns the service endpoint, not the final ZIP URL. Call fetch_wds_table() to resolve and download the file.

Parameters:

  • product_id (str)

  • lang (str)

Return type:

str

synthpopcan.statcan.wds_metadata_url()

Return the WDS endpoint URL for cube metadata lookup.

Return type:

str

Tree Models

Tree-based synthetic population generator contracts.

class synthpopcan.tree.CartTreeModel(spec, feature_categories, target_classes, children_left, children_right, feature, threshold, value, n_node_samples, weighted_n_node_samples, source_format, records_trained, min_samples_leaf, max_depth, release_class='private_working', model_type='cart')

Bases: object

Serialized scikit-learn CART classifier for synthetic row generation.

The object stores the tree structure and class counts needed for generation without storing a live scikit-learn estimator or raw training rows. CART models can be harder to explain than conditional-frequency models, so audit support and purity before treating them as shareable artifacts.

Parameters:

  • spec (TreeModelSpec)

  • feature_categories (dict[str, tuple[str, ...]])

  • target_classes (tuple[dict[str, str], ...])

  • children_left (tuple[int, ...])

  • children_right (tuple[int, ...])

  • feature (tuple[int, ...])

  • threshold (tuple[float, ...])

  • value (tuple[tuple[float, ...], ...])

  • n_node_samples (tuple[int, ...])

  • weighted_n_node_samples (tuple[float, ...])

  • source_format (str)

  • records_trained (int)

  • min_samples_leaf (int)

  • max_depth (int | None)

  • release_class (str)

  • model_type (str)

property feature_names: tuple[str, ...]

Return one-hot encoded feature names in model order.

to_dict()

Serialize the CART model to a JSON-compatible dictionary.

Return type:

dict[str, Any]

classmethod from_dict(payload)

Deserialize a CART model payload.

Parameters:

payload (dict[str, Any])

Return type:

CartTreeModel

class synthpopcan.tree.FrequencyGroup(conditions, support, outcomes)

Bases: object

Observed outcomes for one set of conditioning values.

A frequency group is the transparent equivalent of a tree leaf: it records the conditions, total support, and possible target outcomes observed in the training view.

Parameters:

  • conditions (dict[str, str])

  • support (float)

  • outcomes (tuple[FrequencyOutcome, ...])

to_dict()

Return a JSON-serializable representation.

Return type:

dict[str, Any]

class synthpopcan.tree.FrequencyOutcome(values, weight)

Bases: object

One possible target outcome and its training weight.

values maps target column names to generated category values. weight is the observed weighted support for that outcome within a conditioning group or globally.

Parameters:

  • values (dict[str, str])

  • weight (float)

to_dict()

Return a JSON-serializable representation.

Return type:

dict[str, Any]

class synthpopcan.tree.FrequencyTreeModel(spec, groups, global_outcomes, source_format, records_trained, release_class='private_working', min_support_threshold=5, model_type='conditional-frequency')

Bases: object

Conditional-frequency tree model used for transparent generation.

This model stores aggregate outcome counts by conditioning group rather than source rows, which makes it easier to audit before sharing. Generation chooses a conditioning group and samples among its weighted outcomes.

The model is still a model artifact and should be audited before release: small groups and highly pure groups can reveal too much about the source data even when raw rows are not present.

Parameters:
to_dict()

Serialize the model to a JSON-compatible dictionary.

Return type:

dict[str, Any]

classmethod from_dict(payload)

Deserialize a conditional-frequency model payload.

Parameters:

payload (dict[str, Any])

Return type:

FrequencyTreeModel

class synthpopcan.tree.TreeGenerationRequest(model_spec, rows, geography_values=(), random_seed=None)

Bases: object

Request parameters for generating rows from a tree model.

This dataclass is mainly a reusable contract for callers that want to pass generation requests around before calling generate_tree_rows().

Parameters:

  • model_spec (TreeModelSpec)

  • rows (int)

  • geography_values (tuple[str, ...])

  • random_seed (int | None)

class synthpopcan.tree.TreeModelSpec(level, target_columns, conditioning_columns, geography_column=None, weight_column=None, random_seed=0, model_family='tree-based')

Bases: object

Column roles and generation settings shared by tree model types.

A model spec is the compact description of what a tree model learned: which level it operates at, which columns it generates, which columns it conditions on, and what random seed should be used by default.

Parameters:

  • level (Literal['household', 'person'])

  • target_columns (tuple[str, ...])

  • conditioning_columns (tuple[str, ...])

  • geography_column (str | None)

  • weight_column (str | None)

  • random_seed (int)

  • model_family (str)

as_summary()

Return a JSON-serializable summary of the model specification.

Return type:

dict[str, Any]

class synthpopcan.tree.TreeTrainingSample(level, source_format, records, columns, target_columns, conditioning_columns, geography_column=None, weight_column=None, metadata=<factory>)

Bases: object

Training rows and column roles for a tree-based generator.

Parameters:

  • level (Literal['household', 'person']) – Whether the rows describe households or people.

  • source_format (str) – Name of the adapter or file format that produced the training rows.

  • records (tuple[dict[str, str], ...]) – Training rows as dictionaries of strings.

  • columns (tuple[str, ...]) – Column names available in the training rows.

  • target_columns (tuple[str, ...]) – Columns the model should generate.

  • conditioning_columns (tuple[str, ...]) – Columns used to select or predict target outcomes.

  • geography_column (str | None) – Optional column used to describe geographic scope.

  • weight_column (str | None) – Optional column containing training weights.

  • metadata (dict[str, Any]) – Additional source-specific notes for reports or manifests.

as_summary()

Return a compact, JSON-serializable summary of this sample.

Return type:

dict[str, Any]

synthpopcan.tree.audit_tree_model(model, *, min_support=50, max_purity=0.95)

Check whether a tree model meets basic release-review thresholds.

The report flags low-support groups or leaves, high-purity groups or leaves, source-row/privacy metadata, and release-class status. It is a screening tool for review, not a proof that a model is safe or substantively valid.

Parameters:
Return type:

dict[str, Any]

synthpopcan.tree.generate_linked_population(household_model, person_model, *, households, household_conditions=None, household_size_column='household_size', random_seed=None)

Generate linked household and person records.

The household model first generates household attributes, including household size. The person model then generates the requested number of people for each household using any shared conditioning columns.

Returns:

Household rows and person rows. Both include synthetic household identifiers so the records can be joined.

Return type:

tuple[list[dict[str, str]], list[dict[str, str]]]

Parameters:
synthpopcan.tree.generate_tree_rows(model, *, rows, conditions=None, random_seed=None)

Generate synthetic rows from either supported tree model type.

conditions may provide values for some or all conditioning columns. For conditional-frequency models, unknown or partial conditions fall back toward broader/global outcomes; for CART models, omitted conditions are encoded as empty strings.

Parameters:
Return type:

list[dict[str, str]]

synthpopcan.tree.read_cart_model(path)

Read a CART tree model JSON file.

Raises ValueError if the JSON file contains a different supported model family.

Parameters:

path (Path)

Return type:

CartTreeModel

synthpopcan.tree.read_frequency_model(path)

Read a conditional-frequency tree model JSON file.

Raises ValueError if the JSON file contains a different supported model family.

Parameters:

path (Path)

Return type:

FrequencyTreeModel

synthpopcan.tree.read_tree_model(path)

Read a tree model JSON file and return the matching model class.

The model type is read from the JSON payload and dispatched to either FrequencyTreeModel or CartTreeModel.

Parameters:

path (Path)

Return type:

FrequencyTreeModel | CartTreeModel

synthpopcan.tree.read_tree_training_sample(path, *, level, target_columns, conditioning_columns, geography_column=None, weight_column=None)

Read a CSV training view for tree-model fitting.

The caller must provide explicit target and conditioning columns. The function validates that all requested columns exist and returns a TreeTrainingSample suitable for train_frequency_model() or train_cart_model().

Parameters:

  • path (Path)

  • level (Literal['household', 'person'])

  • target_columns (tuple[str, ...])

  • conditioning_columns (tuple[str, ...])

  • geography_column (str | None)

  • weight_column (str | None)

Return type:

TreeTrainingSample

synthpopcan.tree.train_cart_model(sample, *, random_seed=0, min_samples_leaf=5, max_depth=None)

Train a CART model from a tree-training sample.

Categorical conditioning columns are one-hot encoded in a deterministic order before fitting a scikit-learn DecisionTreeClassifier. The returned model is serialized into plain Python data structures for JSON output.

Parameters:

  • sample (TreeTrainingSample)

  • random_seed (int)

  • min_samples_leaf (int)

  • max_depth (int | None)

Return type:

CartTreeModel

synthpopcan.tree.train_frequency_model(sample, *, random_seed=0, min_support=5)

Train a conditional-frequency model from a tree-training sample.

The model groups training rows by conditioning values and stores weighted target-outcome counts for each group. This is often easier to inspect and teach than CART because the generated probabilities are direct aggregates.

Parameters:
Return type:

FrequencyTreeModel

synthpopcan.tree.validate_linked_population(households, persons, *, household_id_column='synthetic_household_id', person_household_id_column='synthetic_household_id', household_size_column='household_size')

Validate household-person links and household-size consistency.

The report checks that person rows reference generated households and that each household has the number of persons stated in household_size_column.

Parameters:

  • households (list[dict[str, str]])

  • persons (list[dict[str, str]])

  • household_id_column (str)

  • person_household_id_column (str)

  • household_size_column (str)

Return type:

dict[str, Any]

synthpopcan.tree.write_generated_rows(path, rows)

Write generated rows to CSV using the first row’s column order.

This helper is intentionally simple and deterministic. It raises ValueError for empty output because there is no first row from which to infer CSV columns.

Parameters:

  • path (Path)

  • rows (list[dict[str, str]])

Return type:

None

synthpopcan.tree.write_tree_model(path, model)

Write a tree model JSON file.

The JSON schema is the portable artifact format used by the CLI and library. It stores model structure, metadata, and privacy/release metadata, but not raw training rows.

Parameters:
Return type:

None

Validation

Validation reports for generated population artifacts.

synthpopcan.validation.build_control_validation_report(control_table, records, weights, *, tolerance=1e-06, artifact_kind='weights')

Compare weighted records against every cell in a control table.

Returns a JSON-serializable report with margin summaries, cell residuals, and reader-facing issue messages for residuals above tolerance. The report is appropriate for CLI output, notebooks, tests, or provenance files.

Parameters:

  • control_table (ControlTable)

  • records (Sequence[Mapping[str, Any]])

  • weights (list[float])

  • tolerance (float)

  • artifact_kind (str)

Return type:

dict[str, Any]

synthpopcan.validation.build_distribution_comparison(training_rows, training_weights, generated_rows, generated_weights, dimensions)

Build one weighted distribution comparison for a dimension group.

The result includes category-level training and generated proportions, absolute proportion deltas, and the maximum absolute delta for the group.

Parameters:

  • training_rows (Sequence[Mapping[str, Any]])

  • training_weights (list[float])

  • generated_rows (Sequence[Mapping[str, Any]])

  • generated_weights (list[float])

  • dimensions (tuple[str, ...])

Return type:

dict[str, Any]

synthpopcan.validation.build_tree_output_validation_report(*, training_rows, generated_rows, target_columns, conditioning_columns, weight_field=None, tolerance=0.05)

Compare generated tree output with its training distribution.

The report checks target columns, joint target distributions, and selected conditioning columns for large proportion shifts or unseen generated category combinations. Unknown generated categories are reported as errors; distribution shifts beyond tolerance are reported as warnings.

Parameters:

  • training_rows (Sequence[Mapping[str, Any]])

  • generated_rows (Sequence[Mapping[str, Any]])

  • target_columns (tuple[str, ...])

  • conditioning_columns (tuple[str, ...])

  • weight_field (str | None)

  • tolerance (float)

Return type:

dict[str, Any]

synthpopcan.validation.comparison_dimensions(target_columns, conditioning_columns)

Return one-way and joint dimension groups used for tree validation.

The returned groups are the default dimensions compared by build_tree_output_validation_report().

Parameters:

  • target_columns (tuple[str, ...])

  • conditioning_columns (tuple[str, ...])

Return type:

list[tuple[str, …]]

Local Data And Sources

Local data layout checks.

class synthpopcan.localdata.DataLayoutCheck(name, status, detail, path, tip='')

Bases: object

One local data-layout check returned by inspect_local_data_layout().

The object records a check name, status, human-readable detail, checked path, and optional tip for resolving missing or invalid local files.

Parameters:

  • name (str)

  • status (str)

  • detail (str)

  • path (Path)

  • tip (str)

synthpopcan.localdata.inspect_local_data_layout(data_root)

Inspect whether expected local data directories and metadata exist.

The checks are intentionally non-destructive and do not read private source data. They are used by the data doctor command and by scripts that want to explain local setup problems before running a workflow.

Parameters:

data_root (Path)

Return type:

list[DataLayoutCheck]

Local source inspection helpers.

synthpopcan.sources.inspect_source_root(root)

Summarize files under a local source-data root.

The result includes the root path, total file count, and extension counts. It does not parse source files or expose sample rows.

Parameters:

root (Path)

Return type:

dict[str, Any]

synthpopcan.sources.read_source_sample(path, rows)

Read a small sample from a local tabular source file.

The result includes path, delimiter, columns, and up to rows row dictionaries. Callers should treat sampled rows as potentially private.

Parameters:

  • path (Path)

  • rows (int)

Return type:

dict[str, Any]

synthpopcan.sources.read_source_schema(path)

Read the delimiter and header row from a local tabular source file.

Parameters:

path (Path)

Return type:

dict[str, Any]