Skip to content

synthetic dataset generation by housetype - c0 #91

Description

@DesmondCampbell

Hi Vlad,

Further to my previous issue 'synthetic dataset generation by housetype',
this new issue reports a different problem specific to household type 'c0'.
I should have said that in the previous issue I'm generating 10000 rows, and also here.
I wanted to attached the python here but the PyCharm export from a .iypynb file creates a .html file.
Github won't allow .html attachments, so I'll send you the .html file via GU email.

The error occurs after the 100% progress report from
new_data = _syn.generator()

Here is the output up to the error

INFO:  2025-07-27 11:27:49.132365 sHhTypeCurr = c0 minb = 0
INFO:  2025-07-27 11:27:49.132432 sHhTypeCurr = c0 minb = 0 micb = 0
Sampling rows: 100%|██████████| 10000/10000 [00:02<00:00, 4384.75it/s]

Here is the error report

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[9], line 20
     17             print("INFO: already generated files - ", sFileOut)
     18             continue
---> 20 new_data = _syn.generator(_local_model=trainedModelCurr,
     21                           _household_type=sHhTypeCurr,
     22                           _local_dropouts=dropoutsCurr,
     23                           _mini_batch_id=minb,
     24                           _micro_batch_id=micb,
     25                           _total_rows=10000)
     26 if not bSchemaGenerated:
     27      _schema = _syn.generate_schema(new_data)

File ~/HEED/VladHandover/14apr25/synthwave/src/synthwave/synthesizer/uk/generator.py:702, in Syntets.generator(self, _local_model, _household_type, _local_dropouts, _mini_batch_id, _micro_batch_id, _total_rows, _batch_size)
    699 if _household_type.endswith("0"):
    700     # couple, no children
    701     synthetic_data = generate_personal_ids(synthetic_data, contains_couples=True)
--> 702     synthetic_data = Procrustes.stretch_long(synthetic_data, "a")
    703 else:
    704     # couple, several children
    705     if _household_type in ["c1", "c2"]:

File ~/HEED/VladHandover/14apr25/synthwave/src/synthwave/utils/procrustes.py:79, in Procrustes.stretch_long(df_, _postfix)
     76 _column_list = [_c[:-1] for _c in _column_list]
     77 # assume single digit indices only
     78 # move from ["sex_a1", "age_a1"] to ["sex_a", "age_a"]
---> 79 return (pd.
     80         wide_to_long(df_,
     81                      set(_column_list),
     82                      i="id_household",
     83                      j="inter_household_id").
     84         reset_index().
     85         drop(columns=["inter_household_id"]).
     86         rename(columns={_c: _c[:-2] for _c in _column_list if _c.endswith("_a")})
     87         )

File ~/miniconda3/envs/synthwave/lib/python3.12/site-packages/pandas/core/reshape/melt.py:488, in wide_to_long(df, stubnames, i, j, sep, suffix)
    485     stubnames = list(stubnames)
    487 if df.columns.isin(stubnames).any():
--> 488     raise ValueError("stubname can't be identical to a column name")
    490 if not is_list_like(i):
    491     i = [i]

ValueError: stubname can't be identical to a column name

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions