You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
16 households have a negative vehicle count on 1.58.0 (WAS missing-value code in the wealth imputation) #574
On the released 1.58.0 enhanced FRS (enhanced_frs_2024_25.h5, SHA-256 5ba399ea…60b6), 16 households store a negative num_vehicles. The lowest is −8.
Where it comes from
num_vehicles is imputed from the Wealth and Assets Survey by the wealth QRF. generate_was_table (wealth.py) maps vcarnr8 to num_vehicles without cleaning. Two of the 15,128 WAS training rows carry −8, which looks like a WAS missing-value code (not checked against the WAS codebook). A QRF predicts by sampling training values, so the code reaches the dataset.
The has-fuel model in consumption.py already clips the same column at zero for its own training (was["num_vehicles"].fillna(0).clip(lower=0)), so it is unaffected there. Anything that reads num_vehicles from the built dataset sees the negatives.
Other WAS training columns with negatives (num_bedrooms in 14,485 rows, is_renter in 11,548) look like the same missing-value codes and are worth checking at the same time; net_financial_wealth and wealth can be negative legitimately.
Ask
Treat WAS negative codes as missing in generate_was_table before training.
Add a built-dataset test that num_vehicles is non-negative.
Found while tracing #565 (#572), which is about VAT and does not touch this.
Activity