Repository navigation
[20.0][MIG] base - #6068
[20.0][MIG] base#6068hbrunn wants to merge 73 commits into
Conversation
If we don't include it, /jsonrpc route doesn't work.
If you have a module in previous versions that adds data on a model,
and such model is not loaded in the registry in current version because
the module is absent in it, you can't uninstall such module, getting
this error:
File "odoo/odoo/addons/base/models/ir_model.py", line 1945, in _module_data_uninstall
delete(self.env[model].browse(item[1] for item in items))
File "odoo/odoo/api.py", line 463, in __getitem__
return self.registry[model_name]._browse(self, (), ())
File "odoo/odoo/modules/registry.py", line 177, in __getitem__
return self.models[model_name]
KeyError: 'model'
With this patch, data cleanup of such model is skipped and there's no crash.
…ore ; add links to the new OpenUpgrade website
If corresponding field is None, we need to avoid the "AttributeError: 'NoneType' object has no attribute error.
…rg validate as it is not anymore in odoo 15
…e useless after the input argument validate has been removed
Because we captured the exception in _raise_view_error and archived that
view, so info is None, but it is called to info.get('select') in
NameManager.check, which will raise an exception AttributeError, so we
need to override to not raise an exception
|
/ocabot migration base |
FerDeNetlab
left a comment
There was a problem hiding this comment.
Tested on our 19.0 db (131 modules after the upgrade, with our module scripts on top): same result as with our own base scripts, apart from the line below. load_data of noupdate_changes.xml needs OCA/openupgradelib#468 on 20.0.
Drafted with AI assistance (Claude Opus 5.5), reviewed by me.
| WHERE | ||
| ir_model_fields.model_id=ir_model.id | ||
| AND | ||
| ir_model.state='manual' |
There was a problem hiding this comment.
This only covers fields of custom models. Manual fields on standard models lose their index: the analytic plan columns (account.analytic.line.x_plan2_id, project.project.x_plan2_id, ...) are index=True in our 19.0 db and end up with index=NULL. The database index itself is kept, only ir.model.fields is wrong. With this change all 4 come out as btree.
| WHERE | |
| ir_model_fields.model_id=ir_model.id | |
| AND | |
| ir_model.state='manual' | |
| WHERE | |
| ir_model_fields.model_id=ir_model.id | |
| AND | |
| (ir_model.state='manual' OR ir_model_fields.state='manual') |
There was a problem hiding this comment.
thanks, I realize it's nonsense to involve ir_model at all, here
Co-authored-by: FerDeNetlab <fer@netlab.mx>
Co-authored-by: FerDeNetlab <fer@netlab.mx>
when installing auto_install modules
|
The following commit authors need to sign the Contributor License Agreement: |
| model IN ('ir.rule', 'ir.model.access') | ||
| AND | ||
| module NOT IN ( | ||
| '__export__', '__custom__' |
There was a problem hiding this comment.
also add 'studio_customization' just in case
| env, | ||
| [], | ||
| ) | ||
| ir_access(env) |
There was a problem hiding this comment.
I would move this method to end-migration, just before ir_access_end(env).
| FROM ir_model_data | ||
| WHERE model='{model}' | ||
| AND module NOT IN ( | ||
| '__export__', '__custom__' |
There was a problem hiding this comment.
add 'studio_customization'
| for operation, field_name in operation_field_mapping.items(): | ||
| if not row[field_name]: | ||
| continue | ||
| group = ResGroups.browse(row["group_id"]) |
There was a problem hiding this comment.
group = ResGroups.browse(row["group_id"]) or env.ref("base.group_everyone")
In v19, model access without group applies to all users, but in v20 an ir.access without group is a restriction
| f""" | ||
| SELECT | ||
| perm_create, perm_read, perm_write, perm_unlink, domain_force, model_id, | ||
| group_id |
There was a problem hiding this comment.
group_id, {custom_record("ir_rule.id", "ir.rule")} AS is_custom
used below
| accesstuple2domains[key].add(row["domain_force"]) | ||
| if EMPTY_DOMAIN in accesstuple2domains[key]: | ||
| accesstuple2domains[key].remove(EMPTY_DOMAIN) | ||
|
|
There was a problem hiding this comment.
for key in accesstuples_manual:
domains = set()
for group in key.group.all_implied_ids | key.group:
domains |= standard_rule_domains.get((group,) + key[1:], set())
if domains:
accesstuple2domains[key].discard(EMPTY_DOMAIN)
accesstuple2domains[key] |= domainsa user of a group gets the rules of all the groups it implies
| # turn off validation as most models won't be available yet | ||
| _validate_fields = IrAccess.__class__._validate_fields | ||
| IrAccess.__class__._validate_fields = lambda *args: None | ||
| IrAccess.create( | ||
| [ | ||
| { | ||
| "name": "Migrated from v19", | ||
| "operation": operation, | ||
| "group_id": domaintuple.group.id, | ||
| "model_id": domaintuple.model.id, | ||
| "domain": domaintuple.domain | ||
| if domaintuple.domain != EMPTY_DOMAIN | ||
| else None, | ||
| } | ||
| for domaintuple, operation in domaintuple2operation.items() | ||
| ] | ||
| ) | ||
| IrAccess.__class__._validate_fields = _validate_fields |
There was a problem hiding this comment.
replace this full block (that begins in _validate_fields) by :
for domaintuple, operation in domaintuple2operation.items():
vals = {
"name": "Migrated from v19",
"operation": operation,
"group_id": domaintuple.group.id,
"model_id": domaintuple.model.id,
"domain": domaintuple.domain
if domaintuple.domain != EMPTY_DOMAIN
else None,
}
try:
with env.cr.savepoint():
IrAccess.create(vals)
except Exception as e:
_logger.warning(
"Creating inactive ir.access for %s: %s", domaintuple.model.model, e
)
IrAccess.create(dict(vals, active=False))(the constraint only checks active records)
| WHERE | ||
| model='ir.access' | ||
| AND module NOT IN ( | ||
| '__export__', '__custom__' |
There was a problem hiding this comment.
'studio_customization'
| IrAccess.create( | ||
| [ | ||
| { | ||
| "name": "Migrated from v19", |
There was a problem hiding this comment.
BTW, a problem with "Migrated from v19" arises, Every record will have that name. If in end-migration we ask for admins to review ACLs, then we should have to put better names.
maybe we could put "Migrated from v19: " + ", ".join(sorted(source_names)), or change source_names by model + ids.
| # turn off validation as most models won't be available yet | ||
| _validate_fields = IrAccess.__class__._validate_fields | ||
| IrAccess.__class__._validate_fields = lambda *args: None | ||
| IrAccess.create( |
There was a problem hiding this comment.
This create also is a problem. the _export_, _custom_, ..., will still point to ir.model.access and ir.rule ids. I think we should keep track of the source record ids for each created ir.access and then repoint those xmlids. In cases of merged ones, we could point them to the same record, right?
mauro2306
left a comment
There was a problem hiding this comment.
Functional test outside CI at 1faabaf, which includes #6046. A first run at 2d9eb6b on 2026-10-30 gave the same results.
Setup:
- Official
odoo:20Docker image and PostgreSQL 17. - Source: a fresh v19 database with the 88 modules of three production Community databases (account, account_peppol, l10n_be, stock_account, sale_stock, project…), plus
scripts/base/tests/data_base_migration.pyand some data of our own (bank account, Peppol endpoints, tracked changes, a posted invoice). - Same procedure as
test-migration.yml: modules not marked Done are set to uninstalled, then--load=base,web,openupgrade_framework --test-enable --test-tags openupgrade --update=base. - openupgradelib 3.13.7 with the change of OCA/openupgradelib#468 applied. Without it, the post-migration fails in
load_data(), so this needs an openupgradelib release that includes it.
Result:
- As is, the migration fails in
end-migration.pywithKeyError: 'discuss.channel'(see the line comment). - With that fixed, it completes and the 3 tests pass (
0 failed, 0 error(s) of 3 tests). Theres_partner_bankdata is kept:account_numberholds the formeracc_numbervalue. - 6
ir.accessrecords are created as "Migrated from v19", and 2 of them have nogroup_id(operationsrandd), which matches @MiquelRForgeFlow's remark about falling back tobase.group_everyone.
| """ | ||
| ) | ||
| custom_access_models = env.cr.fetchall() | ||
| admin_channel = env.ref("mail.channel_admin", raise_if_not_found=False) |
There was a problem hiding this comment.
raise_if_not_found=False only covers a missing xmlid. Here the xmlid mail.channel_admin exists, but mail isn't loaded, which is exactly the setup of test-migration.yml when the source database had mail installed. So env.ref() raises KeyError: 'discuss.channel', and the whole migration fails in end-migration. Still the case at 1faabaf.
| admin_channel = env.ref("mail.channel_admin", raise_if_not_found=False) | |
| admin_channel = "discuss.channel" in env and env.ref( | |
| "mail.channel_admin", raise_if_not_found=False | |
| ) |
carlosdauden
left a comment
There was a problem hiding this comment.
Reviewed at 1faabaf against origin/20.0, line by line against the base scripts we run on our 20.0 integration branch (19.0 -> 20.0 migration of the OCA test database with ~420 module scripts on top, 132 migration tests, Python 3.12 / PostgreSQL 16). Thanks for the work on the ir.access conversion, it is the hardest part of this module. Every point below was checked against the 19.0 and 20.0 code; happy to open a PR against your branch with our versions of the fixes.
Changes I would ask for:
-
res_device_log:user_id = -1violates the 19.0 foreign keyres_device_log_user_id_fkey(ON DELETE SET NULL). It is still there after the update: the 20.0 field is a plainInteger, and the schema update never drops a foreign key the model no longer declares. So the pre-migration aborts on any database where a user with device logs was deleted, and after the migration, deleting a user with logs fails (SET NULL against the new NOT NULL). Suggestion: drop the foreign key onres_device_log.user_id(found throughpg_constraint) and itsir_model_constraintrow, and delete the rows withuser_id IS NULLinstead of filling them (the log of a deleted user has no value). -
end-migration.py:env.ref("mail.channel_admin", raise_if_not_found=False)raisesKeyError: 'discuss.channel'when the source database hadmailinstalled, because the xmlid exists but the model is not loaded while base is updated (thetest-migration.ymlsetup). mauro2306 reproduced it; the guard"discuss.channel" in env and env.ref(...)from that comment fixes it. -
API keys: 20.0 only accepts a key for the scope it was generated for (
scope = %(scope)sin_check_apikey_credentials, 61c8515704ca), while 19.0 also acceptedscope IS NULLfor any scope, which is how every key generated from the user preferences is stored. After the migration all those keys stop working for RPC. The analysis does not show it becauseres_users_apikeysis_auto = False.UPDATE res_users_apikeys SET scope = 'rpc' WHERE scope IS NULL(the scope 20.0 gives to keys created from the same screen). -
res_partner_bank:- The update from
res_banksetscountry_id = res_bank.country, which wipes the country of every account whose bank has none (or has no bank). UseCOALESCE(res_bank.country, res_partner_bank.country_id). clearing_label_idis new, required, precompute and stored, but theclearing.labeldata is not loaded yet when the column is created, so it stays NULL on every existing account. It needs a fill like_compute_clearing_label_idonce base data is loaded (label of the account's country, elsebase.clearing_label_all); in SQL to avoid writing every account and running the 20.0 clearing number constraints on legacy data.
- The update from
-
res_country.zip_applicability: the post-migration loadsnoupdate_changes.xml(which sets the 20.0 values, includingnot_applicablefor countries such as ao, bz or hk) and then mapsNOT zip_requiredtooptional, overwriting those values. Map first, then apply the 20.0 value only to the countries whosezip_requiredis still the 19.0 data value, so user changes are kept. -
noupdate_changes.xmlhas entries that should not be applied as they are:main_partneris_company = Falseis a generator artefact:is_companybecame a stored compute in 20.0. Loading it turns the main company partner into a person. Please remove it.- UZS
symbol(96cfcaac7429) andbase.default_max_email_sizeare overwritten unconditionally. Update them only where the 19.0 value is unchanged (symbol = 'лв', value10), so user changes survive. - Related:
ir.mail_server.max_email_sizeis removed (left unannotated in the work file) and 20.0 only readsbase.default_max_email_size. When a 19.0 server had its own size, the parameter should get the smallest size an active server accepted, so no mail becomes bigger than what a server took before.
-
ir.accessconversion: MiquelRForgeFlow's comments of 2026-11-03 are still open at 1faabaf:- add
studio_customizationto the custom xmlid modules (3 places); - fall back to
base.group_everyonefor a model access without group (in 19.0 it grants everyone, in 20.0 anir.accesswithout group is a restriction; mauro2306 got 2 such records); - add the domains of the standard rules of the group and its implied groups to the migrated access of manual models;
- a savepoint per
create()(inactive record and a warning on failure) instead of disabling_validate_fields, which can leave invalid domains active; - more useful names than "Migrated from v19" for every record, since the end-migration asks admins to review them;
- repoint the
__export__/__custom__xmlids of the sourceir.model.access/ir.rulerows to the created records; - move
ir_access()to end-migration, when every model, the moduleir.accessdata and the group implications are loaded.
- add
-
CI:
- "test" fails before running anything: the workflow uses Python 3.10, so odoo-old cannot import
babel. 20.0 needs Python 3.12 and PostgreSQL 16 in.github/workflows/test-migration.yml. - "test with Odoo" and "test with OCB" fail because the branch carries an old copy of the #6046 framework test, without
mute_logger("odoo.modules.module"), which logs "manifest not found" foropenupgrade_framework_test_auto_install. Rebasing on the current #6046 head fixes it. - pre-commit needs the pylint-odoo bump to v10.0.12 (OCA/pylint-odoo#590), as in #6046.
- CLA: same history commits as #6046.
- "test" fails before running anything: the workflow uses Python 3.10, so odoo-old cannot import
-
Depends on OCA/openupgradelib#468 (approved, not merged) for
load_data, as FerDeNetlab and mauro2306 noted.
Not blocking, can be follow-ups:
-
States (
res.country.state.csvin 20.0). Besides Odisha, Ladakh and the 2 Egypt merges:- India:
state_in_dd(Daman and Diu) is gone, merged intostate_in_dn. - Lebanon: the
l10n_lb_accountstates move to base with ISO codes (state_lb_01->base.state_lb_ak, ...), and their Arabic duplicates (state_lb_02_ar, ...) are removed. - Bahrain and Oman: the
l10n_bh/l10n_omstates move to base with ISO codes (l10n_bh.state_bh_1->base.state_bh_13,l10n_om.state_om_01->base.state_om_mu, ...). - Saudi Arabia: the 93 city "states" are replaced by the 13 regions; at least the cities that name a region or its capital can be merged (
state_sa_70Riyadh ->state_sa_001, ...).
Without renames or merges, base creates duplicates, and
_process_endcannot delete the old states still used by partners (ondelete='restrict'). The Egypt merge works, but it runs in pre-migration under theversion_info = (19,)hack; done in post-migration withmerge_records(method="sql")it needs no hack. We have the full mapping and can contribute it. - India:
-
Timezones: only 3 values are mapped. The 20.0 selection comes from
zoneinfofiltered by babel (odoo/tools/date_utils.pyall_timezones), so depending on the server tz database every backward link (US/Eastern, Asia/Calcutta, CET, ...) can disappear, andtz_offsetcallsZoneInfo(tz)unguarded. A mapping table built once in base from the tz database links (only links missing from the selection whose target exists) can be reused by the modules owning other tz columns (resource_calendar,resource_resource,calendar_recurrence,mail_guest,event_event,hr_version, ...). The KGS symbol (c78dead6b43e, also in 19.0 but noupdate) can get the same "only if unchanged" update as UZS. -
t-escis removed from server QWeb in 20.0 and renders nothing. Custom views, edited copies of standard views and mail templates keep it after the migration. At-esc->t-outrewrite in base end-migration (ir_ui_view.arch_db/arch_prev,mail_template.body_html/subject) covers all modules at once. -
company_registry->additional_identifiers: skip empty strings, do not overwrite an existing key, and use the enterprise number key of the partner's country when there is one,OTHERotherwise. -
group_user_regularis only added to direct members ofbase.group_user, so users that get it through an implied group are missed. Low impact, as 19.0 stores direct groups only, butNOT shareonres_usersis simpler and covers everything. -
Nit: the
ir_exports_lineid -> sequence copy is not needed (_order = 'sequence, id'with the default sequence keeps the order). Harmless.
depends on:
I'll rebase after those are merged and the
PR has been merged and generated the first updated analysis