Skip to content

[20.0][MIG] base - #6068

Open
hbrunn wants to merge 73 commits into
OCA:20.0from
hbrunn:20.0-base
Open

hbrunn wants to merge 73 commits into
OCA:20.0from
hbrunn:20.0-base

Conversation

@hbrunn

@hbrunn hbrunn commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

depends on:

I'll rebase after those are merged and the

PR has been merged and generated the first updated analysis

legalsylvain and others added 30 commits September 29, 2026 16:02
If we don't include it, /jsonrpc route doesn't work.
If you have a module in previous versions that adds data on a model,
and such model is not loaded in the registry in current version because
the module is absent in it, you can't uninstall such module, getting
this error:

  File "odoo/odoo/addons/base/models/ir_model.py", line 1945, in _module_data_uninstall
    delete(self.env[model].browse(item[1] for item in items))
  File "odoo/odoo/api.py", line 463, in __getitem__
    return self.registry[model_name]._browse(self, (), ())
  File "odoo/odoo/modules/registry.py", line 177, in __getitem__
    return self.models[model_name]
KeyError: 'model'

With this patch, data cleanup of such model is skipped and there's no crash.
…ore ; add links to the new OpenUpgrade website
If corresponding field is None, we need to avoid the "AttributeError: 'NoneType' object has no attribute error.
…e useless after the input argument validate has been removed
Because we captured the exception in _raise_view_error and archived that
view, so info is None, but it is called to info.get('select') in
NameManager.check, which will raise an exception AttributeError, so we
need to override to not raise an exception
@OCA-git-bot OCA-git-bot added mod:openupgrade_scripts Module openupgrade_scripts mod:openupgrade_framework Module openupgrade_framework series:20.0 labels Oct 27, 2026
@hbrunn

hbrunn commented Oct 27, 2026

Copy link
Copy Markdown
Member Author

/ocabot migration base

@OCA-git-bot OCA-git-bot added this to the 20.0 milestone Oct 27, 2026
@OCA-git-bot OCA-git-bot mentioned this pull request Oct 27, 2026
1 of 6 tasks

@FerDeNetlab FerDeNetlab left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on our 19.0 db (131 modules after the upgrade, with our module scripts on top): same result as with our own base scripts, apart from the line below. load_data of noupdate_changes.xml needs OCA/openupgradelib#468 on 20.0.

Drafted with AI assistance (Claude Opus 5.5), reviewed by me.

Comment on lines +190 to +193
WHERE
ir_model_fields.model_id=ir_model.id
AND
ir_model.state='manual'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only covers fields of custom models. Manual fields on standard models lose their index: the analytic plan columns (account.analytic.line.x_plan2_id, project.project.x_plan2_id, ...) are index=True in our 19.0 db and end up with index=NULL. The database index itself is kept, only ir.model.fields is wrong. With this change all 4 come out as btree.

Suggested change
WHERE
ir_model_fields.model_id=ir_model.id
AND
ir_model.state='manual'
WHERE
ir_model_fields.model_id=ir_model.id
AND
(ir_model.state='manual' OR ir_model_fields.state='manual')

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, I realize it's nonsense to involve ir_model at all, here

@oca-cla-bot

oca-cla-bot Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

The following commit authors need to sign the Contributor License Agreement:

CLA not signed

model IN ('ir.rule', 'ir.model.access')
AND
module NOT IN (
'__export__', '__custom__'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also add 'studio_customization' just in case

env,
[],
)
ir_access(env)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would move this method to end-migration, just before ir_access_end(env).

FROM ir_model_data
WHERE model='{model}'
AND module NOT IN (
'__export__', '__custom__'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add 'studio_customization'

for operation, field_name in operation_field_mapping.items():
if not row[field_name]:
continue
group = ResGroups.browse(row["group_id"])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

group = ResGroups.browse(row["group_id"]) or env.ref("base.group_everyone")

In v19, model access without group applies to all users, but in v20 an ir.access without group is a restriction

f"""
SELECT
perm_create, perm_read, perm_write, perm_unlink, domain_force, model_id,
group_id

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

group_id, {custom_record("ir_rule.id", "ir.rule")} AS is_custom

used below

accesstuple2domains[key].add(row["domain_force"])
if EMPTY_DOMAIN in accesstuple2domains[key]:
accesstuple2domains[key].remove(EMPTY_DOMAIN)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

    for key in accesstuples_manual:
        domains = set()
        for group in key.group.all_implied_ids | key.group:
            domains |= standard_rule_domains.get((group,) + key[1:], set())
        if domains:
            accesstuple2domains[key].discard(EMPTY_DOMAIN)
            accesstuple2domains[key] |= domains

a user of a group gets the rules of all the groups it implies

Comment on lines +157 to +174
# turn off validation as most models won't be available yet
_validate_fields = IrAccess.__class__._validate_fields
IrAccess.__class__._validate_fields = lambda *args: None
IrAccess.create(
[
{
"name": "Migrated from v19",
"operation": operation,
"group_id": domaintuple.group.id,
"model_id": domaintuple.model.id,
"domain": domaintuple.domain
if domaintuple.domain != EMPTY_DOMAIN
else None,
}
for domaintuple, operation in domaintuple2operation.items()
]
)
IrAccess.__class__._validate_fields = _validate_fields

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace this full block (that begins in _validate_fields) by :

    for domaintuple, operation in domaintuple2operation.items():
        vals = {
            "name": "Migrated from v19",
            "operation": operation,
            "group_id": domaintuple.group.id,
            "model_id": domaintuple.model.id,
            "domain": domaintuple.domain
            if domaintuple.domain != EMPTY_DOMAIN
            else None,
        }
        try:
            with env.cr.savepoint():
                IrAccess.create(vals)
        except Exception as e:
            _logger.warning(
                "Creating inactive ir.access for %s: %s", domaintuple.model.model, e
            )
            IrAccess.create(dict(vals, active=False))

(the constraint only checks active records)

WHERE
model='ir.access'
AND module NOT IN (
'__export__', '__custom__'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'studio_customization'

IrAccess.create(
[
{
"name": "Migrated from v19",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, a problem with "Migrated from v19" arises, Every record will have that name. If in end-migration we ask for admins to review ACLs, then we should have to put better names.

maybe we could put "Migrated from v19: " + ", ".join(sorted(source_names)), or change source_names by model + ids.

# turn off validation as most models won't be available yet
_validate_fields = IrAccess.__class__._validate_fields
IrAccess.__class__._validate_fields = lambda *args: None
IrAccess.create(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This create also is a problem. the _export_, _custom_, ..., will still point to ir.model.access and ir.rule ids. I think we should keep track of the source record ids for each created ir.access and then repoint those xmlids. In cases of merged ones, we could point them to the same record, right?

@mauro2306 mauro2306 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Functional test outside CI at 1faabaf, which includes #6046. A first run at 2d9eb6b on 2026-10-30 gave the same results.

Setup:

  • Official odoo:20 Docker image and PostgreSQL 17.
  • Source: a fresh v19 database with the 88 modules of three production Community databases (account, account_peppol, l10n_be, stock_account, sale_stock, project…), plus scripts/base/tests/data_base_migration.py and some data of our own (bank account, Peppol endpoints, tracked changes, a posted invoice).
  • Same procedure as test-migration.yml: modules not marked Done are set to uninstalled, then --load=base,web,openupgrade_framework --test-enable --test-tags openupgrade --update=base.
  • openupgradelib 3.13.7 with the change of OCA/openupgradelib#468 applied. Without it, the post-migration fails in load_data(), so this needs an openupgradelib release that includes it.

Result:

  • As is, the migration fails in end-migration.py with KeyError: 'discuss.channel' (see the line comment).
  • With that fixed, it completes and the 3 tests pass (0 failed, 0 error(s) of 3 tests). The res_partner_bank data is kept: account_number holds the former acc_number value.
  • 6 ir.access records are created as "Migrated from v19", and 2 of them have no group_id (operations r and d), which matches @MiquelRForgeFlow's remark about falling back to base.group_everyone.

"""
)
custom_access_models = env.cr.fetchall()
admin_channel = env.ref("mail.channel_admin", raise_if_not_found=False)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

raise_if_not_found=False only covers a missing xmlid. Here the xmlid mail.channel_admin exists, but mail isn't loaded, which is exactly the setup of test-migration.yml when the source database had mail installed. So env.ref() raises KeyError: 'discuss.channel', and the whole migration fails in end-migration. Still the case at 1faabaf.

Suggested change
admin_channel = env.ref("mail.channel_admin", raise_if_not_found=False)
admin_channel = "discuss.channel" in env and env.ref(
"mail.channel_admin", raise_if_not_found=False
)

@carlosdauden carlosdauden left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 1faabaf against origin/20.0, line by line against the base scripts we run on our 20.0 integration branch (19.0 -> 20.0 migration of the OCA test database with ~420 module scripts on top, 132 migration tests, Python 3.12 / PostgreSQL 16). Thanks for the work on the ir.access conversion, it is the hardest part of this module. Every point below was checked against the 19.0 and 20.0 code; happy to open a PR against your branch with our versions of the fixes.

Changes I would ask for:

  1. res_device_log: user_id = -1 violates the 19.0 foreign key res_device_log_user_id_fkey (ON DELETE SET NULL). It is still there after the update: the 20.0 field is a plain Integer, and the schema update never drops a foreign key the model no longer declares. So the pre-migration aborts on any database where a user with device logs was deleted, and after the migration, deleting a user with logs fails (SET NULL against the new NOT NULL). Suggestion: drop the foreign key on res_device_log.user_id (found through pg_constraint) and its ir_model_constraint row, and delete the rows with user_id IS NULL instead of filling them (the log of a deleted user has no value).

  2. end-migration.py: env.ref("mail.channel_admin", raise_if_not_found=False) raises KeyError: 'discuss.channel' when the source database had mail installed, because the xmlid exists but the model is not loaded while base is updated (the test-migration.yml setup). mauro2306 reproduced it; the guard "discuss.channel" in env and env.ref(...) from that comment fixes it.

  3. API keys: 20.0 only accepts a key for the scope it was generated for (scope = %(scope)s in _check_apikey_credentials, 61c8515704ca), while 19.0 also accepted scope IS NULL for any scope, which is how every key generated from the user preferences is stored. After the migration all those keys stop working for RPC. The analysis does not show it because res_users_apikeys is _auto = False. UPDATE res_users_apikeys SET scope = 'rpc' WHERE scope IS NULL (the scope 20.0 gives to keys created from the same screen).

  4. res_partner_bank:

    • The update from res_bank sets country_id = res_bank.country, which wipes the country of every account whose bank has none (or has no bank). Use COALESCE(res_bank.country, res_partner_bank.country_id).
    • clearing_label_id is new, required, precompute and stored, but the clearing.label data is not loaded yet when the column is created, so it stays NULL on every existing account. It needs a fill like _compute_clearing_label_id once base data is loaded (label of the account's country, else base.clearing_label_all); in SQL to avoid writing every account and running the 20.0 clearing number constraints on legacy data.
  5. res_country.zip_applicability: the post-migration loads noupdate_changes.xml (which sets the 20.0 values, including not_applicable for countries such as ao, bz or hk) and then maps NOT zip_required to optional, overwriting those values. Map first, then apply the 20.0 value only to the countries whose zip_required is still the 19.0 data value, so user changes are kept.

  6. noupdate_changes.xml has entries that should not be applied as they are:

    • main_partner is_company = False is a generator artefact: is_company became a stored compute in 20.0. Loading it turns the main company partner into a person. Please remove it.
    • UZS symbol (96cfcaac7429) and base.default_max_email_size are overwritten unconditionally. Update them only where the 19.0 value is unchanged (symbol = 'лв', value 10), so user changes survive.
    • Related: ir.mail_server.max_email_size is removed (left unannotated in the work file) and 20.0 only reads base.default_max_email_size. When a 19.0 server had its own size, the parameter should get the smallest size an active server accepted, so no mail becomes bigger than what a server took before.
  7. ir.access conversion: MiquelRForgeFlow's comments of 2026-11-03 are still open at 1faabaf:

    • add studio_customization to the custom xmlid modules (3 places);
    • fall back to base.group_everyone for a model access without group (in 19.0 it grants everyone, in 20.0 an ir.access without group is a restriction; mauro2306 got 2 such records);
    • add the domains of the standard rules of the group and its implied groups to the migrated access of manual models;
    • a savepoint per create() (inactive record and a warning on failure) instead of disabling _validate_fields, which can leave invalid domains active;
    • more useful names than "Migrated from v19" for every record, since the end-migration asks admins to review them;
    • repoint the __export__/__custom__ xmlids of the source ir.model.access/ir.rule rows to the created records;
    • move ir_access() to end-migration, when every model, the module ir.access data and the group implications are loaded.
  8. CI:

    • "test" fails before running anything: the workflow uses Python 3.10, so odoo-old cannot import babel. 20.0 needs Python 3.12 and PostgreSQL 16 in .github/workflows/test-migration.yml.
    • "test with Odoo" and "test with OCB" fail because the branch carries an old copy of the #6046 framework test, without mute_logger("odoo.modules.module"), which logs "manifest not found" for openupgrade_framework_test_auto_install. Rebasing on the current #6046 head fixes it.
    • pre-commit needs the pylint-odoo bump to v10.0.12 (OCA/pylint-odoo#590), as in #6046.
    • CLA: same history commits as #6046.
  9. Depends on OCA/openupgradelib#468 (approved, not merged) for load_data, as FerDeNetlab and mauro2306 noted.

Not blocking, can be follow-ups:

  1. States (res.country.state.csv in 20.0). Besides Odisha, Ladakh and the 2 Egypt merges:

    • India: state_in_dd (Daman and Diu) is gone, merged into state_in_dn.
    • Lebanon: the l10n_lb_account states move to base with ISO codes (state_lb_01 -> base.state_lb_ak, ...), and their Arabic duplicates (state_lb_02_ar, ...) are removed.
    • Bahrain and Oman: the l10n_bh/l10n_om states move to base with ISO codes (l10n_bh.state_bh_1 -> base.state_bh_13, l10n_om.state_om_01 -> base.state_om_mu, ...).
    • Saudi Arabia: the 93 city "states" are replaced by the 13 regions; at least the cities that name a region or its capital can be merged (state_sa_70 Riyadh -> state_sa_001, ...).

    Without renames or merges, base creates duplicates, and _process_end cannot delete the old states still used by partners (ondelete='restrict'). The Egypt merge works, but it runs in pre-migration under the version_info = (19,) hack; done in post-migration with merge_records(method="sql") it needs no hack. We have the full mapping and can contribute it.

  2. Timezones: only 3 values are mapped. The 20.0 selection comes from zoneinfo filtered by babel (odoo/tools/date_utils.py all_timezones), so depending on the server tz database every backward link (US/Eastern, Asia/Calcutta, CET, ...) can disappear, and tz_offset calls ZoneInfo(tz) unguarded. A mapping table built once in base from the tz database links (only links missing from the selection whose target exists) can be reused by the modules owning other tz columns (resource_calendar, resource_resource, calendar_recurrence, mail_guest, event_event, hr_version, ...). The KGS symbol (c78dead6b43e, also in 19.0 but noupdate) can get the same "only if unchanged" update as UZS.

  3. t-esc is removed from server QWeb in 20.0 and renders nothing. Custom views, edited copies of standard views and mail templates keep it after the migration. A t-esc -> t-out rewrite in base end-migration (ir_ui_view.arch_db/arch_prev, mail_template.body_html/subject) covers all modules at once.

  4. company_registry -> additional_identifiers: skip empty strings, do not overwrite an existing key, and use the enterprise number key of the partner's country when there is one, OTHER otherwise.

  5. group_user_regular is only added to direct members of base.group_user, so users that get it through an implied group are missed. Low impact, as 19.0 stores direct groups only, but NOT share on res_users is simpler and covers everything.

  6. Nit: the ir_exports_line id -> sequence copy is not needed (_order = 'sequence, id' with the default sequence keeps the order). Harmless.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

mod:openupgrade_framework Module openupgrade_framework mod:openupgrade_scripts Module openupgrade_scripts series:20.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.