ADR Suggestion Removal of weakref dicts

[rozyczko]

Removing the Weakref-ed GlobalObject

What it actually does

The singleton GlobalObject + Map serves three purposes:

1. Unique name generation : generate_unique_name() checks existing names in the map to produce ClassName_0, ClassName_1, etc.

2. Object graph tracking : Map
   a. stores parent -> child edges
   b. supports path-finding
   c. supports connectivity checks
   d. supports type classification (argument, created, returned).

3. Undo/redo : GlobalObject.stack holds an UndoStack that records property changes for revert.

The legitimate use cases are:

Need	How GlobalObject serves it
Unique naming	Checks existing names in WeakValueDictionary
Object lookup by name	get_item_by_key in WeakValueDictionary
Undo/redo (needs redesign)	UndoStack referenced via global_object.stack
Script recording (needs redesign)	ScriptManager logs operations for reproducibility

---

Good reasons to get rid of it

1. Constant concurrency bugs!

We've suffered from RuntimeError: dictionary changed size during iteration.
WeakValueDictionary entries vanish when GC runs — even mid-iteration. The retry-loop fix is a band-aid, not a cure; it hides races rather than eliminating them.

2. Hidden global mutable state

Every NewBase.__init__ mutates a process-wide singleton. This makes the code hard to test (you need _clear() between tests), hard to reason about, and impossible to run two independent models in the same process without name collisions. This will be especially important if we are to use multiprocessing.

3. Tight coupling

Every object knows about global_object at construction time. This pulls in Map, UndoStack, ScriptManager, and Logger as transitive dependencies for even the simplest parameter object.

4. Weakrefs + finalizers are fragile

weakref.finalize callbacks run at unpredictable times (during GC, possibly on a different thread in CPython). The prune_type_dict finalizer modifying __type_dict while other code iterates it is a textbook example of this fragility.

5. The graph features are barely used

find_path, find_all_paths, is_connected, reverse_route are generic graph algorithms but typical usage only needs parent - child edges for serialization. Maintaining a full adjacency structure for that is over-engineered.

6. Potential memory/performance overhead

Every object creation does a dict insert + a weakref.finalize registration. For models with thousands of parameters this can adds up.

7. Name-generation is the only hard requirement

And it doesn't need a global graph to work.

---

Simpler alternatives

1. Class-level counter for unique names without global map

from typing import ClassVar

class NewBase:
    _name_counters: ClassVar[dict[str, int]] = {}

    def __init__(self, unique_name=None, ...):
        if unique_name is None:
            prefix = self.__class__.__name__
            count = NewBase._name_counters.get(prefix, 0)
            NewBase._name_counters[prefix] = count + 1
            unique_name = f"{prefix}_{count}"
        self._unique_name = unique_name

A single class-variable dict[str, int] wjcich replaces the entire Map for name generation. Of course all the uniqeness checks need to be added.

2. Explicit parent-child ownership instead of a global graph

Instead of a process-wide Map tracking edges, let containers (EasyList, model classes) hold direct references to their children. Serialization already walks the object tree via to_dict() so it doesn't need the global graph.

class EasyList(NewBase, MutableSequence[ProtectedType_]):
    """Children are stored directly in self._data - no global map needed."""

    def __init__(self, *args, **kwargs):
        super().__init__(**kwargs)
        self._data: list[ProtectedType_] = []
        # Parent - child relationship is implicit: self owns self._data items.
        # No need to register edges in a global Map.

Maybe we can drop path-finding entirely?

If we actually need "which objects depend on parameter X?" for fitting, we cam model that explicitly (e.g., each Parameter stores a back-reference to its parent). The generic graph algorithms in Map (find_path, is_connected, etc.) are unlikely to be needed if ownership is explicit.

class Parameter(NewBase):
    """A parameter that knows its parent explicitly."""

    def __init__(self, value, parent=None, **kwargs):
        super().__init__(**kwargs)
        self._value = value
        self._parent = parent  # Direct back-reference, no global graph needed

---

Current code that would need to change

NewBase.__init__ (registers with global map)

# Current (src/easyscience/base_classes/new_base.py)
class NewBase:
    def __init__(self, unique_name=None, display_name=None):
        self._global_object = global_object
        if unique_name is None:
            unique_name = self._global_object.generate_unique_name(self.__class__.__name__)
            self._default_unique_name = True
        else:
            self._default_unique_name = False
        self._unique_name = unique_name
        self._global_object.map.add_vertex(self, obj_type='created')  # <-- weakref registered here

Map.add_vertex (creates weakref + finalizer)

# Current (src/easyscience/global_object/map.py)
def add_vertex(self, obj, obj_type=None):
    name = obj.unique_name
    if name in self._store:
        raise ValueError(f'Object name {name} already exists in the graph.')
    if name in self.__type_dict:
        del self.__type_dict[name]

    self._store[name] = obj  # <-- WeakValueDictionary entry

    entry_list = _EntryList()
    entry_list.finalizer = weakref.finalize(obj, self.prune_type_dict, name)  # <-- GC finalizer
    entry_list.type = obj_type
    self.__type_dict[name] = entry_list

GlobalObject.generate_unique_name (iterates weakref dict)

# Current (src/easyscience/global_object/global_object.py)
def generate_unique_name(self, name_prefix: str) -> str:
    names_with_prefix = [name for name in self.map.vertices() if name.startswith(name_prefix + '_')]
    # self.map.vertices() calls list(self._store) on a WeakValueDictionary
    # -- entries can disappear mid-iteration if GC runs

---

Migration summary

Component	Current	Replacement
Unique naming	GlobalObject.generate_unique_name -> Map.vertices() -> WeakValueDictionary	Class-level counter (dict[str, int])
Object tracking	Map with weakrefs, finalizers, graph algorithms	Explicit parent->child references on containers
Object lookup by name	Map.get_item_by_key	Walk the object tree

---

Issues to solve

1. What to do with Undo/Redo??
2. How to solve script recording? Do we even need this?

[rozyczko]

I don't think it's that easy... we will probably need some sort of object/dict for tracking names and vertices (or children-parents)