Skip to content

Deadlock possibility in linear_qubit_operator.py #1405

Description

@mhucka

Gemini Code Assist reports a problem with the multiprocessing code in linear_qubit_operator.py:

  1. Deadlock in Multi-Process Path (Lines 192-198):
    Currently, pool.join() is called before consuming the vecs iterator returned by pool.imap_unordered. Since imap_unordered yields results lazily, the worker processes write their results to an internal OS pipe. If the state vector x is large (which is very common in quantum simulations), the pipe buffer (typically 64KB on Linux) will fill up, causing the worker processes to block on write(). Meanwhile, the main process is blocked on pool.join(), resulting in a permanent deadlock.

    Fix: You must consume the iterator completely (e.g., by performing the reduction) before calling pool.join():

    pool = self.options.get_pool(len(self.linear_operators))
    vecs = pool.imap_unordered(
        apply_operator, [(operator, x) for operator in self.linear_operators]
    )
    pool.close()
    # Consume the iterator BEFORE joining the pool to avoid deadlocks
    result = functools.reduce(numpy.add, vecs)
    pool.join()
    return result
  2. Performance/Memory Overhead in Single-Process Path (Lines 189-190):
    Using functools.reduce(numpy.add, ...) creates $N-1$ intermediate numpy arrays, where $N$ is the number of operators. For large state vectors, this causes significant memory churn and garbage collection overhead.

    Fix: Initialize a single zero array and accumulate the results in-place using += as suggested below.

        if self.options.processes <= 1:
            res = numpy.zeros(x.shape, dtype=complex)
            for operator in self.linear_operators:
                res += operator * x
            return res

Originally posted by @gemini-code-assist[bot] in #1404 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions