[Bug]: PickleCoder/_MemoizingPickleCoder.as_deterministic_coder() raises TypeError

[AmirF194]

What happened?

PickleCoder.as_deterministic_coder() and _MemoizingPickleCoder.as_deterministic_coder() raise TypeError on every call instead of returning a working deterministic coder.

Both methods do:

def as_deterministic_coder(self, step_label, error_message=None):
  return FastPrimitivesCoder(self, requires_deterministic=step_label)

FastPrimitivesCoder.__init__(self, fallback_coder=PickleCoder()) has never accepted a requires_deterministic argument, so this raises:

TypeError: FastPrimitivesCoder.__init__() got an unexpected keyword argument 'requires_deterministic'

Repro on current master:

from apache_beam.coders.coders import PickleCoder
PickleCoder().as_deterministic_coder('label')
# TypeError: FastPrimitivesCoder.__init__() got an unexpected keyword argument 'requires_deterministic'

Root cause: commit 9be70c9 (2021-03-08) changed PickleCoder.as_deterministic_coder from the working DeterministicFastPrimitivesCoder(self, step_label) to the broken FastPrimitivesCoder(self, requires_deterministic=...) in the same diff that added a requires_deterministic kwarg to a different class, coder_impl.FastPrimitivesCoderImpl (the Cython impl, not the coders.FastPrimitivesCoder wrapper referenced here). _MemoizingPickleCoder (added later) copied the same already-broken line.

Any caller that resolves to one of these coders and asks for a deterministic version hits this, for example GroupByEncryptedKey.expand() in apache_beam/transforms/util.py, which wraps the call in except ValueError and does not catch the TypeError, so it crashes instead of logging the intended non-deterministic-key warning.

This is a narrower, separate defect from #28558's own headline claim (which is about pickle_library being ignored, and was addressed by @tvalentyn as working as intended for PickleCoder). @tvalentyn identified this specific as_deterministic_coder bug in a comment on that thread in 2023 and invited a PR for it, but #28558 itself is now closed and its title does not match this fix, so filing it here as its own issue.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK

[AmirF194]

.take-issue