feat: implement advanced state API for Go and Python#214
feat: implement advanced state API for Go and Python#214RobertIndie merged 23 commits intoFunctionStream:mainfrom
Conversation
- Go SDK: add state package with codec, keyed state, and structures - codec: bool/bytes/float/int/string/uint/json and ordered types - keyed: Value, List, Map, Aggregating, Reducing, PriorityQueue keyed states - structures: underlying implementations for each state type - Python API: refactor store into codec, keyed, structures submodules - keyed: KeyedValueState, KeyedListState, KeyedMapState, etc. - structures: state structures aligned with Go - Runtime: update fs_context to work with new state API Made-with: Cursor
There was a problem hiding this comment.
Pull request overview
Introduces an “advanced state” layer for both Go and Python SDKs: typed state abstractions (Value/List/Map/PQ/Aggregating/Reducing) built on top of the existing KV store APIs, plus default/built-in codecs and keyed-state factories. Updates runtime context wiring and expands the documentation to include dedicated “Advanced State API” guides.
Changes:
- Go SDK: add
state/codec,state/structures, andstate/keyedpackages implementing typed state + default codecs. - Python API: add
fs_api.store.codec/structures/keyedmodules, re-export them fromfs_api.store/fs_api, and extend theContextinterface + runtimeWitContext. - Docs/packaging: add advanced state API docs (Go/Python), fix guide links/paths, and update Python packaging discovery.
Reviewed changes
Copilot reviewed 71 out of 71 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| python/functionstream-runtime/src/fs_runtime/store/fs_context.py | Expose typed state creation helpers on runtime Context |
| python/functionstream-api/src/fs_api/store/structures/value_state.py | New typed ValueState wrapper |
| python/functionstream-api/src/fs_api/store/structures/reducing_state.py | New typed ReducingState wrapper |
| python/functionstream-api/src/fs_api/store/structures/priority_queue_state.py | New typed PriorityQueueState wrapper |
| python/functionstream-api/src/fs_api/store/structures/map_state.py | New typed MapState wrapper + helpers |
| python/functionstream-api/src/fs_api/store/structures/list_state.py | New typed ListState wrapper |
| python/functionstream-api/src/fs_api/store/structures/aggregating_state.py | New typed AggregatingState wrapper |
| python/functionstream-api/src/fs_api/store/structures/init.py | Export structures API surface |
| python/functionstream-api/src/fs_api/store/keyed/keyed_value_state.py | New keyed ValueState factory/state |
| python/functionstream-api/src/fs_api/store/keyed/keyed_reducing_state.py | New keyed ReducingState factory/state |
| python/functionstream-api/src/fs_api/store/keyed/keyed_priority_queue_state.py | New keyed PriorityQueueState factory/state |
| python/functionstream-api/src/fs_api/store/keyed/keyed_map_state.py | New keyed MapState factory/state |
| python/functionstream-api/src/fs_api/store/keyed/keyed_list_state.py | New keyed ListState factory/state |
| python/functionstream-api/src/fs_api/store/keyed/keyed_aggregating_state.py | New keyed AggregatingState factory/state |
| python/functionstream-api/src/fs_api/store/keyed/_keyed_common.py | Shared keyed helpers (ordered codec enforcement) |
| python/functionstream-api/src/fs_api/store/keyed/init.py | Export keyed API surface |
| python/functionstream-api/src/fs_api/store/codec/base.py | Base Codec interface for Python |
| python/functionstream-api/src/fs_api/store/codec/bool_codec.py | Ordered bool codec |
| python/functionstream-api/src/fs_api/store/codec/bytes_codec.py | Ordered bytes codec |
| python/functionstream-api/src/fs_api/store/codec/string_codec.py | Ordered string codec |
| python/functionstream-api/src/fs_api/store/codec/int_codec.py | Ordered int codec implementation |
| python/functionstream-api/src/fs_api/store/codec/float_codec.py | Ordered float codec implementation |
| python/functionstream-api/src/fs_api/store/codec/json_codec.py | JSON codec implementation |
| python/functionstream-api/src/fs_api/store/codec/pickle_codec.py | Pickle codec implementation |
| python/functionstream-api/src/fs_api/store/codec/default_codec.py | default_codec_for selection logic |
| python/functionstream-api/src/fs_api/store/codec/init.py | Export codec API surface |
| python/functionstream-api/src/fs_api/store/init.py | Re-export codec/structures/keyed from store package |
| python/functionstream-api/src/fs_api/context.py | Expand Context interface with typed state helpers |
| python/functionstream-api/src/fs_api/init.py | Re-export advanced state API from top-level package |
| python/functionstream-api/pyproject.toml | Switch to setuptools package auto-discovery |
| go-sdk/state/common/common.go | Shared helpers for state packages |
| go-sdk/state/codec/interface.go | Go Codec interface + fixed-size helper |
| go-sdk/state/codec/default_codec.go | Go DefaultCodecFor implementation |
| go-sdk/state/codec/bool_codec.go | Go ordered bool codec |
| go-sdk/state/codec/string_codec.go | Go ordered string codec |
| go-sdk/state/codec/json_codec.go | Go JSON codec |
| go-sdk/state/codec/int_codec.go | Go int codec (delegates to int64) |
| go-sdk/state/codec/int8_codec.go | Go ordered int8 codec |
| go-sdk/state/codec/int16_codec.go | Go ordered int16 codec |
| go-sdk/state/codec/int32_codec.go | Go ordered int32 codec |
| go-sdk/state/codec/int64_codec.go | Go ordered int64 codec |
| go-sdk/state/codec/uint_codec.go | Go uint codec (delegates to uint64) |
| go-sdk/state/codec/uint8_codec.go | Go ordered uint8 codec |
| go-sdk/state/codec/uint16_codec.go | Go ordered uint16 codec |
| go-sdk/state/codec/uint32_codec.go | Go ordered uint32 codec |
| go-sdk/state/codec/uint64_codec.go | Go ordered uint64 codec |
| go-sdk/state/codec/float32_codec.go | Go ordered float32 codec |
| go-sdk/state/codec/float64_codec.go | Go ordered float64 codec |
| go-sdk/state/structures/value.go | Go ValueState implementation |
| go-sdk/state/structures/list.go | Go ListState implementation |
| go-sdk/state/structures/map.go | Go MapState implementation |
| go-sdk/state/structures/priority_queue.go | Go PriorityQueueState implementation |
| go-sdk/state/structures/aggregating.go | Go AggregatingState implementation |
| go-sdk/state/structures/reducing.go | Go ReducingState implementation |
| go-sdk/state/keyed/keyed_value_state.go | Go keyed ValueState factory/state |
| go-sdk/state/keyed/keyed_list_state.go | Go keyed ListState factory/state |
| go-sdk/state/keyed/keyed_map_state.go | Go keyed MapState factory/state |
| go-sdk/state/keyed/keyed_priority_queue_state.go | Go keyed PriorityQueueState factory/state |
| go-sdk/state/keyed/keyed_aggregating_state.go | Go keyed AggregatingState factory/state |
| go-sdk/state/keyed/keyed_reducing_state.go | Go keyed ReducingState factory/state |
| go-sdk/impl/context.go | Remove RWMutex usage + minor import change |
| docs/go-sdk-guide-zh.md | Remove old Go guide (relocated under docs/Go-SDK) |
| docs/Go-SDK/go-sdk-guide.md | Add Advanced State API section + link fixes |
| docs/Go-SDK/go-sdk-guide-zh.md | Add (relocated) Chinese Go SDK guide incl. advanced state section |
| docs/Go-SDK/go-sdk-advanced-state-api.md | New dedicated Go Advanced State API doc |
| docs/Python-SDK/python-sdk-guide.md | Link to Python advanced state doc |
| docs/Python-SDK/python-sdk-guide-zh.md | Link to Python advanced state doc (ZH) |
| docs/Python-SDK/python-sdk-advanced-state-api.md | New dedicated Python Advanced State API doc |
| docs/Python-SDK/python-sdk-advanced-state-api-zh.md | New dedicated Python Advanced State API doc (ZH) |
| README.md | Update docs links (add Go SDK guide path) |
| README-zh.md | Update docs links (add Go SDK guide path) |
Comments suppressed due to low confidence (1)
go-sdk/impl/context.go:25
runtimeContextis now unsynchronized while it contains mutable state (closedflag andstoresmap) that is read/written by multiple methods. If user code calls context methods from multiple goroutines (or if the runtime re-enters concurrently), this can lead to data races and even panics like "concurrent map read and map write" inGetOrCreateStore, plus racy reads ofclosed. Either restore thesync.RWMutexprotection, or clearly enforce/document single-goroutine access and make the map/flags concurrency-safe accordingly.
type runtimeContext struct {
config map[string]string
stores map[string]*storeImpl
closed bool
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
|
@cursor review |
|
Skipping Bugbot: Bugbot is disabled for this repository. Visit the Bugbot dashboard to update your settings. |
|
@cursor review |
python/functionstream-api-advanced/src/fs_api_advanced/keyed/keyed_value_state.py
Outdated
Show resolved
Hide resolved
|
You have used all of your free Bugbot PR reviews. To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial. |
|
@cursor review |
|
You have used all of your free Bugbot PR reviews. To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial. |
|
@cursor review |
python/functionstream-api-advanced/src/fs_api_advanced/structures/value_state.py
Show resolved
Hide resolved
|
@cursor review |
python/functionstream-api-advanced/src/fs_api_advanced/keyed/keyed_list_state.py
Outdated
Show resolved
Hide resolved
python/functionstream-api-advanced/src/fs_api_advanced/codec/bytes_codec.py
Show resolved
Hide resolved
|
@cursor review |
|
@cursor review |
python/functionstream-api-advanced/src/fs_api_advanced/codec/float_codec.py
Show resolved
Hide resolved
|
@cursor review |
python/functionstream-api-advanced/src/fs_api_advanced/keyed/keyed_priority_queue_state.py
Show resolved
Hide resolved
|
@cursor review |
Made-with: Cursor
Note
Medium Risk
Adds new Go and Python “advanced state” libraries and updates the Go runtime context concurrency behavior, which could affect state access and build/runtime integration. Main risk is correctness/performance of the new state encodings (ordered codecs, merge/list packing) and any assumptions about thread-safety after removing locking.
Overview
Introduces a new Go advanced SDK (
go-sdk-advanced) providing typed state abstractions (ValueState,ListState,MapState,PriorityQueueState,AggregatingState,ReducingState) plus keyed variants via factories, all backed by the existing low-levelStoreand a new codec layer with ordered primitive codecs andDefaultCodecFor.Adds a new Python advanced state package (
functionstream-api-advanced) with parallel codec/state/keyed APIs and packaging metadata, and wires builds/docs to recognize the new advanced modules (MakefilePYTHONPATHupdate,.gitignorebuild output, README/doc links, and new advanced-state API guides in EN/ZH).Also tweaks the Go SDK runtime context implementation by removing internal mutex guarding
closed/stores, changing thread-safety expectations forEmit/GetOrCreateStore/Close.Written by Cursor Bugbot for commit e93c4a4. This will update automatically on new commits. Configure here.