Skip to content

feat: implement advanced state API for Go and Python#214

Merged
RobertIndie merged 23 commits intoFunctionStream:mainfrom
luoluoyuyu:feature/advanced-state-api
Mar 14, 2026
Merged

feat: implement advanced state API for Go and Python#214
RobertIndie merged 23 commits intoFunctionStream:mainfrom
luoluoyuyu:feature/advanced-state-api

Conversation

@luoluoyuyu
Copy link
Collaborator

@luoluoyuyu luoluoyuyu commented Mar 8, 2026

  • Go SDK: add state package with codec, keyed state, and structures
    • codec: bool/bytes/float/int/string/uint/json and ordered types
    • keyed: Value, List, Map, Aggregating, Reducing, PriorityQueue keyed states
    • structures: underlying implementations for each state type
  • Python API: refactor store into codec, keyed, structures submodules
    • keyed: KeyedValueState, KeyedListState, KeyedMapState, etc.
    • structures: state structures aligned with Go
  • Runtime: update fs_context to work with new state API

Made-with: Cursor


Note

Medium Risk
Adds new Go and Python “advanced state” libraries and updates the Go runtime context concurrency behavior, which could affect state access and build/runtime integration. Main risk is correctness/performance of the new state encodings (ordered codecs, merge/list packing) and any assumptions about thread-safety after removing locking.

Overview
Introduces a new Go advanced SDK (go-sdk-advanced) providing typed state abstractions (ValueState, ListState, MapState, PriorityQueueState, AggregatingState, ReducingState) plus keyed variants via factories, all backed by the existing low-level Store and a new codec layer with ordered primitive codecs and DefaultCodecFor.

Adds a new Python advanced state package (functionstream-api-advanced) with parallel codec/state/keyed APIs and packaging metadata, and wires builds/docs to recognize the new advanced modules (Makefile PYTHONPATH update, .gitignore build output, README/doc links, and new advanced-state API guides in EN/ZH).

Also tweaks the Go SDK runtime context implementation by removing internal mutex guarding closed/stores, changing thread-safety expectations for Emit/GetOrCreateStore/Close.

Written by Cursor Bugbot for commit e93c4a4. This will update automatically on new commits. Configure here.

- Go SDK: add state package with codec, keyed state, and structures
  - codec: bool/bytes/float/int/string/uint/json and ordered types
  - keyed: Value, List, Map, Aggregating, Reducing, PriorityQueue keyed states
  - structures: underlying implementations for each state type
- Python API: refactor store into codec, keyed, structures submodules
  - keyed: KeyedValueState, KeyedListState, KeyedMapState, etc.
  - structures: state structures aligned with Go
- Runtime: update fs_context to work with new state API

Made-with: Cursor
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduces an “advanced state” layer for both Go and Python SDKs: typed state abstractions (Value/List/Map/PQ/Aggregating/Reducing) built on top of the existing KV store APIs, plus default/built-in codecs and keyed-state factories. Updates runtime context wiring and expands the documentation to include dedicated “Advanced State API” guides.

Changes:

  • Go SDK: add state/codec, state/structures, and state/keyed packages implementing typed state + default codecs.
  • Python API: add fs_api.store.codec/structures/keyed modules, re-export them from fs_api.store / fs_api, and extend the Context interface + runtime WitContext.
  • Docs/packaging: add advanced state API docs (Go/Python), fix guide links/paths, and update Python packaging discovery.

Reviewed changes

Copilot reviewed 71 out of 71 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
python/functionstream-runtime/src/fs_runtime/store/fs_context.py Expose typed state creation helpers on runtime Context
python/functionstream-api/src/fs_api/store/structures/value_state.py New typed ValueState wrapper
python/functionstream-api/src/fs_api/store/structures/reducing_state.py New typed ReducingState wrapper
python/functionstream-api/src/fs_api/store/structures/priority_queue_state.py New typed PriorityQueueState wrapper
python/functionstream-api/src/fs_api/store/structures/map_state.py New typed MapState wrapper + helpers
python/functionstream-api/src/fs_api/store/structures/list_state.py New typed ListState wrapper
python/functionstream-api/src/fs_api/store/structures/aggregating_state.py New typed AggregatingState wrapper
python/functionstream-api/src/fs_api/store/structures/init.py Export structures API surface
python/functionstream-api/src/fs_api/store/keyed/keyed_value_state.py New keyed ValueState factory/state
python/functionstream-api/src/fs_api/store/keyed/keyed_reducing_state.py New keyed ReducingState factory/state
python/functionstream-api/src/fs_api/store/keyed/keyed_priority_queue_state.py New keyed PriorityQueueState factory/state
python/functionstream-api/src/fs_api/store/keyed/keyed_map_state.py New keyed MapState factory/state
python/functionstream-api/src/fs_api/store/keyed/keyed_list_state.py New keyed ListState factory/state
python/functionstream-api/src/fs_api/store/keyed/keyed_aggregating_state.py New keyed AggregatingState factory/state
python/functionstream-api/src/fs_api/store/keyed/_keyed_common.py Shared keyed helpers (ordered codec enforcement)
python/functionstream-api/src/fs_api/store/keyed/init.py Export keyed API surface
python/functionstream-api/src/fs_api/store/codec/base.py Base Codec interface for Python
python/functionstream-api/src/fs_api/store/codec/bool_codec.py Ordered bool codec
python/functionstream-api/src/fs_api/store/codec/bytes_codec.py Ordered bytes codec
python/functionstream-api/src/fs_api/store/codec/string_codec.py Ordered string codec
python/functionstream-api/src/fs_api/store/codec/int_codec.py Ordered int codec implementation
python/functionstream-api/src/fs_api/store/codec/float_codec.py Ordered float codec implementation
python/functionstream-api/src/fs_api/store/codec/json_codec.py JSON codec implementation
python/functionstream-api/src/fs_api/store/codec/pickle_codec.py Pickle codec implementation
python/functionstream-api/src/fs_api/store/codec/default_codec.py default_codec_for selection logic
python/functionstream-api/src/fs_api/store/codec/init.py Export codec API surface
python/functionstream-api/src/fs_api/store/init.py Re-export codec/structures/keyed from store package
python/functionstream-api/src/fs_api/context.py Expand Context interface with typed state helpers
python/functionstream-api/src/fs_api/init.py Re-export advanced state API from top-level package
python/functionstream-api/pyproject.toml Switch to setuptools package auto-discovery
go-sdk/state/common/common.go Shared helpers for state packages
go-sdk/state/codec/interface.go Go Codec interface + fixed-size helper
go-sdk/state/codec/default_codec.go Go DefaultCodecFor implementation
go-sdk/state/codec/bool_codec.go Go ordered bool codec
go-sdk/state/codec/string_codec.go Go ordered string codec
go-sdk/state/codec/json_codec.go Go JSON codec
go-sdk/state/codec/int_codec.go Go int codec (delegates to int64)
go-sdk/state/codec/int8_codec.go Go ordered int8 codec
go-sdk/state/codec/int16_codec.go Go ordered int16 codec
go-sdk/state/codec/int32_codec.go Go ordered int32 codec
go-sdk/state/codec/int64_codec.go Go ordered int64 codec
go-sdk/state/codec/uint_codec.go Go uint codec (delegates to uint64)
go-sdk/state/codec/uint8_codec.go Go ordered uint8 codec
go-sdk/state/codec/uint16_codec.go Go ordered uint16 codec
go-sdk/state/codec/uint32_codec.go Go ordered uint32 codec
go-sdk/state/codec/uint64_codec.go Go ordered uint64 codec
go-sdk/state/codec/float32_codec.go Go ordered float32 codec
go-sdk/state/codec/float64_codec.go Go ordered float64 codec
go-sdk/state/structures/value.go Go ValueState implementation
go-sdk/state/structures/list.go Go ListState implementation
go-sdk/state/structures/map.go Go MapState implementation
go-sdk/state/structures/priority_queue.go Go PriorityQueueState implementation
go-sdk/state/structures/aggregating.go Go AggregatingState implementation
go-sdk/state/structures/reducing.go Go ReducingState implementation
go-sdk/state/keyed/keyed_value_state.go Go keyed ValueState factory/state
go-sdk/state/keyed/keyed_list_state.go Go keyed ListState factory/state
go-sdk/state/keyed/keyed_map_state.go Go keyed MapState factory/state
go-sdk/state/keyed/keyed_priority_queue_state.go Go keyed PriorityQueueState factory/state
go-sdk/state/keyed/keyed_aggregating_state.go Go keyed AggregatingState factory/state
go-sdk/state/keyed/keyed_reducing_state.go Go keyed ReducingState factory/state
go-sdk/impl/context.go Remove RWMutex usage + minor import change
docs/go-sdk-guide-zh.md Remove old Go guide (relocated under docs/Go-SDK)
docs/Go-SDK/go-sdk-guide.md Add Advanced State API section + link fixes
docs/Go-SDK/go-sdk-guide-zh.md Add (relocated) Chinese Go SDK guide incl. advanced state section
docs/Go-SDK/go-sdk-advanced-state-api.md New dedicated Go Advanced State API doc
docs/Python-SDK/python-sdk-guide.md Link to Python advanced state doc
docs/Python-SDK/python-sdk-guide-zh.md Link to Python advanced state doc (ZH)
docs/Python-SDK/python-sdk-advanced-state-api.md New dedicated Python Advanced State API doc
docs/Python-SDK/python-sdk-advanced-state-api-zh.md New dedicated Python Advanced State API doc (ZH)
README.md Update docs links (add Go SDK guide path)
README-zh.md Update docs links (add Go SDK guide path)
Comments suppressed due to low confidence (1)

go-sdk/impl/context.go:25

  • runtimeContext is now unsynchronized while it contains mutable state (closed flag and stores map) that is read/written by multiple methods. If user code calls context methods from multiple goroutines (or if the runtime re-enters concurrently), this can lead to data races and even panics like "concurrent map read and map write" in GetOrCreateStore, plus racy reads of closed. Either restore the sync.RWMutex protection, or clearly enforce/document single-goroutine access and make the map/flags concurrency-safe accordingly.
type runtimeContext struct {
	config map[string]string
	stores map[string]*storeImpl
	closed bool
}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

@luoluoyuyu
Copy link
Collaborator Author

@cursor review

@cursor
Copy link

cursor bot commented Mar 13, 2026

Skipping Bugbot: Bugbot is disabled for this repository. Visit the Bugbot dashboard to update your settings.

@luoluoyuyu
Copy link
Collaborator Author

@cursor review

@cursor
Copy link

cursor bot commented Mar 13, 2026

You have used all of your free Bugbot PR reviews.

To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

@luoluoyuyu
Copy link
Collaborator Author

@cursor review

@cursor
Copy link

cursor bot commented Mar 13, 2026

You have used all of your free Bugbot PR reviews.

To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

@luoluoyuyu
Copy link
Collaborator Author

@cursor review

@luoluoyuyu
Copy link
Collaborator Author

@cursor review

@luoluoyuyu
Copy link
Collaborator Author

@cursor review

@luoluoyuyu
Copy link
Collaborator Author

@cursor review

@luoluoyuyu
Copy link
Collaborator Author

@cursor review

@luoluoyuyu luoluoyuyu requested a review from RobertIndie March 14, 2026 04:50
@luoluoyuyu
Copy link
Collaborator Author

@cursor review

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

@RobertIndie RobertIndie merged commit 6ab4638 into FunctionStream:main Mar 14, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants