experiement implementation#10
Conversation
|
Caution Review failedThe pull request is closed. WalkthroughThis update introduces significant enhancements and refactoring across the codebase. It adds comprehensive type definitions and dataclasses for monitoring, tracing, logging, generations, evaluators, and experiments. Logging and network layers are refactored for improved type safety and clarity, with new log levels and error handling. Experiment management is now supported in the SDK, and endpoint and DTO handling is modernized. Tests and interfaces are updated to align with new naming conventions and structures. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant MonitorSDK
participant CreateExperimentEndpoint
participant API
User->>MonitorSDK: create_experiment(feature_slug, params)
MonitorSDK->>CreateExperimentEndpoint: prepare_request(dto)
CreateExperimentEndpoint->>API: POST /monitor/experiments
API-->>CreateExperimentEndpoint: Response (experiment data)
CreateExperimentEndpoint-->>MonitorSDK: decode_response(body)
MonitorSDK-->>User: Experiment instance
sequenceDiagram
participant User
participant PromptSDK
participant ListPromptsEndpoint
participant API
User->>PromptSDK: list(feature_slug)
PromptSDK->>ListPromptsEndpoint: prepare_request(dto)
ListPromptsEndpoint->>API: GET /prompts?featureSlug=...
API-->>ListPromptsEndpoint: Response (prompt list)
ListPromptsEndpoint-->>PromptSDK: decode_response(body)
PromptSDK-->>User: ListResult
Poem
Note ⚡️ AI Code Reviews for VS Code, Cursor, WindsurfCodeRabbit now has a plugin for VS Code, Cursor and Windsurf. This brings AI code reviews directly in the code editor. Each commit is reviewed immediately, finding bugs before the PR is raised. Seamless context handoff to your AI code agent ensures that you can easily incorporate review feedback. Note ⚡️ Faster reviews with cachingCodeRabbit now supports caching for code and dependencies, helping speed up reviews. This means quicker feedback, reduced wait times, and a smoother review experience overall. Cached data is encrypted and stored securely. This feature will be automatically enabled for all accounts on May 16th. To opt out, configure 📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (2)
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 16
🔭 Outside diff range comments (7)
tests/test_monitor_sdk.py (1)
50-70: 🛠️ Refactor suggestionUpdate tests to verify experiment-related functionality
The PR description mentions adding experiment management support, but the test modifications don't include verification of this new functionality.
Consider adding new test cases or extending existing ones to verify the experiment-related functionality. For example:
def test_create_trace_with_experiment(self): """Test creating a trace with experiment metadata.""" experiment_id = "exp-123" trace = self.monitor.create_trace( "test-slug", { "input": self.content, "experiment": {"id": experiment_id, "name": "Test Experiment"}, "metadata": {"property1": "value1"} } ) # Assert trace was created correctly with experiment data self.assertIsNotNone(trace) self.assertEqual(trace.feature_slug, "test-slug") self.assertEqual(trace.experiment.get("id"), experiment_id)basalt/endpoints/list_prompts.py (1)
32-38: 🛠️ Refactor suggestionUpdate docstring to reflect the new parameter
The
prepare_requestmethod signature has been updated to accept aPromptListDTOparameter, but the docstring doesn't document this parameter. The docstring should be updated to include thedtoparameter and its purpose.@staticmethod def prepare_request(dto: PromptListDTO) -> Dict[str, Any]: """ Prepare the request dictionary for the ListPrompts endpoint. + + Args: + dto (PromptListDTO): The data transfer object containing the request parameters. Returns: The path, method, and query parameters for getting a prompt on the API. """basalt/basalt_facade.py (1)
20-27: 🛠️ Refactor suggestionUpdate docstring to reflect LogLevel type
The method signature has been updated to use the
LogLeveltype instead of a string, but the docstring still refers tolog_levelas a string and lists the possible values as strings. The docstring should be updated to reflect the new type.def __init__(self, api_key: str, log_level: LogLevel = 'all'): """ Initializes the Basalt client with the given API key and log level. Args: api_key (str): The API key for authenticating with the Basalt SDK. - log_level (str, optional): The log level for the logger. Defaults to 'all'. (all, warn, error, debug, none) + log_level (LogLevel, optional): The log level for the logger. Defaults to 'all'. """basalt/sdk/promptsdk.py (1)
39-41: 🛠️ Refactor suggestionAvoid mutable default for
variablesUsing
{}as a default argument is unsafe because the same dictionary instance is reused across calls.- variables: Dict[str, str] = {}, + variables: Optional[Dict[str, str]] = None,And inside the method:
variables = variables or {}Apply the same pattern to
_replace_vars.🧰 Tools
🪛 Ruff (0.11.9)
39-39: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
basalt/objects/base_log.py (1)
119-129:⚠️ Potential issueMissing evaluators in to_dict method
The
to_dictmethod doesn't include the new_evaluatorsfield in the returned dictionary, which could lead to data loss during serialization.Consider updating the
to_dictmethod to include evaluators:def to_dict(self) -> Dict[str, Any]: """Convert the log to a dictionary for API serialization.""" return { "id": self._id, "type": self._type, "name": self._name, "start_time": self._start_time, "end_time": self._end_time, "metadata": self._metadata, "parent": {"id": self._parent.id} if self._parent else None, + "evaluators": self._evaluators, }basalt/sdk/monitorsdk.py (1)
145-157: 🛠️ Refactor suggestion
⚠️ Potential issue
Noneexperiment causes AttributeError insideTrace– hot-fix pipeline failure
Trace.__init__expectsparams["experiment"]to be either absent or a valid object.
Passing the key with aNonevalue means the constructor triesexperiment.feature_slug, raising the failure seen in CI.- "metadata": params.metadata, - "experiment": params.experiment, + "metadata": params.metadata,If you still need to forward the experiment, guard it:
if params.experiment is not None: params_dict["experiment"] = params.experimentApply the same pattern to
evaluatorsandevaluation_configif they can also beNone.basalt/objects/trace.py (1)
16-28:⚠️ Potential issue
paramsis typed asTraceParamsbut treated like a dict
TraceParamsis a dataclass; calling.get()on it triggers anAttributeError.
Either pass a plaindictor access attributes directly:- self._input = params.get("input") + self._input = params.inputRepeat for every subsequent
.get()usage (input, output, name, …).
Keeping the current mismatch will break at instantiation time, masking other logic errors.
🧹 Nitpick comments (15)
basalt/ressources/monitor/log_type.py (2)
1-17: Consider using Python's Enum class for better type safetyWhile the current implementation works as a simple container for log type constants, consider using Python's built-in
Enumclass for better type safety, autocomplete support, and to prevent invalid assignments.-class LogType: +from enum import Enum, auto + +class LogType(Enum): """Enum-like class for log types. Attributes: SPAN: Represents a span log type GENERATION: Represents a generation log type FUNCTION: Represents a function log type TOOL: Represents a tool log type RETRIEVAL: Represents a retrieval log type EVENT: Represents an event log type """ - SPAN = 'span' - GENERATION = 'generation' - FUNCTION = 'function' - TOOL = 'tool' - RETRIEVAL = 'retrieval' - EVENT = 'event' + SPAN = 'span' + GENERATION = 'generation' + FUNCTION = 'function' + TOOL = 'tool' + RETRIEVAL = 'retrieval' + EVENT = 'event'
17-17: Remove trailing whitespaceThere's a trailing whitespace at the end of line 17.
- EVENT = 'event' + EVENT = 'event'basalt/endpoints/list_prompts.py (1)
39-45: Add handling for None featureSlug valueThe current implementation always includes the
featureSlugparameter in the query, even ifdto.featureSlugisNone. This could potentially cause issues if the API doesn't handle null values for this parameter properly.return { "path": "/prompts", "method": "GET", "query": { - "featureSlug": dto.featureSlug + **({"featureSlug": dto.featureSlug} if dto.featureSlug is not None else {}) } }basalt/ressources/monitor/experiment_types.py (1)
4-8: Consider adding feature_slug to ExperimentParamsThe
ExperimentParamsclass only has anamefield, but theExperimentclass requires afeature_slug. This suggests that the feature slug is provided through some other means when creating an experiment. Consider adding afeature_slugfield toExperimentParamsfor clarity and completeness.@dataclass class ExperimentParams: """Parameters for creating an experiment.""" name: str + feature_slug: strbasalt/sdk/promptsdk.py (1)
3-3: Unused importGetResult– please remove to keep the module clean
GetResultis imported but never used, which will trigger Ruff F401 and adds noise to the diff.-from ..utils.dtos import GetPromptDTO, PromptResponse, DescribePromptResponse, DescribePromptDTO, GetResult, DescribeResult, ListResult, PromptListResponse, PromptListDTO +from ..utils.dtos import ( + GetPromptDTO, + PromptResponse, + DescribePromptResponse, + DescribePromptDTO, + DescribeResult, + ListResult, + PromptListResponse, + PromptListDTO, +)🧰 Tools
🪛 Ruff (0.11.9)
3-3:
..utils.dtos.GetResultimported but unusedRemove unused import:
..utils.dtos.GetResult(F401)
basalt/utils/flusher.py (2)
32-46: Potential serialization issues forexperiment/evaluators/evaluationConfig
_trace_to_dictforwards complex python objects (Experimentinstance, possibly complex evaluator objects) directly.
WhileSendTraceEndpointdereferencesexperiment.id, it leavesevaluatorsandevaluationConfiguntouched – if they contain non-serialisable objects,json.dumpsin the network layer will fail.Consider converting them to plain serialisable structures here:
- "experiment": trace.experiment, + "experiment": {"id": trace.experiment.id} if trace.experiment else None, - "evaluators": trace.evaluators, - "evaluationConfig": trace.evaluation_config + "evaluators": [dict(e) for e in trace.evaluators] if trace.evaluators else None, + "evaluationConfig": dict(trace.evaluation_config) if trace.evaluation_config else None,
93-112: Missing success feedback after flushOn failure the method logs an error and returns, but on success it does nothing.
A lightweight info/debug log helps confirm monitoring is functioning.@@ error, result = self._api.invoke(endpoint, dto) if error: self._logger.error(f"Failed to flush trace {trace.feature_slug}: {error}") return + + self._logger.info(f"Trace {trace.feature_slug} flushed successfully")basalt/objects/log.py (1)
74-80:optionshard-coded – preserve existing optionsOverwriting
generation.optionsoutright may discard legitimate fields (e.g., temperature). Consider merging:- generation.options = {"type": "multi"} + generation.options = {**(generation.options or {}), "type": "multi"}basalt/utils/logger.py (1)
15-17: Consider more robust error logging implementationThe
errormethod uses the samedef error(self, *args): if self._can_error(): - print(*args) + import sys + print(*args, file=sys.stderr)basalt/utils/dtos.py (1)
127-130: Missing from_dict classmethod in PromptListDTOUnlike other DTOs in this file,
PromptListDTOdoesn't have afrom_dictclassmethod for consistent deserialization.Consider adding a
from_dictclassmethod for consistency:@dataclass(frozen=True) class PromptListDTO: featureSlug: Optional[str] = None + + @classmethod + def from_dict(cls, data: Dict[str, Any]): + return cls( + featureSlug=data.get("featureSlug") + )basalt/endpoints/monitor/send_trace.py (1)
45-58: Simplify conditional dictionary lookups with.get()method.The code uses conditional checks for dictionary keys, but the
.get()method would be more concise.# Convert dates and handle parent ID log_data = { "startTime": dict_log["start_time"].isoformat() if isinstance(dict_log["start_time"], datetime) else dict_log["start_time"], "endTime": dict_log["end_time"].isoformat() if isinstance(dict_log["end_time"], datetime) and dict_log["end_time"] else None, "parentId": dict_log["parent"]["id"] if dict_log["parent"] else None, - "inputTokens": dict_log["input_tokens"] if "input_tokens" in dict_log else None, - "outputTokens": dict_log["output_tokens"] if "output_tokens" in dict_log else None, - "cost": dict_log["cost"] if "cost" in dict_log else None, - "variables": [{"label": k, "value": v} for k, v in dict_log["variables"].items()] if "variables" in dict_log else None, - "input": dict_log["input"] if "input" in dict_log else None, - "output": dict_log["output"] if "output" in dict_log else None, - "prompt": dict_log["prompt"] if "prompt" in dict_log else None, - "evaluators": dict_log["evaluators"] if "evaluators" in dict_log else None + "inputTokens": dict_log.get("input_tokens"), + "outputTokens": dict_log.get("output_tokens"), + "cost": dict_log.get("cost"), + "variables": [{"label": k, "value": v} for k, v in dict_log["variables"].items()] if "variables" in dict_log else None, + "input": dict_log.get("input"), + "output": dict_log.get("output"), + "prompt": dict_log.get("prompt"), + "evaluators": dict_log.get("evaluators") }🧰 Tools
🪛 Ruff (0.11.9)
50-50: Use
dict_log.get("input_tokens", None)instead of anifblockReplace with
dict_log.get("input_tokens", None)(SIM401)
51-51: Use
dict_log.get("output_tokens", None)instead of anifblockReplace with
dict_log.get("output_tokens", None)(SIM401)
52-52: Use
dict_log.get("cost", None)instead of anifblockReplace with
dict_log.get("cost", None)(SIM401)
54-54: Use
dict_log.get("input", None)instead of anifblockReplace with
dict_log.get("input", None)(SIM401)
55-55: Use
dict_log.get("output", None)instead of anifblockReplace with
dict_log.get("output", None)(SIM401)
56-56: Use
dict_log.get("prompt", None)instead of anifblockReplace with
dict_log.get("prompt", None)(SIM401)
57-57: Use
dict_log.get("evaluators", None)instead of anifblockReplace with
dict_log.get("evaluators", None)(SIM401)
basalt/ressources/monitor/log_types.py (1)
71-163: Methods are defined but not implemented.The methods
start,end,append, andcreate_generationhave comprehensive docstrings with examples, but their implementations are placeholders (...). This suggests this file defines an interface that will be implemented elsewhere, but it should be clearly documented.Would you like me to verify how these methods are implemented in other parts of the codebase, or help complete the implementations here?
basalt/ressources/monitor/base_log_types.py (1)
61-69: Defaultidstringification is redundant
str(f'log-{uuid4().hex[:8]}')already yields a string, so wrapping it in an extrastr()call is unnecessary.- id: str = field(default_factory=lambda: str(f'log-{uuid4().hex[:8]}')) + id: str = field(default_factory=lambda: f'log-{uuid4().hex[:8]}')basalt/ressources/monitor/trace_types.py (1)
4-6:LogTypeimport is unusedStatic analysis (
ruff F401) is correct – the symbol is never referenced.
Remove the import to keep the module clean.-from .log_type import LogType🧰 Tools
🪛 Ruff (0.11.9)
6-6:
.log_type.LogTypeimported but unusedRemove unused import:
.log_type.LogType(F401)
basalt/objects/trace.py (1)
323-327:to_dictserialises raw objects – risk of non-JSON serialisable payload
self._logs,self._experiment,self._evaluators, andself._evaluation_configare complex objects.
Pushing them directly into a JSON encoder will fail or leak internal representation.Consider:
"logs": [log.to_dict() for log in self._logs], "experiment": self._experiment.to_dict() if self._experiment else None, "evaluators": [e.to_dict() for e in self._evaluators] if self._evaluators else None,Serialise only primitives and nested dicts.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Cache: Disabled due to data retention organization setting
Knowledge Base: Disabled due to data retention organization setting
⛔ Files ignored due to path filters (2)
.DS_Storeis excluded by!**/.DS_Storebasalt/.DS_Storeis excluded by!**/.DS_Store
📒 Files selected for processing (29)
basalt/_version.py(1 hunks)basalt/basalt_facade.py(3 hunks)basalt/basaltsdk.py(2 hunks)basalt/endpoints/list_prompts.py(3 hunks)basalt/endpoints/monitor/create_experiment.py(1 hunks)basalt/endpoints/monitor/send_trace.py(3 hunks)basalt/objects/base_log.py(4 hunks)basalt/objects/experiment.py(1 hunks)basalt/objects/generation.py(4 hunks)basalt/objects/log.py(2 hunks)basalt/objects/trace.py(7 hunks)basalt/ressources/monitor/base_log_types.py(1 hunks)basalt/ressources/monitor/evaluator_types.py(1 hunks)basalt/ressources/monitor/experiment_types.py(1 hunks)basalt/ressources/monitor/generation_types.py(1 hunks)basalt/ressources/monitor/log_type.py(1 hunks)basalt/ressources/monitor/log_types.py(1 hunks)basalt/ressources/monitor/monitorsdk_types.py(1 hunks)basalt/ressources/monitor/trace_types.py(1 hunks)basalt/sdk/monitorsdk.py(5 hunks)basalt/sdk/promptsdk.py(4 hunks)basalt/utils/api.py(1 hunks)basalt/utils/dtos.py(3 hunks)basalt/utils/flusher.py(3 hunks)basalt/utils/logger.py(1 hunks)basalt/utils/networker.py(2 hunks)basalt/utils/protocols.py(3 hunks)tests/test_monitor_sdk.py(1 hunks)tests/test_send_trace_endpoint.py(2 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (7)
basalt/basaltsdk.py (3)
basalt/utils/protocols.py (3)
IPromptSDK(29-32)IBasaltSDK(34-38)monitor(38-38)basalt/basalt_facade.py (1)
monitor(54-58)basalt/ressources/monitor/monitorsdk_types.py (1)
IMonitorSDK(9-176)
basalt/endpoints/list_prompts.py (5)
basalt/utils/dtos.py (2)
PromptListResponse(108-125)PromptListDTO(128-129)basalt/endpoints/monitor/create_experiment.py (1)
prepare_request(37-47)basalt/utils/protocols.py (1)
prepare_request(14-14)basalt/endpoints/get_prompt.py (1)
prepare_request(32-49)basalt/endpoints/describe_prompt.py (1)
prepare_request(32-49)
basalt/utils/flusher.py (5)
basalt/objects/trace.py (6)
feature_slug(90-92)logs(80-82)logs(85-87)experiment(100-102)evaluators(110-112)evaluation_config(105-107)basalt/utils/logger.py (1)
error(15-17)basalt/utils/protocols.py (2)
error(43-43)invoke(18-18)basalt/endpoints/monitor/send_trace.py (1)
SendTraceEndpoint(11-134)basalt/utils/api.py (1)
invoke(41-74)
basalt/ressources/monitor/experiment_types.py (2)
basalt/objects/experiment.py (5)
name(13-14)Experiment(4-22)id(9-10)feature_slug(17-18)created_at(21-22)basalt/endpoints/monitor/create_experiment.py (1)
Experiment(7-21)
basalt/utils/logger.py (1)
basalt/utils/protocols.py (4)
ILogger(40-43)warn(41-41)info(42-42)error(43-43)
basalt/endpoints/monitor/send_trace.py (4)
basalt/objects/base_log.py (4)
trace(70-72)trace(80-82)to_dict(119-129)id(30-32)basalt/objects/trace.py (4)
logs(80-82)logs(85-87)to_dict(311-327)append(227-245)basalt/objects/log.py (1)
append(58-81)basalt/objects/experiment.py (1)
id(9-10)
basalt/ressources/monitor/monitorsdk_types.py (7)
basalt/ressources/monitor/trace_types.py (4)
TraceParams(26-38)Trace(41-301)create_generation(226-254)create_log(256-281)basalt/ressources/monitor/experiment_types.py (2)
ExperimentParams(5-7)Experiment(10-14)basalt/objects/trace.py (4)
Trace(12-339)create_generation(247-270)create_log(272-289)feature_slug(90-92)basalt/ressources/monitor/generation_types.py (2)
GenerationParams(26-59)Generation(62-156)basalt/ressources/monitor/log_types.py (4)
LogParams(11-25)Log(28-188)create_generation(134-162)create_log(164-188)basalt/objects/log.py (3)
Log(6-145)create_generation(103-127)create_log(129-145)basalt/sdk/monitorsdk.py (4)
create_trace(45-65)create_generation(67-81)create_log(83-97)create_experiment(27-42)
🪛 GitHub Actions: Python SDK Tests
tests/test_monitor_sdk.py
[error] 52-239: Multiple test failures in TestMonitorSDK and TestMonitorSDKIntegration due to AttributeError: 'NoneType' object has no attribute 'feature_slug' when calling monitor.create_trace
basalt/sdk/monitorsdk.py
[error] 158-158: Failure in _create_trace method when instantiating Trace object due to experiment being None and causing AttributeError
basalt/objects/trace.py
[error] 39-39: AttributeError: 'NoneType' object has no attribute 'feature_slug' in Trace.init when accessing experiment.feature_slug
🪛 Ruff (0.11.9)
basalt/endpoints/monitor/send_trace.py
50-50: Use dict_log.get("input_tokens", None) instead of an if block
Replace with dict_log.get("input_tokens", None)
(SIM401)
51-51: Use dict_log.get("output_tokens", None) instead of an if block
Replace with dict_log.get("output_tokens", None)
(SIM401)
52-52: Use dict_log.get("cost", None) instead of an if block
Replace with dict_log.get("cost", None)
(SIM401)
54-54: Use dict_log.get("input", None) instead of an if block
Replace with dict_log.get("input", None)
(SIM401)
55-55: Use dict_log.get("output", None) instead of an if block
Replace with dict_log.get("output", None)
(SIM401)
56-56: Use dict_log.get("prompt", None) instead of an if block
Replace with dict_log.get("prompt", None)
(SIM401)
57-57: Use dict_log.get("evaluators", None) instead of an if block
Replace with dict_log.get("evaluators", None)
(SIM401)
basalt/sdk/promptsdk.py
3-3: ..utils.dtos.GetResult imported but unused
Remove unused import: ..utils.dtos.GetResult
(F401)
basalt/ressources/monitor/trace_types.py
6-6: .log_type.LogType imported but unused
Remove unused import: .log_type.LogType
(F401)
basalt/sdk/monitorsdk.py
9-9: Redefinition of unused Generation from line 6
(F811)
10-10: Redefinition of unused Log from line 7
(F811)
🔇 Additional comments (39)
basalt/utils/api.py (1)
23-23: Whitespace cleanup looks goodThis minor whitespace cleanup in the docstring improves code consistency.
basalt/_version.py (1)
1-1: Version bump is appropriate for new experiment implementationThe version bump to 0.2.0 correctly follows semantic versioning for the addition of new features (experiment implementation) without breaking changes.
basalt/basaltsdk.py (1)
1-2: Refactored import structure looks goodThe import statement for
IMonitorSDKhas been correctly updated to import from the dedicated interface module instead of the generic protocols module. This aligns with the broader refactoring to improve type safety and organization.tests/test_send_trace_endpoint.py (2)
17-17: Field name update is consistent with API changesThe renaming from "chain_slug" to "feature_slug" in the test input data aligns with the broader refactoring across the codebase.
50-50: Assertion updated properly for new field nameThe assertion correctly checks for "featureSlug" in the response body, maintaining consistency with the input change from "chain_slug" to "feature_slug".
basalt/utils/networker.py (1)
12-13: Simplified constructor by removing logging dependencyThe Networker class no longer requires a logger, which aligns with the decoupling of logging responsibilities from the network layer.
basalt/ressources/monitor/evaluator_types.py (2)
1-10: Well-defined Evaluator dataclassThe Evaluator dataclass is implemented with clear type annotations and appropriate documentation. It provides a clean structure for representing evaluator configurations.
11-16: Clean implementation of EvaluationConfigThe EvaluationConfig dataclass is well-structured with proper typing and default values. Using Optional for sample_rate with None as default is a good practice.
basalt/objects/experiment.py (1)
1-22: LGTM! Clean wrapper implementationThe
Experimentclass provides a clean wrapper around theIExperimentinterface with properly typed read-only properties. This implementation follows good object-oriented design principles and provides type safety.basalt/basalt_facade.py (1)
29-31: Removal of logger from Networker constructorThe logger parameter has been removed from the Networker constructor. This suggests that logging has been refactored out of the networker, which is a good separation of concerns. Make sure that any necessary logging for network operations is handled elsewhere in the codebase.
basalt/sdk/promptsdk.py (1)
197-208:Details
✅ Verification successful
self._logger.warnmay not exist – verify logger interfaceThe only logger method shown in
basalt/utils/logger.pyiserror().
Ifwarn()(orwarning()) is not implemented, these calls will raiseAttributeErrorthe first time a missing variable is encountered.Request verification:
If absent, either:
- self._logger.warn(...) + self._logger.error(...) # or .info/.warning once implementedor implement a
warn/warningmethod on the logger.
🏁 Script executed:
#!/bin/bash # Check for a warn|warning method on ILogger implementation rg -n "def (warn|warning)" | headLength of output: 154
No action needed:
warn()is implemented on the loggerThe
ILoggerprotocol (protocols.py:41) and theLoggerimplementation (logger.py:7) both define awarn()method, so these calls will not raise an exception. No changes are required here.🧰 Tools
🪛 Ruff (0.11.9)
197-197: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
basalt/utils/protocols.py (5)
1-1: Good addition of the Literal type importThe addition of
Literalto the import statement is appropriate for defining the newLogLeveltype alias.
4-4: Clean refactoring by importing IMonitorSDK from a dedicated type moduleMoving the
IMonitorSDKinterface to a dedicated type module improves code organization and separation of concerns.
32-32: API enhancement for filtering prompts by featureAdding the optional
feature_slugparameter enables filtering prompts by feature, which aligns with the newPromptListDTOin dtos.py.
41-43: Improved logging interface with additional severity levelsThe introduction of
infoanderrormethods enhances the logging interface with more granular control over log levels.
46-46: Type safety improvement with LogLevel type aliasThe
LogLeveltype alias usingLiteralimproves type safety by restricting possible values to "all", "warning", or "none".basalt/objects/base_log.py (5)
2-8: Good restructuring with explicit type importsThe import updates properly reference the new type modules, enhancing code organization and type safety.
14-14: Enhanced type safety with BaseLogParamsReplacing a generic dictionary with the strongly typed
BaseLogParamsimproves type safety and code clarity.
23-23: Adding evaluators supportAdding the
_evaluatorsfield enables tracking evaluator data within logs, supporting experiment tracking functionality.
74-77: Good encapsulation with evaluators propertyThe property getter for
evaluatorsfollows proper encapsulation principles by providing read-only access to the internal_evaluatorsfield.
94-99: Well-implemented evaluator addition methodThe
add_evaluatormethod includes proper null checking and initialization, and follows the method chaining pattern consistently used in the class.basalt/utils/logger.py (3)
1-4: Improved type safety in logger initializationUsing the imported
LogLeveltype for thelog_levelparameter enhances type checking and prevents invalid log level values.
11-17: Enhanced logging granularity with new methodsThe addition of
infoanderrormethods provides more granular control over logging levels, improving debugging capabilities.
19-26: Well-structured logging permission checksThe permission check methods implement a clear hierarchy: errors are always logged, warnings are logged with 'warning' or 'all' levels, and info is logged only with 'all' level.
basalt/utils/dtos.py (3)
4-4: Good integration with monitoring systemImporting the
Generationtype from the monitor module shows good integration between the prompt and monitoring systems.
127-130: Consistent addition of PromptListDTOThe new
PromptListDTOdataclass with an optionalfeatureSlugfield aligns with the updatedlistmethod signature inIPromptSDK.
73-73:Details
❓ Verification inconclusive
Enhanced return type with Generation support
Expanding the
GetResulttype to include an optionalGenerationenables returning monitoring data alongside prompt responses.This change may impact existing code that expects a 2-tuple. Please ensure all consumers of
GetResulthave been updated to handle the 3-tuple result:
🏁 Script executed:
#!/bin/bash # Find all places where GetResult is unpacked to ensure they handle the new 3-tuple format rg -A 2 "GetResult" | grep -A 5 "(err, result)"Length of output: 47
Please verify all
GetResultconsumers handle the new 3-tuple formatI couldn’t find any direct unpacking of
GetResultin the codebase. Please manually ensure that every call site:
- Invokes functions annotated with
-> GetResult- Unpacks three variables (e.g.
err, response, generation = …) rather than only twoYou can use these commands to locate call sites and unpack patterns:
# Find all functions returning GetResult rg -n "->\s*GetResult" -C2 # Find any two-variable unpacking (err, result = …) that may need a third variable rg -n "^[[:space:]]*[[:alpha:]_][[:alnum:]_]*,[[:space:]]*[[:alpha:]_][[:alnum:]_]*[[:space:]]*=" -C2basalt/objects/generation.py (6)
4-4: Import style is consistent with the project's refactoring toward typed interfaces.The import of
GenerationParamsfrom the new types module aligns with the broader effort to improve type safety throughout the codebase.
10-10: Constructor signature changed to use typed parameter.The constructor now explicitly requires a
GenerationParamstype instead of a generic dictionary, improving type safety while maintaining backward compatibility through dictionary-like access patterns.
20-21: Attribute naming convention changed from camelCase to snake_case.This change from
_inputTokens/_outputTokensto_input_tokens/_output_tokensaligns with Python naming conventions and is consistent with the broader codebase refactoring effort.
54-56: Property naming updated to match attribute naming.The
input_tokensproperty declaration and implementation now uses snake_case, consistent with the attribute rename.
59-61: Property naming updated to match attribute naming.The
output_tokensproperty declaration and implementation now uses snake_case, consistent with the attribute rename.
136-137: Update method keys renamed to match the new attribute naming.The
updatemethod now uses the snake_case keys consistent with the property and attribute renames.basalt/endpoints/monitor/send_trace.py (1)
94-106: Updated API payload structure with new fields.The request body now includes experiment, evaluators, and evaluationConfig fields, supporting the new experimental features. The naming convention change from
chainSlugtofeatureSlugis consistently applied.basalt/endpoints/monitor/create_experiment.py (1)
32-56: Well-structured endpoint implementation.The CreateExperimentEndpoint follows the established pattern for endpoints in the codebase, with separate methods for request preparation and response decoding. The error handling is consistent with other endpoints.
basalt/ressources/monitor/log_types.py (1)
36-66: Excellent docstring with detailed usage examples.The comprehensive docstring with real-world usage examples greatly enhances developer understanding of how the
Logclass should be used within the system.basalt/objects/trace.py (2)
33-36: Key name mismatch for evaluation fieldsYou read
evaluationConfigfrom the params, but the dataclass definesevaluation_config.
Even after the dict/dataclass issue is fixed, the camelCase/ snake_case discrepancy will silently drop data.- self._evaluation_config = params.get("evaluationConfig") + self._evaluation_config = params.evaluation_config
330-339:_can_flushlogs viawarnbutLoggerexposeswarningDouble-check the
LoggerAPI. If the method iswarning()(Python std-lib style), the current call will raiseAttributeError, preventing the guard from working.tests/test_monitor_sdk.py (1)
70-70:Details
❌ Incorrect review comment
Fix the implementation error causing NoneType error in
feature_slugcheck.The test has been correctly updated to check
feature_sluginstead ofchain_slug, reflecting the change in the attribute name. However, the pipeline is failing withAttributeError: 'NoneType' object has no attribute 'feature_slug'when callingmonitor.create_trace.This suggests that the implementation of
MonitorSDK.create_traceis either:
- Returning
Noneinstead of a proper Trace object, or- Not correctly assigning the
feature_slugattribute to the value passed as the first argument.
🏁 Script executed:
#!/bin/bash # Check the implementation of MonitorSDK.create_trace echo "Checking MonitorSDK.create_trace implementation..." cat basalt/sdk/monitorsdk.py | grep -A 20 "def create_trace" # Check the Trace class implementation echo "Checking Trace class implementation..." cat basalt/objects/trace.py | grep -A 20 "__init__" | head -n 30Length of output: 1642
🏁 Script executed:
#!/bin/bash echo "Inspecting _create_trace implementation in MonitorSDK..." grep -R -n -A 20 "def _create_trace" basalt/sdk/monitorsdk.py echo echo "Checking for a return of the Trace instance..." grep -R -n "return " -n -A 5 basalt/sdk/monitorsdk.py | grep "_create_trace" echo echo "Verifying feature_slug property in Trace class..." grep -R -n "feature_slug" basalt/objects/trace.pyLength of output: 1674
🏁 Script executed:
#!/bin/bash echo "Displaying full _create_trace implementation..." sed -n '128,260p' basalt/sdk/monitorsdk.pyLength of output: 2843
MonitorSDK.create_trace correctly returns a Trace instance
The implementation of
MonitorSDK.create_trace(and its internal_create_trace) properly constructs and returns aTracewithfeature_slugset to the providedslug. TheTraceclass’s constructor and itsfeature_slugproperty are implemented correctly, so theNoneTypeerror isn’t coming from here.Please check the test setup in
tests/test_monitor_sdk.py:
- How is
monitorinstantiated? Ensure it’s a validMonitorSDKinstance, notNone.- Review any fixtures or mocks for
MonitorSDKto confirm they return a real object.Likely an incorrect or invalid review comment.
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (2)
basalt/objects/trace.py (2)
223-226:update()suffers from same dataclass/dict & camelCase issuesThe method replicates the earlier mistakes (
params.get(...),evaluationConfig).
Refactor using attribute access and correct field names to ensure updates propagate.self._name = params.get("name", self._name) -self._evaluators = params.get("evaluators", self._evaluators) -self._evaluation_config = params.get("evaluationConfig", self._evaluation_config) +self._evaluators = params.get("evaluators", self._evaluators) +self._evaluation_config = params.get("evaluation_config", self._evaluation_config)Or better, use getattr for attribute access:
self._name = params.get("name", self._name) -self._evaluators = params.get("evaluators", self._evaluators) -self._evaluation_config = params.get("evaluationConfig", self._evaluation_config) +self._evaluators = getattr(params, "evaluators", self._evaluators) +self._evaluation_config = getattr(params, "evaluation_config", self._evaluation_config)
37-45:⚠️ Potential issueInitialize
self._experimentbefore conditional code pathThe
_experimentattribute is only initialized in theelsebranch when the experiment is valid. If there's no "experiment" key in params or if the experiment is invalid,self._experimentremains uninitialized which could causeAttributeErrorwhen accessing theexperimentproperty later.def __init__(self, feature_slug: str, params: TraceParams, flusher: 'Flusher', logger: 'Logger'): self._feature_slug = feature_slug self._input = params.get("input") self._output = params.get("output") self._name = params.get("name") self._start_time = params.get("start_time", datetime.now()) self._end_time = params.get("end_time") self._user = params.get("user") self._organization = params.get("organization") self._metadata = params.get("metadata") self._logs: List['BaseLog'] = [] self._flusher = flusher self._is_ended = False self._evaluators = params.get("evaluators") self._evaluation_config = params.get("evaluationConfig") self._logger = logger + self._experiment = None if "experiment" in params: experiment = params["experiment"] if experiment is None: self._logger.warn("Warning: Experiment is None. This experiment will be ignored.") elif experiment.feature_slug != self._feature_slug: self._logger.warn("Warning: Experiment feature slug does not match trace feature slug. This experiment will be ignored.") else: self._experiment = experiment
🧹 Nitpick comments (1)
basalt/objects/trace.py (1)
331-341: Simplify_can_flushmethod logicThe current implementation is confusing - it checks if the trace is ended, logs a warning if it is, then returns the negation of
_is_ended. The method should either return early after logging the warning, or separate the warning logic from the boolean check.def _can_flush(self) -> bool: """ Check if the trace can be flushed. Returns: bool: True if the trace can be flushed, False otherwise. """ if self._is_ended: self._logger.warn('Trace already ended. This operation will be ignored.') + return False - return not self._is_ended + return True
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Cache: Disabled due to data retention organization setting
Knowledge Base: Disabled due to data retention organization setting
📒 Files selected for processing (1)
basalt/objects/trace.py(7 hunks)
🔇 Additional comments (1)
basalt/objects/trace.py (1)
306-309: 🛠️ Refactor suggestionImprove flush control logic in
end()The method calls
_can_flush()which logs a warning if the trace is already ended, but then proceeds to checknot self._is_endedin the return. This creates a situation where you log a warning but then still potentially try to flush.# Send to the API using the flusher -if self._can_flush(): +can_flush = self._can_flush() +if can_flush: self._end_time = datetime.now() self._is_ended = True self._flusher.flush_trace(self)Likely an incorrect or invalid review comment.
| def set_experiment(self, experiment: Dict[str, Any]) -> 'Trace': | ||
| """ | ||
| Set the experiment for the trace. | ||
|
|
||
| Args: | ||
| experiment (Dict[str, Any]): The experiment to set. | ||
|
|
||
| Returns: | ||
| Trace: The trace instance. | ||
| """ | ||
| self._experiment = experiment | ||
| return self | ||
|
|
There was a problem hiding this comment.
Type mismatch in set_experiment method
The method signature accepts a Dict[str, Any] but assigns it directly to self._experiment, which is likely supposed to be an Experiment object based on the property return type and import. This could cause type errors later.
-def set_experiment(self, experiment: Dict[str, Any]) -> 'Trace':
+def set_experiment(self, experiment: 'Experiment') -> 'Trace':
"""
Set the experiment for the trace.
Args:
- experiment (Dict[str, Any]): The experiment to set.
+ experiment ('Experiment'): The experiment to set.
Returns:
Trace: The trace instance.
"""
self._experiment = experiment
return self📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def set_experiment(self, experiment: Dict[str, Any]) -> 'Trace': | |
| """ | |
| Set the experiment for the trace. | |
| Args: | |
| experiment (Dict[str, Any]): The experiment to set. | |
| Returns: | |
| Trace: The trace instance. | |
| """ | |
| self._experiment = experiment | |
| return self | |
| def set_experiment(self, experiment: 'Experiment') -> 'Trace': | |
| """ | |
| Set the experiment for the trace. | |
| Args: | |
| experiment ('Experiment'): The experiment to set. | |
| Returns: | |
| Trace: The trace instance. | |
| """ | |
| self._experiment = experiment | |
| return self |
🤖 Prompt for AI Agents
In basalt/objects/trace.py around lines 172 to 184, the set_experiment method
accepts a Dict[str, Any] but assigns it directly to self._experiment, which
should be an Experiment object. To fix this, change the method to accept an
Experiment instance instead of a dictionary, and update the type hint
accordingly. Ensure that the assigned value to self._experiment is of type
Experiment to maintain type consistency and avoid errors.
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (2)
basalt/sdk/monitorsdk.py (2)
6-10: Remove shadowed imports forGenerationandLogThis duplicates the previous review issue which is still present. The imports on lines 6-7 shadow the ones on lines 9-10, causing Ruff F811 warnings.
-from ..ressources.monitor.generation_types import GenerationParams, Generation -from ..ressources.monitor.log_types import LogParams, Log +from ..ressources.monitor.generation_types import GenerationParams +from ..ressources.monitor.log_types import LogParams🧰 Tools
🪛 Ruff (0.11.9)
9-9: Redefinition of unused
Generationfrom line 6(F811)
10-10: Redefinition of unused
Logfrom line 7(F811)
27-42:⚠️ Potential issueReturn type in docstring doesn't match annotation
The method annotation correctly returns
Tuple[Optional[Exception], Optional[Experiment]], but the docstring incorrectly states it returns justExperiment. Also, the params type in docstring is incorrect.def create_experiment( self, feature_slug: str, params: ExperimentParams ) -> Tuple[Optional[Exception], Optional[Experiment]]: """ Creates a new experiment for monitoring. Args: feature_slug (str): The feature slug for the experiment. - params (Dict[str, Any]): Parameters for the experiment. + params (ExperimentParams): Parameters for the experiment. Returns: - Experiment: A new Experiment instance. + Tuple[Optional[Exception], Optional[Experiment]]: A tuple containing an optional error and an optional Experiment instance. """
🧹 Nitpick comments (1)
basalt/sdk/monitorsdk.py (1)
48-48: TraceParams should have a default value of NoneTo be consistent with the code that handles
Noneparams, the type annotation should allow forNone.- params: Optional[TraceParams] = None + params: Optional[Dict[str, Any]] = NoneAlternatively, modify the implementation to properly handle
NoneforTraceParams.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Cache: Disabled due to data retention organization setting
Knowledge Base: Disabled due to data retention organization setting
📒 Files selected for processing (1)
basalt/sdk/monitorsdk.py(5 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
basalt/sdk/monitorsdk.py (7)
basalt/utils/protocols.py (6)
IApi(17-18)ILogger(40-43)monitor(38-38)get(10-10)get(30-30)invoke(18-18)basalt/ressources/monitor/trace_types.py (2)
TraceParams(26-38)Trace(41-301)basalt/ressources/monitor/experiment_types.py (2)
ExperimentParams(5-7)Experiment(10-14)basalt/objects/trace.py (7)
Trace(12-341)experiment(102-104)feature_slug(92-94)name(47-49)metadata(77-79)evaluators(112-114)evaluation_config(107-109)basalt/endpoints/monitor/create_experiment.py (3)
Experiment(7-21)CreateExperimentEndpoint(32-55)CreateExperimentDTO(24-26)basalt/objects/experiment.py (3)
Experiment(4-22)feature_slug(17-18)name(13-14)basalt/utils/flusher.py (1)
Flusher(11-114)
🪛 Ruff (0.11.9)
basalt/sdk/monitorsdk.py
9-9: Redefinition of unused Generation from line 6
(F811)
10-10: Redefinition of unused Log from line 7
(F811)
🔇 Additional comments (2)
basalt/sdk/monitorsdk.py (2)
153-156: Support for experiment and evaluations dataThe changes correctly implement new experiment-related functionality by adding support for experiments, evaluators, and evaluation configuration in trace creation. The implementation aligns with the changes to the
Traceclass as shown in the relevant code snippets.
158-158: Added logger to Trace constructionGood improvement to pass the logger to the Trace constructor, which allows for better error reporting within the Trace class.
| if params is None: | ||
| params = {} | ||
|
|
||
| trace_params = TraceParams(**params) | ||
|
|
There was a problem hiding this comment.
🛠️ Refactor suggestion
Inconsistent handling of TraceParams
The method first initializes params to an empty dict if None, but then immediately converts it to TraceParams. This is inconsistent with the type annotation.
if params is None:
- params = {}
-
- trace_params = TraceParams(**params)
+ params = TraceParams()Committable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In basalt/sdk/monitorsdk.py around lines 60 to 64, the code initializes params
as an empty dictionary if None, but then converts it to a TraceParams object,
which conflicts with the expected type annotation. To fix this, ensure that
params is consistently treated as a TraceParams instance by either initializing
params as TraceParams() when None or adjusting the type annotation and usage to
match the dictionary input before conversion.
| dto = CreateExperimentDTO( | ||
| feature_slug=feature_slug, | ||
| name=params.get("name"), | ||
| ) | ||
|
|
||
| # Call the API endpoint | ||
| err, result = self._api.invoke(CreateExperimentEndpoint, dto) | ||
|
|
||
| if err is None: |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Verify error handling in _create_experiment
The method returns (None, Experiment) when successful and (err, None) when there's an error. This is a good pattern that follows the API protocol, but make sure all consumers of this method handle both cases correctly.
🏁 Script executed:
#!/bin/bash
# Find all usages of create_experiment to verify error handling
rg -A 5 "create_experiment\(" --type pyLength of output: 2223
Fix error handling in the create_experiment usage example
The only consumer of _create_experiment we found is the sample in the docstring at basalt/ressources/monitor/monitorsdk_types.py, which currently does:
experiment = basalt.monitor.create_experiment('user-query', {'name': 'my-experiment'})
# Create a trace and add it to the experiment
trace = basalt.monitor.create_trace('user-query', {'experiment': experiment})This ignores the (err, experiment) tuple and will break if an error occurs. Please update the snippet to unpack and handle errors:
• File: basalt/ressources/monitor/monitorsdk_types.py
- Replace the single assignment with:
- experiment = basalt.monitor.create_experiment('user-query', {'name': 'my-experiment'})
+ err, experiment = basalt.monitor.create_experiment('user-query', {'name': 'my-experiment'})
+ if err is not None:
+ # TODO: handle error (e.g., log or raise)
+ else:
+ # proceed with valid experiment
+ trace = basalt.monitor.create_trace('user-query', {'experiment': experiment})This ensures every consumer properly checks for errors before using the returned Experiment.
🤖 Prompt for AI Agents
In basalt/ressources/monitor/monitorsdk_types.py around the usage example of
create_experiment, the code currently assigns the result directly to a variable
without unpacking the (err, experiment) tuple returned by create_experiment.
Update the example to unpack the tuple into err and experiment variables, then
add a conditional check to handle the error case before proceeding. This ensures
proper error handling consistent with the API protocol.
| def _create_experiment( | ||
| self, | ||
| feature_slug: str, | ||
| params: ExperimentParams | ||
| ) -> Tuple[Optional[Exception], Optional[Experiment]]: | ||
| """ | ||
| Internal implementation for creating an experiment. | ||
|
|
||
| Args: | ||
| feature_slug (str): The feature slug for the experiment. | ||
| params (ExperimentParams): Parameters for the experiment. | ||
|
|
||
| Returns: | ||
| Experiment: A new Experiment instance. | ||
| """ | ||
| dto = CreateExperimentDTO( | ||
| feature_slug=feature_slug, | ||
| name=params.get("name"), | ||
| ) | ||
|
|
||
| # Call the API endpoint | ||
| err, result = self._api.invoke(CreateExperimentEndpoint, dto) | ||
|
|
||
| if err is None: | ||
| return None, Experiment(result.experiment) | ||
|
|
||
| return err, None |
There was a problem hiding this comment.
Incorrect usage of ExperimentParams and docstring inconsistency
There are several issues with the implementation:
- The docstring return type doesn't match the actual return type
params.get("name")assumes ExperimentParams is a dictionary-like object, but it appears to be a dataclass based on the imports
def _create_experiment(
self,
feature_slug: str,
params: ExperimentParams
) -> Tuple[Optional[Exception], Optional[Experiment]]:
"""
Internal implementation for creating an experiment.
Args:
feature_slug (str): The feature slug for the experiment.
params (ExperimentParams): Parameters for the experiment.
Returns:
- Experiment: A new Experiment instance.
+ Tuple[Optional[Exception], Optional[Experiment]]: A tuple containing an optional error and an optional Experiment instance.
"""
dto = CreateExperimentDTO(
feature_slug=feature_slug,
- name=params.get("name"),
+ name=params.name,
)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def _create_experiment( | |
| self, | |
| feature_slug: str, | |
| params: ExperimentParams | |
| ) -> Tuple[Optional[Exception], Optional[Experiment]]: | |
| """ | |
| Internal implementation for creating an experiment. | |
| Args: | |
| feature_slug (str): The feature slug for the experiment. | |
| params (ExperimentParams): Parameters for the experiment. | |
| Returns: | |
| Experiment: A new Experiment instance. | |
| """ | |
| dto = CreateExperimentDTO( | |
| feature_slug=feature_slug, | |
| name=params.get("name"), | |
| ) | |
| # Call the API endpoint | |
| err, result = self._api.invoke(CreateExperimentEndpoint, dto) | |
| if err is None: | |
| return None, Experiment(result.experiment) | |
| return err, None | |
| def _create_experiment( | |
| self, | |
| feature_slug: str, | |
| params: ExperimentParams | |
| ) -> Tuple[Optional[Exception], Optional[Experiment]]: | |
| """ | |
| Internal implementation for creating an experiment. | |
| Args: | |
| feature_slug (str): The feature slug for the experiment. | |
| params (ExperimentParams): Parameters for the experiment. | |
| Returns: | |
| Tuple[Optional[Exception], Optional[Experiment]]: A tuple containing an optional error and an optional Experiment instance. | |
| """ | |
| dto = CreateExperimentDTO( | |
| feature_slug=feature_slug, | |
| name=params.name, | |
| ) | |
| # Call the API endpoint | |
| err, result = self._api.invoke(CreateExperimentEndpoint, dto) | |
| if err is None: | |
| return None, Experiment(result.experiment) | |
| return err, None |
🤖 Prompt for AI Agents
In basalt/sdk/monitorsdk.py around lines 99 to 125, the method
_create_experiment has a docstring that incorrectly states the return type as
Experiment instead of a tuple of Optional[Exception] and Optional[Experiment].
Also, the code incorrectly uses params.get("name") assuming ExperimentParams is
a dictionary, but it is a dataclass. Fix this by updating the docstring to
reflect the correct return type and access the name attribute directly from
params using params.name instead of params.get("name").
1 participant
Summary by CodeRabbit
New Features
Improvements
chain_slugtofeature_slugfor clarity across traces and related payloads.info,error) and refined logging behavior.Bug Fixes
Documentation
Tests