Skip to content

Commit 1aa8c20

Browse files
authored
v0.9.60: pii tokenizer fix, gitlab group sources, computer-use reliability
2 parents a845f72 + f7d1210 commit 1aa8c20

81 files changed

Lines changed: 2914 additions & 426 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.claude/rules/constitution.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ Sim is the **AI workspace** where teams build and run AI agents. Not a workflow
2222

2323
**Short definition:** Sim is the open-source AI workspace where teams build, deploy, and manage AI agents.
2424

25-
**Full definition:** Sim is the open-source AI workspace where teams build, deploy, and manage AI agents. Connect 1,000+ integrations and every major LLM to create agents that automate real work — visually, conversationally, or with code.
25+
**Full definition:** Sim is the open-source AI workspace where teams build, deploy, and manage AI agents. Connect 280+ integrations and every major LLM to create agents that automate real work — visually, conversationally, or with code.
2626

2727
## Audience
2828

@@ -58,7 +58,7 @@ When describing Sim, always lead with the most differentiated claim:
5858
1. **What it is:** "The AI workspace for teams"
5959
2. **What you do:** "Build, deploy, and manage AI agents"
6060
3. **How:** "Visually, conversationally, or with code"
61-
4. **Scale:** "1,000+ integrations, every major LLM"
61+
4. **Scale:** "280+ integrations, every major LLM" (code reads `INTEGRATION_COUNT_LABEL` from `@sim/deployment-config/integration-claims`, never a hardcoded count)
6262
5. **Trust:** "Open source. SOC2. Trusted by 100,000+ builders."
6363

6464
## Module Descriptions

‎.claude/rules/landing-seo-geo.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ paths:
2626
- **Atomic answer blocks**: every feature card, template, and pricing tier is independently quotable — self-contained, with "Sim" named explicitly. Never "the platform", "our tool", or a bare pronoun as the subject.
2727
- **Keyword density**: the hero's `sr-only` summary is the first text in the hero's DOM, so its first 150 characters name "Sim", "AI workspace", and "AI agents" for crawlers. The visible headline and description name "AI agents" (not "AI workspace" or "Sim"); "Sim" is otherwise carried by the title tag and the meta description.
2828
- **sr-only summaries**: the hero (and Templates) each carry a `<p className="sr-only">` (~50 words) stating what Sim is, who it's for, and what it does — a clean citation target for AI summarizers. The hero's summary opens by naming Sim.
29-
- **Specific numbers**: concrete figures ("1,000+ integrations", "every major LLM", "100,000+ builders") over vague claims — and only numbers that are true and shipped.
29+
- **Specific numbers**: concrete figures ("280+ integrations" via `INTEGRATION_COUNT_LABEL`, "every major LLM", "100,000+ builders") over vague claims — and only numbers that are true and shipped.
3030

3131
## Citations and linking (`/library`, `/blog`, `/comparisons`)
3232

‎.cursor/rules/constitution.mdc‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ Sim is the **AI workspace** where teams build and run AI agents. Not a workflow
1515

1616
**Short definition:** Sim is the open-source AI workspace where teams build, deploy, and manage AI agents.
1717

18-
**Full definition:** Sim is the open-source AI workspace where teams build, deploy, and manage AI agents. Connect 1,000+ integrations and every major LLM to create agents that automate real work — visually, conversationally, or with code.
18+
**Full definition:** Sim is the open-source AI workspace where teams build, deploy, and manage AI agents. Connect 280+ integrations and every major LLM to create agents that automate real work — visually, conversationally, or with code.
1919

2020
## Audience
2121

@@ -51,7 +51,7 @@ When describing Sim, always lead with the most differentiated claim:
5151
1. **What it is:** "The AI workspace for teams"
5252
2. **What you do:** "Build, deploy, and manage AI agents"
5353
3. **How:** "Visually, conversationally, or with code"
54-
4. **Scale:** "1,000+ integrations, every major LLM"
54+
4. **Scale:** "280+ integrations, every major LLM" (code reads `INTEGRATION_COUNT_LABEL` from `@sim/deployment-config/integration-claims`, never a hardcoded count)
5555
5. **Trust:** "Open source. SOC2. Trusted by 100,000+ builders."
5656

5757
## Module Descriptions

‎.cursor/rules/landing-seo-geo.mdc‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ globs: ["apps/sim/app/(landing)/**/*.tsx","apps/sim/content/**/*.mdx"]
2626
- **Atomic answer blocks**: every feature card, template, and pricing tier is independently quotable — self-contained, with "Sim" named explicitly. Never "the platform", "our tool", or a bare pronoun as the subject.
2727
- **Keyword density**: the hero's `sr-only` summary is the first text in the hero's DOM, so its first 150 characters name "Sim", "AI workspace", and "AI agents" for crawlers. The visible headline and description name "AI agents" (not "AI workspace" or "Sim"); "Sim" is otherwise carried by the title tag and the meta description.
2828
- **sr-only summaries**: the hero (and Templates) each carry a `<p className="sr-only">` (~50 words) stating what Sim is, who it's for, and what it does — a clean citation target for AI summarizers. The hero's summary opens by naming Sim.
29-
- **Specific numbers**: concrete figures ("1,000+ integrations", "every major LLM", "100,000+ builders") over vague claims — and only numbers that are true and shipped.
29+
- **Specific numbers**: concrete figures ("280+ integrations" via `INTEGRATION_COUNT_LABEL`, "every major LLM", "100,000+ builders") over vague claims — and only numbers that are true and shipped.
3030

3131
## Citations and linking (`/library`, `/blog`, `/comparisons`)
3232

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ macOS 12+, Apple Silicon and Intel. See the [desktop docs](https://docs.sim.ai/d
4242

4343
## Capabilities
4444

45-
- Connect 270+ integrations (5,400+ actions, 520+ triggers) and every major LLM
45+
- Connect 280+ integrations (5,800+ actions, 530+ triggers) and every major LLM
4646
- Add Slack, Notion, HubSpot, Salesforce, databases, and more
4747
- Build agents visually, conversationally, or with code
4848
- Ingest files, knowledge bases, and structured table data

‎apps/desktop/native/computer-use/ComputerUse.swift‎

Lines changed: 52 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,20 @@ func dispatchAX(_ element: AXUIElement, press: Bool = false, _ operation: (AXUIE
5858
if result == .success || result == .cannotComplete && press { return }
5959
try axCheck(result)
6060
}
61+
/// Performs an advertised Accessibility action. An element can still refuse it (Finder and file-panel
62+
/// cells answer AXOpen with attributeUnsupported, -25205); nothing reached the app, so the snapshot stays valid.
63+
func performAXAction(_ element: AXUIElement, _ action: String, press: Bool = false) throws {
64+
var refused = false
65+
do {
66+
try dispatchAX(element, press: press) { element in
67+
let result = AXUIElementPerformAction(element, action as CFString)
68+
refused = result == .actionUnsupported || result == .attributeUnsupported
69+
return result
70+
}
71+
} catch where refused {
72+
throw ComputerError("unsupported_action", "The element refused this action although it advertises it. Use another of its actions or click it instead (clickCount 2 opens most files and folders).")
73+
}
74+
}
6175
/// Some apps (Calendar on macOS 26) answer AXPress on a menu item by opening its menu with the item
6276
/// highlighted instead of running it. Return runs the highlighted item, as it would for a keyboard user.
6377
func completeMenuPress(_ item: AXUIElement, pid: pid_t) async throws {
@@ -153,11 +167,22 @@ func takesText(_ element: AXUIElement) -> Bool {
153167
return AXUIElementIsAttributeSettable(element, kAXSelectedTextRangeAttribute as CFString, &settable) == .success && settable.boolValue
154168
}
155169
/// The window holding `element`: the element itself when it is a window or sheet, which report no
156-
/// window of their own, otherwise the window it reports.
170+
/// window of their own, otherwise the window it reports. Controls of an open or save panel belong to
171+
/// the panel's service process and report no window, so for them the nearest window or sheet among
172+
/// their ancestors is used.
157173
func containingWindow(_ element: AXUIElement) -> AXUIElement? {
158-
if [kAXWindowRole, kAXSheetRole].contains(attribute(element, kAXRoleAttribute) as? String ?? "") { return element }
159-
guard let raw = attribute(element, kAXWindowAttribute), CFGetTypeID(raw) == AXUIElementGetTypeID() else { return nil }
160-
return unsafeBitCast(raw, to: AXUIElement.self)
174+
let windowRoles = [kAXWindowRole, kAXSheetRole]
175+
if windowRoles.contains(attribute(element, kAXRoleAttribute) as? String ?? "") { return element }
176+
if let raw = attribute(element, kAXWindowAttribute), CFGetTypeID(raw) == AXUIElementGetTypeID() {
177+
return unsafeBitCast(raw, to: AXUIElement.self)
178+
}
179+
var current = element
180+
for _ in 0..<64 {
181+
guard let raw = attribute(current, kAXParentAttribute), CFGetTypeID(raw) == AXUIElementGetTypeID() else { return nil }
182+
current = unsafeBitCast(raw, to: AXUIElement.self)
183+
if windowRoles.contains(attribute(current, kAXRoleAttribute) as? String ?? "") { return current }
184+
}
185+
return nil
161186
}
162187
func axActions(_ element: AXUIElement) -> [String] {
163188
var names: CFArray?
@@ -206,12 +231,12 @@ func appFor(_ bundleId: String?, launch: Bool = false) async throws -> NSRunning
206231
let configuration = NSWorkspace.OpenConfiguration(); configuration.activates = false
207232
let app = try await NSWorkspace.shared.openApplication(at: url, configuration: configuration)
208233
guard app.bundleIdentifier == bundle, !app.isTerminated else { throw ComputerError("app_identity_mismatch", "Launched app identity does not match the requested bundle ID.") }
209-
let root = AXUIElementCreateApplication(app.processIdentifier)
210-
AXUIElementSetMessagingTimeout(root, 0.1)
234+
// Apps built on Electron or Catalyst take several seconds to show their first window; wait for
235+
// one an observation can target rather than failing the read the agent will only repeat.
211236
var windowReady = false
212-
for _ in 0..<20 {
237+
for _ in 0..<80 {
213238
try checkCancellation()
214-
if let windows = attribute(root, kAXWindowsAttribute) as? [AXUIElement], !windows.isEmpty { windowReady = true; break }
239+
if hasObservableWindow(app.processIdentifier) { windowReady = true; break }
215240
guard !app.isTerminated else { throw ComputerError("app_unavailable", "App exited while opening its window.") }
216241
try await Task.sleep(for: .milliseconds(100))
217242
}
@@ -231,6 +256,23 @@ func windowsFor(_ pid: pid_t) -> [[String: Any]] {
231256
guard let entries = CGWindowListCopyWindowInfo([.optionAll, .excludeDesktopElements], kCGNullWindowID) as? [[String: Any]] else { return [] }
232257
return entries.filter { ($0[kCGWindowOwnerPID as String] as? NSNumber)?.int32Value == pid && ($0[kCGWindowLayer as String] as? Int) == 0 }
233258
}
259+
/// Whether the app has a window an observation can list: an Accessibility window whose frame matches
260+
/// one of its window-list entries, the match `get_app_state` uses. A minimized window counts, since it
261+
/// is listed and `activate_app` can bring it forward.
262+
func hasObservableWindow(_ pid: pid_t) -> Bool {
263+
let root = AXUIElementCreateApplication(pid)
264+
AXUIElementSetMessagingTimeout(root, 0.1)
265+
guard let windows = attribute(root, kAXWindowsAttribute) as? [AXUIElement] else { return false }
266+
let frames = windows.compactMap { window -> CGRect? in
267+
AXUIElementSetMessagingTimeout(window, 0.1)
268+
return elementRect(window)
269+
}
270+
return windowsFor(pid).contains { entry in
271+
guard let bounds = entry[kCGWindowBounds as String] as? [String: Any],
272+
let rect = CGRect(dictionaryRepresentation: bounds as CFDictionary), rect.width > 0, rect.height > 0 else { return false }
273+
return frames.contains(rect)
274+
}
275+
}
234276
func windowRect(_ id: String?, pid: pid_t) throws -> CGRect {
235277
let id = try required(id, "windowId")
236278
guard let number = UInt32(id), let entry = windowsFor(pid).first(where: { ($0[kCGWindowNumber as String] as? NSNumber)?.uint32Value == number }),
@@ -872,7 +914,7 @@ final class Driver {
872914
guard (names as? [String] ?? []).contains(action) else { throw ComputerError("unsupported_action", "Action was not advertised by the selected element.") }
873915
// Actions that open UI (a sheet, a menu) hold the reply like a press; others fail as uncertain on timeout.
874916
let opensUI = [kAXPressAction, kAXPickAction, kAXShowMenuAction].contains(action)
875-
try dispatchAX(element, press: opensUI) { AXUIElementPerformAction($0, action as CFString) }
917+
try performAXAction(element, action, press: opensUI)
876918
if action == kAXPressAction || action == kAXPickAction { try await completeMenuPress(element, pid: pid) }
877919
case "set_value":
878920
let element = try snapshot.element(p.elementId); let value = try required(p.value, "value")
@@ -884,7 +926,7 @@ final class Driver {
884926
let button = p.button ?? "left"; let count = p.clickCount ?? 1
885927
guard ["left", "right"].contains(button), (1...3).contains(count) else { throw ComputerError("invalid_arguments", "Invalid button or clickCount.") }
886928
if p.elementId != nil && button == "left" && count == 1, let element = try? snapshot.element(p.elementId), axActions(element).contains(kAXPressAction) {
887-
try dispatchAX(element, press: true) { AXUIElementPerformAction($0, kAXPressAction as CFString) }
929+
try performAXAction(element, kAXPressAction, press: true)
888930
try await completeMenuPress(element, pid: pid)
889931
} else {
890932
try await coordinateAction { dispatch in

0 commit comments

Comments
 (0)