Is it possible to use macOS Accessibility API features from a CLI or library?

128 views Asked by At

I am working on an application that needs to leverages macOS Accessibility APIs to read the selected text in any application. I will call a Swift library from Rust via FFI.

I am able to get a POC working as part of a SwiftUI application, but when I try the same code from a command line Swift application (or call it via FFI), I can no longer access information about other applications.

For example, using the function below, I can get the text selected in Messages if I trigger it via my SwiftUI app. If I run it as a command line app, it still prints trusted: true, but AXUIElementCopyAttributeValue(systemWideElement, kAXFocusedUIElementAttribute as CFString, &focusedElement) returns a cannotComplete AXError.

func getSelectedText() {
    print("trusted: \(AXIsProcessTrusted())")
    sleep(2) // switch windows and select text
    
    let systemWideElement = AXUIElementCreateSystemWide()
    
    var focusedElement: AnyObject?
    let err = AXUIElementCopyAttributeValue(systemWideElement, kAXFocusedUIElementAttribute as CFString, &focusedElement)
    
    if err != .success {
        print("Unable to get focused element: ", String(reflecting: err))
        return
    }
    
    var selectedText: AnyObject?
    AXUIElementCopyAttributeValue(focusedElement as! AXUIElement, kAXSelectedTextAttribute as CFString, &selectedText)
    
    print(selectedText as? String ?? "No selected text found")
}

I'm new to programming for macOS in general and would appreciate any direction here. Thanks!

0

There are 0 answers