UI Browser icon

UI Browser — Support

The ultimate assistant for Accessibility and GUI Scripting

Scroll ↕

For detailed instructions and more information about using UI Browser, launch UI Browser and choose Help > UI Browser Help. If you have questions that aren't answered in the Help book or on this website, send email to us at support@pfiddlesoft.com.

Resources for GUI Scripting

Consult these additional sources of information about GUI scripting:

  • Two popular books about AppleScript include in-depth discussions of GUI Scripting: Lesson 28 in "Apple Training Series: AppleScript 1-2-3" by Sal Soghoian and Bill Cheeseman (Peachpit Press, 2009), and Chapter 24 in "AppleScript: The Definitive Guide" by Matt Neuburg (Second Edition, O'Reilly, 2006).
  • For a complete GUI Scripting syntax reference, open the terminology dictionary of Apple's System Events application in your favorite script editor. System Events is located at /System/Library/CoreServices/System Events.app. GUI Scripting terminology is defined in the Processes Suite in the System Events dictionary.
  • Several useful and informative sample GUI Scripting scripts are installed by the UI Browser installer package in the Local Scripts folder at /Library/Scripts/UI Element Scripts (PFiddlesoft). If the Scripts menu extra is installed in the menu bar, you can reach them by choosing Scripts > Open Scripts Folder > Open Computer Scripts Folder > UI Element Scripts (PFiddlesoft).
  • Apple provides additional sample GUI Scripting scripts. You can find them on your computer in the Local Scripts folder at /Library/Scripts/UI Element Scripts or, if Apple's Scripts menu extra is installed in your menu bar, by choosing Scripts > Open Scripts Folder > Open Computer Scripts Folder > UI Element Scripts.
  • A brief description of GUI Scripting and a few sample scripts are available at Graphic User Interface (GUI) Scripting, part of The macOS Automation Sites managed by Sal Soghoian, Apple's former product manager for AppleScript and automation technologies. At this writing, the downloadable scripts are mostly obsolete but demonstrate how to script System Preferences settings that are not scriptable on their own.
  • Don't forget to read the UI Browser Help book! There's a wealth of information available under UI Browser's Help menu.
  • Read the GUI Scripting Issues and Tips sections, below.

If you aren't sure that GUI Scripting will meet your needs, take advantage of UI Browser's free 30-day trial period to verify that UI Browser and GUI Scripting will work with your target applications.

GUI Scripting Issues

AppleScript script statements generated using UI Browser's AppleScript menu sometimes fail to work even though UI Browser generates valid GUI Scripting statements using correct syntax as defined by Apple. There are two main reasons for this:

  • Some applications do not fully or correctly implement Apple's Accessibility API, on which GUI Scripting depends. A number of well-established Mac OS X applications were originally written using older toolbox code for buttons, menus, and other UI elements. As a result, they can be unresponsive to UI Browser and GUI Scripting. As time goes by, these applications are likely to be updated to incorporate more modern Cocoa and Carbon UI element code, which incorporates full Accessibility support, or to add custom Accessibility support of their own. If you encounter an application that does not appear to work well with UI Browser or the scripts that it generates, this may be the reason.
  • The Accessibility API sometimes doesn't work perfectly. For example, in Mac OS X v10.3 Panther, the statement 'get attribute "AXParent" of scroll area 1 of window 1' returned no result, when it should have returned the UI element object representing window 1. Mac OS X v10.4 Tiger resolved this and all similar UI element navigation issues. Similarly, in Mac OS X v10.5 Leopard and older, text inserted into a text area or text field often failed to "stick", disappearing when the user clicked into another text area or text field. Mac OS X v10.6 Snow Leopard resolved this issue.

GUI Scripting Tips

These are general tips on GUI Scripting, not necessarily related to UI Browser.

Where can I find sample scripts for GUI Scripting?

UI Browser comes with a collection of advanced sample scripts that showcase GUI Scripting features and techniques. These include, for example, scripts showing how to script the menu items in Dock items and in menu extras (menu extras are the menus at the right end of the menu bar). These UI Browser sample scripts are installed in the UI Element Scripts (PFiddlesoft) folder in the Local Scripts folder at /Library/Scripts/UI Element Scripts (PFiddlesoft). You may find several sample scripts from Apple in the same location, at /Library/Scripts/UI Element Scripts. Several sample scripts are available online at the The macOS Automation Sites' Graphic User Interface (GUI) Scripting page. Finally, read Lesson 28 in "Apple Training Series: AppleScript 1-2-3" by Sal Soghoian and Bill Cheeseman (Peachpit Press, 2009), and Chapter 24 in "AppleScript: The Definitive Guide" by Matt Neuburg (Second Edition, O'Reilly, 2006).

Why do GUI Scripting scripts start with nested double tell blocks?

GUI Scripting scripts usually start with a 'tell application' statement followed by a nested 'tell process' block. These statements can be generated automatically from UI Browser's AppleScript menu using the Tell Block Wrapper (Short) or Tell Block Wrapper (Safe) commands.

It is easy to assume that this is just some magic incantation and stop thinking about it, but in fact it is an example of standard AppleScript technique.

You will understand GUI Scripting better if you realize that the 'project' in the second, nested statement is simply an AppleScript object implemented by the System Events application. It is the same as any other AppleScript object, complete with elements, properties, and commands. For example, when you script the Finder, you might tell the Finder to tell the front window to get its position. In such a script, the Finder is the targeted application, the front window is an AppleScript object known to the Finder, and its position is a property of the window object. When you script System Events, you might tell System Events to tell process "Finder" to tell its front window to get its position. In such a script, System Events is the targeted application, the Finder process is an AppleScript object that System Events understands, the Finder's front window is an AppleScript object that the Finder process understands, and its position is a property of the window object.

Will my GUI Scripts work in Mac OS X v10.2 Jaguar or older?

No, unless the computer includes some special beta supporting software that Apple made available for testing under Jaguar.

The scripts generated by UI Browser require Apple's GUI Scripting technology for AppleScript. Starting with Mac OS X v10.3 Panther, the required supporting software is installed automatically, by default, on every Macintosh computer. Apple successfully concluded the GUI Scripting public beta test for Jaguar, and the beta version of the underlying GUI Scripting software for Jaguar is no longer available.

In the unlikely event that you have the Jaguar beta software and want to write scripts for it, you need to be aware that the AppleScript dictionary for GUI Scripting in the beta System Events application was somewhat different from the release version. For example, the beta dictionary defined 'check box' with a space for Jaguar, but in the final version for Panther it defines 'checkbox' without a space. Since the Jaguar version was beta software and is no longer officially available from Apple, UI Browser 1.4 and newer generates scripts for a 'checkbox' using the Panther terminology. If you are still using Jaguar with Apple's beta GUI Scripting software, edit generated 'checkbox' scripts to add a space by hand. Bear in mind that such a script may not work when it is run under Panther or newer.

Another example is the 'keystroke' command when it is used with modifier keys. The correct form for Jaguar was 'keystroke "n" with {command down}'; for Panther and newer, this was changed to 'keystroke "n" using {command down}'.

My script seems to get ahead of the UI. Any suggestions?

Try the 'delay' command. Under Mac OS X v10.3 Panther and newer, however, GUI Scripting is so much faster than it was with the Jaguar System Events public beta that you probably won't need to do this very often.

How do I script an Application's palettes?

Scripts that read or manipulate UI elements in a target application require special design to handle palettes, because a palette is automatically hidden when another application is activated. If you switch to Script Editor to run a script, for example, the target application's palettes become invisible and thus inaccessible to your script. Your script should therefore be written so that it activates the target application before attempting to read or manipulate palettes. In this way, the target application remains active and visible, and all of its UI elements can be scripted.

The same design principle applies to sheets, utility windows, inspectors, and some other kinds of windows that disappear when the application is not frontmost on the screen.

UI Browser is particularly helpful in these situations, because its hot keys allow you to select UI elements in a window while the target application is in front. In addition, almost every control in UI Browser can be used while you hold the Command key down to keep UI Browser in the background so that a palette will remain visible. The handling of menu

How do I script the menu bar using GUI Scripting?

Apple changed the way the menu hierarchy is exposed in Mac OS X v10.3 Panther. Furthermore, the terminology in Panther differed from that in Jaguar, and, to confuse matters still further, in Panther the terminology differed as between the Accessibility API and GUI Scripting. Things became more logical sometime after Panther.

In Jaguar, the children of 'menu bar 1' were 'menu' elements, and each 'menu' element was conceptually a submenu having one or more 'menu item' elements as children. Each 'menu item' element represented a command on which you could click. Some 'menu item' elements had a single child element of the same name that was a 'menu' element, that is to say, an hierarchical submenu containing one or more 'menu item' elements. The 'menu item'/'menu' pairings continued down the hierarchy if there are submenus.

In Panther, a new level in the hierarchy was introduced immediately below the menu bar, called 'menu title' in the Accessibility API and 'menu bar item' in GUI Scripting. Apple did this to make the hierarchy that is exposed in the Accessibility API and in GUI Scripting correspond exactly to the underlying reality in the system. Thus, in Panther, the children of 'menu bar 1' became 'menu title' elements (addressed as 'menu bar item' elements in GUI Scripting). Conceptually, the 'menu bar' was actually a 'menu' containing one or more menu items, each of which was known as a 'menu title' or 'menu bar item' element. Each 'menu title' ('menu bar item') element had a single 'menu' element child of the same name, which was the submenu holding 'menu item' elements at the next level down the hierarchy. If there were submenus below that, they followed the same 'menu item'/'menu' pairings as in Jaguar.

Now, the Accessibility and GUI Scripting terms are the same, a 'menu bar item' is a 'menu bar item', and all is right with the world.

How do I write scripts to send keyboard shortcuts?

Choose a target application in UI Browser, select its application element (or any other element) in UI Browser's browser view, open the Keystrokes drawer, type the desired key (taking care to use a lower-case character instead of an upper-case character, in the case of alphabetic characters), and check the Command key checkbox. Then open the AppleScript menu and scroll down to see the 'Send Defined Keystroke' command highlighted. This will open the Edit AppleScript window with the desired GUI Scripting statement selected and ready to copy or drag to a script editor. If you select appropriate UI Browser preferences, the generated script will already be on the clipboard, or it will be sent directly to your chosen script editor and inserted at the insertion point.

There is also a 'Send Defined Key Code' command in the AppleScript menu, which enables you to send function keys as well as character keys. The appropriate key code is displayed in the Keystrokes drawer when you press the desired function key (or any other key).

Keystroke and key code commands are always sent to the active, or focused, UI element in the target application. In the case of keyboard shortcuts, they are always sent to the target application itself, if it recognizes that shortcut. When sending keystrokes or key codes, it is never necessary to send it to a specific UI element.

The 'keystroke' command should be sent with a 'command down' parameter in a list created using AppleScript's curly braces syntax, like this: 'keystroke "n" using {command down}'. You can send the 'keystroke' command with multiple modifier keys separated by commas in the list, for keyboard shortcuts that take multiple modifier keys, like this: 'keystroke "n" using {shift down, command down}'. The character should always be lower case; if you use an upper case letter, GUI Scripting will think that the Shift key is down and the application will not respond as you intend.

In GUI Scripting, you can use the 'keystroke' command to send a string of characters, as long as they are all lowercase or all uppercase. UI Browser's Keystrokes drawer only lets you send one character at a time, because this is the way the Accessibility API works.

Why are the names of some UI elements different in a script from their names in UI Browser?

The names of some UI elements as they appear in UI Browser's main browser view differ from their AppleScript names as they appear in scripts generated using UI Browser's AppleScript menu. This is because the AppleScript terminology used in Apple's System Events dictionary for GUI Scripting is not identical to the terminology used by the Accessibility API. In UI Browser, we use the Accessibility names except when generating AppleScript statements, because UI Browser is a developer tool for the Accessibility API as well as a tool for GUI Scripting.

Why do some applications require a process name that is different from the application name?

In GUI Scripting, you usually first 'activate' the target application, and you then use nested 'tell' blocks to tell the System Events application to send commands to the target application's process. The 'activate' command is sent to the application itself, as in 'activate application "Adobe Photoshop CS2.app"'. In the inner 'tell' block of the nested 'tell' blocks, you tell System Events to tell the application's process to perform a command. In the case of a few applications, the name of the process is not the same as the name of the application; for example, 'tell process "Photoshop"'.

UI Browser knows which applications have different application and process names. For this reason, the safest course is to use UI Browser's AppleScript menu to generate 'tell' blocks for GUI Scripting. UI Browser always gets the names right.