PFiddlesoft logo
Home Overview Features Screenshots Comparison Store Press Support


UI Browser application icon

UI Browser


For detailed instructions and more information, launch UI Browser and choose Help > UI Browser Help. If you have questions that aren't answered in the Help book or on this website, send email to us at

Tech Support notes about GUI Scripting

GUI Scripting isn't terribly well documented by Apple. Consult these sources of information from Apple and from third parties:

If you aren't sure that GUI Scripting will meet your needs, take advantage of UI Browser's free 30-day trial period to verify that UI Browser and GUI Scripting will work with your target applications.

Known issues in GUI Scripting

AppleScript script statements generated using UI Browser's AppleScript menu sometimes fail to work even though UI Browser generates valid GUI Scripting statements using correct syntax as defined by Apple. There are two main reasons for this:

GUI Scripting tips

These are general tips on GUI Scripting, not necessarily related to UI Browser.

Where can I find sample scripts for GUI Scripting?

UI Browser comes with a collection of advanced example scripts that showcase GUI Scripting features. These include, for example, scripts showing how to script the commands in Dock items and in menu extras (menu extras are the menus at the right end of the menu bar). These UI Browser example scripts are in the UI Element Scripts (PFiddlesoft) folder on the UI Browser installation disk image. Drag the folder into ~/Library/Scripts for easy reference. You will find several sample scripts from Apple in the same location, at ~/Library/Scripts/UI Element Scripts. Several sample scripts are available online at the Mac OS X Automation website. Finally, read Lesson 28 in "Apple Training Series: AppleScript 1-2-3" by Sal Soghoian and Bill Cheeseman (Peachpit Press, 2009), and Chapter 24 in "AppleScript: The Definitive Guide" by Matt Neuburg (Second Edition, O'Reilly, 2006).

Why do GUI Scripting scripts start with nested double tell blocks?

GUI Scripting scripts usually start with a 'tell application' statement followed by a nested 'tell process' block. These statements can be generated automatically from UI Browser's AppleScript menu using the Tell Block Wrapper (Short) or Tell Block Wrapper (Safe) commands. It is easy to assume that this is just some magic incantation and stop thinking about it.

You will understand GUI Scripting better if you realize that the 'project' in the second, nested statement is simply an AppleScript object implemented by the System Events application. It is the same as any other AppleScript object, complete with elements, properties, and commands. For example, when you script the Finder, you might tell the Finder to tell the front window to get its position. In such a script, the Finder is the targeted application, the front window is an AppleScript object known to the Finder, and its position is a property of the window object. When you script System Events, you might tell System Events to tell process "Finder" to tell its front window to get its position. In such a script, System Events is the targeted application, the Finder process is an AppleScript object that System Events understands, the Finder's front window is an AppleScript object that the Finder process understands, and its position is a property of the window object.

Will my GUI Scripts work in Mac OS X 10.2 "Jaguar" or older?

No, unless the computer includes some special beta supporting software that Apple made available for testing under Jaguar.

The scripts generated by UI Browser require Apple's GUI Scripting technology for AppleScript. Starting with Mac OS X 10.3 Panther, the required supporting software is installed automatically, by default, on every Macintosh computer. Apple successfully concluded the GUI Scripting public beta test for Jaguar, and the beta version of the underlying GUI Scripting software for Jaguar is no longer available.

If you have the Jaguar beta software and want to write scripts for it, you need to be aware that the AppleScript dictionary for GUI Scripting, in the System Events application, was somewhat different then. For example, the beta dictionary defined 'check box' with a space for Jaguar, but in the final version for Panther it defines 'checkbox' without a space. Since the Jaguar version was beta software and it is no longer officially available from Apple, UI Browser 1.4 generates scripts for a 'checkbox' using the Panther terminology. If you are still using Jaguar with Apple's beta GUI Scripting software, edit generated 'checkbox' scripts to add a space by hand. Bear in mind that such a script may not work when it is run under Panther.

Another example is the 'keystroke' command when it is used with modifier keys. The correct form for Jaguar is 'keystroke "n" with {command down}'; for Panther and newer, this is changed to 'keystroke "n" using {command down}'.

My script seems to get ahead of the UI. Any suggestions?

Try the 'delay' command. Under Mac OS X 10.3 Panther and newer, however, GUI Scripting is so much faster than it was with the Jaguar System Events public beta that you probably won't need to do this very often.

How do I script an Application's palettes?

Scripts that read or manipulate UI elements in a target application's palette require special design, because a palette window automatically becomes invisible when another application is activated. If you switch to Script Editor to run a script, the target application's preferences window becomes invisible and thus inaccessible to your script. Your script should therefore be written so that it activates the target application and opens the palette before attempting to read or manipulate its UI elements. In this way, the target application remains active and visible, and UI elements in the dialog can be scripted.

The same design principle applies to sheets, utility windows, inspectors, and some other kinds of windows that disappear when the application is not frontmost on the screen.

UI Browser is particularly helpful in these situations, because its hot keys allow you to select UI elements in a window while the target application is in front, and its ability to navigate UI elements that have been "destroyed" lets you bring UI Browser to the front and generate scripts for them even after the sheet or other container has disappared. In addition, almost every control in UI Browser can be used while you hold the Command key down to keep UI Browser in the background so that a palette will remain visible.

How do I script the menu bar using GUI Scripting?

Apple changed the way the menu hierarchy is exposed in Mac OS X 10.3 Panther. Furthermore, the terminology in Panther differs from that in Jaguar, and, to confuse matters still further, in Panther the terminology differs as between the Accessibility API and GUI Scripting. Things became more logical sometime after Panther.

In Jaguar, the children of 'menu bar 1' were 'menu' elements, and each 'menu' element was conceptually a submenu having one or more 'menu item' elements as children. Each 'menu item' element represented a command on which you could click. Some 'menu item' elements had a single child element of the same name that was a 'menu' element, that is to say, an hierarchical submenu containing one or more 'menu item' elements. The 'menu item'/'menu' pairings continued down the hierarchy if there are submenus.

In Panther, a new level in the hierarchy was introduced immediately below the menu bar, called 'menu title' in the Accessibility API and 'menu bar item' in GUI Scripting. Apple did this to make the hierarchy that is exposed in the Accessibility API and in GUI Scripting correspond exactly to the underlying reality in the system. Thus, in Panther, the children of 'menu bar 1' became 'menu title' elements (addressed as 'menu bar item' elements in GUI Scripting). Conceptually, the 'menu bar' was actually a 'menu' containing one or more menu items, each of which was known as a 'menu title' or 'menu bar item' element. Each 'menu title' ('menu bar item') element had a single 'menu' element child of the same name, which was the submenu holding 'menu item' elements at the next level down the hierarchy. If there were submenus below that, they followed the same 'menu item'/'menu' pairings as in Jaguar.

Now, the Accessibility and GUI Scripting terms are the same, a 'menu bar item' is a 'menu bar item', and all is right with the world.

How do I write scripts to send keyboard shortcuts?

Choose a target application in UI Browser, select its application element (or any other element) in UI Browser's browser view, open the Keystrokes drawer, type the desired key (taking care to use a lower-case character instead of an upper-case character, in the case of alphabetic characters), and check the Command key checkbox. Then open the AppleScript menu and scroll down to see the 'Send Defined Keystroke' command highlighted. This will open the Edit AppleScript window with the desired GUI Scripting statement selected and ready to copy or drag to a script editor. If you select appropriate UI Browser preferences, the generated script will already be on the clipboard, or it will be sent directly to your chosen script editor and inserted at the insertion point.

There is also a 'Send Defined Key Code' command in the AppleScript menu, which enables you to send function keys as well as character keys. The appropriate key code is displayed in the Keystrokes drawer when you press the desired function key (or any other key).

Keystroke commands are always sent to the active UI element in the target application. In the case of keyboard shortcuts, they are always sent to the target application itself, if it recognizes that shortcut.

The 'keystroke' command should be sent with a 'command down' parameter in a list created using AppleScript's curly braces syntax, like this: 'keystroke "n" using {command down}'. You can send the 'keystroke' command with multiple modifier keys separated by commas in the list, for keyboard shortcuts that take multiple modifier keys, like this: 'keystroke "n" using {shift down, command down}'. The character should always be lower case; if you use an upper case letter, GUI Scripting will think that the Shift key is down and the application will not respond as you intend.

In GUI Scripting, you can use the 'keystroke' command to send a string of characters, as long as they are all lowercase or all uppercase. UI Browser's Keystroke drawer only lets you send one character at a time, because this is the way the Accessibility API works.

Why are the names of some UI elements different in a script from their names in UI Browser?

The names of some UI elements as they appear in UI Browser's main browser view differ from their AppleScript names as they appear in scripts generated using UI Browser's AppleScript menu. This is because the AppleScript terminology used in Apple's System Events dictionary for GUI Scripting is not identical to the terminology used by the Accessibility API. In UI Browser, we use the Accessibility names except when generating AppleScript statements, because UI Browser is a developer tool for the Accessibility API as well as a tool for GUI Scripting.

Why do some applications require a process name that is different from the application name?

In GUI Scripting, you usually first 'activate' the target application, then use nested 'tell' blocks to tell the System Events application to send commands to the target application's process. The 'activate' command is sent to the application itself, as in 'activate application "Adobe Photoshop"'. In the inner 'tell' block of the nested 'tell' blocks, you tell System Events to tell the application's process to perform a command. In the case of a few applications, the name of the process is not the same as the name of application; for example, 'tell process "Photoshop"'.

UI Browser knows which applications have different application and process names. For this reason, the safest course is to use UI Browser's AppleScript menu to generate 'tell' blocks for GUI Scripting. UI Browser always gets the names right.

This page was first published by PFiddlesoft on May 25, 2010.
Copyright © 2003-2010 Bill Cheeseman. Used by permission. All Rights Reserved.
PFiddlesoft, PFiddle Software, pfiddle, pfiddles, the PFiddlesoft logo, Wheel of Access, and Applidude are trademarks of PreForm Assistive Technologies, LLC.