Skip to main content

Celonis Product Documentation

Capturing UI Automation data

Microsoft UI Automation (UIA) lets applications expose accessibility interfaces so that clients can consume information from them. The Task Mining Client software acts as a UIA client to capture the data that third party applications provide to this API. Whenever a user interacts with a UIA-enabled screen element of an application that exposes UIA attributes, data such as the name and value of a field can be captured and stored. Applications based on Win32 and .NET programs typically expose UIA attributes.

To make the Task Mining Client software capture data from UIA attributes, enable these client settings in your Task Mining project:

  • If you’re using the basic settings, enable the setting Context of application or webpage in the Captured Details section. This setting is disabled by default. When you enable it, the Task Mining Client software captures data from UIA attributes and from some other APIs that provide context.

  • If you’re using the advanced settings in the configuration editor, enable the setting Use UIAA in the general logging options in the Logging section. This setting is enabled by default.

In the advanced settings in the configuration editor, you can configure UIA data capture for specific applications.

  • You can configure UIA data capture for an application using event processing rules. In the Logging section of the visual editor for an event processing rule that covers the application, you can check or uncheck specific logging fields that you do or don’t want to capture for the application. The UIA attributes have the suffix _UIAA.

  • You can exclude an application completely from UIA data capture by adding its Windows process name (case-sensitive, and without the .exe file extension) to the Applications to exclude from UIAA box. This setting is in the general logging options in the Logging section. Finding the process name for a specific application tells you how to find the process name. The Task Mining Client software doesn’t send UIA requests to excluded applications. We exclude Microsoft Excel by default.

Important

The data that third party applications provide to Microsoft UI Automation is up to the implementer of the application. One application might populate a specific attribute, and another application might not. The quality of data provided for an attribute can vary from very detailed to minimal. The Task Mining Client software cannot know in advance what data a specific application will provide for a UIA attribute, and in what format.

Because of this inconsistency, if you intend to use UIA data capture in your Task Mining project, we recommend that you carry out preliminary testing to find what quality of data your target application provides. Start with a small representative user sample, validate that the data provided by the application is helpful for your project, then roll out to more users. This is particularly important if you are working with an in-house custom application, as we won’t have previously tested that with Task Mining.

In addition to unhelpful or missing UI Automation data, you might find these issues when testing with your target application:

  • If a user action closes the active window, or if the element that the user interacted with disappears immediately after the interaction, the Task Mining Client software cannot capture the UIA data reliably. The removal of the element means that the information on it might already be gone by the time the client queries it. For example, this can happen when the user clicks the Close button in the upper right corner of a window, or the Send button in Microsoft Outlook, or a button that is hidden after it is clicked.

  • UIA data capture might cause a noticeable performance slowdown for some applications in some situations, as the application must provide the UIA data to the client as well as doing its standard task. In our testing for standard applications, we found that large Excel files might have this issue, so by default the Task Mining Client software does not send UIA requests to Microsoft Excel. In the advanced settings in the configuration editor, you can specify applications to be excluded from UIA data capture.