Topics

FW: [win10] Version 1709 and beyond: screen readers must support uIA notification event in order to announce various things from apps #winten1709 #winten1803 #advisory


 

Hi all,

 

First, apologies for mentioning competition, but I felt it is best to do so given the overall context. Second, the advisory is deemed major (to severe for some cases) and is applicable for people running Windows 10 Fall Creators Update and beyond. The source of this advisory is a Win10 forum hosted on Groups.IO at https://win10.groups.io/g/win10.

 

Thanks.

 

From: win10@win10.groups.io [mailto:win10@win10.groups.io] On Behalf Of Joseph Lee
Sent: Friday, February 23, 2018 9:42 AM
To: win10@win10.groups.io
Subject: [win10] Version 1709 and beyond: screen readers must support uIA notification event in order to announce various things from apps #ADVISORY #WinTen1709 #WinTen1803

 

Hi everyone,

 

The following advisory is considered major in that it impacts all known screen readers )except for one or two; feel free to pass this on, although please do cite where you’ve got the advisory from):

 

Problem: notifications from apps such as Calculator, Microsoft Store and others are not announced by screen readers.

 

Background: in Windows 10 Version 1709 and later, if apps are written correctly, they can send notification texts to be announced by screen readers. A UIA client such as a screen reader must be able to catch this in order to announce certain notifications, including Calculator’s display window, app update status in Microsoft Store, various notifications from Edge, Alarms and Clock and other apps.

 

Steps to reproduce:

 

  1. While using Windows 10 Version 1709 or later (including current Insider Preview builds), launch an app such as Alarms and Clock (be sure it is the latest version).
  2. In case of Alarms and Clock, go to Stopwatch tab and start and stop the stopwatch.
  3. For Calculator, do some calculations.

 

Expected: screen reader announces messages such as “Stopwatch stopped” in Alarms and Clock and “display is something” in Calculator.

Actual: only Narrator announces such texts.

Cause: other screen readers do not track UIA notification event. This is due to the fact that Version 1709 introduces this event, which requires tracking a newer UIA interface.

 

Known solutions/mitigations/workarounds and impact:

 

  • Narrator: green light/okay; works out of the box (requires Version 1709 and later).
  • JAWS for Windows: red light/major; no support (version 2018.1802.78 did not solve the Calculator display text problem, for instance).
  • NonVisual Desktop Access: yellow light/solution being tested (through an add-on update).

 

This advisory will be closed once major screen readers (in addition to the three listed above) provides at least a basic solution.

 

Cheers,

Joseph


Steve Matzura
 

Joseph,


Here's a strange one for you. I posted this before, but it might have been in a thread in which you weren't interested. But I bet this will be of interest. It's regarding Calculator.


I have two Windows 10 systems, one generated from version 1603 and up-to-date as of this morning, and another generated from the version 1709 Fall Creators Update, also up-to-date as of this morning. On the newer system--the one generated from the FCU--Calculator does not read results automatically JAWS Key + Q reports the application name is Microsoft.WindowsCalculator. On the older system--the one generated from version 1603--calculation results are read correctly, but on that system, the application name is ApplicationFrame.dll. What do you make of that difference--not just the application name difference, but the screenreader behavior difference, too?


On 2/23/2018 12:46 PM, Joseph Lee wrote:

Hi all,

 

First, apologies for mentioning competition, but I felt it is best to do so given the overall context. Second, the advisory is deemed major (to severe for some cases) and is applicable for people running Windows 10 Fall Creators Update and beyond. The source of this advisory is a Win10 forum hosted on Groups.IO at https://win10.groups.io/g/win10.




 

Hi,

It could be both JAWS and the app. If you have latest JAWS 2018 (or 18.0) build, Calculator results might not be read (or at least for me and others, it does not) due to the fact that the app raises a UIA event JAWS doesn’t know about. I thought this is fixed in build 1802.78 but it apparently wasn’t.

Cheers,

Joseph

 

From: main@jfw.groups.io [mailto:main@jfw.groups.io] On Behalf Of Steve Matzura
Sent: Friday, February 23, 2018 10:14 AM
To: main@jfw.groups.io
Subject: Re: FW: [win10] Version 1709 and beyond: screen readers must support uIA notification event in order to announce various things from apps #ADVISORY #WinTen1709 #WinTen1803

 

Joseph,

 

Here's a strange one for you. I posted this before, but it might have been in a thread in which you weren't interested. But I bet this will be of interest. It's regarding Calculator.

 

I have two Windows 10 systems, one generated from version 1603 and up-to-date as of this morning, and another generated from the version 1709 Fall Creators Update, also up-to-date as of this morning. On the newer system--the one generated from the FCU--Calculator does not read results automatically JAWS Key + Q reports the application name is Microsoft.WindowsCalculator. On the older system--the one generated from version 1603--calculation results are read correctly, but on that system, the application name is ApplicationFrame.dll. What do you make of that difference--not just the application name difference, but the screenreader behavior difference, too?

 

On 2/23/2018 12:46 PM, Joseph Lee wrote:

Hi all,

 

First, apologies for mentioning competition, but I felt it is best to do so given the overall context. Second, the advisory is deemed major (to severe for some cases) and is applicable for people running Windows 10 Fall Creators Update and beyond. The source of this advisory is a Win10 forum hosted on Groups.IO at https://win10.groups.io/g/win10.

 

 


 

In Jaws you can simply do a say line to read the calculator result. I know this is not how it should be, but until FS fixes this which I don’t doubt will happen this is an easy work-around.

 

 

From: main@jfw.groups.io <main@jfw.groups.io> On Behalf Of Joseph Lee
Sent: Friday, February 23, 2018 12:39 PM
To: main@jfw.groups.io
Subject: Re: FW: [win10] Version 1709 and beyond: screen readers must support uIA notification event in order to announce various things from apps #ADVISORY #WinTen1709 #WinTen1803

 

Hi,

It could be both JAWS and the app. If you have latest JAWS 2018 (or 18.0) build, Calculator results might not be read (or at least for me and others, it does not) due to the fact that the app raises a UIA event JAWS doesn’t know about. I thought this is fixed in build 1802.78 but it apparently wasn’t.

Cheers,

Joseph

 

From: main@jfw.groups.io [mailto:main@jfw.groups.io] On Behalf Of Steve Matzura
Sent: Friday, February 23, 2018 10:14 AM
To: main@jfw.groups.io
Subject: Re: FW: [win10] Version 1709 and beyond: screen readers must support uIA notification event in order to announce various things from apps #ADVISORY #WinTen1709 #WinTen1803

 

Joseph,

 

Here's a strange one for you. I posted this before, but it might have been in a thread in which you weren't interested. But I bet this will be of interest. It's regarding Calculator.

 

I have two Windows 10 systems, one generated from version 1603 and up-to-date as of this morning, and another generated from the version 1709 Fall Creators Update, also up-to-date as of this morning. On the newer system--the one generated from the FCU--Calculator does not read results automatically JAWS Key + Q reports the application name is Microsoft.WindowsCalculator. On the older system--the one generated from version 1603--calculation results are read correctly, but on that system, the application name is ApplicationFrame.dll. What do you make of that difference--not just the application name difference, but the screenreader behavior difference, too?

 

On 2/23/2018 12:46 PM, Joseph Lee wrote:

Hi all,

 

First, apologies for mentioning competition, but I felt it is best to do so given the overall context. Second, the advisory is deemed major (to severe for some cases) and is applicable for people running Windows 10 Fall Creators Update and beyond. The source of this advisory is a Win10 forum hosted on Groups.IO at https://win10.groups.io/g/win10.

 

 


Steve Matzura
 

But did you not notice the two different application names? One works, one doesn't. That's what I wanted you to comment on--why you think there are two different applications.


On 2/23/2018 2:38 PM, Joseph Lee wrote:

Hi,

It could be both JAWS and the app. If you have latest JAWS 2018 (or 18.0) build, Calculator results might not be read (or at least for me and others, it does not) due to the fact that the app raises a UIA event JAWS doesn’t know about. I thought this is fixed in build 1802.78 but it apparently wasn’t.

Cheers,

Joseph

 

From: main@jfw.groups.io [mailto:main@jfw.groups.io] On Behalf Of Steve Matzura
Sent: Friday, February 23, 2018 10:14 AM
To: main@jfw.groups.io
Subject: Re: FW: [win10] Version 1709 and beyond: screen readers must support uIA notification event in order to announce various things from apps #ADVISORY #WinTen1709 #WinTen1803

 

Joseph,

 

Here's a strange one for you. I posted this before, but it might have been in a thread in which you weren't interested. But I bet this will be of interest. It's regarding Calculator.

 

I have two Windows 10 systems, one generated from version 1603 and up-to-date as of this morning, and another generated from the version 1709 Fall Creators Update, also up-to-date as of this morning. On the newer system--the one generated from the FCU--Calculator does not read results automatically JAWS Key + Q reports the application name is Microsoft.WindowsCalculator. On the older system--the one generated from version 1603--calculation results are read correctly, but on that system, the application name is ApplicationFrame.dll. What do you make of that difference--not just the application name difference, but the screenreader behavior difference, too?

 

On 2/23/2018 12:46 PM, Joseph Lee wrote:

Hi all,

 

First, apologies for mentioning competition, but I felt it is best to do so given the overall context. Second, the advisory is deemed major (to severe for some cases) and is applicable for people running Windows 10 Fall Creators Update and beyond. The source of this advisory is a Win10 forum hosted on Groups.IO at https://win10.groups.io/g/win10.

 

 



 

Hi,

If Task Manager says that both have same executable names, then they are indeed the one and the same. It is a matter of which version of the app is in use.

I do know that JAWS may report DLL names as app names, and that is very inaccurate. The one found in 1709 is correct, as 1607 (I think that’s what you meant to say) is hosted inside a frame DLL (a host window, in this case).

But that’s not really the main culprit: the issue at hand is UIA events. In older versions of apps, they will raise either name change or live region change events, both of which are caught by JAWS, hence it can announce results if told to do so. However, in newer apps (particularly those updated in 2018), JAWS cannot because these apps raise the new UIA notification event, for which JAWS doesn’t know about.

Technical (very geeky here, so my apologies up front): for resident scripters, FSUIA object must be extended to add UIA notification event handler. The thing is, it is unlike anything we’ve seen. Whereas we can find out changes to objects such as focus movement, state changes and what not, notification event is meant to let screen readers announce text, thus it is slightly detached from element handling. The full signature of this UIA COM method is:

result IUIAutomationNotificationEventHandler::HandleNotificationEvent(IUIAutomationElement sender, NotificationKind NotificationKind, NotificationProcessing NotificationProcessing, string displayString, activity activityId) {}

 

Or to put it in a form that is more familiar for those using a certain screen reader:

 

def IUIAutomationNotificationEventHandler_HandleNotificationEvent(sender, NotificationKind, NotificationProcessing, displayString, activityId): pass

 

Sender is the UIA element, NotificationKind is what kind of notification it is (item has been added or removed, action has been completed or aborted, or other notices), NotificationProcessing refers to how screen readers should process incoming text (process all text, announce it when current announcement is completed, etc.), displayString is the notification text, and activityId is the activity associated with the notification (typically something that identifies the UIA element that sent this notification).

 

References:

 

Disclaimer: I do not work for VFO or Microsoft, nor a certified JAWS expert or a Microsoft Valued Professional (MVP) of any kind.

Cheers,

Joseph

 

From: main@jfw.groups.io [mailto:main@jfw.groups.io] On Behalf Of Steve Matzura
Sent: Saturday, February 24, 2018 8:44 AM
To: main@jfw.groups.io
Subject: Re: FW: [win10] Version 1709 and beyond: screen readers must support uIA notification event in order to announce various things from apps #ADVISORY #WinTen1709 #WinTen1803

 

But did you not notice the two different application names? One works, one doesn't. That's what I wanted you to comment on--why you think there are two different applications.

 

On 2/23/2018 2:38 PM, Joseph Lee wrote:

Hi,

It could be both JAWS and the app. If you have latest JAWS 2018 (or 18.0) build, Calculator results might not be read (or at least for me and others, it does not) due to the fact that the app raises a UIA event JAWS doesn’t know about. I thought this is fixed in build 1802.78 but it apparently wasn’t.

Cheers,

Joseph

 

From: main@jfw.groups.io [mailto:main@jfw.groups.io] On Behalf Of Steve Matzura
Sent: Friday, February 23, 2018 10:14 AM
To: main@jfw.groups.io
Subject: Re: FW: [win10] Version 1709 and beyond: screen readers must support uIA notification event in order to announce various things from apps #ADVISORY #WinTen1709 #WinTen1803

 

Joseph,

 

Here's a strange one for you. I posted this before, but it might have been in a thread in which you weren't interested. But I bet this will be of interest. It's regarding Calculator.

 

I have two Windows 10 systems, one generated from version 1603 and up-to-date as of this morning, and another generated from the version 1709 Fall Creators Update, also up-to-date as of this morning. On the newer system--the one generated from the FCU--Calculator does not read results automatically JAWS Key + Q reports the application name is Microsoft.WindowsCalculator. On the older system--the one generated from version 1603--calculation results are read correctly, but on that system, the application name is ApplicationFrame.dll. What do you make of that difference--not just the application name difference, but the screenreader behavior difference, too?

 

On 2/23/2018 12:46 PM, Joseph Lee wrote:


Hi all,

 

First, apologies for mentioning competition, but I felt it is best to do so given the overall context. Second, the advisory is deemed major (to severe for some cases) and is applicable for people running Windows 10 Fall Creators Update and beyond. The source of this advisory is a Win10 forum hosted on Groups.IO at https://win10.groups.io/g/win10.