moderated jaws being smarter requiring less scripts


Josh Kennedy
 

Jhi
I have a comment and questionn. and lets take vlc media player for example. In vlc media player, if I use the jaws invisible vursor and go to the bottom of the screen and up arrow once, I am on the current time in the file I am listening to. Now if I fast forward or rewind through that file with control left and right arrow keys or alt left and right arrows, jaws won't read that information unless I somehow script it which at this point I don't know how to do because of the jaws scan cursor and almost script functions for it and script utilities not, or seeming to not, give me access to the status bar and any buttons. 
OK so my question is this. Why can't jaws be much smarter than it is? Why can't jaws look at the screen of an applications, find out what is happening when keys are pressed and maybe not on the first try but on subsequent tries, analyse the screen to determine what is changing and say... ok well this is changing and a sighted person may be interested in seeing this, therefore it may be relevant for a blind person also and since this is a media player its probably a time index for the file so lets monitor this and read it out for the blind person after first asking him if he wants this info read or not. Oh, I see some unlabeled buttons, lets automatically find out what these buttons do and put them in the tab and shift tab order and label them so the blind user has easier access to them. I don't know what they are after deeply asking the applications, lets OCR them and find out, if successful, buttons are labeled, if not, ok lets picture smart each button and try to label them that way. If nothing comes back, lets upload them to aira or some service that will make meaningful labels so Jaws can learn about them using an
A-I training model. ok now that Jaws knows what these buttons do, lets put them in the tab order because they seem important enough the blind user may want to tab through them. or this scenario... 
A blind user wants to play a game in steam. Jaws sits back analyses the game as other sighted users plays it and sees how it is played to make it as accessible as possible using a combination of OCR, picture smart or some version of it, and A-I, to determine how best to present the info in the game to the blind user. Right now if I want vlc to read the timer as I hit alft arrows or control left and right arrows I have to script this. I think I can do it if only I could have access to jaws scan cursor with scripting. but why , just why can't jaws be smart enough to analyze the application and know that I may want to hear this information automatically? 


 

On Sat, Apr 23, 2022 at 01:11 PM, Josh Kennedy wrote:
Jaws sits back analyses the game as other sighted users plays it and sees how it is played to make it as accessible as possible using a combination of OCR, picture smart or some version of it, and A-I,
-
Eventually, I have to believe that AI functionality will make massive improvements in screen reading technology.

Virtually anything JAWS and the other screen readers do, as they do it now, is light years away from "the sighted experience" of anything.  Formerly sighted users of screen readers can tell you that even better than I can.

But AI is not yet a part of any screen reader technology, and we're nowhere near to having it integrated into screen readers in any substantial way.

You've already answered your own question, and not just in regard to JAWS, but kinda in reverse.   Screen readers are not "smart" technology and are a very long way from being rewritten such that they employ the kinds of AI that would be needed to imitate "visual filtering" both for items of significance as well as the stuff we sighted folks ignore all the time, and only hunt for on the very rarest of occasions.  And on those occasions we do have to hunt for it on the page/screen/window because it's not something that even registers during typical use.
 
--

Brian - Windows 10, 64-Bit, Version 21H2, Build 19044  

Here is a test to find out whether your mission in life is complete.  If you’re alive, it isn’t.
     ~ Lauren Bacall


David Goldfield <david.goldfield@...>
 

Josh, I think you raise some interesting issues that to an extent I’ve spent some time thinking and writing about. I remember twenty years ago showing JAWS to my sighted brother-in-law who wondered why JAWS couldn’t have been using some sort of artificial intelligence to attempt to reclass controls, such as labeling unlabeled elements. This was 20 years ago and so my brother-in-law was perhaps a bit ahead of his time. Now this type of AI is certainly a reality and JAWS is already leveraging a bit of it in the cloud with its convenient OCR and picture smart features. I do agree that it may be time to take some of that cloud-based power and have it more integrated into JAWS itself as a feature which could automatically attempt to add accessibility to otherwise inaccessible apps. Of course, this would have to be highly experimental and such a feature would need to be something the user could toggle as it would be prone to errors. Apple is already doing this with some of its recognition options in iOS and I think on the Mac as well. It can sometimes make apps very accessible and at times it makes things far worse, at least in its current state. If I were a product manager at Vispero, which I definitely am not, this is something I’d be pursuing along with, if possible, the ability to virtualize Windows native apps to allow a user to quickly navigate the app as well as to locate and activate elements such as buttons. I would definitely contact suggestions@... and document your ideas to them. I honestly would not be at all surprised if these or similar ideas were being developed and of course if they were we would be the last people to know about it.

Btw, I like the idea of crowdsourced element labeling. A few years ago I heard about something similar that was being developed which would have been screen reader agnostic but it never materialized and now the name of that product escapes me.

As an aside some of what you were envisioning reminds me of the way the ASAP screen reader for DOS claimed to work, analyzing screen activity and trying to make the most appropriate decisions on what should be announced. It actually did quite a good job in making those decisions and announcing just what you would want to hear. It is regrettable that its developer was never able to get a compatible Windows version to market although he certainly tried for a while.

As another aside one of the things I’ve suggested to Vispero is to have a script writing wizard for users with no experience writing scripts, containing various user-friendly dialogs to allow even a new user to choose different functions. I’d love to see Brian Hartgen produce such a program and I’ve suggested it to him, as well.

 

 

David Goldfield,

Blindness Assistive Technology Specialist

NVDA Certified Expert

 

Subscribe to the Tech-VI announcement list to receive emails regarding news and events in the blindness assistive technology field.

Email: tech-vi+subscribe@groups.io

www.DavidGoldfield.org

 

 

 

 

From: main@jfw.groups.io <main@jfw.groups.io> On Behalf Of Josh Kennedy
Sent: Friday, April 22, 2022 6:51 PM
To: main@jfw.groups.io
Subject: jaws being smarter requiring less scripts

 

Jhi
I have a comment and questionn. and lets take vlc media player for example. In vlc media player, if I use the jaws invisible vursor and go to the bottom of the screen and up arrow once, I am on the current time in the file I am listening to. Now if I fast forward or rewind through that file with control left and right arrow keys or alt left and right arrows, jaws won't read that information unless I somehow script it which at this point I don't know how to do because of the jaws scan cursor and almost script functions for it and script utilities not, or seeming to not, give me access to the status bar and any buttons. 
OK so my question is this. Why can't jaws be much smarter than it is? Why can't jaws look at the screen of an applications, find out what is happening when keys are pressed and maybe not on the first try but on subsequent tries, analyse the screen to determine what is changing and say... ok well this is changing and a sighted person may be interested in seeing this, therefore it may be relevant for a blind person also and since this is a media player its probably a time index for the file so lets monitor this and read it out for the blind person after first asking him if he wants this info read or not. Oh, I see some unlabeled buttons, lets automatically find out what these buttons do and put them in the tab and shift tab order and label them so the blind user has easier access to them. I don't know what they are after deeply asking the applications, lets OCR them and find out, if successful, buttons are labeled, if not, ok lets picture smart each button and try to label them that way. If nothing comes back, lets upload them to aira or some service that will make meaningful labels so Jaws can learn about them using an
A-I training model. ok now that Jaws knows what these buttons do, lets put them in the tab order because they seem important enough the blind user may want to tab through them. or this scenario... 
A blind user wants to play a game in steam. Jaws sits back analyses the game as other sighted users plays it and sees how it is played to make it as accessible as possible using a combination of OCR, picture smart or some version of it, and A-I, to determine how best to present the info in the game to the blind user. Right now if I want vlc to read the timer as I hit alft arrows or control left and right arrows I have to script this. I think I can do it if only I could have access to jaws scan cursor with scripting. but why , just why can't jaws be smart enough to analyze the application and know that I may want to hear this information automatically? 


Jeff Christiansen
 

Not every control, or every window, is a known type to JAWS you are welcome to write a script and share with the world?



Sent From My Mobile


On Apr 23, 2022, at 11:32, David Goldfield <david.goldfield@...> wrote:



Josh, I think you raise some interesting issues that to an extent I’ve spent some time thinking and writing about. I remember twenty years ago showing JAWS to my sighted brother-in-law who wondered why JAWS couldn’t have been using some sort of artificial intelligence to attempt to reclass controls, such as labeling unlabeled elements. This was 20 years ago and so my brother-in-law was perhaps a bit ahead of his time. Now this type of AI is certainly a reality and JAWS is already leveraging a bit of it in the cloud with its convenient OCR and picture smart features. I do agree that it may be time to take some of that cloud-based power and have it more integrated into JAWS itself as a feature which could automatically attempt to add accessibility to otherwise inaccessible apps. Of course, this would have to be highly experimental and such a feature would need to be something the user could toggle as it would be prone to errors. Apple is already doing this with some of its recognition options in iOS and I think on the Mac as well. It can sometimes make apps very accessible and at times it makes things far worse, at least in its current state. If I were a product manager at Vispero, which I definitely am not, this is something I’d be pursuing along with, if possible, the ability to virtualize Windows native apps to allow a user to quickly navigate the app as well as to locate and activate elements such as buttons. I would definitely contact suggestions@... and document your ideas to them. I honestly would not be at all surprised if these or similar ideas were being developed and of course if they were we would be the last people to know about it.

Btw, I like the idea of crowdsourced element labeling. A few years ago I heard about something similar that was being developed which would have been screen reader agnostic but it never materialized and now the name of that product escapes me.

As an aside some of what you were envisioning reminds me of the way the ASAP screen reader for DOS claimed to work, analyzing screen activity and trying to make the most appropriate decisions on what should be announced. It actually did quite a good job in making those decisions and announcing just what you would want to hear. It is regrettable that its developer was never able to get a compatible Windows version to market although he certainly tried for a while.

As another aside one of the things I’ve suggested to Vispero is to have a script writing wizard for users with no experience writing scripts, containing various user-friendly dialogs to allow even a new user to choose different functions. I’d love to see Brian Hartgen produce such a program and I’ve suggested it to him, as well.

 

 

David Goldfield,

Blindness Assistive Technology Specialist

NVDA Certified Expert

 

Subscribe to the Tech-VI announcement list to receive emails regarding news and events in the blindness assistive technology field.

Email: tech-vi+subscribe@groups.io

www.DavidGoldfield.org

 

 

 

 

From: main@jfw.groups.io <main@jfw.groups.io> On Behalf Of Josh Kennedy
Sent: Friday, April 22, 2022 6:51 PM
To: main@jfw.groups.io
Subject: jaws being smarter requiring less scripts

 

Jhi
I have a comment and questionn. and lets take vlc media player for example. In vlc media player, if I use the jaws invisible vursor and go to the bottom of the screen and up arrow once, I am on the current time in the file I am listening to. Now if I fast forward or rewind through that file with control left and right arrow keys or alt left and right arrows, jaws won't read that information unless I somehow script it which at this point I don't know how to do because of the jaws scan cursor and almost script functions for it and script utilities not, or seeming to not, give me access to the status bar and any buttons. 
OK so my question is this. Why can't jaws be much smarter than it is? Why can't jaws look at the screen of an applications, find out what is happening when keys are pressed and maybe not on the first try but on subsequent tries, analyse the screen to determine what is changing and say... ok well this is changing and a sighted person may be interested in seeing this, therefore it may be relevant for a blind person also and since this is a media player its probably a time index for the file so lets monitor this and read it out for the blind person after first asking him if he wants this info read or not. Oh, I see some unlabeled buttons, lets automatically find out what these buttons do and put them in the tab and shift tab order and label them so the blind user has easier access to them. I don't know what they are after deeply asking the applications, lets OCR them and find out, if successful, buttons are labeled, if not, ok lets picture smart each button and try to label them that way. If nothing comes back, lets upload them to aira or some service that will make meaningful labels so Jaws can learn about them using an
A-I training model. ok now that Jaws knows what these buttons do, lets put them in the tab order because they seem important enough the blind user may want to tab through them. or this scenario... 
A blind user wants to play a game in steam. Jaws sits back analyses the game as other sighted users plays it and sees how it is played to make it as accessible as possible using a combination of OCR, picture smart or some version of it, and A-I, to determine how best to present the info in the game to the blind user. Right now if I want vlc to read the timer as I hit alft arrows or control left and right arrows I have to script this. I think I can do it if only I could have access to jaws scan cursor with scripting. but why , just why can't jaws be smart enough to analyze the application and know that I may want to hear this information automatically? 


Josh Kennedy
 

never mind about vlc, I have leasey that makes vlc more accessible and does what I want it to with jaws.


Josh Kennedy
 

yes and a script-writing wizard is something I would pay for definitely!


Josh Kennedy
 

also Jaws is a $1000 plus or so screen reader if you buy the traditional perpetual licenses. so I would imagine Freedom Scientific is researching A-I and things but yes we're probably the last people to know about it. I wish Jaws would have had the subscription-based home anual licenses 20 years ago because back in college and mostly a few years after some time afterwards I could not afford jaws so had to pirate it but I don't like pirating because it doesn't work reliably not as nice and reliably as it does when you have the money to spend and get a real legal license. If the home anual license would have existed 20 years ago I would have just purchased jaws back then.


Udo Egner-Walter
 

Hi Josh & all,

I have been scripting for quite a while now and would therefore like to write my opinion on your suggestions. First of all, the interface to access controls is not developed by FS but by Microsoft. FS must work as with what is provided via these interfaces (UIA, MSAA, Windows32). Nowadays, this is usually the UIA interface. Each control is an object of a certain type. The properties of the object, such as the name that is displayed, are determined by the developer of a software.

While the name is often determined by the developer, other properties are often not used by the developer. This is bad, because you could find the object through certain properties. If such a unique property is missing, it is difficult to uniquely identify the controls.

Sometimes the object can be identified by the name property. But this is also bad because the name property is different in other countries And sometimes there is even no name property at all, which makes identification even more difficult.

A clear identification is important because you have to assign the text to the correct object at the focus of a control or by pressing the Tab key.

So if the developers of software do not do this properly, it is very difficult for scripters and JAWS to make the program accessible.

For example, you can count the position as a small help to identify a control, but this is not possible if new controls are dynamically added before it.

In the other email about VLC, I talked about my ObjectCursor. Here, for example, I tried to identify the controls so that you can assign your own name or name unlabeled controls in the scripts. If you use the ObjectCursor in the VLC, you will notice the limits, because identifying what is due to the QT interface does not work here. QT is a framework designed to use programs on other operating systems without having to rewrite the code.

I hope I was able to convey this in an understandable way.

Greetings in the round

Udo

Am 23.04.2022 um 00:50 schrieb Josh Kennedy <joshknnd1982@...>:

Jhi
Ich habe einen Kommentar und eine Frage. und nehmen wir zum Beispiel den vlc Media Player. Wenn ich im vlc Media Player den unsichtbaren Backen vursor verwende und einmal zum unteren Rand des Bildschirms und zum Pfeil nach oben gehe, bin ich auf der aktuellen Zeit in der Datei, die ich höre. Wenn ich jetzt mit linken und rechten Pfeiltasten oder Alt-Links- und Rechtspfeilen durch diese Datei vorspule oder zurückspule, werden Kiefer diese Informationen nicht lesen, es sei denn, ich schreibe sie irgendwie, was ich zu diesem Zeitpunkt aufgrund des Kieferscan-Cursors und fast Skriptfunktionen dafür nicht oder scheinbar nicht tun weiß, geben mir Zugriff auf die Statusleiste und alle Schaltflächen.

OK, meine Frage ist also diese. Warum können Kiefer nicht viel schlauer sein, als sie sind? Warum können Kiefer nicht auf den Bildschirm einer Anwendung schauen, herausfinden, was passiert, wenn Tasten gedrückt werden und vielleicht nicht beim ersten Versuch, sondern bei nachfolgenden Versuchen, den Bildschirm analysieren, um festzustellen, was sich ändert und sagt... ok, nun, das ändert sich und eine sehende Person könnte daran interessiert sein, dies zu sehen, daher kann es auch für eine blinde Person relevant sein, und Informationen gelesen oder nicht. Oh, ich sehe einige unbeschriftete Schaltflächen, lassen Sie uns automatisch herausfinden, was diese Schaltflächen tun, und sie in die Tabulator- und Shift-Tab-Reihenfolge setzen und beschriften, damit der blinde Benutzer leichter darauf zugreifen kann. Ich weiß nicht, was sie sind, nachdem ich die Anwendungen gründlich gefragt habe, lasse sie OCR und finde heraus, wenn sie erfolgreich sind, sind Schaltflächen beschriftet, wenn nicht, ok, lass uns jede Taste intelligent darstellen und versuchen, sie so zu beschriften. Wenn nichts zurückkommt, laden wir sie auf aira oder einen Dienst hoch, der aussagekräftige Etiketten erstellt, damit Jaws mit einem

A-I-Trainingsmodell. Ok, jetzt, da Jaws weiß, was diese Schaltflächen tun, lassen Sie sie in die Tab-Reihenfolge bringen, weil sie wichtig genug erscheinen, dass der blinde Benutzer sie vielleicht durch sie tabulieren möchte. oder dieses Szenario...

Ein blinder Benutzer möchte ein Spiel in Steam spielen. Jaws lehnt sich zurück, analysiert das Spiel, während andere sehende Benutzer es spielen, und sieht, wie es gespielt wird, um es mit einer Kombination aus OCR, picture smart oder einer Version davon und A-I so zugänglich wie möglich zu machen, um zu bestimmen, wie die Informationen im Spiel dem blinden Benutzer am besten präsentiert werden können. Im Moment, wenn ich möchte, dass vlc den Timer liest, während ich alft Pfeile drücke oder linke und rechte Pfeile drücke, muss ich dies skripten. Ich denke, ich kann es tun, wenn ich nur Zugriff auf den Backen-Scan-Cursor mit Skripting haben könnte. Aber warum, warum können Kiefer nicht schlau genug sein, um die Anwendung zu analysieren und zu wissen, dass ich diese Informationen möglicherweise automatisch hören möchte?

_. _,_. _,_

Groups.io Links:

Sie erhalten alle Nachrichten, die an diese Gruppe gesendet werden.

Online Anzeigen/Antworten (#96807) | Auf Gruppe Antworten | Auf Absender Antworten | Dieses Thema Stummschalten | Neues Thema

--

JFW-bezogene Links:

JFW-Homepage: http://www.freedomscientific.com/

Anweisungen zur JFW-Liste:

Um eine Nachricht an die Liste zu senden, senden Sie sie an jfw@groups.io

Um sich von dieser Mailingliste abzumelden, senden Sie eine Nachricht an jfw+unsubscribe@groups.io

Archiv befindet sich unter: http://www.groups.io/g/jfw/threads

Alternative Archive finden Sie unter: http://n2.nabble.com/JAWS-for-Windows-f2145279.html

Wenn Sie Bedenken bezüglich der Liste, der von der Liste erhaltenen Beiträge oder der Art und Weise haben, wie die Liste ausgeführt wird, veröffentlichen Sie sie nicht in der Liste. Wenden Sie sich lieber an den Listenbesitzer unter jfw+owner@groups.io.

Ihr Abonnement | Gruppeninhaber kontaktieren | Abmelden [udo.egnerwalter@...]

_. _,_. _,_


 

Hi,
I can confirm the below message from the other side of the pond (see below). While Vispero does provide an interface called FSUIA to help obtain UI Automation properties and patterns, in some ways, screen readers are at the mercy of app developers, including, believe it or not, Microsoft. Without digressing a lot, NVDA uses COM (Component Object Model) to access UI Automation elements and properties. Because I do know what the post below is talking about, I always do my best to avoid use of element labels (names) when detecting specific objects on screen - I rather use UIA properties such as Automation Id, a unique string designed to distinguish between screen controls (trust me, it is better to obtain properties using the facility provided by accessibility API's, but experience has shown that not all UIA controls behave the same).
Let me address two things that are related to the topic at hand:
* Can you make a screen reader smarter by including many features into the core such as artificial intelligence and machine learning based screen reading, or somehow rely less on scripts: I can tell you that the answer is a resounding no. While useful, there is one big issue with AI: it relies on past data more than present and speculations about the future to make decisions. In essence, artificial intelligence is a way to let computers make decisions about the current environment and future directions with help from data from the past. Contrary to what folks may say, it doesn't really help screen readers for at least two reasons: past data on accessibility, screen control labels, and scripts were designed at a time when lack of accessibility guidelines and varying standards was the norm (until early 2000's), and given rapid advances in technology, it takes time to build additional data, and as a result, screen readers may still rely on outdated assumptions to make decisions. But at least it is a bit better than the current screen reading model we have: data gathering -> interpreting -> processing and filtering -> output (after all, screen readers are sophisticated text and data processors). Screen reader vendors could use machine learning (ML) models for some tasks, but that still relies on training the screen reader based on past data. Why am I keep saying "past data"? Because screen readers are event-driven, therefore by the time an event occurs such as a key press, screen changes, and what not, from the viewpoint of the screen reader, it is a past event (remember that humans take fraction of a second or so to react to events; and since computers can calculate some things a bit faster than a human brain, they, too will see "current event" as a "past event"; if the event is not seen as accessible, then it will take hours to days (or even years) for humans to provide the needed data to screen reading AI, by which time folks may encounter other inaccessible data; the best known cases are websites, and increasingly, progressive web apps (PWA's) which are websites acting just like installed programs). I won't go into data bias for AI - if you have time, think critically about it (Why am I not giving you the puzzle pieces for answers? Because you must learn to find things instead of eating things people tell you in order to survive better in a society that's changing rapidly; that's a fact, not an opinion, folks).
* Why are screen readers at the mercy of app developers, including, believe it or not, Microsoft? I'm paraphrasing something Brian Vogel always says to screen reader users: as far as app development is concerned, unless the app is designed with accessibility in mind, the primary target audience of any app is the sighted majority. I'm not saying this lightly: if you think about the majority versus minority in society, you will see that, for some app developers, accessibility comes in later. I understand that programming students these days learn a thing or two about inclusive and accessible design somewhere in their training, but that's still a minority right now. The app development philosophy is the reason why some of us (me included) always advise users to contact app developers BEFORE telling folks that it is an issue with the screen reader (this goes across screen readers). I've been living in the screen reader development universe for a decade (more if you count providing support for a specific braille hardware in the past), and I have seen people say an issue lies with the screen reader only later to find out that the issue has to do with either the ap in question, misunderstood screen reader settings, or not knowing how to persuade ap developers to change their minds. You can't make a screen reader smarter by incorporating more features into it simply because you find technology attractive - improving screen readers is an exercise in collaboration, cooperation, persuasion, interpretation, and inclusion.
Hope this helps (contact me privately if you would like to talk more about all this).
Cheers,
Joseph

-----Original Message-----
From: main@jfw.groups.io <main@jfw.groups.io> On Behalf Of Udo Egner-Walter via groups.io
Sent: Sunday, April 24, 2022 8:18 AM
To: main@jfw.groups.io
Subject: Re: jaws being smarter requiring less scripts

Hi Josh & all,

I have been scripting for quite a while now and would therefore like to write my opinion on your suggestions. First of all, the interface to access controls is not developed by FS but by Microsoft. FS must work as with what is provided via these interfaces (UIA, MSAA, Windows32). Nowadays, this is usually the UIA interface. Each control is an object of a certain type. The properties of the object, such as the name that is displayed, are determined by the developer of a software.

While the name is often determined by the developer, other properties are often not used by the developer. This is bad, because you could find the object through certain properties. If such a unique property is missing, it is difficult to uniquely identify the controls.

Sometimes the object can be identified by the name property. But this is also bad because the name property is different in other countries And sometimes there is even no name property at all, which makes identification even more difficult.

A clear identification is important because you have to assign the text to the correct object at the focus of a control or by pressing the Tab key.

So if the developers of software do not do this properly, it is very difficult for scripters and JAWS to make the program accessible.

For example, you can count the position as a small help to identify a control, but this is not possible if new controls are dynamically added before it.

In the other email about VLC, I talked about my ObjectCursor. Here, for example, I tried to identify the controls so that you can assign your own name or name unlabeled controls in the scripts. If you use the ObjectCursor in the VLC, you will notice the limits, because identifying what is due to the QT interface does not work here. QT is a framework designed to use programs on other operating systems without having to rewrite the code.

I hope I was able to convey this in an understandable way.

Greetings in the round

Udo


Am 23.04.2022 um 00:50 schrieb Josh Kennedy <joshknnd1982@...>:

Jhi
Ich habe einen Kommentar und eine Frage. und nehmen wir zum Beispiel den vlc Media Player. Wenn ich im vlc Media Player den unsichtbaren Backen vursor verwende und einmal zum unteren Rand des Bildschirms und zum Pfeil nach oben gehe, bin ich auf der aktuellen Zeit in der Datei, die ich höre. Wenn ich jetzt mit linken und rechten Pfeiltasten oder Alt-Links- und Rechtspfeilen durch diese Datei vorspule oder zurückspule, werden Kiefer diese Informationen nicht lesen, es sei denn, ich schreibe sie irgendwie, was ich zu diesem Zeitpunkt aufgrund des Kieferscan-Cursors und fast Skriptfunktionen dafür nicht oder scheinbar nicht tun weiß, geben mir Zugriff auf die Statusleiste und alle Schaltflächen.

OK, meine Frage ist also diese. Warum können Kiefer nicht viel
schlauer sein, als sie sind? Warum können Kiefer nicht auf den
Bildschirm einer Anwendung schauen, herausfinden, was passiert, wenn
Tasten gedrückt werden und vielleicht nicht beim ersten Versuch,
sondern bei nachfolgenden Versuchen, den Bildschirm analysieren, um
festzustellen, was sich ändert und sagt... ok, nun, das ändert sich
und eine sehende Person könnte daran interessiert sein, dies zu sehen,
daher kann es auch für eine blinde Person relevant sein, und
Informationen gelesen oder nicht. Oh, ich sehe einige unbeschriftete
Schaltflächen, lassen Sie uns automatisch herausfinden, was diese
Schaltflächen tun, und sie in die Tabulator- und Shift-Tab-Reihenfolge
setzen und beschriften, damit der blinde Benutzer leichter darauf
zugreifen kann. Ich weiß nicht, was sie sind, nachdem ich die
Anwendungen gründlich gefragt habe, lasse sie OCR und finde heraus,
wenn sie erfolgreich sind, sind Schaltflächen beschriftet, wenn nicht,
ok, lass uns jede Taste intelligent darstellen und versuchen, sie so
zu beschriften. Wenn nichts zurückkommt, laden wir sie auf aira oder
einen Dienst hoch, der aussagekräftige Etiketten erstellt, damit Jaws
mit einem

A-I-Trainingsmodell. Ok, jetzt, da Jaws weiß, was diese Schaltflächen tun, lassen Sie sie in die Tab-Reihenfolge bringen, weil sie wichtig genug erscheinen, dass der blinde Benutzer sie vielleicht durch sie tabulieren möchte. oder dieses Szenario...

Ein blinder Benutzer möchte ein Spiel in Steam spielen. Jaws lehnt sich zurück, analysiert das Spiel, während andere sehende Benutzer es spielen, und sieht, wie es gespielt wird, um es mit einer Kombination aus OCR, picture smart oder einer Version davon und A-I so zugänglich wie möglich zu machen, um zu bestimmen, wie die Informationen im Spiel dem blinden Benutzer am besten präsentiert werden können. Im Moment, wenn ich möchte, dass vlc den Timer liest, während ich alft Pfeile drücke oder linke und rechte Pfeile drücke, muss ich dies skripten. Ich denke, ich kann es tun, wenn ich nur Zugriff auf den Backen-Scan-Cursor mit Skripting haben könnte. Aber warum, warum können Kiefer nicht schlau genug sein, um die Anwendung zu analysieren und zu wissen, dass ich diese Informationen möglicherweise automatisch hören möchte?

_. _,_. _,_

Groups.io Links:

Sie erhalten alle Nachrichten, die an diese Gruppe gesendet werden.

Online Anzeigen/Antworten (#96807) | Auf Gruppe Antworten | Auf
Absender Antworten | Dieses Thema Stummschalten | Neues Thema

--

JFW-bezogene Links:

JFW-Homepage: http://www.freedomscientific.com/

Anweisungen zur JFW-Liste:

Um eine Nachricht an die Liste zu senden, senden Sie sie an
jfw@groups.io

Um sich von dieser Mailingliste abzumelden, senden Sie eine Nachricht
an jfw+unsubscribe@groups.io

Archiv befindet sich unter: http://www.groups.io/g/jfw/threads

Alternative Archive finden Sie unter:
http://n2.nabble.com/JAWS-for-Windows-f2145279.html

Wenn Sie Bedenken bezüglich der Liste, der von der Liste erhaltenen Beiträge oder der Art und Weise haben, wie die Liste ausgeführt wird, veröffentlichen Sie sie nicht in der Liste. Wenden Sie sich lieber an den Listenbesitzer unter jfw+owner@groups.io.

Ihr Abonnement | Gruppeninhaber kontaktieren | Abmelden
[udo.egnerwalter@...]

_. _,_. _,_


Steve Nutt
 

Hi Brian,

 

You’d also want to turn that AI off too, because it may ignore something you might want to hear.

 

All the best


Steve

 

--

To subscribe to our News and Special Offers list, go to https://www.comproom.co.uk/subscribe

 

Computer Room Services

77 Exeter Close

Stevenage

Hertfordshire

SG1 4PW

T: +44(0)1438-742286

M: +44(0)7956-334938

F: +44(0)1438-759589

E: steve@...

W: https://www.comproom.co.uk

 

From: main@jfw.groups.io <main@jfw.groups.io> On Behalf Of Brian Vogel
Sent: 23 April 2022 18:17
To: main@jfw.groups.io
Subject: Re: jaws being smarter requiring less scripts

 

On Sat, Apr 23, 2022 at 01:11 PM, Josh Kennedy wrote:

Jaws sits back analyses the game as other sighted users plays it and sees how it is played to make it as accessible as possible using a combination of OCR, picture smart or some version of it, and A-I,

-
Eventually, I have to believe that AI functionality will make massive improvements in screen reading technology.

Virtually anything JAWS and the other screen readers do, as they do it now, is light years away from "the sighted experience" of anything.  Formerly sighted users of screen readers can tell you that even better than I can.

But AI is not yet a part of any screen reader technology, and we're nowhere near to having it integrated into screen readers in any substantial way.

You've already answered your own question, and not just in regard to JAWS, but kinda in reverse.   Screen readers are not "smart" technology and are a very long way from being rewritten such that they employ the kinds of AI that would be needed to imitate "visual filtering" both for items of significance as well as the stuff we sighted folks ignore all the time, and only hunt for on the very rarest of occasions.  And on those occasions we do have to hunt for it on the page/screen/window because it's not something that even registers during typical use.
 
--

Brian - Windows 10, 64-Bit, Version 21H2, Build 19044  

Here is a test to find out whether your mission in life is complete.  If you’re alive, it isn’t.
     ~ Lauren Bacall


Josh Kennedy
 

I remember years ago with asap for dos, you could turn its A-I, off, and make conventional set files because sometimes the A-I, however it worked back then, was good and sometimes it was bad. 


 

On Mon, Apr 25, 2022 at 05:46 AM, Steve Nutt wrote:
You’d also want to turn that AI off too, because it may ignore something you might want to hear.
-
And therein lies the problem - a damned if you do, damned if you don't situation.

Screen readers now, sans AI, announce everything, most of which you'll never want to hear when it comes to commercial webpages, anyway.   With AI, it will be the AI's "best guess" as to what it thinks the user wants to hear.

There is no perfection.   And it's another of those situations where my oft-repeated statement, "Sometimes, there's just no substitute for sight," comes into play.  All accessibility is a workaround that is substituting audition for sight when it comes to actual content, and as things stand now there is no filtering at all when it comes to "what's important to actually read" in situations where most of what's available is dross that is accessed once in way less than a blue moon to never.

Regardless of the sensory modality under discussion, for whichever ones we happen to be equipped with we develop filtering mechanisms that occur beneath the level of conscious awareness.  Good examples of this are for someone who lives under the flight path near an airport or near a train track very quickly "ceases to hear" (not literally, but it gets filtered out) the noises characteristic of those locations that do not pose a threat and where attending to them takes away from real tasks at hand (including sleep).  We are not consciously aware of how our own clothing feels at every given second of the day, but we generally instantly recognize when something about the fit has "gone wrong" because the sensations when that occurs rise above the filtering.  The same thing is absolutely true of visual stimulus when you can see.  I'd say that the majority of it, at any given moment, is actively or passively filtered out to allow focus on finding the thing it is you're looking for.  In the case of webpages as an example, no one initially looks at all the links at the top of say, a newspaper's main page for all the sections unless they're looking to go straight to a section, and even then you can zero in on the name of that section visually seemingly almost instantly.  I don't know of anyone who routinely looks at any of the links that are at the bottom of a newspaper's main page like these, taken directly from the New York Times main page:  

You virtually never, ever need these and even the way they're presented, in tiny print and in grey rather than black color, implicitly acknowledges that.  And for those of you who cannot see, and have never been able to see, those things that are clear visual cues that carry meaning to those who can see can't do the same thing for you.  God forbid that screen readers in "dumb" read all mode would be announcing font sizes, type color, etc., every blessed time it changed.

Things that have clear meaning in a given sensory modality are often sui generis, they cannot be fully captured in any other modality.  My favorite example of that is taste.  You cannot describe a taste unless someone else is capable of also tasting the same thing.  "Salty" means nothing, other than being an abstract class, to a person who cannot taste salt.  A good cross species real world example of this is hot peppers, and really hot peppers, and what humans taste versus what parrots do.  Parrots do not possess the taste receptors for the chemical, capsaicin, that makes hot peppers hot to us.  You give a ghost pepper to a human and they take a bite out of it the results border on catastrophic [they're that hot].  Give the same to a parrot - nada - they can't detect capsaicin.  There is no way I can make a parrot understand "hot pepper hot," just as there is no way I can make someone who has never seen really understand "blue."  You can, and do, understand that it is a classifier for how something appears to someone who can see, but you have no idea how it differs from red, green, violet, turquoise, teal, etc., in any meaningful way because you have to see (or have seen, at some point) to have a real understanding of exactly what color is.  Just as you have to hear, or have heard, to really understand what a piano versus a trumpet versus a train versus a car sounds like.  No amount of description can accurately capture and communicate sensory experience across senses.  It doesn't even come close.  How salty is salty in any given instance?  You can't know, exactly, until you taste.  You know only that it's somewhere outside what the person who said, "salty," considers within typical limits for that flavor.

Thus, since AI relies on some collective profile of salience it uses, there will be occasions where what you want is the "odd thing out, this time," and it will get it wrong.  So being able to turn that off will be essential when it becomes a feature.
--

Brian - Windows 10, 64-Bit, Version 21H2, Build 19044  

Here is a test to find out whether your mission in life is complete.  If you’re alive, it isn’t.
     ~ Lauren Bacall


Steve Nutt
 

Yep this was true of Vocal-Eyes as well, which was the best DOS screen reader I ever had.

 

All the best


Steve

 

From: main@jfw.groups.io <main@jfw.groups.io> On Behalf Of Josh Kennedy
Sent: 25 April 2022 15:43
To: main@jfw.groups.io
Subject: Re: jaws being smarter requiring less scripts

 

I remember years ago with asap for dos, you could turn its A-I, off, and make conventional set files because sometimes the A-I, however it worked back then, was good and sometimes it was bad.