Top
Best
New

Posted by RZelaya 6 hours ago

Apple rejected my dictation app for using the accessibility API(www.mitmllc.com)
233 points | 138 commentspage 2
hirako2000 5 hours ago|
Some non apple apps get access to accessibility APIs. What gives?

This API is sensitive. I imagine Apple is particularly stringent as to how the access is justified. Not how it uses it but how the reason for using it is explained.

It's not like someone tests the app and all api calls to deem them reasonable or not.

taormina 4 hours ago|
They do literally pay people to do that. Then one of those people chose to reject this anyway.
hombre_fatal 5 hours ago||
What API are you using? I have a sandboxed app on the Mac Store that synthesizes CGEvents to simulate arbitrary keyboard actions on behalf of the user. It needs accessibility permission, of course.
RZelaya 5 hours ago||
Same approach: CGEventPost with Accessibility permission. The wrinkle was that my App Store reviewer wasn't comfortable with how I was using AX permission for auto-paste, even though the mechanism is the same as other apps already in the store. The clipboard-only version of WhisperPad needs no AX permission and that's what got through. Interesting that your sandboxed app with similar mechanics is approved.
jchigg2000 5 hours ago||
Wondering the same, there is some weirdness around the clipboard and CGEvents though. Are you avoiding the clipboard entirely in your implementation?
luca-ctx 3 hours ago||
I’ve had lots of inconsistent app reviews from Apple. Just appeal and/or re-word your language and you’ll be ok. Plan on it taking a few weeks to fully sort out.
stokedbits 2 hours ago||
This is well documented by them which is why a majority of the apps doing this are released outside of the App Store. I built something similar, and I just publish it separately https://github.com/moxiebytescode/speakeasy.
jchigg2000 5 hours ago||
Quick question, I assume you're getting caught by the CGEvent(PostEvent)...but I want to be sure. AX API has been gimped for over a decade so you'd have never made it into the app store that way. Just making certain, in case you have another path. It doesn't appear CGEvent is a universal approval anymore either though.

Have fought similar demons lately, feel your pain.

RZelaya 5 hours ago|
The direct version uses CGEventPost to synthesize the paste, which requires Accessibility permission. The App Store version writes to the clipboard only, so no AX permission needed and the user presses Cmd+V manually. The 2.4.5 rejection was specifically about the Accessibility permission use case. Your read sounds right that this path has been gimped for a long time.
longnguyen 2 hours ago||
If you emulate command+V, make sure to check the keyboard layout. You may need to translate the keycode V for the current keyboard layout like DVORAK etc
RZelaya 2 hours ago|
Good catch. Easy edge case to miss if you only test on QWERTY. I'll double-check the implementation, thanks for the heads up.

update: You're right, this is a real bug. The Direct version's auto-paste hardcodes the QWERTY keycode for V instead of translating for the active layout, so Dvorak / Colemak / AZERTY users would all hit it. The MAS version is unaffected (clipboard-only; the user presses their own Cmd+V, which is layout-correct). Fix is going into the next release. Thanks for the careful read.

longnguyen 1 hour ago||
Good luck. I’ve been building a native AI client[0] for the past 3 years and I didn’t catch this edge case until some of my users asked for it.

[0]: https://boltai.com

nullbio 4 hours ago||
Doesn't Wispr Flow do this though? How did they get past these limitations?
RZelaya 3 hours ago||
From what I understand Wispr Flow distributes directly from their website and doesn't ship through the Mac App Store, so they don't go through Apple's App Store review at all. They use the Accessibility API the same way the direct version of WhisperPad does. The 2.4.5 limitation really only kicks in if you want App Store presence.
sangeeth96 4 hours ago||
not in the app store?
DelightOne 5 hours ago||
I don't want random apps to paste potentially dangerous things into other apps. Its understandable.

Imagine a banking app, and for example an IBAN field.

kuboble 5 hours ago||
Them you are free to not install them? Why ban them outright?

I'm using https://github.com/cjpais/Handy whichseems to be doing exactly what this app does, and has a very similar background story (author couldn't type die to injury).

SyneRyder 5 hours ago|||
Handy is excellent and cross platform, and really elegant. They've got a direct website here which might be easier to navigate than the Github repo:

https://handy.computer/

RZelaya 5 hours ago||
Handy looks great. More tools in this space is a good thing for people who need them.
mrweasel 5 hours ago||||
In this case it feels like it's a feature that the operating system should be providing or something that could be marked as an accessibility tool, which would allow it to use that API.

The problem from Apples perspective could be that there is a ton of tools that require access to the accessibility API because they want to do stuff that Apple have deemed a security risk and the only way to do it is by abusing the API. Some of these are also because macOS simply lacks certain APIs.

I think Apple overreacting due to previous API misuse by other apps.

RZelaya 5 hours ago||
[dead]
amazingamazing 5 hours ago|||
To their defense you cannot rollback apps, so if you did install and only an update had this function, you are out of luck
applfanboysbgon 5 hours ago||
"In their defense, the OS is even more insane with mandatory forced application updates that you have no control of". I hope I won't ever happen to have you representing me as a defense attourney!
RZelaya 5 hours ago|||
I see, that's a really fair point. And I can understand that banking field example. So I can see why they're guarding against it. My disagreement was less with the rule itself and whether Whisperpad's specific use case for users with mobility needs falls on the right side of it.
notlive 5 hours ago|||
I would like the option to allow the behaviour selectively
DelightOne 5 hours ago||
That's what install outside of the App Store is for. On your own risk-
boxed 5 hours ago||
Pasting doesn't seem very unsafe. Especially not when the app can't know what it's pasting into.
-mlv 5 hours ago|
No surprises here, Google has also been restricting access to its accessibility API.
RZelaya 5 hours ago|
Useful context, thanks. I hadn't realized Google was tightening similarly. Would be interesting to see how the rationales compare.
More comments...