Repository navigation
Remote keyboard only active when editing text #892
Description
Describe the bug
Remote keyboard don't stay active even if the "Handle remote keys only when editing" option is disabled
Steps To Reproduce:
- Active KDE Connect Remote Keyboard on Android system settings
- Go to plugin's settings
- Disable the "Handle remote keys only when editing" option
- Try to use remote keyboard
- See that isn't active
Expected behavior
Remote keyboard should stay active even if isn't selected any text entry (If it isn't possible, sorry for misunderstand the plugin option)
Screenshots
Receive remote keypresses plugin options
System Details:
- GSConnect version: 39
- Installed from: GNOME Extensions Website
- GNOME/Shell version: 3.36.2
- Distro/Release: Manjaro GNOME
GSConnect environment (if applicable):
- Paired Device(s): Semp Toshiba TA 0708G
- KDE Connect app version: 1.14.2
- Plugin(s): Receive remote keypresses
Additional Notes:
My tablet touchscreen is broken, the only way that I can control the tablet is from touching on a thin line that is working, and controlling from a keyboard with micro usb. I wanted to control from my PC, but I have to disconnect the micro usb keyboard, so I can't accept the authorization dialog for usb debugging. So I was trying to use the remote keyboard. (Sorry for any english mistake)
Activity
andyholmes commentedon Jul 10, 2020
Hi, thanks for reporting.
This may be a bug in either the Android app or GSConnect. I haven't really tested this use-case, so I will try to reproduce and debug soon.
felipemarins commentedon Jul 10, 2020
I don't know if it's important, but it's Android 4.4.2
andyholmes commentedon Jul 10, 2020
That's a good thought (there is some code where that might matter), but I just had a look and there are no SDK-versioned guards in the Android app plugin I see.
Actually, looking briefly I don't see that the preference is actually hooked up to the plugin anywhere...
felipemarins commentedon Jul 10, 2020
So it might be a bug in the Android app and not in GSConnect?
andyholmes commentedon Jul 10, 2020
It seems like that may be the case. I've asked in the KDE Connect developer channel about this, and I'll update you when I hear back.
felipemarins commentedon Jul 10, 2020
Ok, thanks
andyholmes commentedon Jul 11, 2020
Ok, so after talking with the upstream developers, this seems to be a case of the protocol being a little ambiguous.
The short version is that Android sends us a flag (
keyboardstate = true/false) letting us know if the keyboard is ready to accept input, but what it actually means is "keyboard is visible on the screen". But it won't tell us if the preference allows for keypresses when the keyboard is not visible.So hopefully we can improve that behaviour in the long term, but in the short term I will modify GSConnect to consider that to be more of a warning, rather than an error.
felipemarins commentedon Jul 11, 2020
Thank you for explaining and taking the time for check this!
Mousepad Plugin: ignore remote keyboard state
andyholmes commentedon Jul 11, 2020
I've just pushed a release candidate here: https://github.com/andyholmes/gnome-shell-extension-gsconnect/releases/tag/v40-rc1
You can start using that (and testing 😉) by following the instructions in the Wiki.
felipemarins commentedon Jul 11, 2020
I will try :)
felipemarins commentedon Jul 11, 2020
It's working everywhere now! (But it's not working on the usb debugging authorization dialogue, that is where I wanted to work. I guess Android prevents from third party keyboards working on that :( )
felipemarins commentedon Jul 11, 2020
And it doesn't work on the notifications area and on any window that overlaps another (as a dialogue or a menu. So maybe this is the cause that I can't work on the authorize dialog), but I think it may be improved on long-term.
felipemarins commentedon Jul 11, 2020
I think that all these problems are things that has to be improved in the Android app, so maybe I have to report it to them instead of here.
felipemarins commentedon Jul 11, 2020
I will list here some of the problems that I have noticed (if there is any that is unrelated to GSConnect, you may disconsider):
andyholmes commentedon Jul 11, 2020
Well, this issue I'm going to leave open to track the "keyboardstate" protocol problem.
Those particular key issues, I'll have to poke around a bit when I have time. They could be in GSConnect, because handling keys like Esc, Enter and so on can get a bit tricky because we're trying to "steal" them from the desktop.
It could also be some keys aren't being mapped correctly by the Android app, because for example Enter and keypad Enter (the part of your keyboard like phone/calculator) aren't necessarily the same.
I just don't know yet, but feel free to keep posting your results; it saves me time :)
felipemarins commentedon Jul 11, 2020
Well, today I tried to test again, but I'm not getting do it to work. (it's in the same state that was in the previous version) I will keep trying here for a while.
felipemarins commentedon Jul 11, 2020
If I disable the option "Physical keyboard" on the Android settings, it works until a small number of keys is inputted. If I press some keys in the micro usb keyboard, it enables again, but after a small number of keys inputted, it disables.
If I unplug the micro usb keyboard, it enables, but works the same way, it disables after a small number of keys inputted. The same happens if I pull the notifications area.
felipemarins commentedon Jul 11, 2020
I don't know how I managed to do it to work yesterday.
felipemarins commentedon Jul 11, 2020
If I move through the interface with the arrow keys of the micro usb, it enables again, but works the same way.
I noticed that pressing Ctrl + Right causes the open application to crash.
felipemarins commentedon Jul 11, 2020
I actually was able to make it work with a functionality of the Hacker's Keyboard, that opens the keyboard manually through the notifications area, so I open the Hacker's Keyboard and change to the KDE Connect Remote Keyboard while it's open. Sorry for not report it. I will try to enable without this workaround. (Hacker's Keyboard is open source, maybe the developer of the Android app can implement it)
felipemarins commentedon Jul 11, 2020
Without the Hacker's Keyboard workaround, it can be enabled by selecting some text entry (as the Google app) and it's ready, it works. Please, disconsider all today's reports. (but the Ctrl + Right really causes the open app to crash)
felipemarins commentedon Jul 11, 2020
All yesterday's reports are valid without the Hacker's Keyboard workaround, and the Ctrl + Right causing applications to crash too
felipemarins commentedon Jul 11, 2020
In my smartphone (LG K10 with Android 6.0), it didn't work. Maybe it only work if initially the micro usb keyboard was plugged, but I'm not being able to connect it to the LG K10 to test.
ferdnyc commentedon Jul 12, 2020
I was going to say, if you were initially testing with Android 4.4.2, then unfortunately the odds are very good (or, bad?) that things will be worse with newer builds, if anything.
Android 8 - 10 really clamped down hard on a lot of potential security/permissions exploit vectors, and preventing alteration of any sort of system resources from third-party software (which KDE Connect of course is), when those changes can't be verified as coming directly from the user and not the software itself, was a major focus. That's how we lost access to the clipboard-mirroring functionality, it's why we'll almost certainly never be able to support any sort of device-screen remoting, and I wouldn't be surprised at all if it's the reason why this functionality ends up becoming increasingly crippled over time.
It's important to keep in mind that the difference between a microUSB keyboard and the KDE Connect remote keyboard is that the microUSB keyboard is a directly-connected hardware interface — it's for all intents and purposes a system keyboard, and its input is processed directly by the Android OS with elevated permissions compared to the input method apps.
But when GSConnect is sending keyboard events over the network to the KDE Connect app, there's no way the OS on the device side could possibly hope to verify that it's actually you, the authorized device owner, doing the typing. The possibility exists, and the potential is real (even if the likelihood is incredibly low) for those events to be fake keyboard inputs generated by an unknown (and potentially malicious) third party.
Ultimately the KDE Connect remote keyboard is just another untrusted, third-party Android input method app (a soft keyboard), so it's subject to all the limitations of every other soft keyboard. The same way you can't, say, use the on-screen keyboard to navigate the notification shade, you can't use KDE Connect's either. In fact, you can't do anything with the KDE Connect remote keyboard that you can't do with an on-screen keyboard — which is a lot.
In particular and especially, since roughly Android 7 or 8 all of the permissions dialogs are going to be off-limits to input methods for security reasons. (The same way that all browser web extensions are blocked from running on Chrome's internal
about:pages, for example.) Because it's obvious how tempting that could be, for some less-than-upstanding app developer who wants to sneak a "feature" into their input method so maybe it "helps" unsuspecting users, say by auto-accepting any distracting authorization pop-ups — if something like that were possible.A quick few experiments on my Android 9 phone seems to indicate it is not, though. The moment a system dialog or menu takes focus on my phone, the remote input session effectively goes completely dead. With
v40-rc1the remote input no longer gets disconnected like it used to withv39, but it's still completely unresponsive to any input until I dismiss the on-screen system menu/popup/interface and drop back to interacting with unprivileged user-space apps.(If Hacker's Keyboard really is able to do an end-run around some of those protections, even on more recent/secure Android releases, then I wouldn't be at all surprised if the app eventually ends up getting pulled from Google Play temporarily, until that can be addressed by either Google or the H K developers.)
andyholmes commentedon Jul 12, 2020
Good news is @nicolasfella submitted an MR yesterday, so I'll give that a test later tonight and maybe we can revert the workaround in
v40-rc1.For things like focusing the notification tray, it might be worth looking up that Android-x86 project since they could have a list of secret hotkeys that aren't well known.
felipemarins commentedon Jul 13, 2020
I only tested on Android 4.4.2 (and the result was better without the Hacker's Keyboard), but I will test on Android 6.0 (these are the only available for me at the moment)
ferdnyc commentedon Jul 16, 2020
Oh, my phone has that built in. But, tellingly, this list is found at
Settings > General management > Language and input > Physical keyboard > Keyboard shortcuts.
Not a one of 'em works from the software input apps (including KDE Connect's):