Android 12 may use machine learning to fix the Back Gesture Problem

Google released the first Developer Preview of Android 12 the other day, and we’ve been digging into the code to find everything that’s new. One of the most exciting changes we’ve spotted is an overhaul to how Android detects back swipe gestures. If implemented, Android 12 will use machine learning models to predict when the user intends to use the back gesture.

With the launch of Android 10, Google introduced its fullscreen gestural navigation system. Android’s gesture navigation system places a pill at the bottom of the screen that you can interact with to switch between apps, open up the recent apps interface, or go to the homescreen. The back button, meanwhile, was replaced with an inward swipe gesture that can be triggered from the left or right side of the screen. Much ink has been spilled about the problem with Android’s back gesture, but to Google’s credit, they’ve made the experience consistent across the ecosystem and have provided APIs for developers to ensure compatibility with the gesture. While plenty of apps have shifted away from using a Navigation Drawer, there are still plenty of apps where the back gesture can conflict with the in-app UI. To solve this problem, Google is testing a new machine learning-based approach to back gesture detection in Android 12.

How Android’s back gesture currently works is as follows. An invisible trigger area exists at nearly all times on both sides of the screen. This trigger area extends between 18dp-40dp in width from the sides of the screen depending on the user-defined back sensitivity setting. The user can trigger a back gesture by simply placing a finger anywhere within the inset and then moving that finger inward past a minimum distance. Google used phone screen heatmaps when designing the back gesture insets, and they settled on recognition areas that users feel are ergonomic and one-handed friendly.

Gesture navigation in Android 10+. Source: Google.

The problem with this approach, as Google themselves admit, is that some users still swipe to open navigation drawers, which conflicts with the back gesture. Every app is designed differently, but the back gesture trigger area still says the same. This one-size-fits-all approach to the back gesture thus doesn’t play nicely with how some apps are designed, so that’s why Google is experimenting with machine learning to replace the current model.

While investigating the changes that Google made to the double-tap back gesture in Android 12, XDA Recognized Developer Quinny899 discovered the presence of a new TensorFlow Lite model and vocab file called “backgesture.” The latter contains a list of 43,000 package names for both popular and obscure Android apps, including 2 of Quinny899’s own apps. We believe this list contains the apps that Google trained its machine learning model against — ie. they determined the most frequent start and end points for the back gesture on an app-by-app basis. Digging deeper, we discovered the machine learning model is referenced in the updated EdgeBackGestureHandler class in the SystemUI of Android 12. If a feature flag is enabled, then it seems that Android 12 will use the ML model to predict if the user intended to perform a back gesture or if they simply wanted to navigate in the app. The data fed to the ML model for inferencing includes the start and end points of the gesture, whether the app is in the list, and the display’s width in pixels. Alternatively, if the feature flag is disabled, then Android 12 simply reverts back to the standard back swipe detection method (ie. insets).

Currently, the ML-based back gesture prediction is disabled by default in Android 12 Developer Preview 1. It’s possible that Google may scrap this approach if it ends up not being superior to the existing inset-based model. However, we won’t know for sure until Google unveils the Android 12 Beta in a couple of months, as that’s the time that Google usually reveals its bigger changes to Android.

Thanks to PNF Software for providing us a license to use JEB Decompiler, a professional-grade reverse engineering tool for Android applications.

The post Android 12 may use machine learning to fix the Back Gesture Problem appeared first on xda-developers.

from xda-developers https://ift.tt/2M5tej1
via IFTTT