Glassistant Google Assistant on Google Glass Part I

Note: This port isn't really complete but its been sitting on the backburner for over a year now so I'm pushing it out as a draft.

Update 2019-12-07: Google has announced a final software update for the explorer edition that "removes the need and ability to use your Google account on Glass". I'll likely not come back to finish this experiment at this point.

Google announced Enterprise Edition 2 of Google Glass. I'm very jealous I cannot buy one but that is probably for the best considering I bought a used Google Glass XE-C (the last Glass revision available to consumers) off of Ebay back in February of 2018 and never finished my project with it...

The goal of the project was to add Google assistant connectivity to the Google Glass. At the time (end of 2017) Google had just announced updates to the Google Assistant SDK that allowed additional ways to call the assistant API. My plan was to adapt the examples given for running assistant on Android Things to instead run on the Glass. I decided to call the project Glassistant because it simultaneously breaks the branding guidelines both Glass and Assistant.

Previous Implementation

Back when I had implemented the project in 2018 I was able to issue Assistant requests over the API:

However the project was lacking in a couple areas and Glass runs an older version of Android (4.4 KitKat) which complicates matters. The primary areas I wanted to address this time are:

• There was a bug in GRPC that prevented it from using the correct Security Provider. I had to depend on a custom build of GRPC while waiting for the changes to be released.
• There was no pure voice driven invocation like on normal android devices. You had to first say "Ok Glass" to bring up the voice triggers menu and then say "ask Assistant....".
• There was essentially no user interface. It would display the response from Assistant on a Glass Card but there was no clear flow and no way to make multiple requests without restarting.

On the problem with Security Providers in GRPC:

Android 4.4 KitKat does not come with newer Java security providers required for GRPC. Normally the recommended way to update security providers on Android is through Google Play Services. Unfortunately play services is not available for Glass. Instead GRPC recommends using the conscript library by Google and manually registering the new security provider. Recent versions of GRPC-Java will now use conscrypt and you no longer need a custom build.

On the problem of voice invocation and cleaner UX

This is still a work in progress and enough work that I'd like to provide more detail in a future post.

Porting Process

The Glass SDK is still available for use in Android Studio. In the Android SDK section of the settings insure you have installed the 4.4 (KitKat) SDK Platform. It doesn't mention Glass but if you check the "Show Package Details" checkbox you will see it includes "Glass Development Kit Preview".

Next I created a new project starting with the Glass Immersion Activity as the base.

To issue unregistered Glass voice commands during development you need to add a permission to the manifest.

The assistant integration is derived from sample Google Assistant project for Android Things. I first started by migrating the GRPC module over into the base Glass project. I also had to move to a newer version of GRPC to fix the security provider bug.

Next step was adapting the SDK integration they had to work on Glass. The EmbeddedAssistant class provides most of the heavy lifting but there are some Android Things related pieces that need to be excised since the demo can invoke assistant with a button press. Additionally since the Glass is running 4.4 there are are number of audio api updates (AudioDeviceInfo, AudioTrack, etc) that are not available and need to be replaced with calls for the older API.

Testing

There is no way to emulate Glass you can only test by running directly on the device. You also need to enable debug on your Glass. Also since I was developing on Windows 10 I needed to perform a fix for adb to pick up Glass.