Enable customer's products to interact with the Google Home based assistants.
With Google Home, your customers can command the connected products by voice commands.
Hey Google, set the thermostat to 21 degrees!
Hey Google, warm up the oven to 180°C
Configuration
To integrate the Google Home assistant on customer's products, you need to:
- Create (if not yet done) an account to access the Google Actions Console.
- Create a Google Action Project and configure it.
- Configure the Google Home plugin.
- Configure the Google Home Assistant for a Thing Definition you want to support.
- Install the Google Home App on your mobile phone (Android or iOS).
Note that this is required to test the Google actions you are going to configure.
To start using Google Home, your customers need:
- An account into the tenant where the Google Home plugin is configured.
- A mobile phone (Android or iOS) with the Google Home app installed.
- Optionally, a Google Assistant-based device (e.g. Google Nest Mini).
Devices that are connected to Google Home must declare a type and traits (capabilities).
For more details about traits, you can refer to the Google documentation.
Google Actions Configuration
Within the Google Actions Console, you can access the list of your Google Actions Projects. Select the desired one, or create a new project, by clicking the New Project button.
You can refer to this documentation https://developers.google.com/assistant/smarthome/overview.
Within the project configuration, you must follow the steps resumed in the Overview tab.
Share Smart Home Actions
In case the Google Action account is owned by an external user (for example, you are the system integrator), you can ask them to share the Action project with you by following the Share Action guide and assigning the right roles (e.g. Editor). This allows the Google action to be configured on behalf of the account holder.
Account Linking
Within the Quick Setup -> Setup account linking section, you must configure the API key and the endpoint URLs for OAuth2 authentication.
Client ID
The API key you have selected within the Google Home plugin.
Client secret
The secret key of the API key you have selected within the Google Home plugin.
Authorization URL
https://google-home.servitly.com/oauth/authorize
Token URL
https://google-home.servitly.com/oauth/token
Within the Build your actions section click on the Add Action(s) link.
Actions
Select the Develop tab, and provide the fulfillment URL, which represents the endpoint where Google can retrieve the list of the customer's devices and communicate with them.
Fulfillment URL
https://google-home.servitly.com/api/google_assistant
Home Assistant Configuration
For each thing definition, you can now configure how the product can interact with the Google Home assistant.
To configure the Home Assistant you need to:
- Enter the Remote Control / Home Assistants page.
- Select the Thing Definition to edit.
- Press the Add Home Assistant button and select Google Home.
- Confgiure the Devices to display in the Google Home App.
If the product is complex, you can configure multiple devices that will appear in the Google Home App as separate devices. For example, considering a heating system, you can have several zones to be controlled separately. Each device will have its own capabilities, and will go to interact with and control a particular aspect of the connected product.
In case the number of devices depends on the type of installation, you can choose a metric of the thing that provides the number of devices to be generated. In this case, each Device will also need to define the Index property needed to determine which Devices to enable depending on Index Device Count metric vlaue (metric.value <= device.index).
Within each Device you can configure:
- Name: the name to be displayed in the Google Home App.
You can use placeholders to include thing properties (e.g. ${thing.name}) - Device Type: the type of the device to generate (e.g. Thermostat).
For a complete list of available devices and their description, refer to the Google Home documentation. - Device Name Metric: the metric whose value can be used as device name.
- Index: in case of dynamic devices, the index used to enable the device according to the Indexed Device Count Metric.
- Capabilities: the list of capabilities associated to this device and corresponding to the Google Home traits. Refer to the Capabilities section below.
Capabilities
Here is the list of all capabilities you can use to integrate connected products with Google Home.
Dispense
The dispense functionalities any device that can dispense items.
For more details you can refer to the Google Home documentation page.
OnOff
The basic on and off functionality for any device that has binary on and off.
For more details you can refer to the Google Home documentation page.
PROPERTIES | |
---|---|
ON/OFF Status Metric | The metric providing the ON/OFF status. Type: METRIC | Mandatory |
ON Values | The comma-separated list of ON values. Type: STRING | Mandatory |
Turn ON Command | The command used to turn on the device. Type: COMMAND | Optional |
Turn OFF Command | The command used to turn off the device. Type: COMMAND | Optional |
FanSpeed
The basic functionality for any device that support setting the speed of a fan.
For more details you can refer to the Google Home documentation page.
PROPERTIES | |
---|---|
Speed Metric | The metric providing the current fan speed. Type: METRIC | Mandatory |
Speed Parameter | The parameter used to change the fan speed. Type: CONFIGURATION_PARAMETER | Optional |
OpenClose
The open and close functionality for any device that can be opened or closed.
For more details you can refer to the Google Home documentation page.
PROPERTIES | |
---|---|
Open/Close Status Metric | The metric providing the Open/Close status. Type: METRIC | Optional |
Open Values | Comma-separated list of values identifying that the device is open. Type: STRING | Optional |
Open Command | The command used to open the device. Type: COMMAND | Optional |
Close Command | The command used to close the device. Type: COMMAND | Optional |
StartStop
The start and stop functionality for any device that can be started, stoped and optionally resumed after a pause.
For more details you can refer to the Google Home documentation page.
PROPERTIES | |
---|---|
ON/OFF Status Metric | The metric providing the START/STOP status. Type: METRIC | Mandatory |
Start Values | Comma-separated list of values identifying that the device is started. Type: STRING | Mandatory |
Start Command | The command used to start the device. Type: COMMAND | Optional |
Stop Command | The command used to stop the device. Type: COMMAND | Optional |
Pause Command | The command used to put the device in pause. Type: COMMAND | Optional |
Resume Command | The command used to resume the device. Type: COMMAND | Optional |
TemperatureControl
Functionalities for devices (other than thermostats) that support controlling temperature, either within or around the device.
For more details you can refer to the Google Home documentation page.
PROPERTIES | |
---|---|
Temperature Metric | The metric providing the current measured temperature. Type: METRIC | Mandatory |
Temperature Setpoint Parameter | The parameter used to update the temperature setpoint. Type: CONFIGURATION_PARAMETER | Optional |
Temperature Setpoint Metric | The metric providing the current temperature setpoint. Type: METRIC | Optional |
Min Temperature Threshold | The minimum temperature value that can be set. Type: FLOAT | Optional |
Max Temperature Threshold | The maximum temperature value that can be set. Type: FLOAT | Optional |
Temperature Step Value | The minimum temperature increment that can be set. Type: FLOAT | Optional |
TemperatureSetting
Functionalities for thermostat-style devices supporting both temperature setpoints and modes.
For more details you can refer to the Google Home documentation page.
PROPERTIES | |
---|---|
Temperature Metric | The metric providing the current measured temperature. Type: METRIC | Mandatory |
Humidity Metric | The metric providing the current humidity value. Type: METRIC | Optional |
Temperature Setpoint Parameter | The parameter used to update the temperature setpoint. Type: CONFIGURATION_PARAMETER | Optional |
Temperature Setpoint Metric | The metric providing the current temperature setpoint. Type: METRIC | Optional |
Min Temperature Threshold | The minimum temperature value that can be set. Type: FLOAT | Optional |
Max Temperature Threshold | The maximum temperature value that can be set. Type: FLOAT | Optional |
Modes | The enabled modes and optional mappings. Type: KEY_VALUE | Optional | Values: off
heat
cool
eco
|
Mode Metric | The metric providing the current mode. Type: METRIC | Optional |
Mode Parameter | The metric providing the current humidity value. Type: CONFIGURATION_PARAMETER | Optional |
Command Execution and Timeout
When a request arrives from Google Home, the Servitly backend reads the payload and, based on the capabilities in the Home Assistant definition, updates the configured parameters or executes the commands.
Before returning the result of the command execution, the Servitly backend waits for feedback from the connected thing, in the case of configuration parameters it waits for the new value of the feedback metric associated with the parameter, instead for commands it waits for a specific metric based condition (e.g., the state has changed to RUNNING).
The Google Home application waits at least 9 seconds for a response before saying "The remote device is unreachable" or something similar. Note that Google Home suggests a latency less than or equal to 700ms.
To avoid this bad user experience, the Servitly backend waits a maximum of 6 seconds for thing feedback, after which it returns PENDING as the response status. If the thing feedback arrives immediately, it returns SUCCESS, otherwise ERROR in case of problems (e.g., invalid parameter, offline thing).
Note that SUCCESS or ERROR are determined statuses, instead PENDING means that the backend is still processing the command request, and the result is not guaranteed to be SUCCESS. In the latter case, the only way the user can verify that the command has succeeded is to check it in the Google Home app (e.g., the mode or the temperature setpoint has changed as desired).
In the case of executing commands on multiple devices (e.g., turning off all fan coil units), a PENDING result is most likely to be obtained because of the time required to send all commands to the remote things and wait for all feedback.
In order to keep response time low, avoid the reference commands having an On Condition to determine which parameters must be sent each execution (e.g. toggle command). It is preferred to define ad-hoc commands (e.g. one for the Start and one for the Stop), in this way the Google Home app interface can correctly handle the status, even if the user clicks the Start/Stop buttons very fast.
Do not use computed data (Counters or Derived metrics), nor Insight, as feedback metrics; these types of metrics are asynchronously calculated, and the value can take many seconds to update.
This may cause inaccurate results when interacting with the Google Home app.
Google Home App configuration
This is the final step to control your connected product through Google Home, this step generally is up to the end-users.
On your mobile phone, open the Google Home App, press the Add button, then click on "Set up device" and finally "Have something already set up?".
Google Home displays a list of providers, you must select the one that has been configured into the Google Actions. If you are already in a testing phase, it is enough to search for "test".
Once found and selected, you will be redirected to the login page where you must sign in with your application credentials, when done, Google Home retrieves the list of connected devices and add them to the main control view.
Utterance Translation
Voice interaction is available in several languages and is up to Google to interpret the utterance and call the Servitly backend API passing it the relative parameters (e.g. temperature setpoint).
Thing Discovery
By default, when the user has connected the account to the Google Home app, discovery will return all things visible to the user.
Alternatively, you can let the user decide which things should be visible in the Google Home app. To do this, you need to add a boolean property to the things, named "assistantEnabled."
In this way, for each thing, the user is free to enable or disable each from the home assistant discovery and control.
Limitations
- Depending on the user language there may be some problems in interpreting voice commands. Some requests may not be correctly interpreted, you will need to try rephrasing the command. For instance, in Italian the command "Dammi una tazza d'acqua" is not recognized, instead, this works correctly "Dammi un tazza d'acqua".
- Google Home expects a command to be executed in less than a second (700 ms), it waits for feedback for up to 8 seconds, after which if the command has not been executed it returns "There was a problem." Please verify the above paragraph Command Execution and Timeout .
- If the device support Modes (e.g. Thermostat ECO, HEAT, COOL), in the Google Home app, the mode selector may be not correctly displayed, not all supported modes have an icon, and also the relative translation may be triky for the user language.
- The TemperatureSetting capability at the moment does not implement the Temperature Set Range command available for devices supporting the heatcool mode.
State Update Notifications
Google Home supports notifications that are sent to users to notify them of a change in device status. For example, after a command is executed, notification of the new device state is required.
Asynchronous notifications are not mandatory, but are required for:
- Avoid strange behavior when using the user interface of the Google Home app. For example, an ON/OFF switch may change several times before being updated to the actual state once tapped.
- Avoid running manual tests. Without status update notifications, the test suite cannot be fully executed, and some steps must be performed manually, even to provide guidance to Google reviewers.
To enable notifications, from the Servitly Console, in the Google Home plugin, a key for the service account must be uploaded.
For more details about how to create a service account key, you can refer to this guide, and follow these stesps:
Testing your action
In order to test your Google Home integration by using the Google Home app installed on your mobile phone, you must enable testing in the Test tab within the Google Action project page.
For more details, you can follow the Set up testing guide.
Running test suite
In addition to manual testing, it is possible to run a test suite, which is automatically generated based on the configured traits. For more details, you can follow Test suite guide.
Here are some tips for running and passing the test suite:
- On your mobile phone, in the Google Home app you need to log in with the same account used to access the Google Actions console, which may be different from the one used to access the DPS and link the account.
-
On the test suite configuration page:
- Select a device (one is enough).
- You can set 2 seconds as the timeout to reduce the total duration of the test.
- Select the Test Request Sync option.
- In the browser open in a new tab the DPS application with an account that can edit the product name. Navigate to the device details page and click the Edit button to be ready to change the name when prompted.
- Before starting the test suite, make sure your device is synchronized. From the Google Home App, Device section, simply drag the screen down with your finger to activate device synchronization; a spinner will appear.
- During the test, at the end, a dialog box will ask you to modify a device to test device synchronization. Before clicking continue, you should:
- From the browser, within the DPS, edit the device name (add a suffix) and save.
- From the Google Home page, Devices, update devices (drag the screen down), the device name will be updated with the new one.
- On the test suite page, click the Continue button in the dialog box; the test should be marked as passed.
Release your action
To promote your Action to production and make it public, you must submit the Action for certification, and this requires to:
- provide all graphic resources and texts;
- run a test suite and provide the result file;
- complete the quesionaires about securtity and privacy;
- verify your brand, you must ensure that you are authorized to use the name you gave the Action.
For a complete list of steps to perform you can ref to the Release your Action documetation page.
Note that you can wait up to a several days to get feedback from the certification team, and you will probably have to resubmit your skill to correct minor problems. Therefore, be careful to plan the release of Action well in advance.
Comments
0 comments
Please sign in to leave a comment.