Interactive Canvas

Interactive Canvas is a framework based on Google Assistant that can be used to complement a voice skill with a visual component.

This visual component is a web application the voice Google Assistant accesses during a conversation with a client.

Google terms currently allow publishing gaming skills created using the Interactive Canvas. Such skills must also be validated by the Google reviewer team.

Devices

Interactive Canvas skills are available on:

• Smart displays with Google Assistant
• Android-based mobile devices

When an Interactive Canvas skill is started on a device without a screen, it will function correctly without the visual part.

Skill operation

A skill used by Interactive Canvas consists of two main components:

• A conversation interface of skill script you can create on JAICP.
• A web application with a visual component displayed by the skill to the users on the screen during a conversation. You can use HTML, JavaScript and CSS to create a web application.

Connecting a web application

Specify the link to the web application in the Web app URL field in order to connect it when you create a Google Assistant channel.

The script

Use $response.replies with the htmlResponse type to output the visual component of a skill.

For example:

    script:
        var videolink = getLink("main_scene.mp4");
        var poster = getLink("main_menu.gif");
        $response.replies = $response.replies || []
        $response.replies.push({
                "type":"htmlResponse",
                "updatedState": {
                    "command": "UPDATE_STATE",
                    "video": videolink,
                    "poster": poster,
                    "buttons": [{
                        title: "Skip"
                    }]
                }
            });