$reactions.ttsWithVariables

The method configures the audio and text template to use for speech synthesis with variables via Yandex SpeechKit Brand Voice Adaptive.

In the phone channel settings, the Enable Yandex v3 and Use variables switches must be enabled. Otherwise calling the method will result in an error.

Syntax

The method accepts an object containing the audioTemplate and textTemplate properties.

audioTemplate

audioTemplate describes the audio where the variable part should be inserted. It contains three properties:

• audio – the template audio URL.

The file must be publicly accessible via HTTP/HTTPS and satisfy the audio file requirements imposed by Yandex.

• textTemplate describes the audio transcription:

  • template is the template text, with variable parts replaced with variable names in curly braces.
  • textVariables is an array of objects describing each variable used. Every object contains two properties:
    • name is the variable name.
    • value is the transcription of the audio segment replaced with the variable.

• audioVariables is an array of objects describing the placement of variables in the recording. Every object contains three properties:

  • name is the variable name.
  • startMs is the time from the audio start to the beginning of the variable part, in milliseconds.
  • lengthMs is the length of the variable part playback, in milliseconds.

textTemplate

textTemplate describes the text to be synthesized by the bot. It contains two properties:

• template is the template text, with variable parts replaced with variable names in curly braces.
• textVariables is an array of objects describing the variables to be inserted into the template. Every object contains two properties:
  • name is the variable name.
  • value is the text to be synthesized.

The structure of audioTemplate and textTemplate follows the specification of AudioTemplate and TextTemplate as described in the Yandex SpeechKit documentation.

How to use

This is an example call to the method from the script:

$reactions.ttsWithVariables({
    "audioTemplate": {
        "audio": "https://mybucket.s3-ap-southeast-1.amazonaws.com/myfilename.wav",
        "textTemplate": {
            "template": "You have been approved a loan of {limit} dollars. Would you like to proceed?",
            "textVariables": [
                {
                    "name": "limit",
                    "value": "thirty two thousand"
                }
            ]
        },
        "audioVariables": [
            {
                "name": "limit",
                "startMs": 1850,
                "lengthMs": 1400
            }
        ]
    },
    "textTemplate": {
        "template": "You have been approved a loan of {limit} dollars. Would you like to proceed?",
        "textVariables": [
            {
                "name": "limit",
                "value": "sixty seven thousand"
            }
        ]
    }
});

The audio file for this example has thirty two thousand mentioned as the loan amount. audioTemplate specifies the audio file URL and the variable placement in the recording.

To improve code readability, consider storing audio template objects separately from the method call sites. For example, you can make use of YAML dictionaries.

textTemplate describes the phrase synthesized by the bot in the state where this method is called. Here, the limit variable will be replaced with sixty seven thousand. In a real script, the actual variable values will be different based on the dialog context.