Skip to main content

Testing

This article is part of a tutorial about creating an outbound call campaign using NLU.

  1. Telephony setup
  2. Script setup
  3. Campaign launch
  4. Analytics
  5. Additional features
  6. Testing (you are here)

Let’s wrap up this tutorial by testing the script we have developed.

Automatic tests​

When writing automated tests, the following telephony features should be taken into account:

  • The behavior of $dialer.getPayload.
  • Telephony-specific events.
  • Audio playback.

Main script branches​

Open the test.xml file and add test cases for the main branches of script walkthrough:

<test-case id="Positive case">
  <q>/start</q>
  <a state="/Start"/>
  <a state="/Start"/>
  <a state="/Symptoms"/>
  <q>yes</q>
  <a state="/Symptoms/Positive"/>
  <a state="/Goodbye"/>
</test-case>

Write a similar test case for when the client has no COVID-19 symptoms.

Template strings​

Cover the case when the client’s response about their symptoms cannot be recognized.

Additionally, in those places where the a tags are followed by answers containing template strings, specify the actual values of the expected client name and answer to the question:

a: Hello, {{$session.userName}}!
a: So you say: {{$parseTree.text}}. Remember about proper hygiene and wearing face masks.
tip
Anywhere but the telephony channel, $dialer.getPayload always returns an empty object, which is why a neutral address is expected in the test case.
<test-case id="Unrecognized case">
  <q>/start</q>
  <a>Hello, stranger!</a>
  <a state="/Start"/>
  <a state="/Symptoms"/>
  <q>it’s complicated</q>
  <a>So you say: it’s complicated. Remember about proper hygiene and wearing face masks.</a>
  <a state="/Goodbye"/>
</test-case>

Slot filling​

Intents with slot filling can also be covered by automated tests. For the /Callback state, create several test cases to check the different ways of specifying the callback time.

You can also use the <state> tag in order to avoid duplicating the same beginning of the dialog and shorten the amount of code:

<test-case id="Callback request, time is incorrect">
  <state>Symptoms</state>
  <q>could you please call back later</q>
  <a>When should I call you back?</a>
  <q>at breakfast</q>
  <a>At what time will you be available?</a>
  <q>at 11 am</q>
  <a state="/Callback"/>
  <a state="/Goodbye"/>
</test-case>

<test-case id="Callback request, time is correct">
  <state>Symptoms</state>
  <q>sorry I can’t talk at the moment</q>
  <a>When should I call you back?</a>
  <q>after 6 pm</q>
  <a state="/Callback"/>
  <a state="/Goodbye"/>
</test-case>

<test-case id="Callback request with time provided">
  <state>Symptoms</state>
  <q>please ring me back at 5 pm</q>
  <a state="/Callback"/>
  <a state="/Goodbye"/>
</test-case>

Unrecognized speech​

Let’s begin writing a test case for the /NoInput state.

First of all, check the simple processing of unrecognized speech. You have to use the <event> tag in order to emulate sending an event:

<test-case id="Bad connection">
  <state>Symptoms</state>
  <event>speechNotRecognized</event>
  <a state="/NoInput"/>
</test-case>

Audio playback​

Continue the previous test case and verify the audio playback upon experiencing connection issues.

A more versatile <responseData> tag is needed to check the contents of replies with the audio type. Using it to check only the first reply is enough, because the text reply that follows can still be covered by the regular <a> tag. Specify the first element of the replies array in the field attribute.

The full test case may look like this:

<test-case id="Bad connection">
  <state>Symptoms</state>
  <event>speechNotRecognized</event>
  <a state="/NoInput"/>
  <event>speechNotRecognized</event>
  <a state="/NoInput"/>
  <event>speechNotRecognized</event>
  <responseData field="replies[0]">
    {
      "type": "audio",
      "audioUrl": "https://example.com/comms-issue.wav",
      "state": "/NoInput"
    }
  </responseData>
  <a state="/Goodbye"/>
</test-case>
tip
Learn more about testing replies with the audio type

Manual testing​

You can open the test widget and test the script in text mode.

caution
Features specific to telephony, such as the speechNotRecognized event or $dialer methods, are not available in the widget.

Testing in the widget