Web actions

When the flow decides to act on the page, the agent sends a web action alongside its speech. The SDK finds the element by what it's called on screen (its label, text or placeholder), does what a person would, and tells the agent whether it worked. Each action runs in the pause after the words that go with it, so the page moves while the agent says what it's doing.

Web actions are on in every voice session and widget unless you turn them off.

#Built-in actions

#Limit where actions run

Keep actions inside your app and away from other sites:

1createSession({
2  flowKey: "YOUR_FLOW_KEY",
3  webActions: {
4    root: () => document.querySelector("#app"),     // where to look (default: the whole page)
5    allowedOrigins: ["https://shop.example.com"],   // where navigate may go (default: this origin)
6    timing: "after-speech",                         // or "immediate"
7  },
8});

#Add your own actions

To replace a built-in action or add a new one, give a handler for its name. Return "ok" or "error", and the agent hears which:

1createSession({
2  flowKey: "YOUR_FLOW_KEY",
3  webActions: {
4    handlers: {
5      add_to_cart: async (action) => ((await addToCart(action.value)) ? "ok" : "error"),
6    },
7  },
8});

An action carries action (its name) and, as needed, target, within, value, selector, url and timeout.

#Handle actions yourself

To run every action with your own code, turn the runner off and answer each one:

1const session = createSession({ flowKey: "YOUR_FLOW_KEY", webActions: false });
2
3session.on("webAction", async (message, ack) => {
4  const ok = await myRouter.handle(message.action);
5  ack(ok ? "ok" : "error");
6});

#Next steps

• Headless sessions: see every session option and event.
• Flows: manage the workflow that sends the actions.