Receptionist A/B tests · Entagl Help Center

Overview

When experiments are enabled in your paid Receptionist workspace, open A/B tests from the bottom bar in Agent Flow or Agent Builder. Use Current, Drafts and History to find a test. Compare one change on a selected account or website target, then review the recorded outcomes before choosing what to keep.

Features

Saved agent versions

Compare original A with draft B for one exact account or supported target. The tested configuration is saved at launch. Keep the follow-up policy identical. Running versions are read-only; copy a finished test to make a new draft.

AI follow-up on or off

Compare the saved automatic follow-up policy with no experiment-controlled AI re-engagement. Staff messages, reminders, confirmations and unrelated campaigns remain separate. Count every assigned eligible conversation, not only successful sends.

Website chat presentation

Compare launcher only, welcome bubble or auto-open. Select desktop, mobile or both. Consent is required before enrollment. Keep the chat content and offer fixed. Count all enrolled eligible browsers, including those that never chat. A browser is not a unique person across devices.

Recorded outcomes

Choose one goal and a 24, 72, 168 or 336-hour measurement window. Goals use saved events, not an AI opinion. Pending and unknown outcomes stay visible. A generated greeting, opened panel, imported message or preview does not count as a customer message.

Choose the event that matches your goal

Depending on the test type, goals include a customer reply; 2, 4 or 6 customer messages; a booking request; a confirmed booking; a new lead; a newly captured phone or email; completed intake for a selected form and source; a business handover for selected reasons; a callback request; a created order; a paid order; or a website conversation start. Voice transcripts, imported history, system messages and previews do not qualify as customer messages. Tool errors and safety escalations are not automatically business handovers.

Pause, stop and collect outcomes

Pause stops new enrollment while existing measurement windows continue. Resume requires unchanged settings and an open original enrollment period. Normal stop returns new traffic to current normal routing. Emergency stop changes handling at a safe response boundary; it cannot unsend messages or undo a booking already saved. On a downgrade to Free, enrollment and experiment access close; ordinary Inbox records remain.

Common tasks

Create and review a test

Open A/B tests, then Create test. Select pipeline behavior, follow-up on/off or widget presentation.
Choose the exact account or website target. Check the handle, number, inherited routing and affected shared widget settings.
Keep A as the saved original. Edit B in the experiment draft. For widget tests, choose two or three supported presentation modes.
Write your hypothesis. Select one primary goal, its required form or handover reasons where relevant, and the outcome window.
Set fixed traffic shares and an enrollment end condition. Keep normal routing at the end or explicitly choose saved A, B or C where available. Review any permitted credit threshold, costs and readiness checks.
Launch only when checks pass. Use Flow and Changes to inspect saved versions, Results for counts and spend, and Activity for recorded actions. Permitted conversation links open Inbox with test and version filters.
When enrollment ends, allow existing outcome windows to finish. Review tracking and uncertainty. Use as main is a separate manual decision. Keep an inconclusive test in History or copy it to a new draft.

Saved A and B flows — Illustrative local QA screenshot of the actual interface. All accounts, messages, results and credits are synthetic; this is not production evidence.

Provisional results with pending outcomes — Illustrative local QA screenshot of the actual interface. All accounts, messages, results and credits are synthetic; this is not production evidence.

Tips

Availability depends on your workspace and team permissions. A test can be inconclusive and does not guarantee higher conversion.

AI usage consumes credits. Contextual widget welcomes may cost credits before a chat starts. A threshold controls new enrollment; already-started work may still incur cost.

Native mobile supports results and permitted controls. Create and edit tests on the web. Team permissions determine which results, conversations and saved prompts you can see.