> For the complete documentation index, see [llms.txt](https://boundaryai.gitbook.io/boundaryai-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://boundaryai.gitbook.io/boundaryai-docs/analysing-your-feedback/custom-monitoring.md). # Custom Monitoring Themes tell you what people are talking about. Sentiment tells you how they feel. **Custom Monitoring** tells you when something specific has happened (a safety risk in an open-ended response, a churn signal in a support ticket, a feature request buried in a Trustpilot review) and routes that signal to the right people, automatically. You do this by defining **monitors**: rules you write up front, rather than patterns the AI infers on its own. Where themes are emergent, a monitor is deliberate, you decide what matters to your team, and BAI Analytics watches every incoming response for it. A monitor can watch a single question on a single survey, or every source in an entire Feedback Group, continuously, with its own alerts and trend line over time. The setup and detection mechanics are the same either way; scope is just a choice you make when you create one. Monitors work across every kind of feedback BAI Analytics ingests: surveys, uploads, connector tickets, and scraped reviews. For where they're stored and how they roll up across a group, see [Feedback Groups](/boundaryai-docs/feedback-groups/feedback-groups.md). For surveys and uploads, see [Creating a Survey](/boundaryai-docs/bringing-in-your-feedback/surveys/editor.md) and [Uploading an Existing Dataset](/boundaryai-docs/bringing-in-your-feedback/uploading-an-existing-dataset.md). *** ### When to use a monitor Reach for a monitor when you know in advance that a particular signal needs to be watched, not just measured. * **Risk monitoring**: safety mentions in employee or student feedback, compliance concerns in customer support tickets, escalation signals in partner surveys. * **Operational triggers**: churn risk in NPS comments, urgent customer issues in support data, regulatory mentions in scraped reviews. * **Opportunity detection**: feature requests, competitor mentions, positive testimonials worth amplifying. * **Routing & escalation**: alerting the right team the moment a relevant comment lands, instead of waiting for someone to read the analysis. Monitors are not a replacement for themes. Themes surface what's emergent in the data; monitors surface what's important whether or not it's frequent. *** ### Anatomy of a monitor Most monitors are **topic monitors**: they describe a signal in plain language and the AI reads every comment for it. (A second kind, the **score monitor**, watches numeric answers instead; see Score monitors below.) A topic monitor has three layers: #### 1. The monitor itself A short label and description for what you're watching for. Example: *"Safety or Well-Being Risk"*. #### 2. Sub-categories Up to **five** more specific categories within the monitor. Example sub-categories for *Safety or Well-Being Risk*: *Physical harm*, *Mental health*, *Self-harm*, *Harassment*, *Other*. Sub-categories let you triage at scale, *all* safety mentions are caught, but the team handling mental health doesn't need to wade through harassment reports. #### 3. The detection result on a comment When the AI determines a comment matches a monitor, it stores three things on the comment: * The **monitor name**. * The matching **sub-category** (or *Other* if the comment fits the monitor but no specific sub-category). * A short **explanation** of *why* it matched, taken from the language of the comment itself. A single comment can match **multiple monitors simultaneously**: a customer ticket can be both a churn risk and a feature request, and BAI Analytics will record both. *** ### The predefined library BAI Analytics ships with eight built-in monitors, fully translated into the languages BAI Analytics supports. They cover the operational signals most teams ask for first: | Monitor | Use it to surface | | ----------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | | **Urgency or Critical Situation** | Comments that need immediate attention: outages, urgent business risks, time-sensitive incidents. | | **Safety or Well-Being Risk** | Physical, mental, or emotional safety concerns from respondents, employees, students, or customers. | | **Ethical, Legal, or Compliance Concern** | Mentions of legal exposure, regulatory issues, ethical breaches, or policy violations. | | **Escalation or Relationship Risk** | Comments that suggest a relationship is deteriorating: churn-adjacent, partner conflict, key-account dissatisfaction. | | **Integrity or Misuse Concern** | Suspected fraud, abuse, plagiarism, gaming of metrics, or other integrity issues. | | **Customer Churn Risk** | Direct or implicit signals that a customer or account is preparing to leave. | | **Feature Request** | Suggestions for new functionality, often hidden inside complaints or general feedback. | | **Positive Feedback** | Strong, quotable testimonials worth amplifying: for marketing, internal recognition, or product validation. | Each predefined monitor comes with a default name, description, and a starter set of sub-categories you can keep, edit, or replace. They are starting points, not constraints. *** ### Custom monitors For anything outside the predefined library, you can define your own monitor. Custom monitors can be created in two modes, chosen with the **Auto / Manual** toggle in the picker: * **Auto** (default) — you describe what to detect and the AI handles naming and matching. * **Manual** — you name and define the monitor yourself. #### How to create one 1. Open the monitor picker on a question (during survey creation or in the analysis view), or open **Manage monitoring** from a Feedback Group's Quick Overview for a group-wide monitor. 2. Click **New monitor** and pick **Auto** or **Manual**. 3. Provide: * **Name** (up to 100 characters). * **Description** (up to 1,000 characters). This is the most important field, because it's what the AI uses to decide whether a comment matches. * **Sub-categories**: (Optional, most choose to leave blank so they are automatically created based on matched data) up to five, each with its own name and short description. 4. Save. On a single question, the monitor starts running immediately; at the group level you can choose to run it now or save it for later. #### Tips for a strong custom monitor * **Lead with the signal you want to catch, not the wording.** Describe the *meaning* a comment must convey, not specific phrases. *"Mentions of regulatory or legal exposure (lawsuits, audits, regulator inquiries, compliance violations)"* outperforms *"contains the word 'lawsuit'."* * **Use sub-categories only when triage matters.** If your team handles every match the same way, a monitor with no sub-categories is enough. Sub-categories pay off when different sub-types go to different reviewers. * **Avoid overlap with other monitors.** BAI Analytics removes duplicates automatically, but two monitors with very similar descriptions ("Customer Churn Risk" and "At-Risk Account") will both fire on the same comments and add noise. #### Limits * Up to **150 monitors per survey** (predefined + custom + template). * Up to **5 sub-categories per monitor**. * Names: 100 characters; descriptions: 1,000 characters. *** ### Score monitors (quantitative alerts) Monitors are not limited to text. A **score monitor** watches a quantitative question (NPS or a numeric scale) and counts new answers that match a condition you set, for example *"NPS score at or below 6"* or *"rating below 3"*. To create one, open **Manage monitoring** on the Feedback Group and switch the composer from **Topic** to **Score**, then pick: * The **question type**: NPS or scale. * An **operator and value**: at or below 6, above 8, exactly 10, and so on. * The **sources** to watch. Score monitors reuse the same alerting as topic monitors: the per-monitor alert bell, the *"alert when N new matches arrive"* threshold, and the group's shared recipients, channels, and frequency (see Alerts below). Score alerts deep-link straight to the results view. Because matching is a plain numeric comparison, score monitors consume **none of your Usage Allowance**: no AI detection runs and no analysis is triggered. Conditions are evaluated every few minutes as new responses land, and counting starts from the moment the monitor is created, so you are never alerted about a backlog of old scores. Editing the condition restarts the count from that point for the same reason. {% hint style="info" %} Score monitors are available wherever monitor alerts are available on your plan. {% endhint %} *** ### Monitor templates When the same set of monitors is used across many surveys or groups, save it once and reuse it. A **monitor template** is an organisation-wide saved definition. Anyone in the organisation can pick it from the picker; selecting it copies the monitor into the survey or group, where it can then be edited locally without affecting the template. #### Working with templates * **Save a template**: from any custom monitor, click *Save as template* to expose it across the organisation. * **Mark as default**: mark any template as a default and BAI Analytics will pre-select it on new surveys. * **Permissions**: every member of the organisation can use a template; only the creator (or an admin) can edit or delete it. * **Recently used**: the picker tracks usage and bubbles recent templates to the top, which keeps everyday sets one click away. {% hint style="info" %} **Templates are starting points, not links.** When you add a template it is *forked*. Later edits to the template do not retroactively update the surveys or groups that have already used it. {% endhint %} #### Bulk-importing templates If you already maintain a monitor taxonomy elsewhere, import it in one go instead of recreating each one by hand. From the picker, click the import icon next to **New monitor** to open **Bulk import monitors**. * Upload a CSV or Excel file with two columns: the monitor's **name** (up to 100 characters) and its **description** (up to 1,000 characters). A header row is optional, BAI Analytics recognises it automatically, and a **Download example** template is provided if you want a starting point. * Up to **200 monitors** per import. * You get a preview with per-row validation before anything is saved. If a row's name collides with an existing template, you choose whether to keep both (renamed) or overwrite the existing one, and you can apply that choice to every conflict at once. *** ### Setting up monitors You can set a monitor up on a single question, or across an entire Feedback Group. The output looks the same either way, a monitor watching for matches, but where you configure it and what it applies to differ. #### On a single question Monitors can be attached to any Short Answer or Long Answer question at several points in the survey lifecycle: * **During survey creation**: in the builder, every Short Answer or Long Answer question has a monitoring picker. Open it to add predefined monitors, custom monitors, or templates. Monitors applied during creation start running on the very first response. * **From the analysis view**: for surveys that are already collecting data, open the survey's qualitative analysis, switch to the **Custom Monitoring** tab, and edit the list there. After saving, BAI Analytics re-runs detection across all historical comments so the analysis reflects the new definitions. * **On connector and scraper data**: connector imports and scraped data both produce open-ended fields the same way a survey or upload does. Monitors configured on the question apply to every ticket or review that lands in it, on every sync, automatically. {% hint style="info" %} **Monitoring is no longer configured while adding a source.** The upload, agent, and connector flows used to carry their own monitoring step; that step is gone. Instead, the final step of each flow shows a read-only note listing which of the group's monitors will watch the incoming data (see Automatic coverage below). To change what's watched, use the group's **Manage monitoring** panel. {% endhint %} #### Across a whole Feedback Group Open a Feedback Group's **Quick Overview** and switch to the **Custom Monitoring** tab, then **Manage monitoring** to open a side panel where you create and configure group-wide monitors. Two ways to start one there: * **New monitor** — the same composer described above, plus a choice of which sources in the group it should run on. * **From library** — pick one or more saved monitors from your organisation's library in a single multi-select picker, choose sources, and create them all at once. Either way you can **create and run it immediately**, or **save it for later** if you want to define it now and switch it on when you're ready (saving for later doesn't run detection, so it doesn't cost anything until you activate it). #### Automatic coverage of new sources Every group-level monitor has an **Auto-cover new sources** setting: a checkbox in the composer when you create it, and a toggle on each monitor row next to the alert bell. * **On** (the default for newly created monitors): whenever a source is added to the group (an upload, a new survey, a connector import, an agent run, or data pushed through the API), the monitor starts watching it immediately. Detection is seeded before the source's first analysis, so matches show up on the first pass, with no extra scan and no extra cost. * **Off**: the monitor watches only the sources you explicitly selected. The Manage monitoring panel shows an amber *"N sources not covered"* cue when coverage has gaps, with a shortcut to extend it. Monitors that existed before this feature keep manual coverage until you switch the toggle on, so nothing starts consuming your allowance without an explicit choice. *** ### How detection works Detection is **AI-based, not keyword-based**. The model reads each comment in context, decides whether it matches each configured monitor, picks the best sub-category if a match is found, and records a short explanation in the language of the comment. #### When detection runs * **As responses arrive**: new survey responses, upload rows, connector tickets, and scraped items are scanned shortly after they're analysed, rather than waiting for a full analysis pass to finish. New matches show up on the Custom Monitoring tab and trigger notifications without any manual action. * **On re-run**: when you change a monitor's definition on an existing survey, you can re-run detection across the historical data so the entire analysis reflects the new rules. At the group level, use **Re-scan all** to do the same after editing a monitor's description, or **Compute now** to re-aggregate results from data that's already been analysed, useful right after adding a monitor to a group with existing history. #### Detection mechanics in brief * The detection step batches comments and monitors together for efficiency and ranks evidence inside each batch, automatically screening out weak matches so results stay precise without you having to dismiss false positives by hand. * Multilingual: detection works in every language BAI Analytics supports, including French and Spanish. * A single comment can match many monitors at once. All matches are stored. * Sub-category fallback: if a comment matches a monitor but no specific sub-category, it lands under *Other*, visible in the UI as an *Other* sub-bucket so it never goes silently unreviewed. #### Re-running Editing a monitor's definition on an existing survey does not lose the old data. It queues a re-run that: 1. Loads every historical comment in the analysis. 2. Re-evaluates each comment against the new definition. 3. Updates the results in place. 4. Triggers any notifications that the new state warrants (subject to your notification thresholds). Re-runs run as background jobs, so you can keep working while they finish. The Custom Monitoring tab shows a *Re-running* state until the job completes; at the group level, a progress indicator shows which sources are being checked, with a completion toast when it's ready to review. *** ### Reviewing matches The **Custom Monitoring** tab, on a survey's qualitative analysis or on a Feedback Group's Quick Overview, is the primary review surface. #### What you see * **Counts per monitor**, with a clear secondary breakdown by sub-category (*Safety: 23, Physical 15, Mental 8, Other 0*). * **Trend indicators** showing how match volume has moved over time. * A **comments list** for each monitor, with the comment text, sub-category label, AI explanation, sentiment, and any segmentation context (channel, country, source). You can filter the list by monitor, sub-category, status (reviewed / unreviewed), and segmentation tags, and you can sort by recency or sentiment. #### Per-comment actions For each matched comment a reviewer can: * **Mark as reviewed**: track which comments have been triaged. Tracked per-comment with reviewer ID and timestamp. * **Reassign sub-category**: fix a misfire (e.g. a comment placed under *Other* belongs in *Mental health*). * **Dismiss**: remove the match from a comment that doesn't actually warrant it. The dismissal is preserved and won't reappear on re-run. * **Add a note**: leave context for downstream reviewers or report consumers. These actions create an auditable trail; bulk actions are available for sweeping similar comments at once. #### Exporting Matched comments can be exported alongside their explanation, sub-category, and metadata for offline triage or for sharing with teams that don't use BAI Analytics day-to-day. Exports respect any active filters, so an "Urgent / Last 7 days" export only includes the matching subset. *** ### Alerts (Monitor Notifications) Monitors are most useful when they reach a human quickly. Delivery is configured once **per Feedback Group**, shared across every monitor on it, under **Monitor Notifications**, so you can wire up different recipients for different programmes (Customer Support EU vs Engineering Bugs vs Campus Life Insights). Each monitor also has its own **alert toggle** and its own **threshold** ("alert when N new mentions arrive"), so noisy monitors don't have to page anyone on every single hit. #### Configurable per Feedback Group For each group you can set: * **Channels**: email and/or SMS. * **Frequency**: * **Immediate**: fires as soon as a monitor is triggered. * **Daily digest**: accumulates hits and emails them once per day at a chosen hour and timezone. * **Weekly digest**: same, but once a week on a chosen day. * **Recipients**: list of people, each with name, email, and (optionally) phone number. Recipients are scoped to the group, not to a single user account, so on-call rotations can be modelled by editing the recipient list. * **Which monitors to notify on**: *all monitors* by default, or a specific subset (e.g. only *Safety* and *Urgency*). * **Minimum threshold**: the smallest number of new matches worth notifying anyone about. Useful for daily digests on noisy datasets where a single match isn't worth interrupting anyone. #### Behaviour * **Immediate** notifications are sent as soon as a comment has been analysed. * **Digest** notifications are collected and sent on the schedule you choose. A recipient set to "daily at 09:00 Europe/Paris", for example, reliably receives their digest in that window. * **Empty digests are skipped.** If nothing matches in the digest window, no email is sent. *** ### Monitors in reports Reports scoped to a Feedback Group can include a section that rolls up: * Total matched comments across the report's time window. * Counts by monitor and sub-category. * Sample comments per category. * Trend over time. Because reports are scoped at the group level, a single PDF can summarise monitoring activity across surveys, uploads, connectors, and scrapers, useful for monthly readouts to stakeholders who don't log into BAI Analytics day-to-day. For one-off ad-hoc reviews, the Custom Monitoring tab's **Export** action is usually faster than a scheduled report. The exported Excel includes a **Sub-Flag** column (the sub-category breakdown) and a **Why Flagged** column (the AI's explanation). *** ### Limits and plan availability * Custom Monitoring is available on the **Standard plan and above**. Starter shows the picker but the *Save* action is replaced with an upgrade prompt. * **150 monitors** per survey (predefined + custom + template). * **5 sub-categories** per monitor. * **100 characters** per name; **1,000 characters** per description. * All eight predefined monitors and all sub-category depth are included on every entitled plan. There's no premium-only tier. *** ### Best practices * **Start with one or two monitors, not ten.** Most teams over-monitor at first and drown in low-priority hits. Add monitors as the need surfaces. * **Make descriptions specific.** A monitor's description is the AI's brief. Vague monitors catch too much and miss too little; precise ones do the opposite. * **Use sub-categories only where triage diverges.** If safety reports all go to the same person, *Safety or Well-Being Risk* without sub-categories is fine. Add them when a real human routing rule exists. * **Save proven sets as templates.** A set that works for one programme often works for the next; templates remove the repeat configuration cost. * **Set a threshold before turning on alerts.** A threshold of 1 on a busy group can mean constant notifications; start higher and lower it if you're missing things that matter. * **Re-run after every meaningful definition change.** Edits only apply going forward unless you re-run; re-running brings the historical data into line with current rules. * **Start group-wide monitors from your existing ones.** If a monitor already works well on one survey, add it to the group's library instead of redefining it from scratch. * **Leave Auto-cover new sources on.** The most common monitoring failure is a source added after setup that nobody remembered to cover. Auto-coverage closes that gap; switch it off only for monitors that are deliberately scoped to specific sources. * **Match monitor scope to Feedback Group scope.** A *Customer Churn Risk* monitor belongs on customer-feedback groups, not on employee-pulse surveys. Monitors only catch what's actually in the data they run on.