Chat analytics

Chat analytics reports how people use Qubit, the AI assistant, on every surface: the browser extension, the web app, Slack, Teams and pipeline runs. It shows who uses Qubit, how long replies take, which tools it calls, what it costs and where it fails, and it lets you read individual conversations. The sidebar link under Analytics & Reporting is named Qubit Chat Analytics. The page heading is Chat analytics.

Before you start:

• You need the Administrator role. The sidebar link is shown only to Administrators, and anyone else who types the address gets an access-denied page. See Access denied and sign-in errors.
• Treat this page as sensitive. Administrators can read any user's conversation here, including their questions and what tools returned. In Chat with Qubit each person sees only their own conversations. Tell your team that Administrators can read transcripts.

Choose the range and filters

1. Pick a range with Today, 7 days (the default), 30 days or Custom. With Custom, choose the dates in the range picker.
2. Read the line beside the buttons. It shows the range in your organisation's time zone and what the figures are compared with, for example "the previous 7 days".
3. Narrow the data with the filters Surface, User, Tool, Model and Tool outcome. Each applies as soon as you change it, and the lists offer only values that exist in the data. Select Clear to remove them.
4. Select Refresh to read the data now. Tick Auto-refresh (30 s) to read it every 30 seconds. Auto-refresh waits while a transcript is open.

If the page could not read some of its data, a warning appears above the figures.

Read the headline figures

Each tile shows the change from the previous period of the same length, as a percentage, no change or new.

A strip below the tiles adds Messages, Avg latency, Success rate, Model errors, Ticket-linked threads, Grants created and Legacy web chat. The success rate is the share of turns without a model error.

Overview tab

The Overview tab opens first. It has four charts: Turns per day, stacked by surface; Tool outcomes; Cost by surface; and Turn latency.

Under the charts are three tables:

• Surfaces has one row per surface, and the numbers behind the charts. A row named Unattributed (approx.) holds older usage that is not linked to any conversation.
• Top users lists the ten people with the most turns. Select the filter icon on a row to show that person's conversations.
• Tool sources chosen shows which tool sources people had selected in the tools button (All sources) of Qubit chat on their active conversations.

Conversations tab

The Conversations tab lists conversations, most recent first. When there are more than 500, it shows the latest 500 and says so. The columns are Last activity, Surface, User, Title, Turns, Tools, Errors, Tokens and Cost. A title can carry the ticket number, a routine mark or an archived mark. Select a column heading to sort. Select Export to download the list as an Excel file.

To read a conversation:

1. Select the eye icon on its row (Read the transcript). The Conversation transcript window opens.
2. Read the details at the top: surface, user, title, start and last activity, the number of turns, tools and errors, and tokens and cost.
3. Read the messages in order. Each is marked User, Qubit or Tool. Tool calls show the tool and, under Arguments, what was sent. Results show a Preview. The window also shows approval requests and the answers, and any errors.
4. Under Tool audit trail, check each tool run: Time, Tool, Outcome, Duration and Detail. A mark beside the time means the run was matched to the conversation by person and time, not recorded against it.

The transcript is read-only. A link to the page can open a tab or a conversation directly. For a tab, add ?tab= and one of conversations, tools, cost, errors or legacy to the address. For a conversation, add ?thread= and the Thread id shown in the details at the top of its transcript.

Tools & approvals tab

• Most-used tools and Slowest tools (p95) chart the ten leading tools.
• Tool performance has one row per tool: Calls, Success, Failure, p50, p95, Avg, Approval events, Denied and Last used. Calls are executions, meaning executed, failed and rejected for authorisation. Approvals and policy denials are counted separately. Select the filter icon on a row to filter the whole page to that tool.
• Approvals counts what happened to approval requests: Requested, Approved, Denied by the person, Left open (a new message moved on), Answered automatically (policy allows), Answered by a standing grant and Grants created.
• Other governed surfaces lists tools run by pipeline research, agent runners and copilot actions, for comparison. They are not part of the chat figures.

See Approving Qubit's actions and Tool Governance for how these decisions are made.

Cost & models tab

• Cost per day is stacked by surface. Turns priced only from the transcript add tokens but no cost.
• Model mix lists each model with its threads, calls, tokens and cost. An approx. mark means the row includes usage that is not linked to a conversation.
• Cost by user ranks people by cost and tokens. It is empty until usage is linked to a person.

Older usage that is not linked to a conversation appears as Unattributed (approx.). It can include some AI spend that was not chat. For all AI spend, see AI Usage Analytics.

Errors tab

• Error types charts the errors by kind.
• Error statistics gives Model errors (assistant replies), Error rate per turn, Failed tool calls and Policy denials.
• Common errors lists up to 50 distinct errors with their Type, Count, where they happened, Last seen, the message and who was affected. It shows up to three names, then +N. Start here when the error rate moves.

Legacy tab

The Legacy tab holds the older web chat and the interactive guide logs. It is history only: its figures are not part of the other tabs, and the older chat series stops at the date shown on the tab.

• Legacy messages per day charts the older web chat.
• Legacy chat interaction details lists the newest 200 matching rows. Filter with Search messages, Chat type, MCP tool and Status, then select Apply. Export downloads the rows, and Refresh reads them again. On a row, View details shows one exchange and View session shows the whole session.
• Interactive guide analysis lists guide steps that may need work: an average time over 5 minutes, or more than 20% of people abandoning the step.

With a Surface, User, Tool, Model or Tool outcome filter applied, Legacy messages per day and the Legacy web chat figure show nothing. Select Clear to see them again.

Where the figures come from

The Data sources card at the bottom of the page explains how each figure is measured and how many rows could be matched to their conversation. Read it when a number looks low. The page measures use, speed, errors and cost. It has no rating of answer quality.

Related pages

• Chat with Qubit: the assistant these figures describe.
• AI Usage Analytics: all AI spend, not only chat.
• Approving Qubit's actions: approvals and standing grants.
• Tool Governance: which tools may run.