Chat-Magics — Usage Dashboard

Every ChatMagics telemetry event in this Kusto endpoint over the last 90 days — 188,800 events across 11 commands — carries Tenant = A365BBCRunner. That tenant is Microsoft's internal BBC (Build/VHD validation) CI runner, used to validate Spark/Fabric VHD images before release. The chat-magics package is shipped into those VHDs via the ADO feed BBC-VHD_PublicPackages (see .pipelines/chat-magics-Official.yml:19) and validated by re-running test notebooks against each new VHD build.

Evidence A — only one MDS tenant emits ChatMagics here (90 d)

Evidence B — per-command runner-vs-real-user split (90 d)

Evidence C — CI infrastructure fingerprints on every row

A single raw row from the table reveals the CI stack:

Evidence D — rolling VHD builds, not human sessions

• 211 unique VhdId values over 31 days (Feb 16 — Mar 18) — a new VHD validated every ~3.5 hours.
• 7,624 unique notebooks in the same window, ~245 ephemeral notebooks created per day.
• 47% of notebooks finished all their events in < 5 min (single CI run), and another 23% finished within 30 min — classic short-lived test fixture lifetime, not human exploration.
• Top-traffic VHD 2db9f8fa… ran 15,604 %ignore calls against 805 different notebooks over 12 days, then was retired and replaced by the next VHD.
• Same tenant emits %chat, %code, %refresh_context alongside %ignore — it's exercising the whole chat-magics surface as a smoke test, with %%ignore wrapped around every cell.

Evidence E — chat-magics pipeline publishes to BBC-VHD feed

.pipelines/chat-magics-Official.yml:19 — ADO_FEEDS: "chat-magics-fabric,BBC-VHD_PublicPackages" ADO

BBC-VHD pulls the chat-magics package from this feed, bakes it into Spark/Fabric runtime VHD images, then runs validation notebooks (e.g. tests/magics/DataSummaries.ipynb) on each new VHD build. That validation suite is what produces all the telemetry visible here.

Validation KQL — reproduce these claims

// 1) All distinct Tenants emitting ChatMagics (90 d) — only A365BBCRunner shows up
SynapseMLLogs
| where TIMESTAMP between (datetime(2026-02-11) .. datetime(2026-05-12))
| where TelemetryApplicationInfo has "ChatMagics"
| where Message has "Telemetry: command="
| summarize Count = count() by Tenant, EnvironmentType
| order by Count desc

// 2) Per-command runner-vs-real-user split (90 d)
SynapseMLLogs
| where TIMESTAMP between (datetime(2026-02-11) .. datetime(2026-05-12))
| where TelemetryApplicationInfo has "ChatMagics"
| where Message has "Telemetry: command="
| extend Command  = extract(@"Telemetry: command=(\w+)", 1, Message)
| extend IsRunner = (Tenant == "A365BBCRunner")
| summarize Total     = count(),
            Runner    = countif(IsRunner),
            RealUser  = countif(not(IsRunner)),
            RunnerArt = dcountif(tostring(parse_json(TelemetryApplicationInfo).ArtifactId), IsRunner),
            RealArt   = dcountif(tostring(parse_json(TelemetryApplicationInfo).ArtifactId), not(IsRunner))
            by Command
| extend PctRunner = round(100.0 * Runner / Total, 1)
| order by Total desc

// 3) VHD rotation — 211 unique VhdId in 31 d during peak
SynapseMLLogs
| where TIMESTAMP between (datetime(2026-02-16) .. datetime(2026-03-19))
| where TelemetryApplicationInfo has "ChatMagics"
| where Message has "Telemetry: command=ignore"
| summarize Count = count(),
            Notebooks = dcount(tostring(parse_json(TelemetryApplicationInfo).ArtifactId))
            by VhdId
| top 20 by Count desc

// 4) Filter OUT the runner if you ever want a real-user-only view
SynapseMLLogs
| where TIMESTAMP > ago(30d)
| where TelemetryApplicationInfo has "ChatMagics"
| where Message has "Telemetry: command="
| where Tenant != "A365BBCRunner"   // strip CI noise
| extend Command = extract(@"Telemetry: command=(\w+)", 1, Message)
| summarize Count = count() by Command
| order by Count desc

// 5) 30-day runner-vs-real-user split per command — drives the "Usage by command (30 d)" table
SynapseMLLogs
| where TIMESTAMP > ago(30d)
| where TelemetryApplicationInfo has "ChatMagics"
| where Message has "Telemetry: command="
| extend Command  = extract(@"Telemetry: command=(\w+)", 1, Message)
| where isnotempty(Command)
| extend IsRunner = (Tenant == "A365BBCRunner")
| extend AppInfo  = parse_json(TelemetryApplicationInfo)
| summarize Total              = count(),
            Runner             = countif(IsRunner),
            RealUser           = countif(not(IsRunner)),
            UniqueWorkspaces   = dcount(Workspace),
            UniqueArtifacts    = dcount(tostring(AppInfo.ArtifactId)),
            RunnerArtifacts    = dcountif(tostring(AppInfo.ArtifactId), IsRunner),
            RealUserArtifacts  = dcountif(tostring(AppInfo.ArtifactId), not(IsRunner))
    by Command
| extend PctRunner = round(100.0 * Runner / Total, 1)
| order by Total desc

• chat_magics/src/chat_magics/chat_magics.py — magic decorators ADO
• ds_copilot/src/ds_copilot/context.py:241 — is_cell_our_magic allow-list ADO
• ds_copilot/src/ds_copilot/telemetry.py:286 — full EventCommandNames enum ADO
• Trident Certified Events Framework — secondary CE telemetry path Trident Wiki
• Where is my telemetry? · How to get access Trident Wiki