Save Guarantee — Error Grouping & Scenario Analysis

A drill-down into the save-guarantee telemetry that detects when local notebook content fails to persist to the server. This page groups the raw error events into actionable buckets, separates real system bugs from expected user behavior, and points at the specific code path that causes each pattern.

Why this page exists

The headline metric on the weekly dashboard is save success rate, which counts attempted saves and whether they returned a 2xx. But many content-loss incidents never reach the save endpoint at all — the autosave callback never fires, or the user leaves a collab session with dirty buffers. The save-guarantee telemetry catches those by running an independent watchdog that compares local and saved content; when they diverge it spins up a save-monitor timer and logs whatever state it observes.

The raw events emitted by that watchdog look alike on the surface (they all say "Save monitor — save is not succeeded during the monitoring"), so the question this page answers is: which of those events represent an actual bug we should fix?

Where the telemetry comes from

Component name: saveGuaranteeHandlers - saveMonitor. Emitted from trident-de-ds-app on the frontend. No backend telemetry is needed for this analysis — every signal we group on is observable in the browser.

The save-guarantee state machine

Two timers run side by side on every collaboration-enabled notebook tab:

1. Content monitor (contentMonitorHandlers.ts) — long-period timer that periodically compares the local notebook JSON against the latest saved blob. If they match, the timer restarts. If they diverge, it kicks off…
2. Save monitor (saveMonitorHandlers.ts) — short-period watchdog that starts after content monitor detects divergence. It captures a snapshot of isDirty, isSaveLeader, and collabJoinStatus at start, and again at timeout. While it runs, the React save handler is expected to fire setSaveTriggerInfo → setSaveHandleInfo('sent') → setSaveHandleInfo('save-response-received'); the AzNB onSave event then clears the timer cleanly.

The three branches above are the only ways the timer can fire without a clean exit. The rest of this page focuses on branch A because in current PROD data it accounts for >99% of all save-monitor failures, and branch B is consistently zero.

Step 1 — Top-level error grouping

Last 7 days, PROD only. Counts are events / unique aznb instances / unique artifacts.

What this tells us:

• Branch A (no save triggered) absorbs ~99.5% of all "save not succeeded" events. The remaining 0.5% are the rare "save sent but no response" case.
• Branch B is consistently zero — i.e., once the React save handler picks up a save request, it always reaches at least the "sent" step. There is no bug to chase on the trident-side handler.
• Tag D (explicit-save fallback) fires 175k times — the safety net the monitor itself triggers when it sees a leader/standalone notebook with no save in flight. This is the recovery mechanism working as designed.

Step 2 — Decomposing branch A by scenario

Branch A fires whenever isSaveTriggered === false at timeout. That predicate is insufficient to decide whether anything is actually wrong — a notebook can be non-dirty by the time the timer expires, the user can be a non-leader in a collab session (where saves are delegated), the collab session can have died, etc. So we slice the same dataset by three more dimensions captured in errorInfo:

• isDirtyWhenTimeout — is there still unsaved content right now?
• isSaveLeaderWhenTimeout — does this client own the save responsibility?
• collabJoinStatusWhenTimeout — is the collab session healthy? (joined, inactive, failed, in-progress)

Step 3 — What each scenario means in code

S1 · Benign — dirty cleared by timeout (78.6%)

The local isDirty flag was true when the monitor started, but is false by the time the timer fires. In practice this happens when:

• The save-leader (could be this client or a collab peer) successfully wrote to the backend and AzNB cleared the dirty flag, but the dedicated onSave event didn't reach the save-monitor before the timer expired.
• The notebook was reloaded (e.g., from external-edit conflict resolution), which clears the flag without going through a save round-trip.
• The user undid all changes locally, dropping the dirty flag back to false.

No follow-up action is needed for S1 events — they are an artifact of the watchdog being more eager than the clean-exit signal. Filtering them out is the first step of any save-guarantee triage.

S2 · System bug — leader + dirty + joined, autosave never fired (8.3%)

This is the only branch worth chasing. The predicate is unambiguous: this client owns the save responsibility, the collab session is healthy, the content is unsaved, and a full save-monitor window elapsed without the autosave callback firing. The relevant call sites are:

// trident-de-ds-app/apps/de-ds-extension/src/notebook/pages/Authoring/FileViews/Notebook/NotebookFileView.tsx:2133
onSaveTriggered: (...) => {
    // ...
    setSaveTriggerInfo(artifact.objectId);   // ← this is what's supposed to flip isSaveTriggered=true
}

// trident-de-ds-app/apps/de-ds-extension/src/notebook/saveGuaranteeHandlers/saveMonitorHandlers.ts:93-122
const timerId = setTimeout(() => {
    if (!existingItem?.isSaveTriggered) {
        handleSaveNotTrigger(...);          // ← we reach here for every S2 event
    }
    // ...
}, saveMonitorTimerIntervalInMs);

Since branch B is zero, the failure is upstream of the React handler: the AzNB runtime's autosave scheduler — not the trident layer — failed to invoke onSaveTriggered during the watch window. Hypotheses to test (instrument these in @azure-notebooks/core-contract):

• The autosave timer was paused (e.g., tab visibility change, throttled background timer).
• The dirty signal that drives autosave wasn't delivered to the AzNB scheduler when the cell ops were applied via CRDT (i.e., dirty-from-collab vs dirty-from-self).
• A previous save errored and the scheduler stopped retrying without surfacing the failure to the save-monitor.

Affected scale: ~2,100 unique aznb instances and ~1,300 unique notebooks per week. Weekday baseline 3.7k–5.0k events/day, weekend dip to ~1.2k/day — pure traffic curve, no regression.

S3a · User scenario — left collab with dirty buffer (8.5%)

The collab session moved to inactive (the user explicitly disconnected, navigated away, or the keepalive died) while the local notebook was still dirty. The save-monitor still fires because the watchdog doesn't know to stand down — but the user has already been warned via the SaveNotCompletedMessageBar, which is rendered as soon as the monitor flags content might be lost.

This is the inherent risk of disconnecting with unsaved changes; the codebase already handles it. No additional action needed unless the goal is to reduce these (e.g., a confirmation dialog on tab close with dirty content, separate from save reliability).

S3b · Watch — collab session failed (0.2%)

When collabJoinStatusWhenTimeout === 'failed' and the client is not the leader, handleSaveNotTrigger takes the fallback branch and calls aznb.save() directly. The fact that we still see the "save not triggered" log for these means the fallback completed without raising — but the dirty flag is still set 30 seconds later. Most likely causes:

• Capacity paused (the early throw CapacityNotActive in useAutoSaveNotebook).
• 401/412 from the artifact endpoint, which the explicit save catches but doesn't flip the dirty flag back to false.

Volume is low (~750 events/week) but the fix is cheap: emit a follow-up log right after aznbInstance.save() in handleSaveNotTrigger that records the outcome, so the failures stop being indistinguishable from S2.

S4 · User scenario — non-leader in collab (4.4%)

In a real-time collab session, only the save leader writes to backend; co-editors are intentional free-riders on that save. isDirty is true on every client that holds local CRDT changes (it tracks divergence from the last-saved version), so a non-leader will always look dirty if anyone has typed since the last save. The save-monitor takes the "content might be lost" branch, which is correct: until the leader saves, the non-leader's view of "saved" is whatever the leader last wrote.

No bug here; the message bar messaging is the right outcome.

Recommendations

Source code references

Source queries (KQL)

let _start = ago(7d);
let base = cluster('pbiclients.eastus').database('appinsights').customEvents
| union cluster('pbiclientseu.northeurope').database('appinsights').customEvents
| where timestamp >= _start
| where customDimensions.TL_extensionName == 'DES'
| where customDimensions.TL_componentName == 'saveGuaranteeHandlers - saveMonitor'
| where customDimensions.TL_environment == 'PROD'
| extend msg = tostring(customDimensions.TL_message);
base
| extend errorType = case(
    msg == 'Save guarantee: Save monitor - save is not triggred', 'A_save_not_triggered',
    msg == 'Save guarantee: Save monitor - save is triggered while it is not handled', 'B_save_not_handled',
    msg == 'Save guarantee: Save monitor - save is not succeeded during the monitoring', 'C_save_not_succeeded',
    msg == 'Save guarantee: Save monitor - trigger save explicitly', 'D_explicit_save_fallback',
    'other')
| where errorType != 'other'
| extend artifactId      = iff(tostring(customDimensions.TU_artifactId) != '',
                               tostring(customDimensions.TU_artifactId),
                               substring(tostring(customDimensions.TU_documentId), 49))
| extend aznbInstanceId  = tostring(customDimensions.TU_aznbInstanceId)
| summarize events           = count(),
            uniqueInstances  = dcount(aznbInstanceId),
            uniqueArtifacts  = dcount(artifactId)
          by errorType
| order by errorType asc

let _start = ago(7d);
cluster('pbiclients.eastus').database('appinsights').customEvents
| union cluster('pbiclientseu.northeurope').database('appinsights').customEvents
| where timestamp >= _start
| where customDimensions.TL_extensionName == 'DES'
| where customDimensions.TL_componentName == 'saveGuaranteeHandlers - saveMonitor'
| where customDimensions.TL_environment == 'PROD'
| where customDimensions.TL_message == 'Save guarantee: Save monitor - save is not triggred'
| extend errorInfo = parse_json(tostring(customDimensions.TL_error))
| extend dirtyOut   = tostring(coalesce(errorInfo.isDirtyWhenTimeout,           customDimensions.TU_isDirtyWhenTimeout)),
         leaderOut  = tostring(coalesce(errorInfo.isSaveLeaderWhenTimeout,      customDimensions.TU_isSaveLeaderWhenTimeout)),
         collabOut  = tostring(coalesce(errorInfo.collabJoinStatusWhenTimeout,  customDimensions.TU_collabJoinStatusWhenTimeout))
| extend artifactId      = iff(tostring(customDimensions.TU_artifactId) != '',
                               tostring(customDimensions.TU_artifactId),
                               substring(tostring(customDimensions.TU_documentId), 49))
| extend aznbInstanceId  = iff(tostring(errorInfo.aznbInstanceId) != '',
                               tostring(errorInfo.aznbInstanceId),
                               tostring(customDimensions.TU_aznbInstanceId))
| extend scenario = case(
    dirtyOut  == 'false',                                                    'S1_benign_dirty_cleared',
    leaderOut == 'true'  and collabOut == 'joined' and dirtyOut == 'true',  'S2_bug_leader_save_callback_missed',
    collabOut == 'inactive' and dirtyOut == 'true',                          'S3a_user_left_collab',
    collabOut == 'failed'   and dirtyOut == 'true',                          'S3b_collab_failed',
    collabOut == 'joined' and leaderOut == 'false' and dirtyOut == 'true',  'S4_collab_member_not_leader',
                                                                              'S5_other')
| summarize events          = count(),
            uniqueInstances = dcount(aznbInstanceId),
            uniqueArtifacts = dcount(artifactId)
          by scenario
| order by scenario asc

let _start = ago(7d);
cluster('pbiclients.eastus').database('appinsights').customEvents
| union cluster('pbiclientseu.northeurope').database('appinsights').customEvents
| where timestamp >= _start
| where customDimensions.TL_extensionName == 'DES'
| where customDimensions.TL_componentName == 'saveGuaranteeHandlers - saveMonitor'
| where customDimensions.TL_environment == 'PROD'
| where customDimensions.TL_message == 'Save guarantee: Save monitor - save is not triggred'
| extend errorInfo = parse_json(tostring(customDimensions.TL_error))
| extend dirtyOut  = tostring(coalesce(errorInfo.isDirtyWhenTimeout,          customDimensions.TU_isDirtyWhenTimeout)),
         leaderOut = tostring(coalesce(errorInfo.isSaveLeaderWhenTimeout,     customDimensions.TU_isSaveLeaderWhenTimeout)),
         collabOut = tostring(coalesce(errorInfo.collabJoinStatusWhenTimeout, customDimensions.TU_collabJoinStatusWhenTimeout))
| where dirtyOut == 'true' and leaderOut == 'true' and collabOut == 'joined'
| extend aznbInstanceId = tostring(customDimensions.TU_aznbInstanceId)
| summarize buggyInstances = dcount(aznbInstanceId) by bin(timestamp, 1d)
| render timechart

let _start = ago(7d);
cluster('pbiclients.eastus').database('appinsights').customEvents
| union cluster('pbiclientseu.northeurope').database('appinsights').customEvents
| where timestamp >= _start
| where customDimensions.TL_extensionName == 'DES'
| where customDimensions.TL_componentName == 'saveGuaranteeHandlers - saveMonitor'
| where customDimensions.TL_environment == 'PROD'
| where customDimensions.TL_message == 'Save guarantee: Save monitor - save is not triggred'
| extend errorInfo = parse_json(tostring(customDimensions.TL_error))
| extend dirtyOut  = tostring(coalesce(errorInfo.isDirtyWhenTimeout,          customDimensions.TU_isDirtyWhenTimeout)),
         leaderOut = tostring(coalesce(errorInfo.isSaveLeaderWhenTimeout,     customDimensions.TU_isSaveLeaderWhenTimeout)),
         collabOut = tostring(coalesce(errorInfo.collabJoinStatusWhenTimeout, customDimensions.TU_collabJoinStatusWhenTimeout))
| where dirtyOut == 'true' and leaderOut == 'true' and collabOut == 'joined'
| extend artifactId     = iff(tostring(customDimensions.TU_artifactId) != '',
                              tostring(customDimensions.TU_artifactId),
                              substring(tostring(customDimensions.TU_documentId), 49))
| extend aznbInstanceId = tostring(customDimensions.TU_aznbInstanceId)
| summarize hitCount = count(), uniqueInstances = dcount(aznbInstanceId) by artifactId
| top 20 by hitCount
| order by hitCount desc

Cross-references

• Weekly Collab & Save Reliability dashboard — the headline save-success metric over time. The save-guarantee signal complements it by catching missing save attempts.
• How the weekly dashboard works — describes the generator pipeline and the broader telemetry source set.
• Fabric Real-Time Dashboard — Notebook-Save-Guarantee-Monitor — interactive version of every chart and table on this page; useful when the static snapshot is stale.