BackDebugging and testing

Delivery timeline

Follow one notification from payload validation to the provider's final answer.

Reading It

Every send that creates a notification record gets a timeline: the events the engine recorded while handling it, in order. Where explain() predicts, the timeline tells you what actually happened.

const result = await notify.send({ /* ... */ })
const events = await notify.timeline(result.notification!.id)

// Only one delivery's events:
await notify.timeline(recordId, { deliveryId })

[
	{
		event: "payload.validated",
		message: "Payload validated successfully",
	},
	{
		event: "recipient.resolved",
		message: 'Recipient "user_123" resolved',
	},
	{
		event: "preferences.resolved",
		message: "Preference resolution complete: inbox, email enabled",
	},
	{
		event: "inbox.created",
		message: 'Inbox item created: "Rey mentioned you"',
	},
	{
		event: "delivery.created",
		message: "Email delivery created via resend",
	},
	{
		event: "delivery.sent",
		message: "Delivery sent on attempt 1",
	},
	{
		event: "provider.message_id_stored",
		message: "Provider message ID: 4f1c...",
	},
]

Results come back oldest first, up to 1,000 by default (pass limit to change that). Each event has these fields:

Field	Contents
`event`	The event type, from the table below
`message`	A readable description
`channel`, `provider`, `deliveryId`	Set on events about one delivery
`metadata`	Extra detail, such as the attempt number, the error, or a skip reason
`timestamp`, `seq`	When it was written; `seq` orders events written together
`recipientId`, `tenantId`, `workspaceId`, `notificationId`	Which send it belongs to

Events

Stopped early

A send stopped before delivery still gets a notification record, with a single event (or two, for suppression) explaining why:

Event	Meaning
`idempotent.replay`	The idempotency key was already used; the earlier result was returned
`deduplicated`	The dedupe key was seen inside its window. `metadata` has the key and window
`preferences.resolved` + `notification.suppressed`	Every channel was turned off or had no address
`rate_limited`	Over the limit. `metadata` has `max`, `windowMs`, and the scope

Normal path

Event	Meaning
`payload.validated`	The payload passed
`recipient.resolved`	The recipient was found. `metadata` says whether they have an email and a phone
`preferences.resolved`	Lists the channels preferences allow
`channel.skipped`	A channel was left out. `metadata.reason` is, for example, `preferences_disabled` or `missing_address`
`quiet_hours.deferred`	A channel is scheduled for when quiet hours end
`inbox.created`	The inbox item was written
`delivery.created`	An email, SMS, or webhook delivery was queued

Delivery

Event	Meaning
`delivery.attempt`	A retry is starting. Written before attempt 2 onward, with the attempt number and delay
`provider.error`	An attempt failed. `metadata` has the error and whether it was permanent
`delivery.sent`	The provider accepted it
`provider.message_id_stored`	The provider's message id, to look up in its dashboard
`delivery.failed`	No attempts left, or a permanent error
`fallback.triggered`	A fallback channel is firing, and why

Triage

The last event usually tells you what to do:

Last event	What happened	Next
`delivery.sent`	NotifyKit handed it over	Check spam, and the provider's bounce log
`delivery.failed`	The provider refused it every time	Read the `provider.error` events before it
`channel.skipped`	The channel never ran	`preferences_disabled` is the user’s choice; `missing_address` means no email or phone
`quiet_hours.deferred`	Waiting for quiet hours to end	See below if it never went out
`deduplicated`, `rate_limited`	Stopped on purpose	Check that the dedupe key or the limit is right

During an Incident

A timeline covers one send. For a wider view, start from the delivery records and pull timelines for the failures:

scripts/failures.ts

const since = Date.now() - 60 * 60_000
const failed = (await notify.deliveries.list(undefined, undefined, 500)).filter(
	(d) => d.status === "failed" && d.failedAt && d.failedAt.getTime() > since,
)

for (const d of failed.slice(0, 10)) {
	const events = await notify.timeline(d.notificationRecordId, { deliveryId: d.id })
	const error = events.findLast((e) => e.event === "provider.error")
	console.log(d.channel, d.recipientId, error?.message)
}

For rates and alerts over time, use hooks instead of reading timelines.

Storage and Retention

The memory adapter and both Drizzle adapters store timelines. A custom adapter without the optional timeline section simply has none.
Events are kept for 7 days and pruned now and then as new ones are written. Change that with timelineRetentionMs; 0 turns pruning off, and notify.pruneTimeline(olderThan?) then prunes on demand.
A failed timeline write never fails the send. It goes to onTimelineError, which logs to the console by default.
Messages and metadata hold ids, reasons, and error text, not the payload, but inbox titles appear in inbox.created messages.

In Tests

An outcome can be right for the wrong reason. Asserting on the timeline checks the path, for example that a fallback fired because the primary failed rather than because it was skipped:

const events = (await notify.timeline(result.notification!.id)).map((e) => e.event)
const failed = events.indexOf("delivery.failed")
expect(failed).toBeGreaterThan(-1)
expect(events.indexOf("fallback.triggered")).toBeGreaterThan(failed)