Webhook Success Key #2: Track SharePoint Change History

John E. Huschka, January 25, 2018

You might assume that SharePoint webhooks are like other event architectures (such as database triggers) where a SharePoint event immediately triggers a webhook, like this:

This is not how SharePoint webhooks work. With SharePoint webhooks, multiple SharePoint events happen and then the webhook is called approximately once per minute:

The webhook request content sent to your handler will appear similar to the following

{
	"value": [{
	"subscriptionId": "b95bbc51-1fe3-4428-b2bc-1af28de60d94",
	"clientState": "CFWebHookDemo",
	"expirationDateTime": "2018-02-04T21:54:46.4940000Z",
	"resource": "038a9b03-4c33-4b2e-aadf-070a724d6337",
	"tenantId": "2b375469-c5ab-424e-9358-232b0848726b",
	"siteUrl": "/sites/WebhookDemo",
	"webId": "ab08cd87-f4c3-497b-b4f9-01f12b241954"
	}]
}

Note that it doesn’t provide details regarding the changes that have occurred. It has provided the SharePoint context (siteUrl, webId, and the resource, such as a list, to which your webhook is attached). It is now up to you to query the SharePoint change log to find out what events have occurred.

Specifically, in our example, the QueueTransactionProcessor submits a ChangeQuery back to SharePoint asking for event details.

The ChangeQuery includes a token (a semi-colon delimited String) that defines the SharePoint events for which it is requesting details. Here is a sample of how the token will appear:

The token is defined this way:

Notice this: Every value in our token except the event time is a known constant for a specific webhook. If we are going to avoid re-querying and re-processing events that we already have processed, we must know the time for which we last processed events.

In other words, our challenge is the following:

This is why our solution maintains a Webhook History list in SharePoint:

Our QueueTransactionProcessor, which is responsible for querying and processing the SharePoint changes, saves the latest SharePoint change token that it receives in this list:

Then, when the next webhook call is received, it retrieves this token from the Webhook History list to construct a new change token for querying the next set of changes:

In Microsoft's SharePoint Web Hooks reference implementation, this state persistence is provided by an Azure database within the ChangeManager. We have chosen to use SharePoint rather than introduce an additional datastore.

We at Collaboration Foundry are experts in SharePoint and Office 365. If you need assistance, we can help. Contact us.
Also, don't forget to take a look at our blog for more helpful tips and techniques.