Michael Shaheen

Eventing And Bulk With Sugar Integrate

Blog Post created by Michael Shaheen Employee on Jun 15, 2020

Integrations are all about enabling communication between disparate systems. So, when building a new integration, one of your top priorities should be to create frameworks for data transfer. Specifically, methods for detecting events in your endpoints and for sending and receiving new data rapidly.

 

Luckily, we like to give your integrations super powers, so every Adapter comes with pre-built, standardized event handling and bulk action frameworks. If you use Sugar Integrate, you can take advantage of these frameworks out of the box to get your integrations ready to go faster than ever.

 

 

Eventing

Normally for detecting events happening within an endpoint, you have two options: webhooks or polling.

 

Some platforms have built-in functionality to post events to webhooks, and you can configure that functionality through the Sugar Integrate UI and make sure that any data you receive is standardized. There are, however, many API vendors that do not offer webhook functionality out of the box. For these APIs you’ll need to use a polling framework.

 

 

Polling works by querying the endpoint for any objects that have changed within a set polling interval. If the query finds anything, it will return the new or changed data. While this sounds like a simple process, successfully implementing polling can be very complicated and varies from vendor to vendor.

 

One of the great features of Sugar Integrate is that all of our Adapters have a pre-built polling framework, so we handle the messy backend and automatically update any new data, just like using a webhook.

 

To set up a simple polling framework in Sugar Integrate, all you have to do is set up the polling event type and provide some objects to monitor for changes. For example if you set up Sugar Integrate to poll contacts every 15 minutes, that will say, “every 15 minutes, go out and check to see if any contacts have changed, and if that is the case, post the changes to a webhook.”

 

 

These events can then be used to trigger procedures. So, when Sugar Integrate polls Sugar Sell and notes a change in contacts, a custom procedure could be kicked off to do something with that data (like send a message to a Slack channel, or manipulate the data and then send it to another platform, etc).

 

Bulk

Sugar Integrate has two ways of implementing bulk actions with our Adapters. For the endpoints that natively support batch or bulk APIs, we can take advantage of those directly, but many do not support this functionality. For these endpoints, we’ve built a custom framework on top of the standard API to sort of create fake bulk actions.

 

For example, if you want to pull all the contacts out of an adapter that doesn't have a native batch or bulk API, you can implement this add-on framework which will one-by-one, pull out all those contacts and push them into a single file that you can then download.

 

One thing to note about our bulk framework is that it works asynchronously. What that means is that in order to create a bulk download file, you’ll actually execute two API calls. The first one will be the POST /bulk/query call, which will create a bulk job and then pull, individually, all the contacts out of that endpoint. The second one you’ll use is the GET /bulk/{id}/{object name} , which will actually grab the downloaded file once that first bulk job is finished. We have a similar process for uploading in bulk to an endpoint as well.

 

 

For any of the endpoints that do not natively support bulk or batch, our bulk framework is definitely something you’ll want to know, since it is an enhancement on top of the base API that is only available through our platform. It gives you a big advantage over a standard DIY integration, because you can use it to sync first-time data or import existing data from one CRM system to another, so it’s definitely worth playing around with.

Outcomes