Black and white ropes forming a loose knot

Modest Proposals: Scoping Third-Party Integrations

The complexities of systems integration and how to describe your needs accurately in RFPs.

Let me tell you a secret: I read a lot of RFPs and project proposals. A lot. How many? Sometimes friends ask me to proofread their proposals for accuracy. At peak, I read three per day. That many. That is to say: Unless you are a professional proposal writer, I read more RFPs than your team writes. If there is one word I dread more than all others when reading RFPs, it's "integrate." By itself, a harmless word. But that's the problem: it is almost always by itself, with little or no context. Here's an example:

SEO strategy Integrate with Google Analytics and Chartbeat

Here's another:

CRM integration We use Salesforce for CRM and need to integrate that to the website.

So, what's the big deal? These are clear feature requests for linking up a website with a well-known service. Right? Well, no. An integration is not a one-size-fits-all solution. Integration means a tight coupling of separate services. An integration with Google Analytics isn't the same level of effort as adding Google Analytics tags to your pages. Here's a helpful guide to "integration" that illustrates what I mean. Lets think of integration in terms of tiers: increasing levels of complexity (and cost, and time) as we move from tier 1 to tier 4.

Tier 1: Push integration

At the lowest level -- the typical Google Analytics integration request -- what the feature really means is: build my website to push data to an external service. That external service (like Salesforce or Google Analytics), has a series of tools for tracking, analyzing and managing that data. All we need to do is make sure the data is there. This is a typical base integration with a service, and it is most often what people need.

It is generally the easiest because these services provide a bit of code that needs to be implemented once and then deployed. The key here is that interaction with the data collected by the service is performed through tools external to your web site. (Such as a Google Analytics dashboard.) When architecting a solution, our concern is simply to ensure that the data gets to the service, and that the process of doing so doesn’t slow down the site’s response time.

Tier 2: Pull integration

At the next level are external data services that we want to pull into the website. Sometimes these are user-facing (such as XML content feeds). Sometimes these are administrative reports (such as an analytics report for each landing page.) Pull integrations add a layer of complexity because the source data feed may break (for a variety of reasons, including power failures).

When that pull breaks, people tend to notice -- as was the case when Twitter changed its API recently and broke pull integrations on thousands of websites. Note that pull integration is often coupled with push integration. The service that collects data may also push data back to the site, generally so that administrators and editors may act on the information provided. With pull integrations, we have to understand the structure and source of the data. Solutions must handle cases when the data is corrupt, or the service unavailable. Both add complexity to the integration.

Tier 3: Active integration

An active integration is one in which the site pulls data from an external service and then uses it to change the behavior of the web site for a particular visitor (or class of visitors). Subscriber services are a good example of active integrations: the site may pull in a feed of user newsletters (managed in a service like Exact Target or MailChimp) and display that list to a user based on her email address. In another scenario, we may pull analytic data from Google in order to suggest links or place content relevant to the target demographic of the user.

Some active integrations can be as easy as push integrations -- but those are generally ad networks which handle the complete request cycle. Where the complexity comes in is when feature requests demand new functionality be created around data sets. In that second case, we must architect the processing of the data (as in Tier 2) and the actions around that data, such as the assignment of a user role. Event registrations are a good example of this type of integration. A typical workflow looks like so:

  • Given that a site visitor purchases an event ticket;
  • List the events schedule for that user on his account page;
  • Link those event listings to a list of other registered attendees.

In this case, we must enable the purchase of the event and then reconcile that action back to a user account so that additional actions may take place. As the complexity here should indicate, when we get to these higher-order integrations, it becomes crucial that the external service provides a solid. consistent, and well-documented method for interactions, generally in the form of a published API (see https://en.wikipedia.org/wiki/Api).

Tier 4: Full integration

At the highest level of complexity is the creation of a user interface on the website that lets people control their interactions with the external service. The most common example is the management of recurring payments or order histories on ecommerce sites. Here, we must depend on the external service having a robust, well-documented API to support the integration.

This level of request generally includes research, development, interface design and user testing. Which is why when your RFP says "I want to integrate with Salesforce," we may immediately assume a three to six-month project just to produce that single feature. Additionally, the user interfaces for such interactions need to be designed before they can be implemented. Certainly, this level of integration is possible -- with the caveat that the service must support a strong, read-write API -- but it is never trivial. Frequently, it requires support from the vendor. And in many cases, it requires that you pay a license fee to access the API, which needs to be figured into the overall project cost.

Costs

Speaking of cost: Be aware that the cost (and time) of implementing these four tiers generally increase exponentially, not linearly. That is not to say, however, that you shouldn't integrate with good services. The worst option, in most cases, is to ask a developer to write a custom replication of a service (such as an ad serving network). People pay for services like Salesforce or MailChimp for a reason: they get the job done well and require little to no internal maintenance. Rebuilding such a service from the ground up would commit resources beyond the length of initial development.

Advice

So what can you do to improve your proposals to clearly state your intentions? The simplest way is to describe the interactions that you expect from the site. A few examples from each tier should help:

Tier 1

Google Analytics All site pages will be instrumented with Google Analytics code to provide tracking and analysis. Salesforce Visitors who sign up for our site will have their name and email address sent to our Salesforce account.

Tier 2

Google Analytics Administrators may view real-time Google Analytics data for any page by pressing a button. Salesforce Administrators may go to a secure page and see a list of all contacts stored in Salesforce.

Tier 3

Google Analytics Visitors will see a list of suggested links based on their browsing history compared with history patterns of similar visitors (determined by demographics, if available). Salesforce Site visitors may see a list of all other visitors who have signed up, sorted by state.

Tier 4

Google Analytics Administrators may go to a secure page on the website to view reports about site usage, sorting reports by site section, date, and user demographics. Salesforce Site visitors may view and update their Salesforce contact information from their account page, and they may choose to opt-out of the service.

Conclusion

Remember that development firms must choose which projects to take and which projects to reject. Poorly articulated goals can land your proposal in the rejection pile. Clearly communicating your intentions will provide a strong foundation for responding to your proposal. By providing concrete examples of tasks to be done—and the benefits that will be gained—your proposal can help us provide a better estimate for the time and cost of your project. Even more than that, the level of detail in a proposal makes us excited to work with you. It shows us how much passion you have for the project.