Skip to main content

Sendgrid

This page contains the setup guide and reference information for the Sendgrid source connector.

Prerequisites​

  • API Key

Setup guide​

Step 1: Set up Sendgrid​

  • Sendgrid Account
  • Create Sendgrid API Key with the following permissions:
    • Read-only access to all resources
    • Full access to marketing resources

Step 2: Set up the Sendgrid connector in Airbyte​

For Airbyte Cloud:​

  1. Log into your Airbyte Cloud account.
  2. In the left navigation bar, click Sources. In the top-right corner, click +new source.
  3. On the Set up the source page, enter the name for the Sendgrid connector and select Sendgrid from the Source type dropdown.
  4. Enter your apikey.
  5. Enter your start_time.
  6. Click Set up source.

For Airbyte OSS:​

  1. Navigate to the Airbyte Open Source dashboard.
  2. Set the name for your source.
  3. Enter your apikey.
  4. Enter your start_time.
  5. Click Set up source.

Supported sync modes​

The Sendgrid source connector supports the following sync modes:

Supported Streams​

Connector-specific features & highlights, if any​

We recommend creating a key specifically for Airbyte access. This will allow you to control which resources Airbyte should be able to access. The API key should be read-only on all resources except Marketing, where it needs Full Access. Sendgrid provides two different kinds of marketing campaigns, "legacy marketing campaigns" and "new marketing campaigns". Legacy marketing campaigns are not supported by this source connector. If you are seeing a 403 FORBIDDEN error message for https://api.sendgrid.com/v3/marketing/campaigns, it might be because your SendGrid account uses legacy marketing campaigns.

Performance considerations​

The connector is restricted by normal Sendgrid requests limitation.

Build instructions​

Build your own connector image​

This connector is built using our dynamic built process. The base image used to build it is defined within the metadata.yaml file under the connectorBuildOptions. The build logic is defined using Dagger here. It does not rely on a Dockerfile.

If you would like to patch our connector and build your own a simple approach would be:

  1. Create your own Dockerfile based on the latest version of the connector image.
FROM airbyte/source-sendgrid:latest

COPY . ./airbyte/integration_code
RUN pip install ./airbyte/integration_code

# The entrypoint and default env vars are already set in the base image
# ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
# ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

Please use this as an example. This is not optimized.

  1. Build your image:
docker build -t airbyte/source-sendgrid:dev .
# Running the spec command against your patched connector
docker run airbyte/source-sendgrid:dev spec

Customizing our build process​

When contributing on our connector you might need to customize the build process to add a system dependency or set an env var. You can customize our build process by adding a build_customization.py module to your connector. This module should contain a pre_connector_install and post_connector_install async function that will mutate the base image and the connector container respectively. It will be imported at runtime by our build process and the functions will be called if they exist.

Here is an example of a build_customization.py module:

from __future__ import annotations

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    # Feel free to check the dagger documentation for more information on the Container object and its methods.
    # https://dagger-io.readthedocs.io/en/sdk-python-v0.6.4/
    from dagger import Container


async def pre_connector_install(base_image_container: Container) -> Container:
    return await base_image_container.with_env_variable("MY_PRE_BUILD_ENV_VAR", "my_pre_build_env_var_value")

async def post_connector_install(connector_container: Container) -> Container:
    return await connector_container.with_env_variable("MY_POST_BUILD_ENV_VAR", "my_post_build_env_var_value")

Changelog​

VersionDatePull RequestSubject
0.4.12023-10-1331377Use our base image and remove Dockerfile
0.4.02023-05-1923959Add unsubscribe_groupsstream
0.3.12023-01-2721939Fix contacts missing records; Remove Messages stream
0.3.02023-01-2521587Make sure spec works as expected in UI - make start_time parameter an ISO string instead of an integer interpreted as timestamp (breaking, update your existing connections and set the start_time parameter to ISO 8601 date time string in UTC)
0.2.162022-11-0218847Skip the stream on 400, 401 - authorization required with log message
0.2.152022-10-1918182Mark the sendgrid api key secret in the spec
0.2.142022-09-0716400Change Start Time config parameter to datetime string
0.2.132022-08-2916112Revert back to Python CDK
0.2.122022-08-2415911Bugfix to allowing reading schemas at runtime
0.2.112022-08-1915800Bugfix to allow reading sentry.yaml at runtime
0.2.102022-08-1715734Fix yaml based on the new schema validator
0.2.92022-08-1115257Migrate to config-based framework
0.2.82022-06-0713571Add Message stream
0.2.72021-09-085910Add Single Sends Stats stream
0.2.62021-07-194839Gracefully handle malformed responses from the API
Was this page helpful?