Harvest

This page contains the setup guide and reference information for the Harvest source connector.

Prerequisites

To set up the Harvest source connector, you'll need the Harvest Account ID and API key.

Setup guide

For Airbyte Cloud:

1. Log into your Airbyte Cloud.
2. Click Sources and then click + New source.
3. On the Set up the source page, select Harvest from the Source type dropdown.
4. Enter the name for the Harvest connector.
5. Enter your Harvest Account ID.
6. For Start Date, enter the date in YYYY-MM-DDTHH:mm:ssZ format. The data added on and after this date will be replicated.
7. For Authentication mechanism, select Authenticate via Harvest (OAuth) from the dropdown and click Authenticate your Harvest account. Log in and authorize your Harvest account.
8. Click Set up source.

For Airbyte Open Source:

1. Navigate to the Airbyte Open Source dashboard.
2. Click Sources and then click + New source.
3. On the Set up the source page, select Harvest from the Source type dropdown.
4. Enter the name for the Harvest connector.
5. Enter your Harvest Account ID.
6. For Start Date, enter the date in YYYY-MM-DDTHH:mm:ssZ format. The data added on and after this date will be replicated.
7. For Authentication mechanism, select Authenticate with Personal Access Token from the dropdown. Enter your Personal Access Token.
8. Click Set up source.

Supported sync modes

The Harvest source connector supports the following sync modes:

• Full Refresh - Overwrite
• Full Refresh - Append
• Incremental - Append
• Incremental - Append + Deduped

Supported Streams

• Client Contacts (Incremental)
• Clients (Incremental)
• Company
• Invoice Messages (Incremental)
• Invoice Payments (Incremental)
• Invoices (Incremental)
• Invoice Item Categories (Incremental)
• Estimate Messages (Incremental)
• Estimates (Incremental)
• Estimate Item Categories (Incremental)
• Expenses (Incremental)
• Expense Categories (Incremental)
• Tasks (Incremental)
• Time Entries (Incremental)
• Project User Assignments (Incremental)
• Project Task Assignments (Incremental)
• Projects (Incremental)
• Roles (Incremental)
• User Billable Rates
• User Cost Rates
• User Project Assignments (Incremental)
• Expense Reports
• Uninvoiced Report
• Time Reports
• Project Budget Report

Performance considerations

The connector is restricted by the Harvest rate limits.

Build instructions

Build your own connector image

This connector is built using our dynamic built process. The base image used to build it is defined within the metadata.yaml file under the connectorBuildOptions. The build logic is defined using Dagger here. It does not rely on a Dockerfile.

If you would like to patch our connector and build your own a simple approach would be:

1. Create your own Dockerfile based on the latest version of the connector image.

FROM airbyte/source-harvest:latest

COPY . ./airbyte/integration_code
RUN pip install ./airbyte/integration_code

# The entrypoint and default env vars are already set in the base image
# ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
# ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

Please use this as an example. This is not optimized.

2. Build your image:

docker build -t airbyte/source-harvest:dev .
# Running the spec command against your patched connector
docker run airbyte/source-harvest:dev spec

Customizing our build process

When contributing on our connector you might need to customize the build process to add a system dependency or set an env var. You can customize our build process by adding a build_customization.py module to your connector. This module should contain a pre_connector_install and post_connector_install async function that will mutate the base image and the connector container respectively. It will be imported at runtime by our build process and the functions will be called if they exist.

Here is an example of a build_customization.py module:

from __future__ import annotations

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    # Feel free to check the dagger documentation for more information on the Container object and its methods.
    # https://dagger-io.readthedocs.io/en/sdk-python-v0.6.4/
    from dagger import Container


async def pre_connector_install(base_image_container: Container) -> Container:
    return await base_image_container.with_env_variable("MY_PRE_BUILD_ENV_VAR", "my_pre_build_env_var_value")

async def post_connector_install(connector_container: Container) -> Container:
    return await connector_container.with_env_variable("MY_POST_BUILD_ENV_VAR", "my_post_build_env_var_value")

Changelog

Version	Date	Pull Request	Subject
0.1.20	2023-10-13	31377	Use our base image and remove Dockerfile
0.1.19	2023-07-26	28755	Changed parameters for Time Reports to use 365 days as opposed to 1 year
0.1.18	2023-05-29	26714	Remove authSpecification from spec in favour of advancedAuth
0.1.17	2023-03-03	22983	Specified date formatting in specification
0.1.16	2023-02-07	22417	Turn on default HttpAvailabilityStrategy
0.1.15	2023-01-27	22008	Set AvailabilityStrategy for streams explicitly to None
0.1.14	2023-01-09	21151	Skip 403 FORBIDDEN for all stream
0.1.13	2022-12-22	20810	Skip 403 FORBIDDEN for EstimateItemCategories stream
0.1.12	2022-12-16	20572	Introduce replication end date
0.1.11	2022-09-28	17326	Migrate to per-stream states.
0.1.10	2022-08-08	15221	Added parent_id for all streams which have parent stream
0.1.9	2022-08-04	15312	Fix started_time and ended_time format schema error and updated report slicing
0.1.8	2021-12-14	8429	Update titles and descriptions
0.1.6	2021-11-14	7952	Implement OAuth 2.0 support
0.1.5	2021-09-28	5747	Update schema date-time fields
0.1.4	2021-06-22	5701	Harvest normalization failure: fixing the schemas
0.1.3	2021-06-22	4274	Fix wrong data type on statement_key in clients stream
0.1.2	2021-06-07	4222	Correct specification parameter name
0.1.1	2021-06-09	3973	Add AIRBYTE_ENTRYPOINT for Kubernetes support
0.1.0	2021-06-07	3709	Release Harvest connector!