topic Re: HubSpot connector in Data Engineering

HubSpot connector

marcell_nagy — Thu, 08 Oct 2026 12:43:26 GMT

The HubSpot Support Lakeflow connector is able to download basic datasets. However, I see no way to get non-basic columns:

1. The suggested method to analyse email events is to use the portalSubscriptionStatus and source properties. The Lakeflow connector does not get these properties, even if the "Include future columns" is checked.
2. Being able to download columns that are set up in HubSpot as Custom Field would be great.

Can we achieve this right now? If not yet, it would be useful to be able to explicity list column names in the pipeline YAML even if the connector is not aware of that, but could dynamically download additional columns.

On the other hand, I think the email events dataset is not incremental, even if the reference tells that. Seemingly the connector awlays downloads all records but only stores the difference, it makes the pipeline slower than reasonable.

Re: HubSpot connector

Louis_Frolio — Thu, 08 Oct 2026 15:27:07 GMT

Greetings @marcell_nagy, I did some digging and here is what I found.

I think there are two separate issues here.

Missing columns (portalSubscriptionStatus, source, and custom fields)

In HubSpot's Email Events API, portalSubscriptionStatus and source aren't general event properties. They only exist on STATUSCHANGE events. The managed connector flattens events into a fixed schema, and the docs say automated schema evolution for new and deleted columns isn't supported, so "Include future columns" won't pull in type-specific properties the connector doesn't already expose. The same applies to CRM custom properties: column selection only works on connector-exposed columns, and the pipeline definition has no way to request an arbitrary source property by name. So the straight answer is: not through the managed connector today.

Two things to check before giving up on it, though. The connector has an incremental email_subscription_change table that may already carry the subscription status data you're after. And the limitations page notes that nested or variable HubSpot fields are stored as strings, so look at whether your email_events table has a string column holding the raw payload you could parse with from_json downstream. I haven't seen a published column list for that table, so I'd want you to confirm what you actually got.

On custom fields specifically: those live on CRM objects, and CRM Hub ingestion is in Beta behind the hubspot_connector_crm_objects workspace preview. I can't confirm from the docs whether the Beta pulls custom properties. Your account team can get you into the preview and route a feature request for explicit column pass-through to the ingestion team. Your post describes the use case well.

Workaround in the meantime: a small custom ingestion job that calls the Email Events API directly with startTimestamp as the cursor (filter to eventType=STATUSCHANGE), lands the raw JSON in a bronze Delta table, and flattens the properties downstream. Keep the HubSpot token in a secret scope or connection, not in the pipeline YAML.

Is email_events incremental?

Per the reference page, yes. The first run is always a full historical load, but later runs should use a cursor. If you're seeing full pulls after that, two things will help pin it down. Query the pipeline event log and compare rows read against rows written for the email_events flow across a few runs (flow_progress events carry those metrics). That tells you whether it's source-side retrieval, destination writes, or change processing. And check whether the slowness is actually rate limiting: HubSpot caps API calls (the docs cite 100 requests per 10 seconds for free apps) and the connector retries silently, which can make a modest incremental pull look like a full one. If the event log shows full reads, open a support ticket with the pipeline ID, table, run timestamps, and row counts. That's a bug, not a feature gap.

References:

HubSpot connector overview and feature table: https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/hubspot-overview
HubSpot connector reference (incremental vs batch tables): https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/hubspot-reference
HubSpot connector limitations: https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/hubspot-limits
Troubleshoot the HubSpot connector (slow pipelines, rate limits): https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/hubspot-troubleshoot
Ingest data from HubSpot (pipeline definition examples): https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/hubspot-pipeline
HubSpot Email Events API overview: https://developers.hubspot.com/docs/api-reference/legacy/reporting/email-analytics/guide

Hope this helps.

Regards, Louis.

Re: HubSpot connector

marcell_nagy — Fri, 09 Oct 2026 12:25:44 GMT

Can you please explain these parts:

"Your account team can get you into the preview and route a feature request" - how-where should I contact them?

"If the event log shows full reads, open a support ticket with the pipeline ID, table, run timestamps, and row counts." - where to open it? We're in Azure, you mean Help+Support system inside Azure Portal?

Re: HubSpot connector

Louis_Frolio — Fri, 09 Oct 2026 13:47:28 GMT

Greetings @marcell_nagy, Fair questions, I should've been more specific.

Account team. That's the Databricks people assigned to your company: account executive, solutions architect, or customer success contact. Separate from your HubSpot admin. If you don't know who they are, ask whoever at your company owns the Azure Databricks spend, or ask your Microsoft account rep to connect you. One thing to try first for the CRM preview: have a workspace admin check the Previews page under workspace settings. Some previews can be switched on there without involving anyone.

Support ticket. Two routes, and which one applies depends on your support contract. If your company routes Databricks support through Microsoft (the common setup on Azure), use the Azure Portal: Help + support, Create a support request, service Azure Databricks. Microsoft escalates to Databricks engineering when it's a product issue. You'll need an Azure support plan that includes technical support. If your company has a direct Databricks support contract, an authorized contact can file through the Databricks Help Center instead, reachable from the Help menu in your workspace. If you're not sure which you have, your account contact can confirm.

For the email_events ticket, include: workspace URL and region, pipeline ID and destination table, the affected update IDs and timestamps, whether each run was an initial load, normal update, or full refresh, rows read versus rows written, and the relevant event log excerpt. Leave HubSpot tokens and other credentials out of the ticket and out of this thread.

Feature request. Quickest route is in the workspace: click your username icon in the top bar, then Send feedback. Describe the use case (explicit column pass-through for the HubSpot connector) and tick the box allowing Databricks to contact you. That goes straight to the product team.

References:

Azure support options: https://azure.microsoft.com/support
Azure support plans: https://azure.microsoft.com/en-us/support/plans/
Submit product feedback from your workspace: https://learn.microsoft.com/en-us/azure/databricks/resources/ideas
Databricks Help Center (direct support contracts): https://help.databricks.com
Databricks support process: https://docs.databricks.com/aws/en/resources/support

Regards, Louis.