topic Re: CREATE CONNECTION - Support for Community Connections? in Data Engineering

CREATE CONNECTION - Support for Community Connections?

ChristianRRL — Tue, 28 Jul 2026 22:18:09 GMT

Hi there, as the title asks, I'm wondering how Databricks Community Connections are being supported. For example, when I try to make a fresh connection via Spark SQL commands, I get an error because `COMMUNITY` type is not supported.

However, when I check on an existing community connection I setup earlier, I see that it's labeled as type "Lakeflow Community Connector":

When I check the same connection, I see it's labeled as `UNKNOWN_CONNECTION_TYPE`:

And when I try and create a connection with that type, I get the same error as earlier:

Re: CREATE CONNECTION - Support for Community Connections?

adnan_alvee — Tue, 28 Jul 2026 22:38:58 GMT

Lakeflow Community Connector connections are supported, but not as a normal Spark SQL CREATE CONNECTION type today. They’re supported through the Lakeflow Community Connectors flow/UI and companion CLI/API path, while SQL/SDK enum surfaces still don’t recognize COMMUNITY as a first-class connection type.

For Community connectors, use one of these paths:

- Lakeflow UI: New → Add or upload data → source under Community connectors.
- Community connector CLI: community-connector create_connection ... which stamps connection_type=COMMUNITY for you.

Re: CREATE CONNECTION - Support for Community Connections?

ChristianRRL — Wed, 29 Jul 2026 18:18:54 GMT

Hi @adnan_alvee , I understand but am hoping to get a bit more information about the Community connector CLI approach. Can I get some help to understand how exactly I would deploy a community connector via CLI? I've tried some approaches but am having no luck so far.

Re: CREATE CONNECTION - Support for Community Connections?

adnan_alvee — Thu, 30 Jul 2026 20:30:09 GMT

Hi @ChristianRRL I tried with FHIR and it worked. The process is the same for every source. To do GitHub instead of the FHIR example, you change exactly three things, the connector name (github), the credential (a GitHub token), and the list of tables (repos, issues, PRs). Every command and every step stays the same.

Step 1 — Install & authenticate
Install the community-connector CLI, point it at your workspace profile. Identical for any connector.

Step 2 — Create the connection
community-connector create_connection <CONNECTOR> <conn_name> -o '<options JSON>'
Registers a Unity Catalog connection securely holding the source config/credentials. Only the connector name and the options JSON differ.

What I ran (FHIR):

community-connector create_connection fhir adnan_fhir_conn \ --spec src/databricks/labs/community_connector/sources/fhir/connector_spec.yaml \ --options '{"base_url": "https://hapi.fhir.org/baseR4", "auth_type": "none"}'

Step 3 — Define what to ingest
A small pipeline_spec.json listing source tables → destination catalog/schema. Only the table names differ.

Generic — a pipeline_spec.json listing source tables → destination: { "connection_name": "<CONN_NAME>", "objects": [ { "table": { "source_table": "<SOURCE_TABLE>", "destination_catalog": "<CATALOG>", "destination_schema": "<SCHEMA>", "destination_table": "<TABLE>" } } ] }

Step 4 — Create the pipeline (Identical shape for any connector.)
community-connector create_pipeline <CONNECTOR> <pipeline_name> \
--pipeline-spec pipeline_spec.json --catalog <cat> --schema <schema>

What I ran (FHIR) — fhir_pipeline_spec.json: { "connection_name": "adnan_fhir_conn", "objects": [ { "table": { "source_table": "Patient", "destination_catalog": "uc_adnan", "destination_schema": "fhir_raw", "destination_table": "patient" } }, { "table": { "source_table": "Condition", "destination_catalog": "uc_adnan", "destination_schema": "fhir_raw", "destination_table": "condition" } }, { "table": { "source_table": "Encounter", "destination_catalog": "uc_adnan", "destination_schema": "fhir_raw", "destination_table": "encounter" } } ] }

Step 5 — Run & verify
community-connector run_pipeline <pipeline_name>
community-connector show_pipeline <pipeline_name>

community-connector create_pipeline fhir adnan_fhir_pipeline \ --pipeline-spec fhir_pipeline_spec.json \ --catalog uc_adnan --schema fhir_raw \ --repo-url https://github.com/databrickslabs/lakeflow-community-connectors.git \ --use-workspace-pipeline

Then query the Delta tables in UC. Identical for any connector.