Hi there, I have what should be a somewhat straightforward implementation of a Custom Community Connector. I've tried to Dry run a few times but am encountering PYTHON.MODULE_NOT_FOUND_ERRORBelow is some context about my setup:Source Relative path: l...
Hi everyone,We're seeing a change in the behavior of multi-select dashboard parameters in Databricks SQL dashboards.Previously, when no value was selected in a multi-select filter, our query logic worked as expected using:SIZE(:my_parameter) = 0Howev...
The platform is aligning with standard SQL semantics (IS NULL etc) and you can follow it. To fix the queries returning zero rows on load, you need to explicitly evaluate for NULL before passing the parameter to any array-specific functions or IN clau...
This gist explains in detail, including root cause and reproduction: https://gist.github.com/rick-boyd-hightouch/35ff96d96edd2d15c39205c4b56527e4Basically the start/end parsing added for SQL scripting counts any `--` in the SQL text as a comment init...
Can someone help me understand what compute the SharePoint Connector is using by default? I'm trying to use Job Compute, but it "seems" like it may be defaulting to serverless.
The managed SharePoint connector uses serverless compute; it can't be switched to classic Job Compute. The Job you see is the orchestration layer: Lakeflow Connect creates it for the schedule and runs the ingestion pipeline as a task, while the pipel...
Hi Folks,My understanding isSELECT *FROM table_changes('mytable', x)should return all changes from version x through the latest available version, regardless of when I run it. However what happen to me is:When immediately after a streaming table upda...
Perhaps you try running one more variation?Your second query groups by date_extraction only which is a business column, so it can't tell us which commit those rows came from. Seeing the 2026-04-09 rows doesn't yet prove the result contains version 2...
Getting 'Unauthorized Access' when Serverless compute is trying to read s3 bucket data, same works fine with normal compute storage and external location, credentials, IAM policy and serverless egress rules all seems to be correct because normal comp...
One data point that may be useful, we run serverless SQL against several external locations that were created manually through the API rather than auto-created, and they read fine from serverless.So I don't think you've hit a general rule that custom...
Hi there,I am regularly getting 403 errors with Lakeflow Connect or Community Connectors accessing our UC catalog/schema. As far as I'm seeing, I believe that we have the proper role on our Azure storage account (Storage Blob Data Contributor), and t...
@ChristianRRL Most likely the compute behind these connectors is serverless, and that would explain both parts of your question. SharePoint and the community connectors run on serverless compute only as far as I know.The docs list serverless as a pre...
Hi, I'd like some help clarifying how lakeflow connect pipelines work (or should work). I went through the steps to (1) set up a connection, and (2) setup the ingestion, but I keep running into the following error:Operation failed: "This request is n...
Hi! The 403 error you're seeing is almost certainly not coming from SharePoint — it's coming from Azure Storage.The key clue is in the error detail: access denied to /cas cloud path. The /cas path is Unity Catalog's managed storage location (Catalog ...
Hi Community,As a senior data engineer migrating ETL workloads to Databricks (with Unity Catalog and Delta Lake), I'm building a cost-effective pipeline to ingest data from a REST API. Goals: minimize DBU costs, handle incremental loads, ensure scala...
HI @TheDataMaverick ,The most efficient approach for your REST API ingestion pipeline on Databricks is to use an external service like Azure Functions (or AWS Lambda) to handle API calls, then land raw JSON/Parquet in ADLS/S3 for Auto Loader ingestio...
I have completed all my course modules for the Associate Data Engineering Learning Pathway. I am still waiting for the coupon
Hello, Could you please explain how to enable the same UI interface that my colleague has (shown in the first picture)? Unfortunately, my interface looks like the second picture, where the Error code filter does not highlight or display the number of...
What you are seeing is likely a classic staged UI rollout. Databricks frequently pushes updates such as the aggregated error code metrics on the Job Runs page using user-level feature flags or testing cohorts. Because these flags can be evaluated at ...
Hi all,#Databricks#DatafactoryI hope you are well.I have bullet pointed my situation to allow other community members to easily diagnose my problem (I want to make your lives easy as well!).I have also included screenshot of data pipeline in DataFact...
The reason you are only seeing SQL Warehouses when clicking Compute is likely because the workspace admin has locked down interactive cluster creation. Check if you have the cluster creation entitlement. Your workspace admin can grant you the correct...
I'm building an Intelligent Document Processing pipeline using Databricks AI Functions (ai_parse_document and ai_extract).I want to ingest PDF files from a Google Drive folder into a Unity Catalog Volume. I can perform a full load successfully using...
You can use the Google Drive Managed connector that does most of it primarily built for syncing structured data directly to Delta Tables or the Standard connector that allows you to use Databricks (SQL, Auto Loader, pipelines) directly against a Goog...
when i change the yam file(not the resources one but can be any config yml) locally and then redeploy that bundle , the changes are not visible in the remote yml but other files(.py, .sql) do reflect these changes. FYI: during dev , i make changes in...
@prakharsachan What you're seeing is designed behavior, not a bug.When you run `bundle deploy`, the CLI does not upload everything in your bundle folder. It builds the upload list by walking the directory and skipping anything your .gitignore matches...
Hello @Jim ,I completed all four modules of Learning Pathway 1 (Associate Data Engineering) for the Advanced Learning Festival on July 6, 2026, the published end date of the event. My Academy account email is [datasci.patel.rupesh@gmail.com]. Screens...
Hi @Ashwin_DSA ,Thank you for the confirmation. Following up as it's now July 17 and I haven't yet received the voucher at the email address associated with my Academy account (I've checked spam/promotions folders as well).For reference, I completed ...
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 580 | |
| 349 | |
| 287 |
