- 914 Views
- 3 replies
- 1 kudos
Lakeflow jobs file trigger thru overwritten files
Is there any way a job can be triggered with overwritten files through a file trigger?I knew the limitations, but I just want to have other opinions or solutions.LimitationsOnly new files trigger runs. Overwriting an existing file with a file of the ...
- 914 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @DazzaiDe, Unfortunately, you can't do that as of today. Is there a specific reason you want to maintain the same name for the triggers? Instead of linking to a static file name, could you point to a root path so that any new files will trigger th...
- 1 kudos
- 474 Views
- 1 replies
- 0 kudos
Error while reading file deletion_vector_5274904e-d64e-44e9-9a4b-1d7e93dfdc0b.bin
org.apache.spark.sql.streaming.StreamingQueryException: [STREAM_FAILED] Query [id = c981c01f-d84b-4f2b-8b78-c7324cd7e752, runId = 7bdd53a0-0d2b-4456-aad8-734e82116c1d] terminated with exception: [FAILED_READ_FILE.NO_HINT] Error while reading file abf...
- 474 Views
- 1 replies
- 0 kudos
- 0 kudos
Tough to say from just this. You should create a support ticket with MSFT. Possible the deletion vector file was deleted out from under Auto Loader mid-read. Which might be a bug or it could be that you have some separate cleanup running at the same ...
- 0 kudos
- 1814 Views
- 1 replies
- 0 kudos
Resolved! Best practices for using autoloader
I’m looking to follow best practices with Databricks Auto Loader. When ingesting different file formats, is it considered good practice to always read data as strings first, or is it better to use format-specific readers (e.g., JSON, CSV, binary) fro...
- 1814 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey @DazzaiDe , Use the format-specific readers from the start and let Auto Loader handle schema, rather than reading everything as generic strings/text yourself. Key points: Always set cloudFiles.format to the real file format (json, csv, xml, parqu...
- 0 kudos
- 780 Views
- 1 replies
- 1 kudos
workspace config
"I am on a Premium AWS trial workspace (dbc-30503d28-2210). I have two issues:Personal access tokens are grayed out and I cannot generate themMy cluster cannot make outbound HTTP requests to external APIs (getting NameResolutionError when calling api...
- 780 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi, On the personal access tokens greyed out, you need to enable this in the workspace settings. (you'll need to make sure you're a workspace admin first). This is in the advanced setting under workspaces ettings. Are you validating you're a workspa...
- 1 kudos
- 3988 Views
- 1 replies
- 3 kudos
2026 update: Postman free plan limits and alternatives?
Hey,I’ve been rethinking my API tooling lately and realized I’ve mostly stayed with Postman out of habit.One thing that stood out again is the free plan limitations. They’re not new, but they make collaboration a bit annoying for small teams unless y...
- 3988 Views
- 1 replies
- 3 kudos
- 3 kudos
Hi @john26 ,I use Bruno quite often these days and it become the go-to Postman replacement for engineering-heavy workflows specifically because collections live as plain files in your repo. For someone working across Databricks, ADF, and Azure servic...
- 3 kudos
- 986 Views
- 3 replies
- 0 kudos
Resolved! RESOURCE_EXHAUSTED - I cannot work with a serverless in my free account. Do you know why?
Hi team, I am seeing this when trying to have a dummy project. Do you know why?
- 986 Views
- 3 replies
- 0 kudos
- 0 kudos
@alfredoigmen Databricks workspace's compute resources will be shut down and unavailable for the rest of the day (a month in extreme cases) if you exceed the quota. While the compute will be unavailable, Data and settings will not be deleted. You can...
- 0 kudos
- 835 Views
- 2 replies
- 1 kudos
When Trying to use Databricks Connect to Free Edition I am getting "Failed to Read Databricks table:
When Trying to use Databricks Connect to Free Edition I am getting "Failed to Read Databricks table: PERMISSION_DENIED: PERMISSION_DENIED: Cannot access Spark Connect. (requietsId=.....So I have not done anything to my instance aside from getting a ...
- 835 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi, I think this is the way you're configuring Databricks Connect. This error often happens when you're trying to configure Databricks Connect for a classic cluster rather than serverless. If you do the following, it should fix the issue. 1. Update ...
- 1 kudos
- 3903 Views
- 4 replies
- 3 kudos
Calling all Chennai residents! Join the Chennai User Group on Community!
Calling all Chennai residents! Join the Chennai User Group on Community! Are you passionate about our vibrant city of Chennai? Do you love connecting with like-minded individuals, expanding your knowledge, and contributing to a thriving Community...
- 3903 Views
- 4 replies
- 3 kudos
- 3 kudos
Thank you, will that be happening in future or any what's up group is available now?
- 3 kudos
- 2270 Views
- 3 replies
- 1 kudos
No views visible via foreign catalog
Hello,I have created a connection to a SQL Server. I have created a foreign catalog using this connection.When I show the catalog in the catalog explorer I can see the schemas and I can also see the tables and views in one schema. In another schema,...
- 2270 Views
- 3 replies
- 1 kudos
- 1 kudos
@benno did you got this solved? In the documentation there is no clear mention about support for views.
- 1 kudos
- 17795 Views
- 18 replies
- 3 kudos
Is it possible to view Databricks cluster metrics using REST API
I am looking for some help on getting databricks cluster metrics such as memory utilization, CPU utilization, memory swap utilization, free file system using REST API.I am trying it in postman using databricks token and with my Service Principal bear...
- 17795 Views
- 18 replies
- 3 kudos
- 3 kudos
@Walter_C Could you please share if there is any latest update on this ?
- 3 kudos
- 4896 Views
- 7 replies
- 0 kudos
Migrating Service Principals from Non-Unity to Unity-Enabled Databricks Workspaces - Entitlements Mi
Hello Databricks Community,I am currently in the process of migrating Service Principals from a non-Unity workspace to a Unity-enabled workspace in Databricks. While the Service Principals themselves seem to be migrating correctly, I am facing an is...
- 4896 Views
- 7 replies
- 0 kudos
- 0 kudos
1) Unity Catalog Is Account‑Level, Not Workspace‑LevelIn legacy Databricks:Entitlements (cluster creation, SQL access, workspace access) were workspace‑scopedOften implicit or auto‑assignedLoosely governedIn Unity Catalog:Governance is account‑centri...
- 0 kudos
- 4746 Views
- 2 replies
- 0 kudos
Issues Migrating Hive Metastore Tables to Unity Catalog
I am in the process of migrating tables from Hive Meta store to Unity Catalog using the Databricks Labs UCX tool. After completing the assessment phase, the dashboard indicates that while external tables are ready to migrate, the managed tables are m...
- 4746 Views
- 2 replies
- 0 kudos
- 0 kudos
We faced the similar issue, and find out the reason was not a tool failure, but a governance and storage incompatibility.By relocating data out of DBFS root and aligning clusters and DBRs with UC requirements, we enabled a clean, auditable, and compl...
- 0 kudos
- 1698 Views
- 3 replies
- 3 kudos
Resolved! Delta table zero downtime
Hello ExpertsHow to load everyday full load to delta table(truncate and load). The reports are pointing to this table.How to ensure the zero downtime while load is on.Thanks
- 1698 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @databrciks, You can achieve this in multiple ways, as shown below.. All of these patterns work. They do a copy-on-write overwrite which means Databricks writes new files and then atomically commits a new table version. There is never a committed ...
- 3 kudos
- 76084 Views
- 66 replies
- 37 kudos
Session expired Error - Databricks Community
Hi Team, After login into Databricks Community Edition within 2mins I am getting the following Error i.e, " Session expired. Your session has expired, please authenticate again." I have tried to login back again but still it is asking the Auth...
- 76084 Views
- 66 replies
- 37 kudos
- 665 Views
- 1 replies
- 1 kudos
Duplicate text in 2713/generative-ai-application-deployment-and-monitoring/lessons/48645/
https://customer-academy.databricks.com/learn/courses/2713/generative-ai-application-deployment-and-monitoring/lessons/48645/deployment-methodsTo whom it may concern (or please forward to the relevant group):I noticed that page 4 or 5 contains duplic...
- 665 Views
- 1 replies
- 1 kudos
- 1 kudos
@matjung Arigato.
- 1 kudos
