- 1135 Views
- 7 replies
- 5 kudos
Resolved! Error when trying to Create a STREAMING TABLE using Databricks SQL
Hi all,I am currently using the Databricks Free Edition, and trying to create a streaming table, using the following SQL script:%sqlCREATE OR REFRESH STREAMING TABLE sql_csv_autoloaderSCHEDULE EVERY 1 WEEKASSELECT *FROM STREAM read_files( '/Volumes/...
- 1135 Views
- 7 replies
- 5 kudos
- 5 kudos
Yes - I was able to do it no problem on Free, with the exact steps above, using the default supplied compute.
- 5 kudos
- 831 Views
- 4 replies
- 1 kudos
Resolved! Writing to data to a .csv file ( in the Databricks free edition)
I have been testing out a notebook. Is is possible to write any tabular outputs to a .csv file and if so would this be the filesystem or an S3 bucket.I get errrors when I try either approaches with the snippet below.#sort by match date to get the mo...
- 831 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @ilir_nuredini ,Thanks very much for the advice as this really helped esp DFS not available on the free edtions.I ran into an errror and I realised I needed to create a volume and schema first. So with with a little help from the AI assistant, -...
- 1 kudos
- 2019 Views
- 5 replies
- 0 kudos
Access S3 Tables from Databricks
Hi, When I was looking for how to access S3 Tables from Databricks, I came across this AWS blog post: https://aws.amazon.com/blogs/big-data/access-amazon-s3-iceberg-tables-from-databricks-using-aws-glue-iceberg-rest-catalog-in-amazon-sagemaker-lakeho...
- 2019 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi @zhouhaijia, Are you trying to access S3? What are you looking to solve in particular? All the best,BS
- 0 kudos
- 757 Views
- 2 replies
- 0 kudos
Issue with the SQL Server ingestion in Databricks Lakflow- Gateway Stuck at "Starting"
Context: I am working on a POC to explore Databricks Lakeflow ingestion feature. I have tried creating Ingestion pipeline with salesforce connector successfully and able to load the data into Delta table. However when I am trying to create ingestion ...
- 757 Views
- 2 replies
- 0 kudos
- 0 kudos
We were able to resolve the above issue by increasing the quotas for two separate compute families. One was the Standard F4s family, which apparently the worker uses and the other was the Standard EDV4 family which I guess is the driver node. Now we'...
- 0 kudos
- 345 Views
- 1 replies
- 2 kudos
Resolved! Need help in creating compute cluster
Hi everyone,I hope you’re doing well.I’m currently facing an issue while trying to create a cluster in Azure Databricks (Personal). Despite trying various approaches, I keep encountering the following error message:The VM size you are specifying is n...
- 345 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @Naveen_Sequeira ,If you increased quota and it didn't help then just try to use different region. In Azure, some machines are sometimes unavailable due to regional capacity issue. They should be available in another region though.You can try with...
- 2 kudos
- 231 Views
- 1 replies
- 0 kudos
Passing a list of images on AI_Query
HiI would like to know how can I pass a list of images to ai_query on the same call, It will be helpful to extract some metadata information on a multipage document, where I want to know if different images to see if it belongs to the same document.T...
- 231 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @Andreyai Has your issue got resolved?
- 0 kudos
- 3275 Views
- 11 replies
- 2 kudos
Issue with Non-Bulk Inserts Using JDBC on Databricks Runtime 14.1
Hello team,I am experiencing an issue with insert operations on Databricks using the JDBC driver. In my SAS Viya system, the DatabricksJDBC42.jar driver version 2.6.40 is configured. I’ve noticed that, up to Databricks Runtime version 13.1, insert op...
- 3275 Views
- 11 replies
- 2 kudos
- 2 kudos
The PR looks closed for now, any news on this? Also, I was wondering if there's a code snippet available on using this functionality
- 2 kudos
- 443 Views
- 3 replies
- 3 kudos
Resolved! Help to get the swags for the event.
Hello Team,I am hosting my first event on 13th September. The date is final, but the venue is something that I am working on, due to I am not creating the event. I'll be able to create it within a day or two. Can you please help me understand how I ...
- 443 Views
- 3 replies
- 3 kudos
- 3 kudos
Thank you for tagging @BS_THE_ANALYST !Hi @Devanshu ,Thanks for reaching out. I’ve sent you a DM with the swag request details, please share your response so we can move forward and support you better. ThanksRishabh
- 3 kudos
- 4591 Views
- 1 replies
- 0 kudos
the inference table table doesn't get updated
I setup a model serving endpoint and created a monitoring dashboard to monitor its performance. The problem is my inference table doesn't get updated by model serving endpoints. To test the endpoint I use the following codeimport random import time ...
- 4591 Views
- 1 replies
- 0 kudos
- 0 kudos
If the inference table is renamed, schema changed, deleted, or if the creator loses data access/the permissions are missing or have recently changed (for the endpoint creator), inference tables may silently stop updating.
- 0 kudos
- 313 Views
- 1 replies
- 0 kudos
How to install python libraries in DLT using databricks asset bundles
Hi Team,Is there any way we can install python packages to DLT using Databricks asset bundles. resources:pipelines:xysz:name: xyxconfiguration:input_file: test.jsonenv: ${var}permissions:- group_name: ${var}level: CAN_MANAGE- group_name: ${var}level...
- 313 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @nkrom456 ,Add a %pip install command at the top of your DLT pipeline notebook to install the required packages. This will ensure the packages are available on all nodes during pipeline execution.
- 0 kudos
- 264 Views
- 1 replies
- 1 kudos
Performance issue with the Databricks Add-on for Splunk 1.4.2
We are currently using the add-on for Databricks in our on-prem Splunk Enterprise environment. The connection goes directly to the cloud without a proxy. Unfortunately, the add-on is very slow and we lose about 20 seconds with every query. We see th...
- 264 Views
- 1 replies
- 1 kudos
- 1 kudos
It’s likely not Databricks but the Splunk add-on causing the delay. The databricksquery command in version 1.4.2 has known performance issues — most of the lag is inside Splunk while parsing results. Try upgrading to the latest add-on (1.5.x+), monit...
- 1 kudos
- 529 Views
- 2 replies
- 0 kudos
DLT Pipeline Design
I am new to DLT and trying to understand the process. My bronze table will receive incremental data from SAP in real time. In my bronze table, we are not maintaining any history and any data older than 2 weeks will be deleted. This data from bronze w...
- 529 Views
- 2 replies
- 0 kudos
- 0 kudos
The scenario you mentioned can be efficiently managed using a Delta Live Table (DLT) combined with a CDC (Change Data Capture) flow. From what I understand, the bronze table is always in append mode. Check out the link below for more details. Hope th...
- 0 kudos
- 511 Views
- 6 replies
- 2 kudos
Execute Immediate not working to fetch table name based on year
I am trying to pass the year as argument so it can be used in the table name. Ex: there are tables like Claims_total_2021 , Claims_total_2022 and so on till 2025. Now I want to pass the year in parameter , say 2024 and it must fetch the table Claims...
- 511 Views
- 6 replies
- 2 kudos
- 2 kudos
Hi, I believe the solution shared by Martison would fix this issue. In Databricks SQL, when using EXECUTE IMMEDIATE, the SQL string must be a single variable or single string literal, not an inline expression using string concatenation ('...' || clai...
- 2 kudos
- 597 Views
- 6 replies
- 20 kudos
Resolved! Databricks Community Innovators - Program
Hi, I'd like to know what's happened with the Databricks Community innovators program? https://community.databricks.com/t5/databricks-community-innovators/bg-p/databricks-community-news-members Is this still alive? I've applied and emailed: databrick...
- 597 Views
- 6 replies
- 20 kudos
- 20 kudos
Hi Mandy! Thanks for the introduction and update. I'm really looking forward to being a part of everything moving forward. Just echoing what @TheOC mentioned above, I'm happy to help/assist in whatever way I can.Absolutely buzzing based off the info ...
- 20 kudos
- 1092 Views
- 11 replies
- 43 kudos
Databricks Monthly Spotlight - Discontinued?
Hi, I'm curious what's happened with the Databricks monthly spotlight? https://community.databricks.com/t5/databricks-community-innovators/bg-p/databricks-community-news-members I can there hasn't been anyone in the spotlight since April 2025. Has th...
- 1092 Views
- 11 replies
- 43 kudos
- 43 kudos
Hi all,I just wanted to jump in this thread to say how motivating this kind of conversation is. It's great that we've got passion from users and Databricks employees alike to ensure integrity in the Community - it's really reassuring as someone start...
- 43 kudos
