Hi All,I am running a sample notebook from Databricks Documentation section on Higher Order Function on my community edition. I am running this notebook on DBR 12.2 LTS.Databricks Documentation URL : https://docs.databricks.com/optimizations/higher-o...
Any suggestions on how to stream data from databricks into snowflake?. Is snowpipe is the only option?. Snowpipe is not faster since it runs copy into in a small batch intervals and not in few seconds. If no option other than snowpipe, how to call it...
Hi @babyhari Hope everything is going great. Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we ...
We were finally able to get DLT pipelines to run the optimize and vacuum automatically. We verified this via the the table history. However I am able to still query versions older than 7 days. Has anyone been experiencing this and how were you a...
Hi @Gil Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you. Thanks!
Dear @Werner Stinckens and @Tyler Retzlaff We would like to express our gratitude for your participation and dedication in the Databricks Community last week. Your interactions with customers have been valuable and we truly appreciate the time...
Let's say I create a table like CREATE TABLE IF NOT EXISTS new_db.data_table ( key STRING, value STRING, last_updated_time TIMESTAMP ) USING DELTA LOCATION 's3://......';Now when I insert into this table I insert data which has say 20 columns a...
I tried running "REFRESH TABLE tablename;" but I still do not see the added columns in the data explorer columns, while I do see the added columns in the sample data
I am trying to write a code for Error Handling in Databricks notebook in case of a SQL magic cell failure. I have a %sql cell followed by some python code in next cells. I want to abort the notebook if the query in %sql cell fails. To do so I am look...
Hi @pjain We haven't heard from you since the last response from @daniel_sahal , and I was checking back to see if her suggestions helped you. Or else, If you have any solution, please share it with the community, as it can be helpful to others. A...
Why does copying a 9GB file from a container to the /dbfs lose me 50GB of memory? (Which doesn't come back until I restarted the cluster)
Hi James,Did you get to the bottom of this? We are experiencing the same issue, and all the suggested solutions don't seem to work.Thanks,Adrian
Hi, I'm trying to write data from RDD to the storage account:Adding storage account key:spark.conf.set("fs.azure.account.key.y.blob.core.windows.net", "myStorageAccountKey")Read and write to the same storage:val path = "wasbs://x@y.blob.core.windows....
Hello @Vadim1 and @User16764241763. I'm wondering if you find a way to avoid adding the hardcoded key in the advanced options spark config section in the cluster configuration. Is there a similar command to spark.conf.set("spark.hadoop.fs.azure.accou...
Does databricks has an oracle connector or it uses odbc?
@Dataengineers You could use the JDBC connector to connect to Oracle. This is also documented here - https://docs.databricks.com/external-data/jdbc.html
Hello Experts,We have data bricks on Azure. We need to provide a user interface to users so that some customizing table entries can be entered by end users which in turn are saved in the Delta table. Is there any feature in Databricks or tools that w...
Hi Viren,We are also looking for solution to a similar problem. what did you end up doing in this case?
On GCP I subscribed to Databricks in one project within the organization.Then I canceled this subscription and subscribed to Databricks in another project.When I try to login onto the newly subscribed databricks with google SSO:> There was an error s...
I can see the issue might be related to organizations or billing accounts. The new Databricks project I tried creating was on a different organization/billing-account than the test Databricks subscription I created a month back.I went back to the ori...
I am looking for a solution for Data Migration.Want to connect SFTP Server to Databricks to New Platform(Don't Want ADF in between)Is it possible?
I need help or insight regarding the following errors. My instructors (Brooke Wenig with Conor Murphy) ran this code successfully on our course video, but I cannot replicate what she did. Here is the code and below it is the outcome from my Cours...
DELETE the original Parquet table as a separate statementHow can this be done? simple query "DROP TABLE .... " still failing with "cannot be found"
Can someone help me by providing steps for creating a new group, as I could not able to find it anywhere? Actually, I wanted to create a new group for Hyderabad, India which I could not able to find in the Groups sections.@Kaniz Fatma @Sujitha Ramam...
Hi, To request a Group be created, please fill out the form linked, and the Community Team will be in touch in 3 business days.
I am facing an issue when assign permission on view created on unity catalog. The problem is I had create a user defined function (UDFs) in order to encrypt sensitive column, I create a view which call the functions and source table within the catalo...
Excited to expand your horizons with us? Click here to Register and begin your journey to success!
Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!
| User | Count |
|---|---|
| 1603 | |
| 744 | |
| 348 | |
| 285 | |
| 247 |
