From past two hours i am trying to create a cluster but it is getting failed after trying of 15-20 minutes.the red sign is coming after the cluster name which says that : Bootstrap Timeout:Node daemon ping timeout in 780000 ms for instance i-05ff43bc...
Hi Team,If a cloud application wants to read some data from Databricks we realize that Azure Databricks can expose its Tables via REST API.For its authentication what is the most recommended method - I see we can have PAT token tagged to a Service Pr...
Hi @Sid1805 ,I am in the same situation as you. I am looking for guidance how to setup the authentication for REST API to connect with Databricks SQL. I started with PAT and used the user id as "token" and pwd as PAT Token to connect with Databricks...
we are making configuration adjustments in the workspaces to disable public access. My concern is whether this will impact the Power BI dashboards that currently consume data from our workspace's SQL Data Warehouse.
If you're using Azure Databricks, you can create a vnet data gateway on Azure.https://learn.microsoft.com/en-us/data-integration/vnet/create-data-gatewayshttps://learn.microsoft.com/en-us/data-integration/vnet/high-availability-load-balancingYou need...
I created a new Databricks community account a few days ago. Unfortunately, I seem to have forgotten the password associated with it. Despite trying the "forgot password" option multiple times on the login page, I haven't received any emails to reset...
Hello all,We have encountered a problem while running a notebook on a unity catalog cluster by using Azure data factory managed identity. When we add the managed identity directly to the unity catalog table it’s working but if we add the managed iden...
currently, we are using PAT token for authentication purposes to generate tableau reports form data in databricks delta tables using databricks tableau connector. Would somebody know if system/function ID can be used from Tableau to Databricks inste...
@-werners- @Retired_mod can you help here plz?
Hi All,my question is in regard to how data in salesforce data cloud gets unified based on client profiles. Can similar action be done on data in databricks. i believe unity catalog just provides unified layer for security and governance. is there a ...
You want to identify actual persons based on one or more profiles (based on e-mail address etc). That is something that is not available out-of-the box in Databricks. The 'unified' in Databricks means you have a single platform for several data top...
Why does something like df.style.hide_index() turn out so ugly in Databricks? That command should show the dataframe pretty like always, but simply with the index column concealed. Instead, here's an image of what happens instead (displayi...
I am trying to move a file from repo local directory to volumes, but I am getting directory not found issue. Can some one guide me.tried using dbfs (dbfs/:Volumes/folder/ , /dbfs/Volumes/folder/) and without dbfs (/Volumes/folder/). None worked.@Reti...
I have become really frustrated because I can't copy and paste cells in Databricks notebook. The keyboard shortcuts of Command + C and Command + V doesn't seem to work. I couldn't find a way to change the keyboard shortcut either.As a data scientist ...
Hi all,I hope you could help me to figure out what I am missing.I'm trying to do a simple thing. To read the data from the data ingestion zone (csv files saved to Azure Storage Account) using the Delta Live Tables pipeline and share the resulting tab...
I'm curious if Databricks plans to address this. We use delta live streaming tables extensively and also planned on using delta sharing to get our data from our production unity catalog (different region). Duplicating the data as a workaround is no...
How to create Delta live tables in Silver layerHi DB Experts,Having basic questions :I am working on Madalian Architecture (B, S, G) Layers.on B i am getting Delta files (Parq) format. with log folders. One folder for one table, multiple files are ge...
Dear Kaniz,Thank you for addressing question :I am getting following error if i follow above: pyspark.errors.exceptions.captured.IllegalArgumentException: Reading from a Delta table is not supported with this syntax. If you would like to consume data...
Thanks to every one of you, the Databricks Community has reached an incredible milestone: 100,000 members and over 50,000 posts! Your dedication, expertise and passion have made this possible. Whether you're a seasoned data professional, a coding en...
I am trying to follow along with a training course, but I am consistently running into an error loading a CSV with Spark from DBFS. Specifically, I keep getting an "Invalid format detected error". Has anyone else encountered this and found a soluti...
Well your error message is telling you that Spark is encountering a Delta table conflict while trying to read a CSV file. The file path dbfs:/mnt/dbacademy... points to a CSV file. This is where the fun begins. Spark detects a Delta transaction log d...
Hi,This is my sample JSON data which is generated from api response and it is all coming in a single row. I want to split this in multiple rows and store it in a dataframe.[{"transaction_id":"F6001EC5-528196D1","corrects_transaction_id":null,"transac...
Yes indeed, it was datatype issue. After changing it to Longtype in the schema definition, it is working now. Thanks once again for all your inputs and time. Much appreciated !!!
