- 5925 Views
- 1 replies
- 1 kudos
How Connect Powerbi with sql endpoint with no public access only private?
we are making configuration adjustments in the workspaces to disable public access. My concern is whether this will impact the Power BI dashboards that currently consume data from our workspace's SQL Data Warehouse.
- 5925 Views
- 1 replies
- 1 kudos
- 1 kudos
If you're using Azure Databricks, you can create a vnet data gateway on Azure.https://learn.microsoft.com/en-us/data-integration/vnet/create-data-gatewayshttps://learn.microsoft.com/en-us/data-integration/vnet/high-availability-load-balancingYou need...
- 1 kudos
- 1101 Views
- 0 replies
- 0 kudos
Unable to reset my Community edition password
I created a new Databricks community account a few days ago. Unfortunately, I seem to have forgotten the password associated with it. Despite trying the "forgot password" option multiple times on the login page, I haven't received any emails to reset...
- 1101 Views
- 0 replies
- 0 kudos
- 3997 Views
- 2 replies
- 0 kudos
Limit the scope of workspace level access token to access only specific REST APIs of Databricks
Hi Community, Is there a way to limit the scope of workspace level token to hit only certain REST APIs of Databricks.In short, Once we generate a workspace level token following this doc. Link: https://docs.databricks.com/en/dev-tools/auth/oauth-m2m....
- 3997 Views
- 2 replies
- 0 kudos
- 2072 Views
- 0 replies
- 0 kudos
Unity catalog unable to identify Managed Identity
Hello all,We have encountered a problem while running a notebook on a unity catalog cluster by using Azure data factory managed identity. When we add the managed identity directly to the unity catalog table it’s working but if we add the managed iden...
- 2072 Views
- 0 replies
- 0 kudos
- 1648 Views
- 1 replies
- 0 kudos
Function/system id for tableau databricks connector
currently, we are using PAT token for authentication purposes to generate tableau reports form data in databricks delta tables using databricks tableau connector. Would somebody know if system/function ID can be used from Tableau to Databricks inste...
- 1648 Views
- 1 replies
- 0 kudos
- 0 kudos
@-werners- @Retired_mod can you help here plz?
- 0 kudos
- 3647 Views
- 3 replies
- 1 kudos
Resolved! Can data be unified based on client profile (unified profile) in databricks?
Hi All,my question is in regard to how data in salesforce data cloud gets unified based on client profiles. Can similar action be done on data in databricks. i believe unity catalog just provides unified layer for security and governance. is there a ...
- 3647 Views
- 3 replies
- 1 kudos
- 1 kudos
You want to identify actual persons based on one or more profiles (based on e-mail address etc). That is something that is not available out-of-the box in Databricks. The 'unified' in Databricks means you have a single platform for several data top...
- 1 kudos
- 2693 Views
- 0 replies
- 2 kudos
pandas .style is ugly in Databricks
Why does something like df.style.hide_index() turn out so ugly in Databricks? That command should show the dataframe pretty like always, but simply with the index column concealed. Instead, here's an image of what happens instead (displayi...
- 2693 Views
- 0 replies
- 2 kudos
- 2588 Views
- 0 replies
- 1 kudos
No able to move a file to volume
I am trying to move a file from repo local directory to volumes, but I am getting directory not found issue. Can some one guide me.tried using dbfs (dbfs/:Volumes/folder/ , /dbfs/Volumes/folder/) and without dbfs (/Volumes/folder/). None worked.@Reti...
- 2588 Views
- 0 replies
- 1 kudos
- 3670 Views
- 0 replies
- 0 kudos
Can't copy notebook cells using keyboard shortcut
I have become really frustrated because I can't copy and paste cells in Databricks notebook. The keyboard shortcuts of Command + C and Command + V doesn't seem to work. I couldn't find a way to change the keyboard shortcut either.As a data scientist ...
- 3670 Views
- 0 replies
- 0 kudos
- 6168 Views
- 5 replies
- 5 kudos
Problem sharing a streaming table created in Delta Live Table via Delta Sharing
Hi all,I hope you could help me to figure out what I am missing.I'm trying to do a simple thing. To read the data from the data ingestion zone (csv files saved to Azure Storage Account) using the Delta Live Tables pipeline and share the resulting tab...
- 6168 Views
- 5 replies
- 5 kudos
- 5 kudos
I'm curious if Databricks plans to address this. We use delta live streaming tables extensively and also planned on using delta sharing to get our data from our production unity catalog (different region). Duplicating the data as a workaround is no...
- 5 kudos
- 4803 Views
- 1 replies
- 0 kudos
How to create Delta live tables in Silver layer
How to create Delta live tables in Silver layerHi DB Experts,Having basic questions :I am working on Madalian Architecture (B, S, G) Layers.on B i am getting Delta files (Parq) format. with log folders. One folder for one table, multiple files are ge...
- 4803 Views
- 1 replies
- 0 kudos
- 0 kudos
Dear Kaniz,Thank you for addressing question :I am getting following error if i follow above: pyspark.errors.exceptions.captured.IllegalArgumentException: Reading from a Delta table is not supported with this syntax. If you would like to consume data...
- 0 kudos
- 10740 Views
- 7 replies
- 24 kudos
Big news: Our Community is now 100,000 members strong with over 50,000 posts🚀
Thanks to every one of you, the Databricks Community has reached an incredible milestone: 100,000 members and over 50,000 posts! Your dedication, expertise and passion have made this possible. Whether you're a seasoned data professional, a coding en...
- 10740 Views
- 7 replies
- 24 kudos
- 7117 Views
- 1 replies
- 0 kudos
Error Spark reading CSV from DBFS MNT: incompatible format detected
I am trying to follow along with a training course, but I am consistently running into an error loading a CSV with Spark from DBFS. Specifically, I keep getting an "Invalid format detected error". Has anyone else encountered this and found a soluti...
- 7117 Views
- 1 replies
- 0 kudos
- 0 kudos
Well your error message is telling you that Spark is encountering a Delta table conflict while trying to read a CSV file. The file path dbfs:/mnt/dbacademy... points to a CSV file. This is where the fun begins. Spark detects a Delta transaction log d...
- 0 kudos
- 15383 Views
- 13 replies
- 1 kudos
Resolved! Want to split JSON data into multiple rows
Hi,This is my sample JSON data which is generated from api response and it is all coming in a single row. I want to split this in multiple rows and store it in a dataframe.[{"transaction_id":"F6001EC5-528196D1","corrects_transaction_id":null,"transac...
- 15383 Views
- 13 replies
- 1 kudos
- 1 kudos
Yes indeed, it was datatype issue. After changing it to Longtype in the schema definition, it is working now. Thanks once again for all your inputs and time. Much appreciated !!!
- 1 kudos
- 2435 Views
- 2 replies
- 0 kudos
Bug with show tblproperties directly returning redacted in the result set where "userid" in value
When I use show tblproperties on a view/table to see the metadata, it will redact any value which has "userid" anywhere put in to it.And it is not just through the visual interface, when I query it through python directly, it contains the redacted va...
- 2435 Views
- 2 replies
- 0 kudos
- 0 kudos
I understand that yours is a View. For my case, it's a Table so I could use `desc detail <schema_name>.<table_name>` to get the table properties info that are not redacted in the `properties` column from the `desc detail` output.
- 0 kudos
