- 9154 Views
- 4 replies
- 5 kudos
Resolved! Why does spark.read.csv come back with an error: com.databricks.sql.io.FileReadException: Error while reading file dbfs:/mnt/cntnr/demo/circuits.csv ?
I need help understanding why I can't open a file.In a databricks notebook, I use this code:%fs ls /mnt/cntnr/demoI get back dbfs:/mnt/cntnr/demo/circuits.csv as one of the path values.When I use this code, I get an error:circuits_df = spark.read....
- 9154 Views
- 4 replies
- 5 kudos
- 5 kudos
It turns out my spark config was wrong #Set Spark configuration configs = {"fs.azure.account.auth.type": "OAuth", "fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider", "fs.azu...
- 5 kudos
- 12228 Views
- 4 replies
- 0 kudos
Resolved! Cannot configure VS code databricks extension with a non-standard databricks URL: not a databricks host.
Hello,I'm trying to connect to our databricks instance using the vscode extension. However, when following this guide we cannot get the configuration to proceed past the point that it asks for our instance URL. The prompt appears to expect a URL of t...
- 12228 Views
- 4 replies
- 0 kudos
- 0 kudos
Hello,Yes, the databricks team shared a modified version of the vs code plugin which did not include the URL matching logic. It connects successfully. However, our custom URL is as it is because our organisation is hosting its own instance of Databri...
- 0 kudos
- 14409 Views
- 4 replies
- 4 kudos
Unable to use dbutils in Premium
I am unable to use dbutils commands and mkdir, etc also does not work after upgrading my Databricks Workspace from Standard tier to Premium tier.It throws the following error:py4j.security.Py4JSecurityException: Constructor public com.databricks.back...
- 14409 Views
- 4 replies
- 4 kudos
- 4 kudos
Hi @Abhishek Jain​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...
- 4 kudos
- 3115 Views
- 2 replies
- 0 kudos
Running multiple Databricks queries with golang client
I am using github.com/databricks/databricks-sql-go to connect and run queries on my Databricks warehouse. However when I try to run multiple queries:insert into....; insert into....;It fails and complains on syntax errors.[PARSE_SYNTAX_ERROR] Syntax ...
- 3115 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Gil Omer​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your...
- 0 kudos
- 2239 Views
- 2 replies
- 1 kudos
pandas issue
We identify a potential bug in either DBFS or Pandas that when writting a dataframe using Pandas `to_csv`, `to_parquet`, `to_pickle` etc to a mounted ADLS location with read-only service principle didn't throw permission deny exceptions. However, met...
- 2239 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Yung-Hang Chang​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...
- 1 kudos
- 2397 Views
- 1 replies
- 1 kudos
Concurrent Insert on a delta table fails if the table contains Identity Columns. Error message are added below.MetadataChangedException: The metadata ...
Concurrent Insert on a delta table fails if the table contains Identity Columns. Error message are added below.MetadataChangedException: The metadata of the Delta table has been changed by a concurrent update. Please try the operation again.
- 2397 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @SK ASIF ALI​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 1 kudos
- 1734 Views
- 1 replies
- 0 kudos
We have concern with respect to the feasibility of installing the Databricks runtime supported version on-premise. This is because we are planning to...
We have concern with respect to the feasibility of installing the Databricks runtime supported version on-premise. This is because we are planning to migrate the existing Azure Databricks application code to an on-premise environment based on the cl...
- 1734 Views
- 1 replies
- 0 kudos
- 0 kudos
Any update?
- 0 kudos
- 3827 Views
- 2 replies
- 2 kudos
DLT config/setting json support
hi team,There used to be option to provide DLT pipeline settings either via UI or JSON, but I do not see it anymore after switching to new UI. Is this something expected ? am I missing something ? here is screenshot for reference.
- 3827 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Chhaya Vishwakarma​ This option is available, could you please clear the browser cache and try ? or can you try in an incognito window?
- 2 kudos
- 9430 Views
- 2 replies
- 1 kudos
Trying to mount Azure Data Lake Storage Gen 2 to Azure Databricks
I have validated many many many times all my credentials and I am still getting the following error (at the very end). ChatGPT said to basically recheck everything and I did. The one thing I didn't do was grant the permissions which I have since done...
- 9430 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Chris Sarrico​ Could you please specify the container name before the storage account name like this :so the source looks like this. :source = "abfss://<container-name>@<storage-account-name>.dfs.core.windows.net/"https://learn.microsoft.com/en-u...
- 1 kudos
- 2311 Views
- 1 replies
- 0 kudos
0 byte files appearing in blob storage
When writing out files to azure blob storage using spark there are some 0 byte files being created on the web portal but not when I view it using storage explorer.I was wondering if there is there any work around for this?
- 2311 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Ryan Low​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 2117 Views
- 1 replies
- 0 kudos
Spark read not working in parallel
Hi All,Am trying to run a spark jdbc query and the read is running on only 1 worker and is taking a lot of time. Usually, it takes 4 minutes in aws lambda but now it is taking approx 10 minutes while using databricks. There are only 4m records (write...
- 2117 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Saloni Bhatia​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 2517 Views
- 1 replies
- 0 kudos
Silent failure in DataFrameWriter when loading data to Redshift
Context:I'm using DataFrameWriter to load the dataSet into the Redshift. DataFrameWriter writes the dataSet to S3, and loads data from S3 to Redshift by issuing the Redshift copy command. Issue:In frequently we are observing, the data is present in t...
- 2517 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Kishorekumar Somasundaram​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 3475 Views
- 1 replies
- 0 kudos
Getting JsonParseException: Unexpected character ('<' (code 60))
I have a scheduled job that is executed using a notebook. Within one of the notebook cells, there is a check to determine if a table exists. However, even when the table does exist, it incorrectly identifies it as non-existent and proceeds to execut...
- 3475 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Mahesh Chahare​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 3134 Views
- 1 replies
- 1 kudos
Having issue trying to download a csv file from a website using FireFox Selenium.
Hi, When I clicked on the download button from a website thru Firefox selenium using element.click(), and the download destination is being set as Azure datalake storage. Then, after the download started, those .csv and .csv.part files never gotten m...
- 3134 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Brandon Lim​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 1 kudos
- 1603 Views
- 0 replies
- 2 kudos
Your guide to data and AI governance talks at the Data + AI Summit 2023
The highly anticipated Data+AI summit is just around the corner. Are you curious about redefining your data governance strategy in the era of LLMs and generative AI? Look no further! This year's Data+AI summit offers a dynamic lineup of keynotes, dem...
- 1603 Views
- 0 replies
- 2 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 561 | |
| 349 | |
| 287 |
