- 10301 Views
- 3 replies
- 1 kudos
How can I fix this error. ExecutionError: An error occurred while calling o392.mount: java.lang.NullPointerException
Hello, I'm trying to mount my Databricks to my Azure gen 2 data lake to read in data from the container, but I get an error when executing this line of code: dbutils.fs.mount( source = "abfss://resumes@choisysresume.dfs.core.windows.net/", mount_poin...
- 10301 Views
- 3 replies
- 1 kudos
- 1 kudos
I was using databricks scopes, to get the key which was used in the the config. I received a similar mount error while mounting with "wasbs" driver, "ExecutionError: An error occurred while calling o427.mount.", this was the issue because the scope ...
- 1 kudos
- 12382 Views
- 5 replies
- 7 kudos
Passing proxy configurations with databricks-sql-connector python?
Hi,I am trying to connect to databricks workspace which has IP Access restriction enabled using databricks-sql-connector. Only my Proxy server IPs are added in the allow list.from databricks import sql connection = sql.connect( server_hostname ='...
- 12382 Views
- 5 replies
- 7 kudos
- 7 kudos
Is there any resolution for the above setting up proxy with databricks sql connector
- 7 kudos
- 15390 Views
- 5 replies
- 0 kudos
Help With OSError: [Errno 95] Operation not supported: '/Workspace/Repos/Connectors....
Hello,I am experiencing issues with importing from utils repo the schema file I created.this is the logic we use for all ingestion and all other schemas live in this repo utills/schemasI am unable to access the file I created for a new ingestion pipe...
- 15390 Views
- 5 replies
- 0 kudos
- 0 kudos
@ArbMake sure the below is a file with extension .py and not a notebook. That should resolve the issue./Workspace/Repos/Connectors/Dev/utils/schemas/Comptroller.py'
- 0 kudos
- 6156 Views
- 2 replies
- 2 kudos
How to read a PDF file from Azure Datalake blob storage to Databricks
I have a scenario where I need to read a pdf file from "Azure Datalake blob storage to Databricks", where connection is done through AD access.Generating the SAS token has been restricted in our environment due to security issues. The below script ca...
- 6156 Views
- 2 replies
- 2 kudos
- 2 kudos
@PunithRaj You can try to use PDF DataSource for Apache Spark for read pdf files directly to the DataFrame. So you will have extracted text and rendered page as image in output. More details here: https://stabrise.com/spark-pdf/df = spark.read.forma...
- 2 kudos
- 14501 Views
- 5 replies
- 4 kudos
Azure Databricks: Error Creating Cluster
We have provisioned a new workspace in Azure using our own VNet. Upon creating the first cluster, I encounter this error:Control Plane Request Failure: Failed to get instance bootstrap steps from the Databricks Control Plane. Please check that instan...
- 14501 Views
- 5 replies
- 4 kudos
- 9684 Views
- 7 replies
- 6 kudos
How to connect Cognos 11.1.7 to Azure Databricks
We are trying to connect Cognos 11.1.7 to Azure Databricks, but no success.Can you please help or guide us how to connect Cognos 11.1.7 to Azure Databricks.This is very critical to our user community. Can you please help or guide us how to connect Co...
- 9684 Views
- 7 replies
- 6 kudos
- 6 kudos
Have anyone got the Simba JDBC driver going with CA 11.1.7? The ODBC driver works fine but i can't get the JDBC running.Regd's
- 6 kudos
- 5755 Views
- 3 replies
- 4 kudos
Resolved! Spot instance in Azure Databricks
When I run a job enabling using spot instances , I would like to know how many number of workers are using spot and how many number of workers are using on demand instances for a given job run In order to identify the spot instances we got for any...
- 5755 Views
- 3 replies
- 4 kudos
- 7401 Views
- 5 replies
- 5 kudos
Support of running multiple cells at a time in databricks notebook Hi all,Now databricks notebook supports parallel run of commands in a single notebo...
Support of running multiple cells at a time in databricks notebookHi all,Now databricks notebook supports parallel run of commands in a single notebook that will help run ad hoc queries simultaneously without creating a separate notebook.Once you run...
- 7401 Views
- 5 replies
- 5 kudos
- 5 kudos
Hi Team,I am observing that the functionality is not working as expected in the Trial workspace of Databricks. Is there a setting that needs to be enabled to allow independent SQL cells in a Databricks notebook to run in parallel, while dependent cel...
- 5 kudos
- 34080 Views
- 6 replies
- 10 kudos
Resolved! Table write command stuck "Filtering files for query."
Hello all,Background:I am having an issue today with databricks using pyspark-sql and writing a delta table. The dataframe is made by doing an inner join between two tables and that is the table which I am trying to write to a delta table. The table ...
- 34080 Views
- 6 replies
- 10 kudos
- 10 kudos
Even if I vacuum and optimize, it keeps getting stuck.cluster type is r6gd.xlarge min:4, max:6driver type is r6gd.2xlarge
- 10 kudos
- 23221 Views
- 4 replies
- 26 kudos
How to connect your Azure Data Lake Storage to Azure DatabricksStandard Workspace �� Private link In your storage accounts please go to “Networ...
How to connect your Azure Data Lake Storage to Azure DatabricksStandard Workspace Private linkIn your storage accounts please go to “Networking” -> “Private endpoint connections” and click Add Private Endpoint.It is important to add private links in ...
- 23221 Views
- 4 replies
- 26 kudos
- 9497 Views
- 3 replies
- 4 kudos
Converting SQL Code to SQL Databricks
I am new to Databricks. Please excuse my ignorance. My requirement is to convert the SQL query below into Databricks SQL. The query comes from EventLog table and the output of the query goes into EventSummaryThese queries can be found hereCREATE TABL...
- 9497 Views
- 3 replies
- 4 kudos
- 4 kudos
you may explore the tool and services from Travinto Technologies . They have very good tools. We had explored their tool for our code coversion from Informatica, Datastage and abi initio to DATABRICKS , pyspark. Also we used for SQL queries, stored ...
- 4 kudos
- 3971 Views
- 4 replies
- 5 kudos
Resolved! Databricks -->Workflows-->Job Runs
In Databricks -->Workflows-->Job Runs we have a column "Run As".From where does this value come. We are getting a user id here but need to change it to a generic account. Any help would be appreciated. Thanks
- 3971 Views
- 4 replies
- 5 kudos
- 5 kudos
I'm surprised why there no options to select "Run as" as something like "system user". Why all this complication with Service Principal? Where to report this ?@DataBricks
- 5 kudos
- 7635 Views
- 7 replies
- 4 kudos
need help with Azure Databricks questions on CTE and SQL syntax within notebooks
Hi amazing community folks,Feel free to share your experience or knowledge regarding below questions:-1.) Can we pass a CTE sql statement into spark jdbc? i tried to do it i couldn't but i can pass normal sql (Select * from ) and it works. i heard th...
- 7635 Views
- 7 replies
- 4 kudos
- 4 kudos
Not a comparison, but there is a DB-SQL cheatsheet at https://www.databricks.com/sites/default/files/2023-09/databricks-sql-cheatsheet.pdf/
- 4 kudos
- 35059 Views
- 10 replies
- 10 kudos
Failure starting repl. How to resolve this error? I got this error in a job which is running.
Failure starting repl. Try detaching and re-attaching the notebook.java.lang.Exception: Python repl did not start in 30 seconds. at com.databricks.backend.daemon.driver.IpykernelUtils$.startIpyKernel(JupyterDriverLocal.scala:1442) at com.databricks.b...
- 35059 Views
- 10 replies
- 10 kudos
- 10 kudos
I have had this problem many times, today I made a copy of the cluster and it got "de-saturated", it could help someone in the future
- 10 kudos
- 2515 Views
- 1 replies
- 5 kudos
Notebook cell output results limit increased- 10,000 rows or 2 MB. Hi all, Now, databricks start showing the first 10000 rows instead of 1000 rows.Tha...
Notebook cell output results limit increased- 10,000 rows or 2 MB.Hi all,Now, databricks start showing the first 10000 rows instead of 1000 rows.That will reduce the time of re-execution while working on fewer sizes of data that have rows between 100...
- 2515 Views
- 1 replies
- 5 kudos
- 5 kudos
Hi Ajay,Is there any way to increase this limit?Thanks, Fatima
- 5 kudos