Databricks Community

Resolved! Parsing deeply nested XML in Databricks

Hi Guys,Can someone point me to libraries to parse XML files in Databricks using Python / Scala.Any link to blog / documentations will be helpful.Looked into https://docs.databricks.com/data/data-sources/xml.html.Want to parse XSD's, seem this is exp...

Resolved! I am looking for best practices in implementing Ranger type of Access control in Databricks ?

Need this to do auditing and numbers of users accessing databases and tables created in databricks

Resolved! How to convert Data Chr Strings to Date Strings

Databricks CommunityNew to Databricks and work in R code. I have a data from with a date field that is a chr string and need to convert to a date field. Tried the standard as.Date(x, format = "%Y-%m-%d") , then tried the dplyr::mutate function and th...

Resolved! Cannot connect to Databricks SQL Endpoint using PHP and ODBC

I am trying to connect to our Databricks SQL endpoint using PHP in a Docker container.I setup my Docker container to download and configure the ODBC driver as specified here: https://docs.databricks.com/integrations/bi/jdbc-odbc-bi.html#install-and-c...

Resolved! Azure Databricks container runtime broken in 9.1 LTS, how to fix?

For stability, I've stuck with LTS. Last Friday my containers stopped working with error message:Py4JException: An exception was raised by the Python Proxy. Return Message: Traceback (most recent call last): File "/databricks/spark/python/lib/py4j-...

Resolved! JSON validation is getting failed after writing Pyspark dataframe to json format

Hi We have to convert transformed dataframe to json format. So we used write and json format on top of final dataframe to convert it to json. But when we validating the output json its not in proper json format.Could you please provide your suggestio...

Resolved! What does the run_as_repl parameter do in the databricks Jobs API - https://docs.databricks.com/dev-tools/api/latest/examples.html#jobs-api-examples

Resolved! Where / how does DBFS store files?

I tried to use %fs head to print the contents of a CSV file used in a training%fs head "/mnt/path/file.csv"but got an error saying cannot head a directory!?Then I did %fs ls on the same CSV file and got a list of 4 files under a directory named as a ...

Resolved! Unable to start SQL End point in DATABRICKS SQL

Hello All, I am trying to use Databricks SQL but somehow the SQL end point is not getting started. It is in starting state for long time and then session is getting expired. Please note , the default SQL End point also not getting started. I am using...

Resolved! Hi Experts I want to know the difference between connecting any BI Tool to Spark SQL and Databricks SQL end point?

Its all about spinning the spark cluster and both spark Sql api and databricks does the same operation what difference does it make to BI tools ?