Databricks Community

ramravi · 01-12-2023

I am trying to setup databricks-connect in my windows machine. While doing databricks-connect test I am getting the below error complaining java certificate is not found. ''Caused by: sun.security.validator.ValidatorException: PKIX path building fail...

ramravi · 01-03-2023

py4j.security.Py4JSecurityException: Method public org.apache.spark.sql.streaming.DataStreamReader org.apache.spark.sql.SQLContext.readStream() is not whitelisted on class class org.apache.spark.sql.SQLContextI already disable acl for cluster using "...

ramravi · 01-02-2023

spark is case sensitive?Spark is not case sensitive by default. If you have same column name in different case (Name, name), if you try to select either "Name" or "name" column you will get column ambiguity error.There is a way to handle this issue b...

ramravi · 12-22-2022

ramravi · 01-16-2023

Adding the certificate from the root level worked for me. This problem is solved.

ramravi · 01-12-2023

I receive a voucher today and it saysPlease find your code here (expires 6/1/23):Does it mean it expires on 1-June-2023 ?

ramravi · 01-02-2023

init scripts are great way to handle the dependent libraries installation in cluster.https://stackoverflow.com/questions/62516102/install-python-packages-using-init-scripts-in-a-databricks-cluster

ramravi · 01-02-2023

Databricks runs on linux servers. you can launch windows cmd or windows shell commands in databricks. you can run only linux commands.

ramravi · 01-02-2023

pandas_udf are optimized and faster for grouped operations, like applying a pandas_udf after a groupBy. The grouping allows pandas to perform vectorized operations and will be faster than normal udf. for normal case like a*b, a normal spark udf will ...

Databricks Community

User Stats

User Activity

Unable to connect to databricks cluster from Windows using databricks-connect

Issue while reading data from Kafka topic to Spark strutured streaming

spark is case sensitive? Spark is not case sensitive by default. If you have same column name in different case (Name, name), if you try to select eit...

How to limit number rows to display using display method in Spark databricks notebook ?

Re: Unable to connect to databricks cluster from Windows using databricks-connect

Re: Do not received Databricks Certification: Fully Sponsored after order on Reward Store

Re: Azure DataBricks New Job Cluster Libraries Installation Issues

Re: How to execute Windows commands (.cmd file) from Databricks?

Re: benefit of using vectorized pandas UDFs instead of the standard Pyspark UDFs?