cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Direo
Contributor
since ‎03-01-2022
‎09-20-2023

User Stats

  • 20 Posts
  • 2 Solutions
  • 3 Kudos given
  • 10 Kudos received

User Activity

Unexpected performance behaviors due to changes in the Spark engine or Databricks runtime

09-17-2023
Hi!We have recently upgraded our cluster from Databricks Runtime 10.4 LTS which includes Apache Spark 3.2.1 to to Databricks Runtime 13.3 LTSincludes Apache Spark 3.2.1 powered by Apache Spark 3.3.0 and noticed that one of our jobs runtime has dramat...

Feature store feature table location

05-01-2023
Can Databricks feature tables be stored outside of DBFS?

Operations applied when running fs.write_table to overwrite existing feature table in hive metastore

04-07-2023
Hi,there was a need to query an older snapshot of a table. Therefore ran:deltaTable = DeltaTable.forPath(spark, 'dbfs:/<path>') display(deltaTable.history())and noticed that every fs.write_table run triggers two operations:Write and CREATE OR REPLACE...

Importing CA certificate into a Databricks cluster

03-22-2023
Hi!I was following guide outlined here:https://kb.databricks.com/en_US/python/import-custom-ca-cert(also tried this: https://stackoverflow.com/questions/73043589/configuring-tls-ca-on-databricks)to add ca root certificate into Databricks cluster, but...

How does pyspark work in these two scenarios?

03-19-2023
I have two scenarios with different outcomes:Scenario 1:from pyspark.sql.functions import *# create sample dataframesdf1 = spark.createDataFrame([(1, 2, 3), (2, 3, 4)], ["a", "b", "c"])df2 = spark.createDataFrame([(1, 5, 6, 7), (2, 8, 9, 10)], ["a", ...
View more