Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
As stated. My company changed name and the email address has migrated. I need to change it to the new one. And there is no way to open a support ticket to address that from what I saw (Standard Plan)Please do not tell me to contact AWS as they have ...
Hi,I am quite new to working with Databricks in VS code. I am trying to figure out the best way to plot my data, when running on a cluster. I would like to have the possibility to zoom and move the plot as I have when plotting locally with Matplotlib...
Hi Team, I am getting error that voucher code is invalid error when trying to register for "Databricks Certified Associate Data Engineer Associate. I got this issue once page was reloaded due to slowness of the internet before checkout. and the vouch...
It seems that when I am connecting to Databricks Warehouse, it is using the default catalog which is hive_metastore. Is there a way to define unity catalog to be the default?I know I can run the queryUSE CATALOG MAINAnd then the current session will ...
I would like to know if there is any difference if I save dataframe during tranformation to itself as first code or to new dataframe as second example.Thankslog_df = log_df.withColumn("process_timestamp",from_utc_timestamp(lit(current_timestamp()),"E...
Hi fellasi am working on databricks using icebergat first i have configured my notebook as belowspark.conf.set("spark.sql.catalog.spark_catalog","org.apache.iceberg.spark.SparkCatalog")spark.conf.set("spark.sql.catalog.spark_catalog.type", "hadoop")s...
Hi,we are using databricks jdbc https://mvnrepository.com/artifact/com.databricks/databricks-jdbc/2.6.33it seems like there is a thread leakage when getConnection failscould anyone advice?can be reproduced with @Test
void databricksThreads() {...
Hi,none of the above suggestion will not work...we already contacted databricks jdbc team, thread leakage was confirmed and was fixed in version 2.6.34https://mvnrepository.com/artifact/com.databricks/databricks-jdbc/2.6.34this leakage still exist if...
Hello - I have a foreign catalog which I can access fine in SQL. However, I can't access it from from python notebook.i.e. this works just fine if I have notebook using a Pro SQL Warehouse%sqlUSE CATALOG <my_foreign_catalog_name>;USE SCHEMA public;S...
Hi,I need to process nearly 30 files from different locations and insert records to RDS. I am using multi-threading to process these files parallelly like below. Test data: I have configuration like below based on column 4: If column 4=0:...
Regarding --files option in spark submit task of Databricks jobs, would like to understand how it works and what is the syntax to pass multiple files to --files? I tried using --files and --py-files and my understanding is, it should make available t...
Hi, could you please check if this helps: https://docs.databricks.com/en/files/index.html
Also please tag @Debayan​ with your next response which will notify me, Thank you!
Hi, You can try checking the below resources:
https://www.databricks.com/resources/ebook/migration-guide-hadoop-to-databricks
https://www.databricks.com/solutions/migration/hadoop
https://www.databricks.com/blog/2021/08/06/5-key-steps-to-successfull...
Hi, You can try checking the below resources:
https://www.databricks.com/resources/ebook/migration-guide-hadoop-to-databricks
https://www.databricks.com/solutions/migration/hadoop
https://www.databricks.com/blog/2021/08/06/5-key-steps-to-successfull...
Hi, You can try checking the below resources on Hadoop migration:
https://www.databricks.com/resources/ebook/migration-guide-hadoop-to-databricks
https://www.databricks.com/solutions/migration/hadoop
https://www.databricks.com/blog/2021/08/06/5-key-...
Hi, You can try checking the below resources:
https://www.databricks.com/resources/ebook/migration-guide-hadoop-to-databricks
https://www.databricks.com/solutions/migration/hadoop
https://www.databricks.com/blog/2021/08/06/5-key-steps-to-successfull...