- 2703 Views
- 0 replies
- 1 kudos
VS code 2023
Do I need to save the data locally and run the plotting locally as well or does anyone have a smart solution to this
- 2703 Views
- 0 replies
- 1 kudos
Forum Posts
- 1681 Views
- 0 replies
- 0 kudos
Change the Admin Owner Email Account for Databricks cloud standard account?
As stated. My company changed name and the email address has migrated. I need to change it to the new one. And there is no way to open a support ticket to address that from what I saw (Standard Plan)Please do not tell me to contact AWS as they have ...
- 1681 Views
- 0 replies
- 0 kudos
- 1114 Views
- 0 replies
- 0 kudos
Plotting using Databricks in VS code
Hi,I am quite new to working with Databricks in VS code. I am trying to figure out the best way to plot my data, when running on a cluster. I would like to have the possibility to zoom and move the plot as I have when plotting locally with Matplotlib...
- 1114 Views
- 0 replies
- 0 kudos
- 12024 Views
- 6 replies
- 3 kudos
Resolved! Databricks voucher code error
Hi Team, I am getting error that voucher code is invalid error when trying to register for "Databricks Certified Associate Data Engineer Associate. I got this issue once page was reloaded due to slowness of the internet before checkout. and the vouch...
- 12024 Views
- 6 replies
- 3 kudos
Latest Reply
- 6019 Views
- 2 replies
- 1 kudos
Change default catalog
It seems that when I am connecting to Databricks Warehouse, it is using the default catalog which is hive_metastore. Is there a way to define unity catalog to be the default?I know I can run the queryUSE CATALOG MAINAnd then the current session will ...
- 6019 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
Thanks Brian2. Is there an equivalent config parameter for a SQL Warehouse?
- 1 kudos
- 1211 Views
- 0 replies
- 0 kudos
ronaldo is back
create table SalesReport(TerritoryName NVARCHAR(50), ProductName NVARCHAR(100), TotalSales DECIMAL(10,2), PreviousYearSales DECIMAL(10,2), GrowthRate DECIMAL(10,2)); create table ErrorLog( ErrorID int, ErrorMessage nvarchar(max),ErrorDate datetime);...
- 1211 Views
- 0 replies
- 0 kudos
- 1803 Views
- 0 replies
- 0 kudos
Save dataframe to the same variable
I would like to know if there is any difference if I save dataframe during tranformation to itself as first code or to new dataframe as second example.Thankslog_df = log_df.withColumn("process_timestamp",from_utc_timestamp(lit(current_timestamp()),"E...
- 1803 Views
- 0 replies
- 0 kudos
- 2541 Views
- 0 replies
- 0 kudos
iceberg
Hi fellasi am working on databricks using icebergat first i have configured my notebook as belowspark.conf.set("spark.sql.catalog.spark_catalog","org.apache.iceberg.spark.SparkCatalog")spark.conf.set("spark.sql.catalog.spark_catalog.type", "hadoop")s...
- 2541 Views
- 0 replies
- 0 kudos
- 2484 Views
- 1 replies
- 1 kudos
Resolved! threads leakage when getConnection fails
Hi,we are using databricks jdbc https://mvnrepository.com/artifact/com.databricks/databricks-jdbc/2.6.33it seems like there is a thread leakage when getConnection failscould anyone advice?can be reproduced with @Test void databricksThreads() {...
- 2484 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi,none of the above suggestion will not work...we already contacted databricks jdbc team, thread leakage was confirmed and was fixed in version 2.6.34https://mvnrepository.com/artifact/com.databricks/databricks-jdbc/2.6.34this leakage still exist if...
- 1 kudos
- 1446 Views
- 0 replies
- 0 kudos
Missing records while using limit in multithreading
Hi,I need to process nearly 30 files from different locations and insert records to RDS. I am using multi-threading to process these files parallelly like below. Test data: I have configuration like below based on column 4: If column 4=0:...
- 1446 Views
- 0 replies
- 0 kudos
- 3615 Views
- 1 replies
- 0 kudos
--files in spark submit task
Regarding --files option in spark submit task of Databricks jobs, would like to understand how it works and what is the syntax to pass multiple files to --files? I tried using --files and --py-files and my understanding is, it should make available t...
- 3615 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi, could you please check if this helps: https://docs.databricks.com/en/files/index.html Also please tag @Debayan​ with your next response which will notify me, Thank you!
- 0 kudos
- 1904 Views
- 1 replies
- 0 kudos
Hive Migration best practices
Hi ,Could you please share with us the approach and best practices for migrating from hadoop-hive to Databricks?Regards,Phanindra
- 1904 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi, You can try checking the below resources: https://www.databricks.com/resources/ebook/migration-guide-hadoop-to-databricks https://www.databricks.com/solutions/migration/hadoop https://www.databricks.com/blog/2021/08/06/5-key-steps-to-successfull...
- 0 kudos
- 2239 Views
- 1 replies
- 0 kudos
Sqoop Migration best practices
Hi ,Could you please share with us the approach and best practices for migrating from hadoop-SQOOP to Databricks?
- 2239 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi, You can try checking the below resources: https://www.databricks.com/resources/ebook/migration-guide-hadoop-to-databricks https://www.databricks.com/solutions/migration/hadoop https://www.databricks.com/blog/2021/08/06/5-key-steps-to-successfull...
- 0 kudos
- 1547 Views
- 1 replies
- 0 kudos
OOZE Jobs migration to databricks
Hi ,Could you please share with us the approach and best practices for migrating from hadoop-ooze jobs to Databricks?
- 1547 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi, You can try checking the below resources on Hadoop migration: https://www.databricks.com/resources/ebook/migration-guide-hadoop-to-databricks https://www.databricks.com/solutions/migration/hadoop https://www.databricks.com/blog/2021/08/06/5-key-...
- 0 kudos
- 2990 Views
- 1 replies
- 0 kudos
HDFS to Databricks
Hi ,Could you please share with us the approach and best practices for migrating from hadoop-HDFS to Databricks?
- 2990 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi, You can try checking the below resources: https://www.databricks.com/resources/ebook/migration-guide-hadoop-to-databricks https://www.databricks.com/solutions/migration/hadoop https://www.databricks.com/blog/2021/08/06/5-key-steps-to-successfull...
- 0 kudos
