- 1908 Views
- 1 replies
- 0 kudos
Enhance Documentation for databricks-connect for python
Hello, I'm setting up a python environment to work with databricks in vscode using Windows.I followed the instructions in Install Databricks Connect for Python, and found some issues/conflicts/errors:- There is a useless tip in the Set up the client ...
- 1908 Views
- 1 replies
- 0 kudos
- 0 kudos
Additionally, there are no comments on how to avoid uploading virtual environment to the catalog when using databricks extension for vscode
- 0 kudos
- 2472 Views
- 0 replies
- 0 kudos
Corrupted Python installation on Python restart on DBR 13.3
Hey there, we're using DBR 13.3 (no Docker) as general purpose cluster and init the cluster using the following init script:```#!/usr/bin/env bashexport DEBIAN_FRONTEND=noninteractiveset -euxo pipefailif [[ $DB_IS_DRIVER = "TRUE" ]]; thenecho "I am d...
- 2472 Views
- 0 replies
- 0 kudos
- 4870 Views
- 2 replies
- 2 kudos
Metric Tables not created automatically
I was trying the new lakehouse monitoring feature for one of my unity tables, and when I create a monitoring dashboard for my table, the 2 metric tables({output_schema}.{table_name}_profile_metrics and {output_schema}.{table_name}_drift_metrics) are ...
- 4870 Views
- 2 replies
- 2 kudos
- 2482 Views
- 2 replies
- 1 kudos
DataEngineer Associate Exam got Suspended. Raised Ticket. No Response
Hi Team,This is the second time I am tryng to post a discussion in the community. The first time it was marked SPAM. It has been a terrible experience so far with my Databricks Data Engineer Associate exam getting suspended by the proctor for reasons...
- 2482 Views
- 2 replies
- 1 kudos
- 1 kudos
@RohitB91 the support team is awaiting your response for a date / time you are available to resume your exam. Your registration confirmation email should contain this detail: Preparation for taking your Online proctored Exam Understand the computer r...
- 1 kudos
- 1358 Views
- 1 replies
- 0 kudos
search engine functionality in Databricks
Hi Team,Good morning, I am Sai Rekhawar, currently residing in Hamburg, Germany. I am reaching out regarding an issue with the search engine functionality in Databricks.I have created several catalogs and tables within the catalog, and I am the owner...
- 1358 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @SaiRekhawar , try using the filters in under the table bar in ctrl+P as below
- 0 kudos
- 1173 Views
- 0 replies
- 0 kudos
Improve Query Performance
HelloI have a query in one of my notebooks that took around 3.5 hours on D12_V2 cluster and workers between 5 to 25 .is there a way to write the query in diffrenet way in order to improve performance and cost : select /*+ BROADCAST(b) */ MD5(CONCAT(N...
- 1173 Views
- 0 replies
- 0 kudos
- 4070 Views
- 2 replies
- 1 kudos
User Agent Verfication
Hello Folks, We are recently working on Databricks integration in to our products and one of the Best practices suggested is to send user-agent information for any REST API or JDBC connect we make from the product to Databricks. We have made all the ...
- 4070 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @bdraitcak , Here are the steps to be followed to get the logs: Log in to your Azure portal.Go to / Search for "Log analytics workspace."Create a new Log Analytics workspace by specifying your Resource group and Instance details. [subscription + r...
- 1 kudos
- 1996 Views
- 1 replies
- 0 kudos
bitmap_count() function's output is different in databricks compared to snowflake
I have found that the results of the bitmap_count() function output differs significantly between databricks and snowflake.eg: snowflake returns a value of '1' for this code. "select bitmap_count(X'0001056c000000000000') " while Databricks returns a...
- 1996 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @vigneshp , Good Day! In Databricks, bitmap_count function returns the number of bits set in a BINARY string representing a bitmap. This function is typically used to count distinct values in combination with the bitmap_bucket_number() and the bi...
- 0 kudos
- 3974 Views
- 0 replies
- 0 kudos
Convert pkl file
I am looking to convert pkl file to parquet format. I am reading this data from Azure Data Lake and convert and create parquet file outputs to store them in my hive metastore.
- 3974 Views
- 0 replies
- 0 kudos
- 1791 Views
- 0 replies
- 1 kudos
Init script error 13.3 and 14.3 LTS issues mesa
Hi - we had a few issues with some of our init scripts recently. Investigating I found that mesa packages were throwing issues when trying to install. Posting this to help the community and raise awareness to Databricks to fix itI believe the image f...
- 1791 Views
- 0 replies
- 1 kudos
- 14213 Views
- 5 replies
- 0 kudos
Clusters are failing to launch
Hi Guys,I am a complete newbie to data bricks, we are trying to figure out if our data models and ETL can run on it.I have got the failure to launch message. I have read this message as well.https://community.databricks.com/t5/data-engineering/cluste...
- 14213 Views
- 5 replies
- 0 kudos
- 2093 Views
- 1 replies
- 0 kudos
Unable to login to Azure Databricks
I am trying to login to Azure Databricks as usual but it waits for 5 10 seconds then redirects me to https://westeurope-c2.azuredatabricks.net/aad/redirect The system is up and all of my colleagues can login to the system. I have tried in incognito m...
- 2093 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @egndz , If it was a one-time issue we suspect it was an issue within Azure or Databricks services. Please follow the status page to keep track of the maintenance/outages. https://status.azuredatabricks.net/
- 0 kudos
- 2106 Views
- 1 replies
- 0 kudos
Failed Synchronization of Files Using Databricks Extension in VS Code
hi,I am trying to set up Databricks Extension in VS Code. I follow the steps as per the guide belowhttps://docs.databricks.com/en/dev-tools/vscode-ext/tutorial.html When i move to step 6 (see in the above guide) i follow the steps and i create succes...
- 2106 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Anto23 , Greetings from Databricks. Based on the above information, it seems Files in Workspace is currently disabled for your Databricks environment. This feature allows storing and accessing non-notebook files alongside your notebooks and cod...
- 0 kudos
- 1755 Views
- 0 replies
- 0 kudos
Huge data migration from HDFS to Databricks
Hi Team,Could you please help me what is the best way/best practices to copy around 3 TB of data(parquet) from HDFS to Databricks delta format and create external tables on top of it?Regards,Phanindra
- 1755 Views
- 0 replies
- 0 kudos
- 2618 Views
- 1 replies
- 0 kudos
Oracle table load from Databricks
I am trying to load a dataframe from Databricks to target Oracle table using the write method and using JDBC api. I have the right drivers. The job and its corresponding stages are getting completed and the data is getting loaded in Oracle target tab...
- 2618 Views
- 1 replies
- 0 kudos
- 0 kudos
Thanks for the response. Can you please elaborate on the Apache Spark JDBC Connector. I am using ojdbc8 driver as per the Databricks documentation. I am not using Delta Lake. I have the data in a dataframe and using write method to insert the data to...
- 0 kudos
