- 4943 Views
- 2 replies
- 2 kudos
Metric Tables not created automatically
I was trying the new lakehouse monitoring feature for one of my unity tables, and when I create a monitoring dashboard for my table, the 2 metric tables({output_schema}.{table_name}_profile_metrics and {output_schema}.{table_name}_drift_metrics) are ...
- 4943 Views
- 2 replies
- 2 kudos
- 2514 Views
- 2 replies
- 1 kudos
DataEngineer Associate Exam got Suspended. Raised Ticket. No Response
Hi Team,This is the second time I am tryng to post a discussion in the community. The first time it was marked SPAM. It has been a terrible experience so far with my Databricks Data Engineer Associate exam getting suspended by the proctor for reasons...
- 2514 Views
- 2 replies
- 1 kudos
- 1 kudos
@RohitB91 the support team is awaiting your response for a date / time you are available to resume your exam. Your registration confirmation email should contain this detail: Preparation for taking your Online proctored Exam Understand the computer r...
- 1 kudos
- 1385 Views
- 1 replies
- 0 kudos
search engine functionality in Databricks
Hi Team,Good morning, I am Sai Rekhawar, currently residing in Hamburg, Germany. I am reaching out regarding an issue with the search engine functionality in Databricks.I have created several catalogs and tables within the catalog, and I am the owner...
- 1385 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @SaiRekhawar , try using the filters in under the table bar in ctrl+P as below
- 0 kudos
- 1187 Views
- 0 replies
- 0 kudos
Improve Query Performance
HelloI have a query in one of my notebooks that took around 3.5 hours on D12_V2 cluster and workers between 5 to 25 .is there a way to write the query in diffrenet way in order to improve performance and cost : select /*+ BROADCAST(b) */ MD5(CONCAT(N...
- 1187 Views
- 0 replies
- 0 kudos
- 4121 Views
- 2 replies
- 1 kudos
User Agent Verfication
Hello Folks, We are recently working on Databricks integration in to our products and one of the Best practices suggested is to send user-agent information for any REST API or JDBC connect we make from the product to Databricks. We have made all the ...
- 4121 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @bdraitcak , Here are the steps to be followed to get the logs: Log in to your Azure portal.Go to / Search for "Log analytics workspace."Create a new Log Analytics workspace by specifying your Resource group and Instance details. [subscription + r...
- 1 kudos
- 2021 Views
- 1 replies
- 0 kudos
bitmap_count() function's output is different in databricks compared to snowflake
I have found that the results of the bitmap_count() function output differs significantly between databricks and snowflake.eg: snowflake returns a value of '1' for this code. "select bitmap_count(X'0001056c000000000000') " while Databricks returns a...
- 2021 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @vigneshp , Good Day! In Databricks, bitmap_count function returns the number of bits set in a BINARY string representing a bitmap. This function is typically used to count distinct values in combination with the bitmap_bucket_number() and the bi...
- 0 kudos
- 4029 Views
- 0 replies
- 0 kudos
Convert pkl file
I am looking to convert pkl file to parquet format. I am reading this data from Azure Data Lake and convert and create parquet file outputs to store them in my hive metastore.
- 4029 Views
- 0 replies
- 0 kudos
- 1814 Views
- 0 replies
- 1 kudos
Init script error 13.3 and 14.3 LTS issues mesa
Hi - we had a few issues with some of our init scripts recently. Investigating I found that mesa packages were throwing issues when trying to install. Posting this to help the community and raise awareness to Databricks to fix itI believe the image f...
- 1814 Views
- 0 replies
- 1 kudos
- 14379 Views
- 5 replies
- 0 kudos
Clusters are failing to launch
Hi Guys,I am a complete newbie to data bricks, we are trying to figure out if our data models and ETL can run on it.I have got the failure to launch message. I have read this message as well.https://community.databricks.com/t5/data-engineering/cluste...
- 14379 Views
- 5 replies
- 0 kudos
- 2150 Views
- 1 replies
- 0 kudos
Unable to login to Azure Databricks
I am trying to login to Azure Databricks as usual but it waits for 5 10 seconds then redirects me to https://westeurope-c2.azuredatabricks.net/aad/redirect The system is up and all of my colleagues can login to the system. I have tried in incognito m...
- 2150 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @egndz , If it was a one-time issue we suspect it was an issue within Azure or Databricks services. Please follow the status page to keep track of the maintenance/outages. https://status.azuredatabricks.net/
- 0 kudos
- 2147 Views
- 1 replies
- 0 kudos
Failed Synchronization of Files Using Databricks Extension in VS Code
hi,I am trying to set up Databricks Extension in VS Code. I follow the steps as per the guide belowhttps://docs.databricks.com/en/dev-tools/vscode-ext/tutorial.html When i move to step 6 (see in the above guide) i follow the steps and i create succes...
- 2147 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Anto23 , Greetings from Databricks. Based on the above information, it seems Files in Workspace is currently disabled for your Databricks environment. This feature allows storing and accessing non-notebook files alongside your notebooks and cod...
- 0 kudos
- 1784 Views
- 0 replies
- 0 kudos
Huge data migration from HDFS to Databricks
Hi Team,Could you please help me what is the best way/best practices to copy around 3 TB of data(parquet) from HDFS to Databricks delta format and create external tables on top of it?Regards,Phanindra
- 1784 Views
- 0 replies
- 0 kudos
- 2660 Views
- 1 replies
- 0 kudos
Oracle table load from Databricks
I am trying to load a dataframe from Databricks to target Oracle table using the write method and using JDBC api. I have the right drivers. The job and its corresponding stages are getting completed and the data is getting loaded in Oracle target tab...
- 2660 Views
- 1 replies
- 0 kudos
- 0 kudos
Thanks for the response. Can you please elaborate on the Apache Spark JDBC Connector. I am using ojdbc8 driver as per the Databricks documentation. I am not using Delta Lake. I have the data in a dataframe and using write method to insert the data to...
- 0 kudos
- 2675 Views
- 0 replies
- 0 kudos
Decimal Precision error
when i try to create an dataframe like this lstOfRange = list() lstOfRange = [ ['CREDIT_LIMIT_RANGE',Decimal(10000000.010000),Decimal(100000000000000000000000.000000),'>10,000,000','G'] ] RangeSchema = StructType([StructField("rangeType",St...
- 2675 Views
- 0 replies
- 0 kudos
- 3407 Views
- 1 replies
- 0 kudos
error installing igraph library
Hi, I am trying to install the "igraph" and "networkD3" CRAN packages for use within a notebook, but am receiving the below error.Could someone please assist?Thanks! * installing *source* package ‘igraph’ ... ** package ‘igraph’ successfully unpacked...
- 3407 Views
- 1 replies
- 0 kudos
- 0 kudos
Based on this igraph github issue https://github.com/igraph/rigraph/issues/490#issuecomment-966890059, I followed the instructions to install glpk. After installing glpk, I was able to install igraph.
- 0 kudos
