- 3985 Views
- 2 replies
- 1 kudos
User Agent Verfication
Hello Folks, We are recently working on Databricks integration in to our products and one of the Best practices suggested is to send user-agent information for any REST API or JDBC connect we make from the product to Databricks. We have made all the ...
- 3985 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @bdraitcak , Here are the steps to be followed to get the logs: Log in to your Azure portal.Go to / Search for "Log analytics workspace."Create a new Log Analytics workspace by specifying your Resource group and Instance details. [subscription + r...
- 1 kudos
- 1945 Views
- 1 replies
- 0 kudos
bitmap_count() function's output is different in databricks compared to snowflake
I have found that the results of the bitmap_count() function output differs significantly between databricks and snowflake.eg: snowflake returns a value of '1' for this code. "select bitmap_count(X'0001056c000000000000') " while Databricks returns a...
- 1945 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @vigneshp , Good Day! In Databricks, bitmap_count function returns the number of bits set in a BINARY string representing a bitmap. This function is typically used to count distinct values in combination with the bitmap_bucket_number() and the bi...
- 0 kudos
- 3913 Views
- 0 replies
- 0 kudos
Convert pkl file
I am looking to convert pkl file to parquet format. I am reading this data from Azure Data Lake and convert and create parquet file outputs to store them in my hive metastore.
- 3913 Views
- 0 replies
- 0 kudos
- 1753 Views
- 0 replies
- 1 kudos
Init script error 13.3 and 14.3 LTS issues mesa
Hi - we had a few issues with some of our init scripts recently. Investigating I found that mesa packages were throwing issues when trying to install. Posting this to help the community and raise awareness to Databricks to fix itI believe the image f...
- 1753 Views
- 0 replies
- 1 kudos
- 14002 Views
- 5 replies
- 0 kudos
Clusters are failing to launch
Hi Guys,I am a complete newbie to data bricks, we are trying to figure out if our data models and ETL can run on it.I have got the failure to launch message. I have read this message as well.https://community.databricks.com/t5/data-engineering/cluste...
- 14002 Views
- 5 replies
- 0 kudos
- 2024 Views
- 1 replies
- 0 kudos
Unable to login to Azure Databricks
I am trying to login to Azure Databricks as usual but it waits for 5 10 seconds then redirects me to https://westeurope-c2.azuredatabricks.net/aad/redirect The system is up and all of my colleagues can login to the system. I have tried in incognito m...
- 2024 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @egndz , If it was a one-time issue we suspect it was an issue within Azure or Databricks services. Please follow the status page to keep track of the maintenance/outages. https://status.azuredatabricks.net/
- 0 kudos
- 2043 Views
- 1 replies
- 0 kudos
Failed Synchronization of Files Using Databricks Extension in VS Code
hi,I am trying to set up Databricks Extension in VS Code. I follow the steps as per the guide belowhttps://docs.databricks.com/en/dev-tools/vscode-ext/tutorial.html When i move to step 6 (see in the above guide) i follow the steps and i create succes...
- 2043 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Anto23 , Greetings from Databricks. Based on the above information, it seems Files in Workspace is currently disabled for your Databricks environment. This feature allows storing and accessing non-notebook files alongside your notebooks and cod...
- 0 kudos
- 1726 Views
- 0 replies
- 0 kudos
Huge data migration from HDFS to Databricks
Hi Team,Could you please help me what is the best way/best practices to copy around 3 TB of data(parquet) from HDFS to Databricks delta format and create external tables on top of it?Regards,Phanindra
- 1726 Views
- 0 replies
- 0 kudos
- 2556 Views
- 1 replies
- 0 kudos
Oracle table load from Databricks
I am trying to load a dataframe from Databricks to target Oracle table using the write method and using JDBC api. I have the right drivers. The job and its corresponding stages are getting completed and the data is getting loaded in Oracle target tab...
- 2556 Views
- 1 replies
- 0 kudos
- 0 kudos
Thanks for the response. Can you please elaborate on the Apache Spark JDBC Connector. I am using ojdbc8 driver as per the Databricks documentation. I am not using Delta Lake. I have the data in a dataframe and using write method to insert the data to...
- 0 kudos
- 2598 Views
- 0 replies
- 0 kudos
Decimal Precision error
when i try to create an dataframe like this lstOfRange = list() lstOfRange = [ ['CREDIT_LIMIT_RANGE',Decimal(10000000.010000),Decimal(100000000000000000000000.000000),'>10,000,000','G'] ] RangeSchema = StructType([StructField("rangeType",St...
- 2598 Views
- 0 replies
- 0 kudos
- 3282 Views
- 1 replies
- 0 kudos
error installing igraph library
Hi, I am trying to install the "igraph" and "networkD3" CRAN packages for use within a notebook, but am receiving the below error.Could someone please assist?Thanks! * installing *source* package ‘igraph’ ... ** package ‘igraph’ successfully unpacked...
- 3282 Views
- 1 replies
- 0 kudos
- 0 kudos
Based on this igraph github issue https://github.com/igraph/rigraph/issues/490#issuecomment-966890059, I followed the instructions to install glpk. After installing glpk, I was able to install igraph.
- 0 kudos
- 2251 Views
- 1 replies
- 0 kudos
how do committed-use discounts work?
How do committed-use discounts work for Databricks? Do I purchase a chunk of DBUs for a flat fee and then draw down on them until exhausted? Or am I purchasing a % discount to all DBUs I use until the time period ends?In either case, is this reflec...
- 2251 Views
- 1 replies
- 0 kudos
- 0 kudos
Thanks @Retired_mod that helps! What is the commitment term? one month? one year?
- 0 kudos
- 3248 Views
- 2 replies
- 1 kudos
Resolved! Error installing the igraph and networkD3 R libraries
Hi,I am trying to install the igraph and networkD3 CRAN packages for use within a notebook. However, I am receiving the attached installation error when attempting to do so.Could someone please assist?Thank you!
- 3248 Views
- 2 replies
- 1 kudos
- 23036 Views
- 2 replies
- 0 kudos
POC
Hi, I wonder if you could help me on the below please.We tried Databricks Data Intelligence platform for one of our clients and found that its very expensive when compared to AWS EMR. I understand its not apple-apple comparision as one being platform...
- 23036 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Retired_mod Thanks for getting back with so valuable information.SystemFile sizeDurationSystemDurationCommentsComments1EMR225 GB22 minsDatabricks63 minsEMR is cheaper than Databricks by 5 timesThis involves various S3 writes with m5d4xlargeEMR225...
- 0 kudos
- 4734 Views
- 0 replies
- 0 kudos
Jira Connector
Is there any native connector in development fir JIRA
- 4734 Views
- 0 replies
- 0 kudos
