- 15069 Views
- 2 replies
- 1 kudos
Resolved! Not able to access Account console page
Hello All,I'm not be able to access the account console page. I'm a portal admin, my workspace is premium, and yet the Databricks portal stays in a loop, always returning to the Workspaces Overview page and not going to the accounts console page so t...
- 15069 Views
- 2 replies
- 1 kudos
- 1 kudos
Guys, the problem is that I was not signed in as a Global Administrator in Azure AD. After that, I reset all the Browser settings and I was able to do that. Here's the tip.
- 1 kudos
- 2216 Views
- 1 replies
- 1 kudos
Performance issue while calling Sagemaker Endpoint in pyspark udf
Hi,I have pyspark dataframe which calls pyspark udf which in turn calls sagemaker endpoint. But when dataframe has more rows, endpoint start failing. Also it takes longer to process.Please suggest how to call sagemaker endpoint from pyspark.Regards,S...
- 2216 Views
- 1 replies
- 1 kudos
- 2244 Views
- 0 replies
- 0 kudos
org.apache.spark.SparkException - FileReadException
Sometimes getting this kind of error "org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 12224.0 failed 4 times, most recent failure: Lost task 1.5 in stage 12224.0 (TID ) (12.xxx.x.xxx executor 1): com.datab...
- 2244 Views
- 0 replies
- 0 kudos
- 3998 Views
- 2 replies
- 0 kudos
how to edit or delete the post in this community
how to edit or delete the post in this community
- 3998 Views
- 2 replies
- 0 kudos
- 4765 Views
- 3 replies
- 1 kudos
Add Oracle Jar to Databricks cluster policy
I created a policy for users to use when they create their own Job clusters. When I'm editing the policy, I don't have the UI options for adding library (I can only see Definitions and Permissions tabs). I need to add via JSON the option to allows th...
- 4765 Views
- 3 replies
- 1 kudos
- 1 kudos
@adrianhernandez are you admin to workspace, if not you might be missing permissions, if you have policies enabled, admin can allow you.https://docs.databricks.com/en/administration-guide/clusters/policies.html#librariesif your workspace is Unity cat...
- 1 kudos
- 2399 Views
- 2 replies
- 1 kudos
Web terminal and clusters
Hi, I have come across this piece of documentation:Databricks does not support running Spark jobs from the web terminal. In addition, Databricks web terminal is not available in the following cluster types:Job clustersClusters launched with the DISAB...
- 2399 Views
- 2 replies
- 1 kudos
- 4683 Views
- 0 replies
- 0 kudos
Creating java UDF for Spark SQL
Hello, I have created a sample java UDF which masks few characters of a string. However I facing couple of issues when uploading and using it.First I could only import it, which for now is OK. But when do the following,create function udf_mask as 'ba...
- 4683 Views
- 0 replies
- 0 kudos
- 3090 Views
- 0 replies
- 2 kudos
dbutils.fs.ls MAX_LIST_SIZE_EXCEEDED
Hi!I'm experiencing different behaviours between two DBX Workspaces when trying to list file contents from an abfss: location.In workspace A running len(dbutils.fs.ls('abfss://~~@~~~~.dfs.core.windows.net/~~/')) results in "Out[1]: 1551", while runni...
- 3090 Views
- 0 replies
- 2 kudos
- 22962 Views
- 3 replies
- 4 kudos
Training @ Data & AI World Tour 2023
Join your peers at the Data + AI World Tour 2023! Explore the latest advancements, hear real-world case studies and discover best practices that deliver data and AI transformation. From the Databricks Lakehouse Platform to open source technologies in...
- 22962 Views
- 3 replies
- 4 kudos
- 4 kudos
Introducing Mini Flush: Your Ticket to Ultimate Casino Thrills!Are you ready to embark on an electrifying journey into the world of online gambling? If so, look no further than Vijaybet Online Casino! Our state-of-the-art platform is your gateway to ...
- 4 kudos
- 2893 Views
- 1 replies
- 1 kudos
Resolved! Problem creating external delta table on non-AWS s3 bucket
I am testing Databricks with non-AWS S3 object storage. I can access the non-AWS S3 bucket by setting these parameters:sc._jsc.hadoopConfiguration().set("fs.s3a.access.key", "XXXXXXXXXXXXXXXXXXXX")sc._jsc.hadoopConfiguration().set("fs.s3a.secret.key...
- 2893 Views
- 1 replies
- 1 kudos
- 1 kudos
Found the solution to disable it. Can close this question.
- 1 kudos
- 5673 Views
- 3 replies
- 1 kudos
getArgument works fine in interactive cluster 10.4 LTS, raises error in interactive cluster 10.4 LTS
Hello,I am trying to use the getArgument() function in a spark.sql query. It works fine if I run the notebook via an interactive cluster, but gives an error when executed via a job run in an instance Pool.query:OPTIMIZE <table>where date = replace(re...
- 5673 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @Retired_mod,Would you be able to respond to my last comment? I couldn't manage to get it working yet.Thank you in advance.
- 1 kudos
- 28775 Views
- 0 replies
- 3 kudos
Schema owned by Service Principal shows error in PBI
Background info:1. We have unity catalog enabled. 2. All of our jobs are run by Service Principal that has all necessary access it needs.Issue:One of the jobs checks existing schemas against the ones it is supposed to create in that given run and if ...
- 28775 Views
- 0 replies
- 3 kudos
- 3933 Views
- 1 replies
- 0 kudos
AWS Databricks VS AWS EMR
HiWhich services should I use for data lake implementation?any cost comparison between Databricks and aws emr.which one is best to choose
- 3933 Views
- 1 replies
- 0 kudos
- 0 kudos
@AH that depends on use case, if your implementation involves Data Lake, ML, Data engineering tasks better to go with databricks as it has got good UI and there good governance using unity catalog for your data lake and you have good consumer tool su...
- 0 kudos
- 4290 Views
- 1 replies
- 1 kudos
Resolved! System billing usage table - Usage column
Hello experts,Could someone please explain what is exactly contained into the column usage in the system.billing.usage table?We ran specific queries in a cluster trying to calculate the cost and we observe that the DBUs shown in the system table are ...
- 4290 Views
- 1 replies
- 1 kudos
- 1 kudos
@elgeo both should be same, untill if somehow we miss to pick proper plan DBU price, usage column will have complete information related to sku name and DBU units etc... if you use azure databricks calculator and compare we should see similar result
- 1 kudos
- 1889 Views
- 0 replies
- 0 kudos
Suggestions for python its not working
Tab and Shift tab its not suggestin nothing in python codeTab only work as indentation and shift + tab do nothing.What can I do?
- 1889 Views
- 0 replies
- 0 kudos
