- 6272 Views
- 2 replies
- 2 kudos
Databricks job cluster logs
I am using databricks job cluster for multitask jobs, when my job failed/succeeded I couldn't see any logs, Do I need to add any location in advanced options, cluster logging to see the logs for the failed/succeeded jobs or what it is and how it work...
- 6272 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @swetha kadiyala​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...
- 2 kudos
- 2487 Views
- 1 replies
- 1 kudos
How are you currently analyzing A/B tests data in Databricks?
Super curious about people's workflows for analyzing A/B tests. Trying to learn how to optimize my approach as it is very manual right now.
- 2487 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Ben Mathew​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 1 kudos
- 2663 Views
- 2 replies
- 2 kudos
Resolved! Can we have the powerbi connector step into "hive_metastore" automatically?
We are distributing pbids files providing the connection info to databricks. It contains options passed to the "Databricks.Catalogs " function implementing the connection to databricks. It is my understanding that databricks has made this together wi...
- 2663 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Erik Parmann​ Does @Hubert Dudek​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 2 kudos
- 1652 Views
- 1 replies
- 0 kudos
How to search for an S3 bucket
The S3 buckets are a likely source location for the new EDL builder uploads. Is there a way to search Databricks to find the naming convention for the S3 buckets that have been assigned to our team. We uploaded some files using EDL this morning but...
- 1652 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @James Longstreet​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.T...
- 0 kudos
- 6051 Views
- 4 replies
- 3 kudos
Resolved! SQL Warehouse Serverless Endpoint Error
Our SQL Warehouse Serverless Endpoint started failing from this morning (2022-08-23 18:00:00 UTC):“org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Unable to build AWSGlueClient: com.amazonaws.SdkClientException: Unable to find...
- 6051 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi @Bo Zhu​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 3 kudos
- 8158 Views
- 11 replies
- 2 kudos
Visualizations from notebook keeps on disappearing
I have created a notebook which filters on the main table and generates bar graph. The graphs keeps on disappearing and sometimes they dont generate at all. PFA. 'Visualization 1' has issues. Each time I run notebook manually, I have to build the gra...
- 8158 Views
- 11 replies
- 2 kudos
- 2 kudos
@Andrew Freix​ @Rubel Ahamed​ @Sneha Mulrajani​ eha15Can you please share the workspace URL version output? so that I can check your workspace and let you know whether new Visualization is enabled or not. Please run the below URL in your web browser ...
- 2 kudos
- 11293 Views
- 7 replies
- 6 kudos
Resolved! Making a connection to the tables in Dynamics app through the dataverse TDS endpoint
I wish to do some analysis on tables that are stored in dataverse in databricks. I know that PowerBi uses its Dataverse connector to fetch the data using a Dataverse's TDS endpoint. The tables that we import in PowerBi using this connector is nearly ...
- 11293 Views
- 7 replies
- 6 kudos
- 6 kudos
I finally followed the following method for my requirement: 1) Connected the data verse tables from PowerApps to synapse workspace using the synapse link2) Used the standard template provided in it to convert the multiple .csv files to parquet format...
- 6 kudos
- 6859 Views
- 4 replies
- 16 kudos
Resolved! Unity Catalog - External Table
I am not sure if I am missing something, but I just created External Table using External Location and I can still access both data through the table and directly access files from the storage: documentation:https://docs.databricks.com/data-governanc...
- 6859 Views
- 4 replies
- 16 kudos
- 16 kudos
I got the answer from the Databricks Support on this.The point which has been mentioned in doc "Once a table is created in a path, users can no longer directly access the files in that path even if they have been given privileges on an external locat...
- 16 kudos
- 1510 Views
- 1 replies
- 2 kudos
Trying to find aws access key/secret
Looking at the instance profiles we appear to be using AWS IAM roles, however there is a databricks user in our IAM that cloudtrail currently shows its aws access key is being used to describeinstances and describevolumes and I am trying to track dow...
- 1510 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi, @ggleason​ , you can list secrets , ref: https://docs.databricks.com/security/secrets/secrets.html#list-secrets. You can also check on secrets API: https://docs.databricks.com/dev-tools/api/latest/secrets.html
- 2 kudos
- 944 Views
- 0 replies
- 1 kudos
X axis range for visualization in sql dashboard
Unable to set x axis range for charts in sql dashboard. The automatic range and padding that is set doesn't match across all plot types. For line charts with date type data on x axis, there is a padding on both sides of the plot. But for bar charts, ...
- 944 Views
- 0 replies
- 1 kudos
- 8357 Views
- 3 replies
- 3 kudos
Resolved! Differences between Spark Cluster Manager and Databricks Cluster Manager?
I didn't found any documentation on Databricks Cluster Manager. Could anyone give me some resources on this topic?
- 8357 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @John William​ Databricks clusters use Spark's Standalone cluster manager. Each Databricks cluster has its own standalone Master and Worker processes run inside of the LXC containers and share a lifecycle with the cluster. Each cluster has a singl...
- 3 kudos
- 2625 Views
- 4 replies
- 11 kudos
Resolved! Databricks pyspark - Find columns in xls file.
Hello everyone, every day I extract data into xls files but the column position changes every day. Is there any way to find these columns within the file?Here's a snippet of my code.df = spark.read.format("com.crealytics.spark.excel")\ .option("hea...
- 2625 Views
- 4 replies
- 11 kudos
- 11 kudos
Hi @welder martins​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
- 11 kudos
- 1816 Views
- 2 replies
- 3 kudos
DLT Event Log
I am trying to utilize the Event Log DLT is keeping updated, I noticed some of the fields are consistently empty/null.In the Event Log, located ".../storage/system/events", I see the field "origin" and there are nested fields within which are empty/n...
- 1816 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @Kristian Foster​,The following docs will provide more details on the event log schema. Please refer to this link https://docs.databricks.com/workflows/delta-live-tables/delta-live-tables-event-log.html#monitor-pipelines-with-the-delta-live-tables...
- 3 kudos
- 1724 Views
- 2 replies
- 2 kudos
What is the long-term plan for the official Databricks Docker Containers?
Hi, I'd like to ask you, how much resources do you plan to dedicate to the maintenance/development of the official Databricks Docker images, please? Do you have a view on the longer-term plan for these docker images? It seems to be maintained, but i...
- 1724 Views
- 2 replies
- 2 kudos
- 6392 Views
- 6 replies
- 3 kudos
GRANT permission not working on Storage Credential using DBR 11.2 Shared mode
We were trying to update permissions of Storage credential using DBR 11.2 Shared mode but running into following issue Operation not allowed: GRANT(line 1, pos 0) PFA complete error in file.Is the above issue with cluster permission or changing permi...
- 6392 Views
- 6 replies
- 3 kudos
- 3 kudos
Hi @Parth Salvi​ , could you please paste the full error here, also, it is better to create a support case with Databricks with the whole error.
- 3 kudos