cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
witnessthee
by • New Contributor II
  • 7027 Views
  • 3 replies
  • 2 kudos

Resolved! Error when using pyflink on databricks, An error occurred while trying to connect to the Java server

Hi, right now I am trying to run a pyflink script that can connect to a kafka server. When I run that script, I got an error "An error occurred while trying to connect to the Java server 127.0.0.1:35529". Do I need to install a extra jdk for that? er...

Data Engineering
Reply
  • 7027 Views
  • 3 replies
  • 2 kudos
View Replies
Latest Reply
-werners-
Esteemed Contributor III
  • 2 kudos

did you get Flink running on the Databricks cluster? Because that seems to be the issue here.

  • 2 kudos
2 More Replies
bayu
by • New Contributor III
  • 35923 Views
  • 8 replies
  • 4 kudos

Can't login back to Databricks

Hi,I'm trying the Databricks Premium. Got registered etc. No problem.Then I setup AWS and workspace etc. No problem.Back to Databricks and now can't login. Trying to reset password but get nothing in the email.Trying other browser also doesn't work.C...

Data Engineering
Reply
  • 35923 Views
  • 8 replies
  • 4 kudos
View Replies
Latest Reply
bayu
New Contributor III
  • 4 kudos

Hi @Debayan Mukherjee​ could you please help me out?Is there any way I can login to my Databricks account?

  • 4 kudos
7 More Replies
FerArribas
by • Contributor
  • 5170 Views
  • 2 replies
  • 8 kudos

Resolved! Error cluster launch: Security Daemon Registration

I have created a workspace in AWS with private link. When we launch a cluster we get the following error: "Security Daemon Registration Exception: Failed to set up the spark container due to an error when registering the container to security daemon"...

Data Engineering
Reply
  • 5170 Views
  • 2 replies
  • 8 kudos
View Replies
Latest Reply
FerArribas
Contributor
  • 8 kudos

Thanks @Sivaprasad C S​  It's already working. The problem was that the AWS STS private link was wrongly associated with the correct subnet.

  • 8 kudos
1 More Replies
iajay543
by • New Contributor
  • 2422 Views
  • 2 replies
  • 0 kudos

Unable to sign it at customer-academy.databricks.com but can't do the same at partner-academy.databricks.com

Hi thereI have used my company email to register an account for https://customer-academy.databricks.com/learn/signin a while back.Now what I need to do is create an account with https://partner-academy.databricks.com/learn/signin using my company ema...

Data Engineering
Reply
  • 2422 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Nadia1
Databricks Employee
  • 0 kudos

Hello Ajay,Thank you for reaching out. Please submit a ticket to https://help.databricks.com/s/contact-us?ReqType=training . Thank you!

  • 0 kudos
1 More Replies
ivanychev
by • Contributor II
  • 5028 Views
  • 7 replies
  • 2 kudos

How do I enable new Visualization tools in my Databricks notebooks?

Databricks released new visualization toolshttps://docs.databricks.com/notebooks/visualizations/index.htmlBut I don't see them in my notebook (DBR 10.4) nor I can find out how to enable them. How do I use the new tools?

Screenshot 2022-07-21 at 21.59.19
Data Engineering
Reply
  • 5028 Views
  • 7 replies
  • 2 kudos
View Replies
Latest Reply
cweymouth
New Contributor II
  • 2 kudos

@Atanu Sarkar​ , I am still not seeing this. I am running a cluster on runtime 10.4 LTS ML. The "Legacy Visualizations" are the only option available. This feature should be fully deployed, yes? 

  • 2 kudos
6 More Replies
qanut
by • New Contributor II
  • 1323 Views
  • 1 replies
  • 1 kudos

DBFS Delete Bug on Community Edition

I uploaded some files in Community Edition. But when I try to delete them I get the popup that say do you want to delete, and when I select delete, the popup goes away but the file doesn't delete. Seems the the GUI is bugged. However, when I use d...

Data Engineering
Reply
  • 1323 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
Simranarora
Databricks Employee
  • 1 kudos

Thanks for reporting. We will look into it!!

  • 1 kudos
Harsh1
by • New Contributor II
  • 1074 Views
  • 0 replies
  • 0 kudos

Issues in Metastore Migration using Databricks Migration Tool

Hi Team,As I'm performing the Databricks workspace migration, during Metastore migration I'm facing below issue.As we found differences in the Metastore table count between Legacy and Target workspace, we checked error logs.After going through Failed...

Data Engineering
Reply
  • 1074 Views
  • 0 replies
  • 0 kudos
swetha
by • New Contributor III
  • 6859 Views
  • 2 replies
  • 2 kudos

Databricks job cluster logs

I am using databricks job cluster for multitask jobs, when my job failed/succeeded I couldn't see any logs, Do I need to add any location in advanced options, cluster logging to see the logs for the failed/succeeded jobs or what it is and how it work...

Data Engineering
Reply
  • 6859 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @swetha kadiyala​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...

  • 2 kudos
1 More Replies
bmat
by • New Contributor II
  • 2611 Views
  • 1 replies
  • 1 kudos

How are you currently analyzing A/B tests data in Databricks?

Super curious about people's workflows for analyzing A/B tests. Trying to learn how to optimize my approach as it is very manual right now.

Data Engineering
Reply
  • 2611 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Ben Mathew​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 1 kudos
Erik
by • Valued Contributor III
  • 2826 Views
  • 2 replies
  • 2 kudos

Resolved! Can we have the powerbi connector step into "hive_metastore" automatically?

We are distributing pbids files providing the connection info to databricks. It contains options passed to the "Databricks.Catalogs " function implementing the connection to databricks. It is my understanding that databricks has made this together wi...

Data Engineering
Reply
  • 2826 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Erik Parmann​ Does @Hubert Dudek​  response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!

  • 2 kudos
1 More Replies
Navy__jup-frag-
by • New Contributor
  • 1723 Views
  • 1 replies
  • 0 kudos

How to search for an S3 bucket

The S3 buckets are a likely source location for the new EDL builder uploads. Is there a way to search Databricks to find the naming convention for the S3 buckets that have been assigned to our team. We uploaded some files using EDL this morning but...

Data Engineering
Reply
  • 1723 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @James Longstreet​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.T...

  • 0 kudos
bozhu
by • Contributor
  • 6350 Views
  • 4 replies
  • 3 kudos

Resolved! SQL Warehouse Serverless Endpoint Error

Our SQL Warehouse Serverless Endpoint started failing from this morning (2022-08-23 18:00:00 UTC):“org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Unable to build AWSGlueClient: com.amazonaws.SdkClientException: Unable to find...

Data Engineering
Reply
  • 6350 Views
  • 4 replies
  • 3 kudos
View Replies
Latest Reply
Vidula
Honored Contributor
  • 3 kudos

Hi @Bo Zhu​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 3 kudos
3 More Replies
Sneha1594
by • New Contributor III
  • 8631 Views
  • 11 replies
  • 2 kudos

Visualizations from notebook keeps on disappearing

I have created a notebook which filters on the main table and generates bar graph. The graphs keeps on disappearing and sometimes they dont generate at all. PFA. 'Visualization 1' has issues. Each time I run notebook manually, I have to build the gra...

image
Data Engineering
Reply
  • 8631 Views
  • 11 replies
  • 2 kudos
View Replies
Latest Reply
Abishek
Databricks Employee
  • 2 kudos

@Andrew Freix​ @Rubel Ahamed​ @Sneha Mulrajani​ eha15Can you please share the workspace URL version output? so that I can check your workspace and let you know whether new Visualization is enabled or not. Please run the below URL in your web browser ...

  • 2 kudos
10 More Replies
Pat
by • Honored Contributor III
  • 7405 Views
  • 4 replies
  • 16 kudos

Resolved! Unity Catalog - External Table

I am not sure if I am missing something, but I just created External Table using External Location and I can still access both data through the table and directly access files from the storage: documentation:https://docs.databricks.com/data-governanc...

image.png image.png
Data Engineering
Reply
  • 7405 Views
  • 4 replies
  • 16 kudos
View Replies
Latest Reply
Pat
Honored Contributor III
  • 16 kudos

I got the answer from the Databricks Support on this.The point which has been mentioned in doc "Once a table is created in a path, users can no longer directly access the files in that path even if they have been given privileges on an external locat...

  • 16 kudos
3 More Replies
746837
by • New Contributor II
  • 1589 Views
  • 1 replies
  • 2 kudos

Trying to find aws access key/secret

Looking at the instance profiles we appear to be using AWS IAM roles, however there is a databricks user in our IAM that cloudtrail currently shows its aws access key is being used to describeinstances and describevolumes and I am trying to track dow...

Data Engineering
Reply
  • 1589 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
Debayan
Databricks Employee
  • 2 kudos

Hi, @ggleason​ , you can list secrets , ref: https://docs.databricks.com/security/secrets/secrets.html#list-secrets. You can also check on secrets API: https://docs.databricks.com/dev-tools/api/latest/secrets.html

  • 2 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels