cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Hubert-Dudek
by Esteemed Contributor III
  • 5072 Views
  • 2 replies
  • 9 kudos

Login to ideas.databricks.com from Azure or community accounts.

When I go to ideas.databricks.com it display me screen asking about workspace (so I put there for example westeurope.azuredatabricks.net):then it redirect to login and then to... my azure workspace instead of ideas.When I want to use community (I put...

image.png image.png
  • 5072 Views
  • 2 replies
  • 9 kudos
Latest Reply
Anonymous
Not applicable
  • 9 kudos

@Hubert Dudek​ - Would you be happy to send a message to sales@databricks.com, please? We think they're the ones best suited to help you.

  • 9 kudos
1 More Replies
Logan_Data_Inc_
by New Contributor II
  • 3384 Views
  • 3 replies
  • 4 kudos

Resolved! Will the database be deleted when the cluster terminates ?

Hello,I'm working in Databricks Community Edition. So I will terminate my cluster after my work because anyways it will be terminated after 2 hours. I'm creating a database to store all my transformed data. Will the database be deleted when I termina...

  • 3384 Views
  • 3 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

@Hubert Dudek​ - Thanks for answering so quickly!!@Sriram Devineedi​ - If Hubert's answer solved the issue for you, would you be happy to mark his answer as best? That helps others know where to look.

  • 4 kudos
2 More Replies
Joseph_B
by Databricks Employee
  • 2404 Views
  • 1 replies
  • 0 kudos

How should I tune hyperparameters when fitting models for every item?

My dataset has an "item" column which groups the rows into many groups. (Think of these groups as items in a store.) I want to fit 1 ML model per group. Should I tune hyperparameters for each group separately? Or should I tune them for the entire...

  • 2404 Views
  • 1 replies
  • 0 kudos
Latest Reply
Joseph_B
Databricks Employee
  • 0 kudos

For the first question ("which option is better?"), you need to answer that via your understanding of the problem domain.Do you expect similar behavior across the groups (items)?If so, that's a +1 in favor of sharing hyperparameters. And vice versa....

  • 0 kudos
AryaMa
by New Contributor III
  • 3504 Views
  • 3 replies
  • 4 kudos

Sending email from databricks to google drive attachement

https://stackoverflow.com/questions/67088891/send-email-from-databricks-notebook-with-attachmenti have to send the attachment to the organisation google drive folder directly instead of email any suggestionssample email with attachement codemsg.atta...

  • 3504 Views
  • 3 replies
  • 4 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 4 kudos

Maybe just use azure logic apps or power automate (trigger as http requests with json and then all actions there)

  • 4 kudos
2 More Replies
Sri_H
by New Contributor III
  • 4442 Views
  • 4 replies
  • 4 kudos

Did not receive my certificate after passing Associate Dev

I have completed my Databricks associate developer assessment on 12/05/2021 and received a pass result. On 12/08/2021 I have received an email stating my digital badge for this assessment is available. However, I do not see this badge or my completio...

  • 4442 Views
  • 4 replies
  • 4 kudos
Latest Reply
Sri_H
New Contributor III
  • 4 kudos

Hi @Kaniz Fatma​ , I have still not received the certificate and Badge. Can you please help follow-up once regarding this?Thank You.

  • 4 kudos
3 More Replies
cconnell
by Contributor II
  • 1114 Views
  • 0 replies
  • 2 kudos

www.linkedin.com

Here is an article I wrote that puts Databricks in a historical context (why was it developed?) and provides introductory steps to help a newbie get started. Feel free to copy/link as you want.https://www.linkedin.com/pulse/databricks-introduction-ch...

  • 1114 Views
  • 0 replies
  • 2 kudos
cconnell
by Contributor II
  • 1202 Views
  • 1 replies
  • 1 kudos

www.linkedin.com

Importing JSON to Databricks (PySpark) is simple in the simple case. But of course there are wrinkles for real-world data. Here are some tips/tricks to help...https://www.linkedin.com/pulse/json-databricks-pyspark-chuck-connell/

  • 1202 Views
  • 1 replies
  • 1 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 1 kudos

Nice article

  • 1 kudos
William_Scardua
by Valued Contributor
  • 7632 Views
  • 4 replies
  • 3 kudos

Resolved! Update Databricks Runtime

Hy guys,I need to upgrade my databricks runtime (current 8.0 ) What the precautions should I take ?Thank you very much

  • 7632 Views
  • 4 replies
  • 3 kudos
Latest Reply
Steward475
New Contributor II
  • 3 kudos

If you want to know the version of Databricks runtime in Azure after creation: Go to Azure Data bricks portal => Clusters => Interactive Clusters => here you can find the run time version. For more details, refer "Azure Databricks Runtime versions".R...

  • 3 kudos
3 More Replies
test_data
by New Contributor III
  • 4784 Views
  • 1 replies
  • 1 kudos
  • 4784 Views
  • 1 replies
  • 1 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 1 kudos

Please use Repos and in admin settings please enable "Files in Repo" than you will be able to import class in notebook:from repo_folder.subfolders.file import your_class

  • 1 kudos
test_data
by New Contributor III
  • 5672 Views
  • 2 replies
  • 2 kudos

need to move notebook file from workspace to dbfs.

Hi teami need to move the notebook file from workspace to dbfs. i have tried and getting an error there is no file ?

  • 5672 Views
  • 2 replies
  • 2 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 2 kudos

Not sure what command are you using and what is error? As access to workspace is managed per user on databricks so %sh magic commands will not work as you could see other user files.Dbfs is data storage. Notebook code from Workspace can be moved to R...

  • 2 kudos
1 More Replies
prasadvaze
by Valued Contributor II
  • 1587 Views
  • 1 replies
  • 1 kudos

which table in external hive metastore stores the folder path for delta table's data?

which table in external hive metastore stores the folder path for delta table's data? Is it SDS table?

  • 1587 Views
  • 1 replies
  • 1 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 1 kudos

Image here can be helpful https://analyticsanvil.files.wordpress.com/2016/08/hive_metastore_database_diagram.png

  • 1 kudos
prasadvaze
by Valued Contributor II
  • 6598 Views
  • 6 replies
  • 2 kudos

Resolved! SQL endpoint is unable to connect to external hive metastore ( Azure databricks)

Using Azure databricks, I have set up SQL Endpoint with the connection details that match with global init script. I am able to browse tables from regular cluster in Data Engineering module but i get below error when trying a query using SQL Endpoint...

  • 6598 Views
  • 6 replies
  • 2 kudos
Latest Reply
prasadvaze
Valued Contributor II
  • 2 kudos

@Prabakar Ammeappin​  @Kaniz Fatma​  Also I found out that after delta table is created in external metastore (and the table data resides in ADLS) then in the sql end point settings I do not need to provide ADLS connection details. I only provided...

  • 2 kudos
5 More Replies
BorislavBlagoev
by Valued Contributor III
  • 5885 Views
  • 11 replies
  • 2 kudos

Resolved! Databricks SQL Connector

I started to explore Databricks SQL Connector. And I'm wondering can I execute query from Databricks SQL Connector from notebook or job?sample code:'test_query' lives in Databricks SQL. It is ready to use query!from databricks import sql   hostname =...

  • 5885 Views
  • 11 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@Borislav Blagoev​ - There is a "Select as Best" link underneath each reply. All you need to do, if you choose, is to click on that link for whichever reply best answered your question. The purpose is to help other members find helpful solutions more...

  • 2 kudos
10 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels