cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
ClaudeR
by • New Contributor III
  • 4901 Views
  • 5 replies
  • 1 kudos

Resolved! Can someone help me understand how compute pricing works.

Im looking at using Databricks internally for some Data Science projects. I am however very confused to how the pricing works and would like to obviously avoid high spending right now. Internal documentation and within Databricks All-Purpose Compute...

Data Engineering
Reply
  • 4901 Views
  • 5 replies
  • 1 kudos
View Replies
Latest Reply
GuillermoM
New Contributor II
  • 1 kudos

Hello,I was able to get a very precise cost of Azure Databricks Clusters and Computers jobs, using the Microsoft API and Databricks APIThen I wrote a simple tool to extract and manipulate the API results and generate detailed cost reports that can be...

  • 1 kudos
4 More Replies
Anonymous
by • Not applicable
  • 1756 Views
  • 2 replies
  • 0 kudos

Resolved! Auto optimize config

Does auto-optimize work for existing tables only or will it work for both existing and new tables when we enable at the cluster config level?

Data Engineering
Reply
  • 1756 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Mooune_DBU
Valued Contributor
  • 0 kudos

If you're referring to Delta Tables, Auto-Optimize will work for both.For new tables:CREATE TABLE student (id INT, name STRING, age INT) TBLPROPERTIES (delta.autoOptimize.optimizeWrite = true, delta.autoOptimize.autoCompact = true)For existing tables...

  • 0 kudos
1 More Replies
Priyag1
by • Honored Contributor II
  • 1926 Views
  • 2 replies
  • 11 kudos

Query parameters in dashboardsQueries can optionally leverage parameters or static values. When a visualization based on a parameterized query is adde...

Query parameters in dashboardsQueries can optionally leverage parameters or static values. When a visualization based on a parameterized query is added to a dashboard, the visualization can either be configured to use a:Widget parameterWidget paramet...

Data Engineering
Reply
  • 1926 Views
  • 2 replies
  • 11 kudos
View Replies
Latest Reply
Natalie_NL
New Contributor II
  • 11 kudos

Hi, I build a dashboard with dashboard parameters, it works pretty easy!The advantage of dashboard parameters is that you do not have to set a default (it can be: all). This is convenient when you need to filter on values that change every time the q...

  • 11 kudos
1 More Replies
VVill_T
by • Contributor
  • 17235 Views
  • 13 replies
  • 59 kudos

Resolved! Data Engineering with Databricks V2 or V3 for qualification in a few months time

If I am new to Databricks and is aiming to get qualification some point Dec2022 or Jan 2023, should I be studying the material Data Engineering with Databricks V2 or V3?

Data Engineering
Reply
  • 17235 Views
  • 13 replies
  • 59 kudos
View Replies
Latest Reply
Devarsh
Contributor
  • 59 kudos

I would suggest to go for V3 because the course Data Engineering with Databricks (V3) is the latest version as of now and was released on 14th October 2022. So, this version would have more topics in comparison to V2.

  • 59 kudos
12 More Replies
labtech
by • Valued Contributor II
  • 1722 Views
  • 4 replies
  • 14 kudos

Resolved! Get a new badge or new certified for version 3 of DE exam

I took a certified of DE exam (version 2). Do I receive a new badge or certified when I pass newest version of DE exam?I'm going to take that and review my knowlege.

Data Engineering
Reply
  • 1722 Views
  • 4 replies
  • 14 kudos
View Replies
Latest Reply
Kaniz_Fatma
Community Manager
  • 14 kudos

Hi @Gam Nguyen​, Thank you for reaching out!Let us look into this for you, and we'll circle back with an update.

  • 14 kudos
3 More Replies
SparrowDev
by • New Contributor II
  • 7246 Views
  • 5 replies
  • 3 kudos

Resolved! New branch has a changes

Hi there, I ran into issue with Databricks repo. When I create new repo, it doesn't pull default branch from GitLab. I have default branch 'development', but Databricks repo pulls other branch.Moreover this just now added repo already contains change...

Data Engineering
Reply
  • 7246 Views
  • 5 replies
  • 3 kudos
View Replies
Latest Reply
-werners-
Esteemed Contributor III
  • 3 kudos

I had a similar issue, where a few notebooks popped up as changed in every branch I created (even though they were not touched). So first I manually did the discard changes thing, but that was not a solution as with every new branch they were there ...

  • 3 kudos
4 More Replies
Praveen2609
by • New Contributor
  • 1791 Views
  • 2 replies
  • 0 kudos

dbfs access for job clusters and interactive cluster

Hi All,I am new to databricks need some understanding for my requirement .our requirement:a: we have zip file in azure blob storage and we are bringing that file to dbfs and unzip that file and executing our transformations in multiple steps (3 steps...

Data Engineering
Reply
  • 1791 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @praveen rajak​ Does @Debayan Mukherjee​  response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!

  • 0 kudos
1 More Replies
dslin
by • New Contributor III
  • 2314 Views
  • 3 replies
  • 2 kudos

How to deploy a python script with dependencies by dbx?

Hi,I'm quite new here. I'm trying to perform a deployment of python file with dbx command. The file contains libraries to be installed. How may I deploy the file (together with its dependencies) to databricks?Here are the commands I currently run:`db...

Data Engineering
Reply
  • 2314 Views
  • 3 replies
  • 2 kudos
View Replies
Latest Reply
Vidula
Honored Contributor
  • 2 kudos

Hi @Di Lin​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 2 kudos
2 More Replies
Labels