- 2861 Views
- 7 replies
- 9 kudos
From Associate to Professional: My Learning Plan to ace all Databricks Data Engineer Certifications
In today’s data-driven world, the role of a data engineer is critical in designing and maintaining the infrastructure that allows for the efficient collection, storage, and analysis of large volumes of data. Databricks certifications holds significan...
- 2861 Views
- 7 replies
- 9 kudos
- 9 kudos
As an additional tip for those working towards both the Associate and Professional certifications, I recommend avoiding a long gap between the two exams to maintain your momentum. If possible, try to schedule them back-to-back with just a few days in...
- 9 kudos
- 885 Views
- 0 replies
- 1 kudos
Databricks java.util.NoSuchElementException: None.get Error on "SHOW TABLES IN" command
This is not a question, this is just the solution to a problem we encountered in case someone from the community finds it useful.Recently we encountered an issue, where our users' jobs started failing out of nowhere on the following command, with the...
- 885 Views
- 0 replies
- 1 kudos
- 3119 Views
- 0 replies
- 2 kudos
- 3119 Views
- 0 replies
- 2 kudos
- 556 Views
- 0 replies
- 1 kudos
Terraform support for AI/BI dashboards
AI/BI dashboards can now be managed through Terraform.Dashboard using serialized_dashboard attribute: data "databricks_sql_warehouse" "starter" { name = "Starter Warehouse" } resource "databricks_dashboard" "dashboard" { display_name = "...
- 556 Views
- 0 replies
- 1 kudos
- 2957 Views
- 3 replies
- 4 kudos
Introducing Databricks LakeFlow: A New Solution for Data Engineering
Hello Everyone!!!I hope everyone knows about the exciting new launch from Databricks: LakeFlow. This innovative tool makes data engineering simpler and more efficient. Let’s understand what LakeFlow offers and how it can help data teams. Let’s unders...
- 2957 Views
- 3 replies
- 4 kudos
- 4 kudos
Hi @nihalataskeen Thankyou and I have signed up in the waitlist. Any idea when can we might get access to this??
- 4 kudos
- 755 Views
- 0 replies
- 0 kudos
Databricks User interface updates
AI/BI dashboards:Improvements:The assistant can now help you edit chart axis titles and control the visibility of chart data labels.You can now assign multiple measures to pivot tables.Added more relative datetime options, such as Last 10 seconds and...
- 755 Views
- 0 replies
- 0 kudos
- 3816 Views
- 2 replies
- 0 kudos
Resolved! DevOps CICD for Databricks Schema
Does anyone know how to do DevOps CICD pipeline to deploy changes to a Databricks Catalog? I've added a couple tables in DEV and need to deploy in Prod. Could'nt find any info on this anywhere.
- 3816 Views
- 2 replies
- 0 kudos
- 0 kudos
Thanks @Retired_mod , the links refer mostly deployments of workspace objects rather than changes in database objects, eg. a new schema, table, or new column added to a table etc. I'll keep looking...
- 0 kudos
- 1257 Views
- 1 replies
- 1 kudos
Databricks Terraform Authentication Methods
Databricks, a unified data analytics platform, is widely recognized for its ability to process big data and run complex algorithms. When it comes to infrastructure as code, Terraform stands out as a tool that can manage Databricks resources efficient...
- 1257 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Ajay-Pandey , Thank you for sharing this comprehensive guide on managing Databricks resources with Terraform. The detailed explanation of the different authentication methods and examples is very helpful. I appreciate you taking the time to put t...
- 1 kudos
- 3578 Views
- 0 replies
- 0 kudos
Reading modes in spark
Hello Everyone, 3 type of reading mode into pyspark:1.Permissive Mode : its a default mode. if spark is not able to parse it due to dattype mismatch then make it as null without impacting other result, i.e only issue record are null others are fine.2...
- 3578 Views
- 0 replies
- 0 kudos
- 693 Views
- 0 replies
- 0 kudos
Lakehouse Monitoring GA: Profile, diagnose, and enforce data quality
Lakehouse Monitoring provides automated profiling and a dashboard that visualizes trends and anomalies over time. Track key metrics such as data volume, percent nulls, numerical distribution changes, and categorical distribution. For inference tables...
- 693 Views
- 0 replies
- 0 kudos
- 649 Views
- 0 replies
- 0 kudos
Reference SQL results in Workflows [Private Preview]
Reference the result of SQL tasks in Workflows and use it for control flow (If/else condition, For each task) or as a parameter of another task.Enable it today from your Previews page.
- 649 Views
- 0 replies
- 0 kudos
- 930 Views
- 2 replies
- 3 kudos
Resolved! Observability for materialized views and streaming tables
You can now view a materialized view/streaming table's refresh information and schedules from the Catalog Explorer page.
- 930 Views
- 2 replies
- 3 kudos
- 3 kudos
Thank you for sharing this valuable information with the community! @Ajay-Pandey This will surely help many users manage their data workflows more efficiently. We appreciate your contribution and encourage more insights like this.
- 3 kudos
- 2667 Views
- 1 replies
- 4 kudos
Best Practices for Utilizing Databricks Asset Bundles
Best practices for utilizing Databricks Asset Bundles effectively streamline your data and AI projects, ensuring they are robust, scalable, and maintainable. Here are some key practices to consider:1. Version Control: Always maintain a versioned hist...
- 2667 Views
- 1 replies
- 4 kudos
- 4 kudos
Thank you for sharing these best practices. @Ajay-Pandey We appreciate your contribution to helping the community members streamline their projects effectively.
- 4 kudos
- 742 Views
- 0 replies
- 1 kudos
Enhancements: Markdown Easy Editor
Dears,This idea is to bring the HTML editor to our markdown cells in notebook, this will help the developers add comments and note in fast and easy way.Thanks
- 742 Views
- 0 replies
- 1 kudos
- 1616 Views
- 2 replies
- 4 kudos
Resolved! DLT pipeline development made simple with notebooks
We’re just a couple weeks removed from the biggest Data + AI Summit in history, where we introduced Databricks LakeFlow, a unified, intelligent solution for data engineering. While we are hard at work evolving DLT and Workflows into the new unified s...
- 1616 Views
- 2 replies
- 4 kudos
- 4 kudos
Hi @Rishabh-Pandey Thank you for sharing this insightful update! We're thrilled to see your enthusiasm for the latest enhancements to the DLT development experience. Please keep sharing your experiences and insights with the community. Thanks,Rishab...
- 4 kudos
- 1598 Views
- 1 replies
- 1 kudos
Resolved! Databricks SQL fixes
Added support for customizing scatter point size and opacity in AI/BI dashboards.Improved interactivity in displaying tooltips when hovering over pie, scatter, and heatmap charts with many data points.AI/BI dashboard widgets created by Databricks Ass...
- 1598 Views
- 1 replies
- 1 kudos
- 1 kudos
Thank you for sharing the updates on fixes @Ajay-Pandey .
- 1 kudos
