- 2650 Views
- 1 replies
- 3 kudos
Why we are using shared variables into spark?
Hello Databricks Community!I am excited to share my first blog post with you all. This is a small and basic introduction to the concept of shared variables in Apache Spark. I hope this post will help those who are new to Spark understand why shared ...
- 2650 Views
- 1 replies
- 3 kudos
- 3 kudos
Hi @Yogic24 , Welcome to the Databricks Community! Thank you for sharing your first blog post with us, I am sure it will help our community members. Thank you for your contribution and support! Thanks,Rishabh
- 3 kudos
- 2463 Views
- 2 replies
- 5 kudos
knowledge sharing
Hello Databricks Community,I hope this message finds you well.My name is Yogita, and I am excited to announce that I have started sharing knowledge and insights on various Spark scenarios and theoretical questions on linkdin platform. My aim is to cr...
- 2463 Views
- 2 replies
- 5 kudos
- 5 kudos
Hi @Yogic24 , that's fantastic! Please share your blogs in this space so fellow data practitioners can benefit.
- 5 kudos
- 2487 Views
- 1 replies
- 1 kudos
Enhanced Search in Schema Browser
More easily find models and tables in your organization with our enhanced Schema Browser search experience. Functions and volumes support are coming soon. Opt-in to use this feature by visiting Settings > User > Developer settings.
- 2487 Views
- 1 replies
- 1 kudos
- 1 kudos
Thank you for sharing this update with the community! @Ajay-Pandey Thanks for keeping the community informed.
- 1 kudos
- 2931 Views
- 1 replies
- 2 kudos
Can monitor permission for SQL Warehouses now in Public Preview
Can monitor permission allows users to monitor SQL Warehouses, including query history and query profiles.
- 2931 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @Ajay-Pandey , Thank you for sharing this. I am sure it will help community members. We appreciate your participation. Thanks, Rishabh
- 2 kudos
- 1593 Views
- 1 replies
- 2 kudos
Bulk move and delete available in filebrowser
You can now perform bulk move and delete on your filebrowser assets. Select multiple assets and right-click or use the new actions toolbar.
- 1593 Views
- 1 replies
- 2 kudos
- 2 kudos
Thank you for sharing the quick tricks @Ajay-Pandey I am sure, it will help other community members.
- 2 kudos
- 2540 Views
- 1 replies
- 3 kudos
Caching and data freshness
Dashboards maintain a 24-hour result cache to optimize initial loading times, operating on a best-effort basis. This means that while the system always attempts to use historical query results linked to dashboard credentials to enhance performance, t...
- 2540 Views
- 1 replies
- 3 kudos
- 3 kudos
Thank you for providing this valuable update on caching and data freshness @Ajay-Pandey . It's great to see how these features are designed to optimize performance and ensure data accuracy. We appreciate your detailed explanation and your efforts to ...
- 3 kudos
- 1500 Views
- 1 replies
- 2 kudos
Accelerating discovery on Unity Catalog with a revamped Catalog Explore
Discover favorite and recent UC assets in Quick Access. You'll experience a simplified navigation with the gear icon (top left) for compute, storage, credentials, connections, DBFS, and managements features. Delta Sharing, Clean Rooms, and External D...
- 1500 Views
- 1 replies
- 2 kudos
- 2 kudos
Thank you for sharing this update on the Unity Catalog! @Ajay-Pandey Appreciate the detailed overview!
- 2 kudos
- 10375 Views
- 1 replies
- 5 kudos
Configuring DNS resolution for Private Databricks Workspaces (AWS)
Intro For customers on the E2 Platform, Databricks has a feature that allows them to use AWS PrivateLink to provision secure private workspaces by creating VPC endpoints to both the front-end and back-end interfaces of the Databricks infrastructure. ...
- 10375 Views
- 1 replies
- 5 kudos
- 5 kudos
@alysson_souza love it! Thank you for sharing
- 5 kudos
- 5163 Views
- 0 replies
- 2 kudos
CICD for databricks workflow jobs
This post is to set up Databricks workflow jobs as a CI/CD. Below are the two essential components needed for a complete CI/CD setup of workflow jobs. Databricks Asset Bundles(DABs)AzureDevOps pipeline. Databricks Asset Bundle ( From local terminal )...
- 5163 Views
- 0 replies
- 2 kudos
- 4443 Views
- 1 replies
- 3 kudos
Resolved! RamK - Certification Update
Hi Team,My name is Ram based out of Singapore. I am new to this Community . Recently I have completed my certification in Databricks starting from Data Analyst , Data Engineering and Gen AI. Looking forward to get connected in serving the Data and AI...
- 4443 Views
- 1 replies
- 3 kudos
- 3 kudos
Happy to see you here .
- 3 kudos
- 3064 Views
- 0 replies
- 1 kudos
Free Databricks Professional Data Engineer Practice Tests
Hi All,I came across a very good set of Practice tests on Databricks Professional Data Engineer Certification.For time being It is being given for free by instructor as promotional activity . Enroll if you are planning to go for the certificationhttp...
- 3064 Views
- 0 replies
- 1 kudos
- 2456 Views
- 0 replies
- 1 kudos
How to deal with Slow Jobs?
Definitely configure job timeouts, and configure notifications. This will help you to identify slowness due to various factors. It is crucial to also investigate and fix the issue causing the slowness. The first step is to identify the problem. This ...
- 2456 Views
- 0 replies
- 1 kudos
- 2154 Views
- 0 replies
- 0 kudos
Monitoring a Streaming Job
If you have a streaming job, you need to check the batch metrics to be able to understand the stream progress. However, here are some other suggestions which we can use to monitor a streaming job and be stuck in a "hung" state. Streaming Listeners sp...
- 2154 Views
- 0 replies
- 0 kudos
- 2861 Views
- 0 replies
- 0 kudos
Why configure a job timeout?
If you use Databricks Jobs for your workloads, it is possible you might have run into a situation where you find your jobs to be in "hung" state. Before cancelling the job it is important to collect the thread dump as I described here to be able to f...
- 2861 Views
- 0 replies
- 0 kudos
- 2825 Views
- 1 replies
- 0 kudos
A handy tool called spark-column-analyser
I just wanted to share a tool I built called spark-column-analyzer. It's a Python package that helps you dig into your Spark DataFrames with ease.Ever spend ages figuring out what's going on in your columns? Like, how many null values are there, or h...
- 2825 Views
- 1 replies
- 0 kudos
- 0 kudos
An example added to README in GitHubDoing analysis for column PostcodeJson formatted output{"Postcode": {"exists": true,"num_rows": 93348,"data_type": "string","null_count": 21921,"null_percentage": 23.48,"distinct_count": 38726,"distinct_percentage"...
- 0 kudos
| User | Count |
|---|---|
| 85 | |
| 75 | |
| 72 | |
| 60 | |
| 43 |
