Use Case Sharing Sweepstakes ! Data + AI Summit is in full swing and we know you are just as excited as we are to learn about the new and exciting th...

[Anonymous]

Use Case Sharing Sweepstakes !

Data + AI Summit is in full swing and we know you are just as excited as we are to learn about the new and exciting things happening at Databricks. From notebooks to the Lakehouse, we know some of these new features will be game-changing for a lot of you and we want to hear about it!

In this thread we will be collecting the use cases you want to share about the new product releases and/or features you are most excited to use. The Community Member that shares the most use cases below will win an iPad! 

Contest dates: Tuesday - Fri | Jun 28 - July 17 

Use this template in your reply 

Copy the following template for you contest entry:

Name of the new feature (give examples here)

Describe your environment (optional, but helpful)

Share 3-9 sentences describing your use case and how this new feature will help provide a new solution

• What does this new feature helps you achieve that was either not possible before or required massive efforts
• How can this feature affect your team productivity
• What business goal or use-case is achievable relatively easily using the feature

How to win

Three participants will be chosen at random (per the Sweepstakes rules) as our Grand Prize winners and will receive an iPad Air. 🎉

Click here for official contest rules

[AmanSehgal]

Serverless SQL Warehouses

This is really great work by Databricks. 10 seconds to launch a cluster that can start executing queries right away. It'll help our analytics team to start crunching data for any use case or report they want to generate. It's faster, it's quicker and accessing data is much more easier now.

[AmanSehgal]

Unity Catalog

Another great tool from Databricks. I'd been waiting for this to be launched since the announcement.

Unity has helped in performing data discovery and providing access control with a UI. It helps admin to monitor data access by different users - which in turn serves as audit report as well

[AmanSehgal]

Interested to see Folder structure for Databricks SQL

The way notebooks are managed per account basis within folders, it will be really beneficial to have DB SQLs organized within folders and not be presented in a flat hierarchy.

This will help in searching, organizing and maintaining DB SQL notebooks. Currently, the only way to search a notebook is via tags and that's not very helpful

[AmanSehgal]

SQL REST API for Python, Go clients

Definitely going to assist well in integrating delta lake with web applications used for analytics. If integrated with SQL serverless, then analytics applications can directly run over data lake using REST APIs. Moreover, now Go and nodejs developers can onboard on databricks platform.

[AmanSehgal]

Delta Sharing

There have been several instances, where clients want to share data within departments or to external parties - and usually this is done via CSV files, JSON or XML files. It takes a lot of effort to clean data, validate it, match the schema and then load into target database.

With delta sharing this is going to be lot more easier and streamlined especially now with support for CDF and compatibility with node, powerBi, Go and Airflow.

[AmanSehgal]

Databricks Marketplace

In recent past, I've seen Data as a Service coming up in data-heavy companies.

Clients want to generate revenue within department by selling data and associated services.

Marketplace can enable these DaaS products.

Eg: data engineers can share and sell notebooks and dashboards that directly plugin to data lake and provide data quality analytics.

[AmanSehgal]

ReDash Dashboard

Interested to see if and when reDash dashboards will include more visualizations and features in future. reDash currently is good only for certain use cases. If more visualizations can be brought on board, then it'll be more useful.

Features like, report export with custom text and mail content

Including different fonts and sizes within dashboard

Customizing dimensions of the dashboards

[AmanSehgal]

ReDash Widget iFrame

We have some use cases where clients would like to embed ReDash widgets in to their website via iFrame. This isn't currently available and it would be really good if iFrames can be enabled for widgets.

[AmanSehgal]

Serverless Cluster for Data Engineering

Like Serverless Warehouses, it would be great if there are Serverless clusters within Data Engineering workspace. This will make ad-hoc workflows run quicker and assist in managing cost of operating cluster resources. Also connectivity with SIMBA ODBC driver will improve and the clients will be able to get quicker results

[AmanSehgal]

Local Image of Databricks

Not sure how this will be made possible. For education purpose, if we can run databricks with a cluster of VMs on local machine and say access notebook from localhost and store data on local disk.

This will help students to start learning databricks without having a cloud account and they can work on their own local machine and do PoCs - just like Jupyter notebooks

[AmanSehgal]

Bitbucket/Git for DB SQL

Currently Bitbucket is only supported for notebooks but not for DB SQL. Having a version control connectivity with DB SQL will help in maintaining SQL queries and versioning them.

As of now, the SQL queries have to be exported out manually or via script that use REST APIs.

Looking forward for Bitbucket connectivity in DB SQL.

[AmanSehgal]

Cloning libraries when cloning clusters

Currently when we clone clusters, the externally added libraries aren't copied as part of cloning process.

It's an expected behavior but a missing one. At times new developers end up spending lot of time in debugging the issue due to this only to realize that libraries haven't been copied