- 4402 Views
- 5 replies
- 1 kudos
Writing append blob files to unity catalog volum
The workspace is assigned to unity catalog, and all the access to the ADLS Gen2 is now handled via unity catalog only, means no SPN, no connection string, access keys etc. I have to create append blob files in a volume, Is this is possible in a works...
- 4402 Views
- 5 replies
- 1 kudos
- 1 kudos
Now I got your point. No, you can't create Append Blob files directly in Volumes, as this is a native Azure functionality. A volume is basically just an abstraction over a native storage.You will still need to use libraries like azure-storage-blob wi...
- 1 kudos
- 4583 Views
- 2 replies
- 0 kudos
Resolved! Standardized Framework to update Databricks job definition using CI/CD
Hi Databricks support, I am looking for a standardized Databricks framework to update job definition using DevOps from non-production till it get productionized. Our current process of updating the Databricks job definition is as follows:In our sourc...
- 4583 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi from the Git folders/Repos PM: DAB is the way to go, and we are working on an integration to author DABs directly in the workspace. Here's a DAIS talk where the DAB PM and I demo'ed some recommendations for source controlling jobs: https://www.da...
- 0 kudos
- 13195 Views
- 3 replies
- 5 kudos
Build & Refresh a Calendar Dates Table
IntroductionMaintaining accurate and up-to-date calendar date tables is crucial for reliable reporting, yet manual updates can be time-consuming and prone to error. This fundamental component serves as the backbone for date-based analysis, enabling a...
- 13195 Views
- 3 replies
- 5 kudos
- 9746 Views
- 3 replies
- 1 kudos
Reading Excel files folder
Dears,One of the tasks needed by DE is to ingest data from files, for example, Excel file.Thanks for OnerFusion-AI for the below thread that give us the steps of reading from one file https://community.databricks.com/t5/get-started-discussions/how-to...
- 9746 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @AhmedAlnaqa ,Can we read from ADLS location too by using abfss ?Thanks
- 1 kudos
- 1840 Views
- 1 replies
- 3 kudos
Notification destination API
You can now create, delete, and update notification destinations through the Databricks API.The notification destinations API lets you programmatically manage a workspace's notification destinations. Notification destinations are used to send notific...
- 1840 Views
- 1 replies
- 3 kudos
- 3 kudos
Hi,Thank you for sharing this Ajay. We appreciate you keeping the community informed!Thanks,Anushree
- 3 kudos
- 3485 Views
- 1 replies
- 2 kudos
The Importance of Databricks in Today's Data Market
Unlocking the Power of Data with Databricks In the rapidly evolving landscape of data and analytics, Databricks has emerged as a transformative force, reshaping how organizations handle big data, data engineering, and machine learning. As we naviga...
- 3485 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @Rishabh-Pandey,Thank you for sharing this! Databricks is really making a difference with its unified platform and powerful features. Excited to see how it will continue to shape the future of data! Thanks,
- 2 kudos
- 7047 Views
- 1 replies
- 1 kudos
Databricks on Databricks: Kicking off the Journey to Governance with Unity Catalog
In the world of data engineering and analytics, governance is paramount. Databricks has taken a significant step forward in this arena with the introduction of Unity Catalog, a unified governance solution for all data and AI assets. This journey to e...
- 7047 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi Rishabh ,Unity Catalog is a major advancement in data governance, enhancing how we manage and secure data and AI assets in Databricks. Exciting to see these improvements!Thanks,Anushree
- 1 kudos
- 2137 Views
- 1 replies
- 2 kudos
How to detect the Risks in Claims Data Using Databricks and PySpark
As a data engineer with experience in Databricks and other data engineering tools, I know that processing claims data and detecting risks early can really help in insurance claims processing. In this article, I’ll show you how to use Databricks and P...
- 2137 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi, Thanks for sharing this! Kudos for breaking it down so clearly. I’m sure, it will help other community members. Thanks,Anushree
- 2 kudos
- 1309 Views
- 0 replies
- 2 kudos
September 2024 - Databricks SQL fixes
Added customization options for number formats in dashboard widgets, including currencies and percentages.The width of the Values column in pivot tables now auto-adjusts based on the width of the cell value names.The time displayed in dashboard widge...
- 1309 Views
- 0 replies
- 2 kudos
- 3039 Views
- 1 replies
- 0 kudos
SQL code for appending a notebook result into an existing database table
I am attempting to append the results from a notebook query results table into an existing databricks database table. By chance would someone share an example of the sql code with me?
- 3039 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @wheersink ,So let's say you created following table with some sample values.%sql CREATE TABLE dev.default.employee ( id INT, name STRING, age INT, department STRING ); INSERT INTO dev.default.employee VALUES (1, 'John Doe', 30, 'Financ...
- 0 kudos
- 2954 Views
- 1 replies
- 2 kudos
🚀 Boost Your Data Pipelines with Dynamic, Data-Driven Databricks Workflows (For Each Task)! 💡
Unlock the power of the For Each task in Databricks to seamlessly iterate over collections—whether it's a list of table names or any value—and dynamically run tasks with specific parameter values. This powerful feature lets you automate repetitive pr...
- 2954 Views
- 1 replies
- 2 kudos
- 2 kudos
Thanks for sharing @Ajay-Pandey
- 2 kudos
- 1378 Views
- 0 replies
- 2 kudos
Databricks August Update
August 2024 | Databricks on AWS
- 1378 Views
- 0 replies
- 2 kudos
- 1644 Views
- 0 replies
- 2 kudos
Code patterns / best practices in SparkSQL, pyspark and Scala
Hi all,I am looking for collections of Code patterns for:SparkSQLpySparkScalaI'm sure there are at least a few repos in Github with snippets and will share them as I find them in this thread. If you come across any good collections, please post your...
- 1644 Views
- 0 replies
- 2 kudos
- 1321 Views
- 0 replies
- 2 kudos
SHOW CREATE TABLE for materialized views and streaming tables
Added support for SHOW CREATE TABLE for materialized views and streaming tables. This will show the complete CREATE command used at creation time, including properties and schedulesCatalog Explorer "Overview" tab now shows the full CREATE command for...
- 1321 Views
- 0 replies
- 2 kudos
- 3851 Views
- 0 replies
- 2 kudos
LakeFlow Connect
LakeFlow Connect introduces simple ingestion connectors for databases, enterprise applications, and file sources. Set up efficient, low-maintenance pipelines in just a few clicks or via an API. Current sources include Salesforce, Workday, and SQL Ser...
- 3851 Views
- 0 replies
- 2 kudos
| User | Count |
|---|---|
| 85 | |
| 75 | |
| 72 | |
| 60 | |
| 42 |
