- 685 Views
- 1 replies
- 0 kudos
Can I connect Databricks directly to Salesforce CRM for live data access?
Hi everyone,I'm currently working on integrating Databricks with Salesforce CRM. I want to know if it's possible to connect Databricks directly to Salesforce CRM to access live (real-time or near real-time) data—not just periodic batch exports.Specif...
- 685 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @syamsubrahmanya ,Yes, you can use Lakehouse Federation to query Salesforce data from DatabricksIntroducing Salesforce Connectors for Lakehouse Federation and LakeFlow Connect | Databricks BlogSet Up a Databricks Data Federation Connection | Data ...
- 0 kudos
- 1687 Views
- 2 replies
- 0 kudos
Resolved! Getting start with Databricks
Hi All, I wanted to learn Databricks as an ETL background. Please help me with the roadmap and some qualified certifications that matters most. Thank you.
- 1687 Views
- 2 replies
- 0 kudos
- 0 kudos
I'll recommend my tutorials:Introduction to Databricks: A Beginner’s Guidehttps://medium.com/@mariusz_kujawski/getting-started-with-databricks-a-beginners-guide-8b8db7f6f457Why I Liked Delta Live Tables in Databrickshttps://medium.com/@mariusz_kujaws...
- 0 kudos
- 2569 Views
- 3 replies
- 1 kudos
Databricks Apps with Pyodbc Microsoft SQL Driver
I'm building an app that interfaces with an Azure SQL Database. I need to use Entra auth with a service principal, which is why I'm using the Microsoft ODBC driver. However, this works fine on my local, but I can't figure out how to get the ODBC driv...
- 2569 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @bcodernet ,Each Databricks app can include dependencies for Python, Node.js, or both. You define these dependencies in language-specific files:Use a requirements.txt file to specify additional Python packages.Use a package.json file to specify No...
- 1 kudos
- 1789 Views
- 2 replies
- 0 kudos
Converting Existing Streaming Job to Delta Live Tables with Historical Backfill
Description:I’m migrating a two-stage streaming job into Delta Live Tables (DLT):Bronze: read from Pub/Sub → write to Bronze tableSilver: use create_auto_cdc_flow on Bronze → upsert into Silver tableNew data works perfectly, but I now need to backfil...
- 1789 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @yazz ,I’m wondering if you could use a similar approach to the one in the article below. So, just backfill your bronze table first. Then, the downstream silver and gold layers will pick up the new data from the bronze layer.In that approach you ...
- 0 kudos
- 1280 Views
- 3 replies
- 0 kudos
Enable automatic identity management in Azure Databricks
We have Databricks account admin access but not able to see the option from Databricks admin console to enable automatic identity management.Using the Previews page wanted to enable and fallowed below steps:1. As an account admin, log in to the accou...
- 1280 Views
- 3 replies
- 0 kudos
- 0 kudos
After raising Databrick ticket, today I am able to see the Automatic Identity Management public preview option
- 0 kudos
- 1785 Views
- 2 replies
- 0 kudos
Lakebase auto start/stop
It doesn’t appear that the Lakebase OLTP instances function like Sql warehouses in the following ways:• automatically starting when initiating a connection• automatically stopping after no usage in x minutesI am wondering if others have use cases for...
- 1785 Views
- 2 replies
- 0 kudos
- 0 kudos
I guess that the start/stop pattern is not something you want in a low-latency OLTP database.Perhaps they add it in the future.
- 0 kudos
- 1445 Views
- 1 replies
- 1 kudos
process mongo table to delta table databricks
Hello Guys,I have a table mongo which size is 67GB, I use streaming to ingest but is very slow to copying all data to Delta table. Someone have an answer to this? I use connector mongodb V10.5 this is my code pipeline_mongo_sec = [ { "$u...
- 1445 Views
- 1 replies
- 1 kudos
- 1 kudos
What if you do not update the delta table for each incoming microbatch but f.e. only do this every 15 min/hour/whatever.Like that you can keep on ingesting in a streaming way, but the actual update towards the delta table is more batch approached so ...
- 1 kudos
- 2585 Views
- 2 replies
- 2 kudos
Update Delta Table with Apache Spark connector
Hi everyone. I'd like to ask a question about updating Delta tables using the Apache Spark connector.Let's say I have two tables: one is a product dimension table with items from my shop, and the other contains a single column with the IDs of the pro...
- 2585 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @mr3 ,Yes, it’s perfectly fine to use a MERGE operation solely for updates. The UPDATE statement has many limitations. It doesn't support neither UPDATE FROM nor subqueries. This creates many limitations. There are situations where we would like t...
- 2 kudos
- 4154 Views
- 9 replies
- 2 kudos
Resolved! commit time is coming as null in autoloader
As per the databricks new feature in autoloader that we can use archival and move feature in autoloader however I am trying to use that feature using databricks 16.4.x.scala2.12 however commit time is still coming null as its mentioned in the documen...
- 4154 Views
- 9 replies
- 2 kudos
- 2 kudos
Hey @shrutikatyal I believe the only current route to get a discount voucher would be the following:https://community.databricks.com/t5/events/dais-2025-virtual-learning-festival-11-june-02-july-2025/ev-p/119323I think it’s the last day of the event ...
- 2 kudos
- 2644 Views
- 1 replies
- 0 kudos
Handling Merged Heading Rows When Converting Excel to CSV in Databricks
Hi all,I'm working on a process in Databricks to convert multiple Excel files to CSV format. These Excel files follow a similar structure but with some variations. Here's the situation:Each file contains two header rows:The first row contains merged ...
- 2644 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi MinuN,How are you doing today?That’s a great question, and you're definitely on the right path using BeautifulSoup to extract the table structure from .xls HTML-like files. To generate the repeated first row of main headings for the CSV, one pract...
- 0 kudos
- 1539 Views
- 1 replies
- 1 kudos
Need help on "You cannot enable Iceberg reads on materialized views and streaming tables"
Hi All, As we "cannot enable Iceberg reads on materialized views and streaming tables", Is there any option in private preview to enable Iceberg reads for materialized views and streaming tables. I tried using the option of DLT Sink API with table c...
- 1539 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi sridharplv,How are you doing today? As per my understanding, Databricks does not support Iceberg reads for materialized views and streaming tables, and there’s no official preview or timeline shared publicly for enabling this support. Your workaro...
- 1 kudos
- 1955 Views
- 0 replies
- 0 kudos
How to write ObjectId value using Spark connector 10.2.2
In pySpark mongo connector while updating records how to handle _id as objectId.spark 3.2.4scala2.13sparkMongoConnector 2.12-10.2.2
- 1955 Views
- 0 replies
- 0 kudos
- 6337 Views
- 2 replies
- 1 kudos
Resolved! Lookup dashboard ID in bundle variables
Hi all,I have an asset bundle that contains the following dashboard_task:resources: jobs: my_job: name: my_job_name tasks: - task_key: refresh_my_dashboard dashboard_task: dashboard_id: ${var.my_dashbo...
- 6337 Views
- 2 replies
- 1 kudos
- 1 kudos
Thanks! That does make sense. When I run `databricks lakeview list` I do get the dashboard I want:[ { "create_time": "2025-06-23T08:09:49.595Z", "dashboard_id": "id000000000000000000000", "display_name": "My_Dashboard_Name", "lifecy...
- 1 kudos
- 1300 Views
- 1 replies
- 0 kudos
Widget value not synchronized after detach/reattach
Hello Databricks Team,I hope you are doing well.I’m working with dbutils.widgets in a Databricks notebook using the Accessed Commands mode, and I have encountered some challenges.Specifically, after detaching and reattaching to the cluster:- the widg...
- 1300 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello there Can you please share the code used for widgets?also if you change mannually, is it working? (did it worked before?) Are you trying to load via some parent notebook?Waiting for your response.
- 0 kudos
- 1526 Views
- 1 replies
- 0 kudos
making REORG TABLE to enable Iceberg Uniform more efficient and faster
I am upgrading a large number of tables for Iceberg / Uniform compatibility by running REORG TABLE <tablename> APPLY (UPGRADE UNIFORM(ICEBERG_COMPAT_VERSION=2));and finding that some tables take several hours to upgrade - presumably because they are ...
- 1526 Views
- 1 replies
- 0 kudos
- 0 kudos
HI @JameDavi_51481 , Hope you tried this approach for enabling iceberg metadata along with delta format :ALTER TABLE internal_poc_iceberg.iceberg_poc.clickstream_gold_sink_dltSET TBLPROPERTIES ('delta.columnMapping.mode' = 'name','delta.enableIceberg...
- 0 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 484 | |
| 349 | |
| 287 |
