Date & Time:5 March (Wednesday), 11 AM – 3 PM SGT What You'll Learn:In this training session, you will learn how to self-serve business insights from Databricks Data Intelligence Platform using AI/BI. After a tour of the fundamental components of t...
- 101 Views
- 0 replies
- 0 kudos
Sunday
Today we are announcing a deep partnership with SAP which we think can be game changing for our industry. In short, it is the marriage between the most important business data for enterprises globally (SAP data) and the best data platform in the mark...
- 1112 Views
- 0 replies
- 5 kudos
Thursday
We’re thrilled to announce the General Availability (GA) of Databricks Clean Rooms on AWS and Azure, a significant step forward in enabling secure, privacy-centric data collaboration. Following the Public Preview launch in August, we've engaged close...
- 604 Views
- 0 replies
- 1 kudos
2 weeks ago
Databricks welcomes BladeBridge, a proven provider of AI-powered migration solutions for enterprise data warehouses. Together, Databricks and BladeBridge will help enterprises accelerate the work required to migrate legacy data warehouses like Oracle...
- 568 Views
- 0 replies
- 1 kudos
2 weeks ago
In the rapidly evolving landscape of data engineering and analytics, speed, scalability, and simplicity are invaluable. Serverless compute addresses these needs by eliminating the complexity of managing infrastructure, allowing you to focus on buildi...
- 686 Views
- 0 replies
- 0 kudos
2 weeks ago
Reduce the cost of sharing across platforms Gartner predicts that CDOs who have successfully executed data sharing initiatives are 1.7 times more effective in showing business value and ROI from their data analytics strategy. Databricks provides an ...
- 786 Views
- 0 replies
- 3 kudos
3 weeks ago
Community Activity
- 5783 Views
- 10 replies
- 3 kudos
Resolved! to_date not functioning as expected after introduction of arbitrary replaceWhere in Databricks 9.1 LTS
I am trying to do a dynamic partition overwrite on delta table using replaceWhere option. This was working fine until I upgraded the DB runtime to 9.1 LTS from 8.3.x. I am concatenating 'year', 'month' and 'day' columns and then using to_date functio...
- 5783 Views
- 10 replies
- 3 kudos
Latest Reply
- 3 kudos
SELECT TO_DATE('20250217','YYYYMMDD'); gives the error: PARSE_SYNTAX_ERROR syntax error at or near 'select'. sqlstate: 42601. It datagrip, it works no problem and displays the date.
- 3 kudos
- 1 Views
- 0 replies
- 0 kudos
creating Workspace in AWS with Quickstart is giving error
Hello, While creating workspace in AWS using Quickstart, I get below error. I used both admin Account and root account to create this but both gave the same issue. Any help is appreciated. The resource CopyZipsFunction is in a CREATE_FAILED stateT...
- 1 Views
- 0 replies
- 0 kudos
- 6 Views
- 0 replies
- 0 kudos
Update target column with optional source columns
I am trying to load on prem transaction log table and update a databricks table.In Source table,the column reqdetails table hold all the information and expect ProductID remaining columns are dynamic(not all columns exists in the request) all time th...
- 6 Views
- 0 replies
- 0 kudos
- 14 Views
- 0 replies
- 0 kudos
Suspended Exam Databricks Certified Data Engineer Associate
Hi Team,My exam Databricks Certified Data Engineer Associate got suspended today and I have not received any email yet for the reason. As per me, I followed all the instructions as mentioned and I have given my exam with all honesty. I took off from ...
- 14 Views
- 0 replies
- 0 kudos
- 5810 Views
- 4 replies
- 1 kudos
Permission denied: Lightning Logs
I'm doing parameter tuning for a NeuralProphet model (you can see in the image the parameters and code for training)When I try to parallelize the training, it gives me Permission Error.Why can't I access the folder '/databricks/spark/work/*'? Do I ne...
- 5810 Views
- 4 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi Ruben!I am facing exactly the same error running a similar approach when using runtime 16.2 ML. I didn't have this issue when using runtime 12.2 LTS ML or 13.3 ML. Did you find a solution?Many thanks!
- 1 kudos
- 165 Views
- 2 replies
- 0 kudos
Efficacy of PySpark in Databricks
Hi all,- migrating from an on-premise Oracle -Currently on Oracle, I have a "library" of let's say 300 tables to load, sequentially, based on views (some tables being fed potentially by several views, therefore the number of underlying views is highe...
- 165 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Should be no problem, but the pitfall you want to avoid is treating Databricks like a traditional database and trying to do everything in SparkSQL. Yes, SparkSQL will work but Python or DB-SQL might be better choices since they are higher level and ...
- 0 kudos
- 2410 Views
- 4 replies
- 3 kudos
Failure when deploying a custom serving endpoint LLLM
I'm currently experimenting with vector search using Databricks. Everything runs smoothly when I load the model deployed in Unity Catalog into a notebook session and ask questions using Python. However, when I attempt to serve it, I encounter a gener...
- 2410 Views
- 4 replies
- 3 kudos
Latest Reply
- 3 kudos
Deploying custom serving endpoints can sometimes be challenging, especially when working with advanced models like LLMs. For Roblox enthusiasts, platforms like DTI Codes simplify creativity by offering free resources to enhance the All DTI codes. Sim...
- 3 kudos
- 29639 Views
- 97 replies
- 8 kudos
Suspended Exam? Don't worry! Our support team can help!
Taking exams is so stressful already, and if your exam gets suspended it is extra stressful. Don't worry! Our support team is awesome and will help you get back on track. The most important step is to file a ticket in our Help Center. Be sure to inc...
- 29639 Views
- 97 replies
- 8 kudos
Latest Reply
- 8 kudos
Hi Team,My exam Databricks Certified Data Engineer Associate got suspended today and I have not received any email yet for the reason. As per me, I followed all the instructions as mentioned and I have given my exam with all honesty. I took off from ...
- 8 kudos
- 8393 Views
- 2 replies
- 0 kudos
Resolved! Unable to add column comment on a View. Any way to update comments on multiple columns in bulk?
I noticed that unlike "Alter Table" there is no "Alter View" command to add comment on a column in the existing view. This is a regular view created on Tables (and not Materialized view). If the underlying table column has comment then the View inh...
- 8393 Views
- 2 replies
- 0 kudos
- 227 Views
- 1 replies
- 0 kudos
Would You Join Us for a Local Meetup?
If we organized a local meetup in your area, would you attend? This is your chance to connect with fellow data enthusiasts, share knowledge, and build a stronger community face-to-face. Let us know if you'd be interested in joining, and where you'd ...
- 227 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Is this still available? Would love to see this happen in the Omaha area.
- 0 kudos
- 16 Views
- 0 replies
- 0 kudos
AI BI Genie throwing internal error
For any prompt I am getting INTERNAL_ERROR: AI service did not respond with a valid answer
- 16 Views
- 0 replies
- 0 kudos
- 598 Views
- 3 replies
- 1 kudos
Connection reset by peer logging when importing custom package
Hi! I'm trying to import a custom package I published to Azure Artifacts, but I keep seeing the INFO logging below, which I don't want to display. The package was installed correctly on the cluster, and it imports successfully, but the log still appe...
- 598 Views
- 3 replies
- 1 kudos
Latest Reply
- 1 kudos
Thanks for the suggestions. I investigated all of the above, but they didn't provide a solution. What did work was using another logging package within my custom package: Loguru. Not sure why this helped?
- 1 kudos
- 36 Views
- 2 replies
- 0 kudos
External connection to Azure ADLS Gen2 storage
Hello, I have a problem trying to make an external connection to a blob storage configured as ADLS Gen2 with hierarchical namespace (HNS) enabled. I have setuup the storage account with a container wirh HNS enabled as in the image attached:Next I hav...
- 36 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi, would it work if you try to remove "landing" at the end? Seems the error is complaining that the external location should be a directory.
- 0 kudos
- 39 Views
- 1 replies
- 0 kudos
Resolved! Hostname not resolving using Spark JDBC
Hey guys,Ive ran into a weird error this morning. Last week I was testing a new Oracle Connector and it ran through smooth the whole last week!This morning at 7 it ran again and following it was showing a "SQLRecoverableException: IO Error: Unknown h...
- 39 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
I have even cloned the cluster and it worked on the new one. But after turning the cluster off over night, it started working again the next morning. This is really weird.
- 0 kudos
- 128 Views
- 5 replies
- 0 kudos
Serveless compute does need has cloud accout(AWS、Google 、Azure)
I am a Databricks beginner, and I would like to ask if the Compute created in the Databricks account , it means also exists in the cloud account (e.g., AWS)? If the AWS account is deactivated, the existing compute will not be usable. This is what I h...
- 128 Views
- 5 replies
- 0 kudos
Latest Reply
- 0 kudos
@FanMichelleTW No, Databricks recommends using serverless compute, and you can use serverless compute as well.To do so, open a notebook and check the top-right corner to see if a serverless compute option is in a Ready state. If it is, simply select ...
- 0 kudos
Announcements
| User | Count |
|---|---|
| 106 | |
| 43 | |
| 24 | |
| 14 | |
| 13 |
Latest from our Blog
How to Read Unity Catalog Tables from Trino via Open APIs
Databricks Unity Catalog (UC) is the industry’s only unified and open governance solution for data and AI, built into the Databricks Data Intelligence Platform. Unity Catalog provides a single source ...
Ray Monitoring Made Easy: Prometheus & Grafana with Ray on Databricks Clusters
Intro Ray is rapidly becoming the standard for logic-parallel computing, enabling many Databricks customers to accelerate a wide range of Python workloads. Since its general availability on Databricks...
[CUSTOMER BLOG] Efficient Distributed Energy Load Forecasting with Databricks at EDP E-REDES
Traditionally, power grids have been sized with large safety margins to consider low-probability events. Thus, there can be limits imposed on generation even when the low-probability events do not occ...
Improved Duplicate Match Handling in Delta MERGE Operations
In DBR 16.1+, we’ve improved functionality of MERGE operations where multiple rows of the source dataset match the same row of the target Delta table, but only one row matches the WHEN MATCHED conditi...
