- 9820 Views
- 5 replies
- 0 kudos
Resolved! Can we expose REST API using Databricks python notebook?
We are trying to expose a REST API endpoint using python notebook. Does databricks allow this?
- 9820 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi @Muniyappan Mani​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...
- 0 kudos
- 2738 Views
- 3 replies
- 2 kudos
I attended "The Best Data Warehouse Is a Lakehouse" on September 20 but have not recieved my voucher although I recieved a mail which mentio...
I attended "The Best Data Warehouse Is a Lakehouse" on September 20 but have not recieved my voucher although I recieved a mail which mentioned below details.@Kaniz Fatma​ Please help me on same.Mail Id: bose.innovator8@gmail.comPlease complete the ...
- 2738 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @Arindam Bose​ That's great to hear! Please mark the answer as best.Thanks and Regards
- 2 kudos
- 5429 Views
- 3 replies
- 6 kudos
Resolved! Photon Acceleration not getting enabled for ML runtime in Azure
I tried to enable the photon acceleration in ML runtime 9.1 LTS ML (Scala 2.12,Spark 3.1.2) but getting error "selected runtime version does not support photon".I tried for other versions of ML runtime with single and multinode , access mode being s...
- 5429 Views
- 3 replies
- 6 kudos
- 6 kudos
Hi @Sajid Thavalengal Rahiman​ Does @Kaniz Fatma​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 6 kudos
- 4886 Views
- 6 replies
- 1 kudos
Delta Live Tables @expect compare tables count between two stages
Hello,I'm wondering if there is an option to make an expectation on DLT that will compare the number of records between two stages and e.g. fail if there is a difference between those counts?I mean something like this:@dlt.table()def bronze(): Some...
- 4886 Views
- 6 replies
- 1 kudos
- 1 kudos
Hi @Jacek Dembowiak​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...
- 1 kudos
- 2238 Views
- 3 replies
- 1 kudos
I have cleared Databricks Foundational Accreditation today and received the completion certificate. But I didnt receive the badge for the same. By whe...
I have cleared Databricks Foundational Accreditation today and received the completion certificate. But I didnt receive the badge for the same. By when I will receive the badge ?
- 2238 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @Sakshi Puchnanda​ Does @Hubert Dudek​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 1 kudos
- 2248 Views
- 3 replies
- 1 kudos
Incorrect results with df.query()
I have tried pulling a single row from a .csv using df.query()However, the data being returned doesn't coincide with the data I'm expecting - it is pulling a different row. Here is my code:df = spark.read.option("header",True).csv(data_fldr + "config...
- 2248 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @Peter Ott​ Does @Hubert Dudek​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 1 kudos
- 3129 Views
- 2 replies
- 1 kudos
Multiple DLT Pipelines Sharing a Single Cluster
When you use Workflows to orchestrate standard notebooks, they can share a single cluster. It will be awesome if we can achieve the same for DLT pipelines orchestrated in a Workflows Job.I understand DLT pipelines utilising their own special clusters...
- 3129 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Bo Zhu​ Does @Hubert Dudek​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 1 kudos
- 2612 Views
- 2 replies
- 0 kudos
Databricks-jdbc and vulnerabilities CVE-2022-42004, CVE-2022-42003
The latest version of Databricks-jdbc available through Maven (2.6.29) now has these two vulnerabilities:https://nvd.nist.gov/vuln/detail/CVE-2022-42004https://nvd.nist.gov/vuln/detail/CVE-2022-42003All due to depending on and including in the jar th...
- 2612 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Lars Joreteg​ Does @Hubert Dudek​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 0 kudos
- 3408 Views
- 3 replies
- 1 kudos
Pipeline in Community edition
Hi there,I'm learning Databricks using the community edition. I noticed I don't have way to practice the pipeline in community edition. (The icon below compute). Says I need to upgrade. Is there any way to practice pipeline and follow learning lesson...
- 3408 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @AJ DJ​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 1 kudos
- 13065 Views
- 4 replies
- 5 kudos
SQL CTE in Databricks or something similar?
%sqlWITH genCTE AS (SELECT MAX(PredID) + 1 AS PredID, 145 AS SystemIDFROM TableAUNION ALLSELECT PredID + 1FROM genCTE)SELECT * FROM genCTEWhen I attempt this, I get an error that genCTE does not exists.There may be a better way to what I am trying to...
- 13065 Views
- 4 replies
- 5 kudos
- 5 kudos
Hi @Greg Bradley​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...
- 5 kudos
- 14287 Views
- 7 replies
- 3 kudos
uuid in Merge
Hi guys,I'm trying to use uuid in the merge but I always get an error...import uuid ( df_events.alias("events").merge( source = df_updates.alias("updates"), condition = "events.cod = updates.cod and events.num = updates.num" ).whenMatch...
- 14287 Views
- 7 replies
- 3 kudos
- 3 kudos
Hi @William Scardua​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...
- 3 kudos
- 5512 Views
- 5 replies
- 3 kudos
New Visualization Tools
How do I add new visualization tool option to my databricks? I don't see a plus sign that will let you choose "Visualization" in my display command results :(. Please help.
- 5512 Views
- 5 replies
- 3 kudos
- 3 kudos
Hi @Suky Muliadikara​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.T...
- 3 kudos
- 4158 Views
- 2 replies
- 3 kudos
Issue with identity key column in databricks?
For the identity key I've used both GENERATED ALWAYS AS IDENTITY(start with 1 increment by 1) andGENERATED BY DEFAULT AS IDENTITY(start with 1 increment by 1)but in both cases, if I'm running my script once then it is fine (identity key is working as...
- 4158 Views
- 2 replies
- 3 kudos
- 3 kudos
yes, by default option allow duplicated values per design.I will avoid this option and use only use GENERATED ALWAYS AS IDENTITY Using BY DEFAULT option is worse than not using it at all in BY Default option, If I forget to set starting value, the ID...
- 3 kudos
- 2164 Views
- 1 replies
- 4 kudos
DLT DataPlaneException
I created an Azure Databricks workspace with my Visual Studio Subsciption, so far everything has been working as expected although I have requested to increase CPU core limit once.I am now getting this "DataPlaneException" error in the DTL during "Wa...
- 2164 Views
- 1 replies
- 4 kudos
- 4 kudos
@Bo Zhu​ can we get more error log, looks quota limit exceeded. did you get a chance to check quota in azure portal and see if cores exists for config that you selected. try to select another cluster config and validate
- 4 kudos
- 3574 Views
- 1 replies
- 4 kudos
My DLT pipeline return ACL Verification Failed
Python Commanddf = spark.read.format('csv').option('sep', ';').option("recursiveFileLookup", "true").load('dbfs:/***/data_files/PREVISIONS/')Here is the content of the folder Each folder contain the following files: Full logorg.apache.spark.sql.stre...
- 3574 Views
- 1 replies
- 4 kudos
- 4 kudos
Yes some of the files I don't have the right to access (mistakenly) In this case, how do you think I can tell DTL to handle this exception and ignore the file, since I can read some files but not all?
- 4 kudos
| User | Count |
|---|---|
| 1637 | |
| 791 | |
| 522 | |
| 349 | |
| 287 |
