- 6672 Views
- 8 replies
- 3 kudos
Create workspace in Databricks deployed in Google Cloud using terraform
In the documentation https://registry.terraform.io/providers/databricks/databricks/latest/docs https://docs.gcp.databricks.com/dev-tools/terraform/index.html I could not find documentation on how to provision Databricks workspaces in GCP. Only cre...
- 6672 Views
- 8 replies
- 3 kudos
- 3 kudos
Hi @horatiu guja Does @Debayan Mukherjee response answer your question?If yes, would you be happy to mark it as best so that other members can find the solution more quickly? Else, we can help you with more details.
- 3 kudos
- 6462 Views
- 5 replies
- 1 kudos
DLT Pipeline failed to Start due to "The Execution Contained atleast one disallowed language
Hi , im trying to setup DLT pipeline ,its a basic pipeline for testing purpose im facing the issue while starting the pipeline , any help is appreciated Code :@dlt.table(name="dlt_bronze_cisco_hardware")def dlt_cisco_networking_bronze_hardware(): ret...
- 6462 Views
- 5 replies
- 1 kudos
- 1 kudos
Hi @Arumugam Ramachandran seems like you have a spark config set on your DLT job cluster that allows only python and SQL code. Check the spark config (cluster policy).In any case, the python code should work. Verify the notebook's default language, ...
- 1 kudos
- 6822 Views
- 4 replies
- 10 kudos
Resolved! Date field getting changed when reading from excel file to dataframe
The date field is getting changed while reading data from source .xls file to the dataframe. In the source xl file all columns are strings but i am not sure why date column alone behaves differentlyIn Source file date is 1/24/2022.In dataframe it is ...
- 6822 Views
- 4 replies
- 10 kudos
- 10 kudos
Hi Team, @Merca Ovnerud I am also facing same issue , below is the code snippet which I am using df=spark.read.format("com.crealytics.spark.excel").option("header","true").load("/mnt/dataplatform/Tenant_PK/Results.xlsx")I have a couple of date colum...
- 10 kudos
- 5944 Views
- 2 replies
- 0 kudos
Cluster Modes
Given that there are three different kinda of cluster modes, when is it appropriate to use each one?
- 5944 Views
- 2 replies
- 0 kudos
- 0 kudos
Standard clustersA Standard cluster is recommended for a single user. Standard clusters can run workloads developed in any language: Python, SQL, R, and Scala.High Concurrency clustersA High Concurrency cluster is a managed cloud resource. The key be...
- 0 kudos
- 9335 Views
- 1 replies
- 1 kudos
I am new to Databricks and SQL. My CASE statement is not working and I cannot figure out why. Below is my code and the error message I'm receiving. Grateful for any and all suggestions. I'm trying to put yrs_to_mat into buckets.
SELECT *, yrs_to_mat, CASE WHEN < 3 THEN "under3" WHEN => 3 AND < 5 THEN "3to5" WHEN => 5 AND < 10 THEN "5to10" WHEN => 10 AND < 15 THEN "10to15" WHEN => 15 THEN "over15" ELSE null END AS maturity_bucket FROM mat...
- 9335 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Anne-Marie Wood ,I think it's more SQL general issue:you are not comparing any value to `< 3`it should be something like :WHEN X < 3 THEN "under3" SELECT *, yrs_to_mat, CASE WHEN X < 3 THEN "under3" WHEN X => 3 AND <...
- 1 kudos
- 5949 Views
- 5 replies
- 4 kudos
Resolved! Mount Azure Blob Storage with Cluster access control
Hello.I want to mount and share for the one group the container from Azure Blob Storage (It could be simple blob storage or Azure Data Lake Storage gen 2). But I am not able to do it because I am using Cluster with Table Access Control.This is my cod...
- 5949 Views
- 5 replies
- 4 kudos
- 4 kudos
I have a good solution to the problem:I am using Python library.There are some documentation.Topic to be closed.Best regards,Łukasz
- 4 kudos
- 798 Views
- 0 replies
- 0 kudos
Rs 485 IoT Gateway
RS-485 IoT Gateway is used for transmitting data from one device to another usually far away by using a wired LAN or a Wi-Fi. HashStudioz Technologies Inc. provides Smart IoT Gateway Solutions for Businesses like Pharma industries Etc. Our IoT Gatewa...
- 798 Views
- 0 replies
- 0 kudos
- 2871 Views
- 3 replies
- 3 kudos
How do you access a streaming live table's snapshots?
I have read that delta live tables will keep a history of 7 days. However after creating a streaming live table and using the dlt.apply_changes function. With this codedef run_pipeline(table_name,keys,sequence_by): lower_table_name = table_name.l...
- 2871 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @Logan Nicol Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks...
- 3 kudos
- 10260 Views
- 5 replies
- 0 kudos
Resolved! Can we expose REST API using Databricks python notebook?
We are trying to expose a REST API endpoint using python notebook. Does databricks allow this?
- 10260 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi @Muniyappan Mani Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...
- 0 kudos
- 2877 Views
- 3 replies
- 2 kudos
I attended "The Best Data Warehouse Is a Lakehouse" on September 20 but have not recieved my voucher although I recieved a mail which mentio...
I attended "The Best Data Warehouse Is a Lakehouse" on September 20 but have not recieved my voucher although I recieved a mail which mentioned below details.@Kaniz Fatma Please help me on same.Mail Id: bose.innovator8@gmail.comPlease complete the ...
- 2877 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @Arindam Bose That's great to hear! Please mark the answer as best.Thanks and Regards
- 2 kudos
- 5602 Views
- 3 replies
- 6 kudos
Resolved! Photon Acceleration not getting enabled for ML runtime in Azure
I tried to enable the photon acceleration in ML runtime 9.1 LTS ML (Scala 2.12,Spark 3.1.2) but getting error "selected runtime version does not support photon".I tried for other versions of ML runtime with single and multinode , access mode being s...
- 5602 Views
- 3 replies
- 6 kudos
- 6 kudos
Hi @Sajid Thavalengal Rahiman Does @Kaniz Fatma response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 6 kudos
- 5258 Views
- 6 replies
- 1 kudos
Delta Live Tables @expect compare tables count between two stages
Hello,I'm wondering if there is an option to make an expectation on DLT that will compare the number of records between two stages and e.g. fail if there is a difference between those counts?I mean something like this:@dlt.table()def bronze(): Some...
- 5258 Views
- 6 replies
- 1 kudos
- 1 kudos
Hi @Jacek Dembowiak Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...
- 1 kudos
- 2341 Views
- 3 replies
- 1 kudos
I have cleared Databricks Foundational Accreditation today and received the completion certificate. But I didnt receive the badge for the same. By whe...
I have cleared Databricks Foundational Accreditation today and received the completion certificate. But I didnt receive the badge for the same. By when I will receive the badge ?
- 2341 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @Sakshi Puchnanda Does @Hubert Dudek response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 1 kudos
- 2319 Views
- 3 replies
- 1 kudos
Incorrect results with df.query()
I have tried pulling a single row from a .csv using df.query()However, the data being returned doesn't coincide with the data I'm expecting - it is pulling a different row. Here is my code:df = spark.read.option("header",True).csv(data_fldr + "config...
- 2319 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @Peter Ott Does @Hubert Dudek response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 1 kudos
- 3218 Views
- 2 replies
- 1 kudos
Multiple DLT Pipelines Sharing a Single Cluster
When you use Workflows to orchestrate standard notebooks, they can share a single cluster. It will be awesome if we can achieve the same for DLT pipelines orchestrated in a Workflows Job.I understand DLT pipelines utilising their own special clusters...
- 3218 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Bo Zhu Does @Hubert Dudek response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 1 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 554 | |
| 349 | |
| 287 |
