- 8108 Views
- 4 replies
- 5 kudos
Resolved! DLT pipeline is unable to process struct with hyphen in nested column name
Hello,I have some nested columns with hyphen i.e. sample-1 in struct column, recently DLT pipeline has started throwing synatx error. Before May 24, 2023, this was working fine.Is this a new bug in May 2023 release?After clearing table and table's da...
- 8108 Views
- 4 replies
- 5 kudos
- 5 kudos
Hi @Rishabh Tomar We haven't heard from you since the last response from @Kaniz Fatma . Kindly share the information with us, and in return, we will provide you with the necessary solution. Thanks and Regards
- 5 kudos
- 11155 Views
- 7 replies
- 5 kudos
Timestamp in databricks are getting converted to different timezone
Timestamp columns which are extracted from source databases using jdbc read are getting converted to different timezone and is not matching with source timestamp. Could anyone suggest how can we get same timestamp data like source data?
- 11155 Views
- 7 replies
- 5 kudos
- 5 kudos
Hi @Dinu Sukumara We haven't heard from you since the last response from @Werner Stinckens . Kindly share the information with us, and in return, we will provide you with the necessary solution.Thanks and Regards
- 5 kudos
- 5220 Views
- 4 replies
- 2 kudos
Dear Community,I want to understand from you all - How do you debug your codes when using Databricks? Have you tried the Variable Explorer of Databr...
Dear Community,I want to understand from you all - How do you debug your codes when using Databricks?Have you tried the Variable Explorer of Databricks? This allows the users to view at-a-glance all the variables defined in their notebooks, inspect...
- 5220 Views
- 4 replies
- 2 kudos
- 2 kudos
I just create code in notebooks that allow me to check outputs at different steps. These methods usually include print statements or .display() of dataframes. If youre working with lots of data the .show(truncate=100,vertical=True) may help you. I ha...
- 2 kudos
- 5698 Views
- 3 replies
- 2 kudos
- 5698 Views
- 3 replies
- 2 kudos
- 2 kudos
@Gerard Blackburns :Calculating the cost or billing for Azure SQL Server instances involves considering the Azure SQL Database Unit (DBU) pricing model. DBUs are the unit of measure for the consumption of Azure SQL Database resources. To calculate t...
- 2 kudos
- 6792 Views
- 3 replies
- 3 kudos
Setup for Unity Catalog, autoloader, three-level namespace, SCD2
I am trying to setup delta live tables pipelines to ingest data to bronze and silver tables. Bronze and Silver are separate schema. This will be triggered by a daily job. It appears to run fine when set as continuous, but fails when triggered.Table...
- 6792 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @Jennette Shepard Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answ...
- 3 kudos
- 10778 Views
- 2 replies
- 3 kudos
Missing Credential Scope - Unable to call databrick(Scala) notebook from ADF
Hi Team ,I am using job cluster while setting Linked Service in ADF to call Data bricks Notebook activity .Cluster Detail - Policy - UnrestrictedAccess Mode - Single userUnity Catalog Enabled.databrick run time - 12.2 LTS (includes Apache Spark 3.3.2...
- 10778 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @Akshay Patni We haven't heard from you since the last response from @Debayan Mukherjee . Kindly share the information with us, and in return, we will provide you with the necessary solution. Thanks and Regards
- 3 kudos
- 23658 Views
- 4 replies
- 7 kudos
Resolved! Want to load a high volume of CSV rows in the fastest way possible (in excess of 5 billion rows). I want the best approach, in terms of speed, for loading into the bronze table.
My source can only deliver CSV format (pipe delimited).My source has the ability to generate multiple CSV files and transfer them to a single upload folder.All rows must go to the same target bronze delta table.I do not care about the order in which ...
- 23658 Views
- 4 replies
- 7 kudos
- 7 kudos
Hi @Michael Popp Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers ...
- 7 kudos
- 17699 Views
- 5 replies
- 8 kudos
Resolved! Converting the mon-yy format to date, but showing null for output
I have a date column that comes with month-year format and I am trying to convert that into dd-mm-yyyy format in pyspark for example I have date column with value Jan-2019Feb-2020Mar-2020the output I am expecting is 01/01/201901/02/202001/03/2020here...
- 17699 Views
- 5 replies
- 8 kudos
- 8 kudos
Hi @vikram sinhha We haven't heard from you since the last response from @Suteja Kanuri . Kindly share the information with us, and in return, we will provide you with the necessary solution. Thanks and Regards
- 8 kudos
- 20293 Views
- 5 replies
- 6 kudos
Resolved! Databricks and Web app connectivity to build a interactive application
Hi All,Could you please guide and provide me some material on "Databricks and Web app connectivity to build a interactive application"Objective is to build web app so that user can interact for getting different data/visuals from databricks (pyspark ...
- 20293 Views
- 5 replies
- 6 kudos
- 6 kudos
Hi @ananthakrishna raikar We haven't heard from you since the last response from @Werner Stinckens . Kindly share the information with us, and in return, we will provide you with the necessary solution. Thanks and Regards
- 6 kudos
- 19054 Views
- 7 replies
- 11 kudos
Resolved! Unzip Files
Hi all, I am trying to unzip a file in databricks but facing an issue,Please help me if you have any doc or codes to share.
- 19054 Views
- 7 replies
- 11 kudos
- 11 kudos
Hey ajay,You can follow this module to unzip your zip file.To give your brief idea about this, it will unzip your file directly into your driver node storage.So If your compressed data is inside DBFS then you first have to move that to drive node and...
- 11 kudos
- 5936 Views
- 2 replies
- 4 kudos
Resolved! Azure DataBricks REST API 2.0 - Deactivate User by ID. I can't seem to find any more details on this anymore.
Hello team,I am looking to run Azure Logic Apps to deactivate user by ID. Right now there is only the REST API for delete user but I only want to deactivate the user.Can anyone help with the operations op/path/schema that I am missing? Thank you!
- 5936 Views
- 2 replies
- 4 kudos
- 4 kudos
Hi @Daren Liew Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers yo...
- 4 kudos
- 4544 Views
- 2 replies
- 1 kudos
Resolved! Decimal Automatic Rounding on sum
I am facing an issue, I have implemented a code in which I am performing sum on decimal values. I have set precision as Decimal(19,2). When I am calculating for big data I am getting different value as I got from my .Net Utility application. ex. From...
- 4544 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Pragya Sharma We haven't heard from you since the last response from @Werner Stinckens . Kindly share the information with us, and in return, we will provide you with the necessary solution. Thanks and Regards
- 1 kudos
- 12364 Views
- 3 replies
- 3 kudos
Resolved! Cluster SQL Warehouse fails to Start.
I'm a new user of Databricks and I'm taking the Academy course, but I'm having difficulty starting a SQL Warehouse cluster. I've tried several different configurations, but I always get the same error:Clusters are failing to launch. Cluster launch wi...
- 12364 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @Luiz Ribeiro Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers ...
- 3 kudos
- 8184 Views
- 4 replies
- 5 kudos
Resolved! Can we use %run or any magic commands in a Delta Live Table pipeline?
When we try, it says magic commands are not supported. Is there a way to import other notebooks first so that its functions can be referenced/used while we build a DLT pipeline?
- 8184 Views
- 4 replies
- 5 kudos
- 5 kudos
Hi @Manimeghala Vemula Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best an...
- 5 kudos
- 12051 Views
- 5 replies
- 1 kudos
Connecting to AWS MSK
how to connect to the AWS MSK which has I-Am based authentication?
- 12051 Views
- 5 replies
- 1 kudos
- 1 kudos
Hi @Sudhir Jaiswal Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...
- 1 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 586 | |
| 349 | |
| 287 |
