by
Soma
• Valued Contributor
- 1794 Views
- 5 replies
- 3 kudos
Hi Team for secure connection we created secured cluster withNPIP(https://learn.microsoft.com/en-us/azure/databricks/security/secure-cluster-connectivity) WORKSPACE hosted in a private VNET.We had a hub vnet with private endpoint for key vault ,We pe...
- 1794 Views
- 5 replies
- 3 kudos
Latest Reply
Hi @somanath Sankaran , We haven’t heard from you since the last response from @Hubert Dudek, and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be helpful to others....
4 More Replies
- 5024 Views
- 2 replies
- 6 kudos
Hi All,i am getting the below error when i am ingesting the data from source file , source file is also attached , i have tried in both Community edition and Azure databricks as well getting the same error , can any one suggest me the solution ? # ...
- 5024 Views
- 2 replies
- 6 kudos
Latest Reply
Hi @Rakesh Reddy Gopidi , We haven’t heard from you since the last response from me, and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be helpful to others. Otherwise...
1 More Replies
by
Ken1
• New Contributor III
- 913 Views
- 4 replies
- 7 kudos
I have this error - com.databricks.WorkflowException: com.databricks.NotebookExecutionException: FAILEDwhen i ran dbutils.notebook.run(.......)
- 913 Views
- 4 replies
- 7 kudos
Latest Reply
Hi @Godswill Mbata , We haven’t heard from you since the last response from @Debayan Mukherjee , and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be helpful to othe...
3 More Replies
- 7021 Views
- 2 replies
- 0 kudos
Hello, i have a problem.When I try to run the MLlib Assembler (from pyspark.ml.feature import VectorAssembler) I get this error and I don't know what to do anymore. Please help.
- 7021 Views
- 2 replies
- 0 kudos
Latest Reply
Is it High Concurrency cluster with credential passthrough enabled? In that case, you can use a different cluster mode.https://docs.azuredatabricks.net/spark/latest/data-sources/azure/adls-passthrough.htmlThis exception is thrown when you have access...
1 More Replies
by
jm99
• New Contributor III
- 1492 Views
- 2 replies
- 3 kudos
Using Azure Databricks:I can create a DLT table in python usingimport dlt
import pyspark.sql.functions as fn
from pyspark.sql.types import StringType
@dlt.table(
name = "<<landingTable>>",
path = "<<storage path>>",
comment = "<< descri...
- 1492 Views
- 2 replies
- 3 kudos
Latest Reply
Hi @John Mathews did you find a way to progress here?i am stuck in the same point...
1 More Replies
by
jon1
• New Contributor II
- 487 Views
- 1 replies
- 0 kudos
Hi!We're working with change event data from relational and NoSQL databases then processing and ingesting that into DataBricks. It's streamed from source to our messaging platform. Then, our connector is pushing to DataBricks.Right now we're doing th...
- 487 Views
- 1 replies
- 0 kudos
Latest Reply
Update on the theory we are looking at. It'd be similar to below (with necessary changes to support best practices for MERGE such as reducing the search space):-- View for deduping pre-merge CREATE OR REPLACE TEMPORARY VIEW {view} AS SELECT * EXCEPT ...
- 2330 Views
- 3 replies
- 7 kudos
I am writing/reading data from Azure databricks to data lake. I wrote dataframe to a path in delta format using query a below, later I realized that I need the data in parquet format, and I went to the storage account and manually deleted the filepat...
- 2330 Views
- 3 replies
- 7 kudos
Latest Reply
Hi @Kris Koirala , We haven’t heard from you since the last response from @Jose Gonzalez , and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be helpful to others. Ot...
2 More Replies
- 1445 Views
- 2 replies
- 1 kudos
Is there a way to read the CSV files automatically from on-premises network locations and write back to the same from Databricks?
- 1445 Views
- 2 replies
- 1 kudos
Latest Reply
Hi @Raagavi Rajagopal , We haven’t heard from you since the last response from @Debayan Mukherjee, and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be helpful to ot...
1 More Replies
- 897 Views
- 3 replies
- 2 kudos
I've always used alert e-mail notifications with my custom message, written in HTML. The problem is that today it suddenly is not working anymore and I'm getting the alert e-mail notification distorted, as HTML doesn't work anymore.Does anyone know w...
- 897 Views
- 3 replies
- 2 kudos
Latest Reply
Apparently, it has been corrected and it is working again. Thank you everyone
2 More Replies
by
g96g
• New Contributor III
- 3274 Views
- 2 replies
- 1 kudos
Im doing the self study and want pass df column name as a parameter.I have defined the widget column_name= dbutils.widgets.get('column_name')which is executing succefuly ( giving me a column name)then Im reading the df and do some transformation and ...
- 3274 Views
- 2 replies
- 1 kudos
Latest Reply
Hi @Givi Salu , We haven’t heard from you since the last response from @Hubert Dudek, and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be helpful to others. Otherwis...
1 More Replies
- 646 Views
- 2 replies
- 1 kudos
I am experimenting with spark, on my local machine. So, is there some tool/api available to check the performance of the code I write?For eg. I write:val startTime = System.nanoTime()
invoicesDF
.select(
count("*").as("Total Number Of Inv...
- 646 Views
- 2 replies
- 1 kudos
Latest Reply
Hi @mghildiy , We haven’t heard from you since the last response from @Hubert Dudek, and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be helpful to others. Otherwis...
1 More Replies
- 2035 Views
- 5 replies
- 4 kudos
- 2035 Views
- 5 replies
- 4 kudos
Latest Reply
Hi @NOOR BASHA SHAIK, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.
4 More Replies
- 3222 Views
- 4 replies
- 4 kudos
Hi guys,How you suggestion about how to create a medalion archeterure ? how many and what datalake zones, how store data, how databases used to store, anuthing I think that zones:1.landing zone, file storage in /landing_zone - databricks database.bro...
- 3222 Views
- 4 replies
- 4 kudos
Latest Reply
Hi @William Scardua , We haven’t heard from you since the last response from @Jose Gonzalez , and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be helpful to others....
3 More Replies
- 4569 Views
- 3 replies
- 4 kudos
I have several delta live table notebooks that are tied to different delta live table jobs so that I can use multiple target schema names. I know it's possible to reuse a cluster for job segments but is it possible for these delta live table jobs (w...
- 4569 Views
- 3 replies
- 4 kudos
Latest Reply
Hi @John Fico , We haven’t heard from you since the last response from @Hubert Dudek and @Jose Gonzalez , and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be helpf...
2 More Replies
