HI All,I use the latest version of Databricks (2.6.34) in my Mulesoft Application to and the code it deployed successfully in local and while testing i have observed the below mentioned error.Kindly check and help resolving the issue.at com.databrick...
We have two environments for our Azure Databricks. Dev and Prod. We had clusters created and tested in Dev environment, then they were exported to the prod environment through APIs. The clusters in Dev are performing as expected. Whereas, the cluster...
Both Prod and Dev are connect to unity catalog and I am working with the same table in both the envs. Can something done during the creation of workspace itself affect the performance of clusters? Do clusters update to latest Databricks runtime versi...
Hi, I've been struggling to run webdriver.Chrome() in my Databricks. 1) I installed a chromedriver : %sh# Download ChromeDriverwget https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/119.0.6045.105/linux64/chromedriver-linux64.zip -O /tmp/ch...
I’m keen to learn more about employing Distributed Ledger Technology (DLT) in a production environment. Databricks is indeed promoting Delta Live Tables (DLT) as a comprehensive framework for creating robust, maintainable, and testable data processin...
Just a small remark: I definitely meant Delta Live Tables, not Distributed Ledger Technology (DLT)
Hello Databricks Community,I'm working with a DLT pipeline where I consume Protobuf-serialized messages and attempt to decode them using the Spark "from_protobuf" function. My function kafka_msg_dlt_view is outlined as follows:def kafka_msg_dlt_view(...
I finally found the appropriate method for configuring the PERMISSIVE mode. With this setup, corrupted protobuf messages will be processed without throwing an exception .withColumn("msg",from_protobuf(col("value"), message_name, desc_file_path, {"mo...
Reference:PY SDK I can't find any API directly to download the logs. Did I miss anything?Otherwise, I can download via DBFS cluster logs location.
Hi @Cert-TeamI hope this message finds you well. I am writing to request a review of my recently suspended exam. I believe that my situation warrants reconsideration, and I would like to provide some context for your understanding.I applied for Datab...
Hi @Sujitha or @Kaniz Can you please respond to the above query swiftly. As I need to complete my test quickly. Kindly help me to resume or restart my test.Regards,Fasih Ahmed
Hi,My scenario is this: autoloader loads data from ADLS once a day.data are merged into the bronze zone.bronze table has CDF turned on.read CDF and merge into the silver.all works fine, but when there is not new file for autoloader to process, CDF ta...
we have enabled system tables, under system tables we are able to see compute schema, under that we are not able to see cluster and node_type in azure databricks. do we have any limitations for above tables
Hello! I'm following this documentation : https://docs.databricks.com/en/getting-started/community-edition.htmlAnd I can't succeed the step 4) because I don't receive any email to verify my email address. I have checked the spam, no email.Is there so...
@data_game please check the junk mail folder and also message quarantine if you are using M$ services.The verification mail goes out fine. However, it may show up in Junk | Quarantine.
Hi all,I'm trying to do creating one job cluster with one configuration or specification which has a workflow and this workflow needs to have 3 dependent tasks as a straight line. For example, t1->t2->t3. In databricks there are some constraints also...
Hello,We are using DLT pipelines for many of our jobs with notifications on failures to slack.Wondering if there is a clean way to disable the alerts when in development mode. It does make sense to have it turned off in dev, doesn't it?
Hi Team,I have attended the Advantage Lakehouse: Fueling Innovation in the Era of Data and AI webinar.Also completed Databricks Lakehouse Fundamentals and feedback survey, but still I have not received the Databricks voucher.Could you please look i...
1. How to use cloudFiles.backfillInterval option in a notebook?2. Does It need to be any set of the property?3. Where is exactly placed readstream portion of the code or writestream portion of the code?4. Do you have any sample code?5. Where we find ...
1.Is the following code correct for specifying the .option("cloudFiles.backfillInterval", 300)?df = spark.readStream.format("cloudFiles") \.option("cloudFiles.format", "csv") \.option("cloudFiles.schemaLocation", f"dbfs:/FileStore/xyz/back_fill_opti...
Is there a way to set up a workflow with multiple tasks, so that different tasks can share the same compute resource, at the same time?I understand that an instance pool may be an option, here. Wasn't sure if there were other possible options to cons...
