Hi, I use Terraform databricks_job to submit a job with many tasks. IFAIK, each task is a separated Spark Application with a specific applicationId (https://spark.apache.org/docs/3.1.3/api/python/reference/api/pyspark.SparkContext.applicationId.html)...
Databricks is a unified set of tools for building, deploying, sharing, and maintaining enterprise-grade data solutions at scale. The Azure Databricks Lakehouse Platform integrates with cloud storage and security in your cloud account, and manages and...
Hi,I have created my account using my email address on https://community.databricks.com, and my credentials work on https://community.databricks.com.But, when I try login using the same credentials on partner-academy.databricks.com, my login fails. ...
Hi @nikhiljatiwal,Thank you for reaching out!Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training and our team will get back to you shortly.
QuickBooks Desktop Pro is a complete accounting system designed specially for small and medium-sized businesses. This solution provides accounting software application functionality, such as account payable and receivable, project accounting, expendi...
Hi,Background:In Azure-Datebricks:We have a personal account that created a workspace, this workspace is our main workspace. The users Microsoft Office account is already deleted but the user still exist in Databricks.Question 1 is:What happens to th...
Is the Access token the only uthentication method ?
The authentication method may vary depending on the specific implementation and the data source you are connecting to. While an access token is one common authentication method, it is not the only method available. Here are a few authentication optio...
I'm trying to read a message from a confluent kafka topic using databricks but i keep getting below error. can you let me know if I'm missing something?codeoptions = {"kafka.sasl.jaas.config": 'org.apache.kafka.common.security.plain.PlainLoginModule ...
The error message you're encountering suggests an issue with fetching the earliest offsets for the Kafka topic. It seems that the KafkaOffsetReaderConsumer is unable to retrieve the initial partition offsets, causing the error to occur. To troublesho...
It’s a wonderful summit! I met so many women in tech during this event, it’s a very good opportunity to exchange thought and ideas with such a divers ppl to modernize Data & AI! I actually join here from Japan, and unfortunately not so many women in ...
Thanks Sakura for dropping by at the Community cove and asking these questions. Together folks like yourself can move the needle and bring in more women to Data and AI space by being inspirational. I am sure you will attract many of us to be in this ...
Below are 2 versions of codes I tried and the failure messages. Does anyone have advice how to form this properly?I tried thisfor x in range(2,num_organization_page,1): file_str = f"dbfs:/mnt/dlsi2023/raw/ds_pet_finder_organization{x}.json", print(fi...
Your code works, you just have to remove the comma at the end of the second linefile_str = f"dbfs:/mnt/dlsi2023/raw/ds_pet_finder_organization{x}.json", Should be file_str = f"dbfs:/mnt/dlsi2023/raw/ds_pet_finder_organization{x}.json" This is what is...
Hello, Community members! I hope this message finds you well. I wanted to reach out and inquire about your experience with DAIS and how you are absorbing all the valuable information it provides. Additionally, I would love to hear what aspects of Gen...
Dear DBricks Team, Could you kindly look at the Back End Service issues? Thanks a Million.
Hello, My requirement is to created a materialized view (Live table) from a downstream table. The down stream table gets refreshed frequently and hence the materialized view needs to be recalculated. I don't want the pipe line to be in continuous mod...
Hi @Chengcheng I am fine with manual refresh also and have tried "Refresh Table <TableName>" command but it doesn't refreshes the table with new data.The Full refresh command "REFRESH TABLE <tableName> FULL; is giving below error inside notebook.REFR...
I'm trying to run the Dolly demo on a nc6s_v3 128 GB single node cluster but get the following out of memory. Does anyone have any suggestions as to why this is/how to solve it? ThxOutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB (GP...
I thought this would be a good place to share your favorite talks from the Databricks Data+AI Summit 2023 Heck feel free to share your favorite talk from a different conference or previous summit Today's talk 'Delta-rs, Apache Arrow, Polars, WASM: Is...
I am having an issue where my permissions aren't being passed to workflows. I can use the same cluster to access files in an S3 repo but if I try to do spark-submit or a jar task on workflows I am unable to reach my buckets. I am trying to use Unity ...
