- 5673 Views
- 1 replies
- 0 kudos
kafkashaded.org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
I'm trying to read a message from a confluent kafka topic using databricks but i keep getting below error. can you let me know if I'm missing something?codeoptions = {"kafka.sasl.jaas.config": 'org.apache.kafka.common.security.plain.PlainLoginModule ...
- 5673 Views
- 1 replies
- 0 kudos
- 0 kudos
The error message you're encountering suggests an issue with fetching the earliest offsets for the Kafka topic. It seems that the KafkaOffsetReaderConsumer is unable to retrieve the initial partition offsets, causing the error to occur. To troublesho...
- 0 kudos
- 1948 Views
- 1 replies
- 2 kudos
Resolved! First time at Data + AI summit
It’s a wonderful summit! I met so many women in tech during this event, it’s a very good opportunity to exchange thought and ideas with such a divers ppl to modernize Data & AI! I actually join here from Japan, and unfortunately not so many women in ...
- 1948 Views
- 1 replies
- 2 kudos
- 2 kudos
Thanks Sakura for dropping by at the Community cove and asking these questions. Together folks like yourself can move the needle and bring in more women to Data and AI space by being inspirational. I am sure you will attract many of us to be in this ...
- 2 kudos
- 3803 Views
- 1 replies
- 1 kudos
Using F Strings in filenames to read in files from mounted containers
Below are 2 versions of codes I tried and the failure messages. Does anyone have advice how to form this properly?I tried thisfor x in range(2,num_organization_page,1): file_str = f"dbfs:/mnt/dlsi2023/raw/ds_pet_finder_organization{x}.json", print(fi...
- 3803 Views
- 1 replies
- 1 kudos
- 1 kudos
Your code works, you just have to remove the comma at the end of the second linefile_str = f"dbfs:/mnt/dlsi2023/raw/ds_pet_finder_organization{x}.json", Should be file_str = f"dbfs:/mnt/dlsi2023/raw/ds_pet_finder_organization{x}.json" This is what is...
- 1 kudos
- 1005 Views
- 0 replies
- 1 kudos
Generation AI - would love to know your thoughts
Hello, Community members! I hope this message finds you well. I wanted to reach out and inquire about your experience with DAIS and how you are absorbing all the valuable information it provides. Additionally, I would love to hear what aspects of Gen...
- 1005 Views
- 0 replies
- 1 kudos
- 885 Views
- 0 replies
- 0 kudos
4-7-2023 [Back End Service Not Available] - DBCE
Dear DBricks Team, Could you kindly look at the Back End Service issues? Thanks a Million.
- 885 Views
- 0 replies
- 0 kudos
- 5073 Views
- 3 replies
- 1 kudos
How to refresh materialized view (Delta Live Table) without running a pipeline every time.
Hello, My requirement is to created a materialized view (Live table) from a downstream table. The down stream table gets refreshed frequently and hence the materialized view needs to be recalculated. I don't want the pipe line to be in continuous mod...
- 5073 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @Chengcheng I am fine with manual refresh also and have tried "Refresh Table <TableName>" command but it doesn't refreshes the table with new data.The Full refresh command "REFRESH TABLE <tableName> FULL; is giving below error inside notebook.REFR...
- 1 kudos
- 1064 Views
- 0 replies
- 0 kudos
dbdemos - Build your Chat Bot with Dolly memory issue
I'm trying to run the Dolly demo on a nc6s_v3 128 GB single node cluster but get the following out of memory. Does anyone have any suggestions as to why this is/how to solve it? ThxOutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB (GP...
- 1064 Views
- 0 replies
- 0 kudos
- 5464 Views
- 5 replies
- 2 kudos
Resolved! Please share your favorite talks from the Databricks Data+AI Summit 2023 :)
I thought this would be a good place to share your favorite talks from the Databricks Data+AI Summit 2023 Heck feel free to share your favorite talk from a different conference or previous summit Today's talk 'Delta-rs, Apache Arrow, Polars, WASM: Is...
- 5464 Views
- 5 replies
- 2 kudos
- 927 Views
- 0 replies
- 0 kudos
Workflow permissions
I am having an issue where my permissions aren't being passed to workflows. I can use the same cluster to access files in an S3 repo but if I try to do spark-submit or a jar task on workflows I am unable to reach my buckets. I am trying to use Unity ...
- 927 Views
- 0 replies
- 0 kudos
- 999 Views
- 0 replies
- 0 kudos
Using Delta Live Tables Structured Streaming for small batches?
Hi Databricks community, I have a blob storage folder that will receive file drops, with 3 files in each distinct drop: ex. Files A1,B1,C1 is one drop, A2,B2,C2 is the next drop. The DLT pipeline I've got setup has a lot of joins and aggregations, cu...
- 999 Views
- 0 replies
- 0 kudos
- 3358 Views
- 1 replies
- 2 kudos
Free Certification vouchers in Databricks Rewards Store
When will the Databricks Community rewards store gets upgraded with new 100% certification vouchers.
- 3358 Views
- 1 replies
- 2 kudos
- 2 kudos
@KVNARK same i am also looking for 100% certification voucher
- 2 kudos
- 905 Views
- 0 replies
- 0 kudos
Certification Number required to register for external certificate in organization.
Hi Databricks Team,@Anonymous I have successfully completed databricks certification and received the certificate with a URL in which i can see my certificate and badge respectively for databricks data engineering exam and Apache spark developer exa...
- 905 Views
- 0 replies
- 0 kudos
- 929 Views
- 0 replies
- 0 kudos
Missing 13.x runtimes on dockerhub
When is Databricks going to publish the 13.x runtimes on Dockerhub?We are using custom containers, but we need to upgrade to Pandas >= 2.0,.. and for that we need to upgrade Spark to >=3.4, and for that we need to upgrade our custom containers to the...
- 929 Views
- 0 replies
- 0 kudos
- 1553 Views
- 0 replies
- 0 kudos
Update a part of parquet partition is deleting existing data.
I have 4 months of data and I partitioned it on Year and Month column, so my parquet partition looks like Data is present inside each monthly partition folder in parquet format.Then I loaded data for July and also modified some values in August. Afte...
- 1553 Views
- 0 replies
- 0 kudos
- 6004 Views
- 6 replies
- 6 kudos
Resolved! Setting Up Databricks Workspace
Hi Team, Need some advice on setting up a single workspace or multiple workspaces.Please read below for the problem statement - There is an ADLS account says ADLS1 which has a root folder say root1This root1 folder has multiple sub foldersteam1team2...
- 6004 Views
- 6 replies
- 6 kudos
- 6 kudos
Hi @Loki Thank you for posting your question in our community! We are happy to assist you. To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your que...
- 6 kudos
