cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

shahebaj
by New Contributor
  • 1157 Views
  • 1 replies
  • 0 kudos

2023 AI+Data Summit

Hello Everyone,I'm here to learn about technology advancements and how they can help my company. Keynotes were great and learned lot of things through breakout sessions and looking forward to giving back to the community 

  • 1157 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @shahebaj, We're thrilled to hear about your positive experience at DAIS 2023 and your eagerness to give back to the community! Your participation and engagement are what make events like DAIS so valuable. We appreciate your support and look forwa...

  • 0 kudos
Hyperparam42
by New Contributor
  • 723 Views
  • 1 replies
  • 0 kudos

multi-cloud data streaming talk today at Data + AI Summit 2023

The talk highlighted the benefits of using an open data lake for unified batch and streaming workloads and showcased features like Autoloader for data discovery, streaming triggers for seamless switching, and streaming aggregation for incremental com...

  • 723 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @Hyperparam42,  Thank you for sharing your experience and insights from the Data + AI Summit 2023! We're thrilled to hear that you found the talk on multicloud data streaming to be enlightening. We wanted to let you know that the Databricks Commun...

  • 0 kudos
dy
by New Contributor
  • 945 Views
  • 1 replies
  • 0 kudos

Great Conference!!

Great Conference, it was cool learning about creating my own LLM, and the risks associated with LLM.sMy favorite activity was the Spin at Dark on Tuesday night!  

  • 945 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @dy, Great to hear you had a valuable experience at DAIS 2023! We appreciate your attendance and participation. We wanted to share that the Databricks Community Team will be returning to San Francisco to host the Databricks Community booth at DAIS...

  • 0 kudos
DatabricksBofA
by New Contributor
  • 618 Views
  • 1 replies
  • 0 kudos

Training at Summit

Enrolled in the “Machine Learning in Production” and the “LLMs in Production” Classes and completed. Great Training.. looking forward to implementing soon!!!

  • 618 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @DatabricksBofA, That's fantastic to hear! It's always rewarding to see attendees like yourself finding value in the training and looking forward to implementing what you've learned. Your enthusiasm is truly appreciated! I wanted to share some exc...

  • 0 kudos
ValMir
by New Contributor II
  • 740 Views
  • 1 replies
  • 1 kudos

First timer at Summit and super impressed!

What a great community of people and practitioners! Loved to learn more about the path forward and all the ways this community will certainly have an impact on the future of AI.

  • 740 Views
  • 1 replies
  • 1 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 1 kudos

Hi @ValMir, That's a wonderful sentiment! It's fantastic to hear that you found the DAIS event so enriching. Your enthusiasm for the community and its impact on the future of AI is truly inspiring. We greatly appreciate your attendance and participat...

  • 1 kudos
mohaimen_syed
by New Contributor III
  • 1320 Views
  • 3 replies
  • 0 kudos

VS Code integration with Python Notebook and Remote Cluster

Hi, I'm trying to work on VS code remotely on my machine instead of using the Databricks environment on my browser. I have went through documentation to set up the Databricks. extension and also setup Databricks Connect but don't feel like they work ...

  • 1320 Views
  • 3 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @mohaimen_syed, It sounds like you’re trying to use Databricks Connect to run a Python notebook on a remote Azure Databricks cluster from your local machine. Let’s break down the steps to achieve this: Configure Azure Databricks Authentication...

  • 0 kudos
2 More Replies
rahuja
by New Contributor III
  • 1557 Views
  • 2 replies
  • 0 kudos

Py4JError: An error occurred while calling o992.resourceProfileManager

Hello I am trying to run the SparkXGBoostRegressor and I am getting the following error:SpoilerPy4JError: An error occurred while calling o992.resourceProfileManager. Trace: py4j.security.Py4JSecurityException: Method public org.apache.spark.resource...

  • 1557 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @rahuja, The error you’re encountering might be related to the interaction between PySpark and XGBoost. Let’s explore some potential solutions: PySpark Version Compatibility: Ensure that your PySpark version is compatible with the XGBoost vers...

  • 0 kudos
1 More Replies
Surajv
by New Contributor III
  • 606 Views
  • 1 replies
  • 0 kudos

Getting databricks-connect com.fasterxml.jackson.databind.exc.MismatchedInputException parse warning

Hi community, I am getting below warning when I try using pyspark code for some of my use-cases using databricks-connect. Is this a critical warning, and any idea what does it mean?Logs: WARN DatabricksConnectConf: Could not parse /root/.databricks-c...

  • 606 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi, @Surajv, The warning you’re encountering is related to using Databricks Connect with PySpark.  Databricks Connect: Databricks Connect is a Python library that allows you to connect your local development environment to a Databricks cluster. I...

  • 0 kudos
rafal_walisko
by New Contributor II
  • 955 Views
  • 1 replies
  • 0 kudos

Optimal Strategies for downloading large query results with Databricks API

Hi everyone,I'm currently facing an issue with handling a large amount of data using the Databricks API. Specifically, I have a query that returns a significant volume of data, sometimes resulting in over 200 chunks.My initial approach was to retriev...

  • 955 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @rafal_walisko, Handling large volumes of data using the Databricks API can indeed be challenging, especially when dealing with numerous chunks.   Let’s explore some strategies that might help you optimize your approach: Rate Limits and Paral...

  • 0 kudos
ymt
by New Contributor II
  • 1738 Views
  • 1 replies
  • 1 kudos

connection from databricks to snowflake using OKTA

Hi team,This is how I connect to Snowflake from Jupyter Notebook:import snowflake.connector snowflake_connection = snowflake.connector.connect( authenticator='externalbrowser', user='U1', account='company1.us-east-1', database='db1',...

  • 1738 Views
  • 1 replies
  • 1 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 1 kudos

Hi @ymt, It seems you’ve encountered an issue while connecting to Snowflake from your Databricks Notebook. The error message you received is: ImportError: cannot import name 'NamedTuple' from 'typing_extensions' (/databricks/python/lib/python3.9/s...

  • 1 kudos
mh_db
by New Contributor III
  • 4020 Views
  • 5 replies
  • 1 kudos

Job parameters to get date and time

I'm trying to set up a workflow in databricks and I need my job parameter to get the date and time. I see in the documentation there's some options for dynamic values.I'm trying to use this one: {{job.start_time.[argument]}}For the "argument" there, ...

  • 4020 Views
  • 5 replies
  • 1 kudos
Latest Reply
brockb
Valued Contributor
  • 1 kudos

Then please change the code to:```iso_datetime = dbutils.widgets.get("LoadID")```

  • 1 kudos
4 More Replies
Surajv
by New Contributor III
  • 3842 Views
  • 2 replies
  • 0 kudos

Getting python version errors when using pyspark rdd using databricks connect

Hi community, When I use pyspark rdd related functions in my environment using databricks connect, I get below error: Databricks cluster version: 12.2. `RuntimeError: Python in worker has different version 3.9 than that in driver 3.10, PySpark cannot...

  • 3842 Views
  • 2 replies
  • 0 kudos
Latest Reply
Surajv
New Contributor III
  • 0 kudos

Got it. As a side note, I tried above methods, but the error persisted, hence upon reading docs again, there was this statement: You must install Python 3 on your development machine, and the minor version of your client Python installation must be t...

  • 0 kudos
1 More Replies
sharpbetty
by New Contributor II
  • 2610 Views
  • 3 replies
  • 0 kudos

Workflows: Running dependent task despite earlier task fail

I have a scheduled task running in workflow.Task 1 computes some parameters then these are picked up by a dependent reporting task: Task 2.I want Task 2 to report "Failure" if Task 1 fails. Yet creating a dependency in workflows means that Task 2 wil...

Get Started Discussions
tasks
Workflows
  • 2610 Views
  • 3 replies
  • 0 kudos
Latest Reply
NerdSan
New Contributor II
  • 0 kudos

Hi @sharpbetty , Any suggestions how I can keep the parameter sharing and dependency from Task 1 to Task 2, yet also allow Task 2 to fire even on failure of Task 1?Setup:Task 2 dependent on Task1 Challenge: To Fire Task 2 even on Task 1 FailureSoluti...

  • 0 kudos
2 More Replies
Prashanthkumar
by New Contributor III
  • 2668 Views
  • 2 replies
  • 1 kudos

Databricks Users Access Control via Azure AAD?

Hi All,Looking for suggestions to see if it is possible to control users via Azure AD (outside of Azure Databricks). As i want to create a new users in Azure and then I want to give RBAC to individual users and rather than control their permissions f...

  • 2668 Views
  • 2 replies
  • 1 kudos
Latest Reply
Prashanthkumar
New Contributor III
  • 1 kudos

Thank you Kaniz, let me try some of the options as my Databricks is integrated with AAD. Let me try Option 1 as thats my primary requirement.

  • 1 kudos
1 More Replies
jvk
by New Contributor III
  • 3843 Views
  • 4 replies
  • 1 kudos

Resolved! Cant create cluster: "Aws Authorization Failure:" .. not authorized to perform: sts:AssumeRole

Full error here:Aws Authorization Failure:Failure happened when talking to AWS, AWS API error code: AccessDenied AWS error message: User: arn:aws:iam::414351767826:user/ConsolidatedManagerIAMUser-ConsolidatedManagerUser-VX02FYW0SSCY is not authorized...

  • 3843 Views
  • 4 replies
  • 1 kudos
Latest Reply
jvk
New Contributor III
  • 1 kudos

Never mind, I see it now thanks!

  • 1 kudos
3 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels