cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

bhagyashree-ds
by New Contributor III
  • 627 Views
  • 3 replies
  • 0 kudos

Help to reschedule of my exam

Hi team,My Databricks Certified Data Engineer Associate exam got suspended within 40 minutes. I had also shown my exam room to the proctor. My exam got suspended due to watch. I was not using anything. Some questions are so big in the exam so I had t...

  • 627 Views
  • 3 replies
  • 0 kudos
Latest Reply
bhagyashree-ds
New Contributor III
  • 0 kudos

Thankyou for replying

  • 0 kudos
2 More Replies
otara_geni
by New Contributor
  • 1059 Views
  • 1 replies
  • 0 kudos

How to Resolve ConnectTimeoutError When Registering Models with MLflow

Hello everyone,I'm trying to register a model with MLflow in Databricks, but encountering an error with the following command: model_version = mlflow.register_model(f"runs:/{run_id}/random_forest_model", model_name)   The error message is as follows:...

  • 1059 Views
  • 1 replies
  • 0 kudos
Latest Reply
some-rsa
Databricks Employee
  • 0 kudos

@otara_geni if you are still struggling, try this - set the environmental variable in your code just before logging the model with the URL of the regional S3 endpoint (from the error it looks like MLFlow is attempting to use global one, which may not...

  • 0 kudos
viniciuscini
by New Contributor
  • 1152 Views
  • 1 replies
  • 0 kudos

Improve query performance of direct query with Databricks

I’m building a dashboard in Power BI’s Pro Workspace, connecting data via Direct Query from Databricks (around 60 million rows from 15 combined tables), using a SQL Serverless (small size and 4 clusters).The problem is that the dashboard is taking to...

  • 1152 Views
  • 1 replies
  • 0 kudos
Latest Reply
Brahmareddy
Valued Contributor III
  • 0 kudos

Hi @viniciuscini, How are you doing?It sounds like you've already taken a lot of the right steps to optimize your Power BI dashboard, especially given the constraints you mentioned. Since you've ruled out many common optimizations and noticed that th...

  • 0 kudos
dvl_priyansh
by New Contributor
  • 391 Views
  • 2 replies
  • 0 kudos

What exactly is Vectorized query processing and columnar acceleration

Hey folks! I want to know and understand while using photon acceleration, there is a feature called columnar acceleration which basically is a method of storing data in columns rather than rows, which is particularly advantageous for analytical datab...

  • 391 Views
  • 2 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 0 kudos

Hi @dvl_priyansh ,Take a look at below article. It has a great explanation and answers you questions:A Closer Look Into Databricks’s Photon Engine | by Vu Trinh | Data Engineer Things (det.life)

  • 0 kudos
1 More Replies
valjas
by New Contributor III
  • 1535 Views
  • 3 replies
  • 1 kudos

How do I create spark.sql.session.SparkSession?

When I create a session n Databricks it is defaulting to spark.sql.connect.session.SparkSession. How can I connect to spark with out spark connect?

  • 1535 Views
  • 3 replies
  • 1 kudos
Latest Reply
miguel_ortiz
New Contributor II
  • 1 kudos

Is there any solution to this? Pandera, Evidently and Ydata Profiling break because they don't speak a sql.connect session object. They expect a spark.sql.session.SparkSession it's very frustrating not being to use any of these libraries with the new...

  • 1 kudos
2 More Replies
itsmejoeyong
by New Contributor II
  • 1120 Views
  • 3 replies
  • 1 kudos

Resolved! Best Approach for Handling ETL Processes in Databricks

I am currently managing nearly 300 tables from a production database and considering moving the entire ETL process away from Azure Data Factory to Databricks.This process, which involves extraction, transformation, testing, and loading, is executed d...

  • 1120 Views
  • 3 replies
  • 1 kudos
Latest Reply
Brahmareddy
Valued Contributor III
  • 1 kudos

Hi,Instead of 300 individual files or one massive script, try grouping similar tables together. For example, you could have 10 scripts, each handling 30 tables. This way, you get the best of both approches—This way you will have a freedom of easy deb...

  • 1 kudos
2 More Replies
Brahmareddy
by Valued Contributor III
  • 1103 Views
  • 4 replies
  • 3 kudos

Understanding Flight Cancellations and Rescheduling in Airlines Using Databricks and PySpark

In the airline industry, it’s important to manage flights efficiently. Knowing why flights get canceled or rescheduled helps improve customer satisfaction and operational performance. In this article, I’ll show you how to use Databricks and PySpark t...

Community Platform Discussions
airlines
artificial intelligence
feature engineering
machine learning
  • 1103 Views
  • 4 replies
  • 3 kudos
Latest Reply
Rishabh-Pandey
Esteemed Contributor
  • 3 kudos

@Brahmareddy Interesting one , thanks for sharing

  • 3 kudos
3 More Replies
AyushPandey
by New Contributor II
  • 4170 Views
  • 8 replies
  • 0 kudos

Unable to reactive an inactive user

Hi all,I am facing an issue with reactivating an inactive user i tried the following json with databricks cli run_update = {  "schemas": [ "urn:ietf:params:scim:api:messages:2.0:PatchOp" ],  "Operations": [    {      "op": "replace",      "path": "ac...

  • 4170 Views
  • 8 replies
  • 0 kudos
Latest Reply
bencsik
New Contributor III
  • 0 kudos

@FunkybunchOO Thank you for your response! I will look into other connections, but we are not currently using SCIM. There must be something similar blocking the activation.

  • 0 kudos
7 More Replies
priyansh
by New Contributor III
  • 215 Views
  • 0 replies
  • 0 kudos

UCX

Hey folks! I want to know what are the features that UCX does not provides in UC or specially Hive to UC Migration that can be done manually but not using UCX. As UCX is currently in developing mode so there are so many drawbacks, can someone share t...

  • 215 Views
  • 0 replies
  • 0 kudos
TinaN
by New Contributor III
  • 463 Views
  • 1 replies
  • 0 kudos

Resolved! Translating XMLNAMESPACE in SQL Databricks

We are loading a data source that contains XML. I am translating their queries to create views in Databricks. They use 'XMLNAMESPACES' to construct/parse XML.  Below is an example.  What is best practice for translating 'XMLNAMESPACES' in Databricks?...

  • 463 Views
  • 1 replies
  • 0 kudos
Latest Reply
" src="" />
This widget could not be displayed.
This widget could not be displayed.
This widget could not be displayed.
  • 0 kudos

This widget could not be displayed.
We are loading a data source that contains XML. I am translating their queries to create views in Databricks. They use 'XMLNAMESPACES' to construct/parse XML.  Below is an example.  What is best practice for translating 'XMLNAMESPACES' in Databricks?...

This widget could not be displayed.
  • 0 kudos
This widget could not be displayed.
ozbieG
by New Contributor II
  • 537 Views
  • 1 replies
  • 0 kudos

Databricks Certification exam got Suspended - Need Support

Hello Team, @Cert-Team , @Cert-TeamOPS I faced a very bad experience while attempting my 1st DataBricks certification.I was asked to exit the exam multiple times by the support team saying technical issues. My test got rescheduled multiple times with...

  • 537 Views
  • 1 replies
  • 0 kudos
Latest Reply
ozbieG
New Contributor II
  • 0 kudos

@Retired_mod 

  • 0 kudos
bytetogo
by New Contributor
  • 475 Views
  • 1 replies
  • 0 kudos

What API Testing Tool Do You Use?

Hi Databricks!I am a relatively new developer that's looking for a solid API testing tool. I am interested in hearing about other developers, new or experienced, about their experiences with API testing tools, regardless if they are good or bad. I've...

  • 475 Views
  • 1 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 0 kudos

Hi @bytetogo,In my daily work I use Postman. It has user-friendly interface, supports automated testing and has support for popular patterns and libraries. It is also compatible with Linux, MacOs, Windows.

  • 0 kudos
vigneshkannan12
by New Contributor
  • 1944 Views
  • 2 replies
  • 0 kudos

typing extensions import match error

I am trying to install the stanza library and try to create a udf function to create NER tags for my chunk_text in the dataframe.Cluster Config: DBR 14.3 LTS SPARK 3.5.0 SCALA 2.12below code:def extract_entities(text    import stanza    nlp = stanza....

  • 1944 Views
  • 2 replies
  • 0 kudos
Latest Reply
SaadhikaB
New Contributor II
  • 0 kudos

Was this resolved?I installed openai and tried to import and faced the below error. I also tried upgrading the libraries. However I ended up with the same error.Code: import openaiError: ImportError: cannot import name 'override' from 'typing_extensi...

  • 0 kudos
1 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Top Kudoed Authors