cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

hillel1
by New Contributor II
  • 567 Views
  • 0 replies
  • 0 kudos

benchmark tpc-ds from external parquet hive structure in S#

Hi I am just getting started in databricks would appreciate some help here.I have 10TB TPCDS in S3 i a hive partition structure.My goal is to benchmark a data bricks cluster on this data.after setting all IAM credentials according to this https://doc...

Get Started Discussions
External Table
S3
  • 567 Views
  • 0 replies
  • 0 kudos
crvkumar
by New Contributor II
  • 2586 Views
  • 3 replies
  • 0 kudos

Databricks Notebook says "Connecting.." for some users

For some users, after clicking on a notebook the screen says "connecting..." and the notebook does not open.The users are using Chrome browser and the same happens with Edge as well.What could be the reason?

  • 2586 Views
  • 3 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @crvkumar, If some users are unable to open a notebook and the screen displays “connecting…”, it could be due to a network connectivity issue or a problem with the browser.    Here are some steps you can try to resolve the issue:   Check network c...

  • 0 kudos
2 More Replies
sanjay
by Valued Contributor II
  • 1142 Views
  • 3 replies
  • 1 kudos

Performance issue while calling Sagemaker Endpoint in pyspark udf

Hi,I have pyspark dataframe which calls pyspark udf which in turn calls sagemaker endpoint. But when dataframe has more rows, endpoint start failing. Also it takes longer to process.Please suggest how to call sagemaker endpoint from pyspark.Regards,S...

  • 1142 Views
  • 3 replies
  • 1 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 1 kudos

Hi @sanjay ,  Here are some resources that could help you call a SageMaker endpoint from PySpark: Call an Amazon SageMaker model endpoint using Amazon API Gateway and AWS Lambda | AWS Machine Learning Bloghow to use sagemaker inside pyspark - Stack O...

  • 1 kudos
2 More Replies
NeerajSharma
by New Contributor
  • 749 Views
  • 0 replies
  • 0 kudos

Unable to run query- Insert into with 'Nan' value in SQL Editor. Getting the error.

Unable to run query- Insert into with 'Nan' value in SQL Editor.Query :-Insert into ABC with values('xyz',123,Nan);Error :-org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation cannot be cast to org.apache.spark.sql.execution.datasources...

  • 749 Views
  • 0 replies
  • 0 kudos
alesventus
by New Contributor III
  • 604 Views
  • 0 replies
  • 0 kudos

How to check if autoloader processed new file

Hi,My scenario is this: autoloader loads data from ADLS once a day.data are merged into the bronze zone.bronze table has CDF turned on.read CDF and merge into the silver.all works fine, but when there is not new file for autoloader to process, CDF ta...

  • 604 Views
  • 0 replies
  • 0 kudos
andre_gonzalez
by New Contributor
  • 1365 Views
  • 3 replies
  • 0 kudos

SQL wharehouse do not work with power bi online service

Whenever i try to use a SQL Wharehouse serverless cluster on a power bi dataset it does not refresh on the power bi online service. It does work normally for other types of databricks clusters. The catalog is being defined on the power query import.I...

2023-10-17_14-36.png
  • 1365 Views
  • 3 replies
  • 0 kudos
Latest Reply
ChuckyDee25
New Contributor II
  • 0 kudos

Hi,We have the exact same issue, even if we specify the catalog in the connection parameters.However, Oauth authentication through a dataflow (instead of from Power Query Desktop) works fine. In Desktop we are in version 2.122.746.0, but the issue is...

  • 0 kudos
2 More Replies
farah
by New Contributor
  • 490 Views
  • 1 replies
  • 0 kudos

Issue: Verification email not received to sign up for Community Edition

Hello! I'm following this documentation : https://docs.databricks.com/en/getting-started/community-edition.htmlAnd I can't succeed the step 4) because I don't receive any email to verify my email address. I have checked the spam, no email.Is there so...

  • 490 Views
  • 1 replies
  • 0 kudos
Latest Reply
mskcognizant
New Contributor II
  • 0 kudos

@farah please check the junk mail folder and also message quarantine if you are using M$ services.The verification mail goes out fine. However, it may show up in Junk | Quarantine.

  • 0 kudos
Kiranrathod
by New Contributor III
  • 1698 Views
  • 3 replies
  • 0 kudos

Auotoloader-"cloudFiles.backfillInterval"

1. How to use cloudFiles.backfillInterval option in a notebook?2. Does It need to be any set of the property?3. Where is exactly placed readstream portion of the code or writestream portion of the code?4. Do you have any sample code?5. Where we find ...

  • 1698 Views
  • 3 replies
  • 0 kudos
Latest Reply
Kiranrathod
New Contributor III
  • 0 kudos

1.Is the following code correct for specifying the  .option("cloudFiles.backfillInterval", 300)?df = spark.readStream.format("cloudFiles") \.option("cloudFiles.format", "csv") \.option("cloudFiles.schemaLocation", f"dbfs:/FileStore/xyz/back_fill_opti...

  • 0 kudos
2 More Replies
suru
by New Contributor
  • 951 Views
  • 1 replies
  • 0 kudos

Resolved! Unable to Login to community edition

I am unable to login in databricks community account, my email was "sureshsuthar1251@gmail.com". When i try to login it says account not found.

  • 951 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @suru , Please look at this link related to the Community - Edition, which might solve your problem. I appreciate your interest in sharing your Community-Edition query with us. However, at this time, we are not entertaining any Community-Edition q...

  • 0 kudos
Ankur_K
by New Contributor II
  • 2462 Views
  • 2 replies
  • 0 kudos

Connecting to Databricks using Python(VS Code)

I'm trying to connect to tables/views in Databricks using Python(via VS Code). However, I'm getting the following error:- File "C:\Users\XXXXXXXX\AppData\Roaming\Python\Python311\site-packages\urllib3\util\retry.py", line 592, in incrementraise MaxRe...

Get Started Discussions
SSLCertificateError
  • 2462 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @Ankur_K, The error message you're encountering indicates an issue with verifying the SSL certificate for the provided URL. This "certificate verify failed" error can be due to various factors. Here are some steps to resolve the problem: ...

  • 0 kudos
1 More Replies
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels
Top Kudoed Authors