- 17150 Views
- 5 replies
- 6 kudos
cannot convert Parquet type INT64 to Photon type string
I am receiving an error similar to the post in this link: https://community.databricks.com/s/question/0D58Y00009d8h4tSAA/cannot-convert-parquet-type-int64-to-photon-type-doubleHowever, instead of type double the error message states that the type can...
- 17150 Views
- 5 replies
- 6 kudos
- 6 kudos
@John Laurence Sy :It sounds like you are encountering a schema conversion error when trying to read in a Parquet file that contains an INT64 column that cannot be converted to a string type. This error can occur when the Parquet file has a schema t...
- 6 kudos
- 8948 Views
- 6 replies
- 2 kudos
Resolved! Accessing a FastAPI endpoint using Personal Access Token (PAT)
Hello Community,I have a FastAPI endpoint on a cluster with addess 0.0.0.0:8084/predict. And I want to send a request to this endpoint from a React App which is locally hosted on my computer. I have a Personal access token for the workspace but dont ...
- 8948 Views
- 6 replies
- 2 kudos
- 2 kudos
@Aakash Bhandari :To send a request from a React App to a FastAPI endpoint on a Databricks cluster using a Personal Access Token (PAT), you can use the requests module in Python to make HTTP requests.Here's an example of how to use requests to send ...
- 2 kudos
- 3415 Views
- 1 replies
- 1 kudos
"[PARSE_SYNTAX_ERROR] Syntax error at or near 'ROW'(line 2, pos 4)".
Alter table <TABLE_NAME> SET ROW FILTER <func_name> on (COLUMN)Got the below error while running the below code "[PARSE_SYNTAX_ERROR] Syntax error at or near 'ROW'(line 2, pos 4)". Please help on this issue. we tried this code as part of access polic...
- 3415 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello,Please can you confirm which DBR version are you using & do you use unity catalog?
- 1 kudos
- 22885 Views
- 8 replies
- 2 kudos
Resolved! Help with Parameters in Databricks SQL
I am following the documentation on query parameters (https://docs.databricks.com/sql/user/queries/query-parameters.html) and it's not working for me. I can't get a parameter inserted at the text caret and I can't get cmd + p to work. I've changed sp...
- 22885 Views
- 8 replies
- 2 kudos
- 2 kudos
I don't see the need for spaces around the parameter name... I have just been typing {{param_name}} and it appears automatically in SQL editor
- 2 kudos
- 3155 Views
- 2 replies
- 0 kudos
Downstream delta live table is unable to read data frame from upstream table
I have been trying to work on implementing delta live tables to a pre-existing workflow. Currently trying to create two tables: appointments_raw and notes_raw, where notes_raw is "downstream" of appointments_raw. Following this as a reference, I'm at...
- 3155 Views
- 2 replies
- 0 kudos
- 0 kudos
@Anna Wuest : Could you please send me the code snippet here? Thanks.
- 0 kudos
- 3643 Views
- 2 replies
- 3 kudos
NoSuchObjectException(message:There is no database named global_temp)
I can see the following error message in my driver logs. whats does it means? and how to solve it?ERROR RetryingHMSHandler: NoSuchObjectException(message:There is no database named global_temp)
- 3643 Views
- 2 replies
- 3 kudos
- 3 kudos
should one create it in the work space manually via UI? would it get overwritten if work space is created via terraform?I use 10.4 LTS runtime.
- 3 kudos
- 3052 Views
- 2 replies
- 0 kudos
ERROR : Failure starting repl. Try detaching and re-attaching the notebook. Getting this error when running any python command in 10.4LTS cluster configured with https://github.com/mspnp/spark-monitoring to send databricks spark logs to Log Analytics.
ERROR Description :java.lang.Exception: Cannot run program "/local_disk0/pythonVirtualEnvDirs/virtualEnv-5acc1ea9-d03f-4de3-b76b-203d42614000/bin/python" (in directory "."): error=2, No such file or directory at java.lang.ProcessB...
- 3052 Views
- 2 replies
- 0 kudos
- 0 kudos
@Akanksha Gupta :The error message suggests that the Python executable file specified in the configuration of the Databricks cluster cannot be found or accessed. Specifically, it seems that the Python executable file at the path "/local_disk0/python...
- 0 kudos
- 2502 Views
- 3 replies
- 0 kudos
Issue withpassing parameters to the queries in spark sql temporary function
I have created a function like belowcreate function test(location STRING, designation STRING, name STRING)RETURNS TABLE (cnt INT)RETURN(SELECT CASE WHEN location = 'INDIA' THEN (SELECT COUNT(*) FROM tbl_customers where job_role = design...
- 2502 Views
- 3 replies
- 0 kudos
- 0 kudos
@Raghu Dandu :The error message suggests that the column "designation" does not exist in the table "tbl_customers". There could be several reasons for this error, such as a typo in the column name, a missing or deleted column, or a difference in the...
- 0 kudos
- 16623 Views
- 2 replies
- 7 kudos
�� �� Weekly Raffle to Win Ticket to Data + AI Summit 2023 �� �� NO PURCHASE NECESSARY TO ENTER OR WIN. A PURCHASE OF ANY...
Weekly Raffle to Win Ticket to Data + AI Summit 2023 NO PURCHASE NECESSARY TO ENTER OR WIN. A PURCHASE OF ANY KIND WILL NOT INCREASE YOUR CHANCES OF WINNING. VOID WHERE PROHIBITED.We are giving away one ticket to Data + AI Summit 2023 every week ...
- 16623 Views
- 2 replies
- 7 kudos
- 7 kudos
@Fjoraldo Mamutaj , @Aviral Bhardwaj and @Shubham Soni : we have emailed you about some clarifications on your week 1 participation . Request you to reply to my email
- 7 kudos
- 1805 Views
- 2 replies
- 0 kudos
Tuesday get to know each other series: Hey everyone! I'm curious to know what tech books you've been reading lately. As a community of tech en...
Tuesday get to know each other series:Hey everyone! I'm curious to know what tech books you've been reading lately. As a community of tech enthusiasts, I'm sure we all have some great recommendations to share with each other.To get the ball rolling, ...
- 1805 Views
- 2 replies
- 0 kudos
- 0 kudos
If you are interested in distributed machine learning I would suggest Scaling Machine Learning with Spark by Adi Pola k . It is just out so it has all the new goodies including the latest in ML Flow. I am half way though it already but it looks like ...
- 0 kudos
- 5714 Views
- 3 replies
- 1 kudos
When I aggregate over more data, I get the below error message. I've tried multiple ways of diagnosis like going back to a version I know it was w...
When I aggregate over more data, I get the below error message. I've tried multiple ways of diagnosis like going back to a version I know it was working fine (but still got the same error below). Please advise as this is a critical report where the b...
- 5714 Views
- 3 replies
- 1 kudos
- 1 kudos
@Jeff Wu :The error message suggests that there is a syntax error in a SQL statement, specifically near the end of the input. Without the full SQL statement or additional information, it's difficult to pinpoint the exact cause of the error. However,...
- 1 kudos
- 4706 Views
- 4 replies
- 2 kudos
Resolved! master notebook cannot find the udf registered in the child notebook
The master notebook is calling a child notebook using dbutils.notebook.run("PathToChildnotebook"). The child notebook defines a user-defined function (UDF) and registers it using spark.udf.register. However, when the child notebook finishes running a...
- 4706 Views
- 4 replies
- 2 kudos
- 2 kudos
@andrew li :The reason why the UDF cannot be found is that when the child notebook finishes running, the Spark context that was used to define and register the UDF is destroyed. Therefore, the UDF is no longer available in the Spark context used by ...
- 2 kudos
- 4092 Views
- 3 replies
- 2 kudos
AutoLoader issue - java.lang.AssertionError
The below error I am encountering . I am using microbatch for autoloader. please help to rectify this issuejava.lang.AssertionError: assertion failed: Invalid batch: path#36188,modificationTime#36189,length#36190L,content#36191,PROVIDER#36192,LOCATIO...
- 4092 Views
- 3 replies
- 2 kudos
- 2 kudos
@Ayesha Rahmatali :The error message you provided suggests that there is an assertion failure due to invalid batch data in your AutoLoader implementation. The error specifically indicates that the schema of the incoming data is not matching with the...
- 2 kudos
- 14053 Views
- 4 replies
- 5 kudos
How can i use the same spark session from onenotebook to another notebook in databricks
I want to use the same spark session which created in one notebook and need to be used in another notebook in across same environment, Example, if some of the (variable)object got initialized in the first notebook, i need to use the same object in t...
- 14053 Views
- 4 replies
- 5 kudos
- 5 kudos
You can use %run and then use the location of the notebook - %run "/folder/notebookname"
- 5 kudos
- 5041 Views
- 2 replies
- 0 kudos
pip install in cluster using web UI and extra index
In an init script or a notebook, we can:pip install --index-url=<our private pypi url> --extra-index-url=https://pypi.org/simple <a module>In the cluster web UI (libraries -> install library), we can give only the url of our private repository, but n...
- 5041 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Philippe CRAVE Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...
- 0 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 484 | |
| 349 | |
| 287 |
