- 19144 Views
- 8 replies
- 2 kudos
Resolved! Help with Parameters in Databricks SQL
I am following the documentation on query parameters (https://docs.databricks.com/sql/user/queries/query-parameters.html) and it's not working for me. I can't get a parameter inserted at the text caret and I can't get cmd + p to work. I've changed sp...
- 19144 Views
- 8 replies
- 2 kudos
- 2 kudos
I don't see the need for spaces around the parameter name... I have just been typing {{param_name}} and it appears automatically in SQL editor
- 2 kudos
- 2623 Views
- 2 replies
- 0 kudos
Downstream delta live table is unable to read data frame from upstream table
I have been trying to work on implementing delta live tables to a pre-existing workflow. Currently trying to create two tables: appointments_raw and notes_raw, where notes_raw is "downstream" of appointments_raw. Following this as a reference, I'm at...
- 2623 Views
- 2 replies
- 0 kudos
- 0 kudos
@Anna Wuest : Could you please send me the code snippet here? Thanks.
- 0 kudos
- 3363 Views
- 2 replies
- 3 kudos
NoSuchObjectException(message:There is no database named global_temp)
I can see the following error message in my driver logs. whats does it means? and how to solve it?ERROR RetryingHMSHandler: NoSuchObjectException(message:There is no database named global_temp)
- 3363 Views
- 2 replies
- 3 kudos
- 3 kudos
should one create it in the work space manually via UI? would it get overwritten if work space is created via terraform?I use 10.4 LTS runtime.
- 3 kudos
- 2615 Views
- 2 replies
- 0 kudos
ERROR : Failure starting repl. Try detaching and re-attaching the notebook. Getting this error when running any python command in 10.4LTS cluster configured with https://github.com/mspnp/spark-monitoring to send databricks spark logs to Log Analytics.
ERROR Description :java.lang.Exception: Cannot run program "/local_disk0/pythonVirtualEnvDirs/virtualEnv-5acc1ea9-d03f-4de3-b76b-203d42614000/bin/python" (in directory "."): error=2, No such file or directory at java.lang.ProcessB...
- 2615 Views
- 2 replies
- 0 kudos
- 0 kudos
@Akanksha Gupta :The error message suggests that the Python executable file specified in the configuration of the Databricks cluster cannot be found or accessed. Specifically, it seems that the Python executable file at the path "/local_disk0/python...
- 0 kudos
- 2123 Views
- 3 replies
- 0 kudos
Issue withpassing parameters to the queries in spark sql temporary function
I have created a function like belowcreate function test(location STRING, designation STRING, name STRING)RETURNS TABLE (cnt INT)RETURN(SELECT CASE WHEN location = 'INDIA' THEN (SELECT COUNT(*) FROM tbl_customers where job_role = design...
- 2123 Views
- 3 replies
- 0 kudos
- 0 kudos
@Raghu Dandu :The error message suggests that the column "designation" does not exist in the table "tbl_customers". There could be several reasons for this error, such as a typo in the column name, a missing or deleted column, or a difference in the...
- 0 kudos
- 14108 Views
- 2 replies
- 7 kudos
�� �� Weekly Raffle to Win Ticket to Data + AI Summit 2023 �� �� NO PURCHASE NECESSARY TO ENTER OR WIN. A PURCHASE OF ANY...
Weekly Raffle to Win Ticket to Data + AI Summit 2023 NO PURCHASE NECESSARY TO ENTER OR WIN. A PURCHASE OF ANY KIND WILL NOT INCREASE YOUR CHANCES OF WINNING. VOID WHERE PROHIBITED.We are giving away one ticket to Data + AI Summit 2023 every week ...
- 14108 Views
- 2 replies
- 7 kudos
- 7 kudos
@Fjoraldo Mamutaj , @Aviral Bhardwaj and @Shubham Soni : we have emailed you about some clarifications on your week 1 participation . Request you to reply to my email
- 7 kudos
- 1365 Views
- 2 replies
- 0 kudos
Tuesday get to know each other series: Hey everyone! I'm curious to know what tech books you've been reading lately. As a community of tech en...
Tuesday get to know each other series:Hey everyone! I'm curious to know what tech books you've been reading lately. As a community of tech enthusiasts, I'm sure we all have some great recommendations to share with each other.To get the ball rolling, ...
- 1365 Views
- 2 replies
- 0 kudos
- 0 kudos
If you are interested in distributed machine learning I would suggest Scaling Machine Learning with Spark by Adi Pola k . It is just out so it has all the new goodies including the latest in ML Flow. I am half way though it already but it looks like ...
- 0 kudos
- 5132 Views
- 3 replies
- 1 kudos
When I aggregate over more data, I get the below error message. I've tried multiple ways of diagnosis like going back to a version I know it was w...
When I aggregate over more data, I get the below error message. I've tried multiple ways of diagnosis like going back to a version I know it was working fine (but still got the same error below). Please advise as this is a critical report where the b...
- 5132 Views
- 3 replies
- 1 kudos
- 1 kudos
@Jeff Wu :The error message suggests that there is a syntax error in a SQL statement, specifically near the end of the input. Without the full SQL statement or additional information, it's difficult to pinpoint the exact cause of the error. However,...
- 1 kudos
- 3987 Views
- 4 replies
- 2 kudos
Resolved! master notebook cannot find the udf registered in the child notebook
The master notebook is calling a child notebook using dbutils.notebook.run("PathToChildnotebook"). The child notebook defines a user-defined function (UDF) and registers it using spark.udf.register. However, when the child notebook finishes running a...
- 3987 Views
- 4 replies
- 2 kudos
- 2 kudos
@andrew li :The reason why the UDF cannot be found is that when the child notebook finishes running, the Spark context that was used to define and register the UDF is destroyed. Therefore, the UDF is no longer available in the Spark context used by ...
- 2 kudos
- 3479 Views
- 3 replies
- 2 kudos
AutoLoader issue - java.lang.AssertionError
The below error I am encountering . I am using microbatch for autoloader. please help to rectify this issuejava.lang.AssertionError: assertion failed: Invalid batch: path#36188,modificationTime#36189,length#36190L,content#36191,PROVIDER#36192,LOCATIO...
- 3479 Views
- 3 replies
- 2 kudos
- 2 kudos
@Ayesha Rahmatali :The error message you provided suggests that there is an assertion failure due to invalid batch data in your AutoLoader implementation. The error specifically indicates that the schema of the incoming data is not matching with the...
- 2 kudos
- 12751 Views
- 4 replies
- 5 kudos
How can i use the same spark session from onenotebook to another notebook in databricks
I want to use the same spark session which created in one notebook and need to be used in another notebook in across same environment, Example, if some of the (variable)object got initialized in the first notebook, i need to use the same object in t...
- 12751 Views
- 4 replies
- 5 kudos
- 5 kudos
You can use %run and then use the location of the notebook - %run "/folder/notebookname"
- 5 kudos
- 4378 Views
- 2 replies
- 0 kudos
pip install in cluster using web UI and extra index
In an init script or a notebook, we can:pip install --index-url=<our private pypi url> --extra-index-url=https://pypi.org/simple <a module>In the cluster web UI (libraries -> install library), we can give only the url of our private repository, but n...
- 4378 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Philippe CRAVE Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...
- 0 kudos
- 939 Views
- 1 replies
- 0 kudos
I just passed the Lakehouse Fundamentals Accreditation and I haven't received my badge yet and the certification seems to be messed up
I just passed the Lakehouse Fundamentals Accreditation and I haven't received my badge yet and the certification seems to be messed up
- 939 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Sabyasachi Samaddar We are going through a contract renewal with our vendor, Accredible. Once our new contract goes through you will get your badge this week.Thank you for understanding.
- 0 kudos
- 2396 Views
- 1 replies
- 0 kudos
Kafka consumer groups in Databricks
I was trying to find information about configuring the consumer groups for kafka stream in databricks. By doing so I want to parallelize the stream and load it into databricks tables. Does the databricks handle this internally? If we can configure th...
- 2396 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi, we have a few examples on stream processing using Kafka (https://docs.databricks.com/structured-streaming/kafka.html), there is no straight public document for Kafka consumer group creation. You can refer to https://kafka.apache.org/documentation...
- 0 kudos
- 3934 Views
- 4 replies
- 2 kudos
Delta table property is not set.
I have set the delta table property at cluster level.spark.databricks.delta.retentionDurationCheck.enabled falseWhen I create a new table, retentionDurationCheck property is not shown in the table details. But when I set this with ALTER TABLE for a s...
- 3934 Views
- 4 replies
- 2 kudos
- 2 kudos
Hi @Mahesh Chahare Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...
- 2 kudos
| User | Count |
|---|---|
| 1611 | |
| 768 | |
| 348 | |
| 286 | |
| 252 |
