Hi all,I'm trying to run some functions from another notebook (data_process_notebook) in my main notebook, using the %run command command. When I run the command: %run ../path/to/data_process_notebook, it is able to complete successfully, no path, pe...
I have setup a spring boot application which works as expected as a standalone spring boot app.When i build the jar and try to set it up as a databricks job, i am facing these issues.i am getting same error in local as well.I have tried using maven-s...
could you please try with python terminal and see how that behaves?I am not 100% sure if this is relates to your use case.@Dinesh L​
I need duplicated a new job create in stage A in another stage, automatically. is posible?
you may try to get the job details from our job api https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsGet and get the response to duplicate it.,
Hello everybody,I recently discovered (the hard way) that when a query plan uses cached data, the AQE does not kick-in. Result is that you loose the super cool feature of dynamic partition coalesce (no more custom shuffle readers in the DAG). Is ther...
Hi @Pantelis Maroudis​,Did you check the physical query plan? did you check the SQL sub tab with in Spark UI? it will help you to undertand better what is happening.
Hello community!I have a table with a column that is an array of a struct that has a very very long schema.When the table is written, all works well. Though, when I create a view based on this table and I try to access the view I get the error:rg.apa...
@Pantelis Maroudis​ , were you able to solve this issue? Please advise if you got any tip. Thanks in advance
Afternoon everyone! I logged in hoping to see some suggestions but think maybe I need to reword the question a little How can I create a UDF that converts '30000101' to timestamp and then use it in a query like below?%sqlselectfield1,field2,nvl(some...
Got it working (but going to post a new question momentarily): I needed to use timestamp(date '3000-01-01) instead of to_timestamp
Hi,I am trying to run following query:SELECT table_schema, table_name, COUNT(column_name) FROM {db_name}.INFORMATION_SCHEMA.COLUMNS GROUP BY table_schema, table_nameand I am getting following error:Error in SQL statement: AnalysisException: Catalog n...
Hi @Suman Karki​, can you check if UC is enabled in the advanced settings of the endpoint? Also, what DBR is your DE cluster, and what Security Mode did you choose?
At one of the Databricks presentations in August a new type of table was introduced, I kind of remember these were large tables which did not change much. I think I heard these new table typed were in Beta. Unfortunately I lost my notes for that pres...
Hi, Thanks for reaching out to community.databricks.com. Are you talking about data objects in Databricks lakehouse? https://docs.databricks.com/lakehouse/data-objects.html
Error type: System call error. Cannot make named pipe /tmp/EvtnUM74TA574813952 due to error: No space left on device..))I can't find anything recent online related to Databricks or the SQL warehouses specifically. I don't seem to have enough monitori...
Microstrategy connects to a small SQL Datawarehouse (SQL endpoint), which I guess is serverless by default. I don't believe the Databricks UI gives many options other than size and cost efficient or reliable setting. May be a temporary issue since it...
When writing data from Pyspark to Azure SQL Server (official databricks tutorial here) I am getting an error in the conversion between Spark and Parquet types.I believe this is caused in the temporary storage location mandatory when writing data from...
Hi DatabricksCould we please regarding information about upcoming sessions and recorded videos of past sessions .This would be v helpful for users like meKind RegardsPriya
We have views created with mixed case column names in 7.3 LTS and when we do select * from view everything looks OK I mean column names display in mixed case and problem started as we recently migrated to 10.4 LTS.Same views created with create or re...
@Ravikumar Kumashi​ other than the DBR version, do you have any other configuration on the clusters? Also, is it possible for you to share a sample table and code so we could reproduce this?
%shexport aa=100echo $aOUTPUT: 100in the below cell trying to use the variable a, but it's not printing the value%shecho $aOUTPUT:
Hi, bash is not supported as notebook language. Instead, it is a magic cell executing shell script on the driver node.
Hi All, I am getting below error often while trying to load data from databricks using spark odbc connections. QVX_UNEXPECTED_END_OF_DATA: SQL##f - SqlState: S1000, ErrorCode: 61, ErrorMsg: [Simba][Hardy] (61) Server returned error with no error mess...
How to Migrate Your Data and AI Workloads to Databricks With the AWS Migration Acceleration Programhttps://www.databricks.com/blog/2022/08/19/how-to-migrate-your-data-and-ai-workloads-to-databricks-with-the-aws-migration-acceleration-program.htmlIn t...
Thank your for sharing this information @Abishek Subramanian​
| User | Count |
|---|---|
| 1629 | |
| 790 | |
| 511 | |
| 349 | |
| 287 |
