- 9520 Views
- 2 replies
- 2 kudos
Resolved! ThreadPoolExecutor in Databricks
I am using a threadpool executor and running notebooks in parallel. However, these parallel notebooks are not using executors at all and all the load is going towards the driver node resulting in running out of memory for the driver node and eventual...
- 9520 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @uzair mustafa​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedbac...
- 2 kudos
- 2484 Views
- 2 replies
- 1 kudos
Resolved! Query with ORDER BY fails with HiveThriftServerError "requirement failed: Subquery … has not finished"
Using ODBC or JDBC to read from a table fails when I attempt to use an ORDER BY clause. In one sample case, I have a fairly small table (just 1946 rows).select * from some_table order by some_fieldResult:java.lang.IllegalArgumentException: requiremen...
- 2484 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @petter@hightouch.com Petter​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it doe...
- 1 kudos
- 1053 Views
- 1 replies
- 5 kudos
Starting from #databricks runtime 12.2 LTS, implicit lateral column aliasing is now supported. This feature enables you to reuse an expression defined...
Starting from #databricks runtime 12.2 LTS, implicit lateral column aliasing is now supported. This feature enables you to reuse an expression defined earlier in the same SELECT list, thus avoiding repetition of the same calculation.For instance, in ...
- 1053 Views
- 1 replies
- 5 kudos
- 5 kudos
Thanks for sharing this with the Databricks community.
- 5 kudos
- 1616 Views
- 2 replies
- 2 kudos
Resolved! User Memory problem on driver with DLT
Hi! I have a problem with user memory on driver (I have almost several mb of storage memory, 0 Execution memory and more than 7GB of JVM Memory on Heap in use). How it can be? I don't have any broadcast variables, joins or aggregations. All the pipe...
- 1616 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Yuliya Valava​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...
- 2 kudos
- 1958 Views
- 4 replies
- 0 kudos
- 1958 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Youssef Mrini​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedbac...
- 0 kudos
- 23682 Views
- 12 replies
- 10 kudos
Resolved! Fetching top 1 L rows in Databricks SQL
how to fetch the top 1L rows from a long SQL query as the query is returning 5gb data, I want to check only the first 1L ROWS.
- 23682 Views
- 12 replies
- 10 kudos
- 1177 Views
- 1 replies
- 0 kudos
Resolved! Is it possible to migrate Databricks SQL objects?
I am trying to migrate Databricks SQL Dashboards and Queries from one workspace to another.Recently Databricks allows storing SQL objects in Workspace folders (https://docs.databricks.com/sql/user/workspace-browser/), but archiving the folder into db...
- 1177 Views
- 1 replies
- 0 kudos
- 0 kudos
@Kamting Tsoi​ Import/Export dashboards feature is in private preview. Please reach out to your AE to onboard your workspace
- 0 kudos
- 1095 Views
- 1 replies
- 0 kudos
Resolved! How to use caller's rights in Databricks Workflows?
I have created a job in Databricks Workflow that runs a notebook and have given manage run permissions to users. If any user trigger the job, I expect the job to show their username in Run As Option as it was triggered by them and they have Manage Ru...
- 1095 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Vigneshraja Palaniraj​ Rus as is a private preview feature. Please complete this sign up form which will be reviewed by PM and you'll receive the confirmation once it's enabled.
- 0 kudos
- 1433 Views
- 2 replies
- 5 kudos
"Hey everyone, it seems like there's some confusion about enhanced autoscaling in Databricks lately. If you're feeling lost or unsure abo...
"Hey everyone, it seems like there's some confusion about enhanced autoscaling in Databricks lately. If you're feeling lost or unsure about how it works, don't worry - you're not"Enhanced autoscaling is a feature in Databricks that enables dynamic sc...
- 1433 Views
- 2 replies
- 5 kudos
- 1098 Views
- 0 replies
- 3 kudos
Hey there! I've noticed that many people seem to be confused about the differences between databases, data warehouses, and data lakes. It's un...
Hey there! I've noticed that many people seem to be confused about the differences between databases, data warehouses, and data lakes. It's understandable, as these terms can be easily misunderstood or used interchangeablyHere is the summary for all ...
- 1098 Views
- 0 replies
- 3 kudos
- 12919 Views
- 4 replies
- 9 kudos
Resolved! Databricks CLI configure (using AAD-TOKEN) in the Databricks notebook `%sh` mode
Hello everyone, I am trying to setup Databricks CLI by referring to the Databricks CLI documentation. When I setup using the Personal Access Token, it works fine and I am able to access the workspace and fetch the results from the same workspace in D...
- 12919 Views
- 4 replies
- 9 kudos
- 9 kudos
Hi @Juned Mala​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 9 kudos
- 4026 Views
- 8 replies
- 7 kudos
Command to install spark package from notebook cell
Hi All, I need to install a spark-xml package from a notebook cell (hoping it will work on a DLT cluster). Maven Package: com.databricks:spark-xml_2.12:0.16.0Can anyone help me with the command to install from the notebook cell? Fairly new to all thi...
- 4026 Views
- 8 replies
- 7 kudos
- 7 kudos
Hi @Jason Johnson​ I'm sorry you could not find a solution to your problem in the answers provided.Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.I suggest ...
- 7 kudos
- 4489 Views
- 5 replies
- 4 kudos
Resolved! Databricks Community version - Unable to clone a public git repository
Databricks Community version - Unable to clone a public git repository, as the 'Repository' tab that should appear below 'Workspace' tab on the portal does not appear and I am not aware of any alternate method. I have referred to some documents on th...
- 4489 Views
- 5 replies
- 4 kudos
- 4 kudos
Hi @Jay Kumar​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedback wi...
- 4 kudos
- 3244 Views
- 2 replies
- 1 kudos
Resolved! Is there a way to run Databricks query on vba?
Currently, I just manually copy paste ​a code from an excel sheet and paste it on a databricks notebook and run for results, then, copy paste the results to the same workbook. I'm sure there's a faster way to do it. The only solutions i can find is u...
- 3244 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Hanna Wade​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 1 kudos
- 3150 Views
- 6 replies
- 5 kudos
How to delete strings from the /FileStore/
We have just started working with databricks in one of my university modules, and the lecturers gave us a set of commands to practice saving data in the FileStore. One of the commands was the following:dbutils .fs.cp("/ databricks - datasets / weathh...
- 3150 Views
- 6 replies
- 5 kudos
- 5 kudos
Hi @Konrad Kawka​ I'm sorry you could not find a solution to your problem in the answers provided.Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.I suggest ...
- 5 kudos