Hello all,wanted to know how to install custom python libraries or load the library files on multiuser or shared databrick cluster.thanks
I'm trying to do a select on a column with special characters in Databricks sql.I've tried the following%sql select ex$col from database.table limit 10; %sql select `ex$col ` from database.table limit 10;They both don't work and will return "ex do...
Hi @Jamie Nathan​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...
I am trying to add a button in a notebook to trigger an execution of another notebook, but it does not respond to the command. Any idea why? The run command works if I run in a separate cell
Did you end up finding an answer to this? I am running into the same problem
Hi team,I attended the webinar on 21th feb 2023 and also took Lakehouse fundamentals badge, yet I have not received any certification voucher from databricks.regards,vikas
Hi @Vikas Singh​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedback ...
Databricks now supports event-driven workloads, especially for loading cloud files from external locations. This means you can save costs and resources by triggering your Databricks jobs only when new files arrive in your cloud storage instead of mou...
Hi @Hubert Dudek​ We really appreciate you sharing this bit of information.Cheers!
I would like to find the notebooks that are not required and not being used and then I can review and delete them. If there is a way to find last modified date of a notebook programmatically then I can get a list of notebooks, which I can review and ...
Hi @Naveen Kumar Madas​ you can go through below code block%shls -lt /dbfs/
Hello everyone,I'm new in Databricks SQL, and I'm comming from SQL Server.I would like to know what's the equivalent of :DECLARE @P_Name varchar(50) = 'BackOffice'It's for use it like this : CREATE DATABASE @P_NameThanks.
Hi @Salah K.​ you can go through this code block%python P_Name = 'BackOffice'spark.sql(f""" create database {P_name} """)
Hi, You can refer to https://docs.databricks.com/files/unzip-files.html. You can curl the file you want and then it can be unzipped as mentioned in the doc. Please let us know if this helps.Also, please tag @Debayan with your next update which will n...
Hi Team,I'm not able to create a metastore in a region (us-east-1).It tells me that "This region already contains a metastore. Only a single metastore is allowed per region"But we don't have any metastore. Earlier we had one metastore we had deleted...
@Rajath C​ can you please re-check if it has been properly deleted and if still old one has been tied to any of workspaces. also try to delete that storage if no data exists and retry
Flattening Nested XML - Multiple ways
Thanks for this and educating the community members about it.
I'm trying to access init script which is stored on mounted azure data lake storage gen2 to dbfsI mounted storage to dbfs:/mnt/storage/container/script.shand when i try to access it i got an error:Cluster scoped init script dbfs:/mnt/storage/containe...
I do not think the init script saved under mount point work and we do not suggest that. If you specify abfss , then the cluster need to be configured so that the cluster can authenticate and access the adls gen2 folder. Otherwise, the cluster will no...
ProblemI have a directory in S3 with a bunch of data files, like "data-20221101.parquet". They all have the same columns: timestamp, reading_a, reading_b, reading_c. In the earlier files, the readings are floats, but in the later ones they are double...
1) Can you let us know what was the error message when you don't set the schema & use mergeSchema2) What happens when you define schema (with FloatType) & use mergeSchema ? what error message do you get ?
Tool For Monitoring Security/Health of Databricks WorkspaceSince from a year we have been looking for a tool to monitor health of data bricks workspace in automated way. we used to monitor below few things in workspace manually clusters JobsTablesACL...
Data + AI Summit Virtual - Register Now! This year’s free virtual experience will include access to live-streamed keynotes, select sessions designed and led by data experts, as well as unlimited access to on-demand sessions soon after the live event....
Thank you for sharing @Sujitha Ramamoorthy​ !!
Exciting news for #azure users! The #databricks runtime 12.2 has been officially released as a long-term support (LTS) version, providing a stable and reliable platform for users to build and deploy their applications. As part of this release, the en...
Thank you for sharing @Hubert Dudek​ !!!
