Hi There,Scenario at work is as described in the subject line:Is it possible to author a SQL (Python) scalar UDF IN A SQL SERVERLESS WAREHOUSE, which involves a library NOT included in any Databricks Runtime? And how would one go about it? (Documenta...
Based on [an answer in this community](https://community.databricks.com/s/question/0D58Y00009jfKM1SAM/error-no-metastore-assigned-for-the-current-workspace), we set "data security mode" to LEGACY_SINGLE_USER_STANDARD in a cluster policy which needed...
I've been trying to create Community Edition account, but keep getting: "An error has occurred. Please try again later" message. I searched the other posts, there are some people running into the same issue as well, one post talked about VPN and succ...
is there a place I could find the best practices to migrate Databricks from Azure to GCP?
If you want to migrate Databricks to Databricks, you should use Databricks Teraform Exporter for that job
I was trying to explore Genie room, I observed that I may need to finetune the model. Can I finetuned exisiting model with the same UI?
No. Genie spaces uses a fine tuned specialised text-2-sql model. fine-tuning text to 2 SQL model is not an easy job, so I would not recommend you doing yourself unless your business requires it.
Receiving this error: KeyProviderException: Failure to initialize configuration for storage account adlspersonal.dfs.core.windows.net: Invalid configuration value detected for fs.azure.account.keyI used hive meta store to save my table %python spark....
not sure about that error, but try to do it using SQL, see if it will workcreate table hive_metastore.annual_enterprise_survey as select * from catalog.defaul.table
Where can we see when what feature is expected? can you share the schedule?
You need to subscribe to Databricks newsletter, and tune in for the Quarterly Product Roadmap Webinars
Hi there,This is my first thread and it's a baby-foot step in the Databricks community, especially Data engineering section.am working in the community edition and I found this enhancement needed to be implemented: The need is to make the breadcrumbs...
Good feedback, thank you. we are actully looking to complelty revamp the databricks community edition and the experience will be much simpler. stay tuned.
How to integrate the CI/CD process with Databricks using Azure Devops on Catalog level instead of workspace level. I would like to understand the process if this is possible, given that if the catalog is used in different workspaces in same subscript...
CICD is not related to catalogs, it is related to environment (workspaces), there are lots of tutorials on youtube on how to setup Azure DevOps CICD to move assets from one workspace to another and start a job. you will need to use Databricks Plugin ...
It would be beneficial to have an option like a toggle to activate or deactivate a Task in the Job graph interface. This mainly helps to skip execution of a task and reactivate it as required. Currently there is no option to say I want this task to b...
Maybe just load the task with an empty notebook, and once decided just load the right notebook. not ideal but should do the job I guess
This data frame has two columns name and info. Name has value as John and info has vale as 1® VOC.After writing this data, I can read correct values in databricks but when I download the csv file and load it in notepad ( utf-8 ) , it is showing no va...
Try to read the file back into databricks using spark.read, do you the see the charchaters showing? if yes, then it is an editor problem, use another editor such as Notepad++, if not, then the data is not on the write encoder, try different encoder o...
I am trying to write data in csv format (df. coalesce(1) writeformat("csv")option ("header","true")Option (" sep","|시")option("encoding", "UTF-8”)mode ("overwrite")save("/mnt/Filestore/data/job1))Now here the problem is - there is one column in this ...
I have databricks job with around 70 notebooks. When job starts, only one notebook gets executed and the rest of the notebooks that are at the beginning of the line are in state PENDING (not blocked). Looks like notebooks cannot run in parallel for t...
Maybe something related to autoscaling options? So when databricks detects increased workload it will scale up number of workers and then the rest of notebooks get executed. Do you use DLT ?
Dear Databricks Certification Team,I am writing to request a review of my recent Databricks Certified Data Engineer Associate exam, which was unfortunately suspended on July 4th 2024.I was diligently taking the exam and focused solely on the computer...
When I want to run notebook in databricks all queries, saves and read take really long and I found error message in the clusters event log that says: Metastore is down. So, I think cluster is not able to connect to the metastore right now. Could be t...
This issue is solely related to the VNET. Azure engineer must set up connection within VNET correctly.
| User | Count |
|---|---|
| 1625 | |
| 790 | |
| 506 | |
| 349 | |
| 287 |
