Hi all,I was wondering if it was possible to configure a single SQL warehouse to a specific Blob container?I know you can add configuration properties to the Workspace from the Admin settings, but it appears that those configurations will be shared b...
I have a python notebook A in Azure Databricks having import statement as below: import xyz, datetime,... I have another notebook xyz being imported in notebook A as shown in above code. When I run notebook A, it throws the following error: ImportEr...
Create a repository containing an __init__.py fileAdd your library as .py file(s). Let's imagine that our library is composed by multiple sub-folders consolidated in "my_folder", one of sub-folders is named as "math_library" and contains my_awesome_l...
Hi @Amardeep Singh​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...
Hello,I am trying to create a database in Azure Databricks using the abfss Location in the create database statement and it throws an exception.%sqlCREATE DATABASE IF NOT EXISTS test COMMENT "Database for Test Area" LOCATION "abfss://test@storagetemp...
ok.Can you check this link?Because I suspect something is wrong, and so dbrx uses the account key as a default (fs.azure.account.key)
HI guys,I tried to use the regex_extract but when I put { in pattern I received errordisplay( df_containerlog.withColumn('reg', regexp_extract(col('LogEntry'), "((^.*))({'data':)", 1)) ) have any ideaThank you
Hello William,You have to escape the { as it is a special char in case of regex."((^.*))(\{'data':)"
When installing Notebook-scoped R libraries I don't want to manually specify the custom CRAN mirror each time like this:install.packages("diffdf", repos="my_custom_cran_url'')Instead I want to take the custom CRAN mirror URL by default so that I don'...
Got solution on Stack Overflow for this problem: https://stackoverflow.com/a/76777228/18082636
HelloWhen I try to create an external location I get this error:Failed to access cloud storage: [AbfsRestOperationException] HTTP Error -1CustomTokenProvider getAccessToken threw com.databricks.api.base.DatabricksServiceException : INTERNAL_ERROR: Un...
I think I sould have something missconfigured, the way I solved the problem was to re-create the workspace and start from scratch, it was a small one for testing proposes.
Utilizing GCP instances using the 12.2DBR ML runtime. Prior to ~7:10CT last night, _sqldf commands in notebooks to reference the previously executed %sql cell would work locally as well as in scheduled Workflow Job runs. Now it appears that the code ...
It looks like Azure was having the same issue, it might just be all 12.2 Photon clusters.https://community.databricks.com/t5/data-engineering/sqldf-bugs-in-gcp-workspaces/td-p/38578 That post says it was been fixed, but last I checked it was still fa...
I am getting below error while streaming the data from pubsub using databricks DLT pipelines If anyone can help to increase the gRPC message size will help alot.
Please help to reolve this issue i implemented all possible steps, credentials are correct and i am not using free trial account, i was watching tutorial to mount i used same code and everything is done as said in tutorial but iam getting this weird ...
Hi, I am not so sure. But very likely you have tried with any cluster with Access mode as Shared.Can you try with a single node cluster, if the problem still exists, please share your problem once again here.please tag me in your question again if ne...
When I am trying to read snowflake table from my databricks notebook, it is giving the error as:df1.read.format("snowflake") \.options(**options) \.option("query", "select * from abc") \.save()Getting below errorjava.sql.SQLException: No suitable dri...
hello, am running into in issue while trying to write the data into a delta table, the query is a join between 3 tables and it takes 5 minutes to fetch the data but 3hours to write the data into the table, the select has 700 records. here are the app...
it turned out that the issue was not in the writing side, even when i was getting the results in under 5min, the issue was in the cross join in my query i resolved the issue by doing the same cross joins via dataframes got the results computed and wr...
I am not able to restart our streaming DLT pipeline as it's giving "__tmp_path_dir" number format error, I am using PUB-SUB as a source for streaming.Any solutions will help - #pusub #databricks
Hello. What would be the equivalent of the below in databricks?DECLARE @LastChangeDate as dateSET @LastChangeDate = GetDate()I already tried the below and worked. However I need to know how set a sql variable dynamicallySET da.dbname = test;SELECT "$...
I am using Following job trigger API -https://docs.databricks.com/api/workspace/jobs/runnowHere is what I want to achieve -I have complex workflow with multiple tasks and each task has python parameter named "--conf-file".While using runnow REST API ...
@p4pratikjain wrote:I am using Following job trigger API - Popeyes Guest Experience Surveyhttps://docs.databricks.com/api/workspace/jobs/runnowHere is what I want to achieve - I have complex workflow with multiple tasks and each task has python param...
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 578 | |
| 349 | |
| 287 |
