- 3258 Views
- 1 replies
- 0 kudos
Call databricks notebook from azure flask app
I have an Azure web app running flask web server. From flask server, I want to run some queries on the data stored in ADLS Gen2 storage. I already created Databricks notebooks running these queries. The flask server will pass some parameters in ...
- 3258 Views
- 1 replies
- 0 kudos
- 0 kudos
you can use databricks SDKhttps://docs.databricks.com/en/dev-tools/sdk-python.html#create-a-job
- 0 kudos
- 2417 Views
- 4 replies
- 0 kudos
Pyspark execution error
I am getting a error message when executing a simple pyspark code. Can anyone help me with this.
- 2417 Views
- 4 replies
- 0 kudos
- 0 kudos
Could you please share the entire error message?Are you running the code locally or on databricks?
- 0 kudos
- 1307 Views
- 1 replies
- 0 kudos
LLM using DataBrick
Is there any coding example for how to use LLM?
- 1307 Views
- 1 replies
- 0 kudos
- 0 kudos
I would like to share the following links https://www.databricks.com/product/machine-learning/large-language-models https://docs.databricks.com/en/large-language-models/index.html
- 0 kudos
- 5346 Views
- 5 replies
- 4 kudos
{{job.trigger.type}} not working and throws error on Edit Parameter from Job page
Following the instruction on the Job Parameter Dynamic values, I am able to use {{job.id}}{{job.name}}{{job.run_id}}{{job.repair_count}}{{job.start_time.[argument]}}However, when I set trigger_type as trigger_type: {{job.trigger.type}} and hit SAVE, ...
- 5346 Views
- 5 replies
- 4 kudos
- 4 kudos
Thanks everyone, I decided to use the Sparkcontext instead. dbutils.notebook.entry_point.getDbutils().notebook().getContext().toJson()
- 4 kudos
- 2297 Views
- 0 replies
- 0 kudos
AzureDevOps and Databricks Connection using managed identity or service principal
Hi All! Im in a project where i need to connect azure devops and databricks using managed identity to avoid the using of service account, PAT, etc.The thing is i cant move forward with the connection since i cannot take the ownership of the files wh...
- 2297 Views
- 0 replies
- 0 kudos
- 3314 Views
- 0 replies
- 0 kudos
Boomi integrating with Databricks
Hi Team,Is there any impact when integrating Databricks with Boomi as opposed to Azure Event Hub? Could you offer some insights on the integration of Boomi with Databricks?https://boomi.com/blog/introducing-boomi-event-streams/Regards,Janga
- 3314 Views
- 0 replies
- 0 kudos
- 17960 Views
- 6 replies
- 0 kudos
Resolved! Unity catalog issues
Hi all,I have recently enabled Unity catalog in my DBX workspace. I have created a new catalog with an external location on Azure data storage.I can create new schemas(databases) in the new catalog but I can't create a table. I get the below error wh...
- 17960 Views
- 6 replies
- 0 kudos
- 0 kudos
@Snoonan First of all, check the networking tab on the storage account to see if it's behind firewall. If it is, make sure that Databricks/Storage networking is properly configured (https://learn.microsoft.com/en-us/azure/databricks/security/network/...
- 0 kudos
- 1772 Views
- 1 replies
- 0 kudos
Help Refactor T-SQL Code to Databricks SQL
Hello CommunityCan someone help refactor the following T-SQL Code to Databricks SQLCONVERT(DECIMAL(26, 8), ISNULL(xxx.xxxxxxx * ISNULL(RH.xxxxx, 1 / NULLIF(ST.xxxxxx, 0)), ST.xxxxx)) AS AmountWhen I attempt to execute the above code I get the followi...
- 1772 Views
- 1 replies
- 0 kudos
- 0 kudos
You can use CAST instead.Eg:SELECT cast('2024' as int);
- 0 kudos
- 3064 Views
- 0 replies
- 0 kudos
Custom ENUM input as parameter for SQL UDF?
Hello - We're migrating from T-SQL to Spark SQL. We're migrating a significant number of queries."datediff(unit, start,end)" is different between these two implementations (in a good way). For the purpose of migration, we'd like to stay as consiste...
- 3064 Views
- 0 replies
- 0 kudos
- 6076 Views
- 2 replies
- 1 kudos
Nulls in Merge
If you are going to handle any null values in your MERGE condition, better watch out for your syntax #databricks
- 6076 Views
- 2 replies
- 1 kudos
- 3433 Views
- 3 replies
- 0 kudos
Ingest __databricks_internal catalog - PERMISSION DENIED
Good morning, I have a DLT process with CDC incremental load and I need to ingest the history as CDC transactions are only recent. To do this I need to ingest data in the __databricks_internal catalog. In my case, as I am full admin, I can do it, how...
- 3433 Views
- 3 replies
- 0 kudos
- 0 kudos
The tables do not have specific configuration, and the user who is receiving the error is in a group that has full permission in the INTERNAL catalog
- 0 kudos
- 1802 Views
- 2 replies
- 0 kudos
How can I throw an exception when a .json.gz file has multiple roots?
I have a situation where source files in .json.gz sometimes arrive with invalid syntax containing multiple roots separated by empty braces []. How can I detect this and thrown an exception? Currently the code runs and picks up only record set 1, and ...
- 1802 Views
- 2 replies
- 0 kudos
- 1773 Views
- 1 replies
- 0 kudos
Multiple Tables Migration from one workspace to another.
Hi all!I need to copy multiple tables from one workspace to another with metadata information. Is there any way to do it?Please reply as soon as possible.
- 1773 Views
- 1 replies
- 0 kudos
- 0 kudos
@HaripriyaP - Depends on your use case, Either of the below approach can be chosen. 1) DELTA CLONE(DEEP CLONE) to clone them to the new workspace. 2) Have the same cluster policy/Instance profile of the old workspace to access them in the new worksp...
- 0 kudos
- 3068 Views
- 3 replies
- 0 kudos
Delta Live Tables Materialised View Column Comment Error
While materialised view doc says MVs support columns comments, this does not seem like the case for MVs created by DLT. For example, when trying to add a comment to a MV created by DLT, it errors:Any ideas on when this will be fixed/supported?
- 3068 Views
- 3 replies
- 0 kudos
- 0 kudos
Just to close the loop here that it seems DLT generated MVs now support column comments.
- 0 kudos
- 1350 Views
- 1 replies
- 0 kudos
Installing dlt causing error
I'm trying to use the example in big book of engineering 2nd edition-final.pdf and I had an issue with the statementimport dltSo I created another cell and installed it and I noticed I was getting this error:"dataclass_transform() got an unexpected k...
- 1350 Views
- 1 replies
- 0 kudos
- 0 kudos
could you get the full error stack trace please
- 0 kudos
| User | Count |
|---|---|
| 1632 | |
| 791 | |
| 513 | |
| 349 | |
| 287 |
