- 3848 Views
- 1 replies
- 0 kudos
Call databricks notebook from azure flask app
I have an Azure web app running flask web server. From flask server, I want to run some queries on the data stored in ADLS Gen2 storage. I already created Databricks notebooks running these queries. The flask server will pass some parameters in ...
- 3848 Views
- 1 replies
- 0 kudos
- 0 kudos
you can use databricks SDKhttps://docs.databricks.com/en/dev-tools/sdk-python.html#create-a-job
- 0 kudos
- 3266 Views
- 4 replies
- 0 kudos
Pyspark execution error
I am getting a error message when executing a simple pyspark code. Can anyone help me with this.
- 3266 Views
- 4 replies
- 0 kudos
- 0 kudos
Could you please share the entire error message?Are you running the code locally or on databricks?
- 0 kudos
- 1665 Views
- 1 replies
- 0 kudos
LLM using DataBrick
Is there any coding example for how to use LLM?
- 1665 Views
- 1 replies
- 0 kudos
- 0 kudos
I would like to share the following links https://www.databricks.com/product/machine-learning/large-language-models https://docs.databricks.com/en/large-language-models/index.html
- 0 kudos
- 6344 Views
- 5 replies
- 4 kudos
{{job.trigger.type}} not working and throws error on Edit Parameter from Job page
Following the instruction on the Job Parameter Dynamic values, I am able to use {{job.id}}{{job.name}}{{job.run_id}}{{job.repair_count}}{{job.start_time.[argument]}}However, when I set trigger_type as trigger_type: {{job.trigger.type}} and hit SAVE, ...
- 6344 Views
- 5 replies
- 4 kudos
- 4 kudos
Thanks everyone, I decided to use the Sparkcontext instead. dbutils.notebook.entry_point.getDbutils().notebook().getContext().toJson()
- 4 kudos
- 2580 Views
- 0 replies
- 0 kudos
AzureDevOps and Databricks Connection using managed identity or service principal
Hi All! Im in a project where i need to connect azure devops and databricks using managed identity to avoid the using of service account, PAT, etc.The thing is i cant move forward with the connection since i cannot take the ownership of the files wh...
- 2580 Views
- 0 replies
- 0 kudos
- 4157 Views
- 0 replies
- 0 kudos
Boomi integrating with Databricks
Hi Team,Is there any impact when integrating Databricks with Boomi as opposed to Azure Event Hub? Could you offer some insights on the integration of Boomi with Databricks?https://boomi.com/blog/introducing-boomi-event-streams/Regards,Janga
- 4157 Views
- 0 replies
- 0 kudos
- 21828 Views
- 6 replies
- 2 kudos
Resolved! Unity catalog issues
Hi all,I have recently enabled Unity catalog in my DBX workspace. I have created a new catalog with an external location on Azure data storage.I can create new schemas(databases) in the new catalog but I can't create a table. I get the below error wh...
- 21828 Views
- 6 replies
- 2 kudos
- 2 kudos
@Snoonan First of all, check the networking tab on the storage account to see if it's behind firewall. If it is, make sure that Databricks/Storage networking is properly configured (https://learn.microsoft.com/en-us/azure/databricks/security/network/...
- 2 kudos
- 2213 Views
- 1 replies
- 0 kudos
Help Refactor T-SQL Code to Databricks SQL
Hello CommunityCan someone help refactor the following T-SQL Code to Databricks SQLCONVERT(DECIMAL(26, 8), ISNULL(xxx.xxxxxxx * ISNULL(RH.xxxxx, 1 / NULLIF(ST.xxxxxx, 0)), ST.xxxxx)) AS AmountWhen I attempt to execute the above code I get the followi...
- 2213 Views
- 1 replies
- 0 kudos
- 0 kudos
You can use CAST instead.Eg:SELECT cast('2024' as int);
- 0 kudos
- 3956 Views
- 0 replies
- 0 kudos
Custom ENUM input as parameter for SQL UDF?
Hello - We're migrating from T-SQL to Spark SQL. We're migrating a significant number of queries."datediff(unit, start,end)" is different between these two implementations (in a good way). For the purpose of migration, we'd like to stay as consiste...
- 3956 Views
- 0 replies
- 0 kudos
- 7308 Views
- 2 replies
- 1 kudos
Nulls in Merge
If you are going to handle any null values in your MERGE condition, better watch out for your syntax #databricks
- 7308 Views
- 2 replies
- 1 kudos
- 4150 Views
- 3 replies
- 0 kudos
Ingest __databricks_internal catalog - PERMISSION DENIED
Good morning, I have a DLT process with CDC incremental load and I need to ingest the history as CDC transactions are only recent. To do this I need to ingest data in the __databricks_internal catalog. In my case, as I am full admin, I can do it, how...
- 4150 Views
- 3 replies
- 0 kudos
- 0 kudos
The tables do not have specific configuration, and the user who is receiving the error is in a group that has full permission in the INTERNAL catalog
- 0 kudos
- 2317 Views
- 2 replies
- 0 kudos
How can I throw an exception when a .json.gz file has multiple roots?
I have a situation where source files in .json.gz sometimes arrive with invalid syntax containing multiple roots separated by empty braces []. How can I detect this and thrown an exception? Currently the code runs and picks up only record set 1, and ...
- 2317 Views
- 2 replies
- 0 kudos
- 2222 Views
- 1 replies
- 0 kudos
Multiple Tables Migration from one workspace to another.
Hi all!I need to copy multiple tables from one workspace to another with metadata information. Is there any way to do it?Please reply as soon as possible.
- 2222 Views
- 1 replies
- 0 kudos
- 0 kudos
@HaripriyaP - Depends on your use case, Either of the below approach can be chosen. 1) DELTA CLONE(DEEP CLONE) to clone them to the new workspace. 2) Have the same cluster policy/Instance profile of the old workspace to access them in the new worksp...
- 0 kudos
- 3776 Views
- 3 replies
- 0 kudos
Delta Live Tables Materialised View Column Comment Error
While materialised view doc says MVs support columns comments, this does not seem like the case for MVs created by DLT. For example, when trying to add a comment to a MV created by DLT, it errors:Any ideas on when this will be fixed/supported?
- 3776 Views
- 3 replies
- 0 kudos
- 0 kudos
Just to close the loop here that it seems DLT generated MVs now support column comments.
- 0 kudos
- 1772 Views
- 1 replies
- 0 kudos
Installing dlt causing error
I'm trying to use the example in big book of engineering 2nd edition-final.pdf and I had an issue with the statementimport dltSo I created another cell and installed it and I noticed I was getting this error:"dataclass_transform() got an unexpected k...
- 1772 Views
- 1 replies
- 0 kudos
- 0 kudos
could you get the full error stack trace please
- 0 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 584 | |
| 349 | |
| 287 |
