- 9345 Views
- 9 replies
- 2 kudos
Create a Workflow Schedule with varying Parameters
We aim to reduce the amount of notebooks we create to a minimum and instead make these fairly flexible. Therefore we have a Factory setup that takes in a parameter to varies the logic.However when it comes to Workflows we are forced to create multipl...
- 9345 Views
- 9 replies
- 2 kudos
- 2 kudos
We're also running into this issue on my team where having multiple cron schedules would be handy. We have some pipelines that we want run on multiple schedules, say to refresh data "Run every Sunday at midnight" and "Run on the first day of the mont...
- 2 kudos
- 1552 Views
- 0 replies
- 1 kudos
Unable to login to community addition account
Hello, I created a community edition account yesterday. Earlier today, I was still able to sign in and everything worked as expected. Now, whenever I try to sign in, I get the following error: "We were not able to find a Community Edition workspace w...
- 1552 Views
- 0 replies
- 1 kudos
- 1352 Views
- 0 replies
- 1 kudos
Accessing data in Databricks/Unity Catalog from a SharePoint list
Our organization is moving all of it's corporate master data to Databricks. Our business operations units need access to this data in various tools in the form of look-ups or drop-down lists in tools like Excel and SharePoint lists. It's simple eno...
- 1352 Views
- 0 replies
- 1 kudos
- 1006 Views
- 1 replies
- 1 kudos
Best Practices for Optimizing Delta Lake Performance
I’m currently working on a project using Delta Lake on Databricks and would like to optimize performance for both batch and streaming workloads.
- 1006 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @joshbuttler ,It's a big topic and it really depends on the use case. You can follow some guides/recommendations from databricks documentation:- Optimization recommendations on Databricks | Databricks on AWS- Use liquid clustering for Delta tables...
- 1 kudos
- 2894 Views
- 4 replies
- 0 kudos
Has anyone being able to connect to Databricks using ODBC?
I'm trying to connect to the Databricks cluster from the Excel using ODBC and following this official guide:https://docs.databricks.com/en/integrations/excel.html#connect-from-microsoft-excelBut the authentication is always failing, returning "500 In...
- 2894 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Vadimalk ,Could you check if you have following options enabled in simba driver configuration:
- 0 kudos
- 812 Views
- 0 replies
- 0 kudos
Account Level API GET Call requiring User-Agent Header
We recently Enabled Unity Catalog so using Account Level API and little stuck on a GET call. For some reason, just the GET call requires a User-Agent header while POST, PUT, PATCH.. all work without it. For workspace API, there was no need for User-A...
- 812 Views
- 0 replies
- 0 kudos
- 1541 Views
- 1 replies
- 1 kudos
Summit23
I have really enjoyed the summit so far! I am a biostatistics graduate student at Harvard and am learning so much here.
- 1541 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi,If you had a great time at the Summit, be sure to visit our community events portal for upcoming events and join your regional user group for meetups.Thanks,Anushree
- 1 kudos
- 1534 Views
- 1 replies
- 0 kudos
Summit 2024
It has been an exciting event that brought together industry leaders, researchers, and practitioners to explore the latest advancements in artificial intelligence and data science. With cutting-edge presentations, hands-on workshops, and networking o...
- 1534 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi,If you had a great time at the Summit, be sure to visit our community events portal for upcoming events and join your regional user group for meetups.Thanks,Anushree
- 0 kudos
- 3085 Views
- 1 replies
- 0 kudos
- 3085 Views
- 1 replies
- 0 kudos
- 0 kudos
You can leverage this code base. It works as expected using "next_page_token" parameter-Don't forget to mark this solution as correct if this helped you import requests token = 'your token' url = 'your URL' params = {'expand_tasks': 'true'} heade...
- 0 kudos
- 1300 Views
- 1 replies
- 0 kudos
Unity Catalog
Can MLflow Experiments be incorporated into Unity Catalog (may be in volume or somewhere in unity catalog) similar to models and feature tables?
- 1300 Views
- 1 replies
- 0 kudos
- 0 kudos
As of now Databricks supports DBFS, S3, and Azure Blob storage artifact locations. Our teams are working on intruducing Volumes as repository which might be coming soon
- 0 kudos
- 1196 Views
- 1 replies
- 1 kudos
Troubleshoot in Databricks
Hi everyone,I’m having trouble with a query in Databricks. My code seems correct, but I’m getting unexpected results. Can anyone help me troubleshoot this or provide tips for debugging in Databricks?Thanks in advance! locksmith
- 1196 Views
- 1 replies
- 1 kudos
- 1 kudos
It's hard to advice when you didn't provide any code. But you can try to use debugger inside notebook:
- 1 kudos
- 11798 Views
- 13 replies
- 3 kudos
Resolved! Unable to access Account console under Azure Databricks
I am the only user for the Azure enviornment and assigned global admin.For the resource group, I am the owner.For the Azure Databricks Service I am the service admin already.I would like to try to go into the Account console to make some changes for ...
- 11798 Views
- 13 replies
- 3 kudos
- 2648 Views
- 3 replies
- 0 kudos
INVALID_PARAMETER_VALUE: There are more than 1001 files. Please specify [LIMIT] to limit the num....
I am running Azure Databricks DBX Runtime 13.3 and I am seeing the following error in my workspace:Last execution failed# doesn't work in DBX Runtime 13.3 and above# limitation is it can't read more than 1001 filesdbutils.fs.ls("[path]")AnalysisExcep...
- 2648 Views
- 3 replies
- 0 kudos
- 0 kudos
@Retired_mod how to resolve this issue? I have faced the same issue on dbr 14.3 and dbr 15.4 version unity enabled cluster. Can you please let us know the workaround for it?
- 0 kudos
- 10009 Views
- 6 replies
- 2 kudos
Resolved! Error on UC Liquid Clustering
I know we have 4 keys max on cluster by () for both z-order and partition keys. I got some issues when adding 4 keys and 1 specific key triggers that error (I was not expecting as this is about Create Table) . Stats makes sense if you need to optimiz...
- 10009 Views
- 6 replies
- 2 kudos
- 2 kudos
Is Liquid cluster also helpful for high cardinality columns for a table Join?
- 2 kudos
- 3530 Views
- 2 replies
- 1 kudos
Resolved! Data bricks Bundle - 'include' section does not match any files.
Hello Community,I’m encountering the following error while working on my project:Error: ${var.FILE_NAME} defined in 'include' section does not match any files.Has anyone faced this issue before? I'm using variables to include specific files, but it s...
- 3530 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @AkhilVydyula ,You can start by updating your cli. I've seen some really weird errors with older cli versions.
- 1 kudos
