- 1860 Views
- 1 replies
- 0 kudos
- 1860 Views
- 1 replies
- 0 kudos
- 0 kudos
You can leverage this code base. It works as expected using "next_page_token" parameter-Don't forget to mark this solution as correct if this helped you import requests token = 'your token' url = 'your URL' params = {'expand_tasks': 'true'} heade...
- 0 kudos
- 631 Views
- 1 replies
- 0 kudos
Unity Catalog
Can MLflow Experiments be incorporated into Unity Catalog (may be in volume or somewhere in unity catalog) similar to models and feature tables?
- 631 Views
- 1 replies
- 0 kudos
- 0 kudos
As of now Databricks supports DBFS, S3, and Azure Blob storage artifact locations. Our teams are working on intruducing Volumes as repository which might be coming soon
- 0 kudos
- 6872 Views
- 13 replies
- 3 kudos
Resolved! Unable to access Account console under Azure Databricks
I am the only user for the Azure enviornment and assigned global admin.For the resource group, I am the owner.For the Azure Databricks Service I am the service admin already.I would like to try to go into the Account console to make some changes for ...
- 6872 Views
- 13 replies
- 3 kudos
- 1134 Views
- 3 replies
- 0 kudos
INVALID_PARAMETER_VALUE: There are more than 1001 files. Please specify [LIMIT] to limit the num....
I am running Azure Databricks DBX Runtime 13.3 and I am seeing the following error in my workspace:Last execution failed# doesn't work in DBX Runtime 13.3 and above# limitation is it can't read more than 1001 filesdbutils.fs.ls("[path]")AnalysisExcep...
- 1134 Views
- 3 replies
- 0 kudos
- 0 kudos
@Retired_mod how to resolve this issue? I have faced the same issue on dbr 14.3 and dbr 15.4 version unity enabled cluster. Can you please let us know the workaround for it?
- 0 kudos
- 5713 Views
- 6 replies
- 1 kudos
Resolved! Error on UC Liquid Clustering
I know we have 4 keys max on cluster by () for both z-order and partition keys. I got some issues when adding 4 keys and 1 specific key triggers that error (I was not expecting as this is about Create Table) . Stats makes sense if you need to optimiz...
- 5713 Views
- 6 replies
- 1 kudos
- 1 kudos
Is Liquid cluster also helpful for high cardinality columns for a table Join?
- 1 kudos
- 1112 Views
- 2 replies
- 1 kudos
Resolved! Data bricks Bundle - 'include' section does not match any files.
Hello Community,I’m encountering the following error while working on my project:Error: ${var.FILE_NAME} defined in 'include' section does not match any files.Has anyone faced this issue before? I'm using variables to include specific files, but it s...
- 1112 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @AkhilVydyula ,You can start by updating your cli. I've seen some really weird errors with older cli versions.
- 1 kudos
- 291 Views
- 0 replies
- 0 kudos
Creation of live consumption of MSSQL CDC via DLT
HelloI am bit surprised - but I don't see any default possibility of creating continous pipeline for fetching updates from MSSQL, and inserting them into delta lake - is it true - or am I missing something? To be clear - I know we can do that semi-li...
- 291 Views
- 0 replies
- 0 kudos
- 479 Views
- 0 replies
- 0 kudos
Issues with incremental data processing within Delta Live Tables
Hi!I have an problem with incrementally processing data within my Delta Live Tables (DLT) pipeline. I have a raw file (in Delta format) where new data is added each day. When I run my DLT pipeline I only want the new data to be processed. As an examp...
- 479 Views
- 0 replies
- 0 kudos
- 526 Views
- 1 replies
- 1 kudos
How to get a list of users in a workspace with Account Level APIs
Hi There,We are planning to move to Unity Catalog soon so started replacing workspace API with Account Level APIs. One use case I have is getting a list of users, pretty straight forward with workspace:{workspaceUrl}api/2.0/preview/scim/v2/Users?coun...
- 526 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @unseen007 ,To get a list of all users in a given workspace, you should use api/2.0/preview/scim/v2/Users api endpoint. Why do you assume this API is inappropriat? Unity Catalog has nothing to do with this. One API let's you list user at account l...
- 1 kudos
- 842 Views
- 1 replies
- 1 kudos
Resolved! Task runs missing from system database
I have a workflow with 11 tasks (each task executes one notebook) that run in sequence. The task was run on 9/1 and again today (9/10). I am working reporting task history and status using system table `system.lakeflow.job_task_run_timeline`. The...
- 842 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @DavidKxx ,There are chances that the table has not been updated today.Check tomorrow.Here what the documentation says:
- 1 kudos
- 1773 Views
- 2 replies
- 0 kudos
Resolved! exclude (not like) filter using pyspark
I am trying to exclude rows with a specific variable when querying using pyspark but the filter is not working. Similar to the "Not like" function in SQL. e.g. not like '%var4%'. The part of the code that is not working is: (col('col4').rlike('var...
- 1773 Views
- 2 replies
- 0 kudos
- 868 Views
- 3 replies
- 2 kudos
Resolved! Curl command working in 12.2 but not in 13.3
Hello, one my teammate is trying to put some obversability on our Databricks flows.When he tries to contact our Open Telemetry server, he gets a timeout.I had a look, and the same command (on the same Databricks workspace) works well with runtime 12....
- 868 Views
- 3 replies
- 2 kudos
- 2 kudos
Hello,thank you for your answer.I tried to update library version and it was almost the solution.It realized it neither works in 12.2 but the output was pretty different and it misled me.Probably a network config to set up in the target server.
- 2 kudos
- 242 Views
- 0 replies
- 0 kudos
show document name in AI agent
Hi everyone!I have successfully deployed the Review App of my AI agent following those instructions: Create and log AI agents | Databricks on AWSHowever, one question came up regarding the sources (names). To be precise, is there a possibility to sho...
- 242 Views
- 0 replies
- 0 kudos
- 2645 Views
- 11 replies
- 3 kudos
Resolved! Unity catalog implementation
Hello Databricks Community,We are in the process of planning a Unity Catalog implementation for our organization, and I'd like to seek input on some architectural decisions. We're considering various approaches to workspace separation, storage accoun...
- 2645 Views
- 11 replies
- 3 kudos
- 3 kudos
Hi @prad18 ,I'm glad the previous response was helpful! Let's address your remaining questions:Cost Differences Between Single vs. Multiple Azure Storage Accounts: The cost difference between using a single Azure storage account for both Unity Catalo...
- 3 kudos
- 6235 Views
- 9 replies
- 1 kudos
GeoPandas Insall
hi,I cannot install geopandas in my notebook, ive tried all different forms of generic fix, pip installs etc but always get this error:CalledProcessError: Command 'pip --disable-pip-version-check install geopandas' returned non-zero exit status 1.---...
- 6235 Views
- 9 replies
- 1 kudos
- 1 kudos
@brian999 - Conda is subjected to commercial licensing. Referenced here: https://docs.databricks.com/en/archive/legacy/conda.html
- 1 kudos
