- 4610 Views
- 2 replies
- 1 kudos
Resolved! Data bricks Bundle - 'include' section does not match any files.
Hello Community,I’m encountering the following error while working on my project:Error: ${var.FILE_NAME} defined in 'include' section does not match any files.Has anyone faced this issue before? I'm using variables to include specific files, but it s...
- 4610 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @AkhilVydyula ,You can start by updating your cli. I've seen some really weird errors with older cli versions.
- 1 kudos
- 938 Views
- 0 replies
- 0 kudos
Creation of live consumption of MSSQL CDC via DLT
HelloI am bit surprised - but I don't see any default possibility of creating continous pipeline for fetching updates from MSSQL, and inserting them into delta lake - is it true - or am I missing something? To be clear - I know we can do that semi-li...
- 938 Views
- 0 replies
- 0 kudos
- 5283 Views
- 5 replies
- 6 kudos
Resolved! Looking for Databricks Study Materials/Resources
Hello,I’m looking for study materials or resources to help me learn Databricks more effectively. Any recommendations would be greatly appreciated!
- 5283 Views
- 5 replies
- 6 kudos
- 6 kudos
@igor779, @Rishabh-Pandey and @szymon_dybczak .Thank you all for the suggestions! Unfortunately, I don’t have access to content from partner companies and mainly rely on free resources for my studies.
- 6 kudos
- 1660 Views
- 0 replies
- 0 kudos
Issues with incremental data processing within Delta Live Tables
Hi!I have an problem with incrementally processing data within my Delta Live Tables (DLT) pipeline. I have a raw file (in Delta format) where new data is added each day. When I run my DLT pipeline I only want the new data to be processed. As an examp...
- 1660 Views
- 0 replies
- 0 kudos
- 6385 Views
- 4 replies
- 0 kudos
Resolved! Data Masking Techniques and Issues with Creating Tables
Hello Databricks Team,I understand that the mask function can be used to mask columns, but I have a few questions:When users with access use a masked TABLE to create a downstream TABLE, the downstream TABLE does not inherit the mask function directly...
- 6385 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @weilin0323, How are you doing today?As per my understanding, When creating a downstream table from a masked table, you’ll need to reapply the mask function to the necessary columns in the new table if you want to maintain the same level of data p...
- 0 kudos
- 1913 Views
- 1 replies
- 1 kudos
How to get a list of users in a workspace with Account Level APIs
Hi There,We are planning to move to Unity Catalog soon so started replacing workspace API with Account Level APIs. One use case I have is getting a list of users, pretty straight forward with workspace:{workspaceUrl}api/2.0/preview/scim/v2/Users?coun...
- 1913 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @unseen007 ,To get a list of all users in a given workspace, you should use api/2.0/preview/scim/v2/Users api endpoint. Why do you assume this API is inappropriat? Unity Catalog has nothing to do with this. One API let's you list user at account l...
- 1 kudos
- 2710 Views
- 1 replies
- 1 kudos
Resolved! Task runs missing from system database
I have a workflow with 11 tasks (each task executes one notebook) that run in sequence. The task was run on 9/1 and again today (9/10). I am working reporting task history and status using system table `system.lakeflow.job_task_run_timeline`. The...
- 2710 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @DavidKxx ,There are chances that the table has not been updated today.Check tomorrow.Here what the documentation says:
- 1 kudos
- 8953 Views
- 2 replies
- 0 kudos
Resolved! exclude (not like) filter using pyspark
I am trying to exclude rows with a specific variable when querying using pyspark but the filter is not working. Similar to the "Not like" function in SQL. e.g. not like '%var4%'. The part of the code that is not working is: (col('col4').rlike('var...
- 8953 Views
- 2 replies
- 0 kudos
- 3152 Views
- 3 replies
- 2 kudos
Resolved! Curl command working in 12.2 but not in 13.3
Hello, one my teammate is trying to put some obversability on our Databricks flows.When he tries to contact our Open Telemetry server, he gets a timeout.I had a look, and the same command (on the same Databricks workspace) works well with runtime 12....
- 3152 Views
- 3 replies
- 2 kudos
- 2 kudos
Hello,thank you for your answer.I tried to update library version and it was almost the solution.It realized it neither works in 12.2 but the output was pretty different and it misled me.Probably a network config to set up in the target server.
- 2 kudos
- 1148 Views
- 0 replies
- 0 kudos
show document name in AI agent
Hi everyone!I have successfully deployed the Review App of my AI agent following those instructions: Create and log AI agents | Databricks on AWSHowever, one question came up regarding the sources (names). To be precise, is there a possibility to sho...
- 1148 Views
- 0 replies
- 0 kudos
- 13349 Views
- 10 replies
- 12 kudos
Resolved! how to include checkboxes in markdown cells in databricks notebook
Hi has anyone else tried to include checkboxes in markdown cells in databricks notebook?I believe I followed the correct way for checkbox: - [ ] and - [x]but the result I got is still no checkboxes.Please help! Thanks! %md #to do: - [x] static vari...
- 13349 Views
- 10 replies
- 12 kudos
- 12 kudos
Hi @teaholicI faced the same problem and found the ✓ and ✗ notation that did work for me. Hope that helps.
- 12 kudos
- 871 Views
- 0 replies
- 0 kudos
API Results filename have tomorrow's date
Hi, I'm downloading data in chunks using API api/2.0/preview/sql/queries Today I've realized hat, the chunks have different date.. For example results_2024-09-10T10_30_32Z_acf798b5-2a5c-474f-a3b0-b83a2a8eb35a.csv I'm in the UTC timezone, I know that...
- 871 Views
- 0 replies
- 0 kudos
- 13077 Views
- 9 replies
- 1 kudos
GeoPandas Insall
hi,I cannot install geopandas in my notebook, ive tried all different forms of generic fix, pip installs etc but always get this error:CalledProcessError: Command 'pip --disable-pip-version-check install geopandas' returned non-zero exit status 1.---...
- 13077 Views
- 9 replies
- 1 kudos
- 1 kudos
@brian999 - Conda is subjected to commercial licensing. Referenced here: https://docs.databricks.com/en/archive/legacy/conda.html
- 1 kudos
- 845 Views
- 0 replies
- 0 kudos
Free voucher awaited
Hi team , I cleared datalake fundamentals exam attended webinar and completed survey ( my id s100rab@gmail.com ) when can I expect voucher
- 845 Views
- 0 replies
- 0 kudos
- 803 Views
- 0 replies
- 0 kudos
Googles finds databricks community topics but databricks community search does not
Every time I ask Google a Databricks question, and a link comes up to this community, and I click the link, it goes right to the topic and I get to read the question and the answer. But then a pop-up appears asking me to login to the community. Since...
- 803 Views
- 0 replies
- 0 kudos
