- 70 Views
- 1 replies
- 0 kudos
DLT Query History
Hi guys,I can see that in DLT pipelines we have query history section where we can see the duration of each tables and number of rows read.Is this information stored somewhere in the systems catalog? can I query this information?
- 70 Views
- 1 replies
- 0 kudos
- 0 kudos
This might be available in the query history system table https://docs.databricks.com/aws/en/admin/system-tables/query-history
- 0 kudos
- 670 Views
- 1 replies
- 0 kudos
Bar chart data labels in percent
Hello, I am currently working with bar visualizations in a new workbook editor. When I use labels, I can see the count of rows returned, and hovering over them shows the percentage of the two values returned. How can I make the percentage display on ...
- 670 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @gm_co! Were you able to sort this out?You can display % in two ways: In the General settings, check the box for Normalize values to percentage.As you have enabled Labels, just set the Data labels to {{ @@yPercent }}. This will show the percent...
- 0 kudos
- 12046 Views
- 17 replies
- 13 kudos
How to know, who started a job run?
Hi there!We have different jobs/workflows configured in our Databricks workspace running on AWS and would like to know who actually started the job run? Are they started by a user or a service principle using curl?Currently one can only see, who is t...
- 12046 Views
- 17 replies
- 13 kudos
- 268 Views
- 6 replies
- 3 kudos
Resolved! How can I use Terraform to assign an external location to multiple workspaces?
How can I use Terraform to assign an external location to multiple workspaces?When I create an external location with Terraform, I do not see any option to directly link workspaces. it also only links to the workspace of the databricks profile that I...
- 268 Views
- 6 replies
- 3 kudos
- 3 kudos
@Walter_C I think you need to use databricks_workspace_binding resource for that multi-workspace binding. I was able to achieve it in Terraform. The resource docs seem to agree with result that I have. My Databricks runs on Google Cloud.My Terraform...
- 3 kudos
- 689 Views
- 4 replies
- 0 kudos
Unable to read Delta Table using external tools
I am using the new credential vending API to get tokens and url for my tables in Unity Catalog.I get the token, url and I am able to scan the folder using read_parquet, but NOT any Delta Lake functions. Not TableExists, scan_delta or delta_scan from ...
- 689 Views
- 4 replies
- 0 kudos
- 0 kudos
When copying a problematic delta table and reading the copy the issue disappear, it seems to be related to the new delta checkpointPolicy (v2) not supported by the rust implementation of delta but fine with the scala/java one (deltalake vs delta-spar...
- 0 kudos
- 194 Views
- 1 replies
- 1 kudos
Power bi connection
https://databrickster.medium.com/databricks-will-refresh-your-powerbi-semantic-model-both-dataset-metadata-and-data-4e8279e10b8eAbove is what I am trying to do, I already created the connection apparently all looks good, I added the task to my workfl...
- 194 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @carlos_tasayco, it looks like the issue is due to missing permissions in Power BI. If the role is set to Viewer, try updating it to Contributor or higher. You may need help from the Power BI admin to adjust the access. This should resolve the err...
- 1 kudos
- 242 Views
- 1 replies
- 0 kudos
AWS account databricks account / subscriptions disable before 14days trial over
I had registered trial version for 14days thru AWS, its been deactivated now before 14 days over. when i am going thru manage subscription and it is asking to register product and redirecting me to create and link account which was already there and...
- 242 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @anand010210! Were you able to resolve the issue? It seems that your AWS account is still linked to an active Databricks subscription, even though it's been disabled. I recommend checking your AWS Marketplace subscription to see if Databricks i...
- 0 kudos
- 1890 Views
- 6 replies
- 1 kudos
Resolved! Override ruff linter settings for notebook cells
How can I override the ruff linter settings for my notebooks?I have various projects/git folders in my workspace, and oftentimes, they represent different teams and thus different sets of code formatting patterns. I would like to override the default...
- 1890 Views
- 6 replies
- 1 kudos
- 1 kudos
Databricks defines a “project” as a structured collection of assets (jobs, pipelines, notebooks) organized using Databricks Asset Bundles, which use infrastructure-as-code templates to standardize deployments. Hope this helps. Lou.
- 1 kudos
- 81 Views
- 0 replies
- 0 kudos
Best practice for unified cloud cost attribution (Databricks + Azure)?
Hi! I’m working on a FinOps initiative to improve cloud cost visibility and attribution across departments and projects in our data platform. We do tagging production workflows on department level and can get a decent view in Azure Cost Analysis by f...
- 81 Views
- 0 replies
- 0 kudos
- 402 Views
- 6 replies
- 2 kudos
Resolved! Continuous workflow job creating new job clusters?
Hey.I am testing a continuous workflow job which executes the same notebook, so rather simple and it works well. It seems like it re-creates the job cluster for every iteration, instead of just re-using the one created at the first execution. Is that...
- 402 Views
- 6 replies
- 2 kudos
- 2 kudos
Thank you all for your answers!I did use dbutils.notebook.run() inside a while-loop at first but ultimately would run into OOM errors, even if I tried writing in a clearing of cache after each iteration. I'm curious @RefactorDuncan, if you don't mind...
- 2 kudos
- 192 Views
- 2 replies
- 0 kudos
Resolved! Databricks External table row maximum size
Hi Databricks Team/ Community,We have created a Databricks External table on top of ADLS Gen 2. Both parquet and delta tables. we are loading nested json structure into a table. Few column will have huge nested json data. Im getting results too large...
- 192 Views
- 2 replies
- 0 kudos
- 0 kudos
@Mano99 ktagwrote:Hi Databricks Team/ Community,We have created a Databricks External table on top of ADLS Gen 2. Both parquet and delta tables. we are loading nested json structure into a table. Few column will have huge nested json data. Im getting...
- 0 kudos
- 1414 Views
- 2 replies
- 0 kudos
VACUUM with Azure Storage Inventory Report is not working
Could someone please advise regarding VACUUM with Azure Storage Inventory Report as i have failed to make it work.DBR 15.4 LTS, VACUUM command is being run with USING INVENTORY clause, as follows:VACUUM schema.table USING INVENTORY ( select 'https://...
- 1414 Views
- 2 replies
- 0 kudos
- 0 kudos
After additional investigation it turned out the proper "fully-qualified-URL" path should be'dbfs:/mnt/...''dbfs:/mnt/{endpoint}/' || ir.Name as path,and not 'https://xxx.blob.core.windows.net/' || ir.Name as path,
- 0 kudos
- 317 Views
- 2 replies
- 0 kudos
Costs from cost managem azure portal are not allligned with costs calculated from usage system table
Hello,the costs regarding the databricks service from cost management in azure portal (45,869...) are not allligned with costs calculated from usage system table (75,34). The costs from the portal are filtered based on the desired period (usage_date ...
- 317 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @vziog, The Azure portal typically aggregates costs from various billing categories (such as DBUs, infrastructure, storage, and networking) based on usage logs and pricing. On the other hand, the query designed by you extracts detailed cost estima...
- 0 kudos
- 162 Views
- 3 replies
- 0 kudos
Reading from an S3 bucket using boto3 on serverless cluster
Hello All,I am trying to read a CSV file from my S3 bucket in a notebook running on serverless.I am using the two standard functions below, but I get a credentials error (Error reading CSV from S3: Unable to locate credentials).I don't have this issu...
- 162 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @petitregny ,The issue you’re encountering is likely due to the access mode of your cluster. Serverless compute uses standard/shared access mode, which does not allow you to directly access AWS credentials (such as the instance profile) in the sam...
- 0 kudos
- 146 Views
- 1 replies
- 0 kudos
Resolved! global temp view issue
I am following the doc1 and doc2 but I am getting an error.I was under the impression from the documentation that it is doable in pure sql. What am I doing wrong?I know how to do this in python using dataframe API and I am not looking for that soluti...
- 146 Views
- 1 replies
- 0 kudos
- 0 kudos
it is missing ;
- 0 kudos
| User | Count |
|---|---|
| 1611 | |
| 768 | |
| 348 | |
| 286 | |
| 252 |
