- 2455 Views
- 2 replies
- 1 kudos
Resolved! Problem with SHOW GROUPS command
We have 3 environments (dev, qc and prod). In DEV and QC I can issue this command:SHOW GROUPS from a sql notebook and it will show the group I have created for each workspace. However in production this group is not displayed. This group in all 3 cas...
- 2455 Views
- 2 replies
- 1 kudos
- 1 kudos
Thanks SP_6721, That was exactly my issue. all good now. Have a good day
- 1 kudos
- 1443 Views
- 1 replies
- 0 kudos
The warehouse fails to start
Hi, everyone!Does anyone know the reason for this problem? It says that I do not need to do anything, but if I don't restart the warehouse manually, it will not start.Is there a way to fix it? Or at least, create monitoring for it? We have some jobs ...
- 1443 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @joao_augusto , Maybe you're hitting some quotas on AWS EC2 instances? Could you check it?Regarding monitoring, you can try to use API calls and check health status of you sql warehouses:List warehouses | SQL Warehouses API | REST API reference | ...
- 0 kudos
- 2450 Views
- 5 replies
- 1 kudos
serverless job compute error
My general question:Does a serverless compute job automatically scale?The reason I try serverless job - with Performance optimization Disabled option is to make job run effortless and cost effective.I don't like to do any tuning on spark at all. I di...
- 2450 Views
- 5 replies
- 1 kudos
- 1 kudos
I found a setting about 16 G vs 32 g, but that's is part of memory used by sparkhttps://learn.microsoft.com/en-us/azure/databricks/compute/serverless/dependencies#high-memoryIf you run into out-of-memory errors in your notebook, you can configure the...
- 1 kudos
- 1850 Views
- 3 replies
- 0 kudos
View alter permission for multiple members
Hi I've setup the gold layer for my client by creating views. Currently not using CI/CD for view deployment but it's on roadmap. Now I've another 2-3 people joining the team and they will be making minor updates to the views as a result of change req...
- 1850 Views
- 3 replies
- 0 kudos
- 0 kudos
I have the same situation and it's pretty frustration.If somebody creates a view or table, they seem to be the only person that can alter that view.Is there a way round this?I have tables created by a person who is not around and cannot do anythinig ...
- 0 kudos
- 3750 Views
- 3 replies
- 3 kudos
How to Upload Python Wheel Artifacts to a Volume from a DAB Run?
Hello,I'm currently working on a Databricks Assets Bundle (DAB) that builds and deploys a Python wheel package. My goal is to deploy this package to a Volume so that other DAB jobs can use this common library.I followed the documentation and successf...
- 3750 Views
- 3 replies
- 3 kudos
- 3 kudos
I've got the same issue as Jorge3. I've tried to upload a wheel file from my artifacts directory to a volume but couldn't do it. It tried it with Databricks CLI and databricks-sdk. In both cases I get the error "databricks.sdk.errors.platform.Permiss...
- 3 kudos
- 2325 Views
- 2 replies
- 3 kudos
asset bundle configuration
Hi GuysI want to use this approach to give more flexibility of my asset bundle. But when i try to implement this i have some error like unknown field. SomeOne, could show me an example ? Thanx, databricks-dab-repo/├── databricks.yml # Bundle definiti...
- 2325 Views
- 2 replies
- 3 kudos
- 3 kudos
Actually its works, i have declare all my clusters config in variables field permissions: - level: CAN_VIEW group_name: data-engineer - level: CAN_RUN group_name: data-engineer - level: CAN_MANAGE group_name: data-engineer - level: CAN_...
- 3 kudos
- 5530 Views
- 1 replies
- 0 kudos
Serverless costs, standard vs performance optimized
In the pricing page for Lakeflow Declarative Pipelines (formerly DLT), for serverless it shows a single cost of $0.35/DBU for both standard and performance optimized. But in the feature table below, it says standard is "Up to 70% cheaper than running...
- 5530 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @rcostanza !While the DBU price is the same for both modes, the Performance Optimized mode behaves quite differently:Performance Optimized Mode prioritizes speed and throughput. It aggressively scales resources to minimize processing time, whic...
- 0 kudos
- 2320 Views
- 3 replies
- 0 kudos
Spark read /write jdbc netsuite TIMESTAMP error NullPointerException
I've connected to Netsuite via Suite Analytics JDBC connection in a Databricks cluster. I'm seeing a NullPointerException because of the lastmodifieddate column in my netsuite classification table (ie. if I do Select * from classification I see the N...
- 2320 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @szymon_dybczakI'm not sure why my question looks like this, I thought I fixed it. I've connected to Netsuite via Suite Analytics JDBC connection in a Databricks cluster. I'm seeing a NullPointerException because of the lastmodifieddate column in ...
- 0 kudos
- 2977 Views
- 1 replies
- 1 kudos
Resolved! Query profiler
HI When something like power bi connects to Databricks and askes for data, perhaps with a native query, is there a log or profiler in Databricks itself to see what queries hit it?
- 2977 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @IONA ,Sure, you can see what queries has been send by Power BI using i.e Spark UI
- 1 kudos
- 1301 Views
- 1 replies
- 0 kudos
Resolved! Incremental issue in Materialized views - QUERY FINGERPRINT CHANGED
I have create a Materialized view in the gold layer via DLT pipeline. But the plan is not deterministic due to the reason below. Not able to understand what does this error means and how to resolve this. Can you please help ? planning_information": ...
- 1301 Views
- 1 replies
- 0 kudos
- 0 kudos
hi @Dharinip Cleaning up the shared log shows "Before" and "After" fingerprints. Hopefully can give us more info in how to proceed Before [ { "id": 12 }, { "qualifier": [] }, { "class": "GreaterThan", "num-children": 2, ...
- 0 kudos
- 2496 Views
- 4 replies
- 2 kudos
Resolved! Not able to create mount point in Databricks
Hi Everyone,I am trying to create mount point in Azure Databricks, but mount point creation is failing with below error message.java.lang.UnsupportedOperationException: Managing DBFS mounts is not supported on this type of compute resource. Use Unity...
- 2496 Views
- 4 replies
- 2 kudos
- 2 kudos
You'll need to edit the cluster's configuration. This involves navigating to the compute details UI, finding the "Access mode" setting, and selecting "No Isolation Shared". With dedicated and standard mode you will not be able to create mount points
- 2 kudos
- 1335 Views
- 3 replies
- 0 kudos
Resolved! Data Engineering Learning Plan
HelloThe learning plan for data engineering appears to have changed again. The description at the top of the learning plan still cites 4 modules before you take the associate exam, but the module names and content have all changed. I was half way thr...
- 1335 Views
- 3 replies
- 0 kudos
- 0 kudos
HelloI have raised a ticket but the description of the Data Engineering learning plan and the description of the Associate exam have not changed. They both still refer to the original 4 modules. Why have you replaced the modules but not the accompany...
- 0 kudos
- 2397 Views
- 4 replies
- 2 kudos
Databricks Running jobs are not visible
Hi guys ,I have a issue here . I dont know why suddenly I cannot view Jobs which are running . Basically if I set the time and change the run status as "Active" . No active jobs are being shown . Although If I set run status as "Completed" or anythi...
- 2397 Views
- 4 replies
- 2 kudos
- 4484 Views
- 5 replies
- 1 kudos
DLT pipelines in the same job sharing compute
If I have a job like this that orchestrates N DLT pipelines, what setting do I need to adjust so that they use the same compute resources between steps rather than spinning up and shutting down for each individual pipeline?
- 4484 Views
- 5 replies
- 1 kudos
- 1 kudos
@shan_chandra Hello, I have the same issue, where I have a job that uses serverless compute and in this job I do some different tasks and then I start a DLT pipeline, which also uses serverless compute and this means that the job again have to wait f...
- 1 kudos
- 2818 Views
- 4 replies
- 3 kudos
Resolved! ANALYZE command for stats collection - distinct_count difference for a column
I ran the command ANALYZE TABLE <TABLE_NAME> COMPUTE STATISTICS FOR COLUMNS 'COL1,COL2' to compute statistics for specific columns, but I noticed that the distinct counts returned by this command differ from the results obtained by running a direct C...
- 2818 Views
- 4 replies
- 3 kudos
- 3 kudos
HI @Sainath368 and @Waldar , Please find the below table which shows the difference between each queries which are causing the difference in counts and when to use which one. Query Type Accuracy Based On Why It DiffersSELECT DISTINCTExactFull table ...
- 3 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 553 | |
| 349 | |
| 287 |
