- 90 Views
- 5 replies
- 0 kudos
Issue with Adding New Members to Existing Groups During Migration in User group Service Principle
Hi all,I have implemented a migration process to move groups from a source workspace to a target workspace using the following code. The code successfully migrates groups and their members to the target system, but I am facing an issue when it comes...
- 90 Views
- 5 replies
- 0 kudos
- 0 kudos
I have provided response in https://community.databricks.com/t5/get-started-discussions/migrating-service-principals-from-non-unity-to-unity-enabled/m-p/103017#M4679
- 0 kudos
- 37 Views
- 1 replies
- 1 kudos
Resolved! Why does a join on (df1.id == df2.id) result in duplicate columns while on="id" does not?
Why does a join with on (df1.id == df2.id) result in duplicate columns, but on="id" does not?I encountered an interesting behavior while performing a join on two Data frames. Here's the scenario: df1 = spark.createDataFrame([(1, "Alice"), (2, "Bob"),...
- 37 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Tanay , Your intuition is correct here. In Apache Spark, the difference in behavior between on (df1.id == df2.id) and on="id" in a join stems from how Spark resolves and handles column naming during the join operation.When you use the first synta...
- 1 kudos
- 25 Views
- 2 replies
- 0 kudos
Create multiple dashboard subscription with filters
Hi Databricks community, We developed a dashboard that surfaces up several important KPIs for each project we have.In the top filter, we select the project name and the time frame and the dashboard will present the relevant KPIs and charts. I can eas...
- 25 Views
- 2 replies
- 0 kudos
- 0 kudos
You can achieve this by setting up different schedules for each project and specifying the default filter values accordingly Create the Dashboard: Ensure your dashboard is set up with the necessary filters, including the project filter. Set Defau...
- 0 kudos
- 135 Views
- 3 replies
- 0 kudos
Databricks Champions Sign-Up Page Failing or Throwing Error
Following link is throwing error when tried to Sign-Uphttps://advocates.databricks.com/users/sign_up?join-code=community
- 135 Views
- 3 replies
- 0 kudos
- 0 kudos
Let me check on this internally, I am not sure if this actually requires invitation.
- 0 kudos
- 100 Views
- 5 replies
- 0 kudos
Não estou conseguindo logar na minha conta Databricks communit
Galera não consigo logar na minha conta Databricks Communit fala que meu email não tem nada criado nele, mas eu tenho essa conta a um bom tempo já e nunca me ocorreu isso, já até tentei criar uma outra conta com esse mesmo email, mas não consigo cria...
- 100 Views
- 5 replies
- 0 kudos
- 105 Views
- 2 replies
- 0 kudos
Calling the w.genie function throws a "API is not yet supported in the workspace" error. [0.39.0]
Hi everyone,I've been trying to call the databricks genie function, but even on the latest build, it throws the error stating: w.genie API is not yet supported in the workspace.Here is the output of the logs:> {> "content": "**REDACTED**"> }< {< "err...
- 105 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @gluedhawkeye ,I tested this on my own and getting the same error:This is the same code as used here, but they have an info "This script implements an experimental chatbot that interacts with Databricks' Genie API, which is currently in Private Pr...
- 0 kudos
- 72 Views
- 2 replies
- 0 kudos
Issue with Percentage Calculation in Power BI Using Databricks as Source
Hi everyone,I've created a financial summary report in Power BI, and my source is Databricks. I have created a view for each financial metric name along with the calculations. All my amount fields are accurate, but when calculating percentages, I’m g...
- 72 Views
- 2 replies
- 0 kudos
- 0 kudos
Hello Richie,In Databricks, you can use a combination of NULLIF and COALESCE functions to handle divide-by-zero scenarios effectively. Here's an example of how you can modify your percentage calculation: SELECT MetricNo, MetricName, Amo...
- 0 kudos
- 150 Views
- 4 replies
- 0 kudos
GRPC call are not getting through on Databricks 15.4 LTS
Hi Team,I have updated spark version from 3.3.2 to 3.5.0 and switched to Databricks 15.4 LTS from 12.2 LTS so as to get Spark 3.5 version on the Databricks compute. We have moved from uploading libraries on DBFS to uploading libraries to Volumes as 1...
- 150 Views
- 4 replies
- 0 kudos
- 0 kudos
And this was working before is it correct? When the init was hosted in DBFS
- 0 kudos
- 417 Views
- 6 replies
- 1 kudos
[JAVA_GATEWAY_EXITED] Java gateway process exited before sending its port number.
Hi Team,In a streamlit app (in databricks) while creating the spark session getting below error, this is happening when running the app via web link."[JAVA_GATEWAY_EXITED] Java gateway process exited before sending its port number"Below is the code u...
- 417 Views
- 6 replies
- 1 kudos
- 1 kudos
Per looking internally I see that Spark (Context) is not available in Apps The recommended way would be to use our available SDKs and connect to Clusters/DBSQL. No spark context is available- it’s meant to defer processing to other compute it can con...
- 1 kudos
- 247 Views
- 4 replies
- 2 kudos
Resolved! Understanding Autoscaling in Databricks: Under What Conditions Does Spark Add a New Worker Node?
I’m currently working with Databricks autoscaling configurations and trying to better understand how Spark decides when to spin up additional worker nodes. My cluster has a minimum of one worker and can scale up to five. I know that tasks are assigne...
- 247 Views
- 4 replies
- 2 kudos
- 2 kudos
Hi @h_h_ak ,Short Answer:Autoscaling primarily depends on the number of pending tasks.Workspaces on the Premium plan use optimized autoscaling, while those on the Standard plan use standard autoscaling.Long Answer:Databricks autoscaling responds main...
- 2 kudos
- 152 Views
- 1 replies
- 0 kudos
DESCRIBE Table and SHOW CREAT TABLE shows contradictory NULL constraints
SHOW CREATE TABLE provides correct NULL constraints details of each column where as DESCRIBE TABLE shows wrong NULL constraints details?
- 152 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, you are correct. The SHOW CREATE TABLE command provides accurate details about the NULL constraints for each column in a table, whereas the DESCRIBE TABLE command may show incorrect NULL constraints details. This discrepancy arises because SHOW ...
- 0 kudos
- 54 Views
- 0 replies
- 0 kudos
Things to Consider Before Selecting a Databricks Consultancy
What experience does the consultancy have with similar Databricks projects?Ensure they have relevant experience in your industry. Ask for examples of similar projects they've worked on.How do they manage Databricks costs?Inquire about their strategie...
- 54 Views
- 0 replies
- 0 kudos
- 335 Views
- 7 replies
- 1 kudos
Databricks cleanroom functionality and billing
I'm new to Databricks and have been tasked with exploring Databricks Clean Rooms. I'm a bit confused about how billing works for Clean Rooms and their overall functionality. Specifically, I'm curious about the following:Environment Hosting: Are Clean...
- 335 Views
- 7 replies
- 1 kudos
- 1 kudos
Is it adding foreign catalog table in datacleanroom feature also available after GA release, cause I tried it but was not able to see the foreign catalog in add data assets tab
- 1 kudos
- 1983 Views
- 2 replies
- 0 kudos
Unable to grant catalog access to service principal
Hi everyone,I created a service principals called TestServicePrincipal. I tried to grant the catalog access to the service principals, but the error mentioned that it could not find principal with name TestServicePrincipal. If I grant the access to s...
- 1983 Views
- 2 replies
- 0 kudos
- 0 kudos
The issue could be related to how the service principal is being resolved in your system. Unlike users, service principals are often registered in a directory (like Azure AD), and their names might not match what you’re using. Instead of using TestSe...
- 0 kudos
- 3096 Views
- 3 replies
- 0 kudos
Informatica ETLs
I'm delving into the challenges of ETL transformations, particularly moving from traditional platforms like Informatica to Databricks. Given the complexity of legacy ETLs, I'm curious about the approaches others have taken to integrate these with Dat...
- 3096 Views
- 3 replies
- 0 kudos
- 0 kudos
@DataYoga , you may explore the tool and services from Travinto Technologies . They have very good tools. We had explored their tool for our code coversion from Informatica, Datastage and abi initio to DATABRICKS , pyspark. Also we used for SQL quer...
- 0 kudos
