Hi,How to get the distinct count from the below listed image,keywords = column nametable = appCatalogkeywords (column)"[""data"",""cis"",""mining"",""financial"",""pso"",""value""]""[""bzo"",""employee news"",""news""]""[""core.store"",""fbi""]""[""d...
Hi @SivaPK , Try this :-
# Assuming you have a DataFrame named 'df' with a column 'keywords'from pyspark.sql.functions import explode, col
# Explode the array column 'keywords'exploded_df = df.select(explode(col("keywords")).alias("keyword"))
# Count...
I'd like to create Gantt charts using the dashboard function. It seems like this could be possible by adding some additional parameters in the bar plot functionality, but I don't see how to do it currently (if there is a way, would love an example!)....
Is there a preferred method for hosting an odbc connection to a warehouse on a server for use by a report server (SSRS/PBIRS)? I know the odbc driver doesn't support pass-through authentication, so is there a way to configure it with an unattended ac...
Hi @bradleyjamrozik , Certainly! Configuring an ODBC connection for a report server (such as SSRS or PBIRS) involves several considerations.
Let’s explore some options:
Direct ODBC Connection:
You can create an ODBC data source on the server whe...
I'm deploying a new workspace for testing the deployed notebooks. But when trying to import the python files as module in the newly deployed workspace, I'm getting an error saying "function not found".Two points to note here:1. If I append absolute p...
Hi @Kaniz, I see your suggestion to append the necessary path to the sys.path. I'm curious if this is the recommendation for projects deployed via Databricks Asset Bundles. I want to maintain a project structure that looks something like this:project...
I want to create a simple application using Spark Structured Streaming to alert users (via email, SMS, etc.) when stock price data meets certain requirements.I have a data stream: data_streamHowever, I'm strugging to address the main issue: how users...
Greetings,I use Databricks through Azure. Frequently when I'm working in the SQL Editor, the tab button fails to indent-- instead, it forces my cursor to a seemingly random part of the page. It's been doing this since when I started working in the pl...
Same issue here, sometimes it works by selecting a single but complete line and then hitting tab. If it works, then indentation works on multiple lines again as well. However, the single line select doesn't always work either. Pretty inconsistent beh...
Hi, I would like to know whether there is a feasibility/options available to connect to databricks sql endpoint using a programming language like java/scala/c#. I can see JDBC URL, but would like to whether it can be considered as any other jdbc conn...
I found a similar question on Stackoverflow https://stackoverflow.com/questions/77477103/ow-to-properly-connect-to-azure-databricks-warehouse-from-c-sharp-net-using-jdb
I am having issues with Datatbricks SQL and dbt at the moment. When running a query a string column is somehow converted to a number. Does anybody have any idea why this would be happenning?
Hi, currently we are using Power BI as the semantic layer because it allows us to build custom measures to do aggregates and business logic calculation, and provides native connection to Excel. I am thinking to move these logics to Databricks using S...
Hi,I'm implementing a Databricks connector using the ODBC driver and currently working on the functionality to Cancel an ongoing SQL statement.However, I can't seem to find any ODBC function or SQL function to do so.The only other alternative I see i...
I am trying to connect to DBeaver from Databricks and getting this error message:[Databricks][DatabricksJDBCDriver](500593) Communication link failure. Failed to connect to server. Reason: javax.net.ssl.SSLHandshakeException: PKIX path building faile...
Hello,I created a sql warehouse (cluster size = 2X-Small) and I wanted to use it to execute a query using the sql query api:- url : https://databricks-host/api/2.0/preview/sql/statements- params = {'warehouse_id': 'warehouse_id','statement': 'SELECT ...
@Yahya24 can you please remove preview in query, they are not in preview any more "/api/2.0/sql/statements/", you should see json response, can you please check drop down menu and change to json, some times it may be setted into text, but usual respo...
I've been doing some testing with Partitions vs Z-Ordering to optimize the merge process.As the documentation says, tables smaller than 1TB should not be partitioned and can benefit from the Z-Ordering process to optimize the reading process.Analyzin...
Hi @gmiguel,
I've been testing with Partitions vs. Z-Ordering to optimize the merge process.
As the documentation says, tables smaller than 1TB should not be partitioned and can benefit from the Z-Ordering process to optimize the reading process.Ana...
How do you handle reporting monthly trends within a data lakehouse? Can this be done with timetravel to get the table state at the end of each month or is it better practice to build a data warehouse with SCD types? We are new to databricks and lak...
@Mswedorske IMO it would be better to use SCD.When you do VACUUM on a table, it removes the data files that are necessary for Time Travel, so it's not a best choice to rely on Time Travel.
