Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
I am going through the labs for the CDW Migration Best Practices training on the Databricks Learning Portal and scored 75%. However, it still shows as "In Progress."Can anyone explain the criteria for marking the labs as complete? Do I need a higher ...
Yes, it typically takes 10-15 minutes to generate a grade after checking the labs.
Hello Databricks Community,I have a hard time understanding how is Databricks SQL different from microsoft SQL ? Also, why does databricks provide spark SQL ? If you direct me to a well-written webpage or document its of immense help!Thanks,
Databricks SQL and Spark SQL are built for distributed big data analytic. Databricks SQL is great for business intelligence tools and uses Delta Lake for efficient data storage. Spark SQL works with Spark's programming features for data processing. U...
Hello there,Wasn't sure if this was just an error on my part, but I'm using a Databricks Pro SQL warehouse and unity catalogue to pull some data from my tables. I'm having this issue where whenever I try and use a wildcard operator with my LIKE claus...
Hi @nomnomnom543 ,Try this SELECT * FROM table_name WHERE LEFT(column_name, LENGTH('string')) = 'string';
Hello,I have an Azure sql warehouse serverless instance that I can connect to using databricks-sql-connector. But, when I try to use pyspark and jdbc driver url, I can't read or write.See my code belowdef get_jdbc_url(): # Define your Databricks p...
The error does not look specific to the warehouse that you are connecting to. The error message "Unrecognized conversion specifier [msg] starting at position 54 in conversion pattern" indicates that there is an issue with the logging configuration in...
With Databricks BI, Is there anyway to create a KPI or Measures that can be used in a visual? If so, how to achieved that one?
I have seen in the release notes of 15.3 that this was introduced and couldn't wrap my head around it.Does someone has an example of a plan before and after?Quote:Performance improvement for some window functionsThis release includes a change that im...
Before Optimization: Consider a query that calculates the sum of a column value partitioned by category without an ORDER BY clause or a window_frame parameter: SELECT category, SUM(value) OVER (PARTITION BY category) AS total_value FROM sales; ...
I have two tasks. The first task (Sample_Notebook) returns a JSON array (Input_List). Sample data in Input_List['key1':value1, 'key2':value2, 'key3':value3]The second task is a "For Each" task that executes a notebook for each entry in the Input_List...
To access each item of the iteration within the notebook of the second task in your Databricks workflow, you need to utilize the parameterization feature of the For Each task. Instead of trying to retrieve the entire list using dbutils.jobs.taskValue...
Hey everyone,So I managed to create a Streamlit app on Databricks, works fine deployed in the cloud.However, when I try to run it locally, it complains about "assert os.getenv('DATABRICKS_WAREHOUSE_ID'), "DATABRICKS_WAREHOUSE_ID must be set in app.ya...
Hi @AnaMocanu, How are you setting up: DATABRICKS_WAREHOUSE_ID? When running the app locally, the environment variables set in the app.yaml file might not be picked up automatically. You can manually set the environment variable in your local environ...
https://docs.databricks.com/en/query-federation/index.htmlAre there plans to provide Oracle support for Databricks on AWS lakehouse federation? Not sure if that's possible or not. Thanks!
Federation with Oracle is available https://learn.microsoft.com/en-us/azure/databricks/query-federation/oracle
Point map visualization datapoints labels do not let me input a specific column on them
Previously replied in https://community.databricks.com/t5/warehousing-analytics/datapoints-labels-on-a-point-map-visualization/td-p/101839
Users get confused when querying data with timestamps because UTC is not intuitive for many. It is possible to set TIME ZONE at query level or at SQL Warehouse level, but those options fail to address the need of multiple users working on the same wa...
it is possible to set the time zone at the session level using the SET TIME ZONE statement in Databricks SQL. This allows users to control the local timezone used for timestamp operations within their session. However, there is no direct option of us...
I can see that there is a project to support labels for points on the map but this is still in To do status
I'm trying to use Databricks Apps to host a Streamlit app to serve some interactive applicaton. I face two limitations:1. In environment for App I see it using certain python version, but how to update it to use any other version?It is already set to...
Hi @atikiwala , Good Day! Python 3.11 is currently the only version we support. We are thinking of adding additional options in the future. Would love to hear your feedback on this - https://docs.databricks.com/en/resources/ideas.html#submit-product-...
In a Databricks AI/BI dashboard, I have a field with multiple categories (e.g., district-wise sales with 50 districts). How can I display only the top few categories (like the top 10) based on a specific metric such as sales?
hey @migq2 , @maks in the AI/BI dashboards in your data, add a limit parameter like:select all from my_table limit :limit_number to all your tables. when you're on canvas and adding visualizations, add a filter and create a parameter with single val...
From the DLT documentation it seems that the LIVE TABLE is conceptually the same as MATERIALIZED VIEW. When should I use one over another?
@ImranA and @igorstar I repost my response here again:to create materialized views, you could use CREATE OR REFRESH LIVE TABLE however according to the official docs: The CREATE OR REFRESH LIVE TABLE syntax to create a materialized view is deprecat...
