Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
I am going through the labs for the CDW Migration Best Practices training on the Databricks Learning Portal and scored 75%. However, it still shows as "In Progress."Can anyone explain the criteria for marking the labs as complete? Do I need a higher ...
Hello,I'm trying to finish Delivery Specialization: CDW Migration Best Practices course but I could able to achieve only 50%.could you please share those answer notebooks to this email "betterahyosi2k20@gmail.com" ???
Hey Support Team,We are experiencing an issue with our Databricks warehouse where the auto-stop feature does not seem to be working as expected. Despite setting an idle timeout, the warehouse continues running after the configured auto-stop time has ...
We were able to diagnose and resolve the problem. The problem was caused due to a cube.js JDBC connection repeatedly connecting to our Databricks SQL Warehouse. Databricks SQL Warehouse does not scale down when there are repeated new connections made...
Hi Team,Suddenly our SQL warehouse stopped running, can you please help us how can we check why it stopped.
A similar error was reported, please check this thread to see if it helps resolve the issue.
I have applied row-level security (RLS) on the department column in the table so that users can only see data related to their own department. The security policy works perfectly when I query the table in Databricks SQL.Now, I have built a Databricks...
Hi @Akshay_Petkar , which credential mode did you use to publish the dashboard? In order to make access control work based on the viewer's credential, "Don't embed credentials" mode should be used. If you published the dashboard with "Embed credentia...
Hello,I attempted to update the SQL warehouse ID for all our dashboards using the Databricks SDK lakeview.update method. However, I mistakenly passed a dashboard object without the serialized_dashboard value, which resulted in all our dashboards beco...
Hi @Anton2 You can get the Dashboard-definition with the databricks cli. You need the id of the published Dashboard, which you see in the url, as example:Then, open a Terminal, run databricks cli, and save the output in a json file:databricks lakevie...
Dear all,(migrating for an on-premise Oracle ...)The question is in the subject: "What is the equivalent of Oracle's CLOB in Databricks" ?I saw that the "string" type can go up to 50 thousands characters, which is quite good in most of our cases, but...
Hello;Thanks for the answer.For the concatenation itself, it is not an issue.My question is "is Databricks supporting something bigger than the 'string' data-type" ? Thanks
Hi all,- migrating from an on-premise Oracle -Currently on Oracle, I have a "library" of let's say 300 tables to load, sequentially, based on views (some tables being fed potentially by several views, therefore the number of underlying views is highe...
The common scenario for data processing in Oracle is based on PL/SQL and cursors. In the case of PySpark we don't have such concept as cursors and iteration on data frames can lead to poor performance.I migrated Oracle to Databricks and I learned tha...
I am looking to monitor my SQL Warehouse, especially the 'Running Clusters' metric that is available in the monitoring tab of the warehouse. This shows the start and shut down time as well as the number of running clusters:The issue I have run into i...
Hi. I have a local MongoDB running on an EC2 instance in the same AWS VPC as my Databricks cluster but cannot get Databricks to talk to MongoDB. I've followed the guide at https://docs.databricks.com/aws/en/connect/external-systems/mongodb and have a...
I followed the official Databricks documentation("https://docs.databricks.com/en/_extras/notebooks/source/mongodb.html")to integrate MongoDB Atlas with Spark by setting up the MongoDB Spark Connector and configuring the connection string in my Datab...
Hi @vidya_kothavale ,Could you try to change "spark.mongodb.input.uri" to following?spark.read.format("mongodb").option("spark.mongodb.read.connection.uri"
Hello,I maintain a spark plugin with users who are moving from spark to databricks and want to use shared access mode. My library has a bunch of custom datasources and users are seeing errors when using them. Is there a way to write a custom datasour...
Hello @bcb44, Thanks for your question. Data Source V2 Relations: These are not supported in clusters that are configured with Table ACL or Credential Passthrough. Can you validate if ACL or Passthrough are enabled on the cluster? Also this should w...
Hello communityI am facing an issue when I export a dashboard to an HTML: the problem is that the HTML has white background but the exported tables have black colors, like the example I am showing below Does someone know how to handle this visualizat...
Hi All,Does anyone noticed if you "run all" the first time on the notebook, later if you click "run all below" on a cell, that wouldn't work anymore, and require to click "run all" again.It doesn't happen to me about couple of weeks ago, I used to ru...
Hello @Paddy_chu! This doesn't happen with every notebook, but it's likely due to cell dependencies. When you Run All, the final cell might modify or clear the data. Later, if you use Run All Below from a middle cell, it may not work if the required ...
Hey all,I have encountered a strange issue while validating my dashboard visual against the source table.The visual shows the count(distinct VAL) per day and when I ran a query that does the same calculation I get a difference of 84 (on average) and ...
Hi @yuvala, Are you using same query for the dashboard? could you please share it?
Hi All,How can I maintain 7 years of transactional data in delta table? Can I have log retention of 7 days, but data retention of 7 years?Appreciate your response.Thanks and regards Suhas
