Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
I'm working on returning some statistics for a workflow where the date ranges can either be entered manually or when it runs on a schedule it is setting it's date ranges as a task value. I'm able to return the one's where it has been manually set usi...
When I use following java code to get namespace from AWD Databricks: import java.sql.Connection; import java.sql.DriverManager; import java.sql.ResultSet; import java.sql.Statement; import java.util.Properties; public class DatabricksJDBCExample { ...
@charlie_cai database is not a valid configuration parameter available in the jdbc string. You can use ConnCatalog and ConnSchema to provide this information.This is also documented here - https://docs.databricks.com/en/integrations/jdbc-odbc-bi.html...
I am using Power BI Desktop to connect to Unity Catalog using Delta Sharing. When I connect, I enter my endpoint, enter my bearer token, browse the catalog and can see my tables. But when I try to preview or load a table, I get the error "access to r...
It turned out we had to allow my IP address on the storage account used by Unity Catalog. I guess i wasn't expecting to need that for Delta Sharing, but that indeed did fix the problem.
Hi,While running a notebook during a nightrun on Azure Databricks, it got stuck on Initializing RocksDB. We are not using any streaming data nor have enabled RocksDB. Anyone has any clue how to disable RocksDB or prevent this in the future?Thanks!
Is there a way to build (virtual) multidimensional models in Databricks and query them via MDX? Or any third party addon?
I am using the JDBC driver to load comments saved in Databricks, associated to tables and columns.Comments saved in Chinese are returned in the bad encoding. I useDatabaseMetaData.getColumns().getComments()
@dprutean Can you share an example with the expected results vs the actual results?
Hi community,I suddenly found myself confused, and this might sound like an obvious answer for some but not for me at least in this moment.I am not getting why companies use databricks SQL and Redshift at the same time?I mean, with databricks platfor...
Hi @eimis_pacheco We haven't heard from you since the last response from @-werners- , and I was checking back to see if her suggestions helped you. Or else, If you have any solution, please share it with the community, as it can be helpful to others...
Unable start SQL warehouse AWS. Warehouse is in starting state for a very long time and error is thrown
Can you please check on particular SQL warehouse instances type quota with cloud provider (Azure/AWS) . may be that particular instance type quota is over with your account . Same issue we have faced and requested AWS to increase the quota of that in...
So I've been trying to write a file to S3 bucket giving it a custom name, everything I try just ends up with the file being dumped into a folder with the specified name so the output is like ".../file_name/part-001.parquet". instead I want the file t...
Spark feature where to avoid network io it writes each shuffle partition as a 'part...' file on disk and each file as you said will have compression and efficient encoding by default.So Yes it is directly related to parallel processing !!
The client receives data from a third party as weekly "datadumps" of a MySQL database copied into an Azure Blob Storage account container (I suspect this is done manually, I also suspect the changes between the approx 7GB files are very small). I nee...
Hi @Sylvia VB​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers you...
I have a notebook where i want to use the workflow name and task name that it will be running under. How do i access these information?
Please take a look at these docs, I think they are what you need: https://docs.databricks.com/workflows/jobs/task-parameter-variables.html
WhereScape supports now Databricks as a target platform.
Can't wait for the open testing this early fall!
Whenever I was working on a long notebook which is over 100 cells, long latency makes me not productive. Sometimes I should keep working on a same notebook for the context. How can I keep low latency?
I'd like to create an SQL table in a notebook that's visible to a specific run (session?) of a notebook. Meaning, even if 2 different users run that notebook at the same time, there should be no conflict. And the table should go away once the noteboo...
For now ‘CREATE TEMPORARY VIEW’ is the way to go. Once you read from it once, the following reads are going to be cached so it won’t be recomputed every time.
