Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
Hi community,I suddenly found myself confused, and this might sound like an obvious answer for some but not for me at least in this moment.I am not getting why companies use databricks SQL and Redshift at the same time?I mean, with databricks platfor...
Hi @eimis_pacheco We haven't heard from you since the last response from @-werners- , and I was checking back to see if her suggestions helped you. Or else, If you have any solution, please share it with the community, as it can be helpful to others...
Unable start SQL warehouse AWS. Warehouse is in starting state for a very long time and error is thrown
Can you please check on particular SQL warehouse instances type quota with cloud provider (Azure/AWS) . may be that particular instance type quota is over with your account . Same issue we have faced and requested AWS to increase the quota of that in...
So I've been trying to write a file to S3 bucket giving it a custom name, everything I try just ends up with the file being dumped into a folder with the specified name so the output is like ".../file_name/part-001.parquet". instead I want the file t...
Spark feature where to avoid network io it writes each shuffle partition as a 'part...' file on disk and each file as you said will have compression and efficient encoding by default.So Yes it is directly related to parallel processing !!
The client receives data from a third party as weekly "datadumps" of a MySQL database copied into an Azure Blob Storage account container (I suspect this is done manually, I also suspect the changes between the approx 7GB files are very small). I nee...
Hi @Sylvia VB​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers you...
I have a notebook where i want to use the workflow name and task name that it will be running under. How do i access these information?
Please take a look at these docs, I think they are what you need: https://docs.databricks.com/workflows/jobs/task-parameter-variables.html
WhereScape supports now Databricks as a target platform.
Super excited about learning how AI can help improve performance and throughout for our database customers.
Can't wait for the open testing this early fall!
Whenever I was working on a long notebook which is over 100 cells, long latency makes me not productive. Sometimes I should keep working on a same notebook for the context. How can I keep low latency?
I'd like to create an SQL table in a notebook that's visible to a specific run (session?) of a notebook. Meaning, even if 2 different users run that notebook at the same time, there should be no conflict. And the table should go away once the noteboo...
For now ‘CREATE TEMPORARY VIEW’ is the way to go. Once you read from it once, the following reads are going to be cached so it won’t be recomputed every time.
Does anyone have any experience migrating a gold layer from Azure Analysis Services (AAS) tabular structure to one centered around the lakehouse architecture? My company currently uses AAS to serve data to all levels of users across the company, but ...
Lakehouseiq looks really cool! Lakehouseiq is a Llm agent that builds sql queries from natural language. Ties into their other products like biz context lineage and previous queries.
Love the idea of creating and sharing apps in the Databricks Marketplace as shown during the keynote speech today, but how do I create them?
How do I time how long does my code run in DBX.
Hi, you can use the standard python timeit function. ''' %timeitprint("hello world")'''
I love Databricks! It’s the best modern data platform solution !!
