Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
So I've been trying to write a file to S3 bucket giving it a custom name, everything I try just ends up with the file being dumped into a folder with the specified name so the output is like ".../file_name/part-001.parquet". instead I want the file t...
Spark feature where to avoid network io it writes each shuffle partition as a 'part...' file on disk and each file as you said will have compression and efficient encoding by default.So Yes it is directly related to parallel processing !!
The client receives data from a third party as weekly "datadumps" of a MySQL database copied into an Azure Blob Storage account container (I suspect this is done manually, I also suspect the changes between the approx 7GB files are very small). I nee...
Hi @Sylvia VB​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers you...
I have a notebook where i want to use the workflow name and task name that it will be running under. How do i access these information?
Please take a look at these docs, I think they are what you need: https://docs.databricks.com/workflows/jobs/task-parameter-variables.html
WhereScape supports now Databricks as a target platform.
Super excited about learning how AI can help improve performance and throughout for our database customers.
Can't wait for the open testing this early fall!
Whenever I was working on a long notebook which is over 100 cells, long latency makes me not productive. Sometimes I should keep working on a same notebook for the context. How can I keep low latency?
I'd like to create an SQL table in a notebook that's visible to a specific run (session?) of a notebook. Meaning, even if 2 different users run that notebook at the same time, there should be no conflict. And the table should go away once the noteboo...
For now ‘CREATE TEMPORARY VIEW’ is the way to go. Once you read from it once, the following reads are going to be cached so it won’t be recomputed every time.
Does anyone have any experience migrating a gold layer from Azure Analysis Services (AAS) tabular structure to one centered around the lakehouse architecture? My company currently uses AAS to serve data to all levels of users across the company, but ...
Lakehouseiq looks really cool! Lakehouseiq is a Llm agent that builds sql queries from natural language. Ties into their other products like biz context lineage and previous queries.
Love the idea of creating and sharing apps in the Databricks Marketplace as shown during the keynote speech today, but how do I create them?
How do I time how long does my code run in DBX.
Hi, you can use the standard python timeit function. ''' %timeitprint("hello world")'''
I love Databricks! It’s the best modern data platform solution !!
Today was the first day of summit and learned about how different models can be deployed for different decision points. I'm curious to know what users have found to be model control issues regarding this. Thanks!
With multiple locations for items to exist, duplicate names make it hard to tell which one is which without opening.
@Eric Kieft​ :In Databricks, the Recents view shows the recently accessed notebooks, dashboards, and folders. However, it does not show the exact location of the item. To determine the location of an item in the Recents view, you can try the followin...
