Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
I am building a data pipeline using Delta Live table in Azure Databricks to move data from a raw data table to a feature table and model inference results table. However, I am concerned about the potential for duplication issues in future operations,...
Hi @Chengcheng Guo​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
Hi, I'm currently starting to use SQL Warehouse, and we have most of our lake in a compression different than snappy.How can I set the SQL warehouse to use a compression like gzip, zstd, on CREATE, INSERT, etc?Tried this:set spark.sql.parquet.compre...
Hi @Alejandro Martinez​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
I created the following model: which calls get_identifier_information() which is as follows: This is how I log the model And this is the error I am running into:RuntimeError: It appears that you are attempting to reference SparkContext from a broadc...
Hi @Nikhil Gajghate​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
Hello, we're receiving an error when running glue jobs to try and connect to and read from a Databricks SQL endpoint.Hello, we're receiving an error when running glue jobs to try and connect to and read from a Databricks SQL endpoint. An error occu...
Hello @Vidula Khanna​ @Debayan Mukherjee​ ,I wanted to give you an update that might be helpful for your future customers, we worked with @Pavan Kumar Chalamcharla​ and through lots of trial and error we figured out a combination that works for SQL e...
I am trying to identify errors coming from Databricks. So I can handle them in my code.Sometimes I get a descriptive error, that points me to the exact problem, but then if I run the exact same test, I sometimes get "request error: invalid operation ...
Can you share what command were you executing ? Also were you currently doing exception handling within your code with try/catch exception ? Can you also check the driver logs during the time error happened ? The driver logs should have more detail...
I notice it is very easily to get visualization from sql language inside Databricks. Say you run a SQL query which gives you a table, and you can easily use that table to do its visualization in terms of plots.How about in Python language when we hav...
Hi @THIAM HUAT TAN​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...
For purpose of posting video, can we post it on Google Drive, and share relevant link while submitting? Video will be made accessible to anyone with the link.Tagging @Karen Bajza-Terlouw​ and @Michelle Brain​
Hello! The video should be uploaded to and made publicly visible on YouTube, Vimeo, Facebook Video, or Youku so that it can playback on Devpost. This makes it easier for the judges
https://community.databricks.com/s/question/0D58Y00009fClizSAC/ssh-connection-with-paramiko i read in this post that you can help me with that problem, please can you give me more advice what i have to do to connect to an sftp through ssh
Hi!Could you please let me know what your current blocker is? Are you looking for a code snippet that can help you get file through sftp? Or if you are looking for the spark config that whitelist the port? Or if you are blocked by some other error?
I'd like to expose Delta Lake data to external customers via OData v4 APIs. What's the best way to do that?
Is the best answer to this still to implement the the OData intermediate service yourself? Or is there a better way now?
Availability of SQL Warehouse to Data Science and Engineering persona​Hi All,Now we can use SQL Warehouse in our notebook execution.It's in preview now and soon will be GA.
Hi team,I am looking for a way to find DBU cost for DLT clusters, does it get stored anywhere I have been looking into event_logs but did not find information related to cost. it does have cluster resource utilization details.here is what I found, co...
Hi @Chhaya Vishwakarma​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best an...
Hi @Dhruv Dalsania​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...
Explore the major offerings of the MetaMask WalletIf you participating in the crypto economy, you might have come across the term MetaMask Wallet once in a while. In this read, we will explore why we should go for MetaMask by acknowledging the major ...
Hi @Dhruv Dalsania​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...
I've triend this code in Databricks SQLcreate table people_db.GLAccount USING PARQUET LOCATION "abfss://datamesh@dlseu2dtaedwetldtlak9.dfs.core.windows.net/PricingAnalysis/rdv_60_134.vGLAccount.parquet"But I'm getting a "Invalid configuration v...
you can define 'data access configuration' in the admin panel.go to SQL warehouse settings -> Data Access configurationhttps://learn.microsoft.com/en-us/azure/databricks/sql/admin/data-access-configuration
