- 8112 Views
- 4 replies
- 0 kudos
Filter data by Date using where condition (< TargetDate) giving "Query returned no results"
Code is working good if data greater than target date (>) is selected :SELECT xyz.ID,xyz.Gender,xyz.geography,xyz.code,xyz.delivery_status,abc.department_codeFROM v.table1 as xyzleft join y.table2 as abconxyz.ID = abc.ID AND xyz.code = abc.cod...
- 8112 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Rishabh Shankar​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...
- 0 kudos
- 9788 Views
- 2 replies
- 1 kudos
Error 503 first byte timeout
Hi,I'm receiving error while logging in or signing up on CE.Error 503 first byte timeoutfirst byte timeoutError 54113Details: cache-bom4725-BOM 1659706650 579764974Varnish cache serverscreenshot attached below:- Thanks, any help is appreciated from t...
- 9788 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Ghazanfar Uruj​ Does @Prabakar Ammeappin​ answer help? If it does, would you be happy to mark it as best? If it doesn't, please tell us so we can help you.We'd love to hear from you.Thanks!
- 1 kudos
- 1250 Views
- 2 replies
- 2 kudos
Databricks Sql schedule button not saving
The schedule button isn't saving my schedule information for a databricks sql query. After I hit save and open the schedule again it has reverted to 'Never'. The query itself according to the past executions pane is not running according to the sched...
- 1250 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Wally Plourde​ Does @Rohit Rajendran​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 2 kudos
- 1567 Views
- 2 replies
- 0 kudos
- 1567 Views
- 2 replies
- 0 kudos
- 0 kudos
Hey there @IssacClose​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you....
- 0 kudos
- 4032 Views
- 2 replies
- 0 kudos
How do I get a list of the tables that I personally created?
I know that I can get a list of all of the table names in a given 'database' by using (if the 'database' was named "scratch"): show tables from scratchHow do I get a list just like that, but that only lists the tables that I created?
- 4032 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Barb Krienke​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...
- 0 kudos
- 2659 Views
- 2 replies
- 0 kudos
passing array as a parameter to PandasUDF
Hi Team,My python dataframe is as below.The raw data is quite a long series of approx 5000 numbers. My requirement is to go through each row in RawData column and calculate 2 metrics. I have created a function in Python and it works absolutely fine. ...
- 2659 Views
- 2 replies
- 0 kudos
- 0 kudos
Hello @Kausthub NP​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
- 0 kudos
- 14779 Views
- 4 replies
- 4 kudos
Using variables in Spark SQL
Is there a way to declare variables in Spark SQL like we do it in T-SQL?
- 14779 Views
- 4 replies
- 4 kudos
- 4 kudos
Hi @Hariharan Sambath​, We haven't heard from you on the last response from @Debayan Mukherjee​ and @Hubert Dudek​, and I was checking back to see if his suggestions helped you. Or else, If you have any solution, please share it with the community as...
- 4 kudos
- 1744 Views
- 2 replies
- 2 kudos
Resolved! DLT Cluster accessing to S3 bucket without Instance Profile attached
Hi Team,Can anyone please help me figure out how to configure Delta Live Tables Cluster accessing AWS S3 bucket without Instance profile defined in Cluster's JSON?The idea is, the user who is running the DLT Cluster has Storage Credentials and Extern...
- 1744 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Jason Nam​ , DLT and unity catalog are not integrated yet. The cluster-notebook setup uses UC and can access S3 but not the DLT jobs. Please check the limitations in this document (7th point):https://docs.databricks.com/release-notes/unity-catalo...
- 2 kudos
- 10562 Views
- 10 replies
- 10 kudos
Resolved! Is there a way to capture the notebook logs from ADF pipeline?
Hi,I would like to capture notebook custom log exceptions(python) from ADF pipeline based on the exceptions pipeline should got succeed or failed.Is there any mechanism to implement it. In my testing ADF pipeline is successful irrespective of the log...
- 10562 Views
- 10 replies
- 10 kudos
- 10 kudos
Hi SailajaB,Try this out.Notebook, once executed successfully return a long JSON formatted output. We need to specify appropriate nodes to fetch the output. In below screenshot we can see that when notebook ran it returns empName & empCity as output....
- 10 kudos
- 1666 Views
- 3 replies
- 5 kudos
Strategy for streaming ETL and Delta Lake before Delta Live Tables existed
What was the established architectural pattern for doing streaming ETL with Delta Lake before DLT was a thing? And incidentally, what approach would you take in the context of delta-oss today? The pipeline definitions would not have had to be declara...
- 1666 Views
- 3 replies
- 5 kudos
- 5 kudos
Hi @Veli-Jussi Raitila​ Does @Shanmugavel Chandrakasu​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 5 kudos
- 2337 Views
- 2 replies
- 2 kudos
Can I change the VMs IP address from dynamic to static on Azure without breaking anything?
Hello community;I am using Databricks on Azure and I am curious to know the implications of changing the IP address of the underlying VM's from dynamic to static - that is, the IP addresses that the clusters get, I would just like them to always have...
- 2337 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Jonathan Stern​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
- 2 kudos
- 952 Views
- 2 replies
- 0 kudos
Unable to login to https://community.cloud.databricks.com/login.html
Unable to login to https://community.cloud.databricks.com/login.htmlYesterday only I registered, able to login and create cluster.Today unable to login with same credential. I do not receive any email also while trying Password reset.
- 952 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @SHESADEV SAMAL​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
- 0 kudos
- 3853 Views
- 9 replies
- 3 kudos
Delta Live Table Pipeline - Azure cluster fail
Failed to launch pipeline cluster 0802-171503-4m02lexd: The operation could not be performed on your account with the following error message: azure_error_code: OperationNotAllowed, azure_error_message: Operation could not be completed as it results ...
- 3853 Views
- 9 replies
- 3 kudos
- 3 kudos
Hi there @Aaron LeBato​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you...
- 3 kudos
- 1355 Views
- 2 replies
- 0 kudos
- 1355 Views
- 2 replies
- 0 kudos
- 0 kudos
You can do CDC via Delta Live Tables in GCP.https://docs.gcp.databricks.com/workflows/delta-live-tables/delta-live-tables-cdc.html
- 0 kudos
- 1037 Views
- 0 replies
- 0 kudos
Understanding Dependency Update Failure
Heyooooo!I'm using Azure Databricks and sparklyr to do some geospatial analysis.Before I actually work with Spark Dataframes, I've been using the R packagesstarsandsfto do some preprocessing on my data so that it's easier to interact with later.In or...
- 1037 Views
- 0 replies
- 0 kudos