There are couple of custom state functions like mapgroupswithstate,ApplyinpandaswithStateWhich has a internal state maintained is it maintained in same statestore(rocksdb) as aggregation state store function ​
Hi @somanath Sankaran​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best ans...
Hi @Arun Sharma​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...
I'm trying to access account console in Azure but I only can see the list of workspaces and access them. I didn't find documentation about account console for Azure. Do you know how to access account console?
Hello @Leszek​ . Please tell me is it works for you ?Did you find the root cause ?I still don't understand why I should grant to myself some extra permissions if I already global administrator, owner of subscription, owner of databricks workspace but...
Hi, I have 2 dashboards. 1st Dashboard has a table with two columns, 'Country' and 'NumberOfMajorCities'. When I click on any row of this table/click on 'NumberOfMajorCities' field of a row, I would like to jump to 2nd Dashboard, where I display all ...
Hi @Ghazanfar Sl​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers ...
Hi @Nitya Mehta​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...
Data from Azure sql server was read into databricks through JDBC connection (spark version 2.x) and stored into Gen1. Now the client wants to migrate the data from Gen1 to Gen2. When we ran the same jobs that read data from Azure Sql Server to Databr...
Hi @Mani Teja G​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedback ...
Databricks Lakehouse Fundamentals BadgeI have successfully passed the test after completion of the course "Databricks Lakehouse Fundamentals". But I have'nt recieved any badge. I have been provided with a certificate only. Please provide me with th...
I'm just getting started with Databricks and DLTs. I've followed all the docs and tutorials I can find on this and believe I have set up everything in Azure correctly: service principal, paths, and spark configs. When I run a simple DLT autoloader pi...
Has anyone else experienced the lack of access to the Ray dashboard since this week? Last week worked fine.rom ray.util.spark import setup_ray_cluster
setup_ray_cluster(...)This used to output an HTML block with a link to the dashboard.I can manual...
The reason for the missing dashboard for me was due ​to not having installed some required dependencies. Shout out to the Ray community for their help.​I've submitted a PR (now merged) to add a warning message​ when such dependencies are missing.​​
Problem:When running dbt with persist column docs enabled we get the following error: org.apache.hadoop.hive.ql.metadata.HiveException: at least one column must be specified for the tableBackground:There is an issue on the dbt-spark github that was c...
Also confirming that you do not have any of these limitations:From DBT's website: Some databases limit where and how descriptions can be added to database objects. Those database adapters might not support persist_docs, or might offer only partial su...
I am running a set of commands and to run `pydeck` on jupyter notebook as per the documentation here: https://pydeck.gl/installation.html#enabling-pydeck-for-jupyterHowever, it throws an `moduleNotFoundError` exception. !pip install pydeck
!jupyter n...
Hi @Keval Shah​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers yo...
Hi,I am unable to link my Databricks Lakehouse Fundamentals badge to my Linkedin account.I couldn't sign in to credential.databricks.com webpage.My email id is : abhishek.palit@ageas.com.hk
If you're having trouble linking your Databricks Lakehouse Fundamentals badge to your LinkedIn Sales Navigator extension, there are a few things you can try:Check your badge settings: Make sure that you have enabled sharing for your badge and that yo...
We are migrating our Scala jobs from AWS EMR (6.2.1 and Spark version - 3.0.1) to Lakehouse and few of our jobs are failing due to NullPointerException. We tried in Databricks Runtime 7.3 LTS, it is working fine. Because it had same spark version 3.0...
In one of my code statements, I updated scala Boolean to java.lang.Boolean and this is working fine now. May be in new newer Spark versions, null in scala Boolean isn't supported.
An S3 bucket with the prefix "databricks-workspace-stack-lambdazipsbucket" was created by default when I created my AWS Databricks account. It is set to public access. It has one zip file in it called "lambda.zip". What is the purpose of this S3 buck...
Hi @Dian Germishuizen​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best ans...
https://community.cloud.databricks.com/login.htmlUnable to Login and Unable to reset Password . Password reset link keeps on loading even after entering passwords
