Hi all,I am curious to know how you handled change management when rolling out changes to cluster access modes (single user, shared) applicable to UC, especially when there is a centralized platform team offering Infrastructure Deployment services vi...
Hi @janaki jagadeesh​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answ...
Here is the previous discussion. https://community.databricks.com/s/question/0D58Y0000ACcIv2SQF/does-thrift-only-exist-in-databrick-control-planeI have the following questions.Does CVE-2020-13949 affect the data plane or not?Do you know from which ve...
Hi @Jimin Hsieh​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so w...
Hi Databricks,We want to understand a vulnerability issue - CVE-2023-22946.does this CVE affect Azure databricks users? Does it affect data plane? Is it managed by databricks? Should end users do anything to mitigate it if this issue is a shared res...
We would like to know the number of DBU's utilized cluster level/workspace level which was hosted in Azure. We were able to see the amount spend but not able to see the DBU's. If we get any idea of no.of DBU's it would help us to think of reserved DB...
@Mani Teja G​ we don't have screen like aws for azure, but in azure you can monitor everything in cost management --> cost analysis and select tags as Databricks
Hi,I would like to set up a DBU usage alert, such that I will get an email notification when the DBU usage of last X hour exceeds the budget.How can I do that?Thanks,Kit
Hello @Kit Yam Tse​ You can download and analyse usage logs to monitor the DBUs.Check this out: https://docs.databricks.com/administration-guide/account-settings-e2/usage.htmlwe also have usage analysis dashboard notebook:https://docs.databricks.com/...
Is there a way to get the databricks units (DBU) for an existing Azure databricks workspace?
I'm keen to know this as well.
How to access the raw Ganglia metrics. Is there an endpoint or a DBFS storage it belongs to?
How do I get the cost of my notebook run?
How do I get the cost of my notebook run?
how to update the schedule via the Jobs API (it would be within the Cron Schedule field).
Hi Team,We are currently using AWS cloud service for our AI/ML Project. We wanted to use Databricks based GPU service for model training purposes. We do not find a Pay-as-you-go option in marketplace for this purposes.Can someone help me on this !Tha...
https://databricks.com/product/aws-pricingI guess you try to roll out something else?
Our architecture is according to Domain Driven Design. The data is therefore distributed among different domains.We would like to run workloads on top of our data but we would like to avoid to have a dedicated (duplicated) data lake just for Databric...
So basically you do not want to persist data outside of your source systems.I think the so called 'Kappa architecture' could be a fit, where everything is treated like a stream.Hubert already mentioned Kafka, which is an excellent source to build thi...
How do you work to compliance the GDPR/LGPD, in special to people data ?have any suggestion ?
there is identifying PII data and handling/storing PII data.Identifying can be done with Purview, Macie, other tools. Those are not free ofc, so if your env is pretty big it can be interesting. Otherwise, you could also do manual checks.For storin...
I have sensor data coming into Azure Event Hub and need some help in deciding how to best ingest it into the Data Lake and Delta Lake:Option 1:azure event hub > databricks structured streaming > delta lake (bronze)Option 2:azure event hub > event hu...
If batch job is possible and you need to process data I would use probably:azure event hub from (events after previous job run) > databricks job process as dataframe > save df to delta lakeno streaming or capturing needed in that case
A great place to learn more about Databricks integrations with AWS services is https://www.databricks.com/aws There is information on this page regarding integrations with Glue, SageMaker, Redshift and others. Many of these pages also point to our bl...
