- 4548 Views
- 2 replies
- 7 kudos
need to fetch secrets from key vault in my local
Could you please look into this if I'm missing something. Getting the below error:azure.core.exceptions.ServiceRequestError: Bearer token authentication is not permitted for non-TLS protected (non-https) URLs.Using below function for that.def get_aut...
- 4548 Views
- 2 replies
- 7 kudos
- 7 kudos
Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we can help you. C...
- 7 kudos
- 2291 Views
- 3 replies
- 2 kudos
How to speed up `dbx launch --from-assets`
Hiya,I'm trying to follow the testing workflow of```$ dbx deploy test --assets-only$ dbx launch test --from-assets --trace --include-output stdout```But I find the turnaround time is quite long, even with an instance pool.The `deployment.yaml` looks ...
- 2291 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi, I have no solution, actually I've just registered to open a very similar ticket, when saw yours.According to my experiments getting an already running VM from the pool (times between events: CREATING - INIT_SCRIPTS_STARTED) can take anything betw...
- 2 kudos
- 2581 Views
- 2 replies
- 0 kudos
Resolved! Unable to add password to a user (with SSO enabled)
I am trying to add a non-SSO admin user to my account (not to a workspace). I have SSO backed off to Google for the majority of users.I can create the account OK, then go in and reset the password to something, but when I try and log in I get the err...
- 2581 Views
- 2 replies
- 0 kudos
- 2012 Views
- 3 replies
- 0 kudos
File optimization for delta table (versioning and snapshot ) in storage S3
Delta table generates new file for every insert or update on table and keep the old version files also for versioning and time travel history . I have 1tb data as delta table and every 30 minutes , 90 percent data getting updated so file size will b...
- 2012 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @vinay kumar​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks...
- 0 kudos
- 1372 Views
- 2 replies
- 1 kudos
Running large volume of SQL queries in Python notebooks. How to minimise overheads/maintenance.
I have around 200 SQL queries id like to run in databricks python notebooks. Id like to avoid creating an ETL process for each of the 200 SQL processes.Any suggestions on how to run the queries in a way that it loops through them so i have minimum am...
- 1372 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Chris French​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...
- 1 kudos
- 2102 Views
- 4 replies
- 2 kudos
- 2102 Views
- 4 replies
- 2 kudos
- 2 kudos
Hi @jairo neder monassa moreira​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear...
- 2 kudos
- 1535 Views
- 3 replies
- 0 kudos
Resolved! Is it possible to reset user passwords outside of the Admin Console UI?
There is a business requirement for some of our accounts to have their passwords rotated. This currently requires an admin to go in and manually reset the password for the account via UI. I wanted to know if there's a more automated way to handle thi...
- 1535 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Jordan Gray​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks...
- 0 kudos
- 776 Views
- 1 replies
- 2 kudos
Resolved! Lakehouse Fundamentals Accreditation badge not received
I have passed the test today for Lakehouse Fundamentals Accreditation, but have not yet received the badge yet.Please let me know how and when I can receive the badge for this passed test.Thank you.
- 776 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @Chang Su Lee​ Thank you for reaching out! Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training and our team will get back to you shortly.
- 2 kudos
- 2473 Views
- 2 replies
- 2 kudos
Should I create Azure storage & Metastore in the same region?
I am going to create a Metasore following the documentation. Regarding the storage account, I don't understand if it should be in the same region as Metastore. From documentation:You can create no more than one metastore per region. It is recommended...
- 2473 Views
- 2 replies
- 2 kudos
- 2 kudos
Thanks. But the question is about the region for Storage Account & Metastore.
- 2 kudos
- 3640 Views
- 3 replies
- 2 kudos
Databricks Audit Logs, Where the log files are stored? How to read them?
Hi,I want to access the Databricks Audit Logs to check user activity. For example, the number of times that a table was viewed by a user.I have a few questions in this regard. 1) Where the log files are stored? Are they stored on DBFS?2) Can I read l...
- 3640 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @Mohammad Saber​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
- 2 kudos
- 1330 Views
- 2 replies
- 0 kudos
Resolved! When does delta lake actually compute a table?
Maybe I'm completely wrong, but from my understanding delta lake only calculates a table at certain points, for instance when you display your data. Before that point, operations are only written to the log file and are not executed (meaning no chang...
- 1330 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Lukas Goldschmied​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you....
- 0 kudos
- 11154 Views
- 5 replies
- 0 kudos
Merge upsert in Delta is throwing error "concurrent merge in delta lake tables in Azure Databricks .ConcurrentAppendException"
This is the error which is coming while processing concurrent merge in delta lake tables in Azure Databricks .ConcurrentAppendException: Files were added to the root of the table by a concurrent update. Please try the operation again.. What are the o...
- 11154 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi @Abhishek Dutta​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
- 0 kudos
- 2296 Views
- 2 replies
- 1 kudos
Resolved! Does Databricks supports the Pytorch Distributed Training for multiple devices?
Hi, Im trying to use the databricks platform to do the pytorch distributed training, but I didnt find any info about this. What I expected is using multiple clusters to run a common job using pytorch distributed data parallel (DDP) with the code belo...
- 2296 Views
- 2 replies
- 1 kudos
- 1 kudos
With Databricks MLR, HorovodRunner is provided which supports distributed training and inference with PyTorch. Here's an example notebook for your reference: PyTorchDistributedDeepLearningTraining - Databricks.
- 1 kudos
- 1634 Views
- 3 replies
- 0 kudos
Resolved! Time travel and version control- can create custom version control for each day data load when multiple updates happening in a day.
Time travel and version control- can create custom version control for each day data load when multiple updates happening in a day. For example , let’s assume we are doing multiple operation on table in a day every minute and want to keep time travel...
- 1634 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @vinay kumar​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks...
- 0 kudos
- 3226 Views
- 4 replies
- 3 kudos
Resolved! how does databricks time travel work
Hi, Since it is not very well explained, I want to know if the table history is a snapshot of the whole table at that point of time containing all the data or it tracks only some metadata of the table changes.To be more precise : if I have a table in...
- 3226 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi @data engineer​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so...
- 3 kudos