- 3029 Views
- 3 replies
- 1 kudos
Resolved! Can I add custom cluster tag from init script ?
Hi, is it possible to add custom tags from init script during cluster initialization ? We would like to automatically add custom tags whenever someone creates a new cluster in databricks.
- 3029 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @Md Tahseen Anam​ I don't think there is a possibility to use an init script for cust tags. But the easiest way is to use cluster policies. You can mention a list of custom tags in the policy so that you can simply add the policy to the cluster wh...
- 1 kudos
- 3416 Views
- 3 replies
- 2 kudos
Resolved! How to read multiple tiny XML files in parallel
Hi team, we are trying to read multiple tiny XML files, able to parse them using the data bricks XML jar, but is there any way to read these files in parallel and distribute the load across the cluster? right now our job is taking 90% of the time rea...
- 3416 Views
- 3 replies
- 2 kudos
- 2 kudos
Thank you @Hubert Dudek​ for the suggestion. Similar to your recommendation, we added a step in our pipeline to merge the small files to large files and make them available for the spark job.
- 2 kudos
- 18324 Views
- 1 replies
- 0 kudos
Resolved! Exporting data from databricks to external csv
I need to export some data from the database to csv which will be downloaded to another application. What would be the procedure for that? I don't have a lot of knowledge in DataBricks and I didn't find much information in the documentation.Thanks.
- 18324 Views
- 1 replies
- 0 kudos
- 0 kudos
You can manually download data to your local in CSV from databricks notebook cell and pass it to your another application.Your application can run Databricks notebook inside a workflow via an API that writes data to S3 bucket in CSV and in response y...
- 0 kudos
- 1328 Views
- 2 replies
- 2 kudos
- 1328 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @rajat kumar​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks...
- 2 kudos
- 1474 Views
- 2 replies
- 0 kudos
Azure SA mounted but can't load files
I am attempting to load an excel file that's located in a blob storage that I've mounted. In the first cell, when I use the dbutils.fs.ls command, I can see the file I want to load. However, when I try to actually load it, it can't find the file. It ...
- 1474 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Niels Ota​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 0 kudos
- 1614 Views
- 3 replies
- 0 kudos
Is it possible to install databricks-cli on a shared laptop.
I have a government furnished laptop (GFE). My normal user is not an admin on the laptop. I have a separate admin login on the laptop. I was able to install databricks-cli as the admin user but it installed under that users appdata python path which ...
- 1614 Views
- 3 replies
- 0 kudos
- 0 kudos
Hey there @John Zajic​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you....
- 0 kudos
- 4540 Views
- 3 replies
- 4 kudos
Resolved! DeltaTable' object has no attribute 'clone'
Hello, I use delta on my local machine and I would like to clone a table, however the cloning is not working.I have the last version of delta installed (delta-spark==2.0.0) but the clone method does not exist in the python module.With this code :delt...
- 4540 Views
- 3 replies
- 4 kudos
- 4 kudos
Hello @Bertrand BURCKER​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from yo...
- 4 kudos
- 1397 Views
- 2 replies
- 0 kudos
User's name is empty sometimes when a new user is added from admin console
Hi, when I add a new user from admin console, the name of the user is empty. It does not happen all the time. For some users, the username and name both are available. But for some new users, the value in the name column in the users list is empty. W...
- 1397 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi, when I add a new user from admin console, the name of the user is empty. It does not happen all the time. For some users, the username and name both are available. But for some new users, the value in the name column in the users list is empty. W...
- 0 kudos
- 5209 Views
- 8 replies
- 2 kudos
Resolved! Do compute resources get removed after not being used for x number of days?
Currently we're getting reports of compute resources disappearing from one of our lesser used databricks platforms. I just turned on logging to see if we can find something but I'm wondering if a compute gets removed if it hasn't been used after so l...
- 5209 Views
- 8 replies
- 2 kudos
- 2 kudos
Pest Control Software to Grow Your Business Choosing the best pest control software for your business can have a powerful impact on your productivity. Fieldwork can help your workforce repel downtime, attract clients, get organized and get everything...
- 2 kudos
- 5047 Views
- 4 replies
- 1 kudos
Unable to read data from Elasticsearch with spark in Databricks.
When I am trying to read data from elasticsearch by spark sql, it throw an error like RuntimeException: Error while encoding: java.lang.RuntimeException: scala.collection.convert.Wrappers$JListWrapper is not a valid external type for schema of string...
- 5047 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi there @KARTHICK N​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.T...
- 1 kudos
- 1743 Views
- 2 replies
- 0 kudos
Connect to SSAS
Hello everyone,I need to connect Databricks Pyspark to get information from Power BI XLMA EndPoint - the end point work as an SSAS host.So, I'm trying to find what I need to do to connect to SSAS tabular. Can anyone help?Many thanks.Rodrigo Souza
- 1743 Views
- 2 replies
- 0 kudos
- 0 kudos
Hey there @Rodrigo Camara de Souza​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to h...
- 0 kudos
- 2156 Views
- 3 replies
- 1 kudos
Bulk removal of inactive users?
To make a long story short, through SCIM we accidentally provisioned 3,000+ users into our Databricks workspace who should not be there. We fixed the SCIM issue but now the workspaces tab is flooded with inactive user workspaces. Is there any way to ...
- 2156 Views
- 3 replies
- 1 kudos
- 1 kudos
Hello @David Kruetzkamp​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from yo...
- 1 kudos
- 702 Views
- 0 replies
- 0 kudos
SCD type2 snowflake
I have monthly files which comes in S3 bucket. I want to implement SCD type2 in snowflake.I am ok to read the new files, clean it.My question is about comparing what I have read from the files, with what is stored in the snowflake table already(milli...
- 702 Views
- 0 replies
- 0 kudos
- 929 Views
- 3 replies
- 0 kudos
The Next Databricks Office HoursOur next Office Hours session is scheduled for January 25, 2022 - 8:00 am PDTÂ Do you have questions about how to set u...
The Next Databricks Office HoursOur next Office Hours session is scheduled for January 25, 2022 - 8:00 am PDTDo you have questions about how to set up or use Databricks? Do you want to get best practices for deploying your use case or tips on data ar...
- 929 Views
- 3 replies
- 0 kudos
- 0 kudos
Thank you for the opportunity to communicate. I work at https://www.eliteimagingsystems.com/ and know how important it is for our customers to be able to communicate with us 24/7.
- 0 kudos
- 2745 Views
- 3 replies
- 4 kudos
Resolved! What is the maximum of concurrent streaming jobs for a cluster?
What is the maximum of concurrent streaming jobs for a cluster? How can I have the right amount of concurrent streaming jobs for different cluster configuration?Should I use multiple cluster for different jobs or combine it into a big cluster to hand...
- 2745 Views
- 3 replies
- 4 kudos
- 4 kudos
Hi @John William​ it would be better to use different clusters for each streaming jobs.
- 4 kudos