- 856 Views
- 0 replies
- 0 kudos
Databricks CI/CD process is decoupled or coupled ?
In Databricks, CI/CD process is decoupled or coupled ?
- 856 Views
- 0 replies
- 0 kudos
Forum Posts
- 957 Views
- 0 replies
- 0 kudos
docs.databricks.com
What Serverless features are you using on Databricks? I am curious to know.Is it Databricks SQL Serverless or Model Serving?Proceed here to Compare serverless compute to other Databricks architectureshttps://docs.databricks.com/serverless-compute/ind...
- 957 Views
- 0 replies
- 0 kudos
- 1011 Views
- 0 replies
- 0 kudos
Change Azure Databricks cluster owner
I wanted to add secrets to spark conf of the cluster but i am not able to because i am not the cluster owner. I want to know how can we change the cluster owner?
- 1011 Views
- 0 replies
- 0 kudos
- 1175 Views
- 0 replies
- 0 kudos
Other than the "account admin" permission, is there a small permission or role to collect only catalog information?
I am going to use an open source called "datahub" to collect and share metadata information of databricks. (https://datahubproject.io/)Recently, however, there has been a big challenge. That is, to collect the unity catalog information of databricks,...
- 1175 Views
- 0 replies
- 0 kudos
- 5442 Views
- 1 replies
- 1 kudos
Resolved! Efficiently move multiple files with dbutils.fs.mv command on abfs storage
As part of my batch processing I archive a large number of small files received from the source system each day using the dbutils.fs.mv command. This takes hours as dbutils.fs.mv moves the files one at a time.How can I speed this up?
- 5442 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
@Dean Lovelace​ You can use multithreading.See example here: https://nealanalytics.com/blog/databricks-spark-jobs-optimization-techniques-multi-threading/
- 1 kudos
- 10956 Views
- 2 replies
- 2 kudos
Resolved! Web application integrated with Gradio or streamlit on Databricks
We are trying to run a web application integrated with Gradio on Databricks. Although, we have configured launch parameter with (share="True")The app executes and gives us output but it keeps on running with no Public URL is generated:o/p: Running on...
- 10956 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
Hi @Janga Reddy​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...
- 2 kudos
- 1342 Views
- 1 replies
- 0 kudos
- 1342 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
You can now use Delta Sharing to share notebook files securely using the Databricks-to-Databricks sharing flow.Sharing notebooks empowers users to collaborate across metastores and accounts, and enables providers to demonstrate use cases and visualiz...
- 0 kudos
- 5564 Views
- 1 replies
- 0 kudos
Resolved! Can I use Alation with Partner Connect ?
- 5564 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
You can now connect your Databricks workspace to Alation using Partner ConnectFor more information : https://lnkd.in/ePmyWiVr
- 0 kudos
- 5984 Views
- 3 replies
- 0 kudos
Resolved! Unable to install R geospatial libraries raster, terra, sf, ncdf4, etc
When trying to install any of this R libraries from a cmd cell/block on a notebook, or from the UI in the cluster I receive the same error,seeming that are unable to install dependecies.Warning in utils::install.packages(pkgs, ...) : installation of ...
- 5984 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
For the next soul looking for an answer, I managed to solve the issue with the next 2 Init scripts, it gets tricky in the apt or apt-get, that was the issue, sometimes it did update, sometimes it didn't, making it possible to find libmysqlclient21.1)...
- 0 kudos
- 908 Views
- 0 replies
- 0 kudos
Number of rows displayed in sql cell
Even though the default limit on rows displayed is 10,000, the SQL cell is showing rows less than the limit when my resultant has more rows than 10k.It should alteast show the default limit .
- 908 Views
- 0 replies
- 0 kudos
- 6170 Views
- 4 replies
- 2 kudos
Resolved! how to make databricks job to fail when the application has already given "exit code 1"?
object OurMainObject extends LazyLogging with IOApp { def run(args: List[String]): IO[ExitCode] = { logger.info("Started the application") val conf = defaultOverrides.withFallback(defaultApplication).withFallback(defaultReference) val...
- 6170 Views
- 4 replies
- 2 kudos
Latest Reply
- 2 kudos
my workaround now is to make the code like below, so the databricks jobs becomes failure. case Left(ex) => { IO(logger.error("Glue failure", ex)).map(_ => ExitCode.Error) IO.raiseError(ex) }
- 2 kudos
- 6452 Views
- 3 replies
- 3 kudos
Resolved! Roll back to previous version of an AutoLoader checkpoint file
I know to "reset" AutoLoader, you can delete the checkpoint file entirely. I was wondering if it's possible to and how would someone :Get the checkpoint file to a previous version so I can reload certain files that were already processedDelete certai...
- 6452 Views
- 3 replies
- 3 kudos
Latest Reply
- 1908 Views
- 0 replies
- 4 kudos
databricks Runtime 13.1 has added the sql_keywords() function, which lists all SQL keywords. It is a good practice to refrain from using these keyword...
databricks Runtime 13.1 has added the sql_keywords() function, which lists all SQL keywords. It is a good practice to refrain from using these keywords as names for tables or fields, although, in standard ANSI false mode, it will work without problem...
- 1908 Views
- 0 replies
- 4 kudos
- 4074 Views
- 2 replies
- 1 kudos
Resolved! Notebook activity is getting timed out in ADF pipeline.
Notebook activity is getting timed out after certain time of running (5 hours) in ADF pipeline and getting timeout error.Its just simply getting timed out error. Problem is this will process TB of data daily. can anyone have any idea to fix this.
- 4074 Views
- 2 replies
- 1 kudos
Latest Reply
- 5120 Views
- 2 replies
- 9 kudos
You can use apache hudi in databricks without a problem:Â - in cluster settings, install Maven library org.apache.hudi:hudi-spark3.3-bundle_2.12:0.13.0...
You can use apache hudi in databricks without a problem:- in cluster settings, install Maven library org.apache.hudi:hudi-spark3.3-bundle_2.12:0.13.0 for Databricks 12.2 LTS- in cluster spark config, add three lines:spark.sql.extensions org.apache.sp...
- 5120 Views
- 2 replies
- 9 kudos
Latest Reply
- 9 kudos
I tried installing library and configuring spark configs, restarted the cluster and then in notebook ran the create cmd but it gives me error stating java.io.FileNotFoundException: No such file or directory: s3://incred-databricks-data/hudi_dms_data/...
- 9 kudos
| User | Count |
|---|---|
| 1612 | |
| 769 | |
| 348 | |
| 286 | |
| 252 |
