The Spark driver / executors have callbacks to an external service. I need to find out the IP ranges to be locked down for the firewall on the external service. Here is what I found from Azure (https://www.microsoft.com/en-us/download/details.aspx?...
Hello, Relatively new to Databricks and I've been using the Community Edition for a little bit now. I've recently been having more and more issues with my clusters pending until they time out before ever starting up. I've seen a few other posts here...
It seems that it's working now. At least for me...
Test Latesst Question
Hi everybody! Is it possible to use one job cluster for multiple tasks using new task orchestration tool? If so then please tell how. If it's impossible then this new tool looks useless Regards, Daniil.
The the regular version of databricks, the dbfs is mounted at /dbfs. This does not seem to be the case with community edition. I am seeking more details.
If you are using DBR 7.x. or newer versions the dbfs is disabled. You have to use dbutils.fs.cp commands as a workaround.
I want to use the VS Code IDE to edit/write/work on jupyter notebooks but have the code run off of the databricks cluster. Does anyone know of a way for me to force the jupyter notebook server on VS Code to point towards the databricks cluster and e...
I found that Databricks did not stop and delete ec2 instances of clusters. After the cluster terminate , the ec2 instances are still running.
Please make sure the Databricks IAM role does have all required permission mentioned https://docs.databricks.com/administration-guide/account-api/iam-role.html#0-language-Your%C2%A0VPC,%C2%A0defaultMake sure you did not change the EC2 tags especially...
Hi experts, I need to ingest data from an existing delta path to my own delta lake. The dataflow is as shown in the diagram: Data team reads full snapshot of a database table and overwrite to a delta path. This is done many times per day, but...
I have a group of rows with Information on a nested product calls. example- Trxn1-product1-caller1-local1 Trxn1-Product1-local1-local2 Trxn1-Product1-local2-local3 here’s is a expected calls for a product product1-caller1-local1 Product1-local1-loc...
dbutils.fs.mkdirs("/foobar/")See https://docs.databricks.com/data/databricks-file-system.html
To access DBFS via local file APIs, you can try /dbfs/<foobar>https://docs.databricks.com/data/databricks-file-system.html?_ga=2.41953189.1820496821.1627689131-1247613683.1627514237#local-file-apis
i am getting began with databricks-connect to hook up with my azure databricks cluster the use of a private get admission to token. it looks 192.168.1.254 like this token is being stored in a uncooked text report. if it is proper, is there a extra r...
I want to run some unit tests on my code, but Databricks can't seem to handle running formal unit testing libraries due to the lack of command line. From Googling, it appears it's possible to run notebooks and such from IntelliJ if using Scala, rath...
From Audit logs, I found one workspace user keep sending tokenLogin with a token that I cannot identify. The user personal access tokens are all with different ID.
@Ryan Chynoweth​ and @Sean Owen​ are both right, but I have a different perspective on this.Quick side note: you can also configure your cluster to execute with only a driver, and thus reducing the cost to the cheapest single VM available. In the cl...
How big should my cluster be? How do I know how many nodes to use or the kind of instance to use?
>How big should my cluster be? This would really depend on the use case. Some general guiding principles could be found here https://docs.databricks.com/clusters/cluster-config-best-practices.html
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group| User | Count |
|---|---|
| 1610 | |
| 763 | |
| 345 | |
| 286 | |
| 251 |
