Hey, So currently I'm using Pyspark and utilizing SparkListener to track metrics of my spark jobs. The problem I can't seem to solve is why in databricks the listener cannot be removed with removeSparkListener as it still is attached to the context.#...
Hello, we are developers who have been creating a system in Databricks with Scala. We enabled the Git feature, so the project is in a repository. The project has a lot of notebooks and a lot of calls to other notebooks. Sometimes it is a little overw...
it is true that we can't work without data bricks but we can develop an IDE and send the jar to databricks, this will allow us to create unit tests, and use the IDE capabilities (i.e fast navigation among classes).
is it possible to have multiple tabs in Dashboard? if not is there any workaround for this.
Hey @Janga Reddy​ Hope you are well. Just wanted to see if you were able to find an answer to your question and would you like to mark an answer as best? It would be really helpful for the other members too.Cheers!
< a href="https://tersesayings.com/">Success</a>
Hello,I have written code that produces plotly charts in a python notebook (by facet) - each output is 4 line charts side-by-side with a few points highlighted.When I run this in a loop to produce more than 7 or 8 of my graphs, some of the features ...
unfortunately I'm not able to share as it is in reference to proprietary company dataessentially, I can produce any one plotly graph like:[line graph with scatter]but if i produce, say, 8 of them (looping on a list of inputs), I get something like th...
Hi,I have a solution design question on which I am looking for some help. We have 2 environments in azure (dev and prod), each env has its own ADLS storage account with a different name of course. Within Databricks code we are NOT leveraging the mou...
As of now, if I try to list the jobs via "list job" API then there is a limit of 25 jobs only.Is there a way to list all the available/visible jobs to a user?
Hello @Saurabh Verma​ Can the user generate the API token in the workspace and try to use the API?
I want to use databricks inside vscode and I therefore need Databricks-connect I configure my settings using databricks-connect configure as follows: Databricks Host [https://adb-1409757184094616.16.azuredatabricks.net]Databricks Token [<my token>]Cl...
In case it helps anyone, I ran into this issue and had to remove the trailing / from the host name. It used to work fine with the trailing / so something must have changed.
Hi,I have datalake gen2 with vnet and private endpoint. I do have databricks workspace in same vnet. I am trying to access the datalake from databricks but I keep getting error when I allow access only for selected network in datalake. I get error w...
You can always check the tutorial regarding ADLS and private link under this link https://community.databricks.com/s/feed/0D53f00001eQGOHCA4
HI there,I'm following the course mentioned from Databricks Academy. I downloaded the .dbc archiive and working along side the videos from academy. In ML-08 - Hyperopt notebook, I see the following error in cmd 13. best_hyperparam = fmin(fn=objectiv...
Tagging @Kaniz Fatma​ as there was no response what so ever!By any chance, do you know how to resolve these errors in the notebook?Thanks!
Thanks for the information that you share with us, Grey Cup 2022. I really like your content about this.
Per the documentation, a workspace is limited to 1k concurrent job runs.Can somebody clarify how the concurrency limit is set i.e:Is it 1k concurrent runs across all jobs in the workspaceIs it 1k concurrent runs for a single jobAlso, is there any way...
Hi @Kaniz Fatma​ Weners gave some great suggestions on the issue we are dealing with. But some confirmation on the question from data bricks side would be much appreciated
How can we alter table with auto increment column for a delta tableI have tried this but not working:ALTER TABLE dbgtpTest.student ADD COLUMN Student_Id identity(100,1)any Suggestions will be helpful
@Rakesh Reddy Badam​ , For ALTER ... ADD COLUMN in doc is only SYNC IDENTITY.If you want to add an identity column to the existing table just create a new table with an identity column and then copy the data.
We are working for client in offline environment(VM is private) azure databricks, we store some credentials in azure key vault, since azure databricks is run on offline environment, i am unable to create secret scopes on azure databricks, it always t...
Hi @Kaniz Fatma​ apologies for the delayed response, I earlier mentioned that the data bricks run in a private environment which means no connectivity to InterNet, I already followed the link you have shared but didn't get the result. Either I have t...
I'm trying to configure audit logs for databricks and as a part of it I need to configure S3 storage using endpoint storage-configurations.But for some reason I'm getting invalid endpoint. I'm using following URL with basicAuth.https://accounts.cloud...
Hey there @Aman Sehgal​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you...
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 584 | |
| 349 | |
| 287 |
