- 3922 Views
- 6 replies
- 2 kudos
From a noob Databrickser... concerning Python programming in databricks
The following...We,ve got clients working with us in contracts. Per client several contracts of a certain type with start- and end dates. If I need aggregated info per client in one record like:how many different contract did the client haveof which ...
- 3922 Views
- 6 replies
- 2 kudos
- 12525 Views
- 2 replies
- 0 kudos
How can I somehow run spark.something in a worker? - rdd foreach spark.context
i am using rdd to parallelize a function, in this function i format the record i want to save, how can i store from this function the record with a dataframe? because every time i use spark..... an error is generated Caused by: org.apache.spark.api.p...
- 12525 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @alexander grajales vanegas​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear ...
- 0 kudos
- 4602 Views
- 2 replies
- 3 kudos
Enabling Fair Scheduler from Databricks Notebook
How to enable fair scheduler from Databricks notebook using python commands?
- 4602 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @Hariharan Sambath​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you....
- 3 kudos
- 2367 Views
- 2 replies
- 3 kudos
List mounts for specyfic scope
I have like 40 mounts and some of them were done based on the scopes which now I want to remove. How can I list the mounts for specyfic scope or even better, get the list of all the mounts alongside the scopes?
- 2367 Views
- 2 replies
- 3 kudos
- 3 kudos
Hey there @Bartosz Wachocki​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear fro...
- 3 kudos
- 21317 Views
- 7 replies
- 18 kudos
Resolved! Cluster Modes - High Concurrency
It took me quite some time to find the option to create a cluster in High Concurrency mode. It was hidden in the new UI.What should be the way to access the data with TAC?What is the equivalent mode to work with TAC ?Does it mean that we are being pu...
- 21317 Views
- 7 replies
- 18 kudos
- 6075 Views
- 2 replies
- 3 kudos
Data load from Azure databricks dataframe to cosmos db container
I am trying to load data from Azure databricks dataframe to cosmos db container using below commandcfg = { "spark.cosmos.accountEndpoint" : cosmosEndpoint, "spark.cosmos.accountKey" : cosmosMasterKey, "spark.cosmos.database" : cosmosDatabaseName, "sp...
- 6075 Views
- 2 replies
- 3 kudos
- 3 kudos
Hey @Rama Santosh Ravada​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from y...
- 3 kudos
- 3557 Views
- 2 replies
- 0 kudos
dbfs access for job clusters and interactive cluster
Hi All,I am new to databricks need some understanding for my requirement .our requirement:a: we have zip file in azure blob storage and we are bringing that file to dbfs and unzip that file and executing our transformations in multiple steps (3 steps...
- 3557 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @praveen rajak​ Does @Debayan Mukherjee​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 0 kudos
- 2979 Views
- 2 replies
- 3 kudos
Where do you usually store and manage "JDBC credentials" to use on databricks notebook?
Hi all,I would like to improve the way I use JDBC credenditial information (ID/PW, host, port, etc)Where do you guys usually store and use the jdbc credentials?Thanks for your help in advance!
- 2979 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @Kwangwon Yi​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks...
- 3 kudos
- 3714 Views
- 3 replies
- 4 kudos
TCP/IP Issue
Hello Team,I am trying to read the data from Synapse analytics from databricks.The below is the Query to read the table.%python### Read from Azure Synapse table via spark.read.load df = spark.read \ .format("com.databricks.spark.sqldw") \ .option("ur...
- 3714 Views
- 3 replies
- 4 kudos
- 4 kudos
Hi @Rohit Kulkarni​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
- 4 kudos
- 2389 Views
- 1 replies
- 1 kudos
Resolved! DBFS Rest Api is disabled
Dear Team,I have created a db account using gcp when i tried to create the token and configure databricks cli and tried to connect i get the below errordatabricks fs lserrorError: b'{"error_code":"FEATURE_DISABLED","message":"DBFS Rest Api is disable...
- 2389 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Rakesh Reddy Gopidi​ This is a known limitation with DBFS API and GCP. We are planning to redesign the DBFS API and we wanted to not gain more users that we later might need to migrate to a new API. If this is really required for you, please pro...
- 1 kudos
- 810 Views
- 0 replies
- 0 kudos
New Note
The visualisation of workflow tasks doesn't handle particular layouts well. This is because the arrows can pass underneath task blocks, this makes what would be a nice visualisation for exporting into a confusing one.I have provided an image of a set...
- 810 Views
- 0 replies
- 0 kudos
- 3810 Views
- 4 replies
- 3 kudos
Resolved! Still missing Badge for Apache Spark 3.0 Associate Dev certification
Hello, I have taken my exam 2 weeks ago and have passed it but I still did not received my badge. I have contacted the support team twice but still no response. Could you please help? Thank you!
- 3810 Views
- 4 replies
- 3 kudos
- 3 kudos
Databricks should fix the certification platform ASAP, currently a user needs to login to multiple different sites to get a certification.Each site has its own login that makes it very difficult to remember. There is not integration or synergy among ...
- 3 kudos
- 10396 Views
- 0 replies
- 0 kudos
NULL vs NaN in SQL Mode
In SQL Mode | SQL Editor there seems to be no distinction between NULL and NaN. In some cases it is very misleading as it makes the user to search mistake in the wrong place.DE/DS mode works as expected: UPDATE: a bit later I found this article: http...
- 10396 Views
- 0 replies
- 0 kudos
- 4643 Views
- 5 replies
- 0 kudos
Resolved! GCP auth time out in long running databricks job
I'm wondering if you can help me with a google auth issue related to structured streaming and long running databricks jobs in general. I will get this error after running for 8+ hours. Any tips on this? GCP auth issues for long running jobs?Caused by...
- 4643 Views
- 5 replies
- 0 kudos
- 0 kudos
Thanks, yes this seems to be the best work around - the good ole retry on fail. Thanks for the help.
- 0 kudos
- 3327 Views
- 2 replies
- 2 kudos
Resolved! Does libraries installation happen on Data Plane or Control Plane?
Currently, when I install libraries on my clusters. This errors happens:WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol...
- 3327 Views
- 2 replies
- 2 kudos
- 2 kudos
@John William​ : Yeah that's true. All the clusters will be residing in the data plane.
- 2 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 566 | |
| 349 | |
| 287 |
