Hi All,i am getting the below error when i am ingesting the data from source file , source file is also attached , i have tried in both Community edition and Azure databricks as well getting the same error , can any one suggest me the solution ? # ...
Hello Experts,I am unable to replace nulls with 0 in a dataframe ,please refer to the screen shotfrom pyspark.sql.functions import col emp_csv_df = emp_csv_df.na.fill(0).withColumn("Total_Sal",col('sal')+col('comm')) display(emp_csv_df)erorr desired ...
I bet that it is not real null but the string "null". Please check what is in the source and try luck with replacing it.
Dear Team,While practising few examples i have noticed the below notebook is not fetching the full dataset and also no schema is fetched https://docs.databricks.com/_static/notebooks/widget-demo.htmlcan you please re try and let me know the results N...
I think https://health.data.ny.gov/api/views/myeu-hzra/rows.csv was a public dataset, but now it shows authentication_required, so this error is independent of databricks.But the good news is that I was able to generate a new URL on the New Your heal...
I have completed and passed the short assessment for Lakehouse fundamentals but I didn't receive any badge. I have also checked in credentials.databricks.com but I don't see any badge.
Hey @Subhransu Ranjan Sankhua​ Thank you for reaching out! Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training and our team will get back to you shortly.
Hello all, I'm experiencing this issueConstructor public com.databricks.backend.daemon.dbutils.FSUtilsParallel is not whitelisted when I'm trying to mount a s3 bucket. %python dbutils.fs.mount("s3a://dd-databricks-staging-storage/data/staging/datalak...
WE have this problem running cluster with 11.2 and shared access mode. spark.databricks.pyspark.enablePy4JSecurity false - this does not help because it says spark.databricks.pyspark.enablePy4JSecurity is not allowed when choosing access modehere is ...
Hi,Is there any way to execute jar scala-spark application inside the notebook, without using jobs?I have different jars for different intakes and I want to call them from a notebook, so I could call them in a parameterized way.Thanks
Hi @Sergio Garccia​ ,Just a friendly follow-up. Do you still need help? have you check our docs? This might help https://docs.databricks.com/workflows/jobs/jobs.html#jar-jobs-1
The full error is below: An error occurred while calling o95098.execute.: com.databricks.s3commit.S3CommitRejectException: rejected by server26 times at com.databricks.s3commit.S3CommitClientImpl.commit(S3CommitClient.scala:303)It was a one-off inci...
I have a quick question about %run <notebook path>. I am using the %run command to import functions from a notebook. It works fine when I run %run once. But when I run two %run commands, I lose the reference from the first %run. I get NameError when ...
I read this article and I created a notebook to use like a library but when I tried to import it in other notebook I received this error: No module named 'lib.lib_test' No module named 'lib.lib_*****'
Hey @Marco Antônio de Almeida Fernandes​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love...
The following...We,ve got clients working with us in contracts. Per client several contracts of a certain type with start- and end dates. If I need aggregated info per client in one record like:how many different contract did the client haveof which ...
i am using rdd to parallelize a function, in this function i format the record i want to save, how can i store from this function the record with a dataframe? because every time i use spark..... an error is generated Caused by: org.apache.spark.api.p...
Hi @alexander grajales vanegas​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear ...
How to enable fair scheduler from Databricks notebook using python commands?
Hi @Hariharan Sambath​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you....
I have like 40 mounts and some of them were done based on the scopes which now I want to remove. How can I list the mounts for specyfic scope or even better, get the list of all the mounts alongside the scopes?
Hey there @Bartosz Wachocki​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear fro...
It took me quite some time to find the option to create a cluster in High Concurrency mode. It was hidden in the new UI.What should be the way to access the data with TAC?What is the equivalent mode to work with TAC ?Does it mean that we are being pu...
I am trying to load data from Azure databricks dataframe to cosmos db container using below commandcfg = { "spark.cosmos.accountEndpoint" : cosmosEndpoint, "spark.cosmos.accountKey" : cosmosMasterKey, "spark.cosmos.database" : cosmosDatabaseName, "sp...
Hey @Rama Santosh Ravada​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from y...
| User | Count |
|---|---|
| 1612 | |
| 768 | |
| 348 | |
| 286 | |
| 252 |
