- 1924 Views
- 2 replies
- 0 kudos
Invalid port error when trying to read from PlanetScale MySQL databse
Using the code below I am attempting to connect to a PlanetScale MySQL database. I get the following error: java.sql.SQLException: error parsing url : Incorrect port value. However the port is the default 3306, and I have used the correct url based o...
- 1924 Views
- 2 replies
- 0 kudos
- 0 kudos
HI @Rachel Cunningham​ ,maybe you can share your `driver` and `url` value (masked)?
- 0 kudos
- 3895 Views
- 2 replies
- 0 kudos
Remove a category (slice) from a Pie Chart
I added a grand total row to a "Count" in SQL, which I needed for some counter visualisations. I used the "ROLL UP" command to get the grand total.However, I have a pie chart which references the same count, and so the grand total row has been added...
- 3895 Views
- 2 replies
- 0 kudos
- 0 kudos
hi, as per the picture above, the slice disappears but the name ("null" in this case) remains on the legend.
- 0 kudos
- 6840 Views
- 2 replies
- 2 kudos
Databricks <-> Kafka - SSL handshake failed
I am receiving SSL handshake error even though the trust-store I have created is based on server certificate and the fingerprint in the certificate matches the trust-store fingerprint.kafkashaded.org.apache.kafka.common.errors.SslAuthenticationExcept...
- 6840 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Jayanth Goulla​ , worth a try ,https://stackoverflow.com/questions/54903381/kafka-failed-authentication-due-to-ssl-handshake-failedDid you follow: https://docs.microsoft.com/en-us/azure/databricks/spark/latest/structured-streaming/kafka?
- 2 kudos
- 2461 Views
- 1 replies
- 2 kudos
Disable auto-complete (tab button)
Hello. How could we disable autocomplete that appears with tab button? Thank you
- 2461 Views
- 1 replies
- 2 kudos
- 2 kudos
Thank you @Kaniz Fatma​
- 2 kudos
- 3952 Views
- 1 replies
- 3 kudos
Custom Log4j logs are not being written to the DBFS storage.
I used custom Log4j appender to write the custom logs through the init script and I can see the Custom Log file on the Driver logs but Databricks is not writing those custom logs to the DBFS. I have configured Logging Destination in the Advanced sec...
- 3952 Views
- 1 replies
- 3 kudos
- 3 kudos
Hi @VIjeet Sharma​ , Do you receive any error? This can be an issue using DBFS mount point /dbfs in an init script: the DBFS mount point is installed asynchronously, so at the very beginning of init script execution, that mount point might not be ava...
- 3 kudos
- 2380 Views
- 1 replies
- 1 kudos
Structered Streamin from MongoDB Atlas not parsing JSON correctly
HI all,I have a table in MongoDB Atlas that I am trying to read continuously to memory and then will write that file out eventually. However, when I look at the in-memory table it doesn't have the correct schema.Code here:from pyspark.sql.types impo...
- 2380 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @sharonbjehome​ , This has to be checked thoroughly via a support ticket, did you follow: https://docs.databricks.com/external-data/mongodb.html Also, could you please check with mongodb support, Was this working before?
- 1 kudos
- 1451 Views
- 1 replies
- 1 kudos
How to count DelayCategories?
I would like to know how many count of each categories in each year, When I run count, it doesn't work.
- 1451 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi, @Dara Tourt​ , When you say it does not work, what is the error? You can run count aggregate function. https://docs.databricks.com/sql/language-manual/functions/count.htmlPlease let us know if this helps.
- 1 kudos
- 1322 Views
- 1 replies
- 1 kudos
How to read in csvs from s3 directory with different columns
I can read all csvs under an S3 uri byu doing:files = dbutils.fs.ls('s3://example-path')df = spark.read.options(header='true', encoding='iso-8859-1', dateFormat='yyyyMMdd', ignoreLeadingWhiteSpace='true', i...
- 1322 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Anthony Wang​ As of now, I think that's the only way. Please refer: https://docs.databricks.com/external-data/csv.html#pitfalls-of-reading-a-subset-of-columns. Please let us know if this helps.
- 1 kudos
- 8868 Views
- 3 replies
- 6 kudos
Saving PySpark standard out and standard error logs to cloud object storage
I am running my PySpark data pipeline code on a standard databricks cluster. I need to save all Python/PySpark standard output and standard error messages into a file in an Azure BLOB account.When I run my Python code locally I can see all messages i...
- 8868 Views
- 3 replies
- 6 kudos
- 6 kudos
This is the approach I am currently taking. It is documented here: https://stackoverflow.com/questions/62774448/how-to-capture-cells-output-in-databricks-notebook from IPython.utils.capture import CapturedIO capture = CapturedIO(sys.stdout, sys.st...
- 6 kudos
- 895 Views
- 0 replies
- 0 kudos
usedlaptopcalicut.in
We Are Among The Most Reliable Used Laptop Sellers In Calicut. A Wide Variety Of Laptops From Different Brands To Suit Different Budgets Are Available At Us. The used laptops are in good condition and cost a fraction of what a brand-new laptop would....
- 895 Views
- 0 replies
- 0 kudos
- 1585 Views
- 1 replies
- 2 kudos
I have passed the Fundamentals Accreditation but I haven´t received my badge and certificate.
I have just passed Fundamentals Accreditation i dont have the badge
- 1585 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @FRANCISCO LORA​ @Kaniz Fatma​ knows more than me but you could probably submit a ticket to Databricks' Training Team here: https://help.databricks.com/s/contact-us?ReqType=training who will get back to you shortly.
- 2 kudos
- 8530 Views
- 1 replies
- 4 kudos
Getting Error "java.lang.NoSuchMethodError: org.apache.spark.sql.AnalysisException" while writing data to event hub for streaming. It is working fine if I am writing it to another data brick table
import org.apache.spark.sql._import scala.collection.JavaConverters._import com.microsoft.azure.eventhubs._import java.util.concurrent._import scala.collection.immutable._import org.apache.spark.eventhubs._import scala.concurrent.Futureimport scala.c...
- 8530 Views
- 1 replies
- 4 kudos
- 4 kudos
The dataframe to write needs to have the following schema:Column | Type ---------------------------------------------- body (required) | string or binary partitionId (*optional) | string partitionKey...
- 4 kudos
- 2296 Views
- 1 replies
- 2 kudos
iPython shell `set_next_input` not working
I'm running on 11.3 LTS. Expected Behavior:Databricks Notebook Behavior (it does nothing): You can also do `shell.set_next_input("test", replace=True)` to replace the current cell content which also doesn't work on Databricks. `set_next_input` stores...
- 2296 Views
- 1 replies
- 2 kudos
- 6119 Views
- 8 replies
- 3 kudos
Create workspace in Databricks deployed in Google Cloud using terraform
In the documentation https://registry.terraform.io/providers/databricks/databricks/latest/docs https://docs.gcp.databricks.com/dev-tools/terraform/index.html I could not find documentation on how to provision Databricks workspaces in GCP. Only cre...
- 6119 Views
- 8 replies
- 3 kudos
- 3 kudos
Hi @horatiu guja​ Does @Debayan Mukherjee​ response answer your question?If yes, would you be happy to mark it as best so that other members can find the solution more quickly? Else, we can help you with more details.
- 3 kudos
- 5750 Views
- 5 replies
- 1 kudos
DLT Pipeline failed to Start due to "The Execution Contained atleast one disallowed language
Hi , im trying to setup DLT pipeline ,its a basic pipeline for testing purpose im facing the issue while starting the pipeline , any help is appreciated Code :@dlt.table(name="dlt_bronze_cisco_hardware")def dlt_cisco_networking_bronze_hardware(): ret...
- 5750 Views
- 5 replies
- 1 kudos
- 1 kudos
Hi @Arumugam Ramachandran​ seems like you have a spark config set on your DLT job cluster that allows only python and SQL code. Check the spark config (cluster policy).In any case, the python code should work. Verify the notebook's default language, ...
- 1 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 496 | |
| 349 | |
| 287 |
