- 5249 Views
- 2 replies
- 4 kudos
Resolved! Error in accessing Delta Tables
I'm getting attached error in accessing delta lake tables in the data bricks workspaceSummary of error: Could not connect to md1n4trqmokgnhr.csnrqwqko4ho.ap-southeast-1.rds.amazonaws.com:3306 : Connection resetAttached detailed error
- 5249 Views
- 2 replies
- 4 kudos
- 4 kudos
Caused by: java.sql.SQLNonTransientConnectionException: Could not connect to md1n4trqmokgnhr.csnrqwqko4ho.ap-southeast-1.rds.amazonaws.com:3306 : Connection reset at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.get(ExceptionMapper.java:...
- 4 kudos
- 8790 Views
- 8 replies
- 4 kudos
Resolved! How to write Change Data from Delta Lake to aws dynamodb
Is there some direct way to write data from DeltaLake to AWS DynamoDB.If there is none, Is there any way to do the same.
- 8790 Views
- 8 replies
- 4 kudos
- 4 kudos
Hi @Harikrishnan P H​ ,Did @Werner Stinckens​ reply help you to resolved your issue? if yes, please mark it as best. if not, please let us know.
- 4 kudos
- 8515 Views
- 5 replies
- 2 kudos
Resolved! Databricks Data Type Conversion error
In databricks while writing data to curated layer, see error - Failed to execute user defined function (Double => decimal(38,18)). Does anyone know if faced such issue and how to resolve it.
- 8515 Views
- 5 replies
- 2 kudos
- 2 kudos
What happens if you explicitly cast it?I remember having such issues with implicit casting when goin from spark 2.x to 3.x, but these were solved by using explicit casting (not round()).
- 2 kudos
- 1144 Views
- 1 replies
- 2 kudos
The Next Databricks Office HoursOur next Office Hours session is scheduled for March 23 2022 - 8:00 am PDTÂ Do you have questions about how to set up o...
The Next Databricks Office HoursOur next Office Hours session is scheduled for March 23 2022 - 8:00 am PDTDo you have questions about how to set up or use Databricks? Do you want to get best practices for deploying your use case or tips on data archi...
- 1144 Views
- 1 replies
- 2 kudos
- 2 kudos
Signed in!
- 2 kudos
- 2862 Views
- 1 replies
- 0 kudos
- 2862 Views
- 1 replies
- 0 kudos
- 0 kudos
@Bhagwan Chaubey​ May be you can give this a try, if this is a Blob Storage account.https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-python?tabs=environment-variable-windowsFor Datalake storage, please try belowhttps://do...
- 0 kudos
- 9211 Views
- 4 replies
- 3 kudos
Resolved! Writing Spark data frame to ADLS is taking Huge time when Data Frame is of Text data.
Spark data frame with text data when schema is in Struct type spark is taking too much time to write / save / push data to ADLS or SQL Db or download as csv.
- 9211 Views
- 4 replies
- 3 kudos
- 3 kudos
@shiva Santosh​ Have to checked the count of the dataframe that you are trying to save to ADLS?As @Joseph Kambourakis​ mentioned the explode can result in 1-many rows, better to check data frame count and see if Spark OOMs in the workspace.
- 3 kudos
- 3073 Views
- 1 replies
- 2 kudos
github.com
Do I always need to manually invite members of my AAD tenant to ADB workspace, if I don't have SCIM integration configured?EDIT: solved, it works when you go through Azure Portal and get in with "Launch Workspace" button on the ADB resource overview ...
- 3073 Views
- 1 replies
- 2 kudos
- 2 kudos
Hello @pawelmitrus​ For users with Owner or Contributor roles, they should click on the "Launch Workspace" button in the Azure portal. For other users they should be explicitly granted access to the workspace to be able to login.Regards,Arvind
- 2 kudos
- 2871 Views
- 1 replies
- 4 kudos
Resolved! Databricks Community: Cluster Terminated Reason: Unexpected Launch Failure
Hi,I've been encountering the following error when I try to start a cluster, but the status page says everything is fine. Is something happening or are there other steps I can try?Time2022-03-13 14:40:51 EDTMessageCluster terminated.Reason:Unexpected...
- 2871 Views
- 1 replies
- 4 kudos
- 4 kudos
Hi @Rachel Kelley​ We have some internal service interruptions due to which we had this issue. Our engineering has applied the fix and the cluster startup works as expected. Sincerely apologies for the inconvenience caused here.Regards,Darshan
- 4 kudos
- 7546 Views
- 2 replies
- 0 kudos
How to read a compressed file in spark if the filename does not include the file extension for that compression format?
For example, let's say I have a file called some-file, which is a gzipped text file. If I try spark.read.text('some-file'), it will return a bunch of gibberish since it doesn't know that the file is gzipped. I'm looking to manually tell spark the fil...
- 7546 Views
- 2 replies
- 0 kudos
- 0 kudos
The community is field for the approval of the terms. The struggle of a great site is recommend for the norms. The value is suggested for the top of the vital paths for the finding members.
- 0 kudos
- 6304 Views
- 4 replies
- 3 kudos
- 6304 Views
- 4 replies
- 3 kudos
- 3 kudos
probably it is not in PATH, you can add it to PATH as described here https://ganeshchandrasekaran.com/how-to-install-databricks-cli-and-get-the-path-of-databricks-executable-on-windows-74f83040dde7
- 3 kudos
- 17952 Views
- 9 replies
- 2 kudos
Setup Git Integration via REST API
We are currently setting up CI/CD for our Databricks workspace using Databricks Repos following the approach described in the offical docs: https://docs.databricks.com/repos.html#best-practices-for-integrating-databricks-repos-with-cicd-workflowsObvi...
- 17952 Views
- 9 replies
- 2 kudos
- 3183 Views
- 1 replies
- 1 kudos
Resolved! concat_ws() throws AnalysisException when too many columns are supplied
Hi,i'm using concat_ws in scala to calculate a checksum for the dataframe, i.e.:df.withColumn("CHECKSUM", sha2(functions.concat_ws("", dataframe.columns.map(col): _*), 512))I have one example here with just 24 columns that already throws the followin...
- 3183 Views
- 1 replies
- 1 kudos
- 1 kudos
at least one of column names can have some strange char, whitespace or something,or at least one of column type is not compatible (for example StructType)you can separate your code to two or more steps. First generate list of columns as some variable...
- 1 kudos
- 5884 Views
- 2 replies
- 2 kudos
Resolved! How do i changes the fonts and color of the title of widget and the background color of widget?
Currently there is no documentation on how I can change the fonts and background color of widget? Is there a way to do so?
- 5884 Views
- 2 replies
- 2 kudos
- 2 kudos
I saw on roadmap presentation that in future there will be more widget options but for now is like @Kavya Manohar Parag​ said.
- 2 kudos
- 17329 Views
- 6 replies
- 5 kudos
- 17329 Views
- 6 replies
- 5 kudos
- 5 kudos
Hi @Tony Zhou​ , Were you able to run this after generating a new token?
- 5 kudos
- 3627 Views
- 2 replies
- 3 kudos
Resolved! a user has been deleted from databricks workspace . Is there any way to find who deleted the user?
a user has been deleted from databricks workspace . Is there any way to find who deleted the user?
- 3627 Views
- 2 replies
- 3 kudos
- 3 kudos
To do that you need to have enabled audit logs (if event already happened and it was not "on" I am afraid now it is too late).For Azure https://docs.microsoft.com/en-us/azure/databricks/administration-guide/account-settings/azure-diagnostic-logsFor A...
- 3 kudos
| User | Count |
|---|---|
| 1644 | |
| 791 | |
| 554 | |
| 349 | |
| 287 |
