cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
User16790091296
by • Contributor II
  • 1575 Views
  • 2 replies
  • 2 kudos

How to configure Databricks token inside Docker File?

I have a docker file where I want toDownload the Databricks CLIConfigure the CLI by adding a host and tokenAnd then running a python file that hits the Databricks tokenI am able to install the CLI in the docker image, and I have a working python file...

Data Engineering
Reply
  • 1575 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
sachingawade
New Contributor II
  • 2 kudos

Hi I was facing same issue and searching for the solution but didnt get it, and now after working on it i have the solution if you want to access databricks models/download_artifacts using hostname and access token like how you do on databricks cli ...

  • 2 kudos
1 More Replies
rdobbss
by • New Contributor II
  • 793 Views
  • 2 replies
  • 0 kudos

RPC Disassociate error due to container threshold exceeding and garbage collector error when reading 23 gb multiline JSON file.

I am reading 23 gb multi line json file and flattening it using udf and writing datframe as parquet using psypark.Cluster I am using is 3 node (8 core) 64gb memory with limit to go upto 8 nodes.I am able to process 7gb file with no issue and takes ar...

Data Engineering
Reply
  • 793 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Vidula
Honored Contributor
  • 0 kudos

Hi @Ravi Dobariya​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...

  • 0 kudos
1 More Replies
Dicer
by • Valued Contributor
  • 10476 Views
  • 13 replies
  • 13 kudos

Resolved! Failed to convert Spark.sql to Pandas Dataframe using .toPandas()

I wrote the following code:​data = spark.sql (" SELECT A_adjClose, AA_adjClose, AAL_adjClose, AAP_adjClose, AAPL_adjClose FROM deltabase.a_30min_delta, deltabase.aa_30min_delta, deltabase.aal_30min_delta, deltabase.aap_30min_delta ,deltabase.aapl_30m...

Data Engineering
Reply
  • 10476 Views
  • 13 replies
  • 13 kudos
View Replies
Latest Reply
Dicer
Valued Contributor
  • 13 kudos

I just discovered a solution.Today, I opened Azure Databricks. When I imported python libraries. Databricks told me that toPandas() was deprecated and it suggested me to use toPandas.The following solution works: Use toPandas instead of toPandas() da...

  • 13 kudos
12 More Replies
flachboard
by • New Contributor
  • 2718 Views
  • 4 replies
  • 1 kudos

How do you install R packages?

I've tried this, but it doesn't appear to be working: https://community.databricks.com/s/question/0D53f00001GHVX1CAP/unable-to-install-sf-and-rgeos-r-packages-on-the-clusterWhen I run the following after that init script, I receive an error.library(r...

Data Engineering
Reply
  • 2718 Views
  • 4 replies
  • 1 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hey there @Christopher Flach​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear fr...

  • 1 kudos
3 More Replies
Labels