cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

manasa
by Contributor
  • 4165 Views
  • 4 replies
  • 1 kudos

Need help to insert huge data into cosmos db from azure data lake storage using databricks

I am trying to insert 6GB of data into cosmos db using OLTP ConnectorContainer RU's:40000Cluster Config:cfg = { "spark.cosmos.accountEndpoint" : cosmosdbendpoint, "spark.cosmos.accountKey" : cosmosdbmasterkey, "spark.cosmos.database" : cosmosd...

image.png
  • 4165 Views
  • 4 replies
  • 1 kudos
Latest Reply
ImAbhishekTomar
New Contributor III
  • 1 kudos

Did anyone find solution for this, I’m also using similar clutter and RAU and data ingestion taking lot of time….?

  • 1 kudos
3 More Replies
patdev
by New Contributor III
  • 5179 Views
  • 9 replies
  • 2 kudos

text datatype not supported and data having huge data in text filed how to bring it over

Hello all,I have medical field data file and one of the field is the text field with huge data not the big problem is databrick does not support text data type so how can i bring the data over. i tried conversion, cast in various way but so far not ...

  • 5179 Views
  • 9 replies
  • 2 kudos
Latest Reply
patdev
New Contributor III
  • 2 kudos

Setting escapeQuotes to false has helped to bring huge text data in colomn.thanks

  • 2 kudos
8 More Replies
Labels