cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Veeru245
by New Contributor
  • 530 Views
  • 0 replies
  • 0 kudos

Autoloader Solution for Binary files

We have solution implemented for ingesting binary file ( .ZIP ) into delta lake, Currently we are using the below solution within our pipeline.Unzip the file and extract the XML file.Parse the XML using python libraries.Flatten the nested xml columns...

  • 530 Views
  • 0 replies
  • 0 kudos
data_scientist
by New Contributor II
  • 1206 Views
  • 2 replies
  • 2 kudos

Resolved! how to load a .w2v format saved model in databricks

Hi,I am trying load a pre-trained word2vec model which has been saved in .w2v format in databricks. I am not able to load this file . Help me with the correct command.

  • 1206 Views
  • 2 replies
  • 2 kudos
Latest Reply
Kaniz
Community Manager
  • 2 kudos

Hi @sonam de​ , To save models, use the MLflow functions log_model and save_model. You can also save models using their native APIs onto Databricks File System (DBFS). For MLlib models, use ML Pipelines.To export models for serving individual predict...

  • 2 kudos
1 More Replies
tourist_on_road
by New Contributor
  • 4501 Views
  • 1 replies
  • 0 kudos

How to read binary data in pyspark

I'm reading binary file http://snap.stanford.edu/data/amazon/productGraph/image_features/image_features.b using pyspark.from io importStringIO import array img_embedding_file = sc.binaryRecords("s3://bucket/image_features.b",4106)def mapper(featur...

  • 4501 Views
  • 1 replies
  • 0 kudos
Latest Reply
shyam_9
Valued Contributor
  • 0 kudos

Hi @tourist_on_road, please go through the below spark docs,https://spark.apache.org/docs/2.3.0/api/python/pyspark.html#pyspark.SparkContext.binaryFiles

  • 0 kudos
Labels