cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

MEMORY_ONLY not working

gpzz
New Contributor II

val doubledAmount = premiumCustomers.map(x=>(x._1, x._2*2)).persist(StorageLevel.MEMORY_ONLY)

error: not found: value StorageLevel

2 REPLIES 2

Chaitanya_Raju
Honored Contributor

Hi @Gaurav Poojary​ ,

Can you please try the below as displayed in the image it is working for me without any issues.

2Happy Learning!!

gpzz
New Contributor II

@Ratna Chaitanya Raju Bandaru​ 

how to convert to rdd before storing. I want to do some transformations

i am attaching the whole code

val input = sc.textFile("dbfs:/FileStore/shared_uploads/Recent9/customerorders.csv")

val mappedInput = input.map(x=>(x.split(",")(0), x.split(",")(2).toFloat))

val totalByCustomer = mappedInput.reduceByKey(_+_)

val premiumCustomers = totalByCustomer.filter(x => x._2 > 5000)

val doubledAmount = premiumCustomers.map(x=>(x._1, x._2*2)).persist(StorageLevel.MEMORY_ONLY)

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.