cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

MEMORY_ONLY not working

gpzz
New Contributor II

val doubledAmount = premiumCustomers.map(x=>(x._1, x._2*2)).persist(StorageLevel.MEMORY_ONLY)

error: not found: value StorageLevel

2 REPLIES 2

Chaitanya_Raju
Honored Contributor

Hi @Gaurav Poojaryโ€‹ ,

Can you please try the below as displayed in the image it is working for me without any issues.

2Happy Learning!!

Thanks for reading and like if this is useful and for improvements or feedback please comment.

gpzz
New Contributor II

@Ratna Chaitanya Raju Bandaruโ€‹ 

how to convert to rdd before storing. I want to do some transformations

i am attaching the whole code

val input = sc.textFile("dbfs:/FileStore/shared_uploads/Recent9/customerorders.csv")

val mappedInput = input.map(x=>(x.split(",")(0), x.split(",")(2).toFloat))

val totalByCustomer = mappedInput.reduceByKey(_+_)

val premiumCustomers = totalByCustomer.filter(x => x._2 > 5000)

val doubledAmount = premiumCustomers.map(x=>(x._1, x._2*2)).persist(StorageLevel.MEMORY_ONLY)

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group