โ02-24-2022 05:16 AM
Hi,
Can we use config file to change pyspark dataframe attribute names (root, nested of both struct and array type) .
Actually in input we are getting attributes in lowercase we need to convert them into camel case(please note we don't have any separators in attribute name to use or build camel functions or libraries.
We are trying to implement it in pyspark.
Please add your suggestions over here
Thank you
โ04-29-2022 05:04 AM
We have created config file in csv format and converted that into dict key value pairs to chanage the dataframe attribute names.
โ02-24-2022 06:46 AM
Hi @Sailaja Bโ ! Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question first. Or else I will get back to you soon. Thanks.
โ02-24-2022 03:31 PM
Hi @Sailaja Bโ , Can you show your input file if you're comfortable with it?
โ04-28-2022 09:53 AM
Hi @Sailaja Bโ
Hope you are well.
Just wanted to see if you were able to find an answer to your question and would you be happy to share it with us? It would be really helpful for the other members too.
Thanks!
โ04-29-2022 05:04 AM
We have created config file in csv format and converted that into dict key value pairs to chanage the dataframe attribute names.
โ04-29-2022 07:56 AM
Hi @Sailaja Bโ
This is awesome!
Thanks for coming in and posting the solution. We really appreciate it.
Cheers!
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group