Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-19-2022 08:27 AM
Im doing the self study and want pass df column name as a parameter.
I have defined the widget
column_name= dbutils.widgets.get('column_name')
which is executing succefuly ( giving me a column name)
then Im reading the df and do some transformation and after that I want to wrtie back to datalake.
try:
(df2[column_name].write
.mode('append')
.parquet(f'/mnt/bro/Default/{table_name}'))
except Exception as e:
print(e)
raise Exception(e)
then, idea is that adf pipeline will loop through different files after.
but when i run the above code I get error
'DataFrame' object has no attribute 'column_name'
Im sorry if my explanation is not good enough, I just started learning.
Labels:
- Labels:
-
ADF Pipeline
-
Parameters
-
Widgets
1 ACCEPTED SOLUTION
Accepted Solutions
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-20-2022 07:53 AM
1 REPLY 1
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-20-2022 07:53 AM
df2.select([column_name]).write
OR
df2.select(column_name).write
![](/skins/images/582998B45490C7019731A5B3A872C751/responsive_peak/images/icon_anonymous_message.png)
![](/skins/images/582998B45490C7019731A5B3A872C751/responsive_peak/images/icon_anonymous_message.png)