cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

How to add Columns for Automatic Fill on Pandas Python

omsas
New Contributor

1. I have data x,I would like to create a new column with the condition that the value are 1, 2 or 3

2. The name of the column is SHIFT where this SHIFT column will be filled automatically if the TIME_CREATED column meets the conditions.

3. the condition are :

# 00:00:00 - 07.59.59 = Shift 1

# 08:00:00 - 15:59:59 = Shift 2

# 16:00:00 - 23:59:59 = Shift 3

4. on the above conditions apply only to Time as hours:minutes:seconds

5. I attach the link of the py and ipynb files for test and study

6. I've tried the test only for the selection using a query but it's still stuck and it only shows up for 1 day.

https://drive.google.com/drive/folders/1xdDwNfbDNzYb2SfJ_h-DurY_KXFAj9rl?usp=sharing

Columns TableResult of testedThanks, 

2 REPLIES 2

-werners-
Esteemed Contributor III

seems like a simple case statement, or elif in python, no?

Ryan_Chynoweth
Esteemed Contributor

You an do something like this in pandas. Note there could be a more performant way to do this too.

import pandas as pd
import numpy as np
 
df = pd.DataFrame({'a':[1,2,3,4]})
df.head()
>      a
> 0  1
> 1  2
> 2  3
> 3  4
 
conditions = [(df['a'] <=2), (df['a']>2)]
values = ['value1', 'value2']
 
 
df['b'] = np.select(conditions, values)
 
df.head()
         a       b
> 0  1  value1
> 1  2  value1
> 2  3  value2
> 3  4  value2

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group