- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-01-2022 09:53 AM
Try the below:
def CNN_HOF(train_df_pd, valid_df_pd, test_df_pd, params): #Hyperopt objective function
train_generator = train_data_gen.flow_from_dataframe(dataframe=train_df_pd,
directory=images_dir,
x_col='filename',
y_col=target,
target_size=(150, 150),
class_mode='categorical',
batch_size=train_batch)
valid_generator = valid_data_gen.flow_from_dataframe(dataframe=valid_df_pd,
directory=images_dir,
x_col='filename',
y_col=target,
target_size=(150, 150),
class_mode='categorical',
batch_size=valid_batch,
shuffle=False,
seed=42)
test_generator = test_data_gen.flow_from_dataframe(dataframe=test_df_pd,
directory=images_dir,
x_col='filename',
y_col=target,
target_size=(150, 150),
class_mode='categorical',
batch_size=test_batch,
shuffle=False,
seed=42)
mlflow.tensorflow.autolog()
model = model_builder(params,dense_size)
model.compile(loss="categorical_crossentropy",
optimizer=Adam(),
metrics=["accuracy"])
history = model.fit(train_generator,
steps_per_epoch=train_step,
epochs=tuner_epochs,
validation_data=valid_generator,
validation_steps=valid_step,
verbose=2)
# Evaluate the model
score = model.evaluate(test_generator, steps=1, verbose=0)
obj_metric = score[0]
return float(obj_metric)I'm assuming your dense_size, valid_batch, train_batch, test_batch, image_dir, and target are global variables. Note that train_df_pd = train_df.toPandas(), valid_df_pd = valid_df.toPandas(), and test_df_pd = test_df.toPandas() are outside of the objective function and the pandas dataframes are being brought in as arguments. Then you put the generator in the objective function in here. If this works, then the generator was the issue and we can do something to speed up this process.
I also want to note that I am returning the score as a float. I'm wondering if that score[0] is a dictionary and not a float but I casted it but you can print that type to validate.