python - SciKit-learn grid search own scoring object syntax -
i want search hyper-parameters in convolutional net built keras. using kerasclassifier , gridsearchcv scikit-learn in line intro given here machinelearningmastery. typically scikit-learn optimizes on 'accuracy'
, network runs image segmentation optimizing jaccard index. need define own scoring object grid search using make_scorer explaned here make_scorer , here defining scoring strategy. code section below shows implementation, getting error in model.compile(optimizer=optimizer, loss=eval_loss, metrics=(['eval_func'])
, not know specify in metrics. default 'accuracy'
assume in case 'eval_func'
(which works when not doing grid search) or 'score'
neither of these works in case.
what right syntax?
def eval_func(y_true, y_pred): '''evaluation function dice or jaccard, set global var jaccard=true''' if jaccard: return jaccard_index(y_true, y_pred) else: return dice_coef(y_true, y_pred) def get_unet(batch_size=32, decay=0, dropout_rate=0.5, weight_constraint=0): '''create u-net model''' dim = 32 inputs = input((3, image_cols, image_rows)) # modified take 3 color channel input conv1 = convolution2d(dim, 3, 3, activation='relu', border_mode='same', w_constraint=weight_constraint)(inputs) conv1 = convolution2d(dim, 3, 3, activation='relu', border_mode='same', w_constraint=weight_constraint)(conv1) pool1 = maxpooling2d(pool_size=(2, 2))(conv1) pool1 = dropout(dropout_rate)(pool1) # dropout added layers ... more layers ... conv10 = convolution2d(1, 1, 1, activation='sigmoid')(conv9) model = model(input=inputs, output=conv10) optimizer = adam(lr=lr, decay=decay) model.compile(optimizer=optimizer, loss=eval_loss, metrics=(['eval_func']) return model def run_grid_search(): '''optimize model parameters grid search''' ... loading data ... model = kerasclassifier(build_fn=get_unet, verbose=1, nb_epoch=num_epoch, shuffle=true) # define grid search parameters batch_size = [16, 32, 48] decay = [0, 0.002, 0.004] param_grid = dict(batch_size=batch_size, decay=decay) # create scoring object score = make_scorer(eval_func, greater_is_better=true) grid = gridsearchcv(estimator=model, param_grid=param_grid, scoring=score, n_jobs=1, verbose=1) grid_result = grid.fit(x_aug, y_aug)
here last part of error getting both using 'eval_func' , 'score':
file "c:\program files\anaconda2\lib\site-packages\keras\metrics.py", line 216 , in return get_from_module(identifier, globals(), 'metric') file "c:\program files\anaconda2\lib\site-packages\keras\utils\generic_utils.p y", line 16, in get_from_module str(identifier)) exception: invalid metric: eval_func
Comments
Post a Comment