It can be daunting to try different parameters one by one. Valohai offers a mechanism to do hyperparameter searches using parallel executions and a grid search. These are called tasks.

When starting a task, instead of a single value for a single hyperparameter, you get to define multiple values at once. There are various modes to choose from e.g. Single, Multiple, Linear, Logspace, and Random.

### Single

Single means just a single value for a hyperparameter you do not want to search for.

### Multiple

Multiple means a list of all the values to try for a specific hyperparameter. For example, here we are trying out 4 different values (0.81, 0.84, 0.86, and 0.91).

### Logspace

Logspace is a search with values inside a specific range in logarithmic space. For example, here we are trying out 4 different values between 2^2 - 2^8 (base^start - base^end)

### Random

Finally, if you want to “gamble”, you can search randomly with a specified range and distribution. For example, here we are trying out 10 different random values between 0.001 and 0.002.

### Grid search

When we are searching for multiple values for multiple hyperparameters, all permutations are searched. This is also called grid search.

For example, here we have 6 different values for learning_rate and 6 different values for dropout. In total this is 6*6 = 36 executions. Valohai calculates the number of permutations for you and you can see it in the right bottom corner of this screenshot.

## Comments

0 comments

Please sign in to leave a comment.