focus_opt package

Submodules

focus_opt.config_candidate module

class focus_opt.config_candidate.ConfigCandidate(config: dict, evaluation_function: Callable, score_aggregation: str = 'latest', score_aggregation_function: Callable = None, accepts_fidelity: bool = True, max_fidelity: int = 0)

Bases: object

Represents a candidate configuration for evaluation in an optimization process.

This class is designed to handle the evaluation of a configuration at different fidelity levels, caching the results to optimize performance. It supports aggregation of evaluation scores using different strategies.

Parameters:

  • config (dict) – The configuration dictionary to be evaluated.

  • evaluation_function (Callable) – A callable function that evaluates the configuration.

  • score_aggregation (str, optional) – The method used to aggregate scores from multiple evaluations. Options are “latest” or “average”, defaults to “latest”.

  • score_aggregation_function (Callable, optional) – A custom function to aggregate scores, defaults to None.

  • accepts_fidelity (bool, optional) – Indicates if the evaluation function accepts a fidelity level, defaults to True.

  • max_fidelity (int, optional) – The maximum fidelity level for evaluation, defaults to 0.

aggregate_evaluations()

Aggregates the evaluation scores based on the specified aggregation method.

Returns:

The aggregated evaluation score.

Return type:

float

evaluate(session_context: SessionContext)

Evaluates the candidate solution at the next fidelity level.

Parameters:

session_context (SessionContext) – The session context for managing evaluation budget and logging.

Returns:

The aggregated evaluation score.

Return type:

float

Raises:

ValueError – If the fidelity level exceeds the maximum allowed fidelity.

full_evaluation(session_context: SessionContext)

Evaluates a solution with full fidelity.

Parameters:

session_context (SessionContext) – The session context for managing evaluation budget and logging.

Returns:

The aggregated evaluation score after full evaluation.

Return type:

float

property is_fully_evaluated

Checks if the configuration has been fully evaluated.

Returns:

True if fully evaluated, False otherwise.

Return type:

bool

focus_opt.config_candidate.cached_evaluation_function(config_tuple, fidelity_level, evaluation_function, accepts_fidelity)

Global cached evaluation function to share cache across instances.

focus_opt.helpers module

exception focus_opt.helpers.OutOfBudgetError

Bases: Exception

Exception raised when the budget is exceeded.

class focus_opt.helpers.SessionContext(budget: int, start_time: int = None, total_cost: int = 0, max_time: int = None, cost_increment: int = 1, log_dir: str = 'logs/', log_results: bool = False)

Bases: object

Manages the session context for optimization processes, including budget and logging.

Parameters:

  • budget (int) – The total budget available for the session.

  • start_time (int, optional) – The start time of the session, defaults to the current time.

  • total_cost (int, optional) – The initial total cost, defaults to 0.

  • max_time (int, optional) – The maximum allowed time for the session, defaults to None.

  • cost_increment (int, optional) – The cost increment for each operation, defaults to 1.

  • log_dir (str, optional) – The directory where logs are stored, defaults to “logs/”.

  • log_results (bool, optional) – Whether to log results to a file, defaults to False.

budget_error_checks()

Performs checks for timeouts and budget exceedance.

Raises:

  • TimeoutError – If the session has timed out.

  • OutOfBudgetError – If the budget is exceeded.

can_continue_running() → bool

Determines if the session can continue running.

Returns:

True if the session can continue, False otherwise.

Return type:

bool

increment_total_cost()

Increments the total cost by the cost increment.

log_performance(best_score: float)

Logs the performance of the session to a file.

Parameters:

best_score (float) – The best score achieved in the session.

out_of_budget_check() → bool

Checks if the session has exceeded the budget.

Returns:

True if the budget is exceeded, False otherwise.

Return type:

bool

time_out_check() → bool

Checks if the session has exceeded the maximum allowed time.

Returns:

True if the session has timed out, False otherwise.

Return type:

bool

focus_opt.hp_space module

class focus_opt.hp_space.BooleanHyperParameter(name: str, proba_true: float = 0.5)

Bases: HyperParameter

Boolean hyperparameter with a probability of being True.

Parameters:

  • name (str) – The name of the hyperparameter.

  • proba_true (float, optional) – Probability of sampling True, defaults to 0.5.

sample() → bool

Samples a boolean value based on the probability.

Returns:

A sampled boolean value.

Return type:

bool

sample_neighbors(value: bool) → List[bool]

Returns the opposite boolean value as the neighbor.

Parameters:

value (bool) – The current boolean value.

Returns:

A list containing the opposite value.

Return type:

List[bool]

class focus_opt.hp_space.CategoricalHyperParameter(name: str, values: List[Any])

Bases: HyperParameter

Categorical hyperparameter with a list of possible values.

Parameters:

  • name (str) – The name of the hyperparameter.

  • values (List[Any]) – The list of possible values.

sample() → Any

Samples a value from the list of possible values.

Returns:

A sampled value.

Return type:

Any

sample_neighbors(value: Any) → List[Any]

Samples two random neighboring values from the list, excluding the current value.

Parameters:

value (Any) – The current value.

Returns:

A list of neighboring values.

Return type:

List[Any]

Raises:

ValueError – If no neighbors are available.

class focus_opt.hp_space.ContinuousHyperParameter(name: str, min_value: float, max_value: float, is_int: bool = False, step_size: float = 0.1)

Bases: HyperParameter

Continuous hyperparameter with a range of values.

Parameters:

  • name (str) – The name of the hyperparameter.

  • min_value (float) – The minimum value of the range.

  • max_value (float) – The maximum value of the range.

  • is_int (bool, optional) – Whether the values should be integers, defaults to False.

  • step_size (float, optional) – The step size for sampling neighbors, defaults to 0.1.

sample() → float

Samples a value within the range.

Returns:

A sampled value.

Return type:

float

sample_neighbors(value: float) → List[float]

Samples neighboring values based on the step size.

Parameters:

value (float) – The current value.

Returns:

A list of neighboring values.

Return type:

List[float]

class focus_opt.hp_space.HyperParameter(name: str, values: List[Any] | None = None)

Bases: ABC

Abstract base class for hyperparameters.

Parameters:

  • name (str) – The name of the hyperparameter.

  • values (Union[List[Any], None], optional) – The possible values for the hyperparameter, defaults to None.

property param_type: str

Returns the type of the hyperparameter.

Returns:

The type of the hyperparameter.

Return type:

str

abstract sample() → Any

Abstract method to sample a value for the hyperparameter.

Returns:

A sampled value.

Return type:

Any

abstract sample_neighbors(value: Any) → List[Any]

Abstract method to sample neighboring values for a given value.

Parameters:

value (Any) – The current value of the hyperparameter.

Returns:

A list of neighboring values.

Return type:

List[Any]

class focus_opt.hp_space.HyperParameterSpace(name: str, hyper_parameters: List[HyperParameter] = [])

Bases: object

Represents a space of hyperparameters for configuration sampling.

Parameters:

  • name (str) – The name of the hyperparameter space.

  • hyper_parameters (List[HyperParameter], optional) – A list of hyperparameters in the space, defaults to an empty list.

add_hp(hp: HyperParameter)

Adds a hyperparameter to the space.

Parameters:

hp (HyperParameter) – The hyperparameter to add.

sample_all_neighbors(config: Dict[str, Any]) → List[Dict[str, Any]]

Samples all possible neighboring configurations for a given configuration.

Parameters:

config (Dict[str, Any]) – The current configuration.

Returns:

A list of all neighboring configurations.

Return type:

List[Dict[str, Any]]

sample_config() → Dict[str, Any]

Samples a configuration from the hyperparameter space.

Returns:

A dictionary representing the sampled configuration.

Return type:

Dict[str, Any]

sample_configs(n_configs: int) → List[Dict[str, Any]]

Samples multiple configurations from the hyperparameter space.

Parameters:

n_configs (int) – The number of configurations to sample.

Returns:

A list of sampled configurations.

Return type:

List[Dict[str, Any]]

sample_n_neighbors(config: Dict[str, Any], n_neighbors: int = 1) → List[Dict[str, Any]]

Samples neighboring configurations for a given configuration.

Parameters:

  • config (Dict[str, Any]) – The current configuration.

  • n_neighbors (int, optional) – The number of neighbors to sample, defaults to 1.

Returns:

A list of neighboring configurations.

Return type:

List[Dict[str, Any]]

sample_unique_configs(n_configs: int) → List[Dict[str, Any]]

Samples unique configurations from the hyperparameter space.

Parameters:

n_configs (int) – The number of unique configurations to sample.

Returns:

A list of unique sampled configurations.

Return type:

List[Dict[str, Any]]

class focus_opt.hp_space.OrdinalHyperParameter(name: str, values: List[Any])

Bases: CategoricalHyperParameter

Ordinal hyperparameter with ordered values.

Parameters:

  • name (str) – The name of the hyperparameter.

  • values (List[Any]) – The ordered list of possible values.

sample_neighbors(value: Any) → List[Any]

Samples neighboring values based on the order of values.

Parameters:

value (Any) – The current value.

Returns:

A list of neighboring values.

Return type:

List[Any]

focus_opt.optimizers module

class focus_opt.optimizers.BaseOptimizer(hp_space: HyperParameterSpace, evaluation_function: Callable[[Dict[str, Any], int], float], max_fidelity: int = 1, sh_eta: float = 0.5, maximize: bool = False, score_aggregation: str = 'average', score_aggregation_function: Callable[[List[float]], float] | None = None, initial_config: dict | None = None, log_results: bool = False)

Bases: ABC

Abstract base class for all optimizers in the focus_opt package.

This class provides common functionality and interfaces for different optimization algorithms. It handles hyperparameter space management, evaluation function integration, and tracking of the best candidate found during optimization.

compare_candidates(candidate_1: ConfigCandidate, candidate_2: ConfigCandidate) → bool

Compare two candidates based on the optimization objective.

Parameters:
Returns:

True if candidate_1 is better than candidate_2 based on the objective, False otherwise.

Return type:

bool

config_to_candidate(config: dict) → ConfigCandidate

Instantiate a ConfigCandidate from a configuration dictionary.

Parameters:

config (dict) – Hyperparameter configuration.

Returns:

An instance of ConfigCandidate initialized with the given config.

Return type:

ConfigCandidate

configs_to_candidates(configs: List[dict]) → List[ConfigCandidate]

Instantiate a list of ConfigCandidates from a list of configuration dictionaries.

Parameters:

configs (List[dict]) – List of hyperparameter configurations.

Returns:

List of ConfigCandidate instances.

Return type:

List[ConfigCandidate]

abstract optimize() → ConfigCandidate

Abstract method to perform optimization.

Must be implemented by subclasses.

Returns:

The best candidate found during optimization.

Return type:

ConfigCandidate

successive_halving(candidates: List[ConfigCandidate], session_context: SessionContext, min_population_size: int | None = None) → List[ConfigCandidate]

Perform the successive halving algorithm on a list of candidates.

Parameters:

  • candidates (List[ConfigCandidate]) – List of candidates to evaluate.

  • session_context (SessionContext) – Context managing the optimization session.

  • min_population_size (Optional[int], optional) – Minimum population size to maintain. If None, no lower bound is enforced. Defaults to None.

Returns:

Reduced list of candidates after successive halving.

Return type:

List[ConfigCandidate]

update_best(candidates: List[ConfigCandidate]) → None

Update the best candidate found so far based on the current list of candidates.

Parameters:

candidates (List[ConfigCandidate]) – List of candidates to consider.

class focus_opt.optimizers.GeneticAlgorithmOptimizer(hp_space: HyperParameterSpace, evaluation_function: Callable[[Dict[str, Any], int], float], max_fidelity: int = 1, sh_eta: float = 0.5, maximize: bool = False, score_aggregation: str = 'average', score_aggregation_function: Callable[[List[float]], float] | None = None, population_size: int = 20, crossover_rate: float = 0.8, mutation_rate: float = 0.1, elitism: int = 1, tournament_size: int = 3, min_population_size: int = 5, log_results: bool = False)

Bases: BaseOptimizer

Optimizer that uses a genetic algorithm for hyperparameter optimization.

This optimizer maintains a population of candidate configurations and evolves them through selection, crossover, and mutation to find the best configuration within a given budget.

crossover(parent1: dict, parent2: dict) → dict

Perform crossover between two parent configurations to produce an offspring.

For each hyperparameter, the offspring inherits the value from one of the parents based on the crossover rate.

Parameters:

  • parent1 (dict) – The first parent configuration.

  • parent2 (dict) – The second parent configuration.

Returns:

The offspring configuration resulting from crossover.

Return type:

dict

generate_offspring(parents: List[ConfigCandidate]) → List[dict]

Generate new population configurations through crossover and mutation.

Parameters:

parents (List[ConfigCandidate]) – List of parent candidates selected for reproduction.

Returns:

List of offspring configurations.

Return type:

List[dict]

mutate(config: dict) → dict

Perform mutation on a configuration.

Each hyperparameter has a chance to be mutated based on the mutation rate. Mutation replaces the hyperparameter value with a new sampled value from its space.

Parameters:

config (dict) – The configuration to mutate.

Returns:

The mutated configuration.

Return type:

dict

optimize(budget: int = 100, max_time: int | None = None) → ConfigCandidate

Perform genetic algorithm optimization.

Evolves a population of configurations through selection, crossover, and mutation until the budget or time is exhausted.

Parameters:

  • budget (int, optional) – Total number of evaluations allowed, defaults to 100.

  • max_time (Optional[int], optional) – Maximum time (in seconds) allowed for optimization. If None, no time limit is imposed, defaults to None.

Returns:

The best candidate found during optimization.

Return type:

ConfigCandidate

select_parents(candidates: List[ConfigCandidate]) → List[ConfigCandidate]

Select parents for crossover using tournament selection.

Parameters:

candidates (List[ConfigCandidate]) – List of fully evaluated candidates.

Returns:

List of selected parent candidates.

Return type:

List[ConfigCandidate]

class focus_opt.optimizers.HillClimbingOptimizer(hp_space: HyperParameterSpace, evaluation_function: Callable[[Dict[str, Any], int], float], max_fidelity: int = 1, sh_eta: float = 0.5, maximize: bool = False, score_aggregation: str = 'average', score_aggregation_function: Callable[[List[float]], float] | None = None, initial_config: dict | None = None, warm_start: int = 0, random_restarts: int = 5, log_results: bool = False)

Bases: BaseOptimizer

Optimizer that uses the hill climbing algorithm for hyperparameter optimization.

This optimizer starts from an initial configuration (or random ones) and iteratively explores neighboring configurations to find local optima. It supports multiple random restarts to escape local optima and search for the global optimum.

hill_climbing_round(session_context: SessionContext, restart_number: int = 0) → ConfigCandidate

Perform a single hill climbing round.

Starts from an initial configuration and iteratively explores neighboring configurations to find a better candidate. Stops when no better neighbors are found or when the budget/time is exhausted.

Parameters:

  • session_context (SessionContext) – Context managing the optimization session.

  • restart_number (int, optional) – The current restart iteration number, defaults to 0.

Returns:

The best candidate found in this hill climbing round.

Return type:

ConfigCandidate

optimize(max_time: int | None = None, budget: int = 100) → ConfigCandidate

Perform hill climbing optimization with random restarts.

Initiates multiple hill climbing rounds with random restarts to explore different regions of the hyperparameter space and avoid getting stuck in local optima.

Parameters:

  • max_time (Optional[int], optional) – Maximum time (in seconds) allowed for optimization. If None, no time limit is imposed, defaults to None.

  • budget (int, optional) – Total number of evaluations allowed, defaults to 100.

Returns:

The best candidate found during optimization.

Return type:

ConfigCandidate

class focus_opt.optimizers.RandomSearchOptimizer(hp_space: HyperParameterSpace, evaluation_function: Callable[[Dict[str, Any], int], float], max_fidelity: int = 1, sh_eta: float = 0.5, maximize: bool = False, score_aggregation: str = 'average', score_aggregation_function: Callable[[List[float]], float] | None = None, log_results: bool = False)

Bases: BaseOptimizer

Optimizer that performs random search over the hyperparameter space.

This optimizer samples hyperparameter configurations randomly and evaluates them using successive halving to identify the best configuration within a given budget.

optimize(population_size: int = 10, budget: int = 100, max_time: int | None = None) → ConfigCandidate

Perform random search optimization.

Samples configurations randomly and evaluates them using successive halving until the budget or time is exhausted.

Parameters:

  • population_size (int, optional) – Number of configurations to sample in each iteration, defaults to 10.

  • budget (int, optional) – Total number of evaluations allowed, defaults to 100.

  • max_time (Optional[int], optional) – Maximum time (in seconds) allowed for optimization. If None, no time limit is imposed, defaults to None.

Returns:

The best candidate found during optimization.

Return type:

ConfigCandidate

focus_opt.optimizers.log_trial(candidate: ConfigCandidate, success: bool = True) → None

Log the result of a trial evaluation.

Parameters:

  • candidate (ConfigCandidate) – The candidate that was evaluated.

  • success (bool, optional) – Whether the evaluation was successful, defaults to True.

Module contents