configuration_nonstationary module

Configuration for the simulations, for the piecewise stationary single-player case.

configuration_nonstationary.CPU_COUNT = 4

Number of CPU on the local machine

configuration_nonstationary.HORIZON = 5000

HORIZON : number of time steps of the experiments. Warning Should be >= 10000 to be interesting “asymptotically”.

configuration_nonstationary.DO_PARALLEL = True

To profile the code, turn down parallel computing

configuration_nonstationary.N_JOBS = -1

Number of jobs to use for the parallel computations. -1 means all the CPU cores, 1 means no parallelization.

configuration_nonstationary.REPETITIONS = 100

REPETITIONS : number of repetitions of the experiments. Warning: Should be >= 10 to be statistically trustworthy.

configuration_nonstationary.configuration = {'append_labels': {}, 'change_labels': {0: 'klUCB', 1: 'Thompson Sampling', 4: 'DTS', 5: 'M-klUCB', 6: 'CUSUM-klUCB(Localization)', 7: 'GLR-klUCB(Local, Localization)', 8: 'GLR-klUCB(Global, Localization)'}, 'environment': [{'arm_type': <class 'Arms.Bernoulli.Bernoulli'>, 'params': {'listOfMeans': [[0.3, 0.5, 0.9], [0.3, 0.2, 0.9], [0.3, 0.2, 0.1], [0.7, 0.2, 0.1], [0.7, 0.5, 0.1]], 'changePoints': [0, 1000, 2000, 3000, 4000]}}, {'arm_type': <class 'Arms.Bernoulli.Bernoulli'>, 'params': {'listOfMeans': [[0.4, 0.5, 0.9], [0.5, 0.4, 0.7], [0.6, 0.3, 0.5], [0.7, 0.2, 0.3], [0.8, 0.1, 0.1]], 'changePoints': [0, 1000, 2000, 3000, 4000]}}], 'horizon': 5000, 'n_jobs': -1, 'nb_break_points': 4, 'plot_lowerbound': False, 'policies': [{'archtype': <class 'Policies.klUCB.klUCB'>, 'params': {'klucb': CPUDispatcher(<function klucbBern>)}, 'change_label': 'klUCB'}, {'archtype': <class 'Policies.Thompson.Thompson'>, 'params': {'posterior': <class 'Policies.Posterior.Beta.Beta'>}, 'change_label': 'Thompson Sampling'}, {'archtype': <class 'Policies.OracleSequentiallyRestartPolicy.OracleSequentiallyRestartPolicy'>, 'params': {'changePoints': [0, 1000, 2000, 3000, 4000], 'listOfMeans': [[0.3, 0.5, 0.9], [0.3, 0.2, 0.9], [0.3, 0.2, 0.1], [0.7, 0.2, 0.1], [0.7, 0.5, 0.1]], 'policy': <class 'Policies.klUCB.klUCB'>, 'reset_for_all_change': True, 'reset_for_suboptimal_change': False}}, {'archtype': <class 'Policies.Exp3S.Exp3S'>, 'params': {'horizon': 5000, 'max_nb_random_events': 4}}, {'archtype': <class 'Policies.DiscountedThompson.DiscountedThompson'>, 'params': {'posterior': <class 'Policies.Posterior.DiscountedBeta.DiscountedBeta'>, 'gamma': 0.95}, 'change_label': 'DTS'}, {'archtype': <class 'Policies.Monitored_UCB.Monitored_IndexPolicy'>, 'params': {'policy': <class 'Policies.klUCB.klUCB'>, 'horizon': 5000, 'w': 150, 'delta': 0.1}, 'change_label': 'M-klUCB'}, {'archtype': <class 'Policies.CUSUM_UCB.CUSUM_IndexPolicy'>, 'params': {'horizon': 5000, 'policy': <class 'Policies.klUCB.klUCB'>, 'max_nb_random_events': 4, 'lazy_detect_change_only_x_steps': 20, 'epsilon': 0.1, 'use_localization': True}, 'change_label': 'CUSUM-klUCB(Localization)'}, {'archtype': <class 'Policies.GLR_UCB.BernoulliGLR_IndexPolicy_WithDeterministicExploration'>, 'params': {'policy': <class 'Policies.klUCB_forGLR.klUCB_forGLR'>, 'per_arm_restart': True, 'delta': 0.00408248290463863, 'alpha0': 0.007148647364956354, 'lazy_detect_change_only_x_steps': 10, 'lazy_try_value_s_only_x_steps': 10, 'use_localization': True}, 'change_label': 'GLR-klUCB(Local, Localization)'}, {'archtype': <class 'Policies.GLR_UCB.BernoulliGLR_IndexPolicy_WithDeterministicExploration'>, 'params': {'policy': <class 'Policies.klUCB_forGLR.klUCB_forGLR'>, 'per_arm_restart': False, 'delta': 0.007071067811865475, 'alpha0': 0.00412727348049926, 'lazy_detect_change_only_x_steps': 10, 'lazy_try_value_s_only_x_steps': 10, 'use_localization': True}, 'change_label': 'GLR-klUCB(Global, Localization)'}], 'repetitions': 100, 'verbosity': 6}

This dictionary configures the experiments

configuration_nonstationary.NB_ARMS = 3

Number of arms for non-hard-coded problems (Bayesian problems)

configuration_nonstationary.LOWER = 0.0

Default value for the lower value of means

configuration_nonstationary.AMPLITUDE = 1.0

Default value for the amplitude value of means

configuration_nonstationary.VARIANCE = 0.25

Variance of Gaussian arms, if needed

configuration_nonstationary.ARM_TYPE

alias of Arms.Bernoulli.Bernoulli

configuration_nonstationary.MEANS = [0.05, 0.49999999999999994, 0.95]

Means of arms for non-hard-coded problems (non Bayesian)

configuration_nonstationary.lowerbound_on_sequence_length(horizon, gap)[source]

A function that computes the lower-bound (we will find) on the sequence length to have a reasonable bound on the delay of our change-detection algorithm.

  • It returns the smallest possible sequence length \(L = \tau_{m+1} - \tau_m\) satisfying:
\[L \geq \frac{8}{\Delta^2} \log(T).\]
configuration_nonstationary.check_condition_on_piecewise_stationary_problems(horizon, listOfMeans, changePoints)[source]

Check some conditions on the piecewise stationary problem.

configuration_nonstationary.NB_BREAK_POINTS = 4

Number of true breakpoints. They are uniformly spaced in time steps (and the first one at t=0 does not count).

configuration_nonstationary.nbArms = 3

Number of arms in the first environment

configuration_nonstationary.klucb[source]

Warning: if using Exponential or Gaussian arms, gives klExp or klGauss to KL-UCB-like policies!

configuration_nonstationary.WINDOW_SIZE = 150

Default window size \(w\) for the M-UCB and SW-UCB algorithm.