configuration_all_singleplayer module

Configuration for the simulations, to test all the single-player policies.

configuration_all_singleplayer.CPU_COUNT = 4

Number of CPU on the local machine

configuration_all_singleplayer.HORIZON = 10000

HORIZON : number of time steps of the experiments. Warning Should be >= 10000 to be interesting “asymptotically”.

configuration_all_singleplayer.DO_PARALLEL = True

To profile the code, turn down parallel computing

configuration_all_singleplayer.N_JOBS = -1

Number of jobs to use for the parallel computations. -1 means all the CPU cores, 1 means no parallelization.

configuration_all_singleplayer.REPETITIONS = 4

REPETITIONS : number of repetitions of the experiments. Warning: Should be >= 10 to be statistically trustworthy.

configuration_all_singleplayer.EPSILON = 0.1

Parameters for the epsilon-greedy and epsilon-… policies.

configuration_all_singleplayer.TEMPERATURE = 0.05

Temperature for the Softmax policies.

configuration_all_singleplayer.UNBOUNDED_VARIANCE = 1

Variance of unbounded Gaussian arms

configuration_all_singleplayer.NB_ARMS = 9

Number of arms for non-hard-coded problems (Bayesian problems)

configuration_all_singleplayer.LOWER = 0.0

Default value for the lower value of means

configuration_all_singleplayer.AMPLITUDE = 1.0

Default value for the amplitude value of means

configuration_all_singleplayer.VARIANCE = 0.05

Variance of Gaussian arms

configuration_all_singleplayer.ARM_TYPE

alias of Arms.Bernoulli.Bernoulli

configuration_all_singleplayer.ENVIRONMENT_BAYESIAN = False

True to use bayesian problem

configuration_all_singleplayer.USE_FULL_RESTART = True

True to use full-restart Doubling Trick

configuration_all_singleplayer.configuration = {'append_labels': {}, 'cache_rewards': False, 'change_labels': {}, 'environment': [{'arm_type': <class 'Arms.Bernoulli.Bernoulli'>, 'params': [0.1, 0.2, 0.30000000000000004, 0.4, 0.5, 0.6, 0.7000000000000001, 0.8, 0.9]}], 'horizon': 10000, 'n_jobs': -1, 'plot_lowerbound': True, 'policies': [{'archtype': <class 'Policies.Uniform.Uniform'>, 'params': {}}, {'archtype': <class 'Policies.UniformOnSome.UniformOnSome'>, 'params': {'armIndexes': [0, 1]}}, {'archtype': <class 'Policies.EmpiricalMeans.EmpiricalMeans'>, 'params': {}}, {'archtype': <class 'Policies.TakeRandomFixedArm.TakeRandomFixedArm'>, 'params': {}}, {'archtype': <class 'Policies.UniformOnSome.UniformOnSome'>, 'params': {'armIndexes': [0, 1]}}, {'archtype': <class 'Policies.TakeFixedArm.TakeFixedArm'>, 'params': {'armIndex': 1}}, {'archtype': <class 'Policies.TakeFixedArm.TakeFixedArm'>, 'params': {'armIndex': 0}}, {'archtype': <class 'Policies.TakeFixedArm.TakeFixedArm'>, 'params': {'armIndex': 0}}, {'archtype': <class 'Policies.TakeFixedArm.TakeFixedArm'>, 'params': {'armIndex': 1}}, {'archtype': <class 'Policies.EpsilonGreedy.EpsilonGreedy'>, 'params': {'epsilon': 0.1}}, {'archtype': <class 'Policies.EpsilonGreedy.EpsilonDecreasing'>, 'params': {'epsilon': 0.1}}, {'archtype': <class 'Policies.EpsilonGreedy.EpsilonExpDecreasing'>, 'params': {'epsilon': 0.1, 'decreasingRate': 0.005}}, {'archtype': <class 'Policies.EpsilonGreedy.EpsilonFirst'>, 'params': {'epsilon': 0.1, 'horizon': 10000}}, {'archtype': <class 'Policies.ExploreThenCommit.ETC_KnownGap'>, 'params': {'horizon': 10000, 'gap': 0.05}}, {'archtype': <class 'Policies.ExploreThenCommit.ETC_RandomStop'>, 'params': {'horizon': 10000}}, {'archtype': <class 'Policies.Softmax.Softmax'>, 'params': {'temperature': 0.05}}, {'archtype': <class 'Policies.Softmax.SoftmaxDecreasing'>, 'params': {}}, {'archtype': <class 'Policies.Softmax.SoftMix'>, 'params': {}}, {'archtype': <class 'Policies.Softmax.SoftmaxWithHorizon'>, 'params': {'horizon': 10000}}, {'archtype': <class 'Policies.BoltzmannGumbel.BoltzmannGumbel'>, 'params': {'C': 0.5}}, {'archtype': <class 'Policies.Exp3.Exp3'>, 'params': {'gamma': 0.001}}, {'archtype': <class 'Policies.Exp3.Exp3Decreasing'>, 'params': {'gamma': 0.001}}, {'archtype': <class 'Policies.Exp3.Exp3SoftMix'>, 'params': {}}, {'archtype': <class 'Policies.Exp3.Exp3WithHorizon'>, 'params': {'horizon': 10000}}, {'archtype': <class 'Policies.Exp3.Exp3ELM'>, 'params': {'delta': 0.1}}, {'archtype': <class 'Policies.Exp3PlusPlus.Exp3PlusPlus'>, 'params': {}}, {'archtype': <class 'Policies.ProbabilityPursuit.ProbabilityPursuit'>, 'params': {'beta': 0.5}}, {'archtype': <class 'Policies.Hedge.Hedge'>, 'params': {'epsilon': 0.5}}, {'archtype': <class 'Policies.Hedge.HedgeDecreasing'>, 'params': {}}, {'archtype': <class 'Policies.Hedge.HedgeWithHorizon'>, 'params': {'horizon': 10000}}, {'archtype': <class 'Policies.UCB.UCB'>, 'params': {}}, {'archtype': <class 'Policies.Experimentals.UCBlog10.UCBlog10'>, 'params': {}}, {'archtype': <class 'Policies.Experimentals.UCBwrong.UCBwrong'>, 'params': {}}, {'archtype': <class 'Policies.UCBalpha.UCBalpha'>, 'params': {'alpha': 1}}, {'archtype': <class 'Policies.Experimentals.UCBlog10alpha.UCBlog10alpha'>, 'params': {'alpha': 1}}, {'archtype': <class 'Policies.UCBmin.UCBmin'>, 'params': {}}, {'archtype': <class 'Policies.UCBplus.UCBplus'>, 'params': {}}, {'archtype': <class 'Policies.UCBrandomInit.UCBrandomInit'>, 'params': {}}, {'archtype': <class 'Policies.Experimentals.UCBcython.UCBcython'>, 'params': {}}, {'archtype': <class 'Policies.UCBV.UCBV'>, 'params': {}}, {'archtype': <class 'Policies.UCBVtuned.UCBVtuned'>, 'params': {}}, {'archtype': <class 'Policies.SlidingWindowUCB.SWUCB'>, 'params': {}}, {'archtype': <class 'Policies.SlidingWindowUCB.SWUCBPlus'>, 'params': {}}, {'archtype': <class 'Policies.DiscountedUCB.DiscountedUCB'>, 'params': {}}, {'archtype': <class 'Policies.DiscountedUCB.DiscountedUCBPlus'>, 'params': {}}, {'archtype': <class 'Policies.SlidingWindowRestart.SWR_UCB'>, 'params': {}}, {'archtype': <class 'Policies.SlidingWindowRestart.SWR_UCBalpha'>, 'params': {'alpha': 0.5}}, {'archtype': <class 'Policies.SparseUCB.SparseUCB'>, 'params': {'alpha': 4, 'sparsity': 2}}, {'archtype': <class 'Policies.SparseklUCB.SparseklUCB'>, 'params': {'sparsity': 2}}, {'archtype': <class 'Policies.MOSS.MOSS'>, 'params': {}}, {'archtype': <class 'Policies.MOSSH.MOSSH'>, 'params': {'horizon': 10000}}, {'archtype': <class 'Policies.MOSSAnytime.MOSSAnytime'>, 'params': {'alpha': 1.35}}, {'archtype': <class 'Policies.MOSSExperimental.MOSSExperimental'>, 'params': {}}, {'archtype': <class 'Policies.OCUCB.OCUCB'>, 'params': {'eta': 1.1, 'rho': 1}}, {'archtype': <class 'Policies.CPUCB.CPUCB'>, 'params': {}}, {'archtype': <class 'Policies.DMED.DMEDPlus'>, 'params': {}}, {'archtype': <class 'Policies.DMED.DMED'>, 'params': {}}, {'archtype': <class 'Policies.Thompson.Thompson'>, 'params': {'posterior': <class 'Policies.Posterior.Beta.Beta'>}}, {'archtype': <class 'Policies.Thompson.Thompson'>, 'params': {'posterior': <class 'Policies.Posterior.Gauss.Gauss'>}}, {'archtype': <class 'Policies.Experimentals.ThompsonRobust.ThompsonRobust'>, 'params': {'posterior': <class 'Policies.Posterior.Beta.Beta'>}}, {'archtype': <class 'Policies.klUCB.klUCB'>, 'params': {'klucb': CPUDispatcher(<function klucbBern>)}}, {'archtype': <class 'Policies.SlidingWindowRestart.SWR_klUCB'>, 'params': {'klucb': CPUDispatcher(<function klucbBern>)}}, {'archtype': <class 'Policies.klUCBloglog.klUCBloglog'>, 'params': {'klucb': CPUDispatcher(<function klucbBern>)}}, {'archtype': <class 'Policies.Experimentals.klUCBlog10.klUCBlog10'>, 'params': {'klucb': CPUDispatcher(<function klucbBern>)}}, {'archtype': <class 'Policies.Experimentals.klUCBloglog10.klUCBloglog10'>, 'params': {'klucb': CPUDispatcher(<function klucbBern>)}}, {'archtype': <class 'Policies.klUCBPlus.klUCBPlus'>, 'params': {'klucb': CPUDispatcher(<function klucbBern>)}}, {'archtype': <class 'Policies.klUCBH.klUCBH'>, 'params': {'horizon': 10000, 'klucb': CPUDispatcher(<function klucbBern>)}}, {'archtype': <class 'Policies.klUCBHPlus.klUCBHPlus'>, 'params': {'horizon': 10000, 'klucb': CPUDispatcher(<function klucbBern>)}}, {'archtype': <class 'Policies.klUCBPlusPlus.klUCBPlusPlus'>, 'params': {'horizon': 10000, 'klucb': CPUDispatcher(<function klucbBern>)}}, {'archtype': <class 'Policies.BayesUCB.BayesUCB'>, 'params': {'posterior': <class 'Policies.Posterior.Beta.Beta'>}}, {'archtype': <class 'Policies.AdBandits.AdBandits'>, 'params': {'alpha': 0.5, 'horizon': 10000}}, {'archtype': <class 'Policies.ApproximatedFHGittins.ApproximatedFHGittins'>, 'params': {'alpha': 0.5, 'horizon': 10500}}, {'archtype': <class 'Policies.Experimentals.UnsupervisedLearning.UnsupervisedLearning'>, 'params': {}}, {'archtype': <class 'Policies.OSSB.OSSB'>, 'params': {'epsilon': 0.01, 'gamma': 0.0}}, {'archtype': <class 'Policies.BESA.BESA'>, 'params': {'horizon': 10000, 'minPullsOfEachArm': 1, 'randomized_tournament': True, 'random_subsample': True, 'non_binary': False, 'non_recursive': False}}, {'archtype': <class 'Policies.UCBdagger.UCBdagger'>, 'params': {'horizon': 10000}}, {'archtype': <class 'Policies.UCBoost.UCB_bq'>, 'params': {}}, {'archtype': <class 'Policies.UCBoost.UCB_h'>, 'params': {}}, {'archtype': <class 'Policies.UCBoost.UCB_lb'>, 'params': {}}, {'archtype': <class 'Policies.UCBoost.UCBoost_bq_h_lb'>, 'params': {}}, {'archtype': <class 'Policies.UCBoost.UCBoostEpsilon'>, 'params': {'epsilon': 0.1}}, {'archtype': <class 'Policies.Experimentals.UCBoost_cython.UCB_bq'>, 'params': {}}, {'archtype': <class 'Policies.Experimentals.UCBoost_cython.UCB_h'>, 'params': {}}, {'archtype': <class 'Policies.Experimentals.UCBoost_cython.UCB_lb'>, 'params': {}}, {'archtype': <class 'Policies.Experimentals.UCBoost_cython.UCBoost_bq_h_lb'>, 'params': {}}, {'archtype': <class 'Policies.Experimentals.UCBoost_cython.UCBoostEpsilon'>, 'params': {'epsilon': 0.1}}, {'archtype': <class 'Policies.Experimentals.UCBoost_cython.UCBoostEpsilon'>, 'params': {'epsilon': 0.05}}, {'archtype': <class 'Policies.Experimentals.UCBoost_cython.UCBoostEpsilon'>, 'params': {'epsilon': 0.01}}, {'archtype': <class 'Policies.Experimentals.UCBcython.UCBcython'>, 'params': {'alpha': 4.0}}, {'archtype': <class 'Policies.Experimentals.UCBcython.UCBcython'>, 'params': {'alpha': 1.0}}, {'archtype': <class 'Policies.Experimentals.UCBcython.UCBcython'>, 'params': {'alpha': 0.5}}, {'archtype': <class 'Policies.Exp3PlusPlus.Exp3PlusPlus'>, 'params': {}}, {'archtype': <class 'Policies.DiscountedThompson.DiscountedThompson'>, 'params': {'posterior': <class 'Policies.Posterior.DiscountedBeta.DiscountedBeta'>, 'gamma': 0.99}}, {'archtype': <class 'Policies.Exp3R.Exp3R'>, 'params': {'horizon': 10000}}, {'archtype': <class 'Policies.Exp3R.Exp3RPlusPlus'>, 'params': {'horizon': 10000}}, {'archtype': <class 'Policies.AdSwitch.AdSwitch'>, 'params': {'horizon': 10000}}, {'archtype': <class 'Policies.LM_DSEE.LM_DSEE'>, 'params': {'nu': 0.25, 'DeltaMin': 0.1, 'a': 1, 'b': 0.25}}, {'archtype': <class 'Policies.CUSUM_UCB.CUSUM_IndexPolicy'>, 'params': {'horizon': 10000, 'max_nb_random_events': 1, 'policy': <class 'Policies.UCB.UCB'>, 'per_arm_restart': True}}, {'archtype': <class 'Policies.CUSUM_UCB.PHT_IndexPolicy'>, 'params': {'horizon': 10000, 'max_nb_random_events': 1, 'policy': <class 'Policies.UCB.UCB'>, 'per_arm_restart': True}}, {'archtype': <class 'Policies.SWHash_UCB.SWHash_IndexPolicy'>, 'params': {'alpha': 1, 'lmbda': 1, 'policy': <class 'Policies.UCB.UCB'>}}, {'archtype': <class 'Policies.SlidingWindowRestart.SlidingWindowRestart'>, 'params': {'policy': <class 'Policies.UCB.UCB'>}}, {'archtype': <class 'Policies.SlidingWindowUCB.SWUCB'>, 'params': {'alpha': 1, 'tau': 500}}, {'archtype': <class 'Policies.SlidingWindowUCB.SWUCBPlus'>, 'params': {'horizon': 10000, 'alpha': 1}}, {'archtype': <class 'Policies.DiscountedUCB.DiscountedUCB'>, 'params': {'alpha': 1, 'gamma': 0.9}}, {'archtype': <class 'Policies.DiscountedUCB.DiscountedUCBPlus'>, 'params': {'max_nb_random_events': 1, 'alpha': 1, 'horizon': 10000}}, {'archtype': <class 'Policies.Monitored_UCB.Monitored_IndexPolicy'>, 'params': {'horizon': 10000, 'max_nb_random_events': 1, 'delta': 0.1, 'policy': <class 'Policies.UCB.UCB'>}}, {'archtype': <class 'Policies.Monitored_UCB.Monitored_IndexPolicy'>, 'params': {'policy': <class 'Policies.UCB.UCB'>, 'horizon': 10000, 'w': 80, 'b': 29.196611105981823}}, {'archtype': <class 'Policies.OracleSequentiallyRestartPolicy.OracleSequentiallyRestartPolicy'>, 'params': {'changePoints': [], 'policy': <class 'Policies.UCB.UCB'>, 'per_arm_restart': True}}, {'archtype': <class 'Policies.GLR_UCB.BernoulliGLR_IndexPolicy'>, 'params': {'horizon': 10000, 'policy': <class 'Policies.UCB.UCB'>, 'per_arm_restart': True, 'max_nb_random_events': 1}}, {'archtype': <class 'Policies.GLR_UCB.BernoulliGLR_IndexPolicy_WithTracking'>, 'params': {'horizon': 10000, 'policy': <class 'Policies.UCB.UCB'>, 'per_arm_restart': True, 'max_nb_random_events': 1}}, {'archtype': <class 'Policies.GLR_UCB.GaussianGLR_IndexPolicy'>, 'params': {'horizon': 10000, 'policy': <class 'Policies.UCB.UCB'>, 'per_arm_restart': True, 'max_nb_random_events': 1}}, {'archtype': <class 'Policies.GLR_UCB.GaussianGLR_IndexPolicy_WithTracking'>, 'params': {'horizon': 10000, 'policy': <class 'Policies.UCB.UCB'>, 'per_arm_restart': True, 'max_nb_random_events': 1}}, {'archtype': <class 'Policies.GLR_UCB.SubGaussianGLR_IndexPolicy'>, 'params': {'horizon': 10000, 'policy': <class 'Policies.UCB.UCB'>, 'per_arm_restart': True, 'max_nb_random_events': 1}}], 'repetitions': 4, 'verbosity': 6}

This dictionary configures the experiments

configuration_all_singleplayer.nbArms = 2

Number of arms in the first environment

configuration_all_singleplayer.klucb

Warning: if using Exponential or Gaussian arms, gives klExp or klGauss to KL-UCB-like policies!