1. List of experiments presented with notebooks¶
- Easily creating various Multi-Armed Bandit problems, explains the interface of the
- A simple example of Single-Player simulation, comparing
UCB1(for two values of $\alpha$, 1 and 1/2),
- Do we even need UCB? demonstrates the need for an algorithm smarter than the naive
- Lai-Robbins lower-bound for doubling-tricks algorithms with full restart.
Active research on Single-Player MAB¶
- Exploring different doubling tricks for different kinds of regret bounds.
- Experiments of statistical tests for piecewise stationary bandits
- Demonstrations of Single-Player Simulations for Non-Stationary Bandits.
- A simple example of Multi-Player simulation with 4 Centralized Algorithms, comparing
- A simple example of Multi-Player simulation with 2 Decentralized Algorithms, comparing
Selfish(for the “collision avoidance” part) combined with
Thompson Samplingfor learning the arms. Spoiler:
- Can we use a (non-online) Unsupervised Learning algorithm for (online) Bandit problem ?
- Can we use a computationally expensive Black-Box Bayesian optimization algorithm for (online) Bandit problem ?
2. Question: How to read these documents?¶
2.a. View the notebooks statically 📝¶
2.b. Play with the notebooks dynamically (on MyBinder) 💥¶
Anyone can use the mybinder.org website (by clicking on the icon above) to run the notebook in her/his web-browser. You can then play with it as long as you like, for instance by modifying the values or experimenting with the code.
- Do_we_even_need_UCB.ipynb Binder
- Easily_creating_MAB_problems.ipynb Binder
- Example_of_a_small_Single-Player_Simulation.ipynb Binder
- Example_of_a_small_Multi-Player_Simulation__with_Centralized_Algorithms.ipynb Binder
- Example_of_a_small_Multi-Player_Simulation__with_rhoRand_and_Selfish_Algorithms.ipynb Binder
- Lai_Robbins_Lower_Bound_for_Doubling_Trick_with_Restarting_Algorithms.ipynb Binder
- Exploring different doubling tricks for different kinds of regret bounds.ipynb Binder
- Experiments of statistical tests for piecewise stationary bandits.ipynb Binder
- Demonstrations of Single-Player Simulations for Non-Stationary-Bandits.ipynb Binder
2.c. Play with the notebooks dynamically (on Google Colab) 💥¶
Anyone can use the colab.research.google.com/notebook website (by clicking on the icon above) to run the notebook in her/his web-browser. You can then play with it as long as you like, for instance by modifying the values or experimenting with the code.
- Do_we_even_need_UCB.ipynb Google Colab
- Easily_creating_MAB_problems.ipynb Google Colab
- Example_of_a_small_Single-Player_Simulation.ipynb Google Colab
- Example_of_a_small_Multi-Player_Simulation__with_Centralized_Algorithms.ipynb Google Colab
- Example_of_a_small_Multi-Player_Simulation__with_rhoRand_and_Selfish_Algorithms.ipynb Google Colab
- Lai_Robbins_Lower_Bound_for_Doubling_Trick_with_Restarting_Algorithms.ipynb Google Colab
- Exploring different doubling tricks for different kinds of regret bounds.ipynb Google Colab
- Experiments of statistical tests for piecewise stationary bandits.ipynb Google Colab
- Demonstrations of Single-Player Simulations for Non-Stationary-Bandits.ipynb Google Colab
3. Question: Requirements to run the notebooks locally?¶
3.a. Jupyter Notebook and IPython¶
sudo pip install jupyter ipython
It will also install all the dependencies, afterward you should have a
jupyter-notebook command (or a
jupyter command, to be ran as
jupyter notebook) available in your
$ whereis jupyter-notebook jupyter-notebook: /usr/local/bin/jupyter-notebook $ jupyter-notebook --version # version >= 4 is recommended 4.4.1
3.b. My numerical environment,
- First, install its dependencies (
pip install -r requirements).
- Then, either install it (not yet), or be sure to work in the main folder.
Note: it’s probably better to use virtualenv, if you like it. I never really understood how and why virtualenv are useful, but if you know why, you should know how to use it.