# Jupyter Notebooks 📓 by Naereen @ GitHub¶

This folder hosts some Jupyter Notebooks, to present in a nice format some numerical experiments for my SMPyBandits project.

The wonderful Jupyter tools is awesome to write interactive and nicely presented 🐍 Python simulations!

## 1. List of experiments presented with notebooks¶

### MAB problems¶

- Easily creating various Multi-Armed Bandit problems, explains the interface of the
`Environment.MAB`

module.

### Single-Player simulations¶

- A simple example of Single-Player simulation, comparing
`UCB1`

(for two values of $\alpha$, 1 and 1/2),`Thompson Sampling`

,`BayesUCB`

and`kl-UCB`

. - Do we even need UCB? demonstrates the need for an algorithm smarter than the naive
`EmpiricalMeans`

. - Lai-Robbins lower-bound for doubling-tricks algorithms with full restart.

### Active research on Single-Player MAB¶

- Exploring different doubling tricks for different kinds of regret bounds.
- Experiments of statistical tests for piecewise stationary bandits
- Demonstrations of Single-Player Simulations for Non-Stationary Bandits.

### Multi-Player simulations¶

- A simple example of Multi-Player simulation with 4 Centralized Algorithms, comparing
`CentralizedMultiplePlay`

and`CentralizedIMP`

with`UCB`

and`Thompson Sampling`

. - A simple example of Multi-Player simulation with 2 Decentralized Algorithms, comparing
`rhoRand`

and`Selfish`

(for the “collision avoidance” part) combined with`UCB`

and`Thompson Sampling`

for learning the arms. Spoiler:`Selfish`

beats`rhoRand`

!

## (Old) Experiments¶

- Can we use a (non-online) Unsupervised Learning algorithm for (online) Bandit problem ?
- Can we use a computationally expensive Black-Box Bayesian optimization algorithm for (online) Bandit problem ?

## 2. Question: *How to read these documents*?¶

### 2.a. View the *notebooks* statically 📝¶

- Either directly in GitHub: see the list of notebooks;
- Or on nbviewer.jupiter.org: list of notebooks.

### 2.b. Play with the *notebooks* dynamically (on MyBinder) 💥¶

Anyone can use the mybinder.org website (by clicking on the icon above) to run the notebook in her/his web-browser. You can then play with it as long as you like, for instance by modifying the values or experimenting with the code.

- Do_we_even_need_UCB.ipynb Binder
- Easily_creating_MAB_problems.ipynb Binder
- Example_of_a_small_Single-Player_Simulation.ipynb Binder
- Example_of_a_small_Multi-Player_Simulation__with_Centralized_Algorithms.ipynb Binder
- Example_of_a_small_Multi-Player_Simulation__with_rhoRand_and_Selfish_Algorithms.ipynb Binder
- Lai_Robbins_Lower_Bound_for_Doubling_Trick_with_Restarting_Algorithms.ipynb Binder
- Exploring different doubling tricks for different kinds of regret bounds.ipynb Binder
- Experiments of statistical tests for piecewise stationary bandits.ipynb Binder
- Demonstrations of Single-Player Simulations for Non-Stationary-Bandits.ipynb Binder

### 2.c. Play with the *notebooks* dynamically (on Google Colab) 💥¶

Anyone can use the colab.research.google.com/notebook website (by clicking on the icon above) to run the notebook in her/his web-browser. You can then play with it as long as you like, for instance by modifying the values or experimenting with the code.

- Do_we_even_need_UCB.ipynb Google Colab
- Easily_creating_MAB_problems.ipynb Google Colab
- Example_of_a_small_Single-Player_Simulation.ipynb Google Colab
- Example_of_a_small_Multi-Player_Simulation__with_Centralized_Algorithms.ipynb Google Colab
- Example_of_a_small_Multi-Player_Simulation__with_rhoRand_and_Selfish_Algorithms.ipynb Google Colab
- Lai_Robbins_Lower_Bound_for_Doubling_Trick_with_Restarting_Algorithms.ipynb Google Colab
- Exploring different doubling tricks for different kinds of regret bounds.ipynb Google Colab
- Experiments of statistical tests for piecewise stationary bandits.ipynb Google Colab
- Demonstrations of Single-Player Simulations for Non-Stationary-Bandits.ipynb Google Colab

## 3. Question: *Requirements to run the notebooks locally*?¶

All the requirements can be installed with `pip`

.

### 3.a. Jupyter Notebook and IPython¶

```
sudo pip install jupyter ipython
```

It will also install all the dependencies, afterward you should have a `jupyter-notebook`

command (or a `jupyter`

command, to be ran as `jupyter notebook`

) available in your `PATH`

:

```
$ whereis jupyter-notebook
jupyter-notebook: /usr/local/bin/jupyter-notebook
$ jupyter-notebook --version # version >= 4 is recommended
4.4.1
```

### 3.b. My numerical environment, `SMPyBandits`

¶

- First, install its dependencies (
`pip install -r requirements`

). - Then, either install it (
*not yet*), or be sure to work in the main folder.

Note:it’s probably better to usevirtualenv, if you like it. I never really understood how and why virtualenv are useful, but if you know why, you should know how to use it.

### 💁 More information?¶

- More information about notebooks (on the documentation of IPython) or on the FAQ on Jupyter’s website.
- More information about mybinder.org: on this example repository.

## 📜 License ? GitHub license¶

All the notebooks in this folder are published under the terms of the MIT License (file LICENSE.txt). © Lilian Besson, 2016-18.

Maintenance Ask Me Anything ! Analytics made-with-jupyter made-with-python ForTheBadge uses-badges ForTheBadge uses-git