# Jupyter Notebooks 📓 by Naereen @ GitHub¶

This folder hosts some Jupyter Notebooks, to present in a nice format some numerical experiments for my SMPyBandits project.

The wonderful Jupyter tools is awesome to write interactive and nicely presented 🐍 Python simulations!

## 1. List of experiments presented with notebooks¶

### MAB problems¶

Easily creating various Multi-Armed Bandit problems, explains the interface of the

`Environment.MAB`

module.

### Single-Player simulations¶

A simple example of Single-Player simulation, comparing

`UCB1`

(for two values of $\alpha$, 1 and 1/2),`Thompson Sampling`

,`BayesUCB`

and`kl-UCB`

.Do we even need UCB? demonstrates the need for an algorithm smarter than the naive

`EmpiricalMeans`

.Lai-Robbins lower-bound for doubling-tricks algorithms with full restart.

### Active research on Single-Player MAB¶

Exploring different doubling tricks for different kinds of regret bounds.

Experiments of statistical tests for piecewise stationary bandits

Demonstrations of Single-Player Simulations for Non-Stationary Bandits.

### Multi-Player simulations¶

A simple example of Multi-Player simulation with 4 Centralized Algorithms, comparing

`CentralizedMultiplePlay`

and`CentralizedIMP`

with`UCB`

and`Thompson Sampling`

.A simple example of Multi-Player simulation with 2 Decentralized Algorithms, comparing

`rhoRand`

and`Selfish`

(for the “collision avoidance” part) combined with`UCB`

and`Thompson Sampling`

for learning the arms. Spoiler:`Selfish`

beats`rhoRand`

!

## (Old) Experiments¶

Can we use a (non-online) Unsupervised Learning algorithm for (online) Bandit problem ?

Can we use a computationally expensive Black-Box Bayesian optimization algorithm for (online) Bandit problem ?

## 2. Question: *How to read these documents*?¶

### 2.a. View the *notebooks* statically 📝¶

Either directly in GitHub: see the list of notebooks;

### 2.b. Play with the *notebooks* dynamically (on MyBinder) 💥¶

Anyone can use the mybinder.org website (by clicking on the icon above) to run the notebook in her/his web-browser. You can then play with it as long as you like, for instance by modifying the values or experimenting with the code.

Example_of_a_small_Multi-Player_Simulation__with_Centralized_Algorithms.ipynb

Example_of_a_small_Multi-Player_Simulation__with_rhoRand_and_Selfish_Algorithms.ipynb

Lai_Robbins_Lower_Bound_for_Doubling_Trick_with_Restarting_Algorithms.ipynb

Exploring different doubling tricks for different kinds of regret bounds.ipynb

Experiments of statistical tests for piecewise stationary bandits.ipynb

Demonstrations of Single-Player Simulations for Non-Stationary-Bandits.ipynb

### 2.c. Play with the *notebooks* dynamically (on Google Colab) 💥¶

Anyone can use the colab.research.google.com/notebook website (by clicking on the icon above) to run the notebook in her/his web-browser. You can then play with it as long as you like, for instance by modifying the values or experimenting with the code.

Example_of_a_small_Multi-Player_Simulation__with_Centralized_Algorithms.ipynb

Example_of_a_small_Multi-Player_Simulation__with_rhoRand_and_Selfish_Algorithms.ipynb

Lai_Robbins_Lower_Bound_for_Doubling_Trick_with_Restarting_Algorithms.ipynb

Exploring different doubling tricks for different kinds of regret bounds.ipynb

Experiments of statistical tests for piecewise stationary bandits.ipynb

Demonstrations of Single-Player Simulations for Non-Stationary-Bandits.ipynb

## 3. Question: *Requirements to run the notebooks locally*?¶

All the requirements can be installed with `pip`

.

### 3.a. Jupyter Notebook and IPython¶

```
sudo pip install jupyter ipython
```

It will also install all the dependencies, afterward you should have a `jupyter-notebook`

command (or a `jupyter`

command, to be ran as `jupyter notebook`

) available in your `PATH`

:

```
$ whereis jupyter-notebook
jupyter-notebook: /usr/local/bin/jupyter-notebook
$ jupyter-notebook --version # version >= 4 is recommended
4.4.1
```

### 3.b. My numerical environment, `SMPyBandits`

¶

First, install its dependencies (

`pip install -r requirements`

).Then, either install it (

*not yet*), or be sure to work in the main folder.

Note:it’s probably better to usevirtualenv, if you like it. I never really understood how and why virtualenv are useful, but if you know why, you should know how to use it.

### 💁 More information?¶

More information about notebooks (on the documentation of IPython) or on the FAQ on Jupyter’s website.

More information about mybinder.org: on this example repository.

## 📜 License ? ¶

All the notebooks in this folder are published under the terms of the MIT License (file LICENSE.txt). © Lilian Besson, 2016-18.