Jupyter Notebooks 📓 by Naereen @ GitHub

This folder hosts some Jupyter Notebooks, to present in a nice format some numerical experiments for my SMPyBandits project.

The wonderful Jupyter tools is awesome to write interactive and nicely presented 🐍 Python simulations!

https://img.shields.io/badge/Made%20with-Jupyter-1f425f.svgmade-with-jupyter https://img.shields.io/badge/Made%20with-Python-1f425f.svgmade-with-python


1. List of experiments presented with notebooks

MAB problems

  • Easily creating various Multi-Armed Bandit problems, explains the interface of the Environment.MAB module.

Single-Player simulations

  • A simple example of Single-Player simulation, comparing UCB1 (for two values of $\alpha$, 1 and 1/2), Thompson Sampling, BayesUCB and kl-UCB.
  • Do we even need UCB? demonstrates the need for an algorithm smarter than the naive EmpiricalMeans.
  • Lai-Robbins lower-bound for doubling-tricks algorithms with full restart.

Active research on Single-Player MAB

  • Exploring different doubling tricks for different kinds of regret bounds.
  • Experiments of statistical tests for piecewise stationary bandits
  • Demonstrations of Single-Player Simulations for Non-Stationary Bandits.

Multi-Player simulations

(Old) Experiments

  • Can we use a (non-online) Unsupervised Learning algorithm for (online) Bandit problem ?
  • Can we use a computationally expensive Black-Box Bayesian optimization algorithm for (online) Bandit problem ?

2. Question: How to read these documents?

2.a. View the notebooks statically 📝


3. Question: Requirements to run the notebooks locally?

All the requirements can be installed with pip.

Note: if you use Python 3 instead of Python 2, you might have to replace pip and python by pip3 and python3 in the next commands (if both pip and pip3 are installed).

3.a. Jupyter Notebook and IPython

sudo pip install jupyter ipython

It will also install all the dependencies, afterward you should have a jupyter-notebook command (or a jupyter command, to be ran as jupyter notebook) available in your PATH:

$ whereis jupyter-notebook
jupyter-notebook: /usr/local/bin/jupyter-notebook
$ jupyter-notebook --version  # version >= 4 is recommended
4.4.1

3.b. My numerical environment, SMPyBandits

  • First, install its dependencies (pip install -r requirements).
  • Then, either install it (not yet), or be sure to work in the main folder.
Note: it’s probably better to use virtualenv, if you like it. I never really understood how and why virtualenv are useful, but if you know why, you should know how to use it.

💁 More information?