ELEN0062 - Introduction to machine learning (iML)

Informations

• Course study program
• Theoritical course's webpage
• The reference book's webpage (free download)

Schedule

Assignments

First assignment

Installing Anaconda

There are many ways to install Python on a computer and get all the libraries needed. One quick way is to install anaconda, which comes with all the libraries we will need.

1. Get the anaconda installer for your operating system. Make sure you install a Python 3.5+ version.
2. Open a Python console:
• From a unix command line: python
• Or open spyder IDE, which comes with anaconda
Note that you can use the ipython interpreter, which is much easier to work with.
3. Run the following commands:

     
   import numpy as np
   import pandas as pd
   import sklearn
   import scipy
   
   print(np.__version__)
   print(pd.__version__)
   print(sklearn.__version__)
   print(scipy.__version__)
     
   

   If there is no error, the installation went fine

Third assignment: the challenge

The third project is organized in the form a challenge, where you will compete against each other. This year, the challenge is about movie rating: you must predict the rating that a given user would give to a given movie. All the relevant information can be found on the Kaggle plateform which will hold the challenge.

The project is divided into four parts. All the deadlines can be found in the schedule section above.

1. Setup for the project
   • Create an account on the Kaggle platform. Use your real name so that we can identify you.
   • Use the link given in the mail to enter the challenge. If you did not receive the email, contact us.
   • Form groups of three and register them on the submission platform.
   • Test the toy example.
2. Propose the best model you can before the competition deadline.
3. Submit an archive on the submission platform in tar.gz format, containing a report that describes the different steps of your approach and your main results along with your source code. Use the same ids as for the Kaggle platform. The report must contain the following information:
   • A detailed description of all the approaches that you have used to win the challenge, including the feature engineering you performed.
   • A detailed description of your hyper-parameters optimization approach and your model validation technique.
   • A table summarizing the performance of your differents approaches containing for each approach at least the name of the approach, the validation score, the score on the public and the private leaderboard.
   • Any complementary information or figures that you want to mention.
4. Present succinctly your approach to the rest of the class. (More information coming soon)

Have fun!

Cheat sheet for ML in Python

• Python
• NumPy
• Matplotlib
• Pandas
• Scipy

Check out datacamp for more.

Supplementary material

Here is a very scarce list of supplementary material related to the field of machine learning. I tend to update this section when I come across interesting stuff but if you feel like you need more material on some topic, do not hesitate to ask!

Machine learning in general

• Andrew Ng's online course (Standford): The most popular online course on ML. Archived from coursera.
• Pedro Domingos' online course (Washington).
• Reza Shadmehr (Baltimore) and his slides.
• Jeffrey Ullman's course on mining massive datasets (Standford) based on his reference book. Not everything is related to the course though.

Linear regression

The geometry of Least Squares (1 variable)

Artifical neural networks

• Graham Taylor: An Overview of Deep Learning and Its Challenges for Technical Computing (2014)
• Geoffrey Hinton: Introduction to Deep Learning and Deep Belief Nets (2012)
• Geoffrey Hinton: The Next Generation of Neural Networks (2007)
• Leon Bottou: Multilayer Networks series
• A simplified version of Backprop illustrated.
• An illustrated taxonomy of learning networks.

Learning theory (Bias/Variance...)

• The curse of dimensionality

Support Vector Machines

• Visualizing the kernel trick
• A couple of videos about constraint optimization (by Khan Academy):
  • Constrained optimization introduction
  • Lagrange Multipliers (part 1)
  • Lagrange Multipliers (part 2)

Unsupervised learning

• Implementing a PCA by hand

Misc.

• Sentex: A bit of everything
• Derek Kane: A bit of everything
• Welch Labs: A few videos about Neural Nets
• Two minutes papers: Many articles relate to (applications of) ML
• Siraj Raval (this guy is crazy)
• Introductory online course on ML (covers linear/logistic regression, decision trees/random forests, basics on neural networks and a clustering).

Pre-requisites

Machine learning requires a solid background in maths, especially in linear algebra, (advanced) probability theory and (multivariable) calculus. There are even more resources on those than on deep learning. Here is a short selection, which emphasizes intuition.

Linear algebra

• 3 brown 1 blue serie on linear algebra
• If you prefer paper (or PDF): Practical Linear Algebra: A Geometry Toolbox 2nd Edition by Farin, Gerald, Hansford, Dianne. A K Peters/CRC Press (2004)

Calculus

• 3 brown 1 blue serie on calculus. Saddly, it does not go on to multivariable calculus.
• Khan academy serie
• If you prefer paper (or PDF): Calculus: Concepts and contexts 4th Edition by Stewart, James. (Also available in french)