====== Applied Brain Science - Computational Neuroscience (CNS) ====== **Master of Science in Bionics Engineering** Instructors: **[[http://www.di.unipi.it/~micheli|Alessio Micheli]]** ([[mailto:micheli@di.unipi.it|email]]) - **[[http://pages.di.unipi.it/bacciu/|Davide Bacciu]]** ([[mailto:bacciu@di.unipi.it|email]]) Additional web page: http://www.di.unipi.it/~micheli/DID/CNS.htm ---- ===== News ===== **(07/04/2017)** The lecture missed due to Easter holidays will be recovered on Thursday 20/04/2017, Room B24, from 10.30 to 13.30. **(21/02/2017)** Course Didawiki updated with course information and first lesson for academic year 2016/17. ===== Course Information ===== **Note for Computer Science Students** In academic year 2016/2017, "Machine Learning: neural networks and advanced models" (AA2) (Master programme in Computer Science - Corso di Laurea Magistrale in Informatica) is borrowed from CNS. **Weekly Schedule** The course is held on the second term. The preliminary schedule for **A.A. 2016/17** is provided in table below. ^ Day ^ Time ^ Room ^ | Monday | 11.30-13.30 | SI3 (Polo B Ingegneria) | | Wednsday| 15.30-18.30 | SI3 (Polo B Ingegneria) | **First lecture**: Wednsday 01/03/2017 **Objectives** The content of the Computational Neuroscience course includes: * bio-inspired neural modelling, spiking and reservoir computing neural networks; * advanced computational neural models for learning; * architectures and learning methods for dynamical/recurrent neural networks for temporal data and the analysis of their properties; * the role of computational neuroscience in real-world applications (by case studies). **Textbook and Teaching Materials** The official textbooks of the course are the following: [IZHI] E.M. Izhikevich Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting The MIT press, 2007 [DAYAN] P. Dayan and L.F. Abbott Theoretical Neuroscience The MIT press, 2001 [NN] Simon O. Haykin Neural Networks and Learning Machines (3rd Edition), Prentice Hall, 2009 Additionally: for the part of the course on bio-inspired neural modelling, it is also useful the book [[http://lcn.epfl.ch/~gerstner/SPNM/SPNM.html|freely available online]]: [GERSTNER] W. Gerstner and W.M. Kistler, Spiking Neuron Models: Single Neurons, Population, Plasticity. Cambridge University Press, 2002 for the second module of the course (Unsupervised and Representation Learning), it will be referenced material from a book [[http://neuronaldynamics.epfl.ch/online/index.html|freely available online]]: [PANINSKI] W. Gerstner, W.M. Kistler, R. Naud and L. Paninski Neuronal Dynamics: From single neurons to networks and models of cognition Cambridge University Press, 2014 ===== Lectures ===== ^ ^ Date ^ Topic ^ References & Additional Material ^ | 1 | 01/03/17 (15.30-18.30) | Introduction to the course | {{:bionics-engineering:computational-neuroscience:CNS17-LEZ1.pdf|Lecture 1}} | | 2 | 06/03/17 (11.30 - 13.30)| Introduction to Neural Modeling | {{:bionics-engineering:computational-neuroscience:neuralmodeling_1b.pdf | Lecture 2}} | | 3 | 08/03/17 (15.30 - 18.30)| Conductance-based and Spiking Neuron Models | {{:bionics-engineering:computational-neuroscience:neuralmodeling_2b.pdf | Lecture 3}} | | 4 | 13/03/17 (11.30 - 13.30)| Neural and Neuron-Astrocyte Modeling | {{:bionics-engineering:computational-neuroscience:Neuro-Astro-13317_2.pdf | Lecture 4}} | | 5 | 15/03/17 (15.30 - 18.30)| Implementing Spiking Neurons using Izhikevich's Model| {{:bionics-engineering:computational-neuroscience:LaboratoryAssignment-spikinglab1-1_b.pdf | Lab1-1-assignment}} | | 6 | 20/03/17 (11.30 - 13.30)| Statistics for In-vitro neuro-astrocyte culture| {{:bionics-engineering:computational-neuroscience:Lecture_W_20317.pdf | Lecture 6 - seminar}} | | 7 | 22/03/17 (15.30 - 18.30) | Introduction to Liquid State Machines| {{:bionics-engineering:computational-neuroscience:NeuralModeling_3.pdf | Lecture 7}} | | 8 | 27/03/17 (11.30 - 13.30)| Spikinglab2 - Liquid State Machines| {{:bionics-engineering:computational-neuroscience:LaboratoryAssignment-spikinglab2.pdf | Lab1-2-assignment}} | | 9 | 29/03/17 (15.30-18.30) | Representation Learning - Synaptic Plasticity and Hebbian Learning | {{bionics-engineering:computational-neuroscience:1-unsuplearn-hand-2017.pdf|Lecture 8}}\\ //References//:\\ [DAYAN] Sect. 8.1-8.3\\ [PANINSKI] Sect 19.1, 19.2.1, 19.3.1, 19.3.2 | | 10 | 03/04/17 (11.30-13.30) | Associative Memories I - Hopfield Networks | {{bionics-engineering:computational-neuroscience:2-hopfield-hand-2017.pdf|Lecture 9}}\\ //References//:\\ [DAYAN] Sect. 7.4 (Associative Memory part)\\ [PANINSKI] Sect. 17.1, 17.2 | | 11 | 05/04/17 (15.30-18.30) | Lab 2.1 - Hebbian learning and Hopfield networks | [[lab2.1|Assignment 2.1]] | | 12 | 10/04/17 (11.30-13.30) | Associative Memories II - Stochastic networks and Boltzmann machines | {{bionics-engineering:computational-neuroscience:3-boltz-hand-2017.pdf|Lecture 10}}\\ //References//:\\ [DAYAN] Sect. 7.6\\ \\ //Further readings//:\\ [[bionics-engineering:computational-neuroscience:start#further_readings|[1]]] A clean and clear introduction to RBM | | 13 | 12/04/17 (15.30-18.30) | Lab 2.1b - Hebbian learning and Hopfield networks (continued) | | | | 17/04/17 (11.30-13.30) | No class due to [[https://en.wikipedia.org/wiki/Easter_Monday|Italian national holiday]] | | | 14 | 20/04/17 (10.30-13.30) | Lecture 11\\ Part 1: Adaptive Resonance Theory\\ Part 2: Representation learning and deep NN \\ **Recovery Lesson: will be held in room B24** | {{bionics-engineering:computational-neuroscience:4-art-hand-2017.pdf|Lecture 11 - Part 1}}\\ {{bionics-engineering:computational-neuroscience:5-deep-hand-2017.pdf|Lecture 11 - Part 2}}\\ //References//:\\ [DAYAN] Sect. 10.1\\ \\ //Futher Readings//:\\ A gentle introduction to ART networks (with coding examples) can be found [[http://cannes.itam.mx/Alfredo/English/Publications/Nslbook/MitPress/157_170.CH08.pdf|here]]\\ [[bionics-engineering:computational-neuroscience:start#further_readings|[2]]] A classic divulgative paper from the initiator of Deep Learning \\ [[bionics-engineering:computational-neuroscience:start#further_readings|[3]]] Recent review paper \\ [[bionics-engineering:computational-neuroscience:start#further_readings|[4]]] A freely available book on deep learning from Microsoft RC| | 15 | 03/05/17 (15.30-18.30) | Module conclusions and Lab 2.2 | [[lab2.2|Assignment 2.2]] \\ {{bionics-engineering:computational-neuroscience:topicsmodule2.pdf|List of presentation and project topics for the second module}} | | 16 | 08/05/17 (11.30-13.30) | Introduction to RNN: tasks and basic models | [[http://www.di.unipi.it/~micheli/DID/CNS/CNS-2017/part3/| Lecture and info multifiles]] | | 17 | 10/05/17 (15.30-18.30) | Introduction to RNN: properties and taxonomy; intro to learning by BPTT | [[http://www.di.unipi.it/~micheli/DID/CNS/CNS-2017/part3/| Lecture and info multifiles (also RNN learning)]] | | 18 | 15/05/17 (11.30-13.30) | Introduction to RNN: learning by RTRL| [[http://www.di.unipi.it/~micheli/DID/CNS/CNS-2017/part3/| Lecture and info multifiles (RNN learning)]] plus blackboard notes| | 19 | 17/05/17 (15.30-18.30) | Introduction to RNN: LAB3-1 - learning with IDNN and RNN| [[http://www.di.unipi.it/~micheli/DID/CNS/CNS-2017/part3/| Info and assignment multifiles (see "RNN - LAB3-1" section). New version 1.1]] | | 20 | 22/05/17 (11.30-13.30) | Introduction to RNN: Reservoir Computing | [[http://www.di.unipi.it/~micheli/DID/CNS/CNS-2017/part3/| Lecture and info multifiles (ESN)]] | | 21 | 24/05/17 (15.30-18.30) | Introduction to RNN: LAB 3-2 - learning with ESN | [[http://www.di.unipi.it/~micheli/DID/CNS/CNS-2017/part3/| Info and assignment multifiles (see "RNN - Lab2" section). ]] | | 22 | 29/05/17 (11.30-13.30) | Introduction to RNN: LABs 3-1 and 3-2 continue | [[http://www.di.unipi.it/~micheli/DID/CNS/CNS-2017/part3/| Info and assignment multifiles ]] | ===== Past Editions ===== [[cns2016|Academic Year 2015/2016]] ===== Further Readings ===== [[http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf|[1]]] Geoffrey Hinton, A Practical Guide to Training Restricted Boltzmann Machines, Technical Report 2010-003, Department of Computer Science, University of Toronto, 2010 [[http://www.cs.toronto.edu/~hinton/science.pdf|[2]]] G.E. Hinton, R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks.Science 313.5786 (2006): 504-507. [[http://arxiv.org/pdf/1206.5538.pdf|3]]] Y. Bengio, A. Courville, and P. Vincent. Representation learning: A review and new perspectives. Pattern Analysis and Machine Intelligence, IEEE Transactions on, Vol. 35(8) (2013): 1798-1828. [[http://research.microsoft.com/apps/pubs/default.aspx?id=209355|[4]]] L. Deng and D. Yu. Deep Learning Methods and Applications, 2014 [[http://www.igi.tugraz.at/maass/psfiles/189.pdf|[5]]] W. Maass, Liquid state machines: motivation, theory, and applications. Computability in context: computation and logic in the real world (2010): 275-296. See other references in the slide notes.