====== AA2 - Midterm Reading List 2014-15 ======

In the following, it is a list of the topics and articles for the midterm assignment.
You can express your preference for one of the topics: the decision on topic assignment is ultimately made by the course instructor.

Prepare a 10-minutes presentation answering the associated questions (maximum number of slides should be 5/6).

==== 1. Self-Organizing Map for sequences ====

**Reading Material:** T. Voegtlin "Recursive self-organizing maps." Neural Networks 15.8 (2002): 979-991. {{magistraleinformatica:aa2:rsom02.pdf| pdf}}

**Questions:**  Describe the recursive encoding of sequences in the RSOM.  Report and discuss the network error and the update equations for the network weights. Provide a comparison between RSOM, temporal SOM and recurrent SOM (also showing the differences in the respective activation functions).

==== 2. Echo State Networks for indoor localization ====

**Reading Material:** D. Bacciu, P. Barsocchi, S. Chessa, C. Gallicchio, A. Micheli, An experimental characterization of reservoir computing in ambient assisted living applications, Neural Computing and Applications, vol. 24 (6), pag. 1451–1464, 2014 {{magistraleinformatica:aa2:localization.pdf| pdf}}

**Questions:** Describe the application and the experimental scenario: highlight the differences between the homogenous and heterogeneous settings.  Describe the leaky integrator echo state network: discuss changes (also with equations) with respect to the standard ESN.  Why is the leaky integrator needed? 

==== 3.	Minimum complexity Echo State Networks ====

**Reading Material:** Rodan, P. Tino, Minimum complexity echo state network, IEEE Transactions on Neural Networks, vol. 22(1), pag. 131-144, 2011 {{magistraleinformatica:aa2:minCompESN.pdf| pdf}}

**Questions:** Describe the DLR, DLRB and SCR topologies of an ESN. Sketch the demonstration of the memory capacity MC for an SCR (theorem 1).  Summarize the experimental results:  what minimal topology/parameterization has performance levels comparable to standard ESNs?

==== 4.	Long-Short term memory networks ====

**Reading Material:** Hochreiter, Sepp, and Jürgen Schmidhuber. "Long short-term memory." Neural computation 9.8 (1997): 1735-1780. {{magistraleinformatica:aa2:lstm.pdf| pdf}}

**Questions:**  Explain the vanishing gradient problem. Describe the LSTM architecture and main equations. What is the role of the gate units?

==== 5.	Structure finding in Bayesian Networks ====

**Reading  Material:** D.Bacciu, T.A. Etchells, P.J.G. Lisboa and J. Whittaker, "Efficient identification of independence networks using mutual information", Computational Statistics, Springer, vol 28, no. 2, pp 621-646, Apr. 2013 {{magistraleinformatica:aa2:pcAlgo.pdf| pdf}}

**Questions:** Summarize the standard PC algorithm: describe the test of conditional independence and how it is computed with Mutual Information. Explain what is a False Negative in this scenario and describe the idea of power correction for reducing false negatives.  Describe the concept of strong and weak edges and how/why this is used for the test-the-weakest-first policy.

==== 6.	Image-Denoising with the Ising model ====

**Reading Material:** Section 8.3.3 from Bishop chapter ([[http://research.microsoft.com/en-us/um/people/cmbishop/prml/pdf/Bishop-PRML-sample.pdf|pdf]]). 

**Questions:** Describe the problem and the associated Markov random field. Provide the energy function equations and discuss their interpretation for the particular application.

==== 7.	Bi-Directional Hidden Markov Models ====

**Reading Material:** Baldi, P., Brunak, S., Frasconi, P., Pollastri, G., & Soda, G. (2001). Bidirectional dynamics for protein secondary structure prediction. In Sequence Learning (pp. 80-104). Springer Berlin Heidelberg. {{magistraleinformatica:aa2:bidir-hmm.pdf| pdf}}

**Questions:** Describe the bi-diretional IO-HMM and discuss the equation for its joint distribution factorization: identify the model parameters and what are the stationariety assumptions. Summarize how the transition functions can be implemented using MLP neural networks.

==== 8.	Max-product Algorithm ====

**Reading Material:** Section 5.2.1 of David Barber’s Book [BRML].

**Questions:** Describe what is the typical max-product inference problem: why is different from sum-product? Describe the variable elimination idea in max-product. Describe the max-product message passing using factor graphs.

==== 9.	Markov Chains and Pagerank ====

**Reading Material:** Jia Li, Markov Chain Interpretation of Google Page Rank, Tech Report ({{magistraleinformatica:aa2:pagerankMC.pdf| pdf}}). Integrate with David Barber’s Book [BRML], pages from 461 to 463.

**Questions:** Describe the Pagerank algorithm from a Markov Chain point of view. Define the concepts of stationary and equilibrium distribution and discuss their interpretation in terms of Pagerank.