haku: @supervisor Raiko, Tapani / yhteensä: 4
viite: 1 / 4
« edellinen | seuraava »
Tekijä:Van den Broeke, Gerben
Työn nimi:What auto-encoders could learn from brains - Generation as feedback in deep unsupervised learning and inference
Julkaisutyyppi:Diplomityö
Julkaisuvuosi:2016
Sivut:89      Kieli:   eng
Koulu/Laitos/Osasto:Perustieteiden korkeakoulu
Oppiaine:Machine learning and data mining   (SCI3044)
Valvoja:Raiko, Tapani
Ohjaaja:Rasmus, Antti
Elektroninen julkaisu: http://urn.fi/URN:NBN:fi:aalto-201602161349
Sijainti:P1 Ark Aalto  3502   | Arkisto
Avainsanat:unsupervised learning
deep learning
neural networks
generative models
Tiivistelmä (eng):This thesis explores fundamental improvements in unsupervised deep learning algorithms.
Taking a theoretical perspective on the purpose of unsupervised learning, and choosing learnt approximate inference in a jointly learnt directed generative model as the approach, the main question is how existing implementations of this approach, in particular auto-encoders, could be improved by simultaneously rethinking the way they learn and the way they perform inference.

In such network architectures, the availability of two opposing pathways, one for inference and one for generation, allows to exploit the symmetry between them and to let either provide feedback signals to the other.
The signals can be used to determine helpful updates for the connection weights from only locally available information, removing the need for the conventional back-propagation path and mitigating the issues associated with it.
Moreover, feedback loops can be added to the usual usual feed-forward network to improve inference itself.
The reciprocal connectivity between regions in the brain's neocortex provides inspiration for how the iterative revision and verification of proposed interpretations could result in a fair approximation to optimal Bayesian inference.

While extracting and combining underlying ideas from research in deep learning and cortical functioning, this thesis walks through the concepts of generative models, approximate inference, local learning rules, target propagation, recirculation, lateral and biased competition, predictive coding, iterative and amortised inference, and other related topics, in an attempt to build up a complex of insights that could provide direction to future research in unsupervised deep learning methods.
ED:2016-02-21
INSSI tietueen numero: 53141
+ lisää koriin
« edellinen | seuraava »
INSSI