Lecturer: Prof. Wang Jian-Sheng
Schedule/Venue: Tuesday LT51/Friday LT50, 2:00-4:00, at UTown.
Final exam: 27 Nov 2026, 9:00 am.
Reference books: "Deep Learning, Foundations and Concepts", C. M. Bishop; "Deep Learning", Goodfellow, Bengio, and Courville; "Machine Learning", Lindholm, et al; "Dive into Deep Learning", at d2l.ai by A. Zhang, et al; "Neural Networks", Haykin.
Official Syllabus: This course presents the mathematical and computational foundations of machine learning with an emphasis on deep learning networks, preparing students with sufficient background for more advanced topics such as AI in physics or any of the other sciences. The learning outcomes include sufficient familiarity with the Python programming environment for machine learning, a deeper understanding of the building blocks of neural networks, and numerical training algorithms for machine learning. The course will draw applications in science as examples to illustrate the concepts of deep learning.
Course Outline:
Week 1: 11, 14 Aug, biological neurons and artificial neural networks
Week 2: 18, 21 Aug, python, numpy, pytorch
Week 3: 25, 28 Aug, linear regression, maximum likelihood, cross-validation, homework 1 due
Week 4: 1, 4 Sep, feedforward network, universal approximation, NLL, stochastic gradient descent, and other optimization algorithms, tutorial 1
Week 5: 8, 11 Sep, backprogation
Week 6: 15, 18 Sep, regularization, homework 2 due
Recess week, no classes
Week 7: 29, 2 Oct, physics informed neural network (PINN) (midterm test this week on Friday, 2 Oct)
Week 8: 6 Oct, convolutional network (CNN) (9 Oct is Well-Being day)
Week 9: 13, 16 Oct, recurrent neural network (RNN), Long Short-Term memory, homework 3 due
Week 10: 20, 23 Oct, encoder-decoder, attention, transformer
Week 11: 27, 30 Oct, generative model, diffusion model
Week 12: 3, 6 Nov, unsupervised learning, Boltzmann machine, homework 4 due (no class on Friday)
Week 13: 10, 13 Oct, revision
Homework problem sets are on Canvas in Files. Upload homework as PDF on Canvas under Assignments.
Example jupyter notebook codes: autograd, least squares, least squares (using torch.nn and SGD), ...