Author: multigridAdmin
Summer Course in Peking University
Summer Course in Peking University
Course Description
This is an advanced undergraduate course on the introduction of basic mathematical and practical aspects of deep learning techniques. The course will cover some basic deep learning models such as convolutional neural networks, training algorithms such as stochastic gradient descent methods, popular data bases such as MNIST and CIFAR and specific applications such as image classifications. Examples of other successful applications of deep learning will also be briefly discussed for various tasks of machine learning and artificial intelligence such as computer vision, natural language processing and reinforcement learning. More specifically, the following topics will be covered:
- Basics of machine learning, logistic regression and SVM
- Deep neural networks and mathematical properties
- Multigrid methods and convolutional neural networks
- Stochastic gradient descent methods and convergence analysis
- PyTorch and deep learning for image classification
- Other applications
The course will be held at Peking University during July 6-27, 2019. Lectures and lab hours will be given 8am-9:30am and 2pm-4pm and Q&A sessions will be 10:00am-11:30am every weekday. Students are expected to practice and do assignments during the afternoon sessions. Typed lecture notes will be distributed in class. Both math and computing homework problems will be assigned. We will have four homeworks in the first two weeks, a mid-term and a final project.
Course Lecturer
- Lecturer:
Prof. Jinchao Xu (Penn State) - Guest lecturers:
Prof. Zhanxin Zhu (Peking University) on July 16-17 and
Prof. Bin Dong (Peking University) on July 23-24
Teaching Assistants
- Jonathan Siegel (Penn State)
- Juncai He (Penn State)
- Jianhong Chen (Penn State)
- Xiaodong Jia (Peking University)
- Yuyan Chen (Peking University)
- Huang Huang (Peking University)
Course Time and Location
- Three weeks for 3 credits: July 6-27, 2019
- Time: 8:00-9:30 am and 2:00-3:30 pm
- Place: Room 408 in the Third Teaching Building(三教408), Peking University, Beijing China
Grading Policy
- Assignment #1: 10%
- Assignment #2: 10%
- Assignment #3: 10%
- Assignment #4: 10%
- Midterm: 20%
- Course Project: 40%
Course Related
- Course: Math 497, An Introduction to Deep Learning
- Coordinator: Shaobo Liang and Li Jiang (Peking University)
- Staff Assistant: Shenglan Zhao (slzhao@pku.edu.cn)
- For a detailed timetable of the course, see Timetable
Notes and homeworks
- Introductory slides on July 8. 497_Introduction
- Notes. 2019PSU-PKU497
- Slides for Prof. Zhanxin Zhu‘s talk. CNN
- Homework 1. HW1
- Homework 2. HW2
- Homework 3. HW3
- Homework 4. HW4
- Mid-term. mid
- Final project. Projects
Students gallery
Class Photos in Peking University
Group Activities
Students impressions
- Student 1: We become enthusiastic because all the professors were excited to teach. I really appreciate that they gave us this precious opportunity to taking class in Peking University. Through these three weeks, I did learn a lot. I have more mathematical understanding about the CNN structure and my programming skills are getting better and better.
- Student 2: Our professor is very helpful, and cares for every student. He always asked us if there are any questions or if he talked too fast. With the limit of knowledge, we always feel a little ashamed to ask such a simple question on class, but he is so nice and encouraged us a lot, also talked with us after class, which give me more motivation to learn.
- Student 3: This class is a great opportunity for me to appreciate math and see real examples of how useful it can be.
- Student 4: I had a great time learning the math behind it, also I picked up the coding in just three weeks, thanks to all the graduate and postdoc’s(TA) help.
Protected: 04MSC-Note
Seminar Calendar of Deep Learning
Deep Learning Seminars of 2019-2020
- Time: EST 8: 00 am(BJS 20: 00 pm) of Tuesday
- Period: Each Week
- Topic: Alternative and flexible
- You can get the material of our seminar, just click the underlined hyperlink.
Initial Schedule
Date | Speaker | Topic | Material or Summary |
---|---|---|---|
Jul 31 | Huang Huang | Initialization in DNN | Related Paper |
YuYan Chen | Implement of MgNet | ||
Aug 7 | Jonathan Siegel | Sparse Training Algorithms for Neural Networks | |
Qian Zhang | SGD: General Analysis and Improved Rates | Talk Summary From Qian | |
Sep 3 | Pengfei Yin | Deep Residual Networks 1, Deep Residual Networks 2 | Talk Material From Pengfei |
Jianhong Chen | Learning Structured Sparsity in Deep Neural Networks | Talk Summary From Jianhong | |
Sep 10 | Jianqing Zhu | Lookahead Optimizer: k steps forward, 1 step back | Talk Material From Jianqing |
Lin Li | Deep Neural Networks with Rectified Power Units | Talk Material From Lin | |
Sep 17 | Juncai He | Adaptive Learning Rate Schedule in SGD | |
Sep 24 | Juncai He, Jonathan Siegel | Using Statistics to Automate SGD | |
Oct 1 | Jonathan Siegel | Stop Criteria on SGD | |
Oct 8 | Juncai He | Hypothesis Testing for Stopping Criterion Under Markov Chain | |
Oct 15 | Juncai He | Adaptive Learning Rate of SGD From Markov Chain | |
Oct 22 | Qian Zhang | A Unified Theory of SGD: Variance Reduction, Sampling, Quantization and Coordinate Descent | Talk Material From Qian |
* The row of above table colored aqua is in preparation.
Seminars of PSU-PKU Summer School
Schedule of FIRST Week
Date | Speaker | Recorder | Topic |
---|---|---|---|
Jul 8 | Liang Zhao | Yuyan Chen | Implicit regularization property of SGD method |
Jul 10 | Juncai He | Jianhong Chen | Convergence of backpropagation with momentum for network architectures with skip connections |
Jul 12 | Lin Li | Lian Zhang | Approximation properties of ReLU DNN |
Schedule of SECOND Week
Date | Speaker | Recorder | Topic |
---|---|---|---|
Jul 15 | Shaobo Liang | Qian Zhang | k-means |
Jul 17 | Li Jiang | Pengfei Yin | A Unified View of Multi-class SVM |
Jul 19 | Lian Zhang | Juncai He | Model Compression of CNN |
Schedule of THIRD Week
Date | Speaker | Recorder | Topic |
---|---|---|---|
Jul 22 | Zhe Zheng | Li Jiang | L_p Regularization with Forward-backward Splitting |
Date | Speaker | Recorder | Topic |
---|---|---|---|
Jan 30, 2019 | Li Jiang | Shaobo Liang | Multiclass Support Vector Machine |
Feb 20, 2019 | Shaobo Liang | Pengfei Yin | Work complexity for Large-scale Learning |
Feb 27, 2019 | Qian Zhang | Yuyan Chen | Without-replacement sampling for SGD |
Mar 11, 2019 | Xuhao Diao | Jianhong Chen | LSTM |
Mar 18, 2019 | Juncai He | Lin Li | Nonlinear Approximation via Composition |
Mar 25, 2019 | Xiaodong Jia | Liang Zhao | Introduction to Training Neural Networks with PyTorch |
Apr 08, 2019 | Jianhong Chen | Lian Zhang | Diagnosing Pregnancy Based on Wrist Pulse Wave Using Convolutional Neural Network |
Apr 15, 2019 | Lian Zhang | Xiaodong Jia | A Review of Some Existing Network Pruning Algorithms |
Apr 22, 2019 | Chuanbin | – | A PDE Model for Pulse Wave Modeling |
Apr 29, 2019 | Lin Li | Juncai He | Approximation Properties of ReLU DNN |
May 6, 2019 | Yuyan Chen | Jing Liu | Linear Data-feature Mapping in Different Grids from Classical CNN Models |
May 19, 2019 | Huang Huang | Li Jiang | A Neural Network Based on The Non-standard Wavelet Form |
Jun 3, 2019 | Jing Liu | Pengfei Yin | On The Convergence Rate of Training RNN |
Jun 17, 2019 | Pengfei Yin | Shaobo Liang | Generalization Bounds for Classification Problems |
*Topics colored blue are having been completed, and those colored black are in preparation.
DeepLearning Seminars of 2018-2019 Autumn Term
We have successfully raised 9 seminars last semester. For your information, here are the specfic records.
Date | Topic | Speaker |
---|---|---|
Sep 12, 2018 | Multigrid and CNN | Juncai He |
Sep 19, 2018 | RDA | Liang Zhao & Xiaodong Jia |
Oct 10, 2018 | Multigrid and CNN | Juncai He |
Oct 17, 2018 | Degree of approximation for feedforward networks | Chunyue Zheng |
Oct 24, 2018 | Spectral method | Lin Li |
Nov 28, 2018 | Introduction of Supervised Learning-1 | Pengfei Yin |
Dec 05, 2018 | Introduction of Supervised Learning-2 | Pengfei Yin |
Dec 12, 2018 | Introduction of Supervised Learning-3 | Pengfei Yin |
Jan 16, 2019 | Introduction of Supervised Learning-4 | Pengfei Yin |