Summer Course in Peking University

Summer Course in Peking University

 

Course Description

This is an advanced undergraduate course on the introduction of basic mathematical and practical aspects of deep learning techniques. The course will cover some basic deep learning models such as convolutional neural networks, training algorithms such as stochastic gradient descent methods, popular data bases such as MNIST and CIFAR and specific applications such as image classifications. Examples of other successful applications of deep learning will also be briefly discussed for various tasks of machine learning and artificial intelligence such as computer vision, natural language processing and reinforcement learning. More specifically, the following topics will be covered:

  • Basics of machine learning, logistic regression and SVM
  • Deep neural networks and mathematical properties
  • Multigrid methods and convolutional neural networks
  • Stochastic gradient descent methods and convergence analysis
  • PyTorch and deep learning for image classification
  • Other applications

The course will be held at Peking University during July 6-27, 2019. Lectures and lab hours will be given 8am-9:30am and 2pm-4pm and Q&A sessions will be 10:00am-11:30am every weekday. Students are expected to practice and do assignments during the afternoon sessions. Typed lecture notes will be distributed in class. Both math and computing homework problems will be assigned. We will have four homeworks in the first two weeks, a mid-term and a final project.

 

Course Lecturer

Teaching Assistants

  • Jonathan Siegel (Penn State)
  • Juncai He (Penn State)
  • Jianhong Chen (Penn State)
  • Xiaodong Jia (Peking University)
  • Yuyan Chen (Peking University)
  • Huang Huang (Peking University)

 

Course Time and Location

  • Three weeks for 3 credits: July 6-27, 2019
  • Time: 8:00-9:30 am and 2:00-3:30 pm
  • Place: Room 408 in the Third Teaching Building(三教408), Peking University, Beijing China

Grading Policy

  • Assignment #1: 10%
  • Assignment #2: 10%
  • Assignment #3: 10%
  • Assignment #4: 10%
  • Midterm: 20%
  • Course Project: 40%

Course Related

  • Course: Math 497,  An Introduction to Deep Learning
  • Coordinator: Shaobo Liang and Li Jiang (Peking University)
  • Staff Assistant: Shenglan Zhao (slzhao@pku.edu.cn)
  • For a detailed timetable of the course, see Timetable

Notes and homeworks

Students gallery

Group Photos in Peking University, Tsinghua University, Chinese Academy of Science and the Forbidden City.

       

 

Class Photos in Peking University

 

 

Group Activities

    

 

  

Students impressions

  • Student 1: We become enthusiastic because all the professors  were excited to teach. I really appreciate that they gave us this precious opportunity to taking class in Peking University. Through these three weeks, I did learn a lot. I have more mathematical understanding about the CNN structure and my programming skills are getting better and better.
  • Student 2:  Our professor is very helpful, and cares for every student. He always asked us if there are any questions or if he talked too fast. With the limit of knowledge, we always feel a little ashamed to ask such a simple question on class, but he is so nice and encouraged us a lot, also talked with us after class, which give me more motivation to learn.
  • Student 3: This class is a great opportunity for me to appreciate math and see real examples of how useful it can be.
  • Student 4:  I had a great time learning the math behind it, also I picked up the coding in just three weeks, thanks to all the graduate and postdoc’s(TA) help.

 

More on CCMA

 



Continue reading “Summer Course in Peking University”

Seminar Calendar of Deep Learning

  Seminars of 2019

Deep Learning Seminars of 2019-2020

  • Time: EST 8: 00 am(BJS 20: 00 pm) of  Tuesday
  • Period: Each Week
  • Topic: Alternative and flexible
  • You can get the material of our seminar, just click the underlined hyperlink.

Initial Schedule

Date Speaker Topic Material or Summary
Jul 31 Huang Huang Initialization in DNN Related Paper
YuYan Chen Implement of MgNet
Aug 7 Jonathan Siegel Sparse Training Algorithms for Neural Networks
Qian Zhang SGD: General Analysis and Improved Rates Talk Summary From Qian
Sep 3 Pengfei Yin Deep Residual Networks 1, Deep Residual Networks 2 Talk Material From Pengfei
Jianhong Chen Learning Structured Sparsity in Deep Neural Networks Talk Summary From Jianhong
Sep 10 Jianqing Zhu Lookahead Optimizer: k steps forward, 1 step back Talk Material From Jianqing
Lin Li Deep Neural Networks with Rectified Power Units Talk Material From Lin
Sep 17 Juncai He Adaptive Learning Rate Schedule in SGD
Sep 24 Juncai He, Jonathan Siegel Using Statistics to Automate SGD

* The row of above table colored aqua is in preparation.

 Seminars of PSU-PKU Summer School

Schedule of FIRST Week 

Date Speaker Recorder Topic
Jul 8 Liang Zhao Yuyan Chen Implicit regularization property of SGD method
Jul 10 Juncai He Jianhong Chen Convergence of backpropagation with momentum for network architectures with skip connections
Jul 12 Lin Li Lian Zhang Approximation properties of ReLU DNN

Schedule of SECOND Week 

Date Speaker Recorder Topic
Jul 15 Shaobo Liang Qian Zhang k-means
Jul 17 Li Jiang Pengfei Yin A Unified View of Multi-class SVM
Jul 19 Lian Zhang Juncai He Model Compression of CNN

Schedule of THIRD Week 

Date Speaker Recorder Topic
Jul 22 Zhe Zheng Li Jiang L_p Regularization with Forward-backward Splitting

 Seminars of 2019 Spring Term

Date Speaker Recorder Topic
Jan 30, 2019 Li Jiang Shaobo Liang Multiclass Support Vector Machine
Feb 20, 2019 Shaobo Liang Pengfei Yin Work complexity for Large-scale Learning
Feb 27, 2019 Qian Zhang Yuyan Chen Without-replacement sampling for SGD
Mar 11, 2019 Xuhao Diao Jianhong Chen LSTM
Mar 18, 2019 Juncai He Lin Li Nonlinear Approximation via Composition
Mar 25, 2019 Xiaodong Jia Liang Zhao Introduction to Training Neural Networks with PyTorch
Apr 08, 2019 Jianhong Chen Lian Zhang Diagnosing Pregnancy Based on Wrist Pulse Wave Using Convolutional Neural Network
Apr 15, 2019 Lian Zhang Xiaodong Jia A Review of Some Existing Network Pruning Algorithms
Apr 22, 2019 Chuanbin A PDE Model for Pulse Wave Modeling
Apr 29, 2019 Lin Li Juncai He Approximation Properties of ReLU DNN
May 6, 2019 Yuyan Chen Jing Liu Linear Data-feature Mapping in Different Grids from Classical CNN Models
May 19, 2019 Huang Huang Li Jiang A Neural Network Based on The Non-standard Wavelet Form
Jun 3, 2019 Jing Liu Pengfei Yin On The Convergence Rate of Training RNN
Jun 17, 2019 Pengfei Yin Shaobo Liang Generalization Bounds for Classification Problems

*Topics colored blue are having been completed, and those colored black are in preparation.

DeepLearning Seminars of 2018-2019 Autumn Term

We have successfully raised 9 seminars last semester. For your information, here are the specfic records.

Date Topic Speaker
Sep 12, 2018 Multigrid and CNN Juncai He
Sep 19, 2018 RDA Liang Zhao & Xiaodong Jia
Oct 10, 2018 Multigrid and CNN Juncai He
Oct 17, 2018 Degree of approximation for feedforward networks Chunyue Zheng
Oct 24, 2018 Spectral method Lin Li
Nov 28, 2018 Introduction of Supervised Learning-1 Pengfei Yin
Dec 05, 2018 Introduction of Supervised Learning-2 Pengfei Yin
Dec 12, 2018 Introduction of Supervised Learning-3 Pengfei Yin
Jan 16, 2019 Introduction of Supervised Learning-4 Pengfei Yin