Special Topics in Optimization and Learning: Practicum in Deep Learning

Description: Deep learning has recently become the dominant paradigm in machine learning and artificial intelligence. It has wide-ranging applications in engineering and science, such as computer vision, natural language processing, sequence modeling and physical system simulation. This course is a practical introduction to deep neural networks (DNN) within the broader context of machine learning. Topics to be covered include: practical review of classical ML methods (PCA, logistic regression, naive Bayes, KNN, SVM); feedforward, convolutional and recurrent neural networks; optimization for training DNN; generalization, validation and hyperparameter tuning; overfitting, underfitting and bias-variance trade-off; classification, clustering and regression; representation learning; sequence models; generative models. Students will experiment with architectures and algorithms using Keras, TensorFlow and Wolfram Language. Class time will be allocated approximately equally between structured instruction and group discussions plus practical exercises. Students will collaborate in groups of 5 on a semester-long project. Students can propose their own projects or choose from a list provided by the instructor. Projects should be similar to real-world problems and include a value proposition. Progress will be evaluated throughout the semester. The course will include a final report, and a presentation open to the academic community. Prerequisites: ESE 326 (or ESE 520) and ESE 417 (or CSE 417), experience in programming in Python or Wolfram Language. Second-year graduate or senior undergraduate standing. Enrollment for Fall 2024 is by permission of the instructor only subject to successful assessment of the prerequisites. Waits will be managed by department.