Udemy – Optimization Engineering For Machine Learning and AI 2022-11

Optimization is a core fundamental area for machine learning and AI in general. Moreover, Convex optimization is a subfield of mathematical optimization that studies the problem of minimizing or maximizing convex functions over convex sets. Many classes of convex optimization problems admit polynomial-time algorithms, whereas mathematical optimization is in general NP-hard.In the first lesson/lecture of this course, we will talk about the following points:

• What is Optimization?
• Examples on Optimization
• Factors of Optimization
• Reliable/Efficient Problems
• Goals & Topics of this Course
• Brief History on Optimization

In the second lesson/lecture, we will be covering important points on convex sets, which are the following:

• 00:00:00   Affine Combination
• 00:01:33   Affine Set
• 00:08:21   Convex Combination
• 00:09:25   Convex Set
• 00:13:45   Convex Hull
• …..

In the third lesson/lecture of this course on convex optimization, we will be covering important points on convex functions

In Lecture 4 of this course on convex optimization, we will be covering the fundamental principles of convex optimization

In Lecture 5 of this course on convex optimization, we will be covering Linear Programming and the Simplex algorithm, which was introduced by George Dantzig

In Lecture 6 of this course on convex optimization, we will cover the essentials of Quadratic Programming

In Lecture 7 of this course on convex optimization, we will cover the essentials of Quadratically Constrained Quadratic Programs, i.e. QCQPs.

In Lecture 8 of this course on convex optimization, we will cover Second Order Cone Programming, i.e. SOCPs.

In Lecture 9 of this course on convex optimization, we will cover Geometric Programs, i.e. GPs.

In Lecture 10 of this course on convex optimization, we will cover Generalized Geometric Programs, i.e. GPs

In Lecture 11 of this course on convex optimization, we will cover Semidefinite programming, i.e. SDPs.

n Lecture 12 of this course on convex optimization, we will cover various topics related to Vector optimization, such as Pareto optimal points and the Pareto frontier, which is a well known boundary studied in Game theory, risk and trade-off analysis, portfolio analysis, etc.

In Lecture 13 of this course on convex optimization, we will cover various topics related to Vector optimization, such as Pareto optimal points and the Pareto frontier, which is a well known boundary studied in Game theory, risk and trade-off analysis, portfolio analysis, etc.

In Lecture 14 of this course on Convex Optimization, we introduce the Lagrangian duality theory. In essence, for each optimization problem (convex or not), we can associate a certain function referred to as the Lagrangian function. This function, in turn, has a dual function (which serves as an infimum over the variable of interest x). It turns out that, for any optimisation problem, the dual function is a lower bound on the optimal value of the optimisation problem in hand. This lecture focuses on many examples that derive the Lagrangian and the associated dual functions. MATLAB implementations are also presented to give useful insights.

In Lecture 15 of this course on Convex Optimization, we talk about a very very important topic in convex optimisation that is the Lagrange Dual Problem.

In Lecture 16 of this course on Convex Optimization, we talk about a very practical topic, when it comes to numerical optimization algorithms, and that is the ε-suboptimal inequality, which could report how good of an estimate we have. Said differently, the Lagrangian dual feasible points (λ,ν) provides a proof or certificate of the dual gap.

In Lecture 17 of this course on Convex Optimization, we talk about Complementary Slackness, which could be used a test for optimality, or it could even tell us which constraints are active and which are not !!

In Lecture 18 of this course on Convex Optimization, we talk about KKT conditions for nonconvex and convex optimization problems.

In Lecture 19 of this course on Convex Optimization, we talk about Perturbation and Sensitivity Analysis of general and convex optimization problems.

In Lecture 20 of this course on Convex Optimization, we talk about Equivalent Reformulations  of general and convex optimization problems.

In Lecture 21 of this course on Convex Optimization, we talk about the theorem of weak alternatives of general optimization problems.

In Lecture 22 of this course on Convex Optimization, we talk about the theorem of strong alternatives of convex optimization problems.

In Lecture 23 of this course on Convex Optimization, we focus on algorithms that solve unconstrained minimization type problems. The lecture evolves around unconstrained minimization problems that might or might not enjoy closed form solutions. Descent methods are discussed along with exact line search and backtracking. MATLAB implementations are given along the way.

What you’ll learn