Merge pull request #330 from SciML/constut

Add constraints tutorial

docs/Project.toml

@@ -1,12 +1,17 @@
 [deps]
+AmplNLWriter = "7c4d4715-977e-5154-bfe0-e096adeac482"
 Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
 FiniteDiff = "6a86dc24-6348-571c-b903-95158fe2bd41"
 ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210"
+Ipopt = "b6b21f68-93f8-5de0-b562-5493be1d77c9"
+Ipopt_jll = "9cc047cb-c261-5740-88fc-0cf96f7bdcc7"
 IterTools = "c8e1da08-722c-5040-9ed9-7db0dc04731e"
 ModelingToolkit = "961ee093-0014-501f-94e3-6117800e7a78"
+Optimization = "7f7a1694-90dd-40f0-9382-eb1efda571ba"
 OptimizationBBO = "3e6eede4-6085-4f62-9a71-46d9bc1eb92b"
 OptimizationCMAEvolutionStrategy = "bd407f91-200f-4536-9381-e4ba712f53f8"
 OptimizationEvolutionary = "cb963754-43f6-435e-8d4b-99009ff27753"
+OptimizationMOI = "fd9f6733-72f4-499f-8506-86b2bdd0dea1"
 OptimizationNLopt = "4e6fcdb7-1186-4e1f-a706-475e75c168bb"
 OptimizationOptimJL = "36348300-93cb-4f02-beb5-3c3902f8871e"
 OptimizationOptimisers = "42dfb2eb-d2b4-4451-abcd-913932933ac1"

docs/pages.jl

@@ -5,7 +5,8 @@ pages = [
             "tutorials/intro.md",
             "tutorials/rosenbrock.md",
             "tutorials/minibatch.md",
-            "tutorials/symbolic.md"
+            "tutorials/symbolic.md",
+            "tutorials/constraints.md",
         ],
 
         "API" => [
@@ -30,4 +31,4 @@ pages = [
             "Optimisers.jl" => "optimization_packages/optimisers.md",
             "QuadDIRECT.jl" => "optimization_packages/quaddirect.md"
         ],
-    ]
+    ]

docs/src/tutorials/constraints.md

@@ -1 +1,99 @@
-# [Using Equality and Inequality Constraints](@id constraints)
+# [Using Equality and Inequality Constraints](@id constraints)
+
+Multiple optmization packages available with the MathOptInterface and Optim's `IPNewton` solver can handle non-linear constraints.
+Optimization.jl provides a simple interface to define the constraint as a julia function and then specify the bounds for the output
+in `OptimizationFunction` to indicate if it's an equality or inequality constraint.
+
+Let's define the rosenbrock function as our objective function and consider the below inequalities as our constraints.
+
+```math
+\begin{aligned}
+
+x_1^2 + x_2^2 \leq 0.8 \\
+
+0.0 \leq x_1 * x_2 \leq 5.0
+\end{aligned}
+```
+
+```@example constraints
+using Optimization, OptimizationMOI, OptimizationOptimJL, ForwardDiff, ModelingToolkit
+
+rosenbrock(x, p) = (p[1] - x[1])^2 + p[2] * (x[2] - x[1]^2)^2
+x0 = zeros(2)
+_p = [1.0, 1.0]
+```
+
+Next we define the sum of squares and the product of the optimization variables as our constraint functions.
+
+```@example constraints
+cons(res, x, p) = (res .= [x[1]^2+x[2]^2, x[1]*x[2]])
+```
+
+We'll use the `IPNewton` solver from Optim to solve the problem.
+
+```@example constraints
+optprob = OptimizationFunction(rosenbrock, Optimization.AutoForwardDiff(), cons = cons)
+prob = OptimizationProblem(optprob, x0, _p, lcons = [-Inf, -1.0], ucons = [0.8, 2.0])
+sol = solve(prob, IPNewton())
+```
+
+Let's check that the constraints are satisfied and the objective is lower than at initial values to be sure.
+
+```@example constraints
+res = zeros(2)
+cons(res, sol.u, _p)
+res
+```
+
+```@example constraints
+prob.f(sol.u, _p)
+```
+
+We can also use the Ipopt library with the OptimizationMOI package.
+
+```@example constraints
+sol = solve(prob, Ipopt.Optimizer())
+```
+
+```@example constraints
+res = zeros(2)
+cons(res, sol.u, _p)
+res
+```
+
+```@example constraints
+prob.f(sol.u, _p)
+```
+
+We can also use ModelingToolkit as our AD backend and generate symbolic derivatives and expression graph for the objective and constraints.
+
+Let's modify the bounds to use the function as an equality constraint. The constraint now becomes -
+
+```math
+\begin{aligned}
+
+x_1^2 + x_2^2 = 1.0 \\
+
+x_1 * x_2 = 0.5
+\end{aligned}
+```
+
+```@example constraints
+optprob = OptimizationFunction(rosenbrock, Optimization.AutoModelingToolkit(), cons = cons)
+prob = OptimizationProblem(optprob, x0, _p, lcons = [1.0, 0.5], ucons = [1.0, 0.5])
+```
+
+Below the AmplNLWriter.jl package is used with to use the Ipopt library to solve the problem.
+
+```@example constraints
+using AmplNLWriter, Ipopt_jll
+sol = solve(prob, AmplNLWriter.Optimizer(Ipopt_jll.amplexe))
+```
+
+The constraints evaluate to 1.0 and 0.5 respectively as expected.
+
+```@example constraints
+res = zeros(2)
+cons(res, sol.u, _p)
+println(res)
+```