GPU Acceleration

[BenVanDerMeer]

I've been working on an optimisation problem that needs to solve some quite large linear systems for the calculation of the Jacobian of the constraints. I've managed to perform these calculations on the GPU, and this resulted in a significant speedup (~15 s/iterations on the CPU vs ~2.5 s/iteration on the GPU). I was wondering if a similar process could be applied to Ipopt? I know that linear systems need to be solved for calculating the position of the next iteration, but I'm not sure of the details. What are these systems, and would they be well-suited for GPU acceleration? Has this been attempted or thought about before?

[frapac]

Ipopt computes the descent direction by solving a KKT system. It writes as a large sparse indefinite matrix (and ill-conditioned close to the optimum). For that reason, solving the linear systems on the GPU is not trivial. The KKT system is detailed in the Ipopt paper, equation (11).

To solve the KKT system on the GPU with Ipopt, you have two solutions.

1. Solve the KKT system directly using SPRAL. The linear solver has been integrated recently in Ipopt. However, it is known to be slow compared to its CPU equivalents.
2. Exploit the structure of the KKT system to accelerate the solution on the GPU. The approach can be quite effective, but remains domain specific. Our team has recently investigated the performance of such structure-exploiting solvers for the optimal power flow problem.

Doing (2) is also doable with Ipopt, as the code is well structured (you just have to implement a specific structure to solve your KKT system). But that would require to implement some nontrivial code in C++ and integrate it properly with Ipopt. In case you are interested, we have also developed a port of Ipopt in Julia to experiment how far we can benefit from GPU acceleration in nonlinear programming.

A third alternative is to use the KKT solver described in that recent paper. Our own experiments have shown that this method can give some significant speed-up when running on the GPU. You can find a C++ implementation in that repo, but to the best of my knowledge, it has not yet been integrated inside a nonlinear optimization solver.