Introduction to Optimization Servers

Remote solver execution made easy

The purpose of the PIFOP Optimization Server solution — or just Opt-Server — is to provide a convenient and flexible way to run solvers remotely. Traditionally, if a modeler wanted to offload the solution computation to a remote machine, they had two main options. The first was to SSH into the machine and run the solver that way, maybe automating that process through scripting. The second was to use a built-in solution provided by a commercial solver.

Each one of these methods have their own merits and demerits, which we are not going into. The Opt-Server is an alternative solution that, in general, is as effective as those two methods, with the advantage that an Opt-Server is easier to setup and it is not tied down to any particular solver or modeling language. It can even be used to run solvers that have no built-in remote execution capability whatsoever.

Opt-Servers are user-hosted servers that can run programs — solvers and interpreters — upon user request from the PIFOP IDE. Accounts with optimization server hosting enabled can host Opt-Servers for themselves, for colleagues and/or for students. Most Linux machines can run an Opt-Server: a home computer, a company's server, a personal laptop, a virtual machine at Google Cloud, etc.

How it works

In this section we explain how the Opt-Server works. There is nothing that you need to do in order to make it work the way we describe. Hosting an Opt-Server is as simple as downloading a program, editing a configuration file, and running the program.

The Big Picture


The above figure gives an overview of the system of which Opt-Servers are a part of. Terminal commands entered by users of the PIFOP IDE are sent to the pifop.com central server, which in turn sends the commands to Opt-Servers. Opt-Servers executes the commands and send the output back to pifop.com, which forwards this output to the IDE users.

Opt-Server inner workings

Each command line sent to Opt-Servers is executed within a sandbox of its own. The sandbox isolates the process running the command from the rest of the system, restricting what the proccess can do: what files it can read, where can it write to, how much memory it can consume, etc.

Suppose that you have a project at PIFOP with the following three files: model.mod, data.dat and commands.run. Then you enter the following command line into a terminal while the Opt-Server My server is selected.

> ampl commands.run

What happens at My server when it receives that command?

First, it creates a directory containing all the files of your project as they appear to you at the PIFOP IDE, and a directory for writing temporary files. Then it creates a sandbox that can access some of the files and directories within the system. This includes:

  1. The directory containing the files of your project.
  2. The directory containing the command executable — in this example, the ampl directory.
  3. The directories containing the dynamic library dependencies of the executable.
  4. The directory created for temporary files.

With the exception of the project directory and the tmp directory, which are created just for the execution of the command, the files and directories in the sandbox are read-only. The file system structure of the sandbox for our example would look something like this — directory names can vary:

/user-workspace/        (Project files)
/cmd-executable/        (Executable)
/lib64/                 (Dependencies)
/lib/x86_64-linux-gnu/  (Dependencies)
/tmp/                   (Temporary files)

With this file structure in place, the command line is executed from within the user-workspace directory. As for the computing resources available for the process running the command, these can be limited by the Opt-Server administrator when configuring user groups.