EN

项目2019–2021creatorfirst author

Teeport

Break the wall between optimization algorithms and problems.

这一页还没有中文版，先放英文原文。

At an accelerator like SPEAR3 the optimization problem lives in the control room, usually as a Matlab script that sets some magnets and reads the beam back. The algorithms live on laptops, a lot of them in Python. Trying a new one meant cloning it onto a control-room computer and wiring it to the script by hand, and setting up its environment there often needed admin rights (the setup could also break the control system's own configuration). Then the next one, same thing again. I wanted to run the optimizer from where it already was and leave the script alone, so I came up with Teeport. I wrote the server, the dashboard, the plugins and most of the adapters at SLAC from 2019 to 2021, and Xiaobiao Huang, Minghao Song and I wrote it up in Frontiers in Big Data in 2021.

The wall

In my 2020 slides it was a brick wall with three words on it. Distance and language are the laptop and the Matlab from above, and confidentiality is about the script. An evaluation script can hold the token for the control system, or knobs you would rather nobody else saw, so it should stay where it is. Teeport leaves the code on both sides alone and only moves data through the wall, mostly X (the settings to try) one way and Y (what was measured) the other.

A brick wall labelled confidentiality, distance and language stands between your algorithm, on a laptop and often in Python, and the optimization problem, on a control-room computer and often in Matlab. A Teeport pipe runs through the wall: X goes from the algorithm to the problem and Y comes back. confidentiality · distance · language Teeport XYXY Your algorithm on a laptop, often in Python optimize(evaluate) The problem on a control-room computer, often in Matlab Y = evaluate(X)
Redrawn from my 2020 slides, with Teeport as the pipe through the wall.

run_ and use_

Take that Matlab script. To Teeport it is just a function, Y = evaluate(X). An algorithm is a function that takes one of those, optimize(evaluate). In the control room, teeport.runEvaluator publishes the script and gives you an id. On a laptop, use_evaluator takes that id and hands back an ordinary local function:

from teeport import Teeport
teeport = Teeport('ws://localhost:8080/')
evaluate = teeport.use_evaluator('c4oiY1_oe')

Your optimizer calls it like any other function. It looks local. The measurement still happens in the control room, and whoever published the script decides when it runs, so your optimizer cannot reach the machine except through it. Optimizers get the same pair of calls, so someone with a problem can borrow an algorithm by id.

The hidden optimizer

use_evaluator took the most work (it lives in the Python client). On the server every run is a task, and a task needs an optimizer client, a function that keeps running and produces X. Here there is no such function, just your loop calling evaluate(X) whenever it likes. So your first call quietly starts a private optimizer inside the client and creates the task. Its loop never ends. It waits on a future for the next X, sends it out, and puts the Y that comes back into a second future, the one your evaluate(X) is waiting on. To you it is a blocking function, so an existing algorithm can work on a remote problem unchanged.

One call to evaluate(X) returned by use_evaluator(id): 1, your code calls evaluate with X; 2, evaluate sets a future with the next X; 3, the hidden private optimizer, which awaits that future, sends X to the remote evaluator; 4, Y comes back; 5, the hidden optimizer sets a second future with Y; 6, evaluate, which was waiting on that future, returns Y to your code. Your code calls evaluate(X) from a plain loop or any algorithm evaluate(X) the function use_evaluator(id) gave you; it blocks until Y is back Hidden optimizer private, started by your first call, loops forever Remote evaluator behind the server; the measurement happens there 1 · X 6 · Y 2 · next X 5 · next Y futures 3 · X 4 · Y
Two futures carry X to the hidden optimizer and Y back to you, once per call to evaluate(X).

The pending queue

The server is a small Node.js WebSocket service, and it does as little as it can. Before SLAC I had written the same kind of server at Visual3D, the one every headset and workstation in a mixed-reality navigation system talked through. Teeport's server passes X to the evaluator and Y back to the optimizer, and anyone watching gets a copy of Y. It also keeps the history. Every X lands in a pending queue first and goes on only if the task is running. That one check is all there is to pause and resume. Pause, and the batch waits. Resume, and the server replays the queue to the evaluator. Neither the algorithm nor the problem finds out, and neither side had to add a line of code for it.

All of it lives in memory. A run survives a server restart only if somebody exported it to JSON first. I drew an archive database behind the server and a rollback to an earlier point in a run, and neither got built.

One round trip through the Teeport server, which keeps everything in memory. The optimizer sends X; the server puts it in the pending queue and forwards it to the evaluator only if the task is running, so a paused task leaves X waiting and resuming replays the queue. The evaluator sends Y back; the server appends X and Y to the history and sends Y to the optimizer and to the dashboard's task page, which gets Y only. Teeport server in memory · export to JSON pending running? paused: X waits here · resumed: the queue is replayed history · [X, Y] for every generation Optimizer any algorithm Evaluator the script that talks to the machine Dashboard task page XYY X, only if running Y only
X waits in the pending queue unless the task is running, and the dashboard's task page receives only Y.

The dashboard

The dashboard (React and Plotly) is one more client. A task page subscribes as a monitor that asks for Y only, so the server blanks X in every generation before sending, and X comes down when you download the run. That was for big problems. NSGA-II on a 500-dimensional problem piles up about a gigabyte of history by the time it converges, and a browser tab will not hold that. You pick runs in the task list and press Compare to overlay them, even while some are still going, or benchmark an optimizer over repeated runs.

Teeport task list with two selected task cards, RCDS vs Lossrate Exp created at 2020-06-02 20:42:56 and P-GPO vs Lossrate Exp created at 2020-06-02 19:28:42, under a toolbar with New, Compare, More and a By Created sort.
Two of our SPEAR3 loss-rate runs in the task list, both created on June 2, 2020 and picked for the comparison below.
Teeport dashboard, Evaluation History panel: three runs, PSO, RCDS and P-GPO, overlaid against the SPEAR3 loss-rate experiment evaluator, each with its evaluations as dots and its best-so-far as a dashed step line.
PSO, RCDS and P-GPO (a physics-informed GP optimizer) against the same SPEAR3 loss-rate evaluator, overlaid on one Teeport plot.

On SPEAR3

The three of us used it on storage-ring optimizations at SPEAR3, and Minghao, our first user, had tested the prototype and written part of the Matlab adapter. In our January 2021 report to SSRL we wrote that most of the online optimizations in it, simulated and on the ring, ran through Teeport, the loss-rate runs above among them. We published it from a Matlab session in the control room and optimized it from a laptop. A run could go against the simulated ring first and then the machine, by changing one id.

The wall turned up inside one algorithm as well. MG-GPO is Xiaobiao's algorithm (Minghao and I developed it with him), and it is written in Matlab. We could not find a good Gaussian-process package for Matlab, and Python had GPy. So GPy ran as a Teeport processor (a stateless helper either side can call), teeport.useProcessor gave the Matlab code a predict function, and everything else stayed in Matlab. How the tuning went is on the MG-GPO page.

Cartoon: a developer holds a local optimize and evaluate; the evaluate connects through Teeport to either a Simulation evaluator or an Experiment evaluator, each with its own id.
The same local code against the simulation or the experiment, one id apart.
Diagram: Matlab MG-GPO sends X0, Y0 and X1 through Teeport to a GPy processor in Python, which returns the prediction Y1.
Matlab MG-GPO handing its data to GPy and getting the prediction back.

After Teeport

Teeport was good at remote runs and at benchmarking algorithms. Hands-on tuning in the control room needed something else, and in 2021 I started Badger for that. The public instance at teeport.info still loads. Its task list is empty, since everything lived in memory.

Sources are the paper (Zhang, Huang and Song, Frontiers in Big Data, 2021), our report to SSRL (Huang, Song and Zhang, January 2021), my 2020 slides and the public SPEAR3-ML repositories.