Bounds for measurement feedback in the good-control regime

Context and definitions from the 2014 source

II.3 Limits to quantum control

A practically important and fundamentally interesting question is how well various control objectives can be achieved given specific resources. When controlling mesoscopic systems, a major experimental constraint is the strength, or speed, with which we can interact with the system. The problem with obtaining theoretical limits to feedback control is that to do so we must find optimal protocols. For measurement-based control this is near-impossible, since the equations of motion are both stochastic and nonlinear. But there may be a route to finding upper bounds by exploiting the connection between coherent and measurement-based control. In general, while the equations for coherent feedback control are linear, since they involve only the Schrödinger equation, they are just as complex as the nonlinear equations of MBC. However the problem simplifies if we consider the regime of good control. Good control implies that the control protocol is able to achieve the objective with a fidelity close to unity. This means that the rate at which it is able to extract entropy from the system is much faster than the rate at which the noise generates entropy, and thus the speed of the interaction with the system is correspondingly fast. Fortunately the regime of good control is of practical importance, and its simplifying feature is that the noise can be treated as a perturbation on the dynamics.

In [5] the authors were able to obtain the optimal (reversibly coupled) coherent feedback protocol for preparing pure states (e.g. for cooling) in the presence of thermal noise, in the regime of good control. The authors assumed an ideal auxiliary system so as to obtain the limits to cooling under a constraint on the speed of the interaction with the system. This shows that it is possible to obtain optimal results for CFC in the regime of good control, and it may therefore be possible to use similar results to obtain bounds on measurement-based control.

In [5] the authors did not solve the most general problem of cooling in the regime of good control, but restricted themselves to the regime in which the interaction with the system does not change its energy eigenvalues appreciably. The reason for this restriction is that it corresponds to the regime in which the control operations do not change the dynamics induced by the thermal noise. The Markovian Redfield master equation that describes this noise depends on the system’s energy levels. Thus the authors of [5] solve the problem of optimal CFC for the case in which the master equation is fixed. Relaxing this restriction would provide the answer to the fundamental question of just how well one can prepare pure states given a maximal speed of interaction with the system, taking account of the fact that one can significantly alter the dynamics of the system during the process. While the structure of the more general problem is somewhat different to that solved in [5], the results in [5] suggest that it may be tractable. Further, for a Markovian and therefore memoryless bath, it is easy to determine the Master equation as a function of the control Hamiltonian: the Master equation must immediately adapt to the new Hamiltonian at each point in time. Thus the master equation is merely the Redfield equation implied by the Hamiltonian of the joint system at each time.

Often there is a practical bound on the interaction strength between two mesoscopic systems, but this does not stop us from coupling a system of interest to multiple auxiliary systems. As we increase the number of auxiliary systems, the effective coupling rate between the system and the set of auxiliaries increases.

We mentioned above that it may be possible to bound the performance of continuous MBC by using bounds obtained on CFC, where the CFC is chosen to have equivalent resources (irreversible coupling strength). The reason for this is that CFC subsumes MBC. We also mentioned that reversibly coupled CFC is equivalent to irreversibly coupled CFC with the addition of coupling to specifically chosen baths. Thus by finding optimal solutions for reversibly coupled CFC (as in problems 6 and 7), but with the addition of some irreversible bath coupling, it may be possible to obtain useful bounds on continuous MBC.

Control models

I Introduction: feedback and open-loop control

When one thinks of feedback control, it is likely that the use of measurements will spring to mind. Measurements do not sit easily with quantum theory, and in fact they are somewhat superfluous. Measurements can always be replaced by considering instead the joint evolution of two systems, a primary system and an auxiliary. A measurement process on the primary constitutes the formation of correlations between the two, and feedback is simply the exploitation of these correlations by the joint evolution. In fact it is becoming increasingly clear that the ability to swap between a picture involving explicit measurements and one without measurements is useful. It suggests two alternative ways to physically implement the same control process, one in which measurement results are processed by classical computers, and one that merely involves the application of time-dependent control to a collection of quantum systems. While these two implementations are in a sense equivalent, they are very different from a practical point of view, and both have relative advantages. Further, while coherent feedback subsumes measurement-based feedback, the reverse is not true [6].

To distinguish between the two ways of implementing feedback control we refer to the one that involves explicit measurements as measurement-based feedback control, and the one without measurements as coherent feedback control. Let us now consider the most general form of feedback control and see how the different categories of control appear within this form. Since one can always subsume measurements into unitary evolution by including an auxiliary system, the most general feedback control scenario need not refer to measurements. We can therefore define this control in the following way. A primary system $S$ is coupled to an auxiliary system $C$ (here $C$ stands for controller). Both the system and controller will in general also be coupled to environments (other quantum systems) that inject noise into them, and we denote these respectively by $\mathcal{E}_{S}$ and $\mathcal{E}_{C}$. The system and auxiliary evolve under a time-dependent Hamiltonian $H(t)$, whose purpose is solely to control the evolution of $S$. In addition to designing some or all of the time-dependence of $H$, we may also have some limited control over the coupling of each of the two systems to their environments. This is the most general form of feedback control, and is depicted in Fig. 1(top).

Figure 1: Depictions of the most general implementation of feedback (top) and feedforward (bottom) control of quantum systems. In the former the auxiliary system, or controller, does not have direct access to the noise driving the system. The direction of the arrows represents the direction in which the flow of information impacts the control problem: In feedback control neither bath provides any useful information to the systems, while in feedforward the auxiliary can extract useful information from bath 1 about the noise it injects into the system.

If the controller $C$ is a quantum system whose states are not directly accessible to an observer, then the above scenario is coherent feedback control. If some of the states of the auxiliary, or more generally subspaces of the auxiliary, are macroscopically distinct and accessible to an observer, then the scenario is that of measurement-based feedback control. In this case the auxiliary must have within it a means of amplification. That is, the microscopic auxiliary states that become correlated with the microscopic states of the system must be mapped to macroscopically distinct states, whose energy is necessarily much larger than that of the microscopic states.

Shortly after posting the first version of this monograph, I had the opportunity to discuss the above definitions of feedback and feedforward with Daniel Lidar, Robert Kosut, Edmond Jonckhere, and Herschel Rabitz. Kosut pointed out that there was more than one definition of feedforward in the classical control community, and further, Jonkhere mentioned that historically some categories of control have been very hard to precisely define. One well-known control theorist at a famous discussion of “adaptive” control at an IEEE meeting remarked that “the only way to know if a control protocol was adaptive was to ask the designer”. It was also suggested that clarifying the role of Markovian vs. non-Markovian baths in these definitions would be useful. In the Appendix I discuss these points in more detail, elucidating the different classical notions of feedback and feedforward. It may be that grey areas between these categories are inevitable, and it may also be important not to be overly zealous in asserting a single definition as superior. Nevertheless it is important to understand the motivation and justification for any definition, and to specify clearly the definition used in a given piece of work. Below I explain why feedback can be defined without reference to a feedback “loop”. I thank Francesco Ticozzi for hospitality at a meeting in Padova, and Robin Blume-Kohout and John Gough for discussions at that meeting that were very helpful in clarifying why loops are redundant in the definition of feedback.

To introduce some terminology, we note that the purpose, or objective, of a control procedure may be to prepare a particular state at a given time (state-preparation), to realize a particular unitary evolution for the system over some time interval, or to have the system evolve in some specified way as a function of time. A specific choice for the auxiliary system and the joint time-dependent Hamiltonian, designed to achieve a specific objective is called a control protocol.

There are a few points that are worth noting about the above scenarios, and we pause to discuss these now.

I.1 Everything within open-loop control

If we remove the auxiliary system, or equivalently decide that we wish to control the whole of the joint system, $J=S\otimes A$, then the feedback control scenario reduces to that of open-loop control, traditionally referred to simply as quantum control, in which the Hamiltonian is varied as function time to achieve an objective. In a sense feedback control is open-loop control in which we wish “only” to control a subsystem of the system to which we apply the control.

I.2 Where is the feedback loop?

Traditionally feedback control involves extracting information from a system, and then using this information to determine forces (controls) that are applied to the system. The applied controls are often referred to as being conditional, or conditioned, on the measurement results. The flow of information to the controller via the measurements, and then back to the system via the control forces, is described as a loop. This is why feedback control is also referred to as closed-loop control. In the description of feedback control depicted in Fig. 1 we have dropped the loop, and this is something that tends to confuse control theorists. We now explain why the existence of a loop in feedback control is merely a conceptual device; one can just a well describe feedback without using a loop.

The simplest, and possibly the clearest way to explain the redundancy of a loop structure in feedback control is to use an example, and ours will be the Watt governor. This is a device for controlling the speed of a steam engine. The control device, the “governor”, was a metal ball that would swing outward under the centrifugal force of the circular motion of the engine. The ball was attached to a valve that controlled the flow of fuel to the engine. As the speed of the engine increased, the ball would swing further out, and this would reduce the flow of fuel. The governor thus provided a dynamics that stabilized the speed of the engine. We also see that that the engine and the governor are merely two dynamical systems coupled together: the equations of motion of the engine have terms that depend on the state of the governor, and those of the governor depend on the state of the engine. This relationship is symmetric, in that there is no causal ordering of the effects of each system on the other. There is nothing that says that the effect of the system on the controller is one that provides information to the controller, and that the effect of the controller on the system is based on this information. There is merely an interaction between the two systems. A Hamiltonian formulation of the dynamics of the two systems would have one or more interaction Hamiltonians, but would not involve any “loop”. Thus the “loop” in feedback control is merely a conceptual tool.

We can speak of a Hamiltonian interaction between two systems as implementing feedback control because the controller has no access to the noise driving the system. Thus anything that the controller does to reduce the effects of this noise must exploit information that it obtains from the system. While there is no loop in the description of the feedback, and while there may not be an obvious way to quantify the flow of information, especially for quantum systems, the action of the controller falls within the natural definition of feedback control.

I.3 Wither feedforward control?

Feedforward control, as defined in classical control theory, is a scenario in which the controller has access to the noise that is affecting the system before it reaches the system. The controller can then determine forces to apply to the system based on this noise and then feed these forward to the system to counteract the noise. This is not usually considered for quantum systems because one does not usually have useful access to the environment. But there could be situations in which feedforward is useful for controlling quantum systems, and we depict this situation in Fig. 1(bottom). The only difference between feedback and feedforward is that in the latter the auxiliary system has access to the environment $\mathcal{E}_{S}$.

I.4 Reversible or irreversible coupling?

In measurement-based feedback control one often considers measurements in which information is continually extracted from the system, so called “continuous measurements” [2]. Continuous measurements are realized by coupling the system to an environment, or bath, in which the interaction is irreversible and continuously carries off information about the system. The archetypical example of this is the act of measuring the position of an object by shining a beam of light on it and observing the reflected light. The many modes of the traveling-wave electromagnetic field (the light) are a bath with which the system interacts, and the reflected light continually carries away information.

If instead of measuring the reflected light in the above example, we injected this light into an auxiliary quantum system, and then allowed the auxiliary to act in some way upon the system, we would have replaced the continuous measurement-based feedback control with a version of coherent feedback control. The information extraction by the controller is mediated in this case by a quantum electromagnetic field. We could similarly apply the feedback forces by coupling the auxiliary to the system by a quantum field, where in this case the field travels from the auxiliary to the system. This scenario explicitly involves information traveling in one direction around a loop. While this puts it explicitly in the form of a classical feedback loop, it necessarily involves a loss of information via the irreversible connection to the fields. In fact, field-mediated coupling, often called a cascade connection because of the one-way flow of information [3, 4], is equivalent to a direct (reversible) coupling between the system and auxiliary, with the addition of irreversible coupling of each to separate environments. As a result coherent control via cascade connections is a strict subset of coherent feedback that involves direct coupling as depicted in Fig. 1 (top).

Historical source dated 2014-01-27. Current resolution status has not been independently reassessed; bounds and claims in the source context refer to that date.