Control when the controller is as noisy as the system

Context and definitions from the 2014 source

II.5 Imperfect controllers

So far we have considered feedback control as if the auxiliary system were perfect. For measurement based control we can take the processing of the measurement results to be perfect but not necessarily the actions of the controller on the system. For coherent feedback control we cannot even assume that the controller itself, and thus the processing of the information is noiseless. In fact, if the system that we wish to control is noisy, often the auxiliary system will be just as noisy. If this were not the case then why not use the auxiliary in place of the system for that ever task we have in mind?

Sometimes there is a physical justification for why the controller may be different from the system. One example of this is “resolved-sideband” cooling, a simple example of CFC in which a mechanical resonator is cooled by coupling it to an electrical or optical auxiliary resonator. In this case the electrical/optical resonator has such a high frequency that it has essentially zero entropy at the ambient temperature. It is this fact that allows the interaction to transfer entropy out of the mechanical resonator to the auxiliary, cooling the former. But the auxiliary, while it has zero entropy and is in this sense less noisy, still undergoes damping. While this damping is essential to dump the extracted entropy into a reservoir, and thus to obtain steady-state cooling, the analysis in [7] shows that this damping reduces the ability of the control to prepare the purest possible state at a single time (this state is more pure than the best achievable steady-state).

Control models

I Introduction: feedback and open-loop control

When one thinks of feedback control, it is likely that the use of measurements will spring to mind. Measurements do not sit easily with quantum theory, and in fact they are somewhat superfluous. Measurements can always be replaced by considering instead the joint evolution of two systems, a primary system and an auxiliary. A measurement process on the primary constitutes the formation of correlations between the two, and feedback is simply the exploitation of these correlations by the joint evolution. In fact it is becoming increasingly clear that the ability to swap between a picture involving explicit measurements and one without measurements is useful. It suggests two alternative ways to physically implement the same control process, one in which measurement results are processed by classical computers, and one that merely involves the application of time-dependent control to a collection of quantum systems. While these two implementations are in a sense equivalent, they are very different from a practical point of view, and both have relative advantages. Further, while coherent feedback subsumes measurement-based feedback, the reverse is not true [6].

To distinguish between the two ways of implementing feedback control we refer to the one that involves explicit measurements as measurement-based feedback control, and the one without measurements as coherent feedback control. Let us now consider the most general form of feedback control and see how the different categories of control appear within this form. Since one can always subsume measurements into unitary evolution by including an auxiliary system, the most general feedback control scenario need not refer to measurements. We can therefore define this control in the following way. A primary system $S$ is coupled to an auxiliary system $C$ (here $C$ stands for controller). Both the system and controller will in general also be coupled to environments (other quantum systems) that inject noise into them, and we denote these respectively by $\mathcal{E}_{S}$ and $\mathcal{E}_{C}$. The system and auxiliary evolve under a time-dependent Hamiltonian $H(t)$, whose purpose is solely to control the evolution of $S$. In addition to designing some or all of the time-dependence of $H$, we may also have some limited control over the coupling of each of the two systems to their environments. This is the most general form of feedback control, and is depicted in Fig. 1(top).

Figure 1: Depictions of the most general implementation of feedback (top) and feedforward (bottom) control of quantum systems. In the former the auxiliary system, or controller, does not have direct access to the noise driving the system. The direction of the arrows represents the direction in which the flow of information impacts the control problem: In feedback control neither bath provides any useful information to the systems, while in feedforward the auxiliary can extract useful information from bath 1 about the noise it injects into the system.

If the controller $C$ is a quantum system whose states are not directly accessible to an observer, then the above scenario is coherent feedback control. If some of the states of the auxiliary, or more generally subspaces of the auxiliary, are macroscopically distinct and accessible to an observer, then the scenario is that of measurement-based feedback control. In this case the auxiliary must have within it a means of amplification. That is, the microscopic auxiliary states that become correlated with the microscopic states of the system must be mapped to macroscopically distinct states, whose energy is necessarily much larger than that of the microscopic states.

Shortly after posting the first version of this monograph, I had the opportunity to discuss the above definitions of feedback and feedforward with Daniel Lidar, Robert Kosut, Edmond Jonckhere, and Herschel Rabitz. Kosut pointed out that there was more than one definition of feedforward in the classical control community, and further, Jonkhere mentioned that historically some categories of control have been very hard to precisely define. One well-known control theorist at a famous discussion of “adaptive” control at an IEEE meeting remarked that “the only way to know if a control protocol was adaptive was to ask the designer”. It was also suggested that clarifying the role of Markovian vs. non-Markovian baths in these definitions would be useful. In the Appendix I discuss these points in more detail, elucidating the different classical notions of feedback and feedforward. It may be that grey areas between these categories are inevitable, and it may also be important not to be overly zealous in asserting a single definition as superior. Nevertheless it is important to understand the motivation and justification for any definition, and to specify clearly the definition used in a given piece of work. Below I explain why feedback can be defined without reference to a feedback “loop”. I thank Francesco Ticozzi for hospitality at a meeting in Padova, and Robin Blume-Kohout and John Gough for discussions at that meeting that were very helpful in clarifying why loops are redundant in the definition of feedback.

To introduce some terminology, we note that the purpose, or objective, of a control procedure may be to prepare a particular state at a given time (state-preparation), to realize a particular unitary evolution for the system over some time interval, or to have the system evolve in some specified way as a function of time. A specific choice for the auxiliary system and the joint time-dependent Hamiltonian, designed to achieve a specific objective is called a control protocol.

There are a few points that are worth noting about the above scenarios, and we pause to discuss these now.

I.1 Everything within open-loop control

If we remove the auxiliary system, or equivalently decide that we wish to control the whole of the joint system, $J=S\otimes A$, then the feedback control scenario reduces to that of open-loop control, traditionally referred to simply as quantum control, in which the Hamiltonian is varied as function time to achieve an objective. In a sense feedback control is open-loop control in which we wish “only” to control a subsystem of the system to which we apply the control.

I.2 Where is the feedback loop?

Traditionally feedback control involves extracting information from a system, and then using this information to determine forces (controls) that are applied to the system. The applied controls are often referred to as being conditional, or conditioned, on the measurement results. The flow of information to the controller via the measurements, and then back to the system via the control forces, is described as a loop. This is why feedback control is also referred to as closed-loop control. In the description of feedback control depicted in Fig. 1 we have dropped the loop, and this is something that tends to confuse control theorists. We now explain why the existence of a loop in feedback control is merely a conceptual device; one can just a well describe feedback without using a loop.

The simplest, and possibly the clearest way to explain the redundancy of a loop structure in feedback control is to use an example, and ours will be the Watt governor. This is a device for controlling the speed of a steam engine. The control device, the “governor”, was a metal ball that would swing outward under the centrifugal force of the circular motion of the engine. The ball was attached to a valve that controlled the flow of fuel to the engine. As the speed of the engine increased, the ball would swing further out, and this would reduce the flow of fuel. The governor thus provided a dynamics that stabilized the speed of the engine. We also see that that the engine and the governor are merely two dynamical systems coupled together: the equations of motion of the engine have terms that depend on the state of the governor, and those of the governor depend on the state of the engine. This relationship is symmetric, in that there is no causal ordering of the effects of each system on the other. There is nothing that says that the effect of the system on the controller is one that provides information to the controller, and that the effect of the controller on the system is based on this information. There is merely an interaction between the two systems. A Hamiltonian formulation of the dynamics of the two systems would have one or more interaction Hamiltonians, but would not involve any “loop”. Thus the “loop” in feedback control is merely a conceptual tool.

We can speak of a Hamiltonian interaction between two systems as implementing feedback control because the controller has no access to the noise driving the system. Thus anything that the controller does to reduce the effects of this noise must exploit information that it obtains from the system. While there is no loop in the description of the feedback, and while there may not be an obvious way to quantify the flow of information, especially for quantum systems, the action of the controller falls within the natural definition of feedback control.

I.3 Wither feedforward control?

Feedforward control, as defined in classical control theory, is a scenario in which the controller has access to the noise that is affecting the system before it reaches the system. The controller can then determine forces to apply to the system based on this noise and then feed these forward to the system to counteract the noise. This is not usually considered for quantum systems because one does not usually have useful access to the environment. But there could be situations in which feedforward is useful for controlling quantum systems, and we depict this situation in Fig. 1(bottom). The only difference between feedback and feedforward is that in the latter the auxiliary system has access to the environment $\mathcal{E}_{S}$.

I.4 Reversible or irreversible coupling?

In measurement-based feedback control one often considers measurements in which information is continually extracted from the system, so called “continuous measurements” [2]. Continuous measurements are realized by coupling the system to an environment, or bath, in which the interaction is irreversible and continuously carries off information about the system. The archetypical example of this is the act of measuring the position of an object by shining a beam of light on it and observing the reflected light. The many modes of the traveling-wave electromagnetic field (the light) are a bath with which the system interacts, and the reflected light continually carries away information.

If instead of measuring the reflected light in the above example, we injected this light into an auxiliary quantum system, and then allowed the auxiliary to act in some way upon the system, we would have replaced the continuous measurement-based feedback control with a version of coherent feedback control. The information extraction by the controller is mediated in this case by a quantum electromagnetic field. We could similarly apply the feedback forces by coupling the auxiliary to the system by a quantum field, where in this case the field travels from the auxiliary to the system. This scenario explicitly involves information traveling in one direction around a loop. While this puts it explicitly in the form of a classical feedback loop, it necessarily involves a loss of information via the irreversible connection to the fields. In fact, field-mediated coupling, often called a cascade connection because of the one-way flow of information [3, 4], is equivalent to a direct (reversible) coupling between the system and auxiliary, with the addition of irreversible coupling of each to separate environments. As a result coherent control via cascade connections is a strict subset of coherent feedback that involves direct coupling as depicted in Fig. 1 (top).

Historical source dated 2014-01-27. Current resolution status has not been independently reassessed; bounds and claims in the source context refer to that date.