Tuning via Measurements of the Squared Error

Felipe M Pait^{∗}

pait@lac.usp.br Universidade de São Paulo Laboratório de Automação e Controle – pee São Paulo SP 05508–900 Brasil

7 May 1998

Abstract

Tuning when the estimation error is known in magnitude only. Key Words: Adaptive Control; Parameter Estimation; Direct Control.

1 Introduction

Given data x : [0, t̄) → ℝ^{n}, we wish to adjust a parameter vector p(t) so as to minimize

z(t) = x^{⊤}p − y  (1)

as best we can in some norm sense. If y : [0, t̄) → ℝ (or equivalently z) were available, we might choose as our cost function the integral of z^{2} and minimize it using standard leastsquares algorithms. We consider the case when neither y nor z are available; rather, at each instant we are able to choose p(t) and measure z^{2}(t), that is to say, z’s magnitude but not its sign. The garden-variety least-squares algorithm depends on explicit knowledge of y, and its recursive version, as other recursive gradient-type methods, relies on z(t) to specify the direction of the adjustment. In this note we propose a recursive algorithm to tune estimates of p(t). One application we have in mind is direct adaptive control [1].

Running the risk of boring the reader, we shall briefly review some standard leastsquares parameter estimation concepts. With M symmetric, positive-semidefinite, define

J(t) = p̂^{⊤}Mp̂ + ∫_{0}^{t} dτ (x^{⊤}p̂ − y)^{2}.

J(t) represents a cost to be minimized at a given instant t by choosing an appropriate p̂, which must satisfy

\frac{∂J}{∂p̂}(t) = 2p̂^{⊤}M + 2 ∫_{0}^{t} dτ (x^{⊤}p̂ − y)x^{⊤} = 0,

∗Research supported by cnpq – Brazilian Research Council, grant 523949/95-2.

so that

p̂ = (M + ∫_{0}^{t} dτ xx^{⊤})^{−1} ∫_{0}^{t} dτ xy  (2)

is the solution to the least-squares estimation problem. If M > 0 this solution is unique, and because the Hessian

\frac{∂^{2}J}{∂p̂∂p̂^{⊤}} (t) = M + ∫_{0}^{t} dτ xx^{⊤} > 0,

J(t) is a convex function of p̂ and the solution is a global minimum.

Equation (2) can be used to construct a recursive solution p̂ to the least-squares problem. However, the important thing is that the solution to the least-squares problem exists and is unique. Once this condition is satisfied, a number of techniques can be used. If y were available, perhaps the simplest way of constructing a recursive estimate p̂ of p_{∗} would be to choose the tuning error ẑ = x^{⊤}p̂ − y and set

˙p̂ = −xẑ.

In what follows we assume that y is a noisy measurement of the scalar product of data x with an unknown parameter vector p_{∗}, namely, y = x^{⊤}p_{∗} + w. Then x^{⊤}(p̂ − p_{∗}) = ẑ + w, so the derivative of

V = \frac{1}{2}(p̂ − p_{∗})^{⊤}(p̂ − p_{∗}),

the squared parameter error, is

V̇ = (p̂ − p_{∗})^{⊤} ˙p̂ = −(p̂ − p_{∗})^{⊤}xẑ
= −ẑ^{2} − ẑw.

Using the inequality |ẑw| ≤ \frac{1}{2}ẑ^{2} + \frac{1}{2}w^{2} and integrating results in

V (t) ≤ V (0) − \frac{1}{2} ∫_{0}^{t} ẑ^{2} + \frac{1}{2} ∫_{0}^{t} w^{2}.

Because V ≥ 0, this shows that the energy of the signal ẑ is limited by that of w plus a finite constant; in particular if w ∈ L^{2} then ẑ ∈ L^{2} as well, and p̂ is bounded. If we now choose the adjustable parameters p equal to p_{∗}’s estimate p̂, which of course we have the freedom to do, then ẑ = z and

z = x^{⊤}p − y ∈ L^{2} and p ∈ L^{∞}.

If p are parameters of a direct adaptive feedback controller, then p = p̂ can be considered an expression of the certainty-equivalence principle. The distinction between the adjustable parameters p and the parameter estimate p̂ may appear recherché, but it plays a crucial role in the solution of the more complex problem treated in §2 of this paper.

2 The Tuner

The method just described requires that we set the tuning error equal to z, that is to say, z must be known in sign as well as magnitude. If only the latter were known, we might choose as our tuning error

e_{T} = (p̂ − p)^{⊤}xx^{⊤}(p̂ − p) − z^{2}.  (3)

Intuition motivates the definition of e_{T} above: if the sign of z is unknown, then at each instant we might equally believe that y = x^{⊤}p + |z| or that y = x^{⊤}p − |z|. Unable to minimize x^{⊤}p̂ − y as before, we content ourselves with trying to keep the product (x^{⊤}p̂ − x^{⊤}p − |z|)(x^{⊤}p̂ − x^{⊤}p + |z|) = e_{T} small. Notice that e_{T} is a function of p̂ as well as of time.

Now consider the cost function

J(t) = \frac{1}{4}(p̂^{⊤}Mp̂)^{2} + \frac{1}{4} ∫_{0}^{t} dτ e_{T} (t, τ)^{2}.

The p̂ that minimizes J must satisfy

\frac{∂J}{∂p̂}(t) = p̂^{⊤}Mp̂p̂^{⊤}M + ∫_{0}^{t} dτ e_{T} (p̂, τ)x^{⊤}(p̂ − p(τ))x^{⊤} = 0.

A sufficient condition for this solution to be the unique minimum is that the Hessian

\frac{∂^{2}J}{∂p̂∂p̂^{⊤}} (t) = p̂^{⊤}Mp̂M + 2Mp̂p̂^{⊤}M + ∫_{0}^{t} dτ (2(x^{⊤}(τ)(p̂ − p(τ))^{2} + e_{T} ) xx^{⊤}(τ) > 0.

Given an arbitrary vector v_{i}, let

h_{i} = v_{i}^{⊤}Mv_{i}p̂^{⊤}Mp̂ + 2(v_{i}^{⊤}Mp̂)^{2} + ∫_{0}^{t} dτ ξ_{i}(τ) (2(x^{⊤}(τ)(p̂ − p(τ))^{2} + e_{T} )

where ξ_{i} = (v_{i}^{⊤}x)^{2}. The minimum of h_{i} occurs when

\frac{∂h_{i}}{∂p̂^{⊤}} (t) = 2v_{i}^{⊤}Mv_{i}Mp̂ + 4v_{i}^{⊤}Mp̂Mv_{i} + 6 ∫_{0}^{t} dτ ξ_{i}(τ)xx^{⊤}(p̂ − p) = 0,

namely, when

p̂ = (2v_{i}^{⊤}Mv_{i}M + 4v_{i}^{⊤}MMv_{i} ∫_{0}^{t} +dτ ξ_{i}xx^{⊤})^{−1} ∫_{0}^{t} dτ ξ_{i}xx^{⊤}p := p̄,

so

h_{i_{min}} = 3 ∫ ξ_{i}((x^{⊤}p)^{2} − z^{2}) − 3 (∫ ξp^{⊤}xx^{⊤}) (∫ ξxx^{⊤})^{−1} (∫ ξp^{⊤}xx^{⊤}) .

We are, of course, assuming that ∫ ξxx^{⊤} is invertible. Let p̄ = (∫ ξxx^{⊤})^{−1} (∫ ξp^{⊤}xx^{⊤}) . The patient reader will not complain overmuch if we continue the orgy of derivatives and compute

ḣ_{i_{min}} = 3ξ (x^{⊤}(p − p̄))^{2} − z^{2}.

We are concerned with the magnitude of the output z̄ of the system

\frac{d}{dt}(p − p̄) = − (∫ ξxx^{⊤})^{−1} ξxx^{⊤}(p − p̄) + ṗ
z̄ = x^{⊤}(p − p̄).

As long as |z̄|^{2} > \frac{1}{3}|z|^{2}, ḣ_{i_{min}} remains positive, the minimum value attained by the Hessian is increased, and J stays convex! This needs to happen for all directions v_{i}, of course.

We are finally in a position to justify the fastidious distinction between p and p_{∗}’s estimate p̂. For if p̂(t) were the output of some successful tuner, ˙ p̂ would become small. Thus if we were to set p(t) = p̂(t), the tuner’s success would renderz̄ small and thus possibly undermine convexity — the basis on which its success rests.

References

[1] Felipe M Pait. On the Design of Direct Adaptive Controllers. Submitted.