Systems Non-Proper Transfer Fns: Causality?

[Tasos51]

TL;DR Summary

A major result in control theory with no proof anywhere.

Prove that if a system's transfer function is not proper (order of numerator greater than order of denominator) then the system is not causal.

 

[Baluncore]

Summary:: A major result in control theory with no proof anywhere.

Prove that if a system's transfer function is not proper (order of numerator greater than order of denominator) then the system is not causal.

First you need to understand why it must be the case.
https://lewisgroup.uta.edu/ee4314/lectures/Lecture 3.pdf
“The effects of zeros and poles of a system combine and when the system has more finite
zeros than poles then the overall effect is an anticipative one, and the system is not
causal. When a system has more poles than finite zeros (i.e. the transfer function of the
system is strictly proper) then the system is causal. If the transfer function of a system
has relative degree equal to 0 then the system is causal and there is also instantaneous
transfer between input and output.”

 

[Tasos51]

First you need to understand why it must be the case.
https://lewisgroup.uta.edu/ee4314/lectures/Lecture 3.pdf
“The effects of zeros and poles of a system combine and when the system has more finite
zeros than poles then the overall effect is an anticipative one, and the system is not
causal. When a system has more poles than finite zeros (i.e. the transfer function of the
system is strictly proper) then the system is causal. If the transfer function of a system
has relative degree equal to 0 then the system is causal and there is also instantaneous
transfer between input and output.”

OK, but this is not a rigorous proof by any means. Further the statement "when the system has more finite zeros than poles then the overall effect is an anticipative one" needs proof.

Source https://www.physicsforums.com/threads/causal-systems.1006328/

 

[berkeman]

Source https://www.physicsforums.com/threads/causal-systems.1006328/

You did not reply to the last post in your thread from February. Were you able to look at the textbook recommended by @mpresic3 ?

I looked up in my control theory book Modern Control Theory, by William L Brogan, Third Edition. I have not explored the issue in depth but the discussion after 3.4.4. State Equations from Transfer Functions on page 88 to address your question directly. It gives the transfer function Y(s) / U(s) as the ratio of two polynomials. Please see the discussion. It is probably in the other texts too. what text are you using?

 

[anorlunda]

The discussions upthread make it sound like causal/anticipatory are definitions, not derived results.

 

[Tasos51]

You did not reply to the last post in your thread from February. Were you able to look at the textbook recommended by @mpresic3 ?

I cannot see something relevant on given reference. It would be better if the quotes are more accurate, this would save us precious time, not having to check 10 pages of text.

 

[Mark44]

OK, but this is not a rigorous proof by any means. Further the statement "when the system has more finite zeros than poles then the overall effect is an anticipative one" needs proof.

Not necessarily. I agree with @anorlunda's point that these terms seem to be definitions -- definitions don't need to be proved.

Source https://www.physicsforums.com/threads/causal-systems.1006328/

The link above is to this thread, so isn't at all helpful.

It would be better if the quotes are more accurate, this would save us precious time, not having to check 10 pages of text.

Maybe it would save you precious time in reading 10 pages, but at our expense.

 

[jasonRF]

I'm probably missing something important, as I never took control systems. But it isn't obvious to me that this is true. Here is a possible counterexample.

If $x(t)$ is the input and $y(t)$ is the output, then a differentiator $y = -a\, \frac{dx}{dt}$ will have system function $H(s) = -a\, s$. A circuit with an ideal op-amp, one resistor and one capacitor can realize this system (with $a = RC$), and the circuit would be causal. Of course in the real world such a circuit would have stability problems, but stability and causality are different things. Causality simply requires the impulse response to be zero for $t<0$, and the differentiator has impulse response $h(t) = -a\, \delta^\prime(t)$.

Is there a reason why this example doesn't apply?

jason

 

[DaveE]

I don't think $\delta(t)$ is causal. It's normally defined by the integral whose limits include t<0.

$$ \int_{-\infty}^\infty f(t)\delta(t) \, dt=f(0) $$

Or, consider it's Fourier transform, which must include t<0 components.

 

[DaveE]

Also, not really about causality (I think), but a true differentiator would require infinite gain at infinite frequency, which makes it not physically realisable.

 

[jasonRF]

Also, not really about causality (I think), but a true differentiator would require infinite gain at infinite frequency, which makes it not physically realisable.

Agreed. However, if a system with infinite gain at infinite frequencies is not allowed then the OP makes no sense at all. The matter to be proved (or disproved?) effectively is that systems (edit: with rational system functions) with infinite gain at infinite frequency are not causal.

jason

 

[jasonRF]

I don't think $\delta(t)$ is causal. It's normally defined by the integral whose limits include t<0.

$$ \int_{-\infty}^\infty f(t)\delta(t) \, dt=f(0) $$

Or, consider it's Fourier transform, which must include t<0 components.

The delta distribution and derivatives only have support at the origin, so in that sense are zero for $t < 0$ (as well as for $t>0$) so I had assumed they are causal. If they are not considered causal then then the what the OP wants to prove is pretty trivial. You can simply do synthetic division to make the system function look something like $H(s) = c_0\, s^p + c_1\, s^{p-1} + \cdots + c_{p-1}s + H_p(s)$ where $H_p(s)$ is proper. The impulse response (just the inverse transform) then has derivatives of delta functions so would be non-causal. If that is all there is to it, then I'm surprised there has been more than one thread on this. But to me it seems weird to call the differentiator non-causal. After all, the response to an input does not occur before the input. But if that is the convention then who am I to argue? Could you provide a reference where that convention is explained?

Note that if we want to have even the possibility of including non-causal systems, then the bilateral Laplace transform must be used which is defined by
$$ \mathcal{L} \left[ f(t)\right] = \int_{-\infty}^\infty f(t) \, e^{-st}$$
which does have the limits you want. The unilateral Laplace transform throws away the non-causal part of of a system, so isn't useful for trying to determine whether a system is causal.

edit: I think I misunderstand what you are getting at with your comment about the integral. Could you explain more? Note that the unilateral Laplace transform can be defined multiple ways. I learned the definition $\int_{0^-}^\infty f(t) \, e^{-st}$ (See Lathi's Linear Systems and Signals, or the second edition of Signals and Systems by Oppenheim et al), and the lower limit of $0^-$ means that the transform of the delta function is 1, and also allows the most natural initial values to be used. See http://web.mit.edu/2.737/www/handouts/LaplaceArticle.pdfjason

 

[jasonRF]

I'm pretty tired so am not thinking very quickly. But even if the derivatives of the delta function aren't considered causal (which would be sufficient for 'proving' the original statement), the delta function must be. The system that is simply an ideal wire ($y=x$) obviously has the impulse response of $\delta(t)$. Surely that is a causal, physically realizable system. Of course the system function is $H(s)=1$ so is not included in the subject of the thread.

 

[DaveE]

I'm pretty tired so am not thinking very quickly. But even if the derivatives of the delta function aren't considered causal (which would be sufficient for 'proving' the original statement), the delta function must be. The system that is simply an ideal wire ($y=x$) obviously has the impulse response of $\delta(t)$. Surely that is a causal, physically realizable system. Of course the system function is $H(s)=1$ so is not included in the subject of the thread.

Yes, that makes sense. I neglected that the impulse response is defined as the response to $\delta(t)$, so of course the output can also be non-causal.

I'm not as convinced as you about a theoretical differentiator being causal. What should it's output be at the peak of a triangle wave, where the derivative is discontinuous? Doesn't that discontinuous response require knowledge about the future value of the input? You wouldn't know you're at the peak until after it's occurred. Any delay in the response once it "saw" the peak would mean it has pole(s) in the transfer function. This is related to the problem of infinite gain required at infinite frequency.

 

[Tasos51]

I think looking at the differentiator is a correct approach. So, its impulse response is the unit doublet. Causality requires it to be 0 for t<0. Is it ?
Furtermore, causality does not make sense for signals, only systems.

 

[jasonRF]

I'm not as convinced as you about a theoretical differentiator being causal. What should it's output be at the peak of a triangle wave, where the derivative is discontinuous? Doesn't that discontinuous response require knowledge about the future value of the input? You wouldn't know you're at the peak until after it's occurred. Any delay in the response once it "saw" the peak would mean it has pole(s) in the transfer function. This is related to the problem of infinite gain required at infinite frequency.

The only thing I'm (edit: mostly) convinced of is that in an ideal world, a differentiator circuit built with ideal components (opamp, resistor, capacitor) would be causal. The fact that an ideal opamp is required doesn't bother me since virtually all of the systems we analyze with system theory are idealized. For example, V=IR doesn't actually apply for arbitrarily large currents through a resistor, and a resistor that stays linear regardless of the current through it is not physically realizable. But the mathematical models we build with the idealized components are of course very useful, even if all of them fail to be 'true' with arbitrary system inputs.

Otherwise I think I have left open the possibility that, from a system theory perspective, the differentiator might be considered non-causal. Indeed, if that is the case, then as I showed in post 12 it is pretty easy to prove the statement that is the subject of this thread.

jason

 

[jasonRF]

Furtermore, causality does not make sense for signals, only systems.

Agreed.

I think looking at the differentiator is a correct approach. So, its impulse response is the unit doublet. Causality requires it to be 0 for t<0. Is it ?

From the perspective of distribution theory (generalized functions) the support of the unit doublet is a single point - the origin. So from that perspective it can be thought of as zero for $t \neq 0$. But as someone who knows almost nothing about control theory, I don't claim to know whether systems with such impulse responses are somehow excluded from the class of causal systems. I wouldn't be shocked to learn that they are.

jason

 

[Tasos51]

The only thing I'm (edit: mostly) convinced of is that in an ideal world, a differentiator circuit built with ideal components (opamp, resistor, capacitor) would be causal. The fact that an ideal opamp is required doesn't bother me since virtually all of the systems we analyze with system theory are idealized. For example, V=IR doesn't actually apply for arbitrarily large currents through a resistor, and a resistor that stays linear regardless of the current through it is not physically realizable. But the mathematical models we build with the idealized components are of course very useful, even if all of them fail to be 'true' with arbitrary system inputs.

Otherwise I think I have left open the possibility that, from a system theory perspective, the differentiator might be considered non-causal. Indeed, if that is the case, then as I showed in post 12 it is pretty easy to prove the statement that is the subject of this thread.

jason

"The impulse response (just the inverse transform) then has derivatives of delta functions so would be non-causal": Well, this is not obvious to me. Am I missing something trivial ?

 

[jasonRF]

"The impulse response (just the inverse transform) then has derivatives of delta functions so would be non-causal": Well, this is not obvious to me. Am I missing something trivial ?

This quote is taken a little out of context. Here is a larger chunk of that post

The delta distribution and derivatives only have support at the origin, so in that sense are zero for $t < 0$ (as well as for $t>0$) so I had assumed they are causal. If they are not considered causal then then the what the OP wants to prove is pretty trivial. You can simply do synthetic division to make the system function look something like $H(s) = c_0\, s^p + c_1\, s^{p-1} + \cdots + c_{p-1}s + H_p(s)$ where $H_p(s)$ is proper. The impulse response (just the inverse transform) then has derivatives of delta functions so would be non-causal.

All I was saying is that a system with a transfer function that isn't proper will have derivatives of delta functions as part of the impulse response. So if systems whose impulse responses include such things are considered non-causal, then systems with an improper transfer function are non-causal.

EDIT: also, I was assuming you knew that the inverse Laplace transform of $s^n$ is proportional to $\delta^{(n)}(t)$.

 

[Tasos51]

No they are not considered non-causal. This is to be proved, this is the whole idea. Non-causal are systems whose impule response is NOT 0 for t<0.

 

[jasonRF]

No they are not considered non-causal. This is to be proved, this is the whole idea. Non-causal are systems whose impule response is NOT 0 for t<0.

I understand what you are saying, but when the impulse response includes generalized functions at the origin then it just isn’t obvious to me how it would be treated in the standard control theory literature. After all, the derivative of a delta is actually a functional, so maps functions to numbers. Technically, it doesn’t make sense to ask about the value at a given time. That is why in prior posts I mentioned the support instead of specifying times for which they are non-zero. So again, it is not obvious to me how folks doing control theory treat such things. If you already knew your original question was about derivatives of deltas then you should have phrased it that way; it would have saved everyone a lot of time.

Based on this statement:

Summary:: A major result in control theory with no proof anywhere.

you have looked in a lot of control theory texts searching for a proof. Do any of those texts discuss impulse responses that include the derivatives of deltas?
Jason

 

[Tasos51]

I was not sure this is the only approach (e.g. derivatives of deltas), and still am not. If this is the case, it looks quite complicated.
Some texts do refer to impulse responses that include derivatives of deltas (e.g. Willsky, Signals & Systems), but I would not say "discuss".
Maybe, because it goes beyond the scope of an engineering curiculum.

 

[jasonRF]

At work today I took a look at my copy of Fourier Analysis and Applications by Gasquet and Witimski. I had never read the chapter on Distributions and Filters before. It basically says that a system is causal if and only if the support of $h$ (allowed to be a generalized function here) is in $[0, \infty)$, so the differentiator would be causal. It is unstable, though, in the sense that a bounded input can yield an unbounded output.

During lunch I also wandered to the library and found that not all authors agree. Brogan (modern control systems 3rd edition) claims without proof that a proper transfer function is required for causality. Kudeki and Munson (analog signals and systems) essentially claim that an impulse response of $\delta^\prime(t)$ is causal since the output doesn’t preceed the input. I found another text (forgot to note author/title) that claimed (edit: without proof) a proper transfer function was required for a stable causal system.jason

 

[Tasos51]

(Almost) every textbook on Control Systems contains this assertion-theorem. That's why PID is non-causal. Stability is another matter. And note that we are talking about Linear, Time Invariant (LTI) systems, initially at rest (in order to get the transfer function). In books for distributions, differential equations, signals etc., this may not be the case, when contradicting assertions, like the ones you mention, are made. In any case, I am glad that my question seems to be a valid one.

 

[jasonRF]

While the different texts did have different contexts, it seemed like they used equivalent definitions of causality. Of the few texts I looked at, the only texts that would classify the differentiator as acausal offered no proof or discussion. Since differentiation can be computed using the limit of a backward difference it requires only present and past values; classifying it as acausal requires some explanation I think.

I agree that the question is valid, and I find it tremendously unsatisfying that standard texts assert this without proof - or even a reference to a proof based on the few books I glanced at today. Perhaps this is unfair, but I suspect the various authors are copying this result from the books they learned from and not really scrutinizing it. They may gloss over it because whether improper systems such as differentiators are classified as causal or acausal is purely academic. Improper systems are impractical so are not worth spending time on.

As for me, unless I see a proof to convince me otherwise, I will not accept the assertion that causality requires a proper transfer function. I do accept that systems with improper transfer functions are not realizable.

jason

 

[Tasos51]

Well actually, I started looking at this while writing a textbook on Control (in Greek). Since I did not want to rely on past books, I started searching for a proof which I could not find anywhere. I agree it is of academic importance, but we academics are eager to get to the bottom of things, aren't we ?
Mind you, I have looked all over the internet, let alone textbooks, with no luck.

 

[jasonRF]

I commend your efforts to find the proof. The fact that your long search for a proof in the literature has yielded nothing further increases my skepticism that it is true.

I wish you luck!

jason

 

[DaveE]

Since differentiation can be computed using the limit of a backward difference it requires only present and past values

How does that work at the peak of a triangle wave? Where $$ \lim_{t \rightarrow 0^+} {tri'(t)} \neq \lim_{t \rightarrow 0^-} {tri'(t)}$$

edit:
The output of a differentiator is discontinuous at $t=0$, in this case. The network must switch it's output exactly at $t=0$ without knowledge of the input at $t=\epsilon$ for any $\epsilon>0$. This is not causal.

 

[Tasos51]

It doesn't. The derivative is not defined at the peak.

 

[DaveE]

It doesn't. The derivative is not defined at the peak.

No, it's not but the network still has to change it's output then, Maybe this is the basis of a proof. The input is a continuous function with discontinuous derivatives, but the output is not a continuous function? IDK. The network and the function is well defined The network is LTI and the function is continuous, but the output isn't at $t=0$.

Anyway, let's play a game. You take the role of the network with $H(s)=s$. You ask me at any series of increasing times $t$ what the value of the input is. When I answer, you tell me what your output is at that instant. You do not get any information about future input values when you make your choice.

Plus, I'll tell you everything in advance about my function, just to make the problem clear. Except I'll omit one key parameter, $t_0$. My function is
$$
u(t) =
\begin{cases}
0 & \text{if } t < t_0 \\
(t-t_0) & \text{if } t \geq t_0
\end{cases}
$$
Your output will switch from 0 to 1 at $t=t_0$ (perhaps you should answer "undefined" at $t=t_0$).

I claim you will never be able to answer "undefined". Even if you are lucky enough to ask at $t=t_0$, you won't respond "undefined". You need the information at $t=t_0+\epsilon$ ($\epsilon>0$). When you answer "1" at $t=t_0+\epsilon$, you are too late. You will always be too late by $\Delta t=\epsilon ~ \forall ~ \epsilon>0$.

edit: you can tell I'm not a mathematician by my sloppy use of "well defined", LOL.

 

[eq1]

edit: you can tell I'm not a mathematician by my sloppy use of "well defined", LOL.

But that's not how the definition of a derivative works. The derivative definition uses the limit so one doesn't get to cheat and use only the left or right side for that. To show the derivative is undefined I just need to demonstrate there is a number which breaks the bound at t0. This will be easy to do because at t0 the left side limit is zero and the right side limit is 1.

I was thinking this might be the answer to JasonRF's dilemma too. The derivative might not exist for an improper transfer function when the system is non causal as the frequency approaches infinity (or zero). Like how the limit of 1/x doesn't exist as x->0 but I couldn't think of an easy proof off the top of my head for that and I'm too lazy to work through all the definitions. So it's just a hunch. :)

 

[Tasos51]

I think you are confusing causality with differentiability or realizability. I am not sure how they are connected.

 

[jasonRF]

If we are talking pure math using classical analysis, then the derivative of your input should be a step that does not exist at $t=t_0$, and the second derivative would not exist at $t=t_0$ and be zero everywhere else.

Of course a strictly one-sided limit will never output "does not exist". But I have never seen an author define LTI systems as mappings from $R$, to $\{R,$ does not exist $\}$ before. Have you? A system has a domain and a range - if we are using classical analysis then the logical domain of the differentiator would be a space of differentiable functions. If we allow generalized functions, as EEs very often do (usually without explicitly mentioning it), then there is no problem. The first derivative of your input is a well-defined step and the second derivative is $\delta(t-t_0)$.

This discussion has made me go back and look at the more precise definitions of causality that the more mathematical treatments of linear system theory often use, since the 'output doesn't depend on future inputs' notion provided in introductory treatments leaves a lot of room for interpretation. Let $x_0(t)$ and $x_1(t)$ be two inputs with corresponding outputs $y_0(t)$ and $y_1(t)$. The definition I usually see is that a system is causal if $x_0(t)=x_1(t)$ for $a\leq t < b$ implies $y_0(t)=y_1(t)$ for $a\leq t < b$. Under this definition the differentiator is causal, even if we use classical analysis and feed it your input. No need to try and use a one-sided limit to appeal to an imprecise definition of causality.

Edit: this is wrong. When $t_0=a$ then the differentiator would be acausal for your function under this definition, assuming we use classical analysis and it makes any sense to force systems to do operations that don’t exist. The actual definitions I saw only had the $t<b$ part, and I added the $a\leq$ to make it the same form as the definition in the next paragraph. Oops! I know of one book though, state-space and input-output linear systems by Delchamps, that uses a slightly different definition. He uses more complicated notation to allow for time-varying systems and such, but with the prior notation his definition basically is that a system is causal if $x_0(t)=x_1(t)$ for $a\leq t < b$ implies $y_0(t)=y_1(t)$ for $a\leq t \leq b$. In his book the definition of any system must also include a specification of the vector space of input functions. So a differentiator that is defined with differentiable input functions would therefore be causal, and a differentiator defined with inputs that are not differentiable everywhere (and I'm not certain this would even be allowed) would be acausal.

So if you want to feed a differentiator non-differentiable functions and demand a 'does not exist' output, then the differentiator could be acausal depending on your definition of the system, the definition of causality and whether or not you are happy to use generalized functions. Your function creates problems only at one single point, but mathematicians can construct functions that are continuous everywhere and differentiable nowhere. If we also allow those inputs then the differentiator can have inputs that create outputs that simply don't exist anywhere. Does this make it an illegitimate system to begin with? If so, then a nice proper system defined with a standard Riemann integral, $y(t)=\int_0^t x(p)\, dp$ also has problems, since those same mathematicians can construct bounded functions that are not Riemann integrable...

jason

 

[jasonRF]

One more thing. If $x_1(t)$ is a function differential everywhere with derivative $y_1(t)$, then if we use classical analysis and feed a differentiator your function plus $x_1(t)$ the output is simply 'does not exist' at $t=t_0$, regardless of $y_1(t_0)$. Does this violate linearity? Since we are asking the system to do things that are undefined, perhaps it doesn't matter?

 

[DaveE]

One more thing. If $x_1(t)$ is a function differential everywhere with derivative $y_1(t)$, then if we use classical analysis and feed a differentiator your function plus $x_1(t)$ the output is simply 'does not exist' at $t=t_0$, regardless of $y_1(t_0)$. Does this violate linearity? Since we are asking the system to do things that are undefined, perhaps it doesn't matter?

The way I learned linearity, you would have to have valid functions f(a), f(b), and f(a+b) before you could evaluate f(a+b)=f(a)+f(b). So I would say the concept of linearity is not applicable at $t_0$. But as you say, it all comes down to the definitions you like.