Orthogonality is the Key to Understand Fourier Series Expansion

Video Available

---

I would say that "orthogonality" is the key to understand Fourier series expansion.

In engineering and science, we deal with waveforms. A waveform may represent a characteristic of a phenomenon with respect to a certain variable, or variation of a quantity with the evolution of a certain variable for instance time.

In order to apply a mathematical method to a given waveform, the waveform must be represented by a simple mathematical expression. If a given waveform is represented by a complicated equation or just by numerical data, then applying a certain mathematical method would be quite complicated.

---

For representing a given waveform by a simple mathematical expression, note that there are basically three different categories. If you are interested in local behavior of a waveform, use Taylor series. If a given waveform is periodic and you want to capture the entire behavior of the waveform, then use Fourier series. If a given waveform is single-shot and you want to capture the entire behavior of the waveform, then use Fourier or Laplace transform.

So, when you want to capture the entire behavior of a periodic waveform, you should use Fourier series.

---

Fourier series expansion tries to approximate a periodic waveform by a sum of sinusoidal waves. Each sinusoidal wave must have a period whose multiple is the period of the given waveform. In other words, the frequency of each sinusoidal wave must be an exact multiple of the frequency of the given waveform's period.

I presume that you are familiar with describing a sinusoidal wave by an exponential function with an imaginary argument. If not, watch the video or read the text entitled "Oscillation Kernel."

---

Since Fourier series is often applied to time-periodic waveforms, let's consider a time periodic function $f(t)$ where $t$ is time. When the period of $f(t)$ is $T$, then its frequency is $f=1/T$ and angular frequency is $\omega = 2\pi/T$. Fourier series expansion approximates $f(t)$ by a sum of sinusoidal waves.

$$f(t)=\sum_{n=-\infty}^{\infty} c_n \mathrm{e}^{in\omega t} \nonumber$$

where $c_n$ for $n = \cdots, -2, -1, 0, 1, 2, \cdots$ are called Fourier series coefficients. Note that the angular frequency of each sinusoidal wave $\mathrm{e}^{in\omega t}$ is $n\omega$ which is an exact multiple of the given waveform's angular frequency $\omega$.

---

Why are they exact multiples? It can be clearly explained by the following counterexample. What if a sinusoidal wave whose frequency is not exactly a multiple of the given waveform's frequency is included in the series? Then, that component, included in the summation, will collapse the periodicity of the given waveform! So, each sinusoidal component must have a frequency which is an exact multiple of the given waveform's frequency.

Fourier series expansion says that a periodic waveform can be approximated by a sum of sinusoidal waves having frequencies which are exact multiples of the frequency of the given periodic waveform. Those sinusoidal waves are often called harmonics.

---

From now on, we call the complex sinusoidal wave $\mathrm{e}^{i\omega t}$ as an "oscillation kernel" in this text.

Before talking about the main topic of this text which is orthogonality, let's check the following two interesting points.

---

The Fourier series expansion formula adds up not only for positive frequencies but also for negative frequencies, because the summation starts from negative infinity. What is a negative frequency?

In our real world, causality is always satisfied. Causality means that a physical or engineering system does not give an output for an input before the input is received. Under this situation, an oscillation kernel with its coefficient, $c_n \mathrm{e}^{in\omega t}$, which exists as a part of a waveform in this real world, is always accompanied by its complex conjugate counterpart

$$( c_n \mathrm{e}^{in\omega t} )^* = c_n^* \mathrm{e}^{-in\omega t} \nonumber$$

Look at this equation! The accompanied complex-conjugate wave is actually its negative frequency component. If these two components are superimposed, then applying Euler's formula gives

$$c_n \mathrm{e}^{in\omega t} + c_n^* \mathrm{e}^{-in\omega t} = c_n( \cos n\omega t + i \sin n\omega t) + c_n^*( \cos n\omega t - i \sin n\omega t) \nonumber$$

$$= 2 \mathrm{Re} \{c_n\} \cos n\omega t - 2 \mathrm{Im} \{c_n\} \sin n\omega t \nonumber$$

$$= 2 \vert c_n \vert \sin ( n\omega t + \theta_n ), \quad \theta_n = -\arg( c_n ) \nonumber$$

Amazingly, the result is not a complex number but a real number! Causality, which is always satisfied in this real world, forces the complex-conjugate constraint mentioned above, thus always canceling out the imaginary part and leading to the fact that waveforms observed in this real world are all real.

The zeroth order coefficient $c_0$ is an exception. It is always real and does not have a complex-conjugate counterpart. For electrical engineers, it is a dc (direct current) component.

If you are further interested in this topic, watch the videos or the texts entitled "Oscillation Kernel" and "Understanding Euler's Formula through its Derivation."

---

Another point to note at this point is the fact that the Fourier series coefficients are complex-valued except the zeroth-order coefficient. If a coefficient $c_n$ where $n \ne 0$ is written as the polar form

$$c_n = r_n \mathrm{e}^{i\theta_n} \nonumber$$

then the $n$th-order oscillation kernel with its coefficient can be written as

$$c_n \mathrm{e}^{in\omega t} = r_n \mathrm{e}^{i\theta_n} \mathrm{e}^{in\omega t} = r_n \mathrm{e}^{i( n\omega t +\theta_n )} \nonumber$$

So, the coefficient $c_n$ specifies not only the amplitude $r_n$ of the oscillation kernel but also the phase angle $\theta_n$. The use of a complex-valued coefficient $c_n$ makes it possible to specify both the amplitude and the phase angle by a single number.

---

Ok, let's talk about the main topic, orthogonality.

If we are talking about vectors, orthogonality can be interpreted as perpendicularity. If the angle formed by two vectors is 90 degrees, they are orthogonal. In other words, if the inner product of two vectors are zero, they are orthogonal.

This concept can also be applied to two periodic functions. Let's define the inner product of two periodic functions $p(t)$ and $q(t)$. $p(t)$ is multiplied by the complex conjugate of $q(t)$, and the product is integrated for a a common multiple of the period of $p(t)$ and that of $q(t)$.

$$\left< \, p(t), q(t) \, \right> = \frac{1}{T} \int_{-\frac{T}{2}}^{\frac{T}{2}}{p(t)\,q^*(t) dt} \nonumber$$

where $T$ is the common multiple of the periods mentioned above.

In Fourier series, we use oscillation kernels $\mathrm{e}^{in\omega t}$ for the expansion. So, let's calculate the inner product of $p(t) = \mathrm{e}^{i m\omega t}$ and $q(t) = \mathrm{e}^{i n\omega t}$.

$$\left< \, \mathrm{e}^{i m\omega t}, \mathrm{e}^{i n\omega t} \, \right> = \frac{1}{T} \int_{-\frac{T}{2}}^{\frac{T}{2}}{\mathrm{e}^{i m\omega t} \mathrm{e}^{-i n\omega t} dt} = \frac{1}{T} \int_{-\frac{T}{2}}^{\frac{T}{2}}{\mathrm{e}^{i( m - n )\omega t} dt} \nonumber$$

The period of $\mathrm{e}^{i m\omega t}$ is $T_p = 1/(mf) = 2\pi/(m\omega)$, and that of $\mathrm{e}^{i n\omega t}$ is $T_q = 1/(nf) = 2\pi/(n\omega)$. So, we can use $T = 1/f = 2\pi/\omega$ as a common multiple.

In the case of $n = m$, the integrand becomes one. Thus, the inner product is $1/T$ multiplied by $T$ which is unity. This implies that $\mathrm{e}^{i m \omega t}$ and $\mathrm{e}^{i n \omega t}$ are in the same direction and thus not orthogonal to each other, of course because they are the same functions.

In the case of $n \ne m$, on the other hand, the integrand is $\mathrm{e}^{ik\omega t}$ where $k = m - n$ is a nonzero integer. Since $T$ is a multiple of the period of $\mathrm{e}^{ik\omega t}$, integrating $\mathrm{e}^{ik\omega t}$ for the period $T$ is zero. Thus, the inner product is zero, in other words, $\mathrm{e}^{i m\omega t}$ and $\mathrm{e}^{i n\omega t}$ are perpendicular to and thus orthogonal to each other.

Oscillation kernels for different orders are orthogonal to each other, and they form a set of orthogonal bases.

---

Let's look at the Fourier series expansion formula again.

$$f(t)=\sum_{n=-\infty}^{\infty} c_n \mathrm{e}^{in\omega t} \nonumber$$

Fourier series expansion represents a given periodic waveform by the sum of the orthogonal bases $\mathrm{e}^{i n\omega t}$ for $n = \cdots, -2, -1, 0, 1, 2, \cdots$.

Especially when we use $N$ positive frequencies, their negative counterparts and the zeroth-order term only to approximate a given periodic waveform $f(t)$, we may use

$$f(t) \simeq \sum_{n=-N}^N c_n \mathrm{e}^{in\omega t} \nonumber$$

This Fourier expansion tries to approximate $f(t)$ by the sum of $2N+1$ orthogonal bases.

---

What is the inner product of two vectors? It is a projection of one vector to the other. So, what does this mean for two functions?

$$\left< \, f(t), \mathrm{e}^{i n\omega t} \, \right> = \frac{1}{T} \int_{-\frac{T}{2}}^{\frac{T}{2}}{f(t) \mathrm{e}^{-i n\omega t} dt} \nonumber$$

This is the projection of a periodic waveform $f(t)$ to the basis $\mathrm{e}^{i n\omega t}$. Well, this means that we are calculating how much $\mathrm{e}^{i n\omega t}$ component is included in $f(t)$. This is actually the Fourier series coefficient $c_n$. So, we may be able to calculate $c_n$ by

$$c_n = \frac{1}{T} \int_{-\frac{T}{2}}^{\frac{T}{2}}{f(t) \mathrm{e}^{-i n\omega t} dt} \nonumber$$

Let's call this an "extraction equation" in this text. But, is this equation true? Ok, let's assume that a given periodic waveform $f(t)$ consists of the $n$th-order component with its coefficient, $a\,\mathrm{e}^{i n\omega t}$, and the remaining part $g(t)$.

$$f(t) = a\,\mathrm{e}^{i n\omega t} + g(t) \nonumber$$

Substituting this into the extraction equation to get $c_n$ gives

$$c_n = \frac{1}{T} \int_{-\frac{T}{2}}^{\frac{T}{2}}{\left\{a\, \mathrm{e}^{i n\omega t}+g(t)\right\} \mathrm{e}^{-i n\omega t} dt} \nonumber$$

$$= \frac{a}{T} \int_{-\frac{T}{2}}^{\frac{T}{2}}{\mathrm{e}^{i n\omega t} \mathrm{e}^{-i n\omega t} dt} + \frac{1}{T} \int_{-\frac{T}{2}}^{\frac{T}{2}}{g(t) \mathrm{e}^{-i n\omega t} dt} \nonumber$$

According to orthogonality, the first term is $a$. The second term is zero, since $g(t)$ does not include $\mathrm{e}^{i n\omega t}$ component. So,

$$c_n = a \nonumber$$

We are able to extract $a$ from a given waveform using the extraction equation.

---

So, Fourier series expansion gives us a way to represent a given periodic waveform $f(t)$ by a sum of oscillation kernels

$$f(t)=\sum_{n=-\infty}^{\infty} c_n \mathrm{e}^{in\omega t} \nonumber$$

Although it looks complex-valued, the imaginary parts are cancelled out due to causality. So, it is real-valued. You can calculate the values of the coefficients $c_n$ which are the magnitudes of the oscillation kernels including their phase-angle information using

$$c_n = \frac{1}{T} \int_{-\frac{T}{2}}^{\frac{T}{2}}{f(t) \mathrm{e}^{-i n\omega t} dt} \nonumber$$

In practical applications, we use only up to the $N$th term. $N$ positive frequencies, their negative counterparts and the zeroth-order term, altogether $2N+1$ terms.

---

In other words, Fourier series expansion gives us a way to decompose a periodic function into weighted oscillation kernels. And the weightings which are complex-valued so as to include phase-angle information are the Fourier coefficients and can be obtained as the inner product with respect to oscillation kernels.

The oscillation kernels form a set of orthogonal bases. So, the inner product of different-order oscillation kernels are zero. Calculating a Fourier coefficient is equivalent to calculating the projection of the given periodic waveform to that oscillation kernel using the inner product formula, or the extraction equation in this text, thus extracting that oscillation kernel's weighting. The resultant Fourier coefficient is the weighting of the oscillation kernel.

When we use up to the $N$th term, we are representing a given periodic waveform by the sum of $2N+1$ weighted oscillation kernels. Since the $2N+1$ oscillation kernels form a set of orthogonal bases, they are like unit vectors in a $(2N+1)$-dimensional space. And their weightings, in other words, Fourier coefficients are the lengths of those unit vectors. So, Fourier series expansion can be viewed as a way to represent a periodic waveform by a sum of weighted vectors in different or independent directions.

---

Finally, let's apply Fourier series expansion to a periodic square waveform.

This figure shows a periodic square waveform whose period is $T$.

So, its angular frequency is

$$\omega = 2\pi f = \frac{2 \pi}{T} \nonumber$$

The Fourier coefficients of this waveform are calculated by the extraction equation.

$$c_n = \frac{1}{T} \int_{-\frac{T}{2}}^{\frac{T}{2}}{f(t) \mathrm{e}^{-i n\omega t} dt} = \frac{1}{T} \left\{ \int_{-\frac{T}{2}}^{0}{0 \times \mathrm{e}^{-i n\omega t} dt} + \int_{0}^{\frac{T}{2}}{1 \times \mathrm{e}^{-i n\omega t} dt} \right\} \nonumber$$

$$= \frac{1}{T} \int_{0}^{\frac{T}{2}}{\mathrm{e}^{-i n\omega t} dt} = -\frac{1}{i n \omega T} \left[ \mathrm{e}^{-i n \omega t} \right]_{0}^{\frac{T}{2}} \nonumber$$

Here, let's suspend this calculation and simplify $\omega T$

$$\omega T = \frac{2\pi}{T}T=2\pi \nonumber$$

Ok, let's resume the calculation of $c_n$.

$$c_n = -\frac{1}{i 2\pi n}\left( \mathrm{e}^{-i n \pi} - 1 \right) = \frac{i}{2\pi n}\left\{ (-1)^n - 1 \right\} \nonumber$$

The equation above is not applicable to $n=0$, since there is $n$ in the denominator. To get $c_0$, we calculate from scratch.

$$c_0 = \frac{1}{T} \int_{-\frac{T}{2}}^{\frac{T}{2}}{f(t) \mathrm{e}^0 dt} = \frac{1}{T} \left\{ \int_{-\frac{T}{2}}^{0}{0 \times 1 dt} + \int_{0}^{\frac{T}{2}}{1 \times 1 dt} \right\} \nonumber$$

$$= \frac{1}{T} \int_{0}^{\frac{T}{2}}{ 1 dt} = \frac{1}{2} \nonumber$$

So, we get the Fourier coefficients of the periodic square wave.

$$c_n = \left\{ \begin{array}{cl} \dfrac{1}{2} & (n = 0) \\ -\dfrac{i}{\pi n} & (n \, \mbox{: odd number}) \\ 0 & (n \, \mbox{: even number}) \end{array} \right. \nonumber$$

It is interesting to superimpose

$$f(t) \simeq \sum_{n=-N}^N c_n \mathrm{e}^{in\omega t} \nonumber$$

on the given waveform.

When $N=0$, so only with the constant term, we get

When $N=1$ which means we include $n = 0$ and $n = \pm 1$ only, we get

When $N=3$,

When $N=5$,

If we include up to the 19th order, then we get

As we increase the number of terms included, the Fourier series expansion better approximates the given waveform.

---

I hope you have reached real understanding of Fourier series expansion. Thank you.