Numerical Derivative

• The Goal:

  Given a set of $N$ sample points $\{ (x_i, y_i),\;\;\;\vert\;\;\;(i=1,\cdots,N \}$ of an unknown continuous function $y=f(x)$, we want to estimate the derivative $dy(x)/dx$ of the function at an arbitrary point $x$. In general, the sample points are not assumed to be evenly spaced in the dimension of $x$.

• Lagrange interpolation:

  We first reconstruct a continuous function to fit a set of n+1 sample points around the point $x$ for which the derivative $f'(x)$ needs to be estimated by Lagrange interpolation:
  

  $$P_n(x)=\sum_{i=-n/2}^{n/2} \prod_{j=-n/2 (\neq i)}^{n/2} \frac{ (x-x_j)}{(x_i-x_j)} y_i$$
  
  
  $P_n(x)$ is a polynomial function of $x$ of order $n$ which passes through all n+1 sample points, i.e., $P_n(x_i)=y_i,\;\;\;(i=-n/2,\cdots,n/2)$. For example, when $n=2$, we have
  
  $$P_2(x)=\frac{(x-x_0)(x-x_1)}{(x_{-1}-x_0)(x_{-1}-x_1)}y_{-1}+... ...-x_1)}y_0+ \frac{(x-x_{-1})(x-x_0)}{(x_1-x_{-1})(x_1-x_0)}y_1$$
  
  
  When $n=4$, we have
  

  $$P_4(x)$$

  $$= \frac{(x-x_{-1})(x-x_0)(x-x_1)(x-x_2)}{(x_{-2}-x_{-1})(x_{-2}-x_0... ...-x_0)(x-x_1)(x-x_2)}{(x_{-1}-x_{-2})(x_{-1}-x_0)(x_{-1}-x_1)(x_{-1}-x_2)}y_{-1}$$

  $$+ \frac{(x-x_{-2})(x-x_{-1})(x-x_1)(x-x_2)}{(x_0-x_{-2})(x_0-x_{-1})(x_0-x_1)(x_0-x_2)}y_0$$

  $$+ \frac{(x-x_{-2})(x-x_{-1})(x-x_0)(x-x_2)}{(x_1-x_{-2})(x_1-x_{-1}... ...x_{-2})(x-x_{-1})(x-x_0)(x-x_1)}{(x_2-x_{-2})(x_2-x_{-1})(x_2-x_0)(x_2-x_1)}y_2$$

  
  

• Derivative estimation:

  Now the derivative of function $f(x)$ at any point $x$ can be estimated by differentiating $P_n(x)$. For example, when $n=2$:
  

  $$P'_2(x)=\frac{2x-x_0-x_1}{(x_{-1}-x_0)(x_{-1}-x_1)}y_{-1}+ \... ...x_0-x_1)}y_0+ \frac{2x-x_{-1}-x_0}{(x_1-x_{-1})(x_1-x_0)}y_1$$
  
  
  In particular, when the point $x=x_0$ is the middle point of the set of $n+1$ points, we have
  
  $$P'_2(x_0)=P'_2(x)\vert _{x=x_0}=\frac{x_0-x_1}{(x_{-1}-x_0)(x... ...1})(x_0-x_1)}y_0+ \frac{x_0-x_{-1}}{(x_1-x_{-1})(x_1-x_0)}y_1$$
  
  
  When $n=4$, we have
  

  $$P'_4(x)$$

  $$= \frac{y_{-2}}{(x_{-2}-x_{-1})(x_{-2}-x_0)(x_{-2}-x_1)(x_{-2}-x_2)}\cdot [(x-x_0)(x-x_1)(x-x_2)$$

  $$+(x-x_{-1})(x-x_1)(x-x_2)+(x-x_{-1})(x-x_0)(x-x_2)+(x-x_{-1})(x-x_0)(x-x_1) ]$$

  $$+ \frac{y_{-1}}{(x_{-1}-x_{-2})(x_{-1}-x_0)(x_{-1}-x_1)(x_{-1}-x_2)}\cdot [(x-x_0)(x-x_1)(x-x_2)$$

  $$+(x-x_{-2})(x-x_1)(x-x_2)+(x-x_{-2})(x-x_0)(x-x_2)+(x-x_{-2})(x-x_0)(x-x_1) ]$$

  $$+ \frac{y_0}{(x_0-x_{-2})(x_0-x_{-1})(x_0-x_1)(x_0-x_2)}\cdot [(x-x_{-1})(x-x_1)(x-x_2)$$

  $$+(x-x_{-2})(x-x_1)(x-x_2)+(x-x_{-2})(x-x_{-1})(x-x_2)+(x-x_{-2})(x-x_{-1})(x-x_1) ]$$

  $$+ \frac{y_1}{(x_1-x_{-2})(x_1-x_{-1})(x_1-x_0)(x_1-x_2)}\cdot [(x-x_{-1})(x-x_0)(x-x_2)$$

  $$+(x-x_{-2})(x-x_0)(x-x_2)+(x-x_{-2})(x-x_{-1})(x-x_2)+(x-x_{-2})(x-x_{-1})(x-x_0) ]$$

  $$+ \frac{y_2}{(x_2-x_{-2})(x_2-x_{-1})(x_2-x_0)(x_2-x_1)}\cdot [(x-x_{-1})(x-x_0)(x-x_1)$$

  $$+(x-x_{-2})(x-x_0)(x-x_1)+(x-x_{-2})(x-x_{-1})(x-x_1)+(x-x_{-2})(x-x_{-1})(x-x_0) ]$$

  
  
  When $x=x_0$,
  

  $$P'_4(x_0) = P'_2(x)\vert _{x=x_0}=$$

  $$= \frac{ y_{-2} \cdot (x_0-x_{-1})(x_0-x_1)(x_0-x_2) } {(x_{-2}-x_{-1})(x_{-2}-x_0)(x_{-2}-x_1)(x_{-2}-x_2)}$$

  $$+ \frac{ y_{-1} \cdot (x_0-x_{-2})(x_0-x_1)(x_0-x_2) } {(x_{-1}-x_{-2})(x_{-1}-x_0)(x_{-1}-x_1)(x_{-1}-x_2)}$$

  $$+ \frac{y_0}{(x_0-x_{-2})(x_0-x_{-1})(x_0-x_1)(x_0-x_2)} [ (x_0-x_{-1})(x_0-x_1)(x_0-x_2)$$

  $$+(x_0-x_{-2})(x_0-x_1)(x_0-x_2)+(x_0-x_{-2})(x_0-x_{-1})(x_0-x_2) +(x_0-x_{-2})(x_0-x_{-1})(x_0-x_1) ]$$

  $$+ \frac{ y_1 \cdot (x_0-x_{-2})(x_0-x_{-1})(x_0-x_2) } {(x_1-x_{-2})(x_1-x_{-1})(x_1-x_0)(x_1-x_2)}$$

  $$+ \frac{ y_2 \cdot (x_0-x_{-2})(x_0-x_{-1})(x_0-x_1) } {(x_2-x_{-2})(x_2-x_{-1})(x_2-x_0)(x_2-x_1)}$$

  
  

  In particular when $x_i$'s are equally spaced, i.e. $x_i-x_{i-1}=h$, the estimated derivative at $x=x_0$ can be found to be
  

  $$p'_3(x_0)=(-3y_0+4y_1-y_2)/2h$$
  
  
  It can be shown that the error is $h^2\;f^{'''}(\xi)/3$. Here $f'(x_0)$ is estimated using 3 points $(x_0, y_0), (x_1, y_1), (x_2, y_2)$ and is therefore called the forward estimation.

  We can also get backward estimation of $f'(x_0)$ using $(x_0, y_0), (x_{-1}, y_{-1}), (x_{-2}, y_{-2})$:

  
  

  $$p'_3(x_0)=(3y_0-4y_{-1}+y_{-2})/2h$$
  
  

  The average of the forward and backward estimations can be used as the final estimation of $f'(x_0)$.

  To overcome possible noise, better estimation may be obtained by using more points, such as 4 or 5 points.

  The four-point forward estimation:
  

  $$p'_4(x_0)=(2y_3-9y_2+18y_1-11y_0)/6h$$
  
  
  The four-point forward estimation:
  
  $$p'_4(x_0)=(-2y_{-3}+9y_{-2}-18y_{-1}+11y_0)/6h$$
  
  

  The five-point forward estimation:
  

  $$p'_5(x_0)=(-3y_4+16y_3-36y_2+48y_1-25y_0)/12h$$
  
  
  The five-point backward estimation:
  
  $$p'_5(x_0)=(3y_{-4}-16y_{-3}+36y_{-2}-48y_{-1}+25y_0)/12h$$
  
  

  This method is based on the assumption that the function $y=f(x)$ is continuous and noise-free. If this is not the case, the estimation error could be large.