Differential Calculus

Introduction

Calculus is the mathematical study of continously changing quantities. It has two main branches: differential calculus (as we’ll discuss here) and integral calculus (in the next section). Differential calculus is concerned with the study of the rates at which quantities change. Integral calculus is concerned with the study of the accumulation of quantities, and is often appleid to the areas under and between curves.

In this section, we will revise the rules for differentiating functions, and then look at some examples of common derivatives.

Differentiation

Formal Definition

Given a function, \(f(x)\), over a domain, \(x \in \math{\Omega}, the derivative of \)f(x)\( with respect to \)x$ is defined as:

\[ \frac{df}{dx} = \lim_{h \to 0} \frac{f(x+h) - f(x)}{h}, \]

where-ever this limit exists[1]. To give a concrete example, we can take the function \(f(x)=x^2\) and substitute it into the definition of the derivative:

\[ \frac{f(x+h) - f(x)}{h} = \frac{(x+h)^2 - x^2}{h} = \frac{2xh + h^2 }{h} = 2x +h, \]

so that

\[ \frac{df}{dx} = \lim_{h \to 0} 2x +h = 2x.\]

Note that the notation given here isn’t totally universal, and that while the derivative of a function \(f(x)\) is often denoted as \(f'(x)\) or \(\frac{df}{dx}\), authors sometimes pick other notations for convenience in their own wrting.

As can be seen clearly in the \(f'(x)\) notation, the derivative of a function is itself a function which is (usually) still dependent on the value of \(x\). This means that the derivative of a function can be evaluated at any point (say \(x=a\)) to give the rate of change of the function at that point. If the derivative is smooth enough, then it can also be differentiated again to give the second derivative, \(f''(x)\), which is the rate of change of the rate of change of the function, and so on to higher and higher powers (or until the function is no longer smooth enough to differentiate).

If we plot a function, it’s derivative can shown as a second curve on the same plot. The derivative curve will show the rate of change of the function at each point on the axis. If the function is increasing, then ita derivative will be positive. If the function is decreasing, then its derivative will be negative. If the function is (locally) staying constant, then the derivative will be zero.

Differentiation Rules

Summation Rule

The summation rule states that the derivative of a sum of functions is equal to the sum of the derivatives of the functions:

\[ \frac{d}{dx} \Big( f(x) + g(x) \Big) = \frac{df}{dx} + \frac{dg}{dx}.\]

For those of a mathematical interest this can be proven through an appropriate application of the summation “limit law” to the definition of the derivative at a limit. Explaining this in detail is beyond the scope of this material (and your MSc programme) but does form the basis of many real analysis and university level calculus courses.

\[\lim_{h \to 0} \frac{\left(f(x+h) + g(x+h)\right) - left(f(x)+ g(x)\right)}{h} = \lim_{h \to 0} \frac{f(x+h) - f(x)}{h} + \lim_{h \to 0} \frac{g(x+h) - g(x)}{h}.\]

Product Rule

The product rule states that the derivative of a product of functions is equal to the first function times the derivative of the second function plus the second function times the derivative of the first function:

\[ \frac{d}{dx} \Big( f(x) g(x) \Big) = f(x) \frac{dg}{dx} + g(x) \frac{df}{dx}.\]

Although a little more complicated, again this can be proved using the rules around limits, starting by rewriting the numerator on the left hand side using (say)

\[ f(x+h)g(x+h)-f(x)g(x) = f(x+h)g(x+h) \underbrace{- f(x)g(x+h) +f(x)g(x+h)}_{=0} - f(x)g(x).\]

Chain Rule

The chain rule states that the derivative of a function which takes another function as its input is equal to the derivative of the outer function evaluated at the inner function times the derivative of the inner function:

\[ \frac{d}{dx} f(g(x)) = \frac{df}{dg} \frac{dg}{dx}. \]

Again, the proof is through limits, starting from

\[\lim_{h \to 0} \frac{f(g(x+h)) - f(g(x))}{h} = \lim_{h \to 0} \frac{f(g(x+h)) - f(g(x))}{g(x+h) - g(x)} \frac{g(x+h) - g(x)}{h},\]

separating this into the product of two limits, and substituting in one to be in terms of \(X:=g(x)\) and \(k := g(x+h) - g(x)\to 0\).

Quotient Rule

The quotient rule defines the derivative of a quotient (ie. a ratio) of functions as:

\[ \frac{d}{dx} \left( \frac{f(x)}{g(x)} \right) = \frac{g(x) \frac{df}{dx} - f(x) \frac{dg}{dx}}{g(x)^2}. \]

This result can be derived by combining the product rule and an application of the chain rule fof the function \(1/g(x)\).

Examples

Polynomial Functions

Polynomial functions are functions of the form:

\[ f(x) = a_0 + a_1 x + a_2 x^2 + a_3 x^3 + \dots + a_n x^n \]

where \(a_0, a_1, a_2, \dots, a_n\) are constants. The derivative of a polynomial function is given by:

\[ \frac{df}{dx} = a_1 + 2 a_2 x + 3 a_3 x^2 + \dots + n a_n x^{n-1} \]

This formula can be derived by applying the sum rule and the product rule to the polynomial function, and using the Binomial Theorem to expand the terms in powers of \(h\).

In fact, the formula

\[ \frac{d(x^n)}{dx} = n x^{n-1} \]

holds for all real values of \(n\), including negative values and fractional values, providing we note that \(a^0 = 1\) for all \(a \neq 0\), so there is a slight ambiguity at \(x=0\) and \(n=0\) or \(n=1\).

Exponential and logorithmic Functions

The derivative of an exponential function is given by:

\[ \frac{d}{dx} e^x = e^x, \]

or more generally,

\[ \frac{d}{dx} a^x = a^x \ln(a) \]

where \(a\) is a positive constant. This last value can be derived by applying the chain rule to the function \(e^{x \ln(a)}=a^x\). Indeed, Euler’s number, \(e\), can be defined as the value of \(a\) for which \(a^x\) has a derivative of \(1\) at \(x=0\), which then forms the basis for natural logarithms.

By applying the chain rule to the function \(e^{\ln(x)}=x\) we can also show that the derivative of the natural logarithm is given by:

\[ \frac{d}{dx} \ln(x) = \frac{1}{x}. \]

Trigonometric Functions

The primary Trigonometric functions have derivatives given by:

\[ \frac{d}{dx} \sin(x) = \cos(x), \]

\[ \frac{d}{dx} \cos(x) = -\sin(x), \]

\[ \frac{d}{dx} \tan(x) = \sec^2(x), \]

providing that the argument of the function being used is in radians. The derivatives of the other trigonometric functions can be derived from these using the quotient rule, etc.

Exercises

1. Differentiate the following functions:

   • \(f(x) = x^3 + 2x^2 - 5x + 7\)

   • \(g(x) = \sin(x) + \cos(x)\)

   • \(h(x) = e^{2x} + \ln(x)\)

   • \(k(x) = \frac{x^2 + 1}{x - 1}\)

Answer

1. The derivatives are:

   • \(f'(x) = 3x^2 + 4x - 5\)

   • \(g'(x) = \cos(x) - \sin(x)\)

   • \(h'(x) = 2e^{2x} + \frac{1}{x}\)

   • \(k'(x) = \frac{(2x)(x-1) - (x^2+1)(1)}{(x-1)^2} = \frac{x^2 - 3x + 1}{(x-1)^2}\)

2. Using the chain and product rules, find the derivative of the following functions:

   • \(f(x) = (x^2 + 1)^3\)

   • \(g(x) = \sin(x^2)\)

   • \(h(x) = e^{\sin(x)}\)

Answer

2. The derivatives are:

   • \(f'(x) = 3(x^2 + 1)^2(2x) = 6x(x^2 + 1)^2\)

   • \(g'(x) = 2x \cos(x^2)\)

   • \(h'(x) = e^{\sin(x)} \cos(x)\)

3. Find the gradient of the function \(f(x,y) = 3x^3 + y^2\) at the point \((1, 1)\).

Answer

3. The gradient of the function is given by the vector of partial derivatives:

\[ \nabla f(x,y) = \left( \frac{\partial f}{\partial x}, \frac{\partial f}{\partial y} \right) = (3x^2, 2y). \]

At the point \((1, 1)\), the gradient is: $\( \nabla f(1, 1) = (3, 2). \)$

4. Find the divergence of the vector field \(\mathbf{F}(x,y,z) = (x^2, \ln y +x^2, \cos(\pi z) +2y)\) at the point \((1, 2, 3)\).

Answer

4. The divergence of a vector field is given by the sum of the partial derivatives of its components: $\( \nabla \cdot \mathbf{F} = \frac{\partial F_1}{\partial x} + \frac{\partial F_2}{\partial y} + \frac{\partial F_3}{\partial z} = 2x + \frac{1}{y} - \pi \sin(\pi z) + 2. \)$

At the point \((1, 2, 3)\), the divergence is: $\( \nabla \cdot \mathbf{F}(1, 2, 3) = 2(1) + \frac{1}{2} - \pi \sin(3\pi) + 2 = 2 + \frac{1}{2} + 0 + 2 = 4.5. \)$

Further reading

• Wikipedia: Differentiation

---

[1]

I.e the function is “smooth” enough that the limit exists. This is a technical condition which is often satisfied in practice, but not always. For example, the function \(f(x) = |x|\) is not differentiable at \(x=0\), even though it is continuous there.