Limit: Convergence and divergence – Serlo

In this chapter we will introduce the central concept of a limit (also called the Limes) in the context of sequences. We will discuss what the term convergence is and what it means for a sequence to be convergent. All of the important concepts of Analysis like continuity, derivatives and integrals can only be defined if we have a notion of limits and convergence. Thus the concepts introduced in this chapter will serve as the backbone of our entire discussion about Analysis.

Intuition behind the idea of convergence

Intuition behind the idea of convergence (in German) (Youtube-Video by Math-Intuition)

A video explaining convergence (in German). (YouTube-Video by the channel Quatematik)

Before one goes about to find a precise mathematical definition of a new concept, it is always a good idea to first develop an intuition. Let us therefore consider the harmonic sequence $\left({\tfrac {1}{n}}\right)_{n\in \mathbb {N} }$. The first few elements of the sequence are:

$1,\,{\tfrac {1}{2}},\,{\tfrac {1}{3}},\,{\tfrac {1}{4}},\,{\tfrac {1}{5}},\ldots$

You can already tell that elements become smaller and go towards zero as we increase the index $n$. We can make the following intuitive assertions:

• The sequence will get arbitrary close to $0$.
• The bigger the index $n$, the more the member $a_{n}={\tfrac {1}{n}}$ gets close to $0$.
• The sequence $a_{n}={\tfrac {1}{n}}$ tends to $0$.
• The sequence $a_{n}={\tfrac {1}{n}}$ will reach $0$ at infinity .
• …

All this intuitive statements, already give us an idea of what the limit of a sequence is. To give a first intuitive definition we would say: The limit is the thing (or in this case the value) that the sequence approaches as we go further and further, i.e as the index $n$ goes to infinity. In our case $0$ is the limit of the harmonic sequence $\left({\tfrac {1}{n}}\right)_{n\in \mathbb {N} }$. We say that the harmonic sequence is convergent. Of course not all sequences are convergent and thus have a finite limit: Think about the natural numbers $\mathbb {N}$ as a sequence. They grow bigger and bigger and never get closer to any particular value. We say that the sequence of the natural numbers is divergent.

Finding a definition for the limit

First steps

In mathematics we need an exact definition of what we mean by a "limit" to be able to talk and reason precisely. We can find such a definition if we start with an intuitive idea and gradually concretize it until we have a rigorous mathematical definition. The process of concretion will go on until we have a definition that employs only previously defined terms. Let's start with the following intuitive description of a limit:

„A sequence has a limit $a$, if the members of the sequence get arbitrary close to $a$.”

But what does „arbitrary close" mean in the above sentence? We could translate as follows: Imagine the members of the sequence in a coordinate system, where the $x$-axis are the indices $n$ and the $y$-axis has the members/values of the sequence $a_{n}$. So every member of the sequence corresponds to a point in this coordinate system. The limit $a$ is marked by a dotted line.

Folgenglieder im KOSY

If the members get "arbitrary close" to $a$ then the distance to the limit will keep getting smaller. Try to visualize a really narrow "hose" (maybe think about a garden hose) that has a radius of $\epsilon$. Imagine taking this hose and "threading" it over the limit from the right. As long as the distance of the members from the limit is smaller than the thickness of the hose, we can keep pushing the hose to the left. All points are still inside the hose. But as soon as a point has a bigger distance from the limit, this point will not be inside the hose. At this point we stop threading the hose.

Epsilonschlauch

The point ${\color {forestgreen}a_{N_{0}}}$ is the member of the sequence, from which on all members (so those with index greater than or equal $N_{0}$) will be inside the hose. Directly before $\color {OliveGreen}a_{N_{0}}$ there is a point that is outside of the hose (even by just a tiny amount). If we make the hose smaller, maybe some of the points that were inside the bigger one, now will be outside, and thus if your goal is to capture all points inside you can't "push" the hose as far as the bigger one. But even with this smaller hose it's still possible to capture almost all points:

Epsilonschlauch klein

The points that don't fit into the smaller hose are now more to the right than in the previous image. Let's call the first new member in the hose ${\color {forestgreen}a_{N_{1}}}$. All members with an index greater or equal to $N_{1}$ will be inside the smaller hose.

Of course this "garden hose" has no mathematical meaning. We used it as a mental tool to show that the members of the sequence are closer to the dotted line the bigger the index $n$. They keep approaching $a$ and will indeed not start drifting away from the dotted line, because we have seen that all members starting from a certain index will be inside the hose, no matter how small we choose to make it. If we understood that, we don't need the "hose" anymore. We will replace our arbitrary small "hoses" with radius $\epsilon$ with a mathematical object we call a $\epsilon$-neighbourhood (it sounds more scary than it really is).

Every neighbourhood around the limit contains almost all members

The limit is a number with the property that for every $\epsilon$-neighbourhood almost all members of the sequence are contained in that neighbourhood around that number.

We have found indices $N_{0}$ or $N_{1}$, from which on all later members of the sequence will be contained in the respective $\epsilon$-hose. If we make the hose smaller then we can find another $N_{2}$, from which on all members will be inside the hose and so on. No matter how small we choose our hose, we will always find a point so that all subsequent points are captured inside that "hose".

Since all this starting indices $N_{0}$ are natural numbers, there are only a finite amount of members that are outside of the hose ($N_{0}-1$ to be precise). All other points are inside the hose. Since a sequence has infinitely many members, we can "ignore" the finite amount of members that are outside and say that "almost all" members are inside the hose. This is still true, even if we choose $N_{0}$ to be really big. Compared to infinity a finite amount – no matter how big you choose $N_{0}$ - is still little. Understanding this is important to understand the idea of limits.

Let's repeat this one more time: No matter how small you choose your hose to be, almost all members will be captured inside it. That means that the members will approach $a$. And this is the central idea about the limit. The members $a_{n}$ will get arbitrary close to $a$, if we choose a sufficiently large index $n$.

What is a neighbourhood of a number?

We can construct the neighbourhood of a number $a$ geometrically with a circle. Let $a$ be the center of a circle with radius $0$. Then we have only marked the point $a$ on the number line. If we increase the radius of the circle, we see that the diameter expands from $a$.

On an intuitive level a neighbourhood is set of numbers that encloses $a$.

In one dimension this circle is just an open interval. A neighbourhood of a number $a$ can be mathematically described by such an interval. The radius of the circle is the distance to the left and right boundary of the interval . The radius is a arbitrary (small) positive number $\epsilon >0$.

An interval of this sort is characterized as the set of all numbers that are less than $\epsilon$ apart from $a$. Thus this intervals will have the form $(a-\epsilon ,a+\epsilon )$.

We call this interval $\epsilon$-neighbourhood of $a$, and it looks like this:

Die Epsilon-Umgebung

The $\epsilon$-neighbourhood of $a$ will more generally define a neighbourhood of $a$. A set $M$ is a neighbourhood of $a$, if and only if there exists an $\epsilon$-neighbourhood $(a-\epsilon ,a+\epsilon )$, such that $(a-\epsilon ,a+\epsilon )\;\subseteq M$. Let's see how this works by considering the following set $M$:

Menge M mit inneren Punkt a auf der Zahlengeraden

First we need to find an $\epsilon$-neighbourhood of $a$. Choose an adequate $\epsilon >0$ and draw a circle with radius $\epsilon$ around $a$. So that will mark an interval $(a-\epsilon ,a+\epsilon )$. The set $M$ is also an interval. It encloses the interval $(a-\epsilon ,a+\epsilon )$. Thus $M$ is a superset of the $\epsilon$-neighbourhood of $a$. So our definition says that $M$ is a neighbourhood of $a$.

Menge M mit inneren Punkt a und ε-Umgebung um a

Definition (Neighbourhood)

A set $U$ is a neighbourhood of a number $a$, if there exists an $\epsilon >0$, such that $(a-\epsilon ,a+\epsilon )\;\subseteq U$ (i.e. $\epsilon$-neighbourhood is contained in $U$).

What does „almost all" mean?

To better understand this idea imagine a coordinate system in which we have infinitely many members of a convergent sequence with limit $a$. We thread a small $\epsilon$-hose from the right over the limit. Then there are only a finite amount of members which are outside the hose, because their distance to $a$ is not small enough. But infinitely many members are inside the interval $(a-\epsilon ,a+\epsilon )$ and thus in the $\epsilon$-hose.

The amount of members that are inside the interval $(a-\epsilon ,a+\epsilon )$ is overwhelmingly larger than the ones outside. Therefore is reasonable to say that almost all members are inside $(a-\epsilon ,a+\epsilon )$.

Another way of expressing this idea would be to say that all but finitely many members are contained in the $\epsilon$-hose.

A mathematical definition for the limit

Now that we have an idea about what we are trying to define, we will try to find a rigorous mathematical definition. We start by observing that:

„A sequence has a limit $a$, if for every $\epsilon$-neighbourhood of $a$, i.e. $(a-\epsilon ,a+\epsilon )$, there exists a member, that starting from it all following members are inside the neighbourhood.“

We could already work with this definition. But for practical purposes it's useful to further formalise this definition.

Note that a member $a_{n}$ is an element of $(a-\epsilon ,a+\epsilon )$, if and only if $|a_{n}-a|<\epsilon$. Thus:

„A sequence $\left(a_{n}\right)_{n\in \mathbb {N} }$ has a limit $a$, if for every $\epsilon >0$, there exists a member $a_{N}$, starting from which all following members $a_{n}$ are less than $\epsilon$ away from $a$, that means that they satisfy $|a_{n}-a|<\epsilon$.

The part with „there exists a member, starting from which…“ can be equivalently formulated as „there exists a natural number $N$, so that for all $a_{n}$ with $n\geq N$ follows that …“. Thus:

„A sequence $\left(a_{n}\right)_{n\in \mathbb {N} }$ has a limit $a$, if for every $\epsilon >0$ there exists a natural number $N$, so that $|a_{n}-a|<\epsilon$ for all $n\geq N$.

This is the mathematical definition of the limit.

Definition of limit

Definition (Limit)

A sequence $\left(a_{n}\right)_{n\in \mathbb {N} }$ has a limit $a$, if for every $\epsilon >0$ there exists an index $N\in \mathbb {N}$, so that for every member $a_{n}$ with $n\geq N$ the inequality $|a_{n}-a|<\epsilon$ is true. So $a$ is a limit of $\left(a_{n}\right)_{n\in \mathbb {N} }$ if and only if:

$\forall \epsilon >0\,\exists N\in \mathbb {N} \,\forall n\geq N:|a_{n}-a|<\epsilon$

This is what the individual parts of the above formula mean:

${\begin{array}{l}\underbrace {{\underset {}{}}\forall \epsilon >0:} _{{\text{ For every }}\epsilon >0}\ \underbrace {{\underset {}{}}\exists N\in \mathbb {N} :} _{{\text{ there exists an index }}N}\ \underbrace {{\underset {}{}}\forall n\geq N:} _{{\text{ so that for all indices }}n\geq N}\\[1em]\quad \quad \underbrace {{\underset {}{}}|a_{n}-a|<\epsilon } _{{\text{ the distance between }}a_{n}{\text{ and }}a{\text{ is smaller than }}\epsilon }\end{array}}$

There are a few other important definitions when studying limits:

Convergence

A sequence is convergent, if the sequence has a limit. We say that the sequence converges towards $a$, if there is a limit $a$.

Divergence

If there is no limit a sequence is called divergent. Thus a sequence is divergent if it is not convergent.

Null Sequence

A sequence that has a limit of $0$ is called a Null Sequence.

If a sequence converges to $a$, we also write it like $\lim _{n\to \infty }a_{n}=a$ or „$a_{n}\rightarrow a$ für $n\rightarrow \infty$“. We say „Limes of $a_{n}$ for $n$ goes to infinity, is $a$“.

Question: What is the formula in predicate logic to express that $(a_{n})_{n\in \mathbb {N} }$ is convergent?

As we discussed above the forumla to express that a sequence $(a_{n})_{n\in \mathbb {N} }$ has a limit $a$ is the following:

$\forall \epsilon >0\,\exists N\in \mathbb {N} \,\forall n\geq N:|a_{n}-a|<\epsilon$

If we only want to express convergence, we do it like this:

$\exists a\in \mathbb {R} \,\forall \epsilon >0\,\exists N\in \mathbb {N} \,\forall n\geq N:|a_{n}-a|<\epsilon$

Question: What is the formula in predicate logic to express that $(a_{n})_{n\in \mathbb {N} }$ diverges?

We need to negate the previous formula. We need to invert the all- and existence-quantifiers. The negated statement is:

$\forall a\in \mathbb {R} \,\exists \epsilon >0\,\forall N\in \mathbb {N} \,\exists n\geq N:|a_{n}-a|\geq \epsilon$

In words: For every $a\in \mathbb {R}$ there is a real number $\epsilon >0$, so that for all $N\in \mathbb {N}$ there is a $n\geq N$ with $|a_{n}-a|\geq \epsilon$.

Hint

For the absolute value we have: $|-x|=|x|$. Thus also:

$|a_{n}-a|=|-(a-a_{n})|=|a-a_{n}|$

We could use either $|a_{n}-a|$ or $|a-a_{n}|$ in our definition of limit.

Hint

From the definition of convergence it follows immediately that $(a_{n})_{n\in \mathbb {N} }$ converges towards $a$, if and only if $(|a_{n}-a|)_{n\in \mathbb {N} }$ is a Null Sequence. If $(|a_{n}-a|)_{n\in \mathbb {N} }$ converges to $0$, then from the definition we see:

$\forall \epsilon >0\,\exists N\in \mathbb {N} \,\forall n\geq N:||a_{n}-a|-0|=||a_{n}-a||=|a_{n}-a|<\epsilon$

But this is exactly the definition for what it means for $(a_{n})_{n\in \mathbb {N} }$ to converge towards $a$. If conversely $(a_{n})_{n\in \mathbb {N} }$ converges to $a$, we use the definition to conclude:

$\forall \epsilon >0\,\exists N\in \mathbb {N} \,\forall n\geq N:|a_{n}-a|<\epsilon$

But this implies that $||a_{n}-a|-0|=||a_{n}-a||=|a_{n}-a|<\epsilon$ and the quantifiers and the variables in the formula don't change, so that $(|a_{n}-a|)_{n\in \mathbb {N} }$ converges to $0$ and thus is a Null Sequence.

Warning

A common misconception is that "A sequence is divergent if and only if it is unbounded". This statement is false!

The intuitive mistake that often happens is to assume that: „The opposite of $\forall \epsilon >0\ldots |a_{n}-a|<\epsilon$ is $\forall \epsilon >0\ \ldots \ |a_{n}-a|\geq \epsilon$, thus $(a_{n})_{n\in \mathbb {N} }$ must become arbitrary large.“ But as we have seen this is not the definition of divergence!

What is true, is that every unbounded sequence is divergent (more on this in„Unbounded sequences are divergent“), but not every divergent sequence must necessarily be unbounded. Take for example the following alternating sequence $\left((-1)^{n}\right)_{n\in \mathbb {N} }=(-1,1,-1,1,-1,1,\ldots )$, which is in fact divergent but bounded.

Explanation of convergence

Besides the above derivation of limits there is another intuition for the limit: The quantity $|a_{n}-a|$ is the distance between the n-th member and $a$. It is a measure of the error between $a_{n}$ and $a$. The inequality $|a_{n}-a|<\epsilon$ means, that the error between $a_{n}$ and $a$ is guaranteed to be smaller than $\epsilon$. Therefore we can interpret the definition of limit as follows: No matter how small we choose our maximum error $\epsilon >0$ to be, almost all members are less than $\epsilon$ away from the limit $a$. The error $|a_{n}-a|$ between the members and the limit becomes arbitrary small.

There are also historical reasons for this interpretation. Augustin-Louis Cauchy, who first proposed this definition, maybe wanted $\epsilon$ to remind of the french word „erreur“ which means „error"..

Example: Convergence of the harmonic progression

The first ten elements of the harmonic sequence

Let's see this concepts in action by studying the harmonic progression with the generic member $a_{n}={\tfrac {1}{n}}$. It should be intuitive that this sequences converges towards $0$. If this is correct, then this sequences should satisfy the definition for convergence towads $0$.

Take for example $\epsilon ={\tfrac {1}{2}}$. Starting from the third member $a_{3}={\tfrac {1}{3}}$ the distance between $a_{n}$ and $0$ is smaller than $\epsilon ={\tfrac {1}{2}}$. All later members of this sequence are contained inside the $\epsilon$-neighbourhood of $]-{\tfrac {1}{2}},{\tfrac {1}{2}}[$. For $\epsilon ={\tfrac {1}{10}}$ this is true starting from $n=11$ and for $\epsilon =0,00001$ starting from $n=100001$.

But the definition of limit requires that this is true for all $\epsilon$. Therefore, let us assume we are given an $\epsilon >0$. From the archimedean axiom it follows that there exists an $N\in \mathbb {N}$, so so that ${\tfrac {1}{n}}<\epsilon$ for all $n\geq N$ (See Archimedean Axiom whit the choices of $x=1$ and $y={\frac {2}{\epsilon }}$.) From this $N$ onwards all members are contained inside the $\epsilon$-neighbourhood $]-\epsilon ,\epsilon [$. The definition tells us that $0$ is the limit of the harmonic progression.

The limit is unique

Theorem (Uniqueness of the limit)

The limit of a convergent sequence is unique.

This theorem allows us to write things such as $\lim _{n\to \infty }a_{n}$. Imagine that there is a sequence $\left(b_{n}\right)_{n\in \mathbb {N} }$ with more than one limit. Then it is not clear what the expression $\lim _{n\to \infty }b_{n}$ should stand for (there are two possibilities). But because we know that $\left(b_{n}\right)_{n\in \mathbb {N} }$ has a unique limit, it is clear that $\lim _{n\to \infty }b_{n}$ can only mean one thing, i.e. the (unique) limit of that sequence (if it is convergent). The above theorem allows us to talk about the limit and not a limit.

How to get to the proof? (Uniqueness of the limit)

We can prove this by contradiction. We assume that there is a sequence $\left(a_{n}\right)_{n\in \mathbb {N} }$ with two distinct limits. Now we need to show that this assumption will result in a contradiction, and that it therefore cannot be true.

We label the two distinct limits $a$ and $b$. To find a contradiction, we can use the following method: We try to prove the opposite of what we really want to show. Of course we set ourself up for failure. Why do we even try then? Because by understanding why it fails we get a lot of clues that will help us prove the right thing. So for the sake of gathering insight, let us (try) to prove that there is a sequence with two distinct limits.

Take a piece of paper and draw a Number line. Mark two distinct numbers $a$ and $b$ that will represent our two limits. Now try to find a real sequence that converges to both this limits (Remember that starting from a certain index you sequence has to be in an arbitrary small $\epsilon$-neighbourhood around $a$ and $b$). But for arbitary small $\epsilon$-neighbourhoods of $a$ and $b$, that do not intersect, it is impossible that almost all members are contained in both. The following drawing will highlight this problem:

Zahlengerade mit zwei Zahlen und disjunkten ε-Umgebungen

Thus if we choose $\epsilon$ so small, that $(a-\epsilon ,a+\epsilon )$ and $(b-\epsilon ,b+\epsilon )$ do not overlap, the we know that there must be a contradiction hiding somewhere (the one we need to prove our original proposition!). We choose $\epsilon ={\tfrac {|a-b|}{3}}$. We now that there must be a $a_{n}$ that must be contained both in $(a-\epsilon ,a+\epsilon )$ and $(b-\epsilon ,b+\epsilon )$. But such an $a_{n}$ can't exists, since those two neighbourhoods can't intersect for $\epsilon ={\tfrac {|a-b|}{3}}$ A contradiction emerges from the triangle inequality:

${\begin{aligned}|a-b|&=|a+\overbrace {(-a_{n}+a_{n})} ^{=0}-b|\\[1ex]&=|(a-a_{n})+(a_{n}-b)|\\[1ex]&\quad {\color {OliveGreen}\left\downarrow \ {\text{Triangle inequality}}\right.}\\[1ex]&\leq |a-a_{n}|+|a_{n}-b|\\[1ex]&<{\frac {|a-b|}{3}}+{\frac {|a-b|}{3}}\\[1ex]&={\frac {2\cdot |a-b|}{3}}\end{aligned}}$

We cancel both sides with $|a-b|$ and find the contradiction $1<{\tfrac {2}{3}}$.

Proof (Uniqueness of the limit)

Proof by contradiction: Let $\left(a_{n}\right)_{n\in \mathbb {N} }$ be a sequence with two distinct limits $a$ and $b$. Distinct means that $a\neq b$ and therefore $|a-b|>0$. Per definition there exists $N_{1}$ and $N_{2}$ with

$\forall n\geq N_{1}:\ |a_{n}-a|<{\frac {|a-b|}{3}}$

and

$\forall n\geq N_{2}:\ |a_{n}-b|<{\frac {|a-b|}{3}}$

This follows from the definition of limit if we set $\epsilon ={\tfrac {|a-b|}{3}}$. Thus for all members $a_{n}$ with index $n\geq \mathrm {max} \{N_{1},\,N_{2}\}$, both $|a_{n}-a|<{\tfrac {|a-b|}{3}}$ and $|a_{n}-b|<{\tfrac {|a-b|}{3}}$ is true. In this case we have:

${\begin{aligned}|a-b|&=|a+(-a_{n}+a_{n})-b|\\[1ex]&=|(a-a_{n})+(a_{n}-b)|\\[1ex]&\quad {\color {OliveGreen}\left\downarrow \ {\text{Triangle inequality}}\right.}\\[1ex]&\leq |a-a_{n}|+|a_{n}-b|\\[1ex]&<{\frac {|a-b|}{3}}+{\frac {|a-b|}{3}}\\[1ex]&={\frac {2\cdot |a-b|}{3}}\end{aligned}}$

Because of $a\neq b$ we have $|a-b|>0$ and we can cancel both sides in the inequality with $|a-b|$. This results in the contradiction:

$1<{\frac {2}{3}}$

It is essential for this prove that $a\neq b$ and thus $|a-b|>0$. Else we could not have divided by $|a-b|$ (division by zero!) and we could not have chosen $\epsilon ={\tfrac {|a-b|}{3}}>0$.