Some thoughts about epsilon and delta

By Ben Blum-Smith, Contributing Editor

The calculus has a very special place in the 20th century’s traditional course of mathematical study. It is a sort of fulcrum: both the summit toward which the whole secondary curriculum strives, and the fundamental prerequisite for a wide swath of collegiate and graduate work, both in mathematics itself and in its applications to the sciences, economics, engineering, etc.[1] At its heart is the notion of the limit, described in 1935 by T. A. A. Broadbent as the critical turning point:

The first encounter with a limit marks the dividing line between the elementary and the advanced parts of a school course. Here we have not a new manipulation of old operations, but a new operation; not a new trick, but a new idea.[2]

Humanity’s own collective understanding of this “new idea” was hard-earned. The great length of the historical journey toward the modern definition in terms of \epsilon and \delta mirrors the well-known difficulty students have with it. Although it is the foundation of calculus, it is common to push the difficulty of this definition off from a first calculus course onto real analysis. Indeed, mathematicians have been discussing the appropriate place for the full rigors of this definition in the calculus curriculum for over 100 years.[3]

There is also a rich vein in the mathematics education research literature studying students’ engagement with the \epsilon\delta definition. Researchers have examined student difficulties coming from its multiple nested quantifiers[4] as well as its great distance from the less formal notions of limit with which students typically enter its study,[5] and have also made an effort to chart the paths they take toward a full understanding.[6]

This blog post is a contribution to this conversation, analyzing in detail three learners’ difficulties with \epsilon and \delta.[7] If there is a take-home message, it is to respect the profound subtlety of this definition and the complexity of the landscape through which students need to move as they learn to work with it.

Some history

Many readers will be familiar with the long struggle to find a rigorous underpinning for the calculus of Newton and Leibniz, leading to the modern definition of the limit in terms of \epsilon and \delta. In this section, I excerpt a few episodes, which will become important in the later discussion of student thought. Readers already familiar with the history of the subject are welcome to skim or skip this section.

While Newton and Leibniz published their foundational work on (what we now call) derivatives and integrals in the late 17th century, they based these ideas not on the modern limit, but on notions that look hand-wavy in retrospect.[8] To Leibniz, the derivative, for example, was a ratio of “infinitesimal” quantities — smaller than finite quantities, but not zero. To Newton, it was an “ultimate ratio”, the ratio approached by a pair of quantities as they both disappear. Both authors would calculate the derivative of x^n via a now-familiar manipulation: augment x by a small amount o; correspondingly, x^n augments to (x+o)^n = x^n + nox^{n-1} + \dots + o^n. The ratio of the change in x^n to the change in x is nox^{n-1}+\dots+o^n : o, or nx^{n-1}+\dots+o^{n-1}:1. At this point, they would differ in their explanation of why you can ignore all the terms involving o in this last expression: for Leibniz, it is because they are infinitesimal, and for Newton, it is because they all vanish when the augmentation of x is allowed to vanish.

A famous critique of both of these lines of reasoning was leveled in 1734 by the British philosopher and theologian Bishop George Berkeley, arguing that since to form the ratio of nox^{n-1}+\dots+o^n to o in the first place, it was necessary to assume o is nonzero, it is strictly inconsistent to then decide to ignore it.

Hitherto I have supposed that x flows, that x hath a real Increment, that o is something. And I have proceeded all along on that Supposition, without which I should not have been able to have made so much as one single Step. From that Supposition it is that I get at the Increment of x^n, that I am able to compare it with the Increment of x, and that I find the Proportion between the two Increments. I now beg leave to make a new Supposition contrary to the first, i.e., I will suppose that there is no Increment of x, or that o is nothing; which second Supposition destroys my first, and is inconsistent with it, and therefore with every thing that supposeth it. I do nevertheless beg leave to retain nx^{n-1}, which is an Expression obtained in virtue of my first Supposition, which necessarily presupposeth such Supposition, and which could not be obtained without it: All which seems a most inconsistent way of arguing…

It was a long journey from the state of the art in the early 18th century, to which Berkeley was responding, to the modern reformulation of calculus on the basis of the \epsilon\delta limit. The process took well over a century. I will summarize this story by quoting somewhat telegraphically from William Dunham’s book The Calculus Gallery,[9] from which I first learned it.

Berkeley penned the now famous question:

… They are neither finite quantities nor quantities infinitely small, nor yet nothing. May we not call them the ghosts of departed quantities?

… Over the next decades a number of mathematicians tried to shore up the shaky underpinnings… pp. 71-72

… Cauchy’s “limit-avoidance” definition made no mention whatever of attaining the limit, just of getting and staying close to it. For him, there were no departed quantities, and Berkeley’s ghosts disappeared… p. 78

… If his statement seems peculiar, his proof began with a now-familiar ring, for Cauchy introduced two “very small numbers” \delta and \epsilon… p. 83

… We recall that Cauchy built his calculus upon limits, which he defined in these words:

When the values successively attributed to a variable approach indefinitely to a fixed value, in a manner so as to end by differing from it by as little as one wishes, this last is called the limit of all the others.

To us, aspects of this statement, for instance, the motion implied by the term “approach,” seem less than satisfactory. Is something actually moving? If so, must we consider concepts of time and space before talking of limits? And what does it mean for the process to “end”? The whole business needed one last revision.

Contrast Cauchy’s words with the polished definition from the Weierstrassians:

\lim_{x\to a} f(x) = L if and only if, for every \epsilon > 0, there exists a \delta > 0 so that, if 0 < |x-a| < \delta, then |f(x) - L| < \epsilon.

Here nothing is in motion, and time is irrelevant. This is a static rather than dynamic definition and an arithmetic rather than a geometric one. At its core, it is nothing but a statement about inequalities. pp. 129-130

The Weierstrassian definition (i.e., the modern one!) allows the manipulation to which Berkeley objected to be carried to completion without ever asking o to be zero.

Missing the point completely

The core of this blog post is a discussion of three learners’ encounters with the \epsilon\delta limit, seeking to illuminate some of the subtle challenges that can arise. I begin with my own story.

I “did well” in my college real analysis class, by which I mean that my instructor (a well-known analyst at a major research university) concluded on the basis of my written output that I had mastered the material. However, I walked away from the course with a subtle but important misunderstanding of the \epsilon\delta definition that was not visible in my written work and so went entirely undetected by my instructor, and, for many years, by myself as well.

From my previous experience with calculus, I had concluded that you can often identify the value toward which a function is headed, even if the function is undefined “when you get there.” To the extent I had a definition of limit, this was it: the value toward which the function is headed.

When I studied real analysis as an undergraduate, I found the class easy, including the \delta\epsilon work. I mean, if L is where f(x) is headed as x\to c, then sure, for any \epsilon-neighborhood around L, there is going to be a \delta-neighborhood around c that puts f(x) inside the \epsilon-neighborhood. But I related to the notion that f(x) is headed toward L as conceptually prior to this \epsilon\delta game. The latter seemed like fancy window-dressing to me, possibly useful post-hoc if you need an error estimate. I did not understand that it was a definition — that it was supposed to be constitutive of the meaning of limit. So, I completely missed the point! But I want to stress that you would not have known this from my written work, and of course, I didn’t know it either.

I went on to become a high school math teacher. In the years that followed, I did detect certain limitations in my understanding of limits. For example, I noticed that I didn’t have adequate tools for thinking about if and when the order of two different limit processes could be safely interchanged. But it did not cross my mind that the place to start in thinking clearly about this was a tool I had already been given.

After a few years, I began teaching an AP Calculus course. About 3/4 of the way through my first time teaching it, my student Harry[10] said to me after class, “You know this whole class is based on a paradox, right?” He proceeded to give me what I now recognize as essentially Bishop Berkeley’s critique. At the time, it did not occur to me to reach for epsilon and delta. Instead, I responded like an 18th century mathematician, trying to convince him that the terminus of an unending process is something it’s meaningful to talk about. I hadn’t really understood what the problem was. Of course, Harry left unsatisfied.

The pieces finally came together for me the next year, when I read Dunham’s Calculus Gallery, quoted above. I remember the shift in my understanding: ooooohhhhhh. The \epsilon‘s and \delta‘s are not an addendum to, or a gussying-up of, the idea of identifying where an unending process is headed. They are replacing this idea! It was a revelation to reread the definition from this new point of view. Calculus does not need the infinitesimal! I immediately wished I had a do-over with Harry, whose dissatisfaction I hadn’t comprehended enough to be able to speak to.

I concluded from this that a complete understanding of the \epsilon\delta definition includes an understanding of what it’s for.

But it’s not the same thing at all

Having come to this conclusion, in my own teaching of real analysis I’ve made a great effort to make clear the problem that \epsilon and \delta are responding to. In one course, I began with a somewhat in-depth study of Berkeley’s critique of the 18th century understanding of the calculus, in order to then be able to offer \epsilon and \delta as an answer to that critique.

In doing this, I ran into a new challenge. To illustrate, I’ll focus on the experience of a student named Ty. Ty arrived in my course having already developed a fairly strong form of a more intuitive, 17th-18th century understanding of the limit; essentially the Newtonian one, much like the understanding that had carried me myself through all my calculus coursework. He quickly made sense of Berkeley’s objection, so he was able to see that this understanding was not mathematically satisfactory. I was selling the \epsilon\delta definition as a more satisfactory substitute. However, Ty objected that important aspects of his understanding of the limit (what Tall and Vinner called his concept image[11]) were not captured by this new definition. In particular, what had happened to the notion that the limit was something toward which the function was, or even should have been, headed? The \epsilon\delta definition of \lim_{x\to a}f(x) studiously avoids the point a “at which the limit is taken,” even speculatively. To Ty, it was the \epsilon\delta definition that was, pun intended, missing the point.

Of course, this studious avoidance is precisely how the Weierstrassian definition gets around Berkeley’s objection. The Newtonian “ultimate ratio” and the Leibnizian “infinitesimal” both ask us to imagine something counterfactual, or at least pretty wonky. This is exactly what made them hard for Berkeley to swallow, and as I learned from Dunham’s book, the great virtue of \epsilon and $\delta$ is that they give us a way to uniquely identify the limit that does not ask us to engage in such a trippy flight of fancy that may or may not look sane in the light of day.

But, at the same time, something is lost.[12] What I learned from Ty is that this loss is pedagogically important to acknowledge.[13]

One vs. many

Another subtle difficulty in working with the \epsilon\delta definition is revealed when you use it to try to prove something. I think what I am about to describe is a general difficulty students encounter in learning the methods and conventions of proof-writing, but I speculate that it may be particularly acute with respect to the present topic. Consider this (utterly standard) proof that if f,g are functions of $x$ such that \lim_{x\to a}f = L and \lim_{x\to a}g = M, and h=f+g, then \lim_{x\to a} h = L+M:

Let \epsilon > 0 be given. Because \lim_{x\to a} f = L, there exists \delta_1 > 0 such that 0 < |x - a| < \delta_1 implies |f - L| < \epsilon / 2. Similarly, because \lim_{x\to a} g = M, there exists \delta_2 > 0 such that 0 < |x - a| < \delta_2 implies |g-M| < \epsilon/2.

Take \delta = \min(\delta_1,\delta_2).

Then for values of x satisfying 0 < |x - a| < \delta, it follows from the triangle inequality and the definition of h that

|h - (L+M)| = |f - L + g - M| \leq |f-L| + |g-M| < \epsilon / 2 + \epsilon / 2 = \epsilon.

Since \epsilon > 0 was arbitrary, we can conclude that \lim_{x\to a} h = L+M.

Here is a surprisingly rich question: is the \epsilon in the proof one number, or many numbers?

On one way of looking at it, of course it is only one number: \epsilon is fixed at the outset of the proof. Indeed, if \epsilon were allowed to be more than one thing, equations like $\epsilon / 2 + \epsilon / 2 = \epsilon$ would be meaningless. More subtly, we usually speak about \epsilon as a single fixed quantity when we justify the existence of \delta_1,\delta_2 in terms of the definition of the limit: we know \delta_1 exists because by the definition of the limit, for any \varepsilon there is a \delta, so in particular, there is a \delta_1 for \epsilon / 2, etc. Note the “in particular”: we produce \delta_1 from the definition by specializing it.

But on another way of looking at it, of course $\epsilon$ is many numbers. Indeed, it must represent every positive number, otherwise how can it be used to verify the definition for all \epsilon>0? The singular, fixed \epsilon with which we work in the proof is a sort of chimera: it actually represents all positive numbers at once. That we think of it as a single number is just a psychological device to allow us to focus in a productive way on what we are doing with all these numbers.

This dual nature of \epsilon in the above was driven home for me by working with Ricky. Fast and accurate with calculations and algebraic manipulations, Ricky was thrown for a loop by real analysis, which was her first proof-based class, and in particular by the \epsilon\delta proofs. After a lot of work, she had mastered the definition itself. But in trying to write the proofs, she found the lilting refrain for all \epsilon > 0 to be a kind of siren song, leading her astray. She was constantly re-initializing \epsilon with this phrase, so that reading her work, there were 3 different meanings for \epsilon by the end. “Look at how the proof works,” I would say, referring to the proof of \lim h = L+M above. “You don’t need |f-L| to be less than any old \epsilon. You need it to be less than the particular \epsilon that you are using for h.” “What do you mean the particular \epsilon I am using?” she would ask. “I am trying to prove it works for all \epsilon!

Ricky’s difficulty has led me to a much greater appreciation of the subtle and profound abstraction involved any time an object is introduced into a proof with the words “fix an arbitrary…” In a sense, this is nothing more — nor less! — than the abstraction at the heart of a student’s first encounter with algebra: if we imagine an unspecified number x, and proceed to have a conversation about it, our conversation applies simultaneously at once to all the values x could have taken, even if we were imagining it the whole time as “only one thing.”[14] But I don’t think I appreciated the great demand that “fix an arbitrary…” proofs in general, and \delta\epsilon proofs in particular, place on this abstraction. The mastery of it that is needed here goes far beyond what is needed to get you through years of pushing x around.

Conclusion: respect the subtlety

I offer the above anecdotes primarily as grist for reflection about learning, and especially about the nature of the particular landscape students tread as they encounter \epsilon and \delta.[15] But I would like to articulate some lessons and reminders that I myself draw from them:

(1) A complete understanding of a concept might require to go beyond mastery of its internal structure and its downstream implications, to include an understanding of its purpose, i.e. the situation it was designed to respond to.

(2) Work that successfully responds to the standard set of prompts may still conceal important gaps in understanding, as mine did in my undergraduate real analysis class. More generally, do not assume because a student is “strong” that they have command of any particular thing.

(3) Conversely, take student thought seriously, even when it looks/sounds wrong. Ricky and Ty were producing unsuccessful work for very mathematically rich reasons; I learned something worthwhile by taking the time to understand what each of them was getting at. Harry’s issue, which I didn’t take seriously at the time, could have pushed my own understanding of calculus forward — in fact, it did, albeit belatedly.

Finally, I hope the combination of these anecdotes with the history above serves as a reminder both of the magnitude of the historical accomplishment crystallized in the Weierstrassian \epsilon\delta definition of the limit, and of the corresponding profundity of the journey students take toward its mastery.

Notes and references

[1] There is an important contemporary argument that calculus’ pride of place in the curriculum should be ceded to statistics. (For example, see the TED talk by Arthur Benjamin.) That debate is beyond the scope of this blog post.

[2] The First Encounter with a Limit. The Mathematical Gazette, Vol. 19, No. 233 (1935), pp. 109-123. (link [jstor])

[3] In addition to the 1935 Mathematical Gazette article quoted above, see, e.g., E. J. Moulton, The Content of a Second Course in Calculus, AMM Vol. 25, No. 10 (1918), pp. 429-434 (link [jstor]); E. G. Phillips, On the Teaching of Analysis, The Mathematical Gazette Vol. 14, No. 204 (1929), pp. 571-573 (link [jstor]); N. R. C. Dockeray, The Teaching of Mathematical Analysis in Schools, The Mathematical Gazette Vol. 19, No. 236 (1935), pp. 321-340 (link [jstor]); H. Scheffe, At What Level of Rigor Should Advanced Calculus for Undergraduates be Taught?, AMM Vol. 47, No. 9 (1940), pp. 635-640 (link [jstor]). I thank Dave L. Renfro for all of these references.

[4] E.g., E. Dubinsky and O. Yiparaki, On student understanding of AE and EA quantification, Research in Collegiate Mathematics Education IV, 8 (2000), pp. 239-289 (link). In this and the next two notes, the literature cited only scratches the surface.

[5] E.g., D. Tall and S. Vinner, Concept image and concept definition in mathematics with particular reference to limits and continuity, Educational Studies in Mathematics Vol. 12 (1981), pp. 151-169 (link), S. R. Williams, Models of Limit Held by College Calculus Students, Journal for Research in Mathematics Education Vol. 22, No. 3 (1991), pp. 219-236 (link [jstor]), and C. Swinyard and E. Lockwood, Research on students’ reasoning about the formal definition of limit: An evolving conceptual analysis, Proceedings of the 10th annual conference on research in undergraduate mathematics education, San Diego State University, San Diego, CA (2007) (link).

Findings about students’ informal understandings of limits that generate friction with their study of \epsilon and \delta include that they are often dynamic/motion-based (like Newton), or infinitesimals-based (like Leibniz), and meanwhile, they are also often characterized by a “forward” orientation from x to f(x) — “If you bring x close to a, it puts f(x) close to L.” This is in contrast with the \epsilon\delta definition’s “backward” orientation from f(x) to x — “To make f(x) \epsilon-close to L, you have to find a \delta to constrain x.”

[6] E.g., J. Cottrill, E. Dubinsky, D. Nichols, K. Schwingendorf, K. Thomas, D. Vidakovic, Understanding the Limit Concept: Beginning with a Coordinated Process Scheme, Journal of Mathematical Behavior Vol. 15, pp. 167-192 (1996), Swinyard and Lockwood op. cit. (which responds to Cottrill et. al.), and C. Nagle, Transitioning from introductory calculus to formal limit conceptions, For the Learning of Mathematics Vol. 33, No. 2 (2013), pp. 2-10 (link).

[7] To avoid ambiguity, the learners referred to here are myself and the students I below call Ty and Ricky. The student I call Harry illustrates a difficulty one might have without the \epsilon\delta limit.

[8] The brief account I am about to give represents an orthodox view of the history of calculus, see for example J. V. Grabiner, Who Gave You the Epsilon? Cauchy and the Origins of Rigorous Calculus, The American Mathematical Monthly Vol. 90, No. 3 (1984), pp. 185-194 (link). This orthodoxy is not without its detractors, e.g., B. Pourciau, Newton and the Notion of Limit, Historia Mathematica No. 28 (2001), pp. 18-30 (link) or H. Edwards, Euler’s Definition of the Derivative, Bulletin of the AMS Vol. 44, No. 4 (2007), pp. 575-580 (link).

Readers interested in more comprehensive accounts of the history of the \epsilon\delta limit can consult Judith Grabiner’s monograph The Origins of Cauchy’s Rigorous Calculus, MIT Press, Cambridge, MA (1981) and William Dunham’s The Calculus Gallery: Masterpieces from Newton to Lebesgue, Princeton Univ. Press, Princeton, NJ (2005). A very interesting-looking new book on the topic is David Bressoud’s Calculus Reordered: A History of the Big Ideas, Princeton Univ. Press, Princeton, (2019) (link), which takes a broader view, looking at the development of integration, differentiation, series, and limits across multiple millennia and continents, and viewing the limit as a sort of culmination driven by the needs of research mathematicians in the 19th century. Bressoud’s book also considers questions of pedagogy in relation to this history.

[9] Dunham, op. cit. (see previous footnote).

[10] All names of students are pseudonyms.

[11] D. Tall and S. Vinner. Concept image and concept definition in mathematics with particular reference to limits and continuity. Educational Studies in Mathematics Vol. 12, No. 2 (1981), pp. 151-169. (link)

[12] Relatedly, recovering that which was lost from calculus when \epsilon and \delta superseded the Leibnizian infinitesimals is often given as the rationale behind Abraham Robinson’s development of nonstandard analysis.

[13] This observation is related to the body of research indicated in note [5]. I think it is subtly different though. As I understand that research, the theme is the difficulties students have with the \epsilon\delta definition due to “interference” from their more informal understandings of limits and derivatives. In contrast, my focus here is on a difficulty Ty had not because of “interference,” but rather because he recognized (perhaps more clearly than I did) that this new definition is not actually doing the same thing, so if it was being sold it as a substitute, he was not buying.

[14] To help Ricky contextualize what she needed to do for the \epsilon proof in terms of things she already understood, I asked her to consider this proof that every square number exceeds by one the product of the two integers consecutive with its square root:

Let x be any integer. Then

(x + 1) (x - 1) = x^2 - x + x - 1
= x^2 + 0 - 1
= x^2 - 1,

so any square number x^2 is one more than the product of x+1 and x-1.

“I think of the x in this proof as every number,” she said. “But you have to relate to it as a single number during the calculation itself,” I replied. “Otherwise, how do you know that -x + x = 0?”

[15] I first encountered the metaphor of a “landscape of learning” attendant to particular mathematical topics in the writings of Catherine Twomey Fosnot and Maarten Dolk.

This entry was posted in Faculty Experiences, Mathematics Education Research, Student Experiences and tagged , , , , , , , , . Bookmark the permalink.

6 Responses to Some thoughts about epsilon and delta

  1. David Bressoud says:

    All great points. Thanks for this piece, Ben. For anyone who is interested, I wrote a series of columns on student difficulties with limits in my Launchings columns for July, August, and September of 2014: Beyond the Limit I, II, and III. And I have a chapter on Limits as the Algebra of Inequalities in the new book Calculus Reordered: A History of the Big Ideas, in which I draw on the work of Judith Grabiner.

    • Ben says:

      Thanks, David! I learned of your new book from Al Cuoco during the editorial phase, and pointed to it in note [8]. I would like to know if you feel it is being correctly characterized there; I was only able to read the beginning by press time.

      For readers interested in the Launchings columns mentioned by David, which are extremely relevant to the present piece, here are the URLS (unfortunately this blog doesn’t support links in the comments):

      https://launchings.blogspot.com/2014/06/beyond-limit-i.html
      http://launchings.blogspot.com/2014/08/beyond-limit-ii.html
      https://launchings.blogspot.com/2014/09/beyond-limit-iii.html

  2. Maksim says:

    On the “one vs many”: there is an approach to explaining the definition of limit as an adversarial game: I pick an epsilon, you have to respond with a delta. In this approach, it is clear that in every instance of a game, you have to deal with only one epsilon (the one I picked); but to have a strategy you have to be able to deal with whatever epsilon I throw at you.

    • Ben says:

      This is great. Since the point of the blog post is respect for the subtlety of the difficulties, though, allow me to take this opportunity to stay on message: the adversarial paradigm was *already how Ricky understood the definition.* Indeed, it helped her attain sufficient mastery of the definition to be able to state it with a feeling that she understood all its parts.

      But the difficulty discussed above came after this, revealing itself in the context of work on specific proofs. I’m speculating here, but perhaps one way to see it is that she was struggling with the idea of a uniform strategy; or else with the notion that a uniform strategy can be described in terms of a single (but generic) epsilon.

      For what it’s worth, the analogy mentioned in note [14] did shift something for her, because she felt she understood how the algebraic proof worked.

  3. Japheth Wood says:

    The question of whether epsilon is one positive real number or all of them is a real sticking point. One helpful device that I enjoy using in my teaching is Susanna Epp’s use of a “generic particular” to prove a universally quantified statement:

    Method of Generalizing from the Generic Particular
    To show that every element of a set satisfies a certain property, suppose x is a particular but arbitrarily chosen element of the set, and show that x satisfies the property.
    (from Discrete Mathematics with Applications by Susanna S. Epp)

    Anecdotally, I observed that my students with an understanding of “generic particular” were able to produce more coherent proofs of universally quantified statements, and to understand the proofs process in deeper way.

    • Ben Blum-Smith says:

      Seems like you already found your way to my related post (https://blogs.ams.org/matheducation/2020/05/20/the-things-in-proofs-are-weird-a-thought-on-student-difficulties/) on the strangeness of the generic particular! (I’ve never heard this phrase before, thank you for introducing it to me!)

      I’m extremely interested in the question of how students develop an understanding of this idea and what steps instructors can take to support this development (cf. that other post). The quote from Susanna Epp you give directs students to the right path to take, but there’s also the question of how a student gets convinced that that’s the right path. The tack I took with Ricky, described in note [14], did make some progress with that particular learner, but this is just one trick that worked a little bit, in one context. I’m interested in developing a more comprehensive map of the landscape of learning (in the sense the phrase is used by Cathy Fosnot in her “Young Mathematicians at Work” books) involved in developing this understanding. Excited to be in conversation with you about this!

Comments are closed.