Some thoughts about epsilon and delta

By Ben Blum-Smith, Contributing Editor

The calculus has a very special place in the 20th century’s traditional course of mathematical study. It is a sort of fulcrum: both the summit toward which the whole secondary curriculum strives, and the fundamental prerequisite for a wide swath of collegiate and graduate work, both in mathematics itself and in its applications to the sciences, economics, engineering, etc.^[1] At its heart is the notion of the limit, described in 1935 by T. A. A. Broadbent as the critical turning point:

The first encounter with a limit marks the dividing line between the elementary and the advanced parts of a school course. Here we have not a new manipulation of old operations, but a new operation; not a new trick, but a new idea.^[2]

Humanity’s own collective understanding of this “new idea” was hard-earned. The great length of the historical journey toward the modern definition in terms of and mirrors the well-known difficulty students have with it. Although it is the foundation of calculus, it is common to push the difficulty of this definition off from a first calculus course onto real analysis. Indeed, mathematicians have been discussing the appropriate place for the full rigors of this definition in the calculus curriculum for over 100 years.^[3]

There is also a rich vein in the mathematics education research literature studying students’ engagement with the – definition. Researchers have examined student difficulties coming from its multiple nested quantifiers^[4] as well as its great distance from the less formal notions of limit with which students typically enter its study,^[5] and have also made an effort to chart the paths they take toward a full understanding.^[6]

This blog post is a contribution to this conversation, analyzing in detail three learners’ difficulties with and .^[7] If there is a take-home message, it is to respect the profound subtlety of this definition and the complexity of the landscape through which students need to move as they learn to work with it.

Some history

Many readers will be familiar with the long struggle to find a rigorous underpinning for the calculus of Newton and Leibniz, leading to the modern definition of the limit in terms of and . In this section, I excerpt a few episodes, which will become important in the later discussion of student thought. Readers already familiar with the history of the subject are welcome to skim or skip this section.

While Newton and Leibniz published their foundational work on (what we now call) derivatives and integrals in the late 17th century, they based these ideas not on the modern limit, but on notions that look hand-wavy in retrospect.^[8] To Leibniz, the derivative, for example, was a ratio of “infinitesimal” quantities — smaller than finite quantities, but not zero. To Newton, it was an “ultimate ratio”, the ratio approached by a pair of quantities as they both disappear. Both authors would calculate the derivative of via a now-familiar manipulation: augment by a small amount ; correspondingly, augments to . The ratio of the change in to the change in is , or . At this point, they would differ in their explanation of why you can ignore all the terms involving in this last expression: for Leibniz, it is because they are infinitesimal, and for Newton, it is because they all vanish when the augmentation of is allowed to vanish.

A famous critique of both of these lines of reasoning was leveled in 1734 by the British philosopher and theologian Bishop George Berkeley, arguing that since to form the ratio of to in the first place, it was necessary to assume is nonzero, it is strictly inconsistent to then decide to ignore it.

Hitherto I have supposed that flows, that hath a real Increment, that is something. And I have proceeded all along on that Supposition, without which I should not have been able to have made so much as one single Step. From that Supposition it is that I get at the Increment of , that I am able to compare it with the Increment of , and that I find the Proportion between the two Increments. I now beg leave to make a new Supposition contrary to the first, i.e., I will suppose that there is no Increment of , or that is nothing; which second Supposition destroys my first, and is inconsistent with it, and therefore with every thing that supposeth it. I do nevertheless beg leave to retain , which is an Expression obtained in virtue of my first Supposition, which necessarily presupposeth such Supposition, and which could not be obtained without it: All which seems a most inconsistent way of arguing…

It was a long journey from the state of the art in the early 18th century, to which Berkeley was responding, to the modern reformulation of calculus on the basis of the – limit. The process took well over a century. I will summarize this story by quoting somewhat telegraphically from William Dunham’s book The Calculus Gallery,^[9] from which I first learned it.

Berkeley penned the now famous question:

… They are neither finite quantities nor quantities infinitely small, nor yet nothing. May we not call them the ghosts of departed quantities?

… Over the next decades a number of mathematicians tried to shore up the shaky underpinnings… pp. 71-72

… Cauchy’s “limit-avoidance” definition made no mention whatever of attaining the limit, just of getting and staying close to it. For him, there were no departed quantities, and Berkeley’s ghosts disappeared… p. 78

… If his statement seems peculiar, his proof began with a now-familiar ring, for Cauchy introduced two “very small numbers” and … p. 83

… We recall that Cauchy built his calculus upon limits, which he defined in these words:

When the values successively attributed to a variable approach indefinitely to a fixed value, in a manner so as to end by differing from it by as little as one wishes, this last is called the limit of all the others.

To us, aspects of this statement, for instance, the motion implied by the term “approach,” seem less than satisfactory. Is something actually moving? If so, must we consider concepts of time and space before talking of limits? And what does it mean for the process to “end”? The whole business needed one last revision.

Contrast Cauchy’s words with the polished definition from the Weierstrassians:

if and only if, for every , there exists a so that, if , then .

Here nothing is in motion, and time is irrelevant. This is a static rather than dynamic definition and an arithmetic rather than a geometric one. At its core, it is nothing but a statement about inequalities. pp. 129-130

The Weierstrassian definition (i.e., the modern one!) allows the manipulation to which Berkeley objected to be carried to completion without ever asking to be zero.

Missing the point completely

The core of this blog post is a discussion of three learners’ encounters with the – limit, seeking to illuminate some of the subtle challenges that can arise. I begin with my own story.

I “did well” in my college real analysis class, by which I mean that my instructor (a well-known analyst at a major research university) concluded on the basis of my written output that I had mastered the material. However, I walked away from the course with a subtle but important misunderstanding of the – definition that was not visible in my written work and so went entirely undetected by my instructor, and, for many years, by myself as well.

From my previous experience with calculus, I had concluded that you can often identify the value toward which a function is headed, even if the function is undefined “when you get there.” To the extent I had a definition of limit, this was it: the value toward which the function is headed.

When I studied real analysis as an undergraduate, I found the class easy, including the – work. I mean, if is where is headed as , then sure, for any -neighborhood around , there is going to be a -neighborhood around that puts inside the -neighborhood. But I related to the notion that is headed toward as conceptually prior to this – game. The latter seemed like fancy window-dressing to me, possibly useful post-hoc if you need an error estimate. I did not understand that it was a definition — that it was supposed to be constitutive of the meaning of limit. So, I completely missed the point! But I want to stress that you would not have known this from my written work, and of course, I didn’t know it either.

I went on to become a high school math teacher. In the years that followed, I did detect certain limitations in my understanding of limits. For example, I noticed that I didn’t have adequate tools for thinking about if and when the order of two different limit processes could be safely interchanged. But it did not cross my mind that the place to start in thinking clearly about this was a tool I had already been given.

After a few years, I began teaching an AP Calculus course. About 3/4 of the way through my first time teaching it, my student Harry^[10] said to me after class, “You know this whole class is based on a paradox, right?” He proceeded to give me what I now recognize as essentially Bishop Berkeley’s critique. At the time, it did not occur to me to reach for epsilon and delta. Instead, I responded like an 18th century mathematician, trying to convince him that the terminus of an unending process is something it’s meaningful to talk about. I hadn’t really understood what the problem was. Of course, Harry left unsatisfied.

The pieces finally came together for me the next year, when I read Dunham’s Calculus Gallery, quoted above. I remember the shift in my understanding: ooooohhhhhh. The ‘s and ‘s are not an addendum to, or a gussying-up of, the idea of identifying where an unending process is headed. They are replacing this idea! It was a revelation to reread the definition from this new point of view. Calculus does not need the infinitesimal! I immediately wished I had a do-over with Harry, whose dissatisfaction I hadn’t comprehended enough to be able to speak to.

I concluded from this that a complete understanding of the – definition includes an understanding of what it’s for.

But it’s not the same thing at all

Having come to this conclusion, in my own teaching of real analysis I’ve made a great effort to make clear the problem that and are responding to. In one course, I began with a somewhat in-depth study of Berkeley’s critique of the 18th century understanding of the calculus, in order to then be able to offer and as an answer to that critique.

In doing this, I ran into a new challenge. To illustrate, I’ll focus on the experience of a student named Ty. Ty arrived in my course having already developed a fairly strong form of a more intuitive, 17th-18th century understanding of the limit; essentially the Newtonian one, much like the understanding that had carried me myself through all my calculus coursework. He quickly made sense of Berkeley’s objection, so he was able to see that this understanding was not mathematically satisfactory. I was selling the – definition as a more satisfactory substitute. However, Ty objected that important aspects of his understanding of the limit (what Tall and Vinner called his concept image^[11]) were not captured by this new definition. In particular, what had happened to the notion that the limit was something toward which the function was, or even should have been, headed? The – definition of studiously avoids the point “at which the limit is taken,” even speculatively. To Ty, it was the – definition that was, pun intended, missing the point.

Of course, this studious avoidance is precisely how the Weierstrassian definition gets around Berkeley’s objection. The Newtonian “ultimate ratio” and the Leibnizian “infinitesimal” both ask us to imagine something counterfactual, or at least pretty wonky. This is exactly what made them hard for Berkeley to swallow, and as I learned from Dunham’s book, the great virtue of and $\delta$ is that they give us a way to uniquely identify the limit that does not ask us to engage in such a trippy flight of fancy that may or may not look sane in the light of day.

But, at the same time, something is lost.^[12] What I learned from Ty is that this loss is pedagogically important to acknowledge.^[13]

One vs. many

Another subtle difficulty in working with the – definition is revealed when you use it to try to prove something. I think what I am about to describe is a general difficulty students encounter in learning the methods and conventions of proof-writing, but I speculate that it may be particularly acute with respect to the present topic. Consider this (utterly standard) proof that if are functions of $x$ such that and , and , then :

Let be given. Because , there exists such that implies . Similarly, because , there exists such that implies .

Take .

Then for values of satisfying , it follows from the triangle inequality and the definition of that

.

Since was arbitrary, we can conclude that .

Here is a surprisingly rich question: is the in the proof one number, or many numbers?

On one way of looking at it, of course it is only one number: is fixed at the outset of the proof. Indeed, if were allowed to be more than one thing, equations like $\epsilon / 2 + \epsilon / 2 = \epsilon$ would be meaningless. More subtly, we usually speak about as a single fixed quantity when we justify the existence of in terms of the definition of the limit: we know exists because by the definition of the limit, for any there is a , so in particular, there is a for , etc. Note the “in particular”: we produce from the definition by specializing it.

But on another way of looking at it, of course $\epsilon$ is many numbers. Indeed, it must represent every positive number, otherwise how can it be used to verify the definition for all ? The singular, fixed with which we work in the proof is a sort of chimera: it actually represents all positive numbers at once. That we think of it as a single number is just a psychological device to allow us to focus in a productive way on what we are doing with all these numbers.

This dual nature of in the above was driven home for me by working with Ricky. Fast and accurate with calculations and algebraic manipulations, Ricky was thrown for a loop by real analysis, which was her first proof-based class, and in particular by the – proofs. After a lot of work, she had mastered the definition itself. But in trying to write the proofs, she found the lilting refrain for all … to be a kind of siren song, leading her astray. She was constantly re-initializing with this phrase, so that reading her work, there were 3 different meanings for by the end. “Look at how the proof works,” I would say, referring to the proof of above. “You don’t need to be less than any old . You need it to be less than the particular that you are using for .” “What do you mean the particular I am using?” she would ask. “I am trying to prove it works for all !”

Ricky’s difficulty has led me to a much greater appreciation of the subtle and profound abstraction involved any time an object is introduced into a proof with the words “fix an arbitrary…” In a sense, this is nothing more — nor less! — than the abstraction at the heart of a student’s first encounter with algebra: if we imagine an unspecified number , and proceed to have a conversation about it, our conversation applies simultaneously at once to all the values could have taken, even if we were imagining it the whole time as “only one thing.”^[14] But I don’t think I appreciated the great demand that “fix an arbitrary…” proofs in general, and – proofs in particular, place on this abstraction. The mastery of it that is needed here goes far beyond what is needed to get you through years of pushing around.

Conclusion: respect the subtlety

I offer the above anecdotes primarily as grist for reflection about learning, and especially about the nature of the particular landscape students tread as they encounter and .^[15] But I would like to articulate some lessons and reminders that I myself draw from them:

(1) A complete understanding of a concept might require to go beyond mastery of its internal structure and its downstream implications, to include an understanding of its purpose, i.e. the situation it was designed to respond to.

(2) Work that successfully responds to the standard set of prompts may still conceal important gaps in understanding, as mine did in my undergraduate real analysis class. More generally, do not assume because a student is “strong” that they have command of any particular thing.

(3) Conversely, take student thought seriously, even when it looks/sounds wrong. Ricky and Ty were producing unsuccessful work for very mathematically rich reasons; I learned something worthwhile by taking the time to understand what each of them was getting at. Harry’s issue, which I didn’t take seriously at the time, could have pushed my own understanding of calculus forward — in fact, it did, albeit belatedly.

Finally, I hope the combination of these anecdotes with the history above serves as a reminder both of the magnitude of the historical accomplishment crystallized in the Weierstrassian – definition of the limit, and of the corresponding profundity of the journey students take toward its mastery.

Notes and references

[1] There is an important contemporary argument that calculus’ pride of place in the curriculum should be ceded to statistics. (For example, see the TED talk by Arthur Benjamin.) That debate is beyond the scope of this blog post.

[2] The First Encounter with a Limit. The Mathematical Gazette, Vol. 19, No. 233 (1935), pp. 109-123. (link [jstor])

[3] In addition to the 1935 Mathematical Gazette article quoted above, see, e.g., E. J. Moulton, The Content of a Second Course in Calculus, AMM Vol. 25, No. 10 (1918), pp. 429-434 (link [jstor]); E. G. Phillips, On the Teaching of Analysis, The Mathematical Gazette Vol. 14, No. 204 (1929), pp. 571-573 (link [jstor]); N. R. C. Dockeray, The Teaching of Mathematical Analysis in Schools, The Mathematical Gazette Vol. 19, No. 236 (1935), pp. 321-340 (link [jstor]); H. Scheffe, At What Level of Rigor Should Advanced Calculus for Undergraduates be Taught?, AMM Vol. 47, No. 9 (1940), pp. 635-640 (link [jstor]). I thank Dave L. Renfro for all of these references.

[4] E.g., E. Dubinsky and O. Yiparaki, On student understanding of AE and EA quantification, Research in Collegiate Mathematics Education IV, 8 (2000), pp. 239-289 (link). In this and the next two notes, the literature cited only scratches the surface.

[5] E.g., D. Tall and S. Vinner, Concept image and concept definition in mathematics with particular reference to limits and continuity, Educational Studies in Mathematics Vol. 12 (1981), pp. 151-169 (link), S. R. Williams, Models of Limit Held by College Calculus Students, Journal for Research in Mathematics Education Vol. 22, No. 3 (1991), pp. 219-236 (link [jstor]), and C. Swinyard and E. Lockwood, Research on students’ reasoning about the formal definition of limit: An evolving conceptual analysis, Proceedings of the 10th annual conference on research in undergraduate mathematics education, San Diego State University, San Diego, CA (2007) (link).

Findings about students’ informal understandings of limits that generate friction with their study of and include that they are often dynamic/motion-based (like Newton), or infinitesimals-based (like Leibniz), and meanwhile, they are also often characterized by a “forward” orientation from to — “If you bring close to , it puts close to .” This is in contrast with the – definition’s “backward” orientation from to — “To make -close to , you have to find a to constrain .”

[6] E.g., J. Cottrill, E. Dubinsky, D. Nichols, K. Schwingendorf, K. Thomas, D. Vidakovic, Understanding the Limit Concept: Beginning with a Coordinated Process Scheme, Journal of Mathematical Behavior Vol. 15, pp. 167-192 (1996), Swinyard and Lockwood op. cit. (which responds to Cottrill et. al.), and C. Nagle, Transitioning from introductory calculus to formal limit conceptions, For the Learning of Mathematics Vol. 33, No. 2 (2013), pp. 2-10 (link).

[7] To avoid ambiguity, the learners referred to here are myself and the students I below call Ty and Ricky. The student I call Harry illustrates a difficulty one might have without the – limit.

[8] The brief account I am about to give represents an orthodox view of the history of calculus, see for example J. V. Grabiner, Who Gave You the Epsilon? Cauchy and the Origins of Rigorous Calculus, The American Mathematical Monthly Vol. 90, No. 3 (1984), pp. 185-194 (link). This orthodoxy is not without its detractors, e.g., B. Pourciau, Newton and the Notion of Limit, Historia Mathematica No. 28 (2001), pp. 18-30 (link) or H. Edwards, Euler’s Definition of the Derivative, Bulletin of the AMS Vol. 44, No. 4 (2007), pp. 575-580 (link).

Readers interested in more comprehensive accounts of the history of the – limit can consult Judith Grabiner’s monograph The Origins of Cauchy’s Rigorous Calculus, MIT Press, Cambridge, MA (1981) and William Dunham’s The Calculus Gallery: Masterpieces from Newton to Lebesgue, Princeton Univ. Press, Princeton, NJ (2005). A very interesting-looking new book on the topic is David Bressoud’s Calculus Reordered: A History of the Big Ideas, Princeton Univ. Press, Princeton, (2019) (link), which takes a broader view, looking at the development of integration, differentiation, series, and limits across multiple millennia and continents, and viewing the limit as a sort of culmination driven by the needs of research mathematicians in the 19th century. Bressoud’s book also considers questions of pedagogy in relation to this history.

[9] Dunham, op. cit. (see previous footnote).

[10] All names of students are pseudonyms.

[11] D. Tall and S. Vinner. Concept image and concept definition in mathematics with particular reference to limits and continuity. Educational Studies in Mathematics Vol. 12, No. 2 (1981), pp. 151-169. (link)

[12] Relatedly, recovering that which was lost from calculus when and superseded the Leibnizian infinitesimals is often given as the rationale behind Abraham Robinson’s development of nonstandard analysis.

[13] This observation is related to the body of research indicated in note [5]. I think it is subtly different though. As I understand that research, the theme is the difficulties students have with the – definition due to “interference” from their more informal understandings of limits and derivatives. In contrast, my focus here is on a difficulty Ty had not because of “interference,” but rather because he recognized (perhaps more clearly than I did) that this new definition is not actually doing the same thing, so if it was being sold it as a substitute, he was not buying.

[14] To help Ricky contextualize what she needed to do for the proof in terms of things she already understood, I asked her to consider this proof that every square number exceeds by one the product of the two integers consecutive with its square root:

Let be any integer. Then

,

so any square number is one more than the product of and .

“I think of the in this proof as every number,” she said. “But you have to relate to it as a single number during the calculation itself,” I replied. “Otherwise, how do you know that ?”

[15] I first encountered the metaphor of a “landscape of learning” attendant to particular mathematical topics in the writings of Catherine Twomey Fosnot and Maarten Dolk.