Art

The Geometry of the Endless

How artists have used mathematical structures to encode infinity within a finite frame

Learning Objectives

By the end of this module you will be able to:

  • Describe how Escher used hyperbolic geometry — specifically the Poincaré disk model — to depict infinite tilings within a bounded circle.
  • Explain what a fractal is and why self-similarity at multiple scales creates an impression of infinite depth.
  • Trace the line of influence from Islamic geometric art through Escher's visits to the Alhambra to his breakthrough tessellation works.
  • Identify what mise en abyme means and recognize it in specific visual examples.
  • Articulate why mathematical infinity and aesthetic infinity are related but not the same thing.

Core Concepts

Tessellation: The Rule That Tile the Plane

A tessellation is an arrangement of shapes that cover a flat surface without gaps or overlaps. The mathematical constraints are strict: at any vertex where tiles meet, their interior angles must sum to exactly 360 degrees. This constraint explains why only three regular polygons can tessellate the plane on their own: the equilateral triangle, the square, and the regular hexagon. No other regular polygon satisfies the angle-sum requirement. Beyond those three, there are exactly eight semiregular (Archimedean) tessellations — combinations of two or more polygon types where the same arrangement appears at every vertex.

These are not merely mathematical curiosities. Tessellating systems have been used in artistic traditions across cultures for centuries, from Roman floor mosaics to medieval Islamic tilework. The tiling rule creates the conditions for the infinite: a pattern that could always, in principle, continue.

Why tessellation matters for infinity

A tessellation has no natural endpoint. Unlike a painting bounded by its canvas, a tessellating pattern does not terminate — it simply stops being rendered. The border is always the artist's choice, never a mathematical necessity. This property makes tessellation an ideal language for suggesting infinity.

Fractals and Self-Similarity

A fractal is a structure that looks similar at different scales — zoom in on part of it, and you find something that resembles the whole. The Mandelbrot set is the canonical example: generated by applying a simple iterative formula (z² + c applied repeatedly, with the output fed back as the next input), its boundary is infinitely complex. Zooming into any portion reveals miniature echoes of the whole shape, slightly distorted, connected by filaments — an endlessly detailed coastline with no smooth stretches.

The fractal dimension — a mathematical measure lying between 1 (a line) and 2 (a filled plane) — captures how completely a fractal fills the space it inhabits. Research on aesthetic preference finds that natural scenes, mathematical fractals, and sections of classical artwork tend to cluster in a preferred fractal dimension range of approximately 1.3 to 1.5: complex enough to hold interest, ordered enough to remain legible. This intermediate range appears across coastlines, mountain profiles, and botanical forms, suggesting that this particular quality of structured complexity is deeply compatible with how human perception works.

Mise en Abyme and the Strange Loop

Mise en abyme (literally "placed into the abyss," from heraldry) is a formal technique in which an image contains a smaller copy of itself. The term was first theorized by André Gide in 1893, though the practice appears centuries earlier — Giotto's Stefaneschi Triptych (14th century) shows Cardinal Stefaneschi presenting the triptych itself to Saint Peter. The Droste effect is the same principle applied specifically to images that contain themselves infinitely.

A related but distinct concept is the strange loop, a term developed by cognitive scientist Douglas Hofstadter: a system that, by moving through its own levels, returns to its starting point. Strange loops violate linear causality — they have no clear origin or endpoint. They are self-referential in a structural way, not merely decorative.

Both concepts describe structures that gesture toward infinity by folding back on themselves. The difference is one of degree: mise en abyme nests copies of an image downward in scale; a strange loop creates a circular dependency with no exterior.

Mathematical Infinity vs. Aesthetic Infinity

It is worth drawing a distinction that becomes important throughout this module. Mathematical infinity and aesthetic infinity operate under fundamentally different logics. Mathematical infinity, in the sense used by set theory, denotes an absolutely unbounded cardinality — a genuine endlessness that can even be compared and ranked (some infinities are larger than others). Aesthetic or combinatorial infinity, by contrast, designates the point at which a system exceeds the human capacity to enumerate or exhaust — a psychological limit rather than a mathematical one.

Artists working with rule-based systems are usually in the second territory. Their works are not literally infinite — they are finite structures that produce the experience of boundlessness because no human perceiver could survey them completely. Understanding this distinction helps prevent confusion: when an artwork "encodes infinity," it usually means it instantiates a rule system whose outputs feel inexhaustible, not that it achieves mathematical infinity in Cantor's sense.


Narrative Arc

The Deep Background: Islamic Geometric Art as Theological Program

Before Escher, and before modern mathematics formalized what tessellation could mean, Islamic geometric art had already developed a sophisticated visual language for infinity.

The theological context was decisive. The prohibition against representing living beings in Islamic religious contexts — aniconism — directed artistic energy toward abstraction, calligraphy, and geometric complexity. The underlying theological concern was that image-making that depicted living creatures risked challenging God's uniqueness as creator. What emerged instead was a tradition that found spiritual meaning through pattern.

Islamic geometric patterns systematically exploit tessellation to create the visual impression of infinite extension. Artisans developed mastery over the four symmetry operations — translation, rotation, reflection, and glide reflection — and used them to build patterns that radiate from central axes, interlock and overlap, and appear to continue indefinitely beyond any physical boundary. The impression of endlessness was not incidental: tessellation's property of extending without logical endpoint provided a visual analogy to the Islamic theological concept of the infinite divine. The physical boundary of a wall or ceiling was the builder's limitation, not the pattern's.

The pattern does not end at the wall. The wall ends at the pattern.

The Renaissance Interlude: Perspective and the Vanishing Point

Running parallel to Islamic geometric tradition, European art took a different mathematical path toward representing depth. Around 1415, Filippo Brunelleschi developed the first mathematically precise system of linear perspective, based on a single vanishing point to which all parallel lines converge. Objects diminish in size according to specific mathematical ratios. The vanishing point is, in a literal sense, a representation of infinite distance collapsed to a single location on the picture plane.

Leon Battista Alberti systematized this in De pictura (1435), explicitly grounding his artistic theory in mathematics: "I will take first from the mathematicians those things with which my subject is concerned." Piero della Francesca went further, theoretically defining painting itself as a branch of geometry, proving pictorial propositions through similarity of triangles and proportional relationships.

The vanishing point is an early instance of visual infinity achieved through mathematical rule: apply the convergence principle rigorously and you produce, on a finite surface, the representation of space extending without limit. This mathematical basis for perspective governed European artistic representation until the late 19th century, when artists began to question whether the single viewpoint was the only valid framework.

Escher's Turn: The Alhambra and the Discovery of Tessellation

The connection between Islamic geometric art and 20th-century visual mathematics runs through one building: the Alhambra palace in Granada. Escher visited the Alhambra in 1922 and again in 1936, describing his second visit as "the richest source of inspiration" he had encountered. He spent days making detailed sketches of the tile patterns, studying how forms interlocked and repeated across the walls and floors.

What he observed was the operational logic of symmetry groups — thirteen of the seventeen possible plane symmetry groups are represented at the Alhambra. He wrote of the Moorish artists: "They were masters in the filling of surface with congruent figures and left no gaps". From those sketches, Escher went on to produce 137 drawings based on tessellation principles, extending the Islamic logic in a new direction: where Islamic patterns had used abstract geometric forms, Escher populated his tilings with recognizable creatures — fish, birds, lizards — that transformed into each other across the picture plane.

This move — from abstract geometry to figurative tessellation — was Escher's own invention. But the underlying mathematical logic came directly from the Alhambra.

A self-taught mathematician

Escher had no formal mathematics education beyond secondary school. He worked out the principles of tessellation, symmetry groups, and later hyperbolic geometry through visual exploration and direct collaboration with professional mathematicians — Pólya, Penrose, and Coxeter. His work repeatedly anticipated or paralleled formal mathematical discoveries without being derived from them.

Circle Limits: Hyperbolic Geometry in a Finite Frame

Escher's breakthrough into genuinely infinite structures came through a letter. In 1958, mathematician H.S.M. Coxeter sent Escher a paper referencing Jules Henri Poincaré's model of hyperbolic geometry. Inspired by this correspondence, Escher created the Circle Limit series (1958–1960) — four woodcuts that are among the most mathematically precise artworks ever produced.

The key is the Poincaré disk model: a mathematical system in which the entire infinite hyperbolic plane is represented as a finite disk. In hyperbolic geometry, space curves in the opposite direction from the familiar Euclidean plane. The Poincaré disk encodes this by representing hyperbolic lines as circular arcs that meet the disk's boundary at right angles. Crucially, a conformal transformation preserves angles even as shapes diminish in size, which is why tessellated motifs in the Circle Limits remain visually recognizable all the way to the boundary.

In the Poincaré disk model, points on the limiting circle are at infinite hyperbolic distance from any interior point. This means that the tile elements in the Circle Limits decrease infinitely in size as they approach the boundary — but never reach it, because the boundary represents a genuine mathematical infinity. The result is a visual representation of infinite recursive scaling compressed into a circle that fits on a table.

This resolves a real mathematical paradox: boundedness and infinity are not contradictory when the geometry changes. Shapes that would extend forever in Euclidean space instead diminish toward a boundary that is, in hyperbolic terms, infinitely far away. The disk boundary is not a truncation — it is the infinity itself.

The Circle Limits solve the problem of infinite extension. Two other Escher works address a different kind of infinity: the self-referential loop.

Drawing Hands (1948) shows two hands emerging from a sheet of paper, each drawing the other into existence. Neither hand can logically be said to precede the other — both depend on each other for existence, creating a closed causal cycle. This is the strange loop made literal. The work does not depict infinity as endless extension but as a recursive dependency with no ground level — a structure that, if you follow it, always returns to where you started.

Print Gallery (1956) is more structurally complex. The composition shows a young man standing in a gallery, viewing a print that depicts a town — which contains the gallery he is standing in. The image folds back on itself through an exponential transformation: the gallery interior expands outward until it encompasses the very building that contains it. A blank white void at the center of the work marks the mathematical singularity — the point at which the self-referential loop cannot close completely. In 2003, mathematicians Bart de Smit and Hendrik Lenstra proved that this composition can be treated as drawn on an elliptic curve over the field of complex numbers. They demonstrated that the blank center can be filled by a version of the entire image rotated by approximately 157.63 degrees and scaled by a factor of approximately 22.58 — confirming that the recursion, while interrupted, is mathematically coherent.

Escher left the center blank. He may not have known exactly why he had to, but the mathematicians confirmed that the blank was not a failure of imagination — it was the necessary mark of a genuine logical singularity.

Fractals and the Pollock Question

Escher worked with explicit mathematical collaboration. A parallel development in 20th-century art raised the question of whether fractal structure could emerge without intentional design.

Jackson Pollock's drip paintings are fractal structures: their painted surfaces exhibit self-similar complexity at multiple scales. Fractal dimension analysis can distinguish authentic Pollock paintings from imitations with approximately 93% accuracy — the fractal signature is measurable and consistent. The fractal dimensions of his work increased over his career, from approximately 1.0 to 1.72, suggesting increasing control over the fractal properties of the paint surface rather than random chance.

This raises a question the works themselves cannot answer: did Pollock intend the fractal structure, or was it an emergent consequence of his body's motion and the physics of poured paint? The fractal dimension range of his mature work — falling in the preferred 1.3–1.5 range found across natural scenes and traditional art — suggests that his intuitive method may have been calibrated, however unconsciously, to this perceptually resonant zone. His technique was not accidental: Pollock developed specific pouring methods and controlled body motions. The fractal complexity appears to be a result of deliberate artistic practice, not randomness.

Procedural Generation: Infinity from Algorithm

The contemporary inheritor of rule-based infinity is procedural generation — the use of finite algorithmic rules to produce unbounded variation. In digital art and game design, randomized parameters and recursive algorithms generate virtual landscapes, characters, and narratives without proportional increases in labor or storage. Vera Molnár's algorithmic art from 1968 onward exemplifies the same principle at a smaller scale: tightly bounded rule systems generating systematic variation that no viewer could fully survey.

This is combinatorial infinity in practice — not mathematical unboundedness, but a rule space so large that it exceeds any individual's capacity to explore it exhaustively. The strategy parallels what emerges in minimalist music and complex systems more broadly: infinite apparent complexity from the iteration of simple elementary rules. The artist defines the grammar; the rules do the generating.


Worked Example

Reading a Circle Limit Print

Take Circle Limit III (1959) — the one Escher himself considered the most successful of the series, because it uses white lines (which correspond to geodesics in the hyperbolic plane) rather than the less mathematically precise earlier versions.

What you see: A circular composition divided into interlocking fish that grow progressively smaller toward the outer edge. The fish near the center are large enough to count features; those near the boundary are too small to fully resolve.

What the geometry does: The disk represents the entire infinite hyperbolic plane. The center of the disk is a single location in hyperbolic space; the boundary is infinitely far away in hyperbolic terms. As you move outward, the hyperbolic distance from the center is growing faster than the Euclidean distance — the tiles that appear small to your Euclidean eye are, in hyperbolic geometry, the same size as those at the center. The diminishing is a distortion of representation, not of the actual hyperbolic objects.

What Escher achieves: By choosing the Poincaré disk model, Escher gets two things simultaneously: a bounded, portable artwork (it fits on a wall), and a genuinely infinite mathematical structure. The fish do not stop at the boundary because the pattern runs out — they stop because rendering them at that scale becomes physically impossible. The boundary is not the end of the pattern; it is where the physical medium runs out of resolution. The pattern continues, in principle, forever.

The perceptual effect: The eye is drawn outward, then discovers it cannot arrive. The boundary refuses to be a border. This is what Escher meant by "making infinity visible" — not depicting something infinite, but constructing a viewing experience in which the eye performs an infinite approach.


Annotated Case Study

Fig 1
Town (contains gallery) Viewer Gallery ⊃ Town ⊃ Gallery (loop)
Print Gallery, M.C. Escher, 1956. Lithograph.

The setup: A young man stands in a print gallery, examining a print on the wall. The print depicts a coastal town. Inside that town is a building. That building contains the gallery. The gallery contains the man. The man is looking at the print.

The mathematical move: Escher applied an exponential transformation to the composition's grid. Scale is compressed toward the center and expanded toward the edges, so that the same spatial plane appears at multiple scales simultaneously. The composition is mathematically equivalent to drawing on an elliptic curve over the complex numbers — a surface with a specific, provable self-referential structure.

The blank at the center: The white void in the middle of Print Gallery is where the loop closes — or rather, where it cannot close. It marks the mathematical singularity of a self-referential system: the point at which "the image contains itself" reaches a contradiction that cannot be resolved by any finite representation. Escher put his signature there — inside the void, at the place where the system admits it cannot complete itself.

What the mathematicians found: De Smit and Lenstra demonstrated that the blank can be filled by a version of the entire image, rotated 157.63 degrees and scaled down by a factor of 22.58. The recursion does not fail — it is mathematically sound. But it requires that the center contain an infinitely small version of the whole, which no physical print can render. Escher's blank is not a gap; it is an honest acknowledgment of where visual representation must yield to mathematical abstraction.

Why it matters: Print Gallery is the rare artwork that is literally more mathematically coherent than it appears. The structure Escher intuited — without formal training, working from visual logic alone — turns out to be an exact instance of a well-defined mathematical object. The work proves that rigorous visual reasoning can arrive at truths that formal mathematics later confirms.

Key Takeaways

  1. Tessellation encodes potential infinity. A tessellating pattern has no mathematical endpoint — only physical limits set by material and scale. This property was exploited in Islamic geometric art as a theological statement about infinite divine nature, and later by Escher as a visual argument about structure.
  2. Escher's Circle Limit series achieves genuine mathematical infinity in finite form. By using the Poincaré disk model of hyperbolic geometry, the Circle Limits compress an infinite hyperbolic plane into a bounded circle. The boundary of the disk is not a truncation but a genuine mathematical infinity — infinitely far away in hyperbolic distance.
  3. Escher's debt to Islamic art is direct and documented. His visits to the Alhambra in 1922 and 1936 gave him the operational logic of symmetry groups and the aesthetic conviction that repeating patterns could extend indefinitely. He then transformed that logic by replacing abstract geometric motifs with figurative creatures.
  4. Mise en abyme and strange loops are distinct but related structures. Mise en abyme nests copies of an image within itself, creating an appearance of infinite regression. Strange loops create circular causal dependencies with no ground level. Both gesture toward infinity through self-reference rather than extension.
  5. Mathematical infinity and aesthetic infinity are not the same thing. Mathematical infinity denotes absolute unboundedness; aesthetic infinity denotes the point at which a system exceeds human capacity to enumerate. Most artists working with rule-based systems are producing the second, not the first.

Further Exploration

On Escher and hyperbolic geometry

On Print Gallery and its mathematics

On Islamic geometric art

On fractals

On mise en abyme

  • Mise en abyme — History and examples across visual art, literature, and film.

On infinity in philosophy and mathematics

  • Infinity — Stanford Encyclopedia of Philosophy — Rigorous account of the distinction between mathematical and philosophical infinity.