The Timpson Lyceum · Computation Theory

Lecture 14 (Computation Theory) Church-Turing Thesis and the Definition of an Algorithm

Overview

This lecture pivots from the mechanics of Turing machines to the question they were invented to answer: what is an algorithm? It traces the historical motivation — Hilbert's 1900 list of 23 problems, and especially Hilbert's tenth problem on Diophantine equations — that exposed why a precise definition of "algorithm" was needed. It introduces the Church–Turing thesis: the claim that the informal notion of algorithm is exactly captured by what a Turing machine (equivalently, Church's lambda calculus) can compute. The lecture then practices high-level algorithm descriptions by proving several languages recognizable or decidable, and closes with the real-world consequences of Turing's work: the universal (reprogrammable) machine and the codebreaking that grew out of it.

Key Concepts

Recognizable vs. decidable (review).

• A language is Turing-recognizable if some Turing machine accepts every string in the language (it may loop forever or reject on strings not in the language).
• A language is decidable if some Turing machine accepts every string in the language and rejects every string not in the language — it always halts with a yes/no answer.
• Every decidable language is recognizable; the decidable languages are a proper subset of the recognizable ones.

Why a definition of algorithm was needed.

• Before Turing and Church, "algorithm" was understood only intuitively — e.g. Euclid's algorithm for the greatest common divisor is clearly a finite, mechanical procedure.
• Intuition is enough to recognize an algorithm when you see one, but it cannot prove that no algorithm exists for a given problem. Proving impossibility requires an exact, formal definition of computation.

Hilbert's problems (1900). David Hilbert posed 23 problems to guide the century's mathematics. Several turned out to have no resolution in Hilbert's intended sense:

• Problem 1 (Continuum Hypothesis): whether a set exists with size strictly between $|\mathbb{Z}|$ and $|\mathbb{R}|$ — shown to be independent of the standard axioms (must be taken as an axiom either way).
• Problem 2 (consistency of arithmetic): Gödel's incompleteness theorems show a sufficiently strong system cannot prove its own consistency.
• Problem 10 (Hilbert's tenth problem): find an algorithm that decides whether an arbitrary Diophantine equation has an integer solution.

Diophantine equations. A polynomial in any number of variables with integer coefficients, set equal to zero, where only integer solutions count. For example $x^3 + y^2 - z + 10 = 0$ has the integer solution $x=5,\,y=3,\,z=0$. An "integer root" of the polynomial means integer values for all variables that make it evaluate to $0$.

The Church–Turing thesis.

• In 1936, Alonzo Church proposed a formal definition of algorithm using lambda calculus; his student Alan Turing independently proposed one using Turing machines (Turing's paper, On Computable Numbers, with an Application to the Entscheidungsproblem).
• The two definitions are logically equivalent: any procedure one calls an algorithm, the other does too. Other later proposals have all proven equivalent as well, suggesting a very robust notion.
• The thesis is a claim, not a theorem: it asserts that the intuitive idea of "algorithm" is precisely what a Turing machine can do. Turing machines are easier to reason about than lambda calculus, which is one reason Turing's formulation became dominant.
• Hilbert's tenth problem was eventually proven to have no algorithm — its language is undecidable — but it required exactly this kind of formal definition to prove the negative.

Encoding inputs as strings. A Turing machine takes only strings. Any object $O$ (a polynomial, a graph, even another Turing machine) is first encoded as a string, written $\langle O \rangle$. The encoding scheme does not matter, only that one exists.

Universal Turing machine. Turing described a single machine $U$ that takes $\langle M, w\rangle$ — the encoding of a Turing machine $M$ together with an input string $w$ — and simulates $M$ running on $w$.

• If $M$ accepts $w$, then $U$ accepts; if $M$ rejects $w$, then $U$ rejects; if $M$ loops on $w$, then $U$ loops.
• A program is simply an encoded Turing machine, so $U$ is the theoretical model of a reprogrammable computer — one machine that runs any software, configured by code rather than by rewiring hardware.

Historical consequences.

• Enigma: a rotor cipher machine. A single rotor rewires the 26 letters and advances by one position after each keypress; three rotors chained give $26^3$ wheel positions, times $3! = 6$ rotor orderings. Crucially the wiring is reciprocal (if A maps to Q, then Q maps to A) and no letter ever maps to itself — a weakness Turing exploited.
• The German military version used 5 rotors choose 3 in order ($5\cdot4\cdot3$ ways) plus a plugboard swapping pairs of letters, exploding the keyspace into the hundreds of millions and beyond.
• Turing's bombe systematically searched rotor/plugboard configurations. Like early purpose-built machines, it was wired for one task — when the settings changed the hardware became near-worthless, illustrating why the universal (software-driven) machine was such a leap.
• An early universal electronic machine (ca. 1945) solved in ~30 seconds artillery-trajectory calculations that a human "computer" took ~20 hours to do — roughly 2,400× faster, paying back its ~$1M cost within months.

Worked Example

Claim. Let $D_1 = \{\,\langle p \rangle : p \text{ is a single-variable polynomial over } x \text{ with an integer root}\,\}$. Then $D_1$ is recognizable.

Recognizing strategy: test integer values of $x$ in an order that is guaranteed to reach every integer, and accept if any value drives $p$ to $0$. Enumerating $0, 1, 2, 3, \dots$ would never reach the negatives, so interleave them.

If $p$ has an integer root, $M_1$ eventually tests it and accepts; if not, $M_1$ runs forever. Hence $M_1$ recognizes (but does not decide) $D_1$.

Extending to two variables. For $D_2$ (polynomials over $x$ and $y$), enumerate ordered pairs $(x,y)$ by dovetailing — sweeping the integer grid in expanding diagonals so every pair is eventually reached — and accept if any pair gives $p(x,y)=0$. The same idea generalizes to $k$ variables by dovetailing over a $k$-dimensional grid, so the full Diophantine language is recognizable (though not decidable).

A decidable contrast (connected graphs). Let $A = \{\,\langle G\rangle : G \text{ is a connected graph}\,\}$. The machine marks one node, then repeatedly marks any node adjacent to a marked node until no new node gets marked; it accepts if all nodes are marked and rejects otherwise. Because it always halts with an explicit accept-or-reject, $A$ is decidable.

Summary

• The Church–Turing thesis identifies the informal notion of "algorithm" with what a Turing machine can compute; lambda calculus and Turing machines are logically equivalent formalizations of it.
• A formal definition of algorithm was needed not to recognize algorithms but to prove none exists for a problem — motivated by Hilbert's tenth problem.
• A Diophantine equation is an integer-coefficient polynomial sought to be zero with integer-only solutions; the language of solvable Diophantine equations is recognizable but not decidable.
• Recognizable = accept every member (may loop otherwise); decidable = always halt and correctly accept or reject. Decidable $\subsetneq$ recognizable.
• High-level algorithm descriptions (search-in-the-right-order, dovetailing, mark-and-spread) suffice once the Turing machine has fixed the precise model of computation; any object is fed in via a string encoding $\langle O\rangle$.
• The universal Turing machine simulates any encoded machine on any input — the theoretical basis for the reprogrammable computer, realized physically within a decade of Turing's 1936 paper.