The Timpson Lyceum · Computation Theory

Lecture 15 (Computation Theory) Decidable and Undecidable Languages

Decidable and Undecidable Languages – Lecture 15

Overview

This lecture surveys which language-recognition problems can be solved by an algorithm and which cannot. Building on the Church–Turing thesis (an algorithm is exactly what a Turing machine can do) and the existence of a universal Turing machine, we prove that a whole family of natural questions about DFAs, NFAs, regular expressions, and context-free grammars are decidable. We then cross into undecidability: the acceptance problem for Turing machines, $A_{TM}$, is recognizable but not decidable. To set up that impossibility proof we develop Cantor's theory of set size — bijections, countable sets, and the diagonalization argument showing the real numbers are uncountable. The thread tying it together is the difference between being able to confirm "yes" answers and being able to settle every answer, yes or no.

Key Concepts

The language hierarchy. Every regular language is context-free, every context-free language is decidable, and every decidable language is Turing-recognizable. Each containment is strict.

Decidable vs. recognizable.

• A language is decidable if some Turing machine halts on every input and answers correctly: it accepts every string in the language and rejects every string not in it (it can confirm both "yes" and "no").
• A language is (Turing-)recognizable if some Turing machine accepts exactly the strings in the language, but may loop forever on strings not in it (it can confirm "yes," but not necessarily "no").

Encodings. Angle brackets denote an encoding of an object as a string, e.g. $\langle D, w\rangle$ is the DFA $D$ and string $w$ packaged as a single input. Languages are sets of strings, so any machine, grammar, or pair must first be encoded.

Decidable problems about regular languages. Each of these languages is decidable:

• Acceptance for DFAs: $A_{DFA} = \{\langle D, w\rangle \mid D \text{ is a DFA that accepts } w\}$. Decided by simulating $w$ on $D$; a DFA always halts after reading the finite input, so we accept iff it ends in an accept state.
• Acceptance for NFAs: $A_{NFA}$. Decided by first converting the NFA to an equivalent DFA, then calling the $A_{DFA}$ decider as a subroutine.
• Acceptance for regular expressions: $A_{REX}$. Same idea — convert the regular expression to an equivalent DFA, then reuse the $A_{DFA}$ decider.
• Emptiness for DFAs: $E_{DFA} = \{\langle D\rangle \mid L(D) = \varnothing\}$. Decided by a marking (reachability) procedure: mark the start state, repeatedly mark any state reachable from a marked state until nothing new is marked, then accept iff no accept state was marked.
• Equality for DFAs: $EQ_{DFA} = \{\langle D_1, D_2\rangle \mid L(D_1) = L(D_2)\}$. Decided using the symmetric difference $L(D_1)\,\triangle\,L(D_2) = \big(L(D_1)\cap \overline{L(D_2)}\big)\cup\big(\overline{L(D_1)}\cap L(D_2)\big)$. Because regular languages are closed under union, intersection, and complement, this is regular; build a DFA $D$ for it and run the $E_{DFA}$ decider. The two languages are equal iff their symmetric difference is empty.

Decidable problems about context-free languages.

• Chomsky normal form (CNF): every rule has the form $A \to BC$ (two variables) or $A \to a$ (one terminal). Every CFG can be converted to CNF. Key fact: a derivation of a string of length $n$ in CNF takes exactly $2n-1$ steps ($n-1$ to build the variables, $n$ to convert them to terminals).
• Acceptance for CFGs: $A_{CFG} = \{\langle G, w\rangle \mid G \text{ generates } w\}$. Decided by converting $G$ to CNF and exhaustively listing all derivations of length $2n-1$ (where $n = |w|$); accept iff $w$ appears. Only finitely many such derivations exist, so the search always terminates.
• Emptiness for CFGs: $E_{CFG}$. Decided by a marking procedure that works backwards from the terminals: mark all terminals, then mark any variable $A$ all of whose right-hand-side symbols (for some rule) are marked, repeat until stable, and reject (non-empty) iff the start variable gets marked.
• Every context-free language is decidable: given a CFL $A$, there is a CFG $G$ generating it; the decider for $A$ hard-codes $G$ and runs the $A_{CFG}$ decider on $\langle G, w\rangle$.

Acceptance for Turing machines is undecidable. $A_{TM} = \{\langle M, w\rangle \mid M \text{ is a TM that accepts } w\}$ is recognizable — the universal TM simulates $w$ on $M$ and accepts if $M$ accepts. But we cannot make it a decider: if $M$ loops, the simulation never returns, and unlike DFAs/CFGs there is no finite bound on the number of steps after which a TM must halt. $A_{TM}$ turns out to be recognizable but undecidable, and its complement is not even recognizable.

Cardinality and bijections. Two sets have the same cardinality, $|A| = |B|$, iff there is a bijection $f : A \to B$ — a map that is onto (every element of $B$ is hit, so $B$ is not larger than $A$) and one-to-one (distinct inputs give distinct outputs, so $A$ is not larger than $B$).

Countable vs. uncountable. A set is countable if it is finite or has the same cardinality as $\mathbb{N}$. Examples of countably infinite sets: the even naturals, the integers $\mathbb{Z}$, and (via dovetailing through the grid of fractions) the rationals $\mathbb{Q}$. A set that is neither finite nor the size of $\mathbb{N}$ is uncountable.

Diagonalization — the reals are uncountable. Assume a bijection $f : \mathbb{N} \to (0,1)$ exists. Build a real $x \in (0,1)$ whose $n$-th decimal digit differs from the $n$-th digit of $f(n)$ (e.g. use $8$ if that digit is $9$, otherwise add $1$). Then $x \neq f(n)$ for every $n$, so $x$ is missed — $f$ is not onto, contradicting that it was a bijection. No such bijection exists, so $(0,1)$ (and hence $\mathbb{R}$) is uncountable. This counting machinery is the engine behind proving $A_{TM}$ undecidable.

Worked Example

Claim: the even natural numbers $E = \{n \in \mathbb{N} \mid n \text{ even}\}$ have the same cardinality as $\mathbb{N}$.

To show $|\mathbb{N}| = |E|$ we exhibit a bijection $f : \mathbb{N} \to E$ and verify it is onto and one-to-one.

Every even natural is the image of some natural (onto), and equal outputs force equal inputs (one-to-one), so $f$ is a bijection and $|\mathbb{N}| = |E|$ — halving an infinite set leaves its cardinality unchanged.

Summary

• Decidable means a TM halts and answers correctly on every input (settles yes and no); recognizable means it accepts the yes-instances but may loop on the no-instances.
• All of $A_{DFA}$, $A_{NFA}$, $A_{REX}$, $E_{DFA}$, $EQ_{DFA}$, $A_{CFG}$, $E_{CFG}$ are decidable, and every context-free language is decidable — proofs reuse earlier deciders as subroutines and rely on finite halting bounds (DFAs halt after reading input; CNF derivations of $w$ have exactly $2n-1$ steps).
• $EQ_{DFA}$ is decided via the symmetric difference: $L(D_1) = L(D_2)$ iff $L(D_1)\,\triangle\,L(D_2) = \varnothing$, which works because regular languages are closed under union, intersection, and complement.
• $A_{TM}$ is recognizable (just simulate via the universal TM) but undecidable, because a TM has no finite halting bound and may loop forever.
• Cantor's framework defines equal size via a bijection; $\mathbb{N}$, $\mathbb{Z}$, and $\mathbb{Q}$ are all countable, but diagonalization shows $\mathbb{R}$ is uncountable — the key tool for the undecidability results to come.