Building a Toy Language Processor | Part 1: Introduction

I just returned from Wikipedia. As it turns out, building a compiler isn't exactly something you do off the top of your head. What's the source code we're transforming? What is the syntax of our language? What does our language structure looks like?

I'll start off with the structure of the source code:

PRINT "Hello World, how old are you?"
INPUT age
LET a = 0

WHILE age > 0 REPEAT
    PRINT age - a
    a = a + 1
ENDWHILE

If you're familiar with the programming language, BASIC, you'll notice this is a minimal copy of its structure. I'll refer to the language as STANDARD, since "standard" is a synonym for "basic" lol.

These are the current features of our language:

• Variables

• Arithmetic

• Input

• Output

• While loop

Coming later

• Functions

• Arrays

• (More features as the series evolves 😪)

Compiler Overview

Before we start implementing anything, it's worth understanding what a compiler actually does. A compiler is software that translates computer code in one programming language (the source language) into another language (the target language).

Core Structure of a Compiler

Compilers are generally divided into three major architectural components:

1. The Frontend: It analyses the source code, checks syntax, generates an Abstract Syntax Tree (AST) and flags error.
2. The Middle End: It performs optimisation to make code faster.
3. The Backend: It translates the optimised intermediate code into binary object code for a specific CPU architecture.

Phases of Compilation

Compilation takes several distinct phases to produce its result. Source code moves through these phases sequentially, and are examined during each process. These phases naturally fall into the frontend, middle end, and backend architecture. They are:

1. Lexical Analysis: This phase breaks raw code into meaningful pieces—such as identifiers, keywords, and operators—called tokens.
2. Syntax Analysis: In this phase, the parser scans for syntax errors and ensures that the tokens follow the rules of the programming language. It also generates the AST which represents the structure of the syntax code.
3. Semantic Analysis: Now that the code is syntactically correct, the compiler finds logical errors, and adds semantic information to the parsed code.
4. Code Optimisation: Perform various optimisations to improve the performance of the resulting code.
5. Target Code Generation: The phase where the compiler translates the AST into machine-readable code.

Source Code
      │
      ▼
Lexical Analysis (Frontend)
      │
      ▼
Syntax Analysis
      │
      ▼
Semantic Analysis (Middle End)
      │
      ▼
Code Optimisation
      │
      ▼
Target Code Generation (Backend)
      │
      ▼
Machine Code

Now that we have a grasp of the fundamental concepts behind a compiler, the next step is planning the phases our compiler will implement. I'll be discussing that in the next article of this series.

Thanks for reading, and stay tuned.