Compiler Lab #1 - Lexical Analysis (Scanning)

Created by Todd Cooper & Karen Lemone

This lab will prepare you to do Project, Part 1 which uses lex to generate a scanner

Below in the box on the left is a lex program that will create and run a scanner (lexical analyzer) when you click on the button Edit the text and click me

When you press the button Edit the text and click me, the following occurs:

  1. The web page will be submitted to flex, a version of lex creating a Scanner that will recognize the tokens described there. It is essentially a C program.
  2. Then the C compiler (cc), compiles this to create an executable scanner.
  3. This scanner runs the input program on the right producing a set of tokens.


You can see this list of tokens at the bottom of the output page. You might have to scroll down in your browser to see them.

Step 1 Click on the button and investigate the output.

Lex Program Scanner Input
QUESTION 1 Looking at the lex code and the output, what tokens seem to be described there?

Step 2 Change the input to the following and click on the button again:

void input_a() {
a = b3;
xyz = a + b + c - p / q;
a = xyz * ( p + q );
p = a - xyz - p;
}

QUESTION 2 What tokens didn't get recognized?

Step 3

  1. Change the lex file so that when it comes across an equal, it prints EQUAL.
  2. Now change the lex file so that it recognizes all the tokens in the program in Step 2

Step 4 Change the lex file so that it will also recognize an integer consisting of 1 or more digits

Step 5 Using lex in Unix

  1. Create a file, lex.a containing your lex code from Step 4. (You can change it to print nicer output if you wish - in fact, I recommend this; Try printing the tokens as ordered pairs, e.g., (identifier, void))
  2. Generate the C Program which is the Scanner as follows:

    $ lex lex.a

    Now let's see what files are there:

    $ ls
    lex.a lex.yy.c

    You can see that lex has created a C program called lex.yy.c. This is our Scanner, but we have to compile it first:

  3. Compile the C Program which is the Scanner:

    $ cc lex.yy.c -ll
    $ ls
    a.out lex.a lex.yy.c

    a.out is the executable scanner. Let's try it out!

  4. Running the Scanner:

    $ ./a.out
    23
    integer

Step 6 Now change your lex file so that it displays REALNUM for a real number.
A real number can be defined as a plus or minus followed by a number of digits followed by a dot ".", followed by a number of digits.

You are now ready to do Project, Part 1!

Valid XHTML 1.0 Strict