GNU Info

(bison.info)Ltcalc Lexer

---

Prev: Ltcalc Rules Up: Location Tracking Calc

---

Enter node , (file) or (file)node

The `ltcalc' Lexical Analyzer.
------------------------------

   Until now, we relied on Bison's defaults to enable location
tracking. The next step is to rewrite the lexical analyser, and make it
able to feed the parser with the token locations, as it already does for
semantic values.

   To this end, we must take into account every single character of the
input text, to avoid the computed locations of being fuzzy or wrong:

     int
     yylex (void)
     {
       int c;
     
       /* skip white space */
       while ((c = getchar ()) == ' ' || c == '\t')
         ++yylloc.last_column;
     
       /* step */
       yylloc.first_line = yylloc.last_line;
       yylloc.first_column = yylloc.last_column;
     
       /* process numbers */
       if (isdigit (c))
         {
           yylval = c - '0';
           ++yylloc.last_column;
           while (isdigit (c = getchar ()))
             {
               ++yylloc.last_column;
               yylval = yylval * 10 + c - '0';
             }
           ungetc (c, stdin);
           return NUM;
         }
     
       /* return end-of-file */
       if (c == EOF)
         return 0;
     
       /* return single chars and update location */
       if (c == '\n')
         {
           ++yylloc.last_line;
           yylloc.last_column = 0;
         }
       else
         ++yylloc.last_column;
       return c;
     }

   Basically, the lexical analyzer performs the same processing as
before: it skips blanks and tabs, and reads numbers or single-character
tokens.  In addition, it updates `yylloc', the global variable (of type
`YYLTYPE') containing the token's location.

   Now, each time this function returns a token, the parser has its
number as well as its semantic value, and its location in the text. The
last needed change is to initialize `yylloc', for example in the
controlling function:

     int
     main (void)
     {
       yylloc.first_line = yylloc.last_line = 1;
       yylloc.first_column = yylloc.last_column = 0;
       return yyparse ();
     }

   Remember that computing locations is not a matter of syntax.  Every
character must be associated to a location update, whether it is in
valid input, in comments, in literal strings, and so on.