GNU Info

(bison.info)Calling Convention

---

Next: Token Values Up: Lexical

---

Enter node , (file) or (file)node

Calling Convention for `yylex'
------------------------------

   The value that `yylex' returns must be the numeric code for the type
of token it has just found, or 0 for end-of-input.

   When a token is referred to in the grammar rules by a name, that name
in the parser file becomes a C macro whose definition is the proper
numeric code for that token type.  So `yylex' can use the name to
indicate that type.  Note: Symbols.

   When a token is referred to in the grammar rules by a character
literal, the numeric code for that character is also the code for the
token type.  So `yylex' can simply return that character code.  The
null character must not be used this way, because its code is zero and
that is what signifies end-of-input.

   Here is an example showing these things:

     int
     yylex (void)
     {
       ...
       if (c == EOF)     /* Detect end of file. */
         return 0;
       ...
       if (c == '+' || c == '-')
         return c;      /* Assume token type for `+' is '+'. */
       ...
       return INT;      /* Return the type of the token. */
       ...
     }

This interface has been designed so that the output from the `lex'
utility can be used without change as the definition of `yylex'.

   If the grammar uses literal string tokens, there are two ways that
`yylex' can determine the token type codes for them:

   * If the grammar defines symbolic token names as aliases for the
     literal string tokens, `yylex' can use these symbolic names like
     all others.  In this case, the use of the literal string tokens in
     the grammar file has no effect on `yylex'.

   * `yylex' can find the multicharacter token in the `yytname' table.
     The index of the token in the table is the token type's code.  The
     name of a multicharacter token is recorded in `yytname' with a
     double-quote, the token's characters, and another double-quote.
     The token's characters are not escaped in any way; they appear
     verbatim in the contents of the string in the table.

     Here's code for looking up a token in `yytname', assuming that the
     characters of the token are stored in `token_buffer'.

          for (i = 0; i < YYNTOKENS; i++)
            {
              if (yytname[i] != 0
                  && yytname[i][0] == '"'
                  && strncmp (yytname[i] + 1, token_buffer,
                              strlen (token_buffer))
                  && yytname[i][strlen (token_buffer) + 1] == '"'
                  && yytname[i][strlen (token_buffer) + 2] == 0)
                break;
            }

     The `yytname' table is generated only if you use the
     `%token_table' declaration.  Note: Decl Summary.