Copyright (C) 2000-2012 |
Whole document tree Expat XML ParserExpat is a library, written in C, for parsing XML documents. It's the underlying XML parser for the open source Mozilla project, perl's XML::Parser, and other open-source XML parsers. This library is the creation of James Clark, who's also given us groff (an nroff look-alike), Jade (an implemention of ISO's DSSSL stylesheet language for SGML), XP (a Java XML parser package), XT (a Java XSL engine). James was also the technical lead on the XML Working Group at W3 that produced the XML specification. This is free software, licensed under the MIT/X Consortium license. You may download it from the expat homepage on Source Forge. The bulk of this document was originally commissioned as an article by XML.com. They graciously allowed me to retain copyright and to distribute it with expat. Table of Contents
OverviewExpat is a stream-oriented parser. You register callback (or handler) functions with the parser and then start feeding it the document. As the parser recognizes parts of the document, it will call the appropriate handler for that part (if you've registered one.) The document is fed to the parser in pieces, so you can start parsing before you have all the document. This also allows you to parse really huge documents that won't fit into memory. Expat can be intimidating due to the many kinds of handlers and options you can set. But you only need to learn four functions in order to do 90% of what you'll want to do with it:
These functions and others are described in the reference part of this document. The reference section also describes in detail the parameters passed to the different types of handlers. Let's look at a very simple example program that only uses 3 of the above functions (it doesn't need to set a character handler.) The program outline.c prints an element outline, indenting child elements to distinguish them from the parent element that contains them. The start handler does all the work. It prints two indenting spaces for every level of ancestor elements, then it prints the element and attribute information. Finally it increments the global Depth variable. int Depth; void start(void *data, const char *el, const char **attr) { int i; for (i = 0; i < Depth; i++) printf(" "); printf("%s", el); for (i = 0; attr[i]; i += 2) { printf(" %s='%s'", attr[i], attr[i + 1]); } printf("\n"); Depth++; } /* End of start handler */ The end tag simply does the bookkeeping work of decrementing the Depth. void end(void *data, const char *el) { Depth--; } /* End of end handler */ After creating the parser, the main program just has the job of shoveling the document to the parser so that it can do its work. Building and Installing expatThe expat distribution comes as a compressed (with GNU gzip) tar file. You may download the latest version from Source Forge. After unpacking this, cd into the directory. Then follow either the Win32 directions or Unix directions below. Building under Win32If you're using the GNU compiler under cygwin, follow the Unix directions in the next section. Otherwise if you have Microsoft's Developer Studio installed, then from Windows Explorer double-click on "expat.dsp" in the lib directory and build and install in the usual manner. Alternatively, you may download the win32 binary package that contains the expat.h include file and a pre-built dll. Building under Unix (or GNU)First you'll need to run the configure shell script in order to configure the Makefiles and headers for your system. If you're happy with all the defaults that configure picks for you, and you have permission on your system to install into /usr/local, you can install expat with this sequence of commands: ./configure make make install There are some options that you can provide to this script, but the
only one we'll mention here is the /usr/local/lib and the associated header file in
/usr/local/include .
But if you were to give the option, --prefix=/home/me/mystuff ,
then the library and header would get installed in
/home/me/mystuff/lib and /home/me/mystuff/include
respectively.
Using ExpatCompiling and Linking against expatUnless you installed expat in a location not expected by your compiler
and linker, all you have to do to use expat in your programs is to include
the expat header ( On a Unix based system, here's what a Makefile might look like when expat is installed in a standard location: CC=cc LDFLAGS= LIBS= -lexpat xmlapp: xmlapp.o $(CC) $(LDFLAGS) -o xmlapp xmlapp.o $(LIBS) If you installed expat in, say, CC=cc CFLAGS= -I/home/me/mystuff/include LDFLAGS= LIBS= -L/home/me/mystuff/lib -lexpat xmlapp: xmlapp.o $(CC) $(LDFLAGS) -o xmlapp xmlapp.o $(LIBS) You'd also have to set the environment variable Expat BasicsAs we saw in the example in the overview, the first step in parsing an
XML document with expat is to create a parser object. There are
three functions in the expat API for creating a
parser object.
However, only two of these
( The objects returned by the parser creation functions are good for parsing only one XML document or external parsed entity. If your application needs to parse many XML documents, then it needs to create a parser object for each one. The best way to deal with this is to create a higher level object that contains all the default initialization you want for your parser objects. Walking through a document hierarchy with a stream oriented parser will require a good stack mechanism in order to keep track of current context. For instance, to answer the simple question, "What element does this text belong to?" requires a stack, since the parser may have descended into other elements that are children of the current one and has encountered this text on the way out. The things you're likely to want to keep on a stack are the currently opened element and it's attributes. You push this information onto the stack in the start handler and you pop it off in the end handler. For some tasks, it is sufficient to just keep information on what the depth of the stack is (or would be if you had one.) The outline program shown above presents one example. Another such task would be skipping over a complete element. When you see the start tag for the element you want to skip, you set a skip flag and record the depth at which the element started. When the end tag handler encounters the same depth, the skipped element has ended and the flag may be cleared. If you follow the convention that the root element starts at 1, then you can use the same variable for skip flag and skip depth. void init_info(Parseinfo *info) { info->skip = 0; info->depth = 1; /* Other initializations here */ } /* End of init_info */ void rawstart(void *data, const char *el, const char **attr) { Parseinfo *inf = (Parseinfo *) data; if (! inf->skip) { if (should_skip(inf, el, attr)) { inf->skip = inf->depth; } else start(inf, el, attr); /* This does rest of start handling */ } inf->depth++; } /* End of rawstart */ void rawend(void *data, const char *el) { Parseinfo *inf = (Parseinfo *) data; inf->depth--; if (! inf->skip) end(inf, el); /* This does rest of end handling */ if (inf->skip == inf->depth) inf->skip = 0; } /* End rawend */ Notice in the above example the difference in how depth is manipulated in the start and end handlers. The end tag handler should be the mirror image of the start tag handler. This is necessary to properly model containment. Since, in the start tag handler, we incremented depth after the main body of start tag code, then in the end handler, we need to manipulate it before the main body. If we'd decided to increment it first thing in the start handler, then we'd have had to decrement it last thing in the end handler. Communicating between handlersIn order to be able to pass information between different handlers without using globals, you'll need to define a data structure to hold the shared variables. You can then tell expat (with the XML_SetUserData function) to pass a pointer to this structure to the handlers. This is typically the first argument received by most handlers. Namespace ProcessingWhen the parser is created using the Element type and attribute names that belong to a given namespace are
passed to the appropriate handler in expanded form. By default this expanded
form is a concatenation of the namespace URI, the separator character (which
is the 2nd argument to However if XML_SetReturnNSTriplet
has been called with a non-zero You can set handlers for the start of a namespace declaration and for
the end of a scope of a declaration with the
These handlers are called for each declaration. So if, for instance, a start tag had three namespace declarations, then the StartNamespaceDeclHandler would be called three times before the start tag handler is called, once for each declaration. The namespace.c example demonstrates the use of these features. Like outline.c, it produces an outline, but in addition it annotates when a namespace scope starts and when it ends. This example also demonstrates use of application user data. Character EncodingsWhile XML is based on Unicode, and every XML processor is required to recognized UTF-8 and UTF-16 (1 and 2 byte encodings of Unicode), other encodings may be declared in XML documents or entities. For the main document, an XML declaration may contain an encoding declaration: <?xml version="1.0" encoding="ISO-8859-2"?> External parsed entities may begin with a text declaration, which looks like an XML declaration with just an encoding declaration: <?xml encoding="Big5"?> With expat, you may also specify an encoding at the time of creating a parser. This is useful when the encoding information may come from a source outside the document itself (like a higher level protocol.) There are four built-in encodings in expat:
Anything else discovered in an encoding declaration or in the
protocol encoding specified in the parser constructor, triggers a call
to the Expat places restrictions on character encodings that it can support
by filling in the
One pitfall that novice expat users are likely to fall into is that although expat may accept input in various encodings, the strings that it passes to the handlers are always encoded in UTF-8. Your application is responsible for any translation of these strings into other encodings. Handling External Entity ReferencesExpat does not read or parse external entities directly. Note that any
external DTD is a special case of an external entity.
If you've set no Your handler
isn't actually responsible for parsing the entity, but it is responsible
for creating a subsidiary parser with
Parsing DTDsIn order to parse parameter entities, before starting the parse, you must
call
In order to read an external DTD, you also have to set an external entity reference handler as described above. Some expat ExamplesExpat ReferenceParser Creation
Construct a new parser. If encoding is non-null, it specifies a
character encoding to use for the document. This overrides the document
encoding declaration. There are four built-in encodings:
Constructs a new parser that has namespace processing in effect. Namespace
expanded element names and attribute names are returned as a concatenation
of the namespace URI, sep, and the local part of the name. This
means that you should pick a character for sep that can't be
part of a legal URI.
Construct a new parser using the suite of memory handling functions
specified in typedef struct { void *(*malloc_fcn)(size_t size); void *(*realloc_fcn)(void *ptr, size_t size); void (*free_fcn)(void *ptr); } XML_Memory_Handling_Suite;
Construct a new XML_Parser object for parsing an external general
entity. Context is the context argument passed in a call to a
ExternalEntityRefHandler. Other state information such as handlers, user data,
namespace processing is inherited from the parser passed as the 1st
argument. So you shouldn't need to call any of the behavior changing
functions on this parser (unless you want it to act differently than the
parent parser.)
Free memory used by the parser. Your application is responsible for
freeing any memory associated with UserData.
Parsing
Parse some more of the document. The string
s is a buffer
containing part (or perhaps all) of the document. The number of bytes of s
that are part of the document is indicated by len . This means
that s doesn't have to be null terminated. It also means that
if len is larger than the number of bytes in the block of
memory that s points at, then a memory fault is likely. The
isFinal parameter informs the parser that this is the last
piece of the document. Frequently, the last piece is empty (i.e.
len is zero.)
If a parse error occurred, it returns 0. Otherwise it returns a non-zero
value.
This is just like XML_Parse, except in this case expat provides the buffer.
By obtaining the buffer from expat with the
XML_GetBuffer
function, the application can avoid double copying of the input.
Obtain a buffer of size
len to read a piece of the document
into. A NULL value is returned if expat can't allocate enough memory for
this buffer. This has to be called prior to every call to
XML_ParseBuffer . A typical use would look like this:
for (;;) { int bytes_read; void *buff = XML_GetBuffer(p, BUFF_SIZE); if (buff == NULL) { /* handle error */ } bytes_read = read(docfd, buff, BUFF_SIZE); if (bytes_read < 0) { /* handle error */ } if (! XML_ParseBuffer(p, bytes_read, bytes_read == 0)) { /* handle parse error */ } if (bytes_read == 0) break; } Handler SettingAlthough handlers are typically set prior to parsing and left alone, an
application may choose to set or change the handler for a parsing event
while the parse is in progress. For instance, your application may choose
to ignore all text not descended from a A handler may be unset by providing a NULL pointer to the appropriate handler setter. None of the handler setting functions have a return value. Your handlers will be receiving strings in arrays of type
typedef void (*XML_StartElementHandler)(void *userData, const XML_Char *name, const XML_Char **atts); Set handler for start (and empty) tags. Attributes are passed to the start handler as a pointer to a vector of char pointers. Each attribute seen in a start (or empty) tag occupies 2 consecutive places in this vector: the attribute name followed by the attribute value. These pairs are terminated by a null pointer. Note that an empty tag generates a call to both start and end handlers (in that order). typedef void (*XML_EndElementHandler)(void *userData, const XML_Char *name); Set handler for end (and empty) tags. As noted above, an empty tag generates a call to both start and end handlers. Set handlers for start and end tags with one call. typedef void (*XML_CharacterDataHandler)(void *userData, const XML_Char *s, int len); Set a text handler. The string your handler receives is NOT zero terminated. You have to use the length argument to deal with the end of the string. A single block of contiguous text free of markup may still result in a sequence of calls to this handler. In other words, if you're searching for a pattern in the text, it may be split across calls to this handler. typedef void (*XML_ProcessingInstructionHandler)(void *userData, const XML_Char *target, const XML_Char *data); Set a handler for processing instructions. The target is the first word in the processing instruction. The data is the rest of the characters in it after skipping all whitespace after the initial word. typedef void (*XML_CommentHandler)(void *userData, const XML_Char *data); Set a handler for comments. The data is all text inside the comment delimiters. typedef void (*XML_StartCdataSectionHandler)(void *userData); Set a handler that gets called at the beginning of a CDATA section. typedef void (*XML_EndCdataSectionHandler)(void *userData); Set a handler that gets called at the end of a CDATA section. Sets both CDATA section handlers with one call. typedef void (*XML_DefaultHandler)(void *userData, const XML_Char *s, int len); Sets a handler for any characters in the document which wouldn't otherwise be handled. This includes both data for which no handlers can be set (like some kinds of DTD declarations) and data which could be reported but which currently has no handler set. Note that a contiguous piece of data that is destined to be reported to the default handler may actually be reported over several calls to the handler. Setting the handler with this call has the side effect of turning off expansion of references to internally defined general entities. Instead these references are passed to the default handler. typedef void (*XML_DefaultHandler)(void *userData, const XML_Char *s, int len); This sets a default handler, but doesn't affect expansion of internal entity references. typedef int (*XML_ExternalEntityRefHandler)(XML_Parser p, const XML_Char *context, const XML_Char *base, const XML_Char *systemId, const XML_Char *publicId); Set an external entity reference handler. This handler is also
called for processing an external DTD subset if parameter entity parsing
is in effect. (See
The base parameter is the base to use for relative system identifiers. It is set by XML_SetBase and may be null. The public id parameter is the public id given in the entity declaration and may be null. The system id is the system identifier specified in the entity declaration and is never null. There are a couple of ways in which this handler differs from others.
First, this handler returns an integer. A non-zero value should be returned
for successful handling of the external entity reference. Returning a zero
indicates failure, and causes the calling parser to return
an Second, instead of having userData as its first argument, it receives the parser that encountered the entity reference. This, along with the context parameter, may be used as arguments to a call to XML_ExternalEntityParserCreate. Using the returned parser, the body of the external entity can be recursively parsed. Since this handler may be called recursively, it should not be saving information into global or static variables. typedef int (*XML_UnknownEncodingHandler)(void *encodingHandlerData, const XML_Char *name, XML_Encoding *info); Set a handler to deal with encodings other than the built in set. If the handler knows how to deal with an encoding with the given name, it should fill in the info data structure and return 1. Otherwise it should return 0. typedef struct { int map[256]; void *data; int (*convert)(void *data, const char *s); void (*release)(void *data); } XML_Encoding; The map array contains information for every possible possible leading byte in a byte sequence. If the corresponding value is >= 0, then it's a single byte sequence and the byte encodes that Unicode value. If the value is -1, then that byte is invalid as the initial byte in a sequence. If the value is -n, where n is an integer > 1, then n is the number of bytes in the sequence and the actual conversion is accomplished by a call to the function pointed at by convert. This function may return -1 if the sequence itself is invalid. The convert pointer may be null if there are only single byte codes. The data parameter passed to the convert function is the data pointer from XML_Encoding. The string s is NOT null terminated and points at the sequence of bytes to be converted. The function pointed at by release is called by the parser when it is finished with the encoding. It may be null. typedef void (*XML_StartNamespaceDeclHandler)(void *userData, const XML_Char *prefix, const XML_Char *uri); Set a handler to be called when a namespace is declared. Namespace declarations occur inside start tags. But the namespace declaration start handler is called before the start tag handler for each namespace declared in that start tag. typedef void (*XML_EndNamespaceDeclHandler)(void *userData, const XML_Char *prefix); Set a handler to be called when leaving the scope of a namespace declaration. This will be called, for each namespace declaration, after the handler for the end tag of the element in which the namespace was declared. Sets both namespace declaration handlers with a single call typedef void (*XML_XmlDeclHandler) (void *userData, const XML_Char *version, const XML_Char *encoding, int standalone); Sets a handler that is called for XML declarations and also for
text declarations discovered in external entities. The way to distinguish
is that the typedef void (*XML_StartDoctypeDeclHandler)(void *userData, const XML_Char *doctypeName, const XML_Char *sysid, const XML_Char *pubid, int has_internal_subset); Set a handler that is called at the start of a DOCTYPE declaration,
before any external or internal subset is parsed. Both typedef void (*XML_EndDoctypeDeclHandler)(void *userData); Set a handler that is called at the end of a DOCTYPE declaration, after parsing any external subset. Set both doctype handlers with one call. typedef void (*XML_ElementDeclHandler)(void *userData, const XML_Char *name, XML_Content *model); enum XML_Content_Type { XML_CTYPE_EMPTY = 1, XML_CTYPE_ANY, XML_CTYPE_MIXED, XML_CTYPE_NAME, XML_CTYPE_CHOICE, XML_CTYPE_SEQ }; enum XML_Content_Quant { XML_CQUANT_NONE, XML_CQUANT_OPT, XML_CQUANT_REP, XML_CQUANT_PLUS }; typedef struct XML_cp XML_Content; struct XML_cp { enum XML_Content_Type type; enum XML_Content_Quant quant; const XML_Char * name; unsigned int numchildren; XML_Content * children; }; Sets a handler for element declarations in a DTD. The handler gets called with the name of the element in the declaration and a pointer to a structure that contains the element model. It is the application's responsibility to free this data structure. The For type Types typedef void (*XML_AttlistDeclHandler) (void *userData, const XML_Char *elname, const XML_Char *attname, const XML_Char *att_type, const XML_Char *dflt, int isrequired); Set a handler for attlist declarations in the DTD. This handler is called
for each attribute. So a single attlist declaration with multiple
attributes declared will generate multiple calls to this handler. The
The typedef void (*XML_EntityDeclHandler) (void *userData, const XML_Char *entityName, int is_parameter_entity, const XML_Char *value, int value_length, const XML_Char *base, const XML_Char *systemId, const XML_Char *publicId, const XML_Char *notationName); Sets a handler that will be called for all entity declarations.
The For internal entities ( The typedef void (*XML_UnparsedEntityDeclHandler)(void *userData, const XML_Char *entityName, const XML_Char *base, const XML_Char *systemId, const XML_Char *publicId, const XML_Char *notationName); Set a handler that receives declarations of unparsed entities. These are entity declarations that have a notation (NDATA) field: <!ENTITY logo SYSTEM "images/logo.gif" NDATA gif> This handler is obsolete and is provided for backwards compatibility. Use instead XML_SetEntityDeclHandler. typedef void (*XML_NotationDeclHandler)(void *userData, const XML_Char *notationName, const XML_Char *base, const XML_Char *systemId, const XML_Char *publicId); Set a handler that receives notation declarations. typedef int (*XML_NotStandaloneHandler)(void *userData); Set a handler that is called if the document is not "standalone".
This happens when there is an external subset or a reference to a parameter
entity, but does not have standalone set to "yes" in an XML declaration.
If this handler returns 0, then the parser will throw an
Parse position and error reporting functionsThese are the functions you'll want to call when the parse functions return 0 (i.e. a parse error has ocurred), although the position reporting functions are useful outside of errors. The position reported is the byte position (in the original document or entity encoding) of the first of the sequence of characters that generated the current event (or the error that caused the parse functions to return 0.) The position reporting functions are accurate only outside of the DTD. In other words, they usually return bogus information when called from within a DTD declaration handler.
Return what type of error has occurred.
Return a string describing the error corresponding to code.
The code should be one of the enums that can be returned from
XML_GetErrorCode.
Return the byte offset of the position.
Return the line number of the position.
Return the offset, from the beginning of the current line, of
the position.
Return the number of bytes in the current event. Returns 0 if the event is
inside a reference to an internal entity.
Returns the parser's input buffer, sets the integer pointed at by
This should only be called from within a handler during an active parse and the returned buffer should only be referred to from within the handler that made the call. This input buffer contains the untranslated bytes of the input. Only a limited amount of context is kept, so if the event triggering a call spans over a very large amount of input, the actual parse position may be before the beginning of the buffer. Miscellaneous functionsThe functions in this section either obtain state information from the parser or can be used to dynamicly set parser options.
This sets the user data pointer that gets passed to handlers.
It overwrites any previous value for this pointer. Note that the
application is responsible for freeing the memory associated with
userData when it is finished with the parser. So if
you call this when there's already a pointer there, and you haven't
freed the memory associated with it, then you've probably just leaked
memory.
This returns the user data pointer that gets passed to handlers.
It is actually implemented as a macro.
After this is called, handlers receive the parser in the
userData argument. The userData information can still be obtained using
the XML_GetUserData function.
Set the base to be used for resolving relative URIs in system identifiers.
The return value is 0 if there's no memory to store base, otherwise it's
non-zero.
Return the base for resolving relative URIs.
When attributes are reported to the start handler in the atts vector,
attributes that were explicitly set in the element occur before any
attributes that receive their value from default information in an
ATTLIST declaration. This function returns the number of attributes that
were explicitly set times two, thus giving the offset in the
atts
array passed to the start tag handler of the first attribute set
due to defaults. It supplies information for the last call to a start
handler. If called inside a start handler, then that means the current call.
Returns the index of the ID attribute passed in the atts array
in the last call to XML_StartElementHandler, or -1 if there is no ID
attribute. If called inside a start handler, then that means the current call.
Set the encoding to be used by the parser. It is equivalent to
passing a non-null encoding argument to the parser creation functions.
It must not be called after XML_Parser or XML_ParseBuffer have been
called on the given parser.
This enables parsing of parameter entities, including the external
parameter entity that is the external DTD subset, according to
code .
The choices for code are:
This function only has an effect when using a parser created with
XML_ParserCreateNS, i.e. when namespace
processing is in effect. The If
Return the library version as a string (e.g. "expat_1.95.1").
Return the library version information as a structure.
typedef struct { int major; int minor; int micro; } XML_Expat_Version;
|