Yes, the upstream 'make' only installs gmp.h and mp.h, so why does this
package supply all the other headers?

The answer is: "Several packages in Debian need deeper constructs than those
provided in gmp.h or mp.h alone."

In the past at least two different developers asked for an additional header
file to be included because their package needed it. Rather than be bug report
driven, it seemed the simplest solution was to supply all the header files 
provided in the source. They do not collide with any other name space, so
unless or until they do, all bug reports complaining that too many header
files are provided in the -dev package will be summarily closed, with
reference to this README file.

HTH,

Dwarf