Manpages

IMPATIENT

NAME

SYNOPSIS

DESCRIPTION

Introduction

However with the development of Perl v5, Perl acquired 'Objects'. To put it simply users can define their own special data types, and write custom routines to manipulate them either in low level languages (C and Fortran) or in Perl itself.

This has been fully exploited by the PerlDL developers. The 'PDL' module is a complete Object-Oriented extension to Perl (although you don't have to know what an object is to use it) which allows large N-dimensional data sets, such as large images, spectra, time series, etc to be stored efficiently and manipulated en masse. For example with the PDL module we can write the perl code "$a=$b+$c", where $b and $c are large datasets (e.g. 2048x2048 images), and get the result in only a fraction of a second.

PDL variables (or 'piddles' as they have come to be known) support a wide range of fundamental data types - arrays can be bytes, short integers (signed or unsigned), long integers, floats or double precision floats. And because of the Object-Oriented nature of PDL new customised datatypes can be derived from them.

As well as the PDL modules, that can be used by normal perl programs, PerlDL comes with a command line perl shell, called 'perldl', which supports command line editing. In combination with the various PDL graphics modules this allows data to be easily played with and visualised.  

Help

 perldl> help help
 $ pdldoc

 perldl> foreach ( map{"$_/PDL/HtmlDocs"}@INC ) { p "$_\n" if -d $_ }

Perl Datatypes and how PDL extends them

perl v5 introduces to perl data structures and objects. A simple scalar variable $x now be a user-defined data type or full blown object (it actually holds a reference (a smart ``pointer'') to this but that is not relevant for ordinary use of perlDL)

The fundamental idea behind perlDL is to allow $x to hold a whole 1D spectrum, or a 2D image, a 3D data cube, and so on up to large N-dimensional data sets. These can be manipulated all at once, e.g. "$a = $b + 2" does a vector operation on each value in the spectrum/image/etc.

You may well ask: "Why not just store a spectrum as a simple perl @x style list with each pixel being a list item?" The two key answers to this are memory and speed. Because we know our spectrum consists of pure numbers we can compactly store them in a single block of memory corresponding to a C style numeric array. This takes up a LOT less memory than the equivalent perl list. It is then easy to pass this block of memory to a fast addition routine, or to any other C function which deals with arrays. As a result perlDL is very fast --- for example one can mulitiply a 2048*2048 image in exactly the same time as it would take in C or FORTRAN (0.1 sec on my SPARC). A further advantage of this is that for simple operations (e.g. "$x += 2") one can manipulate the whole array without caring about its dimensionality.

I find when using perlDL it is most useful to think of standard perl @x variables as ``lists'' of generic ``things'' and PDL variables like $x as ``arrays'' which can be contained in lists or hashes. Quite often in my perlDL scripts I have @x contain a list of spectra, or a list of images (or even a mix!). Or perhaps one could have a hash (e.g. %x) of images... the only limit is memory!

perlDL variables support a range of data types - arrays can be bytes, short integers (signed or unsigned), long integers, floats or double precision floats.  

Usage

 use PDL;  # in perl scripts: use the standard perlDL modules

 use PDL::Graphics::TriD;

 use PDL::Graphics::TriD '';

There is also an interactive shell, "perldl", see perldl.  

To create a new PDL variable

 $a = pdl [1..10];             # 1D array
 $a = pdl (1,2,3,4);           # Ditto
 $b = pdl [[1,2,3],[4,5,6]];   # 2D 3x2 array
 $b = pdl 42                   # 0-dimensional scalar
 $c = pdl $a;                  # Make a new copy

 $d = byte [1..10];            # See "Type conversion"
 $e = zeroes(3,2,4);           # 3x2x4 zero-filled array

 $c = rfits $file;             # Read FITS file

 @x = ( pdl(42), zeroes(3,2,4), rfits($file) ); # Is a LIST of PDL variables!

In addition all PDL functions automatically convert normal perl scalars to PDL variables on-the-fly.

(also see ``Type Conversion'' and ``Input/Output'' sections below)  

Arithmetic (and boolean expressions)

 $a = $b + 2; $a++; $a = $b / $c; # Etc.

 $c=sqrt($a); $d = log10($b+100); # Etc

 $e = $a>42; # Vector conditional

 $e = 42*($a>42) + $a*($a<=42); # Cap top

 $b = $a->log10 unless any ($a <= 0); # avoid floating point error

 $a = $a / ( max($a) - min($a) );

 $f = where($a, $a > 10); # where returns a piddle of elements for
                          # which the condition is true

 print $a; # $a in string context prints it in a N-dimensional format

When using piddles in conditional expressions (i.e. "if", "unless" and "while" constructs) only piddles with exactly one element are allowed, e.g.

 $a = pdl (1,0,0,1);
 print "is set" if $a->index(2);

 print "is ok" if $a > 3;

 print "some are > 3" if any $a>3;
 print "can't take logarithm" unless all $a>0;

Matrix functions

perlDL is row-major not column major so this is actually "c(i,j) = sum_k a(k,j) b(i,k)" - but when matrices are printed the results will look right. Just remember the indices are reversed. e.g.:

 $a = [                   $b = [
       [ 1  2  3  0]            [1 1]
       [ 1 -1  2  7]            [0 2]
       [ 1  0  0  1]            [0 2]
      ]                         [1 1]
                               ]

 gives $c = [
             [ 1 11]
             [ 8 10]
             [ 2  2]
            ]

How to write a simple function

 sub dotproduct {
     my ($a,$b) = @_;
     return sum($a*$b) ;
 }
 1;

Of course, this function is already available as the inner function, see PDL::Primitive.  

Type Conversion

 $a = float($b);
 $c = long($d);   # "long" is generally a 4 byte int
 $d = byte($a);

These routines also automatically convert perl lists to allow the convenient shorthand:

 $a = byte [[1..10],[1..10]];  # Create 2D byte array
 $a = float [1..1000];         # Create 1D float array

Piddles and boolean expressions

Printing

 print $a;

 $b = "Answer is = $a ";

Sections

Furthermore [Which modules!?!],
 the convention is that the center of the pixel (0,0) IS at coordinate (0.0,0.0). Thus the above image ranges from "-0.5..99.5, -0.5..99.5" in real space. All perlDL graphics functions conform to this defintion and hide away the unit-offsetness of, for example, the PGPLOT FORTRAN library.

Again following the usual convention coordinate (0,0) is displayed at the bottom left when displaying an image. It appears at the top left when using ""print $a"" etc.

 $b = $a->slice("$x1:$x2,$y1:$y1,$z1:$z2"); # Take subsection

 # Set part of $bigimage to values from $smallimage
 ($tmp = $bigimage->slice("$xa:$xb,$ya:$yb")) .= $smallimage;

 $newimage = ins($bigimage,$smallimage,$x,$y,$z...) # Insert at x,y,z

 $c  = nelem ($a); # Number of pixels

 $val = at($object, $x,$y,$z...)    # Pixel value at position
 $val = $object->at($x,$y,$z...)    # equivalent
 set($myimage, $x, $y, ... $value)  # Set value in image

 $b = xvals($a); # Fill array with X-coord values (also yvals(), zvals(),
                 # axisvals($x,$axis) and rvals() for radial distance
                 # from centre).

Input/Output

PDL::IO::Misc

Ascii, FITS and FIGARO/NDF IO routines.

PDL::IO::FastRaw

Using the raw data types of your machine, an unportable but blindingly fast IO format. Also supports memory mapping to conserve memory as well as get more speed.

PDL::IO::FlexRaw

General raw data formats.

PDL::IO::Browser

A Curses browser for arrays.

PDL::IO::Pnm

Portaple bitmap and pixmap support.

PDL::IO::Pic

Using the previous module and netpbm, makes it possible to easily write GIF, jpeg and whatever with simple commands.

Graphics

PDL::Graphics::PGPLOT

PGPLOT provdes a simple library for line graphics and image display.

There is an easy interface to this in the internal module PDL::Graphics::PGPLOT, which calls routines in the separately available PGPLOT top-level module.

PDL::Graphics::IIS

Many astronomers like to use SAOimage and Ximtool (or there derivations/clones). These are useful free widgets for inspection and visualisation of images. (They are not provided with perlDL but can easily be obtained from their official sites off the Net.)

The PDL::Graphics::IIS package provides allows one to display images in these (``IIS'' is the name of an ancient item of image display hardware whose protocols these tools conform to.)

Karma

The PDL::Graphics::Karma module provides an interface to the Karma visualisation suite. This is a set of GUI applications which are specially designed for visualising noisy 2D and 3D data sets.

PDL::Graphics::TriD

See PDL::Graphics::TriD (the name sucks...). this is a collection of 3D routines for OpenGL and (soon) VRML and other 3D formats which allow 3D point, line, and surface plots from PDL.

Autoloading

One can also write PDL extensions as normal Perl modules.  

perldl shell

e.g.:

 jhereg% perldl
 perlDL shell v1.30
  PDL comes with ABSOLUTELY NO WARRANTY. For details, see the file
  'COPYING' in the PDL distribution. This is free software and you
  are welcome to redistribute it under certain conditions, see
  the same file for details.
 ReadLines enabled                                                               
 Reading PDL/default.perldlrc...
 Found docs database /home/kgb/soft/dev/lib/perl5/site_perl/PDL/pdldoc.db
 Type 'help' for online help
 Type 'demo' for online demos
 Loaded PDL v2.005                                                               
 perldl> $x = rfits 'm51.fits'
 BITPIX =  16  size = 65536 pixels
 Reading  131072 bytes
 BSCALE = 1.0000000000E0 &&  BZERO = 0.0000000000E0

 perldl> imag $x
 Loaded PGPLOT
 Displaying 256 x 256 image from 24 to 500 ...

Miscellaneous shell features:

p

The shell aliases "p" to be a convenient short form of "print", e.g.

   perldl> p ones 5,3
   [
    [1 1 1 1 1]
    [1 1 1 1 1]
    [1 1 1 1 1]
   ]

Initialization

The files "~/.perldlrc" and "local.perldlrc" (in the current directory) are sourced if found. This allows the user to have global and local PDL code for startup.

Help

Type 'help'! One can search the PDL documentation, and look up documentation on any function.

Escape

Any line starting with the "#" character is treated as a shell escape. This character is configurable by setting the perl variable $PERLDL_ESCAPE. This could, for example, be set in "~/.perldlrc".

Overload operators

 + - * / > < >= <= << >> & | ^ == != <=> ** % ! ~
 sin log abs atan2 sqrt cos exp

Object-Orientation and perlDL

By using overloading one can make mathematical operators do whatever you please, and PDL has some built-in tricks which allow existing PDL functions to work unchanged, even if the underlying data representation is vastly changed! See PDL::Objects  

Memory usage and references

NOTE: On some most systems it is better to configure perl (during the build options) to use the system "malloc()" function rather than perl's built-in one. This is because perl's one is optimised for speed rather than consumption of virtual memory - this can result in a factor of two improvement in the amount of memory storage you can use. The Perl malloc in 5.004 and later does have a number of compile-time options you can use to tune the behaviour.

Simple arithmetic

If $a is a big image (e.g. occupying 10MB) then the command

 $a = $a + 1;

eats up another 10MB of memory. This is because the expression "$a+1" creates a temporary copy of $a to hold the result, then $a is assigned a reference to that. After this, the original $a is destroyed so there is no permanent memory waste. But on a small machine, the growth in the memory footprint can be considerable. It is obviously done this way so "$c=$a+1" works as expected.

Also if one says:

 $b = $a;     # $b and $a now point to same data
 $a = $a + 1;

Then $b and $a end up being different, as one naively expects, because a new reference is created and $a is assigned to it.

However if $a was a huge memory hog (e.g. a 3D volume) creating a copy of it may not be a good thing. One can avoid this memory overhead in the above example by saying:

 $a++;

The operations "++,+=,--,-=", etc. all call a special ``in-place'' version of the arithmetic subroutine. This means no more memory is needed - the downside of this is that if "$b=$a" then $b is also incremented. To force a copy explicitly:

 $b = pdl $a; # Real copy

or, alternatively, perhaps better style:

 $b = $a->copy;

Functions

Most functions, e.g. "log()", return a result which is a transformation of their argument. This makes for good programming practice. However many operations can be done ``in-place'' and this may be required when large arrays are in use and memory is at a premium. For these circumstances the operator inplace() is provided which prevents the extra copy and allows the argument to be modified. e.g.:

 $x = log($array);          # $array unaffected
 log( inplace($bigarray) ); # $bigarray changed in situ

WARNINGS:

1

The usual caveats about duplicate references apply.

2

Obviously when used with some functions which can not be applied in situ (e.g. "convolve()") unexpected effects may occur! We try to indicate "inplace()"-safe functions in the documentation.

3

Type conversions, such as"float()", may cause hidden copying.

Ensuring piddleness

 sub myfiddle { my $pdl = topdl(shift); $pdl->fiddle_foo(...); ... }

AUTHOR

Index

NAME

SYNOPSIS

DESCRIPTION

Introduction

Help

Perl Datatypes and how PDL extends them

Usage

To create a new PDL variable

Arithmetic (and boolean expressions)

Matrix functions

How to write a simple function

Type Conversion

Piddles and boolean expressions

Printing

Sections

Input/Output

Graphics

Autoloading

perldl shell

Overload operators

Object-Orientation and perlDL

Memory usage and references

Ensuring piddleness

AUTHOR