This Apache module is intended to reduce the load on your server & bandwidth
generated by popular virtual hosts, directories, locations, or users according
to supported polices (see below) that decide when to delay or refuse requests.
Also mod_throttle can track and throttle incoming connections by IP address
or by authenticated remote user.
Every request now passes through four levels of throttling, which are:
by client's IP address (ThrottleClientIP),
by authenticated remote user name (ThrottleRemoteUser),
by local user ID (ThrottleUser),
and by directory, location, virtual host, or server (ThrottlePolicy).
In the list below, is a detailed description of each policy. Every policy
expects two parameters: a limit and a time period.
None
This policy imposes no restrictions on a request and used as a place holder to allow
monitoring. The limit currently serves no purpose. The period specifies how long
data is accumulated before the counters are reset.
Concurrent
Impose a limit on the number of concurrent requests at any one time.
The period specifies how long data is accumulated before the counters are reset.
This policy cannot be used with either ThrottleClientIP
or ThrottleRemoteUser.
Document
Excluding requests for HTML page elements such as images and style sheets,
impose a limit on the number of requests per period. When this limit is
exceeded all further requests are refused, until the elapsed time exceeds
the period length, at which point the elapsed time and the
counters are reset. Note that the requests (hits) column of the throttle
status display does not include the requests for page elements.
Idle
Impose a mimimum idle time between requests. When the miminum is not
reached, then the request incurs a calculated delay penalty or is
refused.
First, whenever the elapsed time exceeds the period length, then the
counters are reset.
Second, if the idle time between requests exceeds the minimum, then the
the request proceeds without delay. Otherwise the request is delayed
between one and ThrottleMaxDelay seconds.
If the delay would exceed ThrottleMaxDelay,
then we refuse the request entirely to avoid occupying servers unnecessarily.
The delay is computed as the policy minimum less the idle time between requests.
Original
Original mod_throttle 2.0 heuristic.
Impose a limit on the volume (kbytes sent) per period, which when exceeded
the request incurs a counter-based delay penalty or is refused.
First, whenever the elapsed time exceeds the period length, then the volume
and elapsed time are halved.
Second, if the volume is below the limit, then the delay counter is
decreased by one second if it is not yet zero. Otherwise, when the limit
is exeeded, the delay counter is increased by one second. The delay can be
between zero and ThrottleMaxDelay seconds, after
which the request will be refused to avoid occupying servers unnecessarily.
Random
Randomly accept a percentage (limit) of the requests. If the percentage
is zero (0), then every request is refused; if the percentage
is 100, then all requests are accepted. The period specifies how long
data is accumulated before the counters are reset.
Request
Impose a limit on the number of requests per period. When this limit is
exceeded all further requests are refused until the elapsed time exceeds
the period length, at which point the elapsed time and counters are reset.
Speed
Impose a limit on the volume (kbytes sent) per period, which when exceeded
the request incurs a calculated delay penalty or is refused.
First, whenever the elapsed time exceeds the period length, then the limit
(allowance) is deducted from the volume, which cannot be a negative result;
also the period length is deducted from the elapse time.
Second, if the volume is below the limit, in which case the request
proceeds without delay. Otherwise the request is delayed between one and
ThrottleMaxDelay seconds. If the delay would
exceed ThrottleMaxDelay, then we refuse the
request entirely to avoid occupying servers unnecessarily.
The delay is computed as one plus the integer result of the
volume times 10 divided by the limit.
Volume
Impose a limit on the volume (kbytes sent) per period. When this limit is
exceeded all further requests are refused, until the end of the period at
which point the elapsed time and counters are reset.
Depending on your operating system, you must edit the #undef and #define found
at the top of mod_throttle.c source file to specify the semaphore & shared
memory API to use.
For the semaphore support you must #define only one of the following and #undef
the rest. If you do not #define one of these, then mod_throttle will operate
without semaphore support, which will have undefined results:
For the shared memory support you must #define only one of the following and #undef
the rest. If you do not #define one of these, then mod_throttle will operate without
shared memory, which will have undefined results:
cd /usr/local/apache_1.3.14 (or where ever you keep the Apache source)
./configure \ (see below in the Notes for additional information)
--disable-shared=throttle \
--add-module=(path to)/mod_throttle-3.1/mod_throttle.c \
... (your other configuration options) ...
make install
Configure the httpd.conf file. At the very least add following:
Displays the complete list of recently connected client IP addresses.
There are command links to reset all the clients, or reset individual clients.
Each row shows a client IP address,
the percentage of the limit reached,
the total number of requests (hits) made by this client,
number of requests refused to the this client,
volume (kbytes sent) per period,
average volume per request,
the current delay,
the elapsed period time,
and the time idle since the last request.
Displays the complete list of recently authenticated remote users.
There are command links to reset all or individual remote users.
Each row shows the authenticated remote user name,
the percentage of the limit reached,
the total number of requests (hits) made by this remote user,
number of requests refused to the this remote user,
volume (kbytes sent) per period,
average volume per request,
the current delay,
the elapsed period time,
and the time idle since the last request.
Displays the complete list of server, virtual hosts, directories, locations, and
local users who are to be throttled. There are command links to reset all or individual
entries, and preserve & restore runtime data.
Each row shows the item being throttled,
the percentage of the limit reached,
the total number of requests (hits) received,
number of requests refused,
volume (kbytes sent) per period,
average volume per request,
the current delay,
policy, limit, period,
elapsed period time,
and the time idle since the last request.
Similar to SetHandler throttle-status in content,
except it only displays one individual entry for either a ~user or the
server of which the request was made. The intent of this handler is to provide
the system administrator a means by which customers can monitor their
throttles without seeing those belonging to others.
Assuming the handler has been installed as outlined above,
then a URL of the form:
http://my.domain.com/~user/throttle-me
will display the throttle status for user, if a
ThrottleUser was specified for the user in question; otherwise
a Not Found error page. A URL of the form:
http://my.domain.com/throttle-me
will display the throttle status of my.domain.com,
if it has a ThrottlePolicy; otherwise a Not Found error page.
ThrottleClientIP size policy limit period
Context: server
Specify the size of the client IP address list and the global policy
to be applied to all incoming connections based on information recorded for a client IP address.
The policy Concurrent is ignored.
When the list size is greater than zero, then throttling by
client IP address is enabled (the default is zero). Requests from a client IP address can be
tracked for a short period of time and subject to the policy specified. The period of
time that a client IP address is tracked depends on the size of the client IP address
list, which is ordered most recent request to oldest. Every time a new client IP address
connects, the oldest entry in the list is lost and reassigned to the new client IP address.
Every time an existing entry makes a request, it is moved to the top of the list.
ThrottleContentType string
Context: server
Specifies the default Content-Type to be used for the throttle status
display. Only two supported types: text/html,
text/plain. text/html is
a nicely formatted and coloured table. text/plain is
a tab separated list of columns as seen for text/html,
without headers and footers. When not specified, the default is
text/html. This can be overridden by specifying
the content-type in the URL, for example:
ThrottleIndicator green percentage ThrottleIndicator yellow percentage ThrottleIndicator red percentage
Context: server
Specify the default thresholds used for the visual warnings by the
throttle status display. The default indicators are green 50, yellow 75,
and red 90.
ThrottleLockFile string
Context: server
The lock file used with fcntl() or flock() serialization. Must be stored on a
local disk.
ThrottleMaxDelay seconds
Context: server
This directive affects all policies that use delays. It imposes an upper limit on
the throttle delay that can be applied by a policy. The default is 60 seconds. A
value of zero (0) disables the limit, allowing a delay to be as high as the policy
wants.
The policy parameter can be one of the previously above
mentioned policies.
The limit is a number followed by an optional suffix K, M, or G
for kilo-bytes, mega-bytes, or giga-bytes respectively. When no suffix is given the default
is kilo-bytes.
The period is a number followed by an optional suffix s, m, h, d, or w
for seconds, minutes, hours, days, or weeks respectively. When no suffix is given the default
is seconds.
ThrottleRefresh seconds
Context: server
Specifies the refresh time for the throttle status display. When
not specified, the default is 60 seconds. This can be overridden by
specifying the refresh time in the URL as well, for example:
http://www.domain.com/throttle-status?refresh=20
ThrottleRemoteUser size policy limit period
Context: server
Specify the size of the authenticated remote user list and the global policy to be applied
to all requests based on information recorded for an authenticated remote user.
The policy Concurrent is ignored.
When the list size is greater than zero, then throttling by authenticated remote user
is enabled (the default is zero). When the web server restricts access to a resource,
authentication is required to be sent by the remote user. The user name supplied for
that authentication is then used to control short term usage by the policy
specified. The period of
time that a remote user can be tracked depends on the size of the remote user
list, which is ordered most recent request to oldest. Every time a new remote user
connects, the oldest entry in the list is lost and reassigned to the new remote user.
Every time an existing entry makes a request, it is moved to the top of the list.
ThrottleRuntimeFile filename
Context: server
The location and file name of the runtime data file used to preserve state information
across shutdowns & restarts.
ThrottleUser user policy limit period
Context: server
Each request for files belonging to the given user
name or #ID are throttled according to the selected policy as described above.
If the user is an asterisk (*), then all the users from
the system user databases are throttled with the same policy, limit, and period.
If the user is an absolute path to an htpasswd file,
/etc/passwd, or a file with one user per line, then all the users from that file
are throttled with the same policy, limit, and period. User names with no corresponding
system user ID are ignored.
Notes
mod_throttle/3.0 tested on following platforms (please
email me
your platform if you've made it work on something different):
FreeBSD 4.1.1, Apache 1.3.14
The USE_SYSTEM_V_SERIALIZATION, and USE_SYSTEM_V_SHARED_MEMORY code
was successfully tested on this platform.
The USE_POSIX_SERIALIZATION and
USE_POSIX_SHARED_MEMORY compiles and links (with & without -pthread),
but fails on sem_init() when execute - help anyone?.
RedHat Linux 5.1+ (MIPS & Intel), Apache 1.3.14
The USE_FCNTL_SERIALIZATION, USE_FLOCK_SERIALIZATION,
USE_SYSTEM_V_SERIALIZATION, and USE_SYSTEM_V_SHARED_MEMORY code was
successfully tested on this platform.
Solaris 5.7, Apache 1.3.12
To build as a DSO module, Apache must be built with --enable-rule=SHARED_CORE.
The USE_POSIX_SERIALIZATION and USE_POSIX_SHARED_MEMORY code was successfully
tested on this platform, but must be linked with the library "-lrt"; when
compiling DSO, edit the Makefile and uncomment the LIB= macro; when compiling
statically, specify: LIBS='-lrt' ./configure ...
The context diff "patch.proxy_util.c" should be applied to
/src/modules/proxy/proxy_util.c from the Apache 1.3.14 distribution.
This patch allows mod_throttle/3.0 to monitor proxy requests when mod_proxy
is in use. (This patch has been submitted to the Apache Server project.)
The policies Original, Speed,
and Volume all impose limits on volume (kilobytes sent).
The differences lie in how adjustments are made each period and whether to delay
or refuse the conection.
Here is a shell command that can be used for a cron job to restart the Apache server
on a set schedule in order to reload your configuration file. This assumes that you
do not change the location of the "PidFile" in your configuration and that httpd is on
root's PATH.
kill -USR1 $(cat $(httpd -V | sed -n '/DEFAULT_PIDLOG/s/.*"\(.*\)"/\1/p'))
License
This source distribution is made freely available and there is no charge
for its use, provided you retain this notice, disclaimers, author's
copyright, and credits.
Disclaimer
THIS SOFTWARE IS PROVIDE "AS IS" AND ANY EXPRESSED OR IMPLIED WARRANTIES,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO WAY SHALL THE
AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
OF THE POSSIBILITY OF SUCH DAMAGE.
Credits
Original design for mod_throttle/1.0 goes to Mark Lovell
<mlovell@bigrock.com>.
Elements of the critical & shared memory code, as of mod_throttle/3.0,
originally derived from the Apache Web Server source code.
Thank You to Lu Vo <LVo@SUPERB.NET>
for providing a Solaris POSIX compliant machine to test on; to Travis Doherty
<travis@terradelta.net>
for a FreeBSD machine for testing and several suggestions; to David M. Shirley
<david@webquarry.com> for
constructive and helpful reports concerning design and testing.
