I would be interested how to detect an ongoing DDOS attack - finding out what's going on is of course the prerequisite for any decent countermeasures.
Let's keep it simple and assume that there are only HTTP requests involved, coming from distributed sources. Stupid bots hammering the same URL or coming from the same source IP are easy to spot, we feed the logs to some perl scripts or bash-oneliners involving simple grep/awk patterns to spot them if need be, and then we fight them off with rate limits or hardcoded block rules based on IP, Agent or other criteria.
However, if they hide behind appengine or AWS farms or use other various source addresses, Agent strings and target URLs, then it gets more difficult, especially with IPv6 clients.
So is there any 3rd party tool that can be used to detect unusual usage patterns in realtime (possibly by analyzing the latest log files) Or even some helpful built-in function that I am not aware of?