Due to frustration from this previous thread:
http://www.lunarforums.com/lunarpages_web_hosting_email_pc_security/autobanning_ips_that_scan_for_nonexistent_content-t43805.0.html;msg310478#msg310478 -
I created this thread and the Auto IP Ban Script.Many of us web site/domain account holders have been plagued by the dreaded nuisance of site rippers, site scanners, unruly search engine spiders and rogue search engine bots. This is clearly evident whenever you view your error logs and notice a really long list of errors pertaining to files or folders that do not reside on your web account. And it leaves you wondering, why are they trying to find that particular file or folder?
This form of site sniffing is due to someone or some program automatically running a script that looks for certain files or folders. What they are looking for are security holes in your web account for files that can be exploited. Although there is the ever present danger of exploits and the constant need for security; many of these exploits go unfixed or unnoticed until it is much too late for the account owner.
In some cases certain search engines and or robots will crawl your pages and more than often will not respect or obey the parameters set forth in the robots.txt file that you have created and or configured. Some of these spiders/bots will ignore the robots.txt and then traverse wherever they like about your web account folders. This in turn will list or collect everything in your web account regardless of what you have blocked or have not properly hidden.
In response to this annoying practice of sniffing or scanning a web site; I attempted to seek assistance from local hosting providers and other resources. To no avail, not a single person or company had a sure fire (and or free) way to fully combat this problem. I was told to either just ignore the scanning or "manually edit" my .htaccess file each and every single time there was a violation. Confounded by the lack of assistance and the need for being cheap; I set forth to learn more PHP coding (yes, I took the time to read) and came up with a simple but highly effective script that dealt with the problem immediately. Within the first 24 hours of implementing my newly created script, I had effectively and permanently banned five IP addresses that were scanning my web sites. After the first 5 seconds of them scanning my site and hitting the Auto IP Ban Script - THEY WERE STOPPED COLD!
So, here is the script. As you can see it is small, plain, loosely written and has much potential. Try it out. It shouldn't be too hard to get it to work on your web site. It only took me less than a day to write and perfect. Have fun and I hope this works for you rather than the - "Just deal with it" or "Ignore it" responses you will normally get.
Download the ZIP file: http://www.fbi-team.org/_pub/auto_ip_ban.zip
Auto IP Ban Script (Stop Rogue Scanning and Trap Bad Spiders/Bots)(Installation steps [Testing Phase] - Please read carefully)
1.
Copy below code into a file called: mytrap.php(Use an HTML/PHP editor or Notepad. Be sure to remove the trailing spaces if you copy from this forum)
<?php
// * Auto IP Ban Script - v1.0 *
// * Author: Inque187 *
// * Date: (c) Oct 5, 2007 *
// * Works with PHP 4.0+ & 5.0+ *
// * DISCLAIMER: Author of this script is not and will not be held liable and or responsibile for any configuration errors, loss of data, interruption of *
// * service, and or any other means of misuse by neglect pertaining to this script that affects any internet account. User of this script takes sole *
// * responsibility for any resulting problems that arise due to usage of this script in part or in whole and shall hold indemnity against the author. *
$ipad = $_SERVER['REMOTE_ADDR']; // Collects the user/visitor IP address.
$ban = "Deny from $ipad\n"; // What will be written to the .htaccess file if IP needs to be banned.
$file = "htaccess.txt"; // Change to -> .htaccess <- once thoroughly tested. Should be placed in the root directory.
$search = file_get_contents($file); // Prepare the .htaccess file by gathering entire contents.
$check = strpos($search, $ipad); // Checks the .htaccess file if the current user IP address string does exist.
// This next part of the script checks to see if the IP is already banned or not.
// If the IP does not already exist; it will write the ban line to the .htaccess file, display the message, and then email you a copy.
// If the IP is already listed in the .htaccess file; the script ends with only a displayed message.
if ($check === FALSE) {
$open = @fopen($file, "a"); // Open the .htaccess file and get ready for writing only.
$write = @fputs($open, $ban); // Write the banned IP line to the .htaccess file. (Example: Deny from 12.34.56.789)
// Email a copy of ban and info to your admin account (or other email address).
// Make sure you change the email address.
@mail('BlOcK_IpS@YoUr_WeB_sItE.cOm','Banned IP '.$_SERVER['REMOTE_ADDR'].'','
Banned IP: '.$_SERVER['REMOTE_ADDR'].'
Request URI: '.$_SERVER['REQUEST_URI'].'
User Agent: '.$_SERVER['HTTP_USER_AGENT']);
// IP address is not banned - so there is a need to write to .htaccess file.
// Display the error message to the user. (You may change to read what you want).
echo '<html><head><title>IP Address '.$ipad.' - Blocked or Banned!</title></head><body bgcolor="#FF000000" text="#FFFFFF" oncontextmenu="return false;"><center><font face="Verdana, Arial"><h1>THANK YOU - DON\'T COME AGAIN!</h1><b>IP Address '.$ipad.' Has Been Blocked or Banned!<br />Contact the web admin if this ban is by mistake.<p />Have a nice day!</b></font></center></body></html>';
// Close the .htaccess file - all done.
@fclose($open);
} else {
// IP address is already banned - no need to rewrite to .htaccess file.
// Display the error message to the user. (You may change to read what you want).
echo '<html><head><title>IP Address '.$ipad.' - Blocked or Banned!</title></head><body bgcolor="#FF000000" text="#FFFFFF" oncontextmenu="return false;"><center><font face="Verdana, Arial"><h1>THANK YOU - DON\'T COME AGAIN!</h1><b>IP Address '.$ipad.' Has Been Blocked or Banned!<br />Contact the web admin if this ban is by mistake.<p />Have a nice day!</b></font></center></body></html>';
}
// End of File/Script;
exit;
?>
2.
Save mytrap.php file and then FTP
upload to the root folder of your web account.
(You can also use this same file in any subdirectories designated as 'Add-On Domains')
3.
For testing purposes only - create an empty text file called
htaccess.txt and then FTP
upload it to the same root folder of your web account where the mytrap.php file was uploaded to.
(This way while testing the Auto IP Ban script, you will not overwrite or destroy your existing original .htaccess file)
*** WARNING *** - the following step is for those who have some knowledge in editing the .htaccess file.
It is highly recommended that you make a copy of your original .htaccess file and keep in a safe place in the event that an error occurs.- If you
are well versed in manually editing the .htaccess file - proceed with caution.
- If you
are not well versed in manually editing the .htaccess file -
STOP RIGHT NOW!- Please consult the latest documentation regarding the .htaccess file:
http://httpd.apache.org/docs/2.0/howto/htaccess.html 4a.
Open your original .htaccess file in a text editor and add the following lines of code:
(It is recommended that you put these lines of code at the
top of the .htaccess file)
Options +SymLinksifOwnerMatch
<FilesMatch 403.shtml>
Order Allow,Deny
Allow From All
</FilesMatch>
4b. Then add these lines at the
bottom of the .htaccess file - be sure to leave an extra empty line at the bottom too!
RewriteEngine On
RewriteRule ^cgi-bin /mytrap.php [NC]
RewriteRule ^cgi-bin/(.*)$ /mytrap.php [NC]
(Be sure that you are not using this directory/folder "CGI-BIN"- Ensure that you have added the "Options +SymLinksifOwnerMatch" line of code to the .htaccess file or you will ban/block your own web site IP address and or will receive an "Error 500 Page" when accessing your site. If you are in fact using the "CGI-BIN" folder, edit this to reflect a folder not in your root folder. (Example: /home/account/public_html/fumuda)
5.
Save .htaccess file and then FTP
upload to the root folder of your web account.
6.
Open a web browser and enter in the address bar:
http://www.YoUr_WeB_sItE.cOm/cgi-bin(You can use a trailing slash too - Change "YoUr_WeB_sItE.cOm" to your own web domain)
7. You should be presented with a red background screen with white colored text stating that your IP address has been banned/blocked.
8. Go
check your email that you changed in the mytrap.php script and you should have a message from your hosted server with your own IP address (that has been banned), the file/folder you tried to access and the user agent string of your browser.
9a. Download/Browse to:
http://www.YoUr_WeB_sItE.cOm/htaccess.txt10a. Open the htaccess.txt file in Notepad and you should see the line:
Deny from 12.34.56.789 (with your IP address instead)
11a. If you see the red message screen, have received the email with the ban info, and verified the "deny" text was written to the htaccess.txt file - EVERYTHING WORKED PERFECTLY!
How do I unban myself? [Testing Phase]Technically you haven't banned yourself from your own web site. At least not yet. Remember the htaccess.txt file? This was used only for testing purposes. Luckily it wasn't the real .htaccess file or you wouldn't be able to access your web site via a web browser right now.
So, with the testing complete; there is no need to unban yourself.
Setting Auto IP Ban for [Live Banning]A. Follow the instructions in step #1 above
but edit the $file value from
htaccess.txt to
.htaccess(Make sure the double quotes stay around the value and also no extension on the end)
B. Skip step #3 altogether - Testing is no longer needed
C. Continue the rest of the steps until you come to steps #9a, #10a, and #11a - skip those three steps and replace with the below steps #9b, #10b, 11b
- 9b. Now type your web URL in the browser address bar: http://www.YoUr_WeB_sItE.cOm
- 10b. You should be presented with an "Error 403 - Forbidden Access" page.
- 11b. Download your: .htaccess file from your root folder - open it in a text editor and look at the last line of the file - You should see the line: Deny from 12.34.56.789 (with your IP address instead) .
D. If you see the red message screen, have received the email with the ban info, and verified the "deny" text was written to the .htaccess file, and most of all, CAN'T ACCESS YOUR WEB SITE - EVERYTHING WORKED PERFECTLY!
How do I unban myself? [Live Banning]So now you can't access your web site. The Auto IP Ban Script worked perfectly but now you want to access your web site again.
The fix is simple. Download your:
.htaccess file from your root folder - open it in a text editor and delete the entire line that contains your IP address from the file and then FTP
upload the file back to your web account. Then refresh or reload your web page. You should now be able to access and see your web site again.
Adding Extras (Blocking access to other folders and files)With the working functionality of the Auto IP Ban Script, a web owner has the potential flexibility to practically block anything that does not exist in their web directory folders. As it may sound simple, this is the hard work that needs to be done to make the Auto IP Ban Script really effective.
If you want to add more folders or files that these miscreants have been looking sniffing or scanning for; the first thing you need to do is locate what they have already be trying to find. This information can be found in your web account logs. Depending on the frequency of the violations, you may have a huge amount of information or a very little amount to sift through. The best way to find these violations is to use a log viewer. I recommend:
http://www.kiwisyslog.com/kiwi-log-viewer-overview/ - You can sift through large filesize logs easy and filter to highlight specific things you need to find. Your best bet is to look for a cluster of lines by the same IP address that is attempting to access non-existent files or folders. They are usually grouped together in a span of about 1-15 minutes.
Once you have found all the violations and recorded them on paper or a text file, you will need to add them to the .htaccess file. As with in Step 4b. you can add extra lines with the corresponding violations you have found. Just add a new line and change the folder name or file name to match your recordings. If you want to block everything inside a folder, use the character set
/(.*)$ after each folder name. This will block everything inside that folder. It works also for different versions of the folder whether it be plural or numbered variations.
Example Additions:RewriteEngine On
RewriteRule ^cgi-bin /mytrap.php [NC]
RewriteRule ^cgi-bin/(.*)$ /mytrap.php [NC]
# Newly Added Folders
RewriteRule ^administrator /mytrap.php [NC]
RewriteRule ^horde /mytrap.php [NC]
RewriteRule ^phpmyadmin /mytrap.php [NC]
# Newly Added Files
RewriteRule ^cgi-bin/formemail.cgi /mytrap.php [NC]
RewriteRule ^hidden/login.php /mytrap.php [NC]
RewriteRule ^README /mytrap.php [NC]
(You can add as many as you like! Just make sure you are not using the folder or files you add or everyone that visits them will be banned from your web site)
Bot and Spider Trapping (The Tangled Web You Weave)The Auto IP Ban Script has an extended feature to lure and trap search engine spiders and robots that ignore or disobey your configuration settings in the "robots.txt" file. If a search engine spider or robot so happens to ignore your "robots.txt" file; you can catch them in the act and effectively ban them permanently from your web site. The setup is really easy.
1. Install, edit, configure and
verify that Auto IP Ban Script is working properly.
2.
Create or edit your
robots.txt file.
3.
Add the following lines to your robots.txt file
Disallow: mytrap.php
Disallow: /mytrap.php
4. Save and FTP
upload your robots.txt file to your root folder of your web account.
5. You can then choose to add a "blind" link at the top of your web page just below the <body> tag so that search engine spiders and robots will pick it up. If they ignore the robots.txt file - they will be banned.
<a href="http://www.YoUr_WeB_sItE.cOm/mytrap.php" title=""><img src="pixel.gif" alt="" height="1" width="1" /></a>
Make sure the image file you use is put in the proper folder according to the SRC info. Also, it is best to use an image that has a transparent background so it will not be visible to any regular visitor. We don't want to ban our honest visitors, now do we?
(Check with the more common respectful search engines [Google, Yahoo, etc] to see if your web site may be penalized for using "blind" links. Some search engines will mark/grade/rate your web site lower if you have these types of hidden links in your web code)
6. Sit back and watch the fun of spiders and robots being banned!
Maintenance Thereafter [Long Term Effects]You ask what are the drawbacks to using this script? Aside from unbanning regular nosy visitors, there is only one required maintenance feature. Depending on the frequency of sniffing and scanning attacks on your web site; you will have to periodically download and check your .htaccess file for excessive amounts of banned IP addresses. If you have a lot of scanning going on your web pages, your .htaccess can grow to a very large filesize in a few weeks. Moderate your .htaccess file and ascertain if the Auto IP Ban Script is right for your web site. If you are getting sniffed and scanned on a regular basis and it is becoming a serious issue, I highly recommend that you purchase some software that will handle the demand for better security.
ClosingBy no means is the Auto IP Ban Script supposed to be a fix-all to stopping web site scanning and disobedient search engines. Auto IP Ban Script is a primary step in an attempt to curb or discourage web site sniffing/scanning for non-existent files and or folders. Thus it will save precious bandwidth usage and processing time. Not to mention, it will reduce the size of your log files and give you some piece of mind in regards to web site security.
If you have any questions about Auto IP Ban Script; please post here or email me at:
inque187@yahoo.comEnjoy!
