Robots-txt-generator 1.01

    robots-txt generator The spiders that visit your site consumes much of your bandwidth? If you want to control the robots or spiders that visit your site, robots-txt generator is the solution.

It is very common that the webmaster of each site spends a lot of money paying for bandwidth and it is unfair that a significant portion of that bandwidth is consumed spiders.
Robots txt generator, you can stop or control the bots.

Some robots or spiders saturate your server making countless queries per minute?

With Robots txt generator, you can limit how many requests can make any robot or spider per hour or per minute.
In addition, the program has a database of more than 300 bots, allowing you to search on the database the source and owner of any bot you have found in your logs.
You can even specify different restrictions for different bots. Per example, you can allow google bot, but disallow google-image-bot. You can specify a different restriction level for each one of the 300 bots in the database or can specify some restrition to the "general" bots, plus some specific for some specifi bots. Like giving a ban to some specific bot.
Best of all, is that the software is free!
 
Robots-txt-generator

Size

 0.05 MB

License

 Freeware

Price

 0 USD

Category

 Web Development / Site

Operating System

 Windows2000,WinXP,Windows2003,Windows Vista Starter,Windows Vista Home Basic,Windows Vista Home Premium,Windows Vista Business,Windows Vista Enterprise,Windows Vista Ultimate

Developer

 Robots txt
Program system requirements: none
Download Now
File size 0.05 MB
 Buy Now
Tags:  robots txt, robots txt generator, web spider, web bot, bad bot,

Copy below code into your website if you want link to Robots-txt-generator page
<a href="http://www.top-shareware.net/Robotstxtgenerator.html">Download Robots-txt-generator</a>

Top Site / Web Development programs

SiteScan XP - Link Checker & SiteMapper 1)   SiteScan XP - Link Checker & SiteMapper 1.4
All-in-one Broken Link Checker, Performance Monitor & Google SiteMap Generator

 2)   Link Site Maker 1.0
Link Site Maker creates beautiful link sites.

SiteMap XML Dynamic SiteMap Generator 3)   SiteMap XML Dynamic SiteMap Generator 1.1
SiteMap XML Dynamic SiteMap Generator Google, XML, RSS, HTML and Text site maps.

FlippingBook joomla extension 4)   FlippingBook joomla extension 1.0
FlippingBook joomla extension - joomla page flipping engine

Site Builder Software 5)   Site Builder Software 1.0
Build a dynamic CMS web site with automatic RSS Feeds, Meta Tags and valid XHTML

Warden .htaccess Manager 6)   Warden .htaccess Manager 1.1
Password protect your website and manage user accounts in three easy steps.

 7)   Sitemap 4 Traffic 1.0
Sitemap 4 traffic is a Free Google Sitemap and html sitemaps generator.



New Site / Web Development programs

Sitecraft 2008 1)   Sitecraft 2008 4.16
Sitecraft - constructor of sites. Build a site without technical difficulties.

Free Link Checker 2)   Free Link Checker 4.3
Free broken link check tool for a web site, broken link testing software

Inspyder Sitemap Creator 3)   Inspyder Sitemap Creator 1.2.0
Create an HTML or XML sitemap with easy to use software!

RemoteIntegrity Website Scanner 4)   RemoteIntegrity Website Scanner RI_1000
Monitor your website for hacking.

 5)   Web Hosting Buyers Guide 1.5
Choose the right web hosting provider for your web site

Nesox Link Checker Free Edition 6)   Nesox Link Checker Free Edition 2.0
Bulk web link validator, pagerank, link popularity and traffic rank checker

Inspyder InSite 7)   Inspyder InSite 1.4.1
Spell check your website and find broken links, quickly and easily!
 
Sitemap 4 Traffic
Sitemap 4 traffic will help webmasters to build Google sitemap, build Html sitemaps to be uploaded to their website, Check brocken links on their website, Backup their website files (without server code) and Make their website much better to be crawl