|
|
|
|
![]() ![]() |
Aug 27 2005, 05:17 AM
Post
#1
|
|
|
Member [ Level 1 ] Group: Members Posts: 43 Joined: 27-August 05 From: My House, Some Island, Earth, The Milkey Way Member No.: 8,129 |
Do the Google crawlers pick up info about all/most of the websites hosted here? If they do how frequently do they scan through the servers? Do all hosted sites include a robot.txt thingy or .htaccess (forgot what one google looks at)?
lol, my last site got on it but then my host started stuffing up my site so i stopped using it. QUOTE This post has been edited by microscopic^earthling: Today, 07:41 PM What was changed? This post has been edited by TheJeffsta: Aug 27 2005, 08:09 AM |
|
|
|
Aug 27 2005, 04:28 PM
Post
#2
|
|
|
S.P.A.M.S.W.A.T. Group: Members Posts: 814 Joined: 22-January 05 From: San Antonio, Texas (No, I'm not dumb. I just moved here...) Member No.: 2,284 |
Crawlers crawl through links of sites they have already indexed, so Astahost links to your site, then Google should be able to crawl to your site. If you don't want them to, you can specify it in the "robot.txt thingy"
|
|
|
|
![]() ![]() |
Similar Topics
|
Lo-Fi Version | Time is now: 8th October 2008 - 05:15 AM |