Expand / Collapse Navigation
|
|
|
| Web Hosting Guide |
 |
MSN Help, Problem |
 Jul 19 2005, 08:56 PM
Post
#1
|
|
|
Member [ Level 1 ]  Group: Members Posts: 32 Joined: 4-May 05 Member No.: 4,707  |
I have a problem with MSN search, I was under the impression that it followed link on my site, but for some reason it is finding pages that don't exsist on my forums, or really wierd ones, it hasn't read any posts, it just checks the login and registration pages and all sorts of wierd things...
I am sure that the group that the bots are treated as have acess to the forums to read them, but they still don't check posts, it just comes every day and see's if my users long-in differently now... 
|
 |
 
|
|
Jul 20 2005, 06:24 AM
Post
#2
|
|
|
Member - Active Contributor Group: Members Posts: 90 Joined: 29-June 05 Member No.: 6,693 |
They think it's a different page that you uploaded... You can block search engines from searching those 'weird' pages by using robot.txt
|
|
|
|
|
Jul 20 2005, 07:38 AM
Post
#3
|
|
|
Teh Teckeh Trekkeh Group: Members Posts: 682 Joined: 8-September 04 From: Scotland, UK Member No.: 389 |
phpBB by any chance?
Its pretty useless at helping search bots find anything. |
|
|
|
|
Aug 9 2005, 12:25 AM
Post
#4
|
|
|
Member [ Level 1 ] Group: Members Posts: 32 Joined: 4-May 05 Member No.: 4,707 |
No, it was IPB, and the page was /index.php?somethingcrazyhere=somethingcrazyhere
so it wasn't something I uploaded, not really... |
|
|
|
|
Aug 9 2005, 12:35 AM
Post
#5
|
|
|
Klassified Group: Members Posts: 156 Joined: 7-June 05 From: Klass-World Member No.: 5,936 |
Here is my robots.txt for my Vbulletin forum:
CODE User-agent: * Disallow: /cgi-bin/ Disallow: /forums/attachment.php Disallow: /forums/avatar.php Disallow: /forums/editpost.php Disallow: /forums/member.php Disallow: /forums/member2.php Disallow: /forums/misc.php Disallow: /forums/moderator.php Disallow: /forums/newreply.php Disallow: /forums/newthread.php Disallow: /forums/online.php Disallow: /forums/poll.php Disallow: /forums/postings.php Disallow: /forums/printthread.php Disallow: /forums/private.php Disallow: /forums/private2.php Disallow: /forums/report.php Disallow: /forums/search.php Disallow: /forums/sendtofriend.php Disallow: /forums/threadrate.php Disallow: /forums/usercp.php Disallow: /forums/admin/ Disallow: /forums/images/ Disallow: /forums/mod/ Disallow: /admin/ Disallow: /cache/ Disallow: /albums/ Disallow: /gallery/ Disallow: /downloads/ Disallow: /images/ Disallow: /includes/ Disallow: /language/ Disallow: /manuel/ Disallow: /modules/ Disallow: /netpbm/ Disallow: /tmp/ Disallow: admin.php Disallow: auth.inc.php Disallow: banners.php Disallow: config.php Disallow: counter.php Disallow: download.php Disallow: footer.php Disallow: friend.php Disallow: header.php Disallow: index.php Disallow: links.php Disallow: mainfile.php Disallow: modules.php Disallow: reviews.php Disallow: sections.php Disallow: stats.php Disallow: top.php Disallow: topics.php Disallow: user.php Disallow: vbp_includes.php Disallow: vbpolls.php Disallow: viewtopic.php Disallow: voteinclude.php Might help you set yours up for IPB. Yes I have most of it disallowed but what is not disallowed the spider can crawl |
|
|
|
|
Oct 12 2008, 04:19 PM
Post
#6
|
|
|
Super Member  Group: [HOSTED] Posts: 873 Joined: 12-July 06 From: Ontario, Canada Member No.: 14,464 myCENTs:79.31 |
You can try using Live Webmaster Tools to manage what Live Search crawls for a specific site. It isn't as advanced as Google Webmaster Tools, but it is mainly for Live Search and is quite useful actually.
http://webmaster.live.com/ |
|
|
|
|
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:
|
Lo-Fi Version | Time is now: 15th March 2010 - 02:43 PM |
© 2010 AstaHost: Free Web Hosting & Technical Discussion, Free Web Hosting. a member of xisto.
Powered by Invision Board. Skin: IPB Forum Skins
