Welcome Guest ( Log In | Register )



 
 Reply to this topicStart new topic
> MSN Help, Problem
GM-University
post Jul 19 2005, 08:56 PM
Post #1


Member [ Level 1 ]
Group Icon

Group: Members
Posts: 32
Joined: 4-May 05
Member No.: 4,707
I have a problem with MSN search, I was under the impression that it followed link on my site, but for some reason it is finding pages that don't exsist on my forums, or really wierd ones, it hasn't read any posts, it just checks the login and registration pages and all sorts of wierd things...
I am sure that the group that the bots are treated as have acess to the forums to read them, but they still don't check posts, it just comes every day and see's if my users long-in differently now... laugh.gif
Go to the top of the page
 
+Quote Post
runefantasy
post Jul 20 2005, 06:24 AM
Post #2


Member - Active Contributor
Group Icon

Group: Members
Posts: 90
Joined: 29-June 05
Member No.: 6,693
They think it's a different page that you uploaded... You can block search engines from searching those 'weird' pages by using robot.txt
Go to the top of the page
 
+Quote Post
Trekkie101
post Jul 20 2005, 07:38 AM
Post #3


Teh Teckeh Trekkeh
Group Icon

Group: Members
Posts: 682
Joined: 8-September 04
From: Scotland, UK
Member No.: 389
phpBB by any chance?

Its pretty useless at helping search bots find anything.
Go to the top of the page
 
+Quote Post
GM-University
post Aug 9 2005, 12:25 AM
Post #4


Member [ Level 1 ]
Group Icon

Group: Members
Posts: 32
Joined: 4-May 05
Member No.: 4,707
No, it was IPB, and the page was /index.php?somethingcrazyhere=somethingcrazyhere
so it wasn't something I uploaded, not really...
Go to the top of the page
 
+Quote Post
Klass
post Aug 9 2005, 12:35 AM
Post #5


Klassified
Group Icon

Group: Members
Posts: 156
Joined: 7-June 05
From: Klass-World
Member No.: 5,936
Here is my robots.txt for my Vbulletin forum:

CODE
User-agent: *
Disallow: /cgi-bin/
Disallow: /forums/attachment.php
Disallow: /forums/avatar.php
Disallow: /forums/editpost.php
Disallow: /forums/member.php
Disallow: /forums/member2.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/private2.php
Disallow: /forums/report.php
Disallow: /forums/search.php
Disallow: /forums/sendtofriend.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/admin/
Disallow: /forums/images/
Disallow: /forums/mod/
Disallow: /admin/
Disallow: /cache/
Disallow: /albums/
Disallow: /gallery/
Disallow: /downloads/
Disallow: /images/
Disallow: /includes/
Disallow: /language/
Disallow: /manuel/
Disallow: /modules/
Disallow: /netpbm/
Disallow: /tmp/
Disallow: admin.php
Disallow: auth.inc.php
Disallow: banners.php
Disallow: config.php
Disallow: counter.php
Disallow: download.php
Disallow: footer.php
Disallow: friend.php
Disallow: header.php
Disallow: index.php
Disallow: links.php
Disallow: mainfile.php
Disallow: modules.php
Disallow: reviews.php
Disallow: sections.php
Disallow: stats.php
Disallow: top.php
Disallow: topics.php
Disallow: user.php
Disallow: vbp_includes.php
Disallow: vbpolls.php
Disallow: viewtopic.php
Disallow: voteinclude.php


Might help you set yours up for IPB.
Yes I have most of it disallowed but what is not disallowed the spider can crawl
Go to the top of the page
 
+Quote Post
« Next Oldest · MSN Search · Next Newest »
 

Reply to this topicStart new topic

Collapse

> Similar Topics

Topics Topics
  1. Spam Problem On My Forums(25)
  2. Problem With Move_uploaded_file()(5)
  3. XP Problem: Clicking On Folder Opens Search(4)
  4. Trojan / Virus Problem ,please Help(16)
  5. Einstein Quiz(29)
  6. Problem With The ReadOnly Property(3)
  7. Blue Screen - irql_not_less_or_equal(29)
  8. Administrator Account Problem In Microsoft Xp [solved](17)
  9. Mozilla Firefox And Runescape(10)
  10. Problem With Acdsee(4)
  11. Laptop Overheating(15)
  12. Need To Hack An Admin Account On Xp... No Problem!(60)
  13. Frustrating Problem With XP On Laptop(18)
  14. Problem Deleting Empty Folder In Win XP(13)
  15. Problem With Purchase Credits Page(7)
  1. Problem With Drag And Drop (or So It Seems).(10)
  2. Mp3 Player Problem: Sound In One Ear Only(8)
  3. Permission Problem With Mysql Database Creation(8)
  4. Mysql Problem(1)
  5. Live Messenger Display Pic Problem(5)
  6. I Can't Turn Off My Computer(9)
  7. Internet Explorer 7 Problem(6)
  8. Problem: Network Connection Fails Every 2 Mins(12)
  9. DVD-RW Problem: Unformatted Disk(6)
  10. Whats Wrong In Altec Lansing Sound System ?(5)
  11. The Problem With #(1)
  12. Baylan Network Switch Problem(1)
  13. Fedora Core 6 Install Problem(5)
  14. The Mind-body Problem(9)
  15. Html Table Rows Problem [solved](1)


 

Display Mode: Standard · Switch to: Linear+ · Switch to: Outline

Track this topic · Email this topic · Print this topic · Subscribe to this forum

- Lo-Fi Version Time is now: 7th July 2008 - 12:08 AM
Powered By IP.Board 2.2.1 © 2008  IPS, Inc.
Licensed to: Xisto Corporation