Jump to content



Welcome to AstaHost - Dear Guest , Please Register here to get Your own website. - Ask a Question / Express Opinion / Reply w/o Sign-Up!

Toggle shoutbox Shoutbox Open the Shoutbox in a popup

@  agyat : (23 May 2013 - 01:23 AM) Wow! Mr. Sb Back Home.
@  OpaQue : (23 May 2013 - 12:44 AM) Ting
@  OpaQue : (24 April 2013 - 02:44 PM) I guess, Time to run Mycent script.
@  OpaQue : (24 April 2013 - 02:43 PM) wow.. not much spam. except habatt posting lot of links.. :P
@  yordan : (23 April 2013 - 01:04 PM) You're welcome, agyat. Nice to have been helpful. Second lesson: try full words, "you" instead of "EW".
@  agyat : (23 April 2013 - 05:03 AM) @YORDAN: tHANK EW FOR YOUR FIRST LESSON.   :D
@  yordan : (22 April 2013 - 09:43 PM) @agyat : "why don't you help me", or "please help me", or "please teach us"
@  yordan : (22 April 2013 - 09:42 PM) welcome back, velma
@  velma : (22 April 2013 - 07:51 AM) **yawns** Good to be back, wonder what is going on here :)
@  agyat : (22 April 2013 - 03:50 AM) Oh! so, why don't help me learn english..
@  yordan : (21 April 2013 - 08:38 PM) The goal mentioned by shiu : "learning english, learning computer"
@  agyat : (21 April 2013 - 06:31 PM) WHAT GOAL?
@  yordan : (20 April 2013 - 10:39 AM) yes, that's our goal. simultaneouly learning English and teaching/learning computer using.
@  shiyu : (20 April 2013 - 07:30 AM) learning english,learning computer
@  yordan : (19 April 2013 - 01:11 PM) Oh, I see, it's just a trick in order to force people looking at your texte. Somehow smart, maybe.
@  agyat : (19 April 2013 - 02:54 AM) And of course I know it is not SEO friendly.
@  agyat : (19 April 2013 - 02:52 AM) There may be two possible answers for that ....


1) Shout was posted using mobile keypad.

2) To force people read content carefully and/or with more concentration.
@  agyat : (19 April 2013 - 02:49 AM) There may be two possible answers for that ....
@  yordan : (18 April 2013 - 09:35 PM) however, why this mixing of capital letters in the middle of your text?
@  agyat : (18 April 2013 - 11:10 AM) false feelings.

Photo
- - - - -

Crawl.googlebot.com

Started by yordan, Apr 26 2009 02:47 PM

12 replies to this topic

#1 yordan

yordan

    Way Out Of Control - You need a life :)

  • [MODERATOR]
  • 4,677 posts

Posted 26 April 2009 - 02:47 PM

I had a look at my private forum, and I saw a guest connected. I had a look at his IP address, did a whois, and found that this guest was
crawl-66-249-73-137.googlebot.com
How did this googlebot come to my private website ?
Is there a way to prevent these bots from crawling my website ?

#2 hfbvm

hfbvm

    Member - Active Contributor

  • Members
  • 83 posts
  • Gender:Male
  • Location:Saudi arabia
  • myCENTs:24.23

Posted 26 April 2009 - 05:49 PM

just make a robots.txt file.robots.txt will work as you want. You can even disable crawling for specific pages, so you don't have to move your pages into a folder. Something like:
User-agent: *
Disallow: /index.php
Disallow: /main.php
Disallow: /inc/
Disallow: /img/
Disallow: /images/
Disallow: /js/
Disallow: /css/
And so on and so forth its easy more queries ask me maybe i could help you

#3 yordan

yordan

    Way Out Of Control - You need a life :)

  • [MODERATOR]
  • 4,677 posts

Posted 26 April 2009 - 06:08 PM

Seems to be exactly what I want. Thanks, I'll try that way.

#4 hfbvm

hfbvm

    Member - Active Contributor

  • Members
  • 83 posts
  • Gender:Male
  • Location:Saudi arabia
  • myCENTs:24.23

Posted 26 April 2009 - 07:39 PM

where's my rep for this?

#5 yordan

yordan

    Way Out Of Control - You need a life :)

  • [MODERATOR]
  • 4,677 posts

Posted 26 April 2009 - 07:51 PM

where's my rep for this?

Your rep is here : http://www.astahost....s...st&p=136357
Can't you really see it ?

#6 zorba4

zorba4

    Newbie [ Level 2 ]

  • Members
  • 15 posts
  • myCENTs:22

Posted 26 April 2009 - 08:14 PM

where's my rep for this?

I also see your rep, you said :

User-agent: *
Disallow: /index.php
Disallow: /main.php
Disallow: /inc/
Disallow: /img/
Disallow: /images/
Disallow: /js/
Disallow: /css/

does your browser have a problem? I use IE7

#7 yordan

yordan

    Way Out Of Control - You need a life :)

  • [MODERATOR]
  • 4,677 posts

Posted 26 April 2009 - 08:16 PM

does your browser have a problem? I use IE7

I use Firefox 2, and I see hfbvm's rep. Maybe he started sleeping ?

#8 zorba4

zorba4

    Newbie [ Level 2 ]

  • Members
  • 15 posts
  • myCENTs:22

Posted 26 April 2009 - 08:38 PM

Maybe he started sleeping ?

Or he forgot refreshing his browser window?

#9 hfbvm

hfbvm

    Member - Active Contributor

  • Members
  • 83 posts
  • Gender:Male
  • Location:Saudi arabia
  • myCENTs:24.23

Posted 27 April 2009 - 01:30 PM

yes i can now see it sorry for the inconvenience or whatever.
I also use firefox.

#10 aloKNsh

aloKNsh

    Member [ Level 2 ]

  • Members
  • 71 posts

Posted 28 April 2009 - 08:36 AM

yeah i can also see....
and i use safari

#11 yordan

yordan

    Way Out Of Control - You need a life :)

  • [MODERATOR]
  • 4,677 posts

Posted 28 April 2009 - 09:44 AM

yeah i can also see....
and i use safari

Why safari ? Are you a Mac User ?

#12 FirefoxRocks

FirefoxRocks

    Super Member

  • [HOSTED]
  • 988 posts
  • Gender:Male
  • Location:Ontario, Canada
  • myCENTs:92.36

Posted 03 May 2009 - 03:11 AM

Why safari ? Are you a Mac User ?

You do realize that there is also Safari for Windows. It is very similar to Google Chrome minus some features.

I'm using Firefox 3 on Ubuntu 9,04 at the moment. Just installed it today.

#13 yordan

yordan

    Way Out Of Control - You need a life :)

  • [MODERATOR]
  • 4,677 posts

Posted 03 May 2009 - 12:16 PM

I'm using Firefox 3 on Ubuntu 9,04 at the moment. Just installed it today.

Exactly what I said. Linux+windows people use Firefox.
Mac people send you mails with .websave files, and they a surprised learning that safari is not your default browser.
Back to Google

Reply to this topic



  


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

  1. Open Discussion & Free Web Hosting
  2. → Computers & Tech
  3. → Search Engines
  4. → Google
  5. Rules