rockarolla
Feb 13 2008, 03:17 AM
| | Hi,
I have made my web site solely stored in a SQL database...that will say if I need to load a page I take it our from the data base and then display it.
My question is: is any searcg engine able to ``crawl'' into my web site content?
I would appreciate some info so that I can change the way my web works.
|
Reply
Sten
Feb 13 2008, 03:29 AM
Provided this is what you mean, yes. So I gather you've stored everything in a database and using PHP to get a part of it and show it, just like a CMS? PHP is server side, it does everything on the server and then displays it to the user as normal HTML, which if a search engine finds, it will show. If you're not using search engine friendly URLs though then it probably wouldn't find it since it's only picking up the index.php page.
Reply
yordan
Feb 13 2008, 03:36 PM
By the way, pages needing passwords cannot be crawled, because the robots cannot guess your passwords. So, pages you protected or scripts needing you to login before entering make the things not searchable by robots.
Reply
rockarolla
Feb 14 2008, 03:08 AM
QUOTE(yordan @ Feb 13 2008, 03:36 PM)  By the way, pages needing passwords cannot be crawled, because the robots cannot guess your passwords.
So, pages you protected or scripts needing you to login before entering make the things not searchable by robots. Thats my point. The database table has an user/pass so it might not be accessible for a bot or it might be...I think they ``crawl'' a particular site by ``virtually opening'' it and then check the content...at least this is my experience...as I really need to know this I'm making some crawling around too to become an expert...  see how if I'll share it ... here are the advises given by G: http://www.google.com/support/webmasters/b...py?answer=40349
Reply
yordan
Feb 14 2008, 08:56 AM
Hi, RockaRolla, When I do crawling for fun, it simply opens the php scripts, so I get files named somehthing.php down to my PC, nothing more. For instance, in 4gallery photo galleries, I don't have the pictures. However, I have the php scripts, so I can re-create the website. But of course I cannot connect to the database (for instance at asta it's your database so, even guessing the passwords I could not connect) so I cannot reach dynamic pages which are displayed using a particular user's rights and name and passwords.
Reply
Quatrux
Feb 14 2008, 12:26 PM
Well, if your pages are dynamic, it would be very good for you to send the right headers to the bot/crawler and even to the browser, due to when apache or as I know any other http server sends a simple html file with the right content length and etc. but when a PHP file is generated/parsed apache can't really know the length of it, but it can be done with some php output buffering and sending the header with php, I think google is full of these kind of suggestions. 
Reply
Athlon1600
Aug 7 2008, 03:42 AM
that's what I thought too at first don't worry about it, every extension is indexable to some point. here is the useful tool: http://www.seochat.com/seo-tools/spider-simulator/enter your websites url and you will see your page exactly as google search engine spider would see.
Reply
FirefoxRocks
Aug 7 2008, 05:01 AM
My thought on this is that pages needing POST form data (user logins, not database logins) are not accessible by search engines. This is because robots cannot guess the input required to access the content (passwords, email addresses, etc). If you have links to content outputted by PHP from a database, it should be searchable.
Reply
toby
Aug 7 2008, 10:19 AM
You can tell google a user/pass, and sitemaps, but generally what it can't find a link to, it won't index.
Reply
Similar Topics
Keywords : php, file, searchable,
- Mysql And User File_priv
(0)
Re-ordering Welcome Screen & Moving Heavily Fragmented File
(1) On this computer the two accounts are Lane Family and Toby Lane. Can I move the second to the top
without renaming either, or with some hidden character? Are there any commonly fragmented files
that I can move (say to a usb drive), or should I start worrying about the bios battery or hard disk
or any other parts of this 11-ish year old computer failing?....
Need To Edit A Wav File [solved]
can you recommend me a free software ? (1) Hi, For ringtone purposes, I need a wav file smaller than 2 megs. This means editing my favourite
song, and shorten the wav file to the first seconds of the intro in order to be less than 2 megs.
Could you please recommend me a soft able to do that ? Easy to use and free if possible ? Or a
commercial one having a trial version ? Thanks in advance Yordan ------------------ Edit : OK,
Audacity did perfectly the job. I leave this post here, maybe this info could be useful for somebody
else.....
Linux Basic Command - For Storing Compilation Error To File
(1) Ex: Compiling a cpp file using a basic command " g++ filename.cpp " and to run the program use
./a.out , Then to store the compilation error to text file use this command. g++ test.cpp >
log.txt log.txt contains the compile time errors. ....
How To Copy File & Folders From Linux To Windows?.
(12) Many face problem while transfering file from Linux to windows and Vise versa. My solution for
this is "winscp405". Winscp is a windows freeware tool used to copy file from & to Linux machine.
To get this tool just google for " winscp405" tool and download this small package and install.
Enjoy working with Linux and windows /tongue.gif" style="vertical-align:middle" emoid=":P"
border="0" alt="tongue.gif" />....
Use Of Xml Properties File For One Key - Multiple Value Mapping
- should I use some other Class? (0) Here is a situation I have encountered. I know I can write a custom code to get the job done, but I
was wondering if the java.util.Properties class (or something else) had this inbuilt and I was
missing it in the documentation. Situation: Need to load from a file, a set of values (mutliple
values) for an associated single key. To illustrate, here is an example: propfile.properties
CODE <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE properties
SYSTEM "http://java.sun.com/dtd/properties.dtd"> <properties>....
Safari And Hosts File
Overriding the file?? (8) I found this to be quite alarming when I found that sites I blocked using HOSTS were still
accessible by Safari. My HOSTS file consists of many automatically added entries by Spybot S&D and
SpywareBlaster and similar programs, but I also have some user added entries also. It is a 228KB
file, and a small segment that I added looks like this: 127.0.0.1 myspace.com 127.0.0.1
www.myspace.com 127.0.0.1 ca.myspace.com I found it to be inaccessible by all user accounts
using Firefox, Internet Explorer, Opera and Flock. However, I tried accessing it using Safari an....
Problem With Move_uploaded_file()
Permission denied? (5) CODE Warning: move_uploaded_file(images/profile/Mordent.jpg)
[function.move-uploaded-file]: failed to open stream: Permission denied in
/home/mordent/public_html/withvalour/edit_image.php on line 42 Warning:
move_uploaded_file() [function.move-uploaded-file]: Unable to move
'/tmp/phpOWVz7o' to 'images/profile/Mordent.jpg' in
/home/mordent/public_html/withvalour/edit_image.php on line 42 Any ideas? The code seems fine,
as it works a treat when I run it locally (on WAMPServer), but as soon as I tr....
Extplorer
A PHP -and JavaScript- based File Manager (7) Browsing the ExtJS examples website i found this excellent web-based file manager called
eXtplorer . eXtplorer allows you to browse your webserver folders with an intuitive Layout which
makes working with files very easy, and thanks to the great ExtJS Javascript Library you can drag
& drop folders and files, filter directories and sort the file list using various criteria. You can
use eXtplorer to for example: browse directories & files on the server. edit, copy, move, delete
files. search, upload and download files. create and extract archives. create new fil....
Url File-access Is Disabled In The Server Configuration
(7) As per the php documentation:- CODE // Won't work; looks for a file named
'file.php?foo=1&bar=2' on the // local filesystem. include 'file.php?foo=1&bar=2';
// Works. include 'http://www.example.com/file.php?foo=1&bar=2'; So, I used the
following code to include the header.php file which I use to draw the top navigation menu on my
site. CODE <? include "http://" . $_SERVER['HTTP_HOST'] .
"/includes/header.php?d=../../" ?> It worked before, but now I see
the....
How To Download/upload File Through A Ftp Proxy Server With 'curl'
(3) This help is for linux machines behind a FTP proxy.. (Note: Squid is not a true FTP Proxy server )
Say the proxy server is squid configured to be running on ' 192.168.0.10' Port
'3128' then some of the things we can do.. * List Files On FTP Server curl --proxy
192.168.0.10:3128 ftp://xyz:mypass@sharethespace.com:21 * Download a file curl -o
STSServerInstaller.exe --proxy 192.168.0.10:3128
ftp://xyz:mypass@sharethespace.com:21/STSServerInstaller.exe * Upload multiple files to the
ftp server.. curl --upload-file "{rarlinux-3.6.0.tar....
Messed Up My Boot.ini File With Vista...
How can i fix it? (13) I wanted to install Vista without getting rid of my XP installation, so I left Xp on my C drive, and
installed Vista on another physical disk. I started running into problems with the dual booting when
i would switch from one OS to the next, and after a while Xp would start hanging on boot, and only
after several cold reboots would it load properly. So i was planning on uninstalling Vista, but the
hard drive i had it on got formatted by a friend of mine who thought he was doing me a favor. Now
when i switch on the computer i get the OS choice menu, typical Vista pre-sele....
Need Help Urgently (missing Or Corrupt Hal.dll File).
(6) I attempted to boot up my younger brothers computer dismorning, only to be greeted by an error in
white texton a black screen (before windows starts up) claiming that hal.dll was missing or corrupt.
I am not entirely sure what happened, but my searches have lead me to believe that my little sister
may have accidentaly edited the boot.ini file (hopefully not removing it's contents as I
don't remember how I set it up when I had trouble last time). I am wondering if anyone knows of
a simple single floppy boot disk that I could use to edit text files on the drive s....
Deleting A Corrupt File
Cannot delete a corrupt file on Windows XP... (25) I was upgrading some software and came across a corrupt file that halted the installation process. I
went to see what was wrong with the file, and failed to manually delete it. I tried moving
(cut&paste) it to another location, and the installation succeeded. However, the file is still on my
drive, taking up a KB of space. I know that's not a lot, but where am I supposed to put it? In a
folder created especially for all the corrupt files on my computer? /blink.gif"
style="vertical-align:middle" emoid=":blink:" border="0" alt="blink.gif" /> I've heard
something....
How Do You Use The cPanel File Manager?
How to use File Manager? (13) I feel stupid for asking this but I can’t figure out how to use file manager. How do you
open folders and view the file in it? When I click on a certain folder the options does not include
opening it. And can you upload file from the file manager? I’ve been using FTP to manage my
files so far. ....
You Cannot Create A File Named Con
(8) did you know, that on MS Word, you cannot name a document con? nobody can figure out why!
....
How To Use Psd File
(14) I'm redesigning my website, and I found a template for it that is in a PSD format. I have Adobe
Photoshop so I can open it and edit it. I changed the text and images exactly the way I want it, but
what do I do from here? How do I get the images and text from the PSD to the individual files that
makes it up? Or am I supposed to be doing that at all? Thanks....
Uploading Image File Through JSP Code To Server
(9) I have to upload images from client side to server side, without using input tag or browseing of
file. I know the absolute path of file in client side. And that file I want to shift to server with
JSP code or any servlet code. I have done uploading through servlet to the server with input tag
and type is file . Servlet code : CODE import com.oreilly.servlet.MultipartRequest;
import java.io.*; import java.util.*; import javax.servlet.*; import javax.servlet.http.*; public
class UploadPhoto extends HttpServlet { public void doPost(HttpServletRequest req....
What Is Mkv? Need Help With This File Format?
(6) I've download a movie, but I don't know how to open it. What's MKV format. Never used
that before. This is the file name : Final_Fantasy_VII_Last_Order_v2.DVD(H264.AAC) .mkv I'm
very excited to watch the movie. So, please reply as soon as possible. Please include information
about the codec in your post. Where to download the codec ?....
Restore Windows File And Folder Protection
(4) Restore Windows File And Folder Protection Windows XP offers many new and more effective features
to protect your system. For example, Windows hides files and folders stored on Windows, Program
files and system directory to protect them from being changed or deleted. When you want to open
system folders for first time, windows displays below screen These Files Are Hidden This folder
contains files that keep your system working properly. You should not modify its contents Show the
contents of this folder If you click on “Show the contents of this folder” in left p....
Text File Operations VB.NET
(5) In this tutorial you will learn most of the operations you can use on a text file. They include
finding if a file exists, opening/creating a file, reading/writing file, closing file, copying
file, deleting file. You will need a form with two buttons on it. Use the names Step1 and Step2.
First thing we are going to do is import system.IO. To do this go into the forms code view. At
the very top add this line. CODE Imports system.IO This lets us use the file operations
that we need for this tutorial. Next we need to check to see if a file exists. To ....
Create And Email PDF File 'On The Fly' With PHP
(1) Create a pdf on the fly, I am sure that many people know how to do it but not attaching those into
email. You can have a lot tutorials about generating a pdf on the fly without storing anything on
webserver. This technique allow you to customise the pdf as client's requirements. Let's
say you have a huge database of various types of books. On your website, visitors have the option to
download sample booklet (in pdf format) of a book for which they are looking. In the old days, you
have to create all of those pdf files manually or whatever way you can then upload....
How To Increase Windows Shutdown Speed
when you enabled “clear Page File at shu (39) How to increase Windows shutdown speed when you enabled “clear Page File at shutdown”? /blink.gif'
border='0' style='vertical-align:middle' alt='blink.gif' /> Note that making incorrectly changes
to the registry may damage your system; please back up any valued data on your computer. 1) Go to
Start menu > Run > and type regedit and click OK. 2) Find following address: HKEY_LOCAL_MACHINE
\ SYSTEM \ CurrentControlSet \ Control \Session Manager \ Memory Management
3) In right side right click on “ClearPageFileAtShutdown”, select Modify and mak....
FTP Or File Manager - Which Is Better?
Which is better? (37) Hey guys talking about this topic here, what do you guys think is better? Using file manager or Ftp?
I mean, everytime i use ftp it is so slow! File manager is quite fast but you can't do much
with it if you want to upload lots of files at once. Yet many people seem to use ftp more than file
manager. I even edit my sites using the HTML editor in the file manager of cpanel. What is what?
Which is better to use? Which is more efficient? /blink.gif' border='0'
style='vertical-align:middle' alt='blink.gif' /> Please do not take topics off into random
tangents. ....
Recover Tables From A MySQL .frm File
(8) I have a couple of .frm files with no corresponding data or index files. Is it possible to recover
the table structure (field names, types, sizes, rows,col, etc) from these files? The table type is
innodb....
How To Save A Image In Pdf File And Download It?
Programming script need hepl. (10) Can anyone tell how to save an image into a PDF file and download it without need to save it on the
server? Thanks. All sugesstions are welcome! Wrong section. Topic moved to Software >
Graphics & Web Design. ....
How Do I Compress One File Into Several Parts?
Using Winrar (10) Ok, i have a 70 mb folder compressed with winrar, and i tried to uploaded into a "service" on the
internet, but they only allow uploads up to 10mb.. That's why i need to upload the folder in 7
equal parts of 10mb each... I know it's possible in winrar beacuse i have downloaded files with
this specs (separated, like a back-up) So, if u know how to, please tell me Thanks!....
Shut Down, Restart, Log Off XP Using A Batch File
Undocumented feature for XP (23) How to shutdown a XP Pro computer when you are connected using RDC (Remote Desktop Connection) using
a batch file or RPC (Remote Procedure Command) This is useful if you can't get to the computer
but have remote access to the computer via a network or the internet. First To show you what you
will see at a command prompt (command.com) when you type: "shutdown /?" Without the " " on each end.
/smile.gif' border='0' style='vertical-align:middle' alt='smile.gif' /> CODE
c:\>shutdown /? Usage: SHUTDOWN [-i | -l | -s | -r | -a] [-f ....
How Can I Open A *.sit File
(10) I just down a *.sit file. how can I open it?? Which program should I use??....
Php Script To Download File From Another Site
(9) hi i need a php or java script code for downloading files from other sites to my site for example:
http://download.com/file.zip to http://mysite.com/file.zip thanks....
Looking for php, file, searchable,
|
|
Searching Video's for php, file, searchable,
|
advertisement
|
|
