Index
 |
Make Your Own Search Engine? - Is it possible to make your own engine? | ||
Discussion by infamousflame with 30 Replies.
Last Update: April 12, 2012, 3:40 pm (View Latest) | Page 1 of 2 pages. | ||
 |
|||
Does anyone know how to make your own search engine? Does it take lots of time and careful programming or is it quite quick and easy to make a basic one? Post here!
Sun Jun 19, 2005 Reply New Discussion
Youcould start by trying this link and maybe you can decide if that is what you want to do! How Intertnet Search Engines Work! Hope the link will get you going where you want to go.
Sun Jun 19, 2005 Reply New Discussion
Just opnions.
I have never made a search engine though. Never had the need to. But I have some idea. It mostly depends on the search space. It less abouit programming and more about using the right tool. Searching Algorithms are no doubt one of the most studied algos out there with varying implementations..so is the case with search engines them selves. It would be wise to look into already developed search engines.
I need not mention sites like sourceforge.net tigris.com etc which host projects like these. If you find some thing interesting you may want to join them.
just curious , do you want to venture into search engine technology just for the sake of it or do u have an actual problem to solve (ie you are building a website of some sort)
Also search engines are taking new approaches these days , ie sound based , video , 3d etc etc and not just plain old text.
You may want to check http://en.wikipedia.org/wiki/Search_Engine out
I have never made a search engine though. Never had the need to. But I have some idea. It mostly depends on the search space. It less abouit programming and more about using the right tool. Searching Algorithms are no doubt one of the most studied algos out there with varying implementations..so is the case with search engines them selves. It would be wise to look into already developed search engines.
I need not mention sites like sourceforge.net tigris.com etc which host projects like these. If you find some thing interesting you may want to join them.
just curious , do you want to venture into search engine technology just for the sake of it or do u have an actual problem to solve (ie you are building a website of some sort)
Also search engines are taking new approaches these days , ie sound based , video , 3d etc etc and not just plain old text.
You may want to check http://en.wikipedia.org/wiki/Search_Engine out
Sun Jun 19, 2005 Reply New Discussion
I was just curious about how easy it would be to make your own search engines , I don't think I have time to do it now though. It's probably all too technical for me. I don't know so much about programming, so I might not be able to do it. I know you said it isn't so much about programming, but it would still probably be too much of an extra workload for me, I've got a lot going on with my site and with my forum. Thanks for givng me such good info guys. 
Sun Jun 19, 2005 Reply New Discussion
QUOTE (infamousflame)
I was just curious about how easy it would be to make your own search engines , I don't think I have time to do it now though. It's probably all too technical for me. I don't know so much about programming, so I might not be able to do it. I know you said it isn't so much about programming, but it would still probably be too much of an extra workload for me, I've got a lot going on with my site and with my forum. Thanks for givng me such good info guys.
[post="40200"]<{POST_SNAPBACK}>[/post]
Creating a search engine needs a huge database i presume and a large brain to scour through all the algos of searching, sorting and implementing the best to suite your needs... I feel its about a effecient programming than about large programming. You need to have all the knowledge about internet and how it works. How you rate pages and index them for better searh...
i think you should start creating searchengine for a localhost now and make it a big boom on the internet if you succeed in implementing the better algos. Like google does...
Best of luck dude
Tue Aug 9, 2005 Reply New Discussion
Programming the search engine is one thing - hardware is another! You need much space (many many harddisks), much RAM, many processors and voltage ... and that all costs MONEY.
GreetingZ
GreetingZ
Tue Aug 9, 2005 Reply New Discussion
Creating search engine is simple. Really! But if you want search engine like Google and yahoo so its not simple anymore. lol!
There's a lot of free search engine scripts out there ready to use for your site.
There's a lot of free search engine scripts out there ready to use for your site.
Fri Sep 2, 2005 Reply New Discussion
Yes it's true
Sat Sep 2, 2006 Reply New Discussion
QUOTE (infamousflame)
Does anyone know how to make your own search engine? Does it take lots of time and careful programming or is it quite quick and easy to make a basic one? Post here!
Link: view Post: 40168
Best practice to learn some topic is to explore existing implementations. Start by reading sources.
I can recommend MnoGoSearch and AspSeek. First written in perl AFAIK, second C++ and MySQL.
Happy hacking!
Fri Nov 17, 2006 Reply New Discussion
If you would like to create a search section/site search area for your site, there are 3 methods:
1. Use free, easy to use, PHP scripts. You can find plenty here. Just find the free ones underneath the commercial ones and choose the one that is most suitable.
2. Use Google SiteSearch.
3. Investigate and research how to build your own PHP/ASP files to create your own search engine to index your site. You may also want a script for a sitemap.
The first two options are easiest. Good luck!
1. Use free, easy to use, PHP scripts. You can find plenty here. Just find the free ones underneath the commercial ones and choose the one that is most suitable.
2. Use Google SiteSearch.
3. Investigate and research how to build your own PHP/ASP files to create your own search engine to index your site. You may also want a script for a sitemap.
The first two options are easiest. Good luck!
Sun Nov 26, 2006 Reply New Discussion
Think of an unorganized library, with all the books lying here and there. I tell you to find me a book that contains any description of Neural Networks. To make matter worse, all the books have their indexes ripped apart. All that helps you is the title of the book and the contents.
Now just imagine what searching for a topic would be like. Your first step will be to search by the title of the books. But its not surprising that many books have weird titles which may not convey what the book might be containing within.
So in this case having a index in each book would have helped a load. But still, since the library is unorganized and books are lying in a pile, you would have to search the books by picking them up one by one. In the worst case, the book you are finding might be at the last of the heap. In this case the algorithm time complexity is n, where n is the no. of books in the library.
Now what you do is sort the books in sections depending upon their core theme. A book on artificial neural networks shall actually be stashed in Computer Science section and not Neurology/Biology section.
Now further more, to make things even more fast, you introduce something called a frequently issued section, that contain only the books frequently issued. It happens that most of the books requested for issue come from the frequent section. Simply, now you end up making your algorithm faster.
Now remember that you categorized books in sections? Let's do one more thing. The books that appear the most in bibliography of other books reserve a seat for themselves in the top shelf, indicating that they are the most popular and reliable books. This, my dear friend is your introduction to page rank algorithm used by Google.
Why I gave this example is because the very first model described in para 1 can be made in 2 hours. The time required and programming skills needed subsequently increase.
Now lets talk about programming and requirements.
Hardware (you may also do with a single PC with a 160GB HDD):
> Loads of storage space (RAID)
> A well connected database
> Multiple Servers for load sharing
Programming:
> Web crawler (that fetches the web pages and puts them in the Data warehouse)
> Data organizer (that arranges the data so that it can be mined efficiently)
> Search Algorithm
You can read some good white papers regarding search engine technology and settle upon a time frame in which you can make a search engine.
Now just imagine what searching for a topic would be like. Your first step will be to search by the title of the books. But its not surprising that many books have weird titles which may not convey what the book might be containing within.
So in this case having a index in each book would have helped a load. But still, since the library is unorganized and books are lying in a pile, you would have to search the books by picking them up one by one. In the worst case, the book you are finding might be at the last of the heap. In this case the algorithm time complexity is n, where n is the no. of books in the library.
Now what you do is sort the books in sections depending upon their core theme. A book on artificial neural networks shall actually be stashed in Computer Science section and not Neurology/Biology section.
Now further more, to make things even more fast, you introduce something called a frequently issued section, that contain only the books frequently issued. It happens that most of the books requested for issue come from the frequent section. Simply, now you end up making your algorithm faster.
Now remember that you categorized books in sections? Let's do one more thing. The books that appear the most in bibliography of other books reserve a seat for themselves in the top shelf, indicating that they are the most popular and reliable books. This, my dear friend is your introduction to page rank algorithm used by Google.
Why I gave this example is because the very first model described in para 1 can be made in 2 hours. The time required and programming skills needed subsequently increase.
Now lets talk about programming and requirements.
Hardware (you may also do with a single PC with a 160GB HDD):
> Loads of storage space (RAID)
> A well connected database
> Multiple Servers for load sharing
Programming:
> Web crawler (that fetches the web pages and puts them in the Data warehouse)
> Data organizer (that arranges the data so that it can be mined efficiently)
> Search Algorithm
You can read some good white papers regarding search engine technology and settle upon a time frame in which you can make a search engine.
Mon Nov 27, 2006 Reply New Discussion
making a search engine is a difficult and a verymuch financially expensive one ,if you want to make a real search engine like google hehe, the main thing is the crawlers who crawl billions of pages and the data they store in the servers ,big and pwerfull servers are needed to store and retrieve the data , making search engine for a particular site or forum is relatively easy ,just need to use the cron jobs to scan your site at regular interval for scanning all pages and make it available easily in the searches whenever someone search for it
Wed Feb 7, 2007 Reply New Discussion
Not so much about programming? And how exactly would a search engine exist without programming, it would fall flat on it's arse.
The reverse however, may not work as intended, but it will do something. So i'd be willing to bet that programming is far more important.
Creating some form of Search Engine, you don't need to bother reading the garbage that everyone else opinionates on.
You need to think about what you need to be searching and how, start off with something programmed basic and just improve and built it from there.
You certainly won't have the "World OS Final" running from the get go if you know what I mean
.
The reverse however, may not work as intended, but it will do something. So i'd be willing to bet that programming is far more important.
Creating some form of Search Engine, you don't need to bother reading the garbage that everyone else opinionates on.
You need to think about what you need to be searching and how, start off with something programmed basic and just improve and built it from there.
You certainly won't have the "World OS Final" running from the get go if you know what I mean
.
Sat Feb 10, 2007 Reply New Discussion
I think Stumbleupon is doing something interesting by using their stumblers to help rate their sites. People enjoy using the plug-in and community and at the same time are contributing to their search engine.
Sat Feb 10, 2007 Reply New Discussion
@foolakadugie, agreed. StumbleUpon definitely has an interesting system there. I like how often I get an interesting website everything I randomly browse around. It's better than typing in random words to Google and hoping for something interesting on my more boring days.
More on topic: Making a search engine is infinitely hard, especially if you want a successful search engine and not just something that's there for playing purposes. As simple as Google may look, that is only on the outside. If you really want something as successful as Google, it'll take a lot of work, just like CaptainRon has said.
On the other hand, you could also just make a small-scaled version of a search engine for fun so that you understand the basics of how it works. But if you were seriously dedicated, it's time to start setting up a schedule for yourself.
More on topic: Making a search engine is infinitely hard, especially if you want a successful search engine and not just something that's there for playing purposes. As simple as Google may look, that is only on the outside. If you really want something as successful as Google, it'll take a lot of work, just like CaptainRon has said.
On the other hand, you could also just make a small-scaled version of a search engine for fun so that you understand the basics of how it works. But if you were seriously dedicated, it's time to start setting up a schedule for yourself.
Mon Apr 9, 2007 Reply New Discussion
 |
 |
|
 |
 |
| Reply / Comment | Ask a Question? | Share / Bookmark | E-Mail a Friend |
 Netbux.org Get paid to search the net!!! (4)
|
(1) A9 
|