Loading...
HOME
 |
Make Your Own Search Engine? - Is it possible to make your own engine? | ||
Discussion by infamousflame with 30 Replies.
Last Update: April 12, 2012, 3:40 pm | |||
 |
|||
 |
 |
|
 |
 |
| Reply / Comment | New Discussion / Topic | Share / Bookmark | E-Mail a Friend |
 |
|
|
I have never made a search engine though. Never had the need to. But I have some idea. It mostly depends on the search space. It less abouit programming and more about using the right tool. Searching Algorithms are no doubt one of the most studied algos out there with varying implementations..so is the case with search engines them selves. It would be wise to look into already developed search engines.
I need not mention sites like sourceforge.net tigris.com etc which host projects like these. If you find some thing interesting you may want to join them.
just curious , do you want to venture into search engine technology just for the sake of it or do u have an actual problem to solve (ie you are building a website of some sort)
Also search engines are taking new approaches these days , ie sound based , video , 3d etc etc and not just plain old text.
You may want to check http://en.wikipedia.org/wiki/Search_Engine out
QUOTE (infamousflame)
I was just curious about how easy it would be to make your own search engines , I don't think I have time to do it now though. It's probably all too technical for me. I don't know so much about programming, so I might not be able to do it. I know you said it isn't so much about programming, but it would still probably be too much of an extra workload for me, I've got a lot going on with my site and with my forum. Thanks for givng me such good info guys.
Creating a search engine needs a huge database i presume and a large brain to scour through all the algos of searching, sorting and implementing the best to suite your needs... I feel its about a effecient programming than about large programming. You need to have all the knowledge about internet and how it works. How you rate pages and index them for better searh...
i think you should start creating searchengine for a localhost now and make it a big boom on the internet if you succeed in implementing the better algos. Like google does...
Best of luck dude
GreetingZ
There's a lot of free search engine scripts out there ready to use for your site.
QUOTE (infamousflame)
Does anyone know how to make your own search engine? Does it take lots of time and careful programming or is it quite quick and easy to make a basic one? Post here!
Link: view Post: 40168
Best practice to learn some topic is to explore existing implementations. Start by reading sources.
I can recommend MnoGoSearch and AspSeek. First written in perl AFAIK, second C++ and MySQL.
Happy hacking!
1. Use free, easy to use, PHP scripts. You can find plenty here. Just find the free ones underneath the commercial ones and choose the one that is most suitable.
2. Use Google SiteSearch.
3. Investigate and research how to build your own PHP/ASP files to create your own search engine to index your site. You may also want a script for a sitemap.
The first two options are easiest. Good luck!
Now just imagine what searching for a topic would be like. Your first step will be to search by the title of the books. But its not surprising that many books have weird titles which may not convey what the book might be containing within.
So in this case having a index in each book would have helped a load. But still, since the library is unorganized and books are lying in a pile, you would have to search the books by picking them up one by one. In the worst case, the book you are finding might be at the last of the heap. In this case the algorithm time complexity is n, where n is the no. of books in the library.
Now what you do is sort the books in sections depending upon their core theme. A book on artificial neural networks shall actually be stashed in Computer Science section and not Neurology/Biology section.
Now further more, to make things even more fast, you introduce something called a frequently issued section, that contain only the books frequently issued. It happens that most of the books requested for issue come from the frequent section. Simply, now you end up making your algorithm faster.
Now remember that you categorized books in sections? Let's do one more thing. The books that appear the most in bibliography of other books reserve a seat for themselves in the top shelf, indicating that they are the most popular and reliable books. This, my dear friend is your introduction to page rank algorithm used by Google.
Why I gave this example is because the very first model described in para 1 can be made in 2 hours. The time required and programming skills needed subsequently increase.
Now lets talk about programming and requirements.
Hardware (you may also do with a single PC with a 160GB HDD):
> Loads of storage space (RAID)
> A well connected database
> Multiple Servers for load sharing
Programming:
> Web crawler (that fetches the web pages and puts them in the Data warehouse)
> Data organizer (that arranges the data so that it can be mined efficiently)
> Search Algorithm
You can read some good white papers regarding search engine technology and settle upon a time frame in which you can make a search engine.
The reverse however, may not work as intended, but it will do something. So i'd be willing to bet that programming is far more important.
Creating some form of Search Engine, you don't need to bother reading the garbage that everyone else opinionates on.
You need to think about what you need to be searching and how, start off with something programmed basic and just improve and built it from there.
You certainly won't have the "World OS Final" running from the get go if you know what I mean
.
More on topic: Making a search engine is infinitely hard, especially if you want a successful search engine and not just something that's there for playing purposes. As simple as Google may look, that is only on the outside. If you really want something as successful as Google, it'll take a lot of work, just like CaptainRon has said.
On the other hand, you could also just make a small-scaled version of a search engine for fun so that you understand the basics of how it works. But if you were seriously dedicated, it's time to start setting up a schedule for yourself.
Your going to need a big database engine and a fast server...
Alot of brains and you have to know other languages to make other sites like german ect...
Your gonna need alot of money to advertise it for a start.
It is quite possible to build a very complex and stable search engine without the need for ridiculous amounts of space and processing power (to begin with atleast).
You just got to make sure you can scale it well afterwards lol.
Zoom Search
Search Engine Builder
I've made it using Google "Custom Search Engine". It works pretty well. To this moment I've indexed near 2000 sites.
http://www.google.com/coop/cse/
Been debating Desktop search for some time at:
http://channel9.msdn.com/showuserthreads.aspx?userid=31672
It's the only program I really use.
It evolved to do all the search related features I could think of.
But, if you make a search engine which can index and serve results for specific topic for example "golf tour search engine" maybe it can still grab a share in the market. Making general search engine and getting benefit from that is out off the question.
If you are trying to create a search engine from scratch, you should begin creating your robo-spider first.
I have a problem I can't figure out how to connect to my database I post the script below can any one tell me whats wrong with like coding errors
mysql_connect('localhost','root','root');My_select_db('SearchEngine');$GET = mysql("Select * From keywords");While ($getrow = mysql_fetch_assoc($get)){ $keywords =$getrow; echo $keywords,'<br>'; }
-reply by Programer
My own search engine would really make my website look cleaner and I can use it in whatever way I want. Though the existing free services like google custom search engine are also very customizable but they still won't be able to compete with my own creation.
So I would love to hear from someone about how to make my own search engine for my website.
Another thing that was usable- yahoo search boss. I am not sure if it is working anymore. As yahoo is merged with bing, i guess the boss api may not be working anymore. Duckduckgo is the search engine that uses it.
The thing about search engine is that it is necessary to buid spam index and filters before bot wildely indexes everything. So it all depends on lots of coding, better hosting and diskspace and we're talking about datacenter here.
Google app engine which hosts many apps with search feature makes use of something custom. The syntax for accessing database is quite something different. It doesnt look like sql to me.
Couch db, nosql and some file system friendly search engines are in use. I tried python,yaml and app engine for connecting two dbs. I thought about making it look like search engine and it didnt worked for me.
|
|
|
|
|
|
| Reply / Comment | New Discussion / Topic | Share / Bookmark | E-Mail a Friend |
Similar Topics:
Search Engine Optimization
Yahoo Puts Other Search Engines To ...
Profusion.com The Original Meta Se...
 Netbux.org Get paid to search the net!!! (4)
|
(1) A9 
|
How Search Works
Best Search Engine?
What is a Search Engine
What Is Search Engine Optimization / SEO
SIMS 141 - Overview of How Search Engines Work
The Structured Search Engine
SIMS 141 - Search, Google, and Life: Sergey Brin - Google
What is Search Engine Marketing?
Search Engine (SEO) Basics
Search Engine Optimization SEO Tutorial -- WebBizIdeas
