Need help in file search engine crawler, PHP Web Programming

Assignment Help:

Need help in File search engine crawler

Want a web crawler to gather data on an continuous basis from different file hosting services and filter it.

1) Data will be used to provide a files search engine to users. I don't want a clone of any similar kind website but infect something better than that one.

2) Process of data get-together will be a continuous process and will be running on the servers 24x7.

3) Data must be properly filtered to erase duplicate entries and new entries of the same data should merge.

I want the crawler itself part of the websites

It has to crawl multiple websites through given xml feed. Crawler must detect all links to files on rapidshare.com, 4shared, uploading as well as all other file sharing hosts. After detection it must add those links found to some kind of database (prefer MySQL) with addition information found about file Like Meta description, size, title, date if available.

Desired Skills are PHP5, cURL, MySQL Programming


Related Discussions:- Need help in file search engine crawler

Need help in jquery event calendar integration, Need help in JQuery Event C...

Need help in JQuery Event Calendar Integration w/Functionality, API Integration of BBB This module is about Integrating as well as Activating a JQuery Event Calendar as well as

Need help in magento developer for customization and updates, Need help in ...

Need help in Magento developer for customization and updates Our website is built on Community Edition plus uMarketplace (add-on) We will put you in touch with uMarketplace'

In search of experienced cake php coder to help in launch, In search of exp...

In search of experienced Cake PHP coder to help in launch of new site We are seeking someone to assist our small team of developers to handle minor code tweaks as well as error

Fashion content website that must operate under styles, WP - custom Theme ...

WP - custom Theme The website is a fashion content website that must operate under Styles categories (chic, casual, street etc) where each style will have content associated to

Abcdefg, aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

Performance issues on amazon ec2, I presently host my websites on Amazon EC...

I presently host my websites on Amazon EC2. I seem to be having several performance issues such as longer than expected loading times. My sites build with Wordpress. I use Cl

We are in need of front end web developer and designer, We are in need of F...

We are in need of Front End Web Developer and Designer In search of someone who can convert my PSD files to HTML, HTML5, PHP. Who understands as well as works on websites, la

Formatting tags, Now let us look at some more tags which can be used to for...

Now let us look at some more tags which can be used to format text. These are all given in the example illustrated in Figure  XYZ&l

Multilingual wordpress template design, Multilingual Wordpress Template Des...

Multilingual Wordpress Template Design + Wordpress site development Seeking professional Wordpress theme designer as well as developer to provide the following services- 1) O

Web developer with cms applications, Web Developer a) Intermediate to ad...

Web Developer a) Intermediate to advanced skill level with WordPress (Essential) and some experience working with leading CMS applications. b) Strong knowledge of Classic ASP

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd