PDA

View Full Version : Search Database



Luckyrare
28-10-2006, 03:09 PM
Hey guys,

I am currently writing a search spider in php and I have got so far. Please tell me any of your websites and I will run the spider on your site in order for me to build up my website database. I have done about 4 websites so far and it has stored over 100 web pages urls from that.

Thanks
Danny <3

:Blob
28-10-2006, 03:10 PM
www.webshakes.com
www.vbalt.net
www.ry-net.info

sam iz da buz
28-10-2006, 03:11 PM
That's a good idea xD

www.mmoccforum.com
www.samsterteenz.com

Luckyrare
28-10-2006, 03:14 PM
Added them, thank you very much

Jackboy
28-10-2006, 03:14 PM
Ill give ya a list of sites i can think of? Wether they're mine or not

www.habbox.com (http://www.habbox.com)
www.habboxforum.com (http://www.habboxforum.com)
www.habbo-world.co.uk (http://www.habbo-world.co.uk)
www.google.co.uk (http://www.google.co.uk)
www.ask.com (http://www.ask.com)
www.yahoo.com (http://www.yahoo.com)
www.atomicoxide.co.uk (http://www.atomicoxide.co.uk)
www.atomic-network.co.uk (http://www.atomic-network.co.uk)
www.ao-hosting.com (http://www.ao-hosting.com)
www.atomicpanel.co.uk (http://www.atomicpanel.co.uk)
www.hotscripts.com (http://www.hotscripts.com)
www.habboxcanada.com (http://www.habboxcanada.com)
www.vbulletin.org (http://www.vbulletin.org)
www.phpbb.com (http://www.phpbb.com)
www.techtuts.com (http://www.techtuts.com)
www.simpsons.com (http://www.simpsons.com)
www.msn.com (http://www.msn.com)
www.microsoft.com (http://www.microsoft.com)
www.macromedia.com (http://www.macromedia.com)
www.adobe.com (http://www.adobe.com)
www.coffeecup.com (http://www.coffeecup.com)
www.brighton.com (http://www.brighton.com)
www.thefa.com (http://www.thefa.com)
www.jelsoft.com (http://www.jelsoft.com)
www.grisoft.com (http://www.grisoft.com)
www.habstract.com (http://www.habstract.com)
www.bluebobble.com (http://www.bluebobble.com)
www.thinkhabbo.com (http://www.thinkhabbo.com)
www.clubhabbo.net (http://www.clubhabbo.net)
www.habbohotel.co.uk (http://www.habbohotel.co.uk)
www.runescape.com (http://www.runescape.com)
www.runehq.com (http://www.runehq.com)
www.sulake.com (http://www.sulake.com)
www.php.net (http://www.php.net)
www.w3schools.com (http://www.w3schools.com)


theres a few if you need?

I might add more later

Tomm
28-10-2006, 03:16 PM
www.radiopanel.co.uk

:D

Colin-Roberts
28-10-2006, 03:19 PM
www.colin-roberts.net

Luckyrare
28-10-2006, 03:21 PM
@J-RAP - That pulled out 400 websites ;) Thank you very much

Thanks for the links guys. REP+

Also you may post other peoples websites, fav websites and big websites

Thanks

DuHast
28-10-2006, 03:23 PM
http://www.w3.org
http://www.sitepoint.com
http://www.devshed.com
http://www.htmlgoodies.com
http://www.neowin.net
http://www.liveside.net
http://www.digg.com
http://www.slashdot.org
http://www.gamespot.com
http://www.ign.com
http://www.ebay.co.uk
http://www.amazon.co.uk
http://www.amazon.com
http://www.google.co.uk
http://www.gamefaqs.com
http://www.yahoo.co.uk
http://www.msn.com
http://www.v7n.com
http://forums.digitalpoint.com

Jackboy
28-10-2006, 03:24 PM
ill post tons more if ya like

:Blob
28-10-2006, 03:24 PM
here goes. there is probs millions of pages for this. all arent mine:

www.a.com
www.b.com
www.c.com
www.d.com
www.e.com
www.f.com
www.g.com
www.h.com
www.i.com
www.j.com
www.k.com
www.l.com
www.m.com
www.n.com
www.o.com
www.p.com
www.q.com
www.r.com
www.s.com
www.t.com
www.u.com
www.v.com
www.w.com
www.x.com
www.y.com
www.z.com
www.a.net
www.b.net
www.c.net
www.d.net
www.e.net
www.f.net
www.g.net
www.h.net
www.i.net
www.j.net
www.k.net
www.l.net
www.m.net
www.n.net
www.o.net
www.p.net
www.q.net
www.r.net
www.s.net
www.t.net
www.u.net
www.v.net
www.w.net
www.x.net
www.y.net
www.z.net
www.a.org
www.b.org
www.c.org
www.d.org
www.e.org
www.f.org
www.g.org
www.h.org
www.i.org
www.j.org
www.k.org
www.l.org
www.m.org
www.n.org
www.o.org
www.p.org
www.q.org
www.r.org
www.s.org
www.t.org
www.u.org
www.v.org
www.w.org
www.x.org
www.y.org
www.z.org
www.a.co.uk
www.b.co.uk
www.c.co.uk
www.d.co.uk
www.e.co.uk
www.f.co.uk
www.g.co.uk
www.h.co.uk
www.i.co.uk
www.j.co.uk
www.k.co.uk
www.l.co.uk
www.m.co.uk
www.n.co.uk
www.o.co.uk
www.p.co.uk
www.q.co.uk
www.r.co.uk
www.s.co.uk
www.t.co.uk
www.u.co.uk
www.v.co.uk
www.w.co.uk
www.x.co.uk
www.y.co.uk
www.z.co.uk
www.a.info
www.b.info
www.c.info
www.d.info
www.e.info
www.f.info
www.g.info
www.h.info
www.i.info
www.j.info
www.k.info
www.l.info
www.m.info
www.n.info
www.o.info
www.p.info
www.q.info
www.r.info
www.s.info
www.t.info
www.u.info
www.v.info
www.w.info
www.x.info
www.y.info
www.z.info

There you go

Colin-Roberts
28-10-2006, 03:28 PM
www.pixel2life.com
www.techtuts.com
www.paramore.com
www.pcmech.com

Luckyrare
28-10-2006, 03:29 PM
Heh.

999 webpages. Ill do blobs now ;) J-RAP that would be awesome

Jackboy
28-10-2006, 03:35 PM
Heres some more:

www.limewire.com
www.habbo-works.com
www.habbodream.net
www.zonelabs.com
www.pimpmyride.com
www.computeractive.co.uk
www.micromart.co.uk
www.computer.com
www.dell.com
www.savastore.co.uk
www.maplin.co.uk
www.orange.co.uk
www.vodafone.co.uk
www.lg.co.uk
www.bt.com
www.ntl.com
www.sky.com
www.fish.com
www.voip.com
www.msgplus.com
www.mysql.net
www.cpanel.net
www.speedtouch.com
www.sofas.com
www.mozilla.com
www.photos.com
www.futurama.com
www.familyguy.com
www.nfl.com
www.carsthemovie.com
www.cars.com
www.pimpmyride.com
www.tetley.com
www.beer.com
www.germany.com
www.virgin.net
www.thomson.co.uk
www.hamiltonjet.co.uk
www.prismo.com
www.BASF.com
www.sega.com
www.sony.com
www.nintendo.com
www.xbox.com
www.gameplay.com
www.luckyg.co.uk
www.warez-over.net
www.shareaza.com
www.thomas.com
www.bbc.co.uk
www.channel4.com
www.channel5.com
www.123reg.co.uk

Ahh ma gaddzz.

Edit: rofl at blob. Most of em are Page cannot be displayed

Luckyrare
28-10-2006, 03:46 PM
Thanks!

1,306 webpages

More please <3

Jackboy
28-10-2006, 03:53 PM
www.school.com (http://www.school.com)
www.usa.com (http://www.usa.com)
www.lol.com (http://www.lol.com)
www.owned.net (http://www.owned.net)
www.promises.com (http://www.promises.com)
www.holylemon.com (http://www.holylemon.com)
www.petzic.info (http://www.petzic.info)

More will come later

sam iz da buz
28-10-2006, 03:56 PM
www.waushare.com
www.habbo-positive.co.uk
www.nitroncity.com
www.bbc.co.uk
www.habboforum.com
www.lolage.co.uk
www.rockersonly.com

Marsquake
28-10-2006, 03:57 PM
www.myfootballforum.com
www.base-community.com

DuHast
28-10-2006, 03:58 PM
www.waushare.com (http://www.waushare.com)
www.habbo-positive.co.uk (http://www.habbo-positive.co.uk)
www.nitroncity.com (http://www.nitroncity.com)
www.bbc.co.uk (http://www.bbc.co.uk)
www.habboforum.com (http://www.habboforum.com)
www.lolage.co.uk (http://www.lolage.co.uk)
www.rockersonly.com

Tsk Tsk you use Warez.... I hate that site they're **********

Edited by -legoman- (Super Moderator) Please do not avoid the filter.

RNelson
28-10-2006, 04:00 PM
EuropeHabbo.com , EuropeHabboForum.com , Connect-Hosting.com , Connect-Ads.com , Connect-News.com , Connect-Talk.com :D :eusa_danc

sam iz da buz
28-10-2006, 04:06 PM
Tsk Tsk you use Warez.... I hate that site they're wankstains.

Lol it's alright for getting codes etc, but it's always down which is a pain in the ***

And lol at the word above not being filtered when silly little words are filtered

-::Mr.Dave::-
28-10-2006, 04:14 PM
http://www.servers4me.com
http://www.vps4me.com
http://www.4menetwork.net
http://www.xtremehosts.co.uk
http://www.connect-hosting.com
http://www.runescape.com
http://www.habbo.com
http://www.habbo.com.au
http://www.buildyourown.org.uk/forums

Recursion
28-10-2006, 04:54 PM
www.skyehosting.net
www.wildfireclan.com (???)

Icarus
28-10-2006, 05:11 PM
http://www.habblio.com :D

Luckyrare
28-10-2006, 06:52 PM
I have logged 3,808 webpages

If anyone would like to tell me any popular community sites feel free, or any sites at all that would be great...

Thanks

lew!
28-10-2006, 07:00 PM
www.ravefm.net
www.habbodreams.net
www.a.com
www.b.com
www.c.com
www.d.com
www.e.com
www.f.com
www.g.com
www.h.com
www.i.com
www.j.com
www.k.com
www.l.com
www.z.com
ww.aa.com
www.bb.com

And the list goes on.... ;]

uber
28-10-2006, 07:01 PM
www.rockersonly.com
www.habmoo.net

Recursion
28-10-2006, 08:07 PM
Link to your search engine :D??

Luckyrare
28-10-2006, 08:23 PM
I have not made the search public yet, Once I have got pently of webpages (about 100,000) I will make it public.

Recursion
28-10-2006, 08:27 PM
:( can you please PM me :D so i can BETA it ;)

Luckyrare
28-10-2006, 08:32 PM
I will allow people to beta test in a few days just need to tweek.

Ryan.
28-10-2006, 08:36 PM
www.urbangfx.com

headset
28-10-2006, 08:36 PM
www.speed-networks.net
www.speed-networks.org
www.speed-networks.co.uk
www.nickcocchiarella.org
www.scammersmostwanted.com
www.google.co.uk
www.bbc.co.uk
www.alltheweb.com
www.ebay.co.uk
www.thinkhabboxforum.com
www.alexxturnerdesigns.com
www.ebayshoutcasts.com
www.gothell.net
www.habboxlive.eu
www.iron-horse-media.com
www.headshotradio.co.uk
www.jamiemiddleton.net
www.lazygeneration.co.uk
www.magnetic-dollies.com
www.nickcocchiarella.org
www.speco-******.com - seems this one is a be rude


My list

Recursion
28-10-2006, 08:42 PM
Nick, half of those are empty, and change your freakin' sig "/

Ryan.
28-10-2006, 08:44 PM
Can you add any?

My list added :p

www.chaosgfx.com
www.gameover.com
www.gamerenders.com
www.adminfusion.com
www.allgraphicdesign.com
www.just-expose.com
runescape.salmoneus.net/forums
www.extremepixels.com

Mainly recources and graphics help :p

Luckyrare
28-10-2006, 09:07 PM
Can you add any?

My list added :p

www.chaosgfx.com (http://www.chaosgfx.com)
www.gameover.com (http://www.gameover.com)
www.gamerenders.com (http://www.gamerenders.com)
www.adminfusion.com (http://www.adminfusion.com)
www.allgraphicdesign.com (http://www.allgraphicdesign.com)
www.just-expose.com (http://www.just-expose.com)
runescape.salmoneus.net/forums
www.extremepixels.com (http://www.extremepixels.com)

Mainly recources and graphics help :p

Thank you very much, that found me 1033 webpages (well more but if I kept searching it would go on forever) rep+


www.speed-networks.net (http://www.speed-networks.net)
www.speed-networks.org (http://www.speed-networks.org)
www.speed-networks.co.uk (http://www.speed-networks.co.uk)
www.nickcocchiarella.org (http://www.nickcocchiarella.org)
www.scammersmostwanted.com (http://www.scammersmostwanted.com)
www.google.co.uk (http://www.google.co.uk)
www.bbc.co.uk (http://www.bbc.co.uk)
www.alltheweb.com (http://www.alltheweb.com)
www.ebay.co.uk (http://www.ebay.co.uk)
www.thinkhabboxforum.com (http://www.thinkhabboxforum.com)
www.alexxturnerdesigns.com (http://www.alexxturnerdesigns.com)
www.ebayshoutcasts.com (http://www.ebayshoutcasts.com)
www.gothell.net (http://www.gothell.net)
www.habboxlive.eu (http://www.habboxlive.eu)
www.iron-horse-media.com (http://www.iron-horse-media.com)
www.headshotradio.co.uk (http://www.headshotradio.co.uk)
www.jamiemiddleton.net (http://www.jamiemiddleton.net)
www.lazygeneration.co.uk (http://www.lazygeneration.co.uk)
www.magnetic-dollies.com (http://www.magnetic-dollies.com)
www.nickcocchiarella.org (http://www.nickcocchiarella.org)
www.speco-******.com (http://www.speco-******.com) - seems this one is a be rude


My list

Thanks for them but quite a few of them where dead

rep+

Ryan.
28-10-2006, 09:40 PM
How did it find you 1033?

And what exactly are you doing coz i just added a few webpages lol...

Luckyrare
28-10-2006, 09:52 PM
What it does is it opens up that webpage and finds all the links of that site, it logs them down and then finds all the links on them sites, then it opens all the other pages and logs them down. This could go on forever (untill pages stop linking)

Ryan.
28-10-2006, 09:59 PM
:o... And you made it?
Was it hard? Coz i wanna learn php :p +rep for this great tool

JoeComins
28-10-2006, 10:16 PM
Primaldesigns.co.uk
Primalhosting.co.uk
Primalservers.co.uk
SQPublicists.com
Can you tell me how many from the 4 above?

And of cause . . .
Deviantart.com

Ryan.
28-10-2006, 10:25 PM
Lol deviantart.. How manys that got i wonder...

What about dafont.com
That would have loadsa links i'm suggesting lol.. if i find any more really popular ones i shall get back to you for now i'm empty :p

http://www.google.co.uk/search?hl=en&q=a&meta

Tell me how many ya get..

Also
http://www.bbc.co.uk/

Sygon.
28-10-2006, 10:32 PM
www.forumtrends.com
www.sitepoint.com
www.earnersforum.com
www.luvwii.com
www.habboxforum.com
www.habbox.com
www.habbohotel.co.uk
www.digitalpoint.com
http://forums.digitalpoint.com
www.adminfusion.com
www.arcademee.com
www.milliondollarhomepage.com
www.youtube.com
www.pagehole.com
www.microvisuals.com
www.dnforum.com
www.namepros.com
www.ebay.com
www.apple.com
www.dell.com
www.rockersonly.com
www.xboxlodge.com
www.uploadz.co.uk
www.****************

Thats enough for me so far

adm
28-10-2006, 10:34 PM
amazing how habbohotel came before anything else on that list.

Sygon.
28-10-2006, 10:34 PM
I was thinking what sites would have the most outbound links -.-'

Also i forgot heres a good idea, Directories :D And topsites

http://dir.yahoo.com/
www.dmoz.org
http://directory.v7n.com/
http://www.arcadetopsites.com/

Theres some just google for lists youll get millions of links if you set up a chain :)

Ryan.
28-10-2006, 10:37 PM
Nice list sygon i would +rep but i can't lol.. stupid limits ;(

Also is Danny offline coz not been replying in a while :(

Sygon.
28-10-2006, 10:41 PM
He doesnt live on here you know :D

Ryan.
28-10-2006, 10:51 PM
He's online though :p
I live on here.. Well today :p

Luckyrare
28-10-2006, 11:07 PM
Joe them 4 picked up 7 webpages. This is because you done most links like "/page.htm" rather than "http://server/page.htm" I will fix this soon, but I patched it up so I could start adding pages


Sygon great help thanks dude


As I said to joe some websites dont get pages on the database because of the way they have been writen on the coding and something I need to change to allow these.

Sygon.
28-10-2006, 11:15 PM
Linking to "/page.html" Is general bad for SEO and SE results though

ALso have younamed your crawler i look out for it in my stats :)

Luckyrare
28-10-2006, 11:53 PM
Well the hostname is s4.myhostcenter.com and the ip is 72.36.156.6

Also them websites you gave me Sy have found me 4,760 and not finished yet

Sygon.
29-10-2006, 12:32 AM
I feel special surely theirs more?

Also what does this crawler do?

Just collect outbound links or does it do a collect outbound links then crawl that outbound link for outbound links? and whats it for?

Luckyrare
29-10-2006, 12:57 AM
Currently it grabs all the weblinks on the website/page. I have been playing with the coding for a short while and am working on a system that goes on for 10 terms.

eg.

website.com would have 3 websites (website1.com, website2.com and website3.com), they would find the websites lets say they find 9 (website4.com, website5.com, website6.com, website7.com, website8.com, website9.com, website10.com, website11.com, website12.com) and so on...

What is this for...
well I am wanting to build a search engine so I have started collecting webpages.

additional
29-10-2006, 02:50 AM
R-ob.com
Robertoland.com
Qontrol.net
Ouruploads.com
Robmail.info
Cheeseandcrackers.info
Youtool.us

Ryan.
29-10-2006, 10:43 AM
And whats this spider gonna do for you/us??

uber
29-10-2006, 10:51 AM
And whats this spider gonna do for you/us??

Search engine...

Ryan.
29-10-2006, 10:52 AM
Oh.. What like google?

Thats a point why not run google on it?

Recursion
29-10-2006, 11:24 AM
rofl, google would give you most websites wouldnt it??

English
29-10-2006, 11:29 AM
Hey Danny - tis Craig
www.bbc.co.uk
www.bbc.co.uk/sport [subdomain onto it lol]
www.newsground.com
www.play.com
www.amazon.co.uk

Last 2 should reach loads;)

Ryan.
29-10-2006, 11:31 AM
rofl, google would give you most websites wouldnt it??
Thats what i'm thinking lol..

Danny search google :p

Luckyrare
29-10-2006, 11:37 AM
rofl, google would give you most websites wouldnt it??

m. nope it gives me 0

Some sub-sites of google give me more, google searches/google homepage give me nill

Matt,
29-10-2006, 12:40 PM
www.radioforteens.co.uk

headset
29-10-2006, 12:45 PM
you have a redirect scripd?

Ryan.
29-10-2006, 01:22 PM
m. nope it gives me 0

Some sub-sites of google give me more, google searches/google homepage give me nill
one work 'Shocking'..

Steven.
29-10-2006, 01:56 PM
www.vbulletin.com
www.zybez.net
www.tip.it
www.hab-world.com
www.thehabbos.com
www.teenhelp.org
www.netbusinesstalk.com
www.stevehoffman.tv/forums/ (http://www.stevehoffman.tv/forums/)
www.superherohype.com/forums/ (http://www.superherohype.com/forums/)
www.superhonda.com/forum/ (http://www.superhonda.com/forum/)
www.sybermoms.com/forums/ (http://www.sybermoms.com/forums/)
www.tampaforums.com/forums/ (http://www.tampaforums.com/forums/)
www.techimo.com/forum/ (http://www.techimo.com/forum/)
www.teenspot.com/boards/ (http://www.teenspot.com/boards/)
www.thedvdforums.com/forums/ (http://www.thedvdforums.com/forums/)
www.thefiringline.com/forums/ (http://www.thefiringline.com/forums/)
www.theologyweb.com/campus/ (http://www.theologyweb.com/campus/)
www.timebomb2000.com/vb/ (http://www.timebomb2000.com/vb/)
www.tivocommunity.com/tivo-vb/ (http://www.tivocommunity.com/tivo-vb/)
www.tournament.com/forums/ (http://www.tournament.com/forums/)
www.trackforum.com/forums/ (http://www.trackforum.com/forums/)
www.trackpads.com/forum/ (http://www.trackpads.com/forum/)
www.tranceaddict.com/forums/ (http://www.tranceaddict.com/forums/)
www.tribemagazine.com/board/ (http://www.tribemagazine.com/board/)
www.turkeyforum.com/satforum/ (http://www.turkeyforum.com/satforum/)
www.turkforum.net/ (http://www.turkforum.net/)
www.tverlebnis.com/forum/ (http://www.tverlebnis.com/forum/)
www.tvlinksonline.com/boards/ (http://www.tvlinksonline.com/boards/)
www.ultimatemetal.com/forum/ (http://www.ultimatemetal.com/forum/)
www.unrealforum.de/forum/ (http://www.unrealforum.de/forum/)
www.vbforums.com/ (http://www.vbforums.com/)
www.vbulletin.com/forum/ (http://www.vbulletin.com/forum/)
www.visordown.com/forums/ (http://www.visordown.com/forums/)
www.warriornation.net/Forum/ (http://www.warriornation.net/Forum/)
www.wass-up.com/forum/ (http://www.wass-up.com/forum/)
www.wetcanvas.com/forums/ (http://www.wetcanvas.com/forums/)
www.xtratime.org/forums/ (http://www.xtratime.org/forums/)
www.masalatalk.com/masalaboard (http://www.masalatalk.com/masalaboard)
www.xboard.us/bbb (http://www.xboard.us/bbb)
www.hayle.eu
www.hayletowncouncil.net
www.allotments-uk.com
www.gaiaonline.com
www.imdb.com
www.answers.com
www.imageshack.us
www.photobucket.com
www.allotments4all.co.uk
www.neopets.com
www.runescapecommunity.com
www.habbopaper.co.uk
www.habbomobile.com
www.teenmag.com
www.teenpeople.com
www.pcworld.co.uk
www.diy.com
www.myspace.com
www.bebo.com
www.htmlhelp.com
www.mozilla.com
www.mozilla.org
www.blogspot.com
www.blogger.com
www.thesun.co.uk
www.zymic.com

Uhm, thats all i can think of and i got to go. :P

Luckyrare
29-10-2006, 02:50 PM
Thank you very much ;)
I now have 21,308 webpages and I have made a spider that gets the webpages from the mysql, then searches them but it tends to crash if you try to do to many webpages (you can do about 100 webpages at once, so it can pull out a few 1000 per crawl)

Ryan.
29-10-2006, 03:20 PM
Hmm so tell me what exactly is this going to make...

Is it going to make a search engine like google?.. Or..? Coz i stil don't quite get what ya trying to make apart from it's a spider that leeches on peoples sites and grabs the sites from them and so thorth..

Luckyrare
29-10-2006, 03:33 PM
This has already been said, it will go to make a search engine. 30,908 Websites logged

Ryan.
29-10-2006, 03:45 PM
i shall get some more links :p..
I will edit this post when i got links..

Edit:

www.microsoft.com (http://www.microsoft.com)
www.piczo.com (http://www.piczo.com)
www.adobe.com (http://www.adobe.com)
www.learnflash.com (http://www.learnflash.com)
www.good-tutorials.com (http://www.good-tutorials.com)
www.gimp.org (http://www.gimp.org)
www.smashhits.net (http://www.smashhits.net)
www.mtv.com (http://www.mtv.com)
www.music.com (http://www.music.com)

Tell me how many ya get :p

dark-as-death
29-10-2006, 03:53 PM
http://www.time-for-time.com/default.htm
www.msn.com
www.hotmail.co.uk

Liberation
29-10-2006, 04:22 PM
www.rockersonly.com
www.sitepoint.com
www.bebo.com
www.vbulletin.com
www.phpbb.com
www.imageshack.us
www.tesco.co.uk
www.amazon.com
www.youtube.com
www.techtuts.com
www.adobe.com
www.sxc.hu
www.lipsum.com
www.paypal.com

Thats all for now from me. PLase say how many i got ;].
Will try and post more later.
Thanks

Ryan.
29-10-2006, 04:30 PM
:o.. You tube lol.. They must have alot?

Luckyrare
29-10-2006, 04:37 PM
Rhy - They had 537, I dont pick links up on youtube but it does pick up links to youtube/youtube videos

I cant count them anymore because they wont be correct due to cron jobs working at the same time as I work.

dark-as-death - Thanks ;) rep added

Liberation - Thanks for that list rep added They had 796 but I had to stop, the cron will do the rest of the work for me

Liberation
29-10-2006, 04:51 PM
Awsome. Thanks for the rep, Will Rep Back

dark-as-death
29-10-2006, 05:19 PM
1. rep returned
2. more links =]

E-buyer (http://www.habboxforum.com/www.ebuyer.com)
Random (http://www.habboxforum.com/www.random.org)
Wikipedia (http://www.habboxforum.com/www.wikipedia.org)
AA (http://www.habboxforum.com/www.theaa.com)< - the insurance company
Babyworld (http://www.habboxforum.com/www.babyworld.co.uk/) < - suggested by my mum lol

Please tell me how many these get =]

Luckyrare
29-10-2006, 06:30 PM
Sorry dude I cant tell you how many that added (it was about 100-150) due to the cron task

I now have 77,665

It runs every 5 minutes 5AM untill 12AM (Midnight) and during the early hours in the morning 1-4am it runs every 2 minutes. This is to try get most out of the server.

JamesRouale
29-10-2006, 06:34 PM
I suggest you add the Yellow Pages, (http://www.yell.com/).

headset
29-10-2006, 06:43 PM
http://www.speed-networks.net/forums/index.php

is my forum its quite populer

dark-as-death
29-10-2006, 06:53 PM
Oh oh oh, www.orange.com =]

Luckyrare
29-10-2006, 08:12 PM
Very happy to say, we have reached 102,553! 1mil here we come!

I have 2 cron jobs running. One of popular websites and the other is just one searches a random webpages (in batches of 100)

thanks guys

DuHast
29-10-2006, 08:14 PM
www.playboy.com (http://www.playboy.com)
http://www.hustler.com

They're both big sites..?

Luckyrare
29-10-2006, 08:29 PM
www.playboy.com (http://www.playboy.com)
http://www.hustler.com

They're both big sites..?

true, added

George Morgan
29-10-2006, 08:37 PM
www.sitepoint.com
www.techtuts.com
www.pixelresources.co.uk
www.primalhosting.co.uk
www.pweb.info
www.devshed.com
www.cnet.com
www.zymic.com
www.freelayouts.co.uk
www.zdnet.com
www.gmpte.com
www.mediacatch.com
www.aol.com
www.ntlworld.com
www.blueyonder.co.uk
www.bbc.co.uk
www.itv.com
www.xbox.com
www.sony.com
www.manutd.com
www.mcfc.com
www.habbari.com
www.altavista.com
www.waushare.com
www.rapidshare.com
www.imageshack.us
www.computing.info
www.ubuntuforums.org
www.arcticmonkeys.co.uk
www.thishabbo.com
www.php.net
www.wordpress.org
www.piczo.com
www.apple.com
www.dell.com
www.adobe.com
www.helpwithpcs.com
www.pcworld.co.uk
www.about.com
www.habboforum.com

Jackboy
29-10-2006, 08:39 PM
www.jack.com (http://www.jack.com)
www.jon.com (http://www.jon.com)
www.dave.com (http://www.dave.com)
www.milliondollarhomepage.com (http://www.milliondollarhomepage.com)
^^^^^^^ that'll get ya loads mate!
www.dave.com (http://www.dave.com)
www.billy.com (http://www.billy.com)
www.luke.com (http://www.luke.com)
www.glen.com (http://www.glen.com)
www.beryl.com (http://www.beryl.com)
www.karen.com (http://www.karen.com)
www.brian.com (http://www.brian.com)
www.frank.com (http://www.frank.com)
www.saltworkz.com (http://www.saltworkz.com)
www.myspace.com (http://www.myspace.com)
www.jimmy.com (http://www.jimmy.com)
www.bensons.com (http://www.bensons.com)
www.asda.com (http://www.asda.com)
www.tesco.com (http://www.tesco.com)
www.sainsburys.com (http://www.sainsburys.com)
www.morrisons.com (http://www.morrisons.com)
www.salisburys.com (http://www.salisburys.com)
www.bob.com (http://www.bob.com)
www.dave.com (http://www.dave.com)
www.jonathan.com (http://www.jonathan.com)
www.jonny.com (http://www.jonny.com)
www.smileyface.com (http://www.smileyface.com)
www.bbc.co.uk/cbbc (http://www.bbc.co.uk/cbbc)
www.jollyroger.com (http://www.jollyroger.com)
www.pirates.com (http://www.pirates.com)
www.slap.com (http://www.slap.com)
www.fishing.com (http://www.fishing.com)
www.hook.com (http://www.hook.com)
www.preview.com (http://www.preview.com)
www.mikerowe.com (http://www.mikerowe.com)
www.software.com (http://www.software.com)
www.download.com (http://www.download.com)

Recursion
29-10-2006, 09:07 PM
www.rockstargames.com
www.sony.com
www.dell.com
www.microsoft.com - GET A HELL OF ALOT OF PAGES!!
www.pepsi.com
www.cocacola.com
www.panasonic.com
www.clock.co.uk
www.thomson.com
www.sky.com
www.skydigital.com
www.door.com
www.laptop.com
www.templatemonster.com - Will get you quite alot of pages
www.4templates.com - will get you quite alot of pages
www.habboxforum.com
www.habbox.com
www.habboxforum.us
www.hiphabbo.co.uk
www.intel.com
www.nvidia.com
www.d-link.co.uk
www.celeron.com
www.fireplaces.com
www.thebible.com - :D random!!
www.nick.com
www.msn.com
www.live.com
www.zonelabs.com
www.hotmail.com
www.conversation.com
www.runescape.com
www.playstation.com
www.psp.com
www.nintendo.com
www.wii.com
www.luvwii.com
www.books.com
www.thecentermk.com
www.pokeme.com
www.onlytuesday.com
www.seats.com
www.sofa.com
www.imonmylaptop.co.uk
www.gays.co.uk
www.itv.com
www.bbc.co.uk
www.tiles.com
www.maplin.com
www.gameplay.co.uk
www.ben.com
www.feguson.com
www.glasses.com

london
29-10-2006, 09:27 PM
www.bbc.co.uk
www.msn.com
www.hellomoto.com
www.express.co.uk
www.boots.com
www.wikipedia.com
www.bored.com
www.lbc.co.uk
www.itunes.com
www.apple.com
www.upn.com
www.milliondollarhomepage.com
www.piczo.com
www.i-am-bored.com

dark-as-death
29-10-2006, 09:37 PM
Ok, don't blame me if these have been posted before, im just using the links in my favorates

http://www.justpaintball.co.uk/
http://www.fatbobspaintball.co.uk/
http://www.paintballmadness.co.uk
http://www.paintballtimes.com
http://www.thedollpalace.com/
http://www.channel4.com
http://www.bigbrotheronline.co.uk/
http://bigbrother.digitalspy.co.uk/
http://www.boomgames.com/
http://www.gotfuturama.com
http://www.fetchfido.com
http://www.free-games.com.au
http://www.funny-games.biz/main.html
http://www.hamster-heaven.com
http://www.midlandhamsterclub.co.uk/
http://www.petwebsite.com/
http://dictionary.reference.com/
http://www.bluecross.org.uk
http://www.raceforlife.org/
https://www.amazon.co.uk
http://www.freebies4everyone.co.uk/
http://www.goingplaces.co.uk/
http://www.sandals.co.uk/
https://www.bankcardservices.co.uk
https://www.halifax-online.co.uk
http://www.petstogoodhomes.com/default.asp
http://www.theallergysite.co.uk/
http://www..myspace.com
http://www.buildyourown.org.uk/
http://www.glittermaker.com/
http://www.underfivepounds.com/
http://www.roddas.co.uk
http://www.hotelsandguesthouses.net/
http://www.fishtec-coarse.co.uk
http://www.itsyourturn.com/
http://www.slicingguide.com
http://www.ebay.co.uk/

could probably get more, but i think thats enough for now =]

Rob
30-10-2006, 12:13 AM
http://www.zebramuzik.com
http://www.kerrang.com
http://www.rollersnakes.co.uk
http://www.speed-networks.net
http://www.skstudios.biz
http://www.santahome.co.uk
http://www.sonyericsson.com
http://www.myspace.com
http://www.vampirefreaks.com
http://www.lordi.org
http://www.oyah.co.uk
http://www.teenagecasketcompany.com
http://www.patchworkgrace.co.uk
http://www.censoredonline.com
http://www.piczo.com
http://www.nokia.com
http://www.gibson.com
http://www.web-systems.co.uk
http://www.sygon.net
http://www.luvwii.com
http://www.thybag.co.uk


Just a few I could think of.

Rob

craigg.
30-10-2006, 08:45 AM
http://www.webdeveloper.com/
http://www.webdevforums.com/
http://www.ozzu.com/
http://www.mediatemple.net/
http://www.2advanced.net/
http://www.blackicehosting.com/
http://billybussey.com/
http://www.bbstudios.com/
http://www.billybussey.com/forum/


Just a few, not alot I must say :)

sam iz da buz
30-10-2006, 12:34 PM
www.vbulletingames.com
www.bebo.com
www.naptunes.com
www.autotrader.co.uk
www.dictionary.com

Luckyrare
01-11-2006, 05:45 PM
Thanks guys, I will add them later. I got my account suspended for overloading with a cron job (every 5 minutes and it was running 7 times at once) I am sorting it now

Jackboy
03-11-2006, 05:04 PM
are u back?

Want to hide these adverts? Register an account for free!