getting deep crawled

[eMEraLdwPn]

i just noticed only about 35,000 of my 200,000 or so subpages have been crawled / indexed by google, and i've been floating between a PR4 and 5 for several months now.. is there anything i can do to get it to crawl the rest of the pages? it would probably boost my PR back to 5 and up my traffic significantly if i got all the pages indexed

[AndyH]

It appears to me that the higher PR you have, the more pages Google will crawl.

[nohaber]

What site is it? How many levels deep are most of the pages? If you have 4,5 etc. level pages it should take a while for Google to index them.
If you change the linking structure to accomodate less clicks from the home page to get to all of the pages, that should help.

[eMEraLdwPn]

it's my lyrics site. most of the links it never grabbed are like 4-5 pages in... the format goes something like: home --> browse --> artist --> album --> song

it has all the home and browse pages (all PR4), i think it has all the artist pages (like 13,000)... then it has some of the album pages and some of the song pages

i can't really think of any easier way to get to the album / song pages quicker, even if i made a site map...

[nohaber]

i can't really think of any easier way to get to the album / song pages quicker, even if i made a site map... Yeah, I can see this.
Maybe it's a matter of waiting then. When Google indexes all of your pages, your PR should rise with such an amount of content.

[eMEraLdwPn]

yea, i probably just have to go back to waiting then... i guess it's better that i'm getting the traffic i am now with only 1/6 the site indexed rather than the entire thing, it's just more frustrating knowing i could be getting so much more ;-)

[joeychgo]

Ive been getting deep crawls from googl for weeks - jeteye also

about every other day

[Emancipator]

interesting. Google this and spiders me daily for my content, i use it to get my friends sites spidered since it is daily hits. Google however doesnt do a deep crawl, it only goes a few pages deep.

I am interested in hearing approaches for deep crawls.

[Blue Cat Buxton]

Would it help to throw some PR towards a subpage (sort of steer Google in the right direction?

[paulfitz]

Jeteye has been annoying me recently - the hits mess up my 'unique visitor' stats as there are various Jeteye subdomains coming in on different IP addresses.

Well hopefully it will pay off with search engine results.