Subdomains, SEO and getting crawled

January 17th, 2007

Hi Everyone –

This blog was previously hosted on a subdomain – http://blog.utheguru.com. That subdomain was submitted using google webmaster tools, at the same time as the top level domain, www.utheguru.com .

The top level domain was crawled, cached and searchable within approximately two days, whereas the subdomain, four months later, is still waiting to be indexed, even though my logs tell me it is being crawled.

I’ve seen this same reticence from Google to index subdomains before on my other sites. As far as I can glean from my research, google doesn’t have a problem with subdomains – see here for another site that talks about it – but, really, in all honesty it’s really hard to find an OFFICIAL policy about subdomains on google.

I know in the past, there have been examples of people trying to manipulate PR by registering hundreds or millions of subdomains, and carefully interlinking the pages so as to impart an artificially high number of backlinks to the TLD. I’m pretty sure that google has cracked down on it now, as Matt Cutts talks about the problem of subdomain spamming in his online blog.

I’ve tried asking about subdomains, SEO and indexing on the Google Groups Webmaster Help Forum – in my first post, I asked the question:-

I’m considering starting a new site with a bit of a revolutionary
business model.

But let’s forget about that for now..

What I’m interested in is whether seperating a site into subdomains is
beneficial in terms of a site’s page rank and overall ability of users
to find information relevant and helpful to them.

There were a few interesting responses, but the general feeling I got was that I had entered a no-go zone and people were a bit reluctant to give any real answers. A few weeks later, I asked directly whether google indexes top level domains in preference to subdomains. Again, no real authoritative answers.

I’m thinking that this problem is worth asking about. At the end of the day, I have just 301 redirected blog.utheguru.com to www.utheguru.com, while I try and get answers.

I’d love to hear from you all about your experiences with subdomains.

Cheers,

D

---

Ok – so I think I’ve got the solution. I did eventually get ONE of my pages indexed, and that was by backlinking to that page a great deal (ironically, it’s this one) from other sites. Subsequently I got the poo’s with waiting for google, and transferred all pages over to the TLD (as you can see above). This ‘Solution’ has quickly paid dividends, as the site was indexed within a few days after the change over.So, my ideas –

1. Google is reticent and reluctant to quickly index subdomains.
2. If you want to get subdomains indexed quickly, you’d better have a strong backlinking program.
3. I think google has a ‘delay’ between crawling and indexing a sub-domain – I saw my subdomains regularly crawled, but indexing hadn’t occured up until two months after they were first crawled. The equivalent wait for a top level domain was only about a week.

If you want to use subdomains –

1. Initially add your new site to a subdirectory of the TLD of your existing website.
2. Get it indexed.
3. 301 redirect that directory to your subdomain.

Cheers and all the best!

Duck

Entry Filed under: SEO Discussions