Building google sitemaps

[Masetek]

What doyou use to build the xml google sitemap? I found a tool called site map builder (http://www.sitemapbuilder.net/products.aspx) that works but it ignores robots.txt. Hence, it adds all the forum stuff in the that I dont want indexed. This creates a huge sitemap (3mb+) and most of it is useless stuff like search.php, private.php etc from vb.

[stymiee]

I code it by hand. I use PHP to create the sitemap for the dynamic parts of my site and use apache forcetype to make it recognize the sitemap.xml file as a php file. I now only need to go and edit file whener I add a major page to my site. Otherwise all articles, blog posts, directory submissions, and forum posts are automatically added to it.

[ozgression]

My not be what you need, but there is a good sitemap generator plugin for wordpress that allows you to add/take away pages/folders that you don't want included.

I'll go and look for the name of it.

[KelliShaver]

I suually just write a script to generate it from the database content, but if that's not practical or if I'm feeling lazy, I'll just use this:

http://sitemap.xmlecho.org/

[agua]

I just found this Sitemap Generator and it listened to my robots.txt file :)

http://www.xml-sitemaps.com/

[Giles]

I only have a few sites and they are all wordpress so the WP sitemap builder plugin is my saviour, it automatically updates itself too.

[Emancipator]

cool aqua I am going to check that out and see what I think.

[dc dalton]

I have and LOVE gSite Crawler ... really awesome tool!

Check it out here:

http://johannesmueller.com/gs/

[Johnny Gulag]

I only have a few sites and they are all wordpress so the WP sitemap builder plugin is my saviour, it automatically updates itself too. Hey thanks for that. I did not know that existed. Now i do. You rock! :)

[jeremycurtis]

I've use www.xml-sitemaps.com on several sites. It offers an online free service for sites with <500 pages. For sites of more than this you need to buy it and install it on your domain - not expensive though. I've used the bought version on www.childrenwebmag.com quite successfully for several months.

The only problem with the free online version is that it will parse multiple versions of the same page with many strings (e.g. page?sort=asc and page?sort=desc etc.) which leads to more than 500 pages even on a small site. If it still is less than 500 pages then a quick regex is needed to remove all the duplicate pages.

The paid for version lets you filter this out automtically.

[Masetek]

Just some follow up...

I tested loads of free sitemap builders and this one is the best one I can find that builds a clean sitemap

http://enarion.net/google/phpsitemapng/

You download it and install on your server. Theres loads of config options. A couple of test runs and I had a clean sitemap.

[jeremycurtis]

Thanks Masetek,
I've been looking for an Open Source Google XML sitemap generator for a while and never found this one. I've just installed it and it is as expected a breeze.

It would nice to have the ability to setup a "cron" style job that reran it every day/week/month. But as it is only a click or two away its OK.

You also have to deal with security yourself with .htaccess files.

[Masetek]

Thanks Masetek,
I've been looking for an Open Source Google XML sitemap generator for a while and never found this one. I've just installed it and it is as expected a breeze.

It would nice to have the ability to setup a "cron" style job that reran it every day/week/month. But as it is only a click or two away its OK.

You also have to deal with security yourself with .htaccess files.

Have u had a look at the source? I'm sure it wouldnt be hard to modify so it can run on the cron

[Giles]

There is also the google official sitemap builder, that runs on perl, it is available from google.