Nelson's Weblog: tech / good / googleSitemaps

How well is Google indexing you?

Google's relationship with webmasters is weird. Google depends on webmasters for content, webmasters depend on Google for referrals, and yet the relationship is generally adversarial as Google defends itself from spammers. About a year ago Google reached out to webmasters with the Webmaster tools. It was good at launch and has improved a lot. If you care about how Google indexes your website, you should learn about it.

The statistics are the most interesting webmaster tool for me. They show you how Google is crawling your site, how user queries lead to your site, and what keywords Googlebot is seeing on your pages. For instance, I learned today that I get a lot of clicks for the query pee jokes. I'm #2 on Google for that highly valuable term, although I think #1 would be more appropriate.

The webmaster tools also give you diagnostics for when Googlebot has problems. The Crawl rate shows you how many requests Googlebot is making (3 megabytes a day!) and gives you the option to slow it down. The crawl errors are cool too, particularly for finding robots.txt problems.

The above tools show you what Googlebot is doing; sitemaps lets you help it along. A sitemap is a simple text document that lists all the URLs on your site. Great for dynamic sites with hard-to-crawl pages. This file is understood by all three major search engines, so it's well worth the effort to create one. It's like aliweb all over again!

tech • good
2007-01-25 18:00 Z


Mastodon @nelson@tech.lgbt Linkblog Sat 2024-12-21 License Plate Game Fri 2024-12-20 The Loves of my Life Musk + AfD Thu 2024-12-19 Another $2B Wed 2024-12-18 mail-tester.com Kenji and alcohol Amazon Time Sync hydroxychloroquine fraud withdrawn Tue 2024-12-17 A New Social Tacticalgramma Sun 2024-12-15 Drone nonsense Sat 2024-12-14 OK Nintendo 64 Fri 2024-12-13 Unix Windows LocalThunk interview Sora AI gymnastics Thu 2024-12-12 Queer interview Tue 2024-12-10 OldInsuranceMaps.net Sanborn Maps Harmeet Dhillon (CW: anti-trans) Uranian (sexuality) Search Archives 2023 12 11 10 09 08 07 06 05 04 03 02 01 2022 12 11 10 09 08 07 06 05 04 03 02 01 2021 12 11 10 09 08 07 06 05 04 03 02 01 2020 12 11 10 09 08 07 06 05 04 03 02 01 2019 12 11 10 09 08 07 06 05 04 03 02 01 2018 12 11 10 09 08 07 06 05 04 03 02 01 2017 12 11 10 09 08 07 06 05 04 03 02 01 2016 12 11 10 09 08 07 06 05 04 03 02 01 2015 12 11 10 09 08 07 06 05 04 03 02 01 2014 12 11 10 09 08 07 06 05 04 03 02 01 2013 12 11 10 09 08 07 06 05 04 03 02 01 2012 12 11 10 09 08 07 06 05 04 03 02 01 2011 12 11 10 09 08 07 06 05 04 03 02 01 2010 12 11 10 09 08 07 06 05 04 03 02 01 2009 12 11 10 09 08 07 06 05 04 03 02 01 2008 12 11 10 09 08 07 06 05 04 03 02 01 2007 12 11 10 09 08 07 06 05 04 03 02 01 2006 12 11 10 09 08 07 06 05 04 03 02 01 2005 12 11 10 09 08 07 06 05 04 03 02 01 2004 12 11 10 09 08 07 06 05 04 03 02 01 2003 12 11 10 09 08 07 06 05 04 03 02 01 2002 12 11 10 09 08 07 06 05 04 03 02 01 2001 12 11 10 09 08 07 One good site MDN Nelson Minar nelson@monkey.org Blog licensed under a Creative Commons License		How well is Google indexing you? Google's relationship with webmasters is weird. Google depends on webmasters for content, webmasters depend on Google for referrals, and yet the relationship is generally adversarial as Google defends itself from spammers. About a year ago Google reached out to webmasters with the Webmaster tools. It was good at launch and has improved a lot. If you care about how Google indexes your website, you should learn about it. The statistics are the most interesting webmaster tool for me. They show you how Google is crawling your site, how user queries lead to your site, and what keywords Googlebot is seeing on your pages. For instance, I learned today that I get a lot of clicks for the query pee jokes. I'm #2 on Google for that highly valuable term, although I think #1 would be more appropriate. The webmaster tools also give you diagnostics for when Googlebot has problems. The Crawl rate shows you how many requests Googlebot is making (3 megabytes a day!) and gives you the option to slow it down. The crawl errors are cool too, particularly for finding robots.txt problems. The above tools show you what Googlebot is doing; sitemaps lets you help it along. A sitemap is a simple text document that lists all the URLs on your site. Great for dynamic sites with hard-to-crawl pages. This file is understood by all three major search engines, so it's well worth the effort to create one. It's like aliweb all over again! tech • good 2007-01-25 18:00 Z Nelson's Weblog • tech • good