Those are called “sitelinks” and are automated but you can partially configure them in Google’s webmaster’s tools. In webmaster’s tools, click “sitelinks” in the navigation menu on the left. From the sitelinks page: Sitelinks are links to a site’s interior pages. Not all sites have sitelinks. Google generates these links automatically, but you can remove … Read more
According to the Latest Update: 01 Jan 2017 Any Sitemap file is limited to 50MB (uncompressed) with a maximum of 50,000 URLs. And a Sitemap index file (not to be confused with a Sitemap file) can include up to 50,000 Sitemaps. So, for a single Sitemap index file, the maximum capacity of URLs and storage … Read more
There is no standard, so there is no guarantee. With that said, its common for the sitemap to be self labeled and on the root, like this: example.com/sitemap.xml Case is sensitive on some servers, so keep that in mind. If its not there, look in the robots file on the root: example.com/robots.txt If you don’t … Read more
The Jeff Prosise version from MSDN magazine works pretty well, but it has a few flaws: AddNode freaks out with links to external sites on your menu (www.google.com, etc.) Here’s my fix in BuildSiteMap(): SiteMapNode node = GetSiteMapNodeFromReader(reader); string url = node.Url; if (url.Contains(“:”)) { string garbage = Guid.NewGuid().ToString(); // SiteMapNode needs unique URLs node.Url … Read more
Yes it is possible to have more than one sitemap-index-file: You can have more than one Sitemap index file. Highlight by me. Yes it is possible to list multiple sitemap-files within robots.txt, see as well in the sitemap.org site: You can specify more than one Sitemap file per robots.txt file. Sitemap: http://www.example.com/sitemap-host1.xml Sitemap: http://www.example.com/sitemap-host2.xml Highlight … Read more
Add this route towards the bottom of your config/routes.rb file (more specific routes should be listed above it): map.sitemap ‘/sitemap.xml’, :controller => ‘sitemap’ Create the SitemapController (app/controllers/sitemap_controller): class SitemapController < ApplicationController layout nil def index headers[‘Content-Type’] = ‘application/xml’ last_post = Post.last if stale?(:etag => last_post, :last_modified => last_post.updated_at.utc) respond_to do |format| format.xml { @posts = … Read more
On Stackoverflow (and all Stack Exchange sites), a sitemap.xml file is created which contains a link to every question posted on the system. When a new question is posted, they simply append another entry to the end of the sitemap file. It isn’t that resource intensive to add to the end of the file but … Read more
The difference between text/xml and application/xml is the default character encoding if the charset parameter is omitted: Text/xml and application/xml behave differently when the charset parameter is not explicitly specified. If the default charset (i.e., US-ASCII) for text/xml is inconvenient for some reason (e.g., bad web servers), application/xml provides an alternative (see “Optional parameters” of … Read more
According to the official documentation on sitemaps.org it needs to be a full URL: You can specify the location of the Sitemap using a robots.txt file. To do this, simply add the following line including the full URL to the sitemap: Sitemap: http://www.example.com/sitemap.xml