1 Sitemap-Simply a sitemap gives the brief details of your website to the search engines. There are several types of sitemap available for the websites. A sitemap is an XML file that tells the search engine the complete details of your website page and url in descriptive way. Sitemap gives hints to the web crawler to crawl your website.
-XML sitemap
-RSS sitemap
2 Robots.TXT-With the help of robots.txt webmaster can easily allow or disallow the content of web page to the crawler.
Example Of robots.txt
#Robots.txt www.example.com/robots.txt
User-agent: *
Disallow:
# Don’t allow spambot to crawl any pages
User-agent: spambot
disallow: /
3 Meta Robots-An Example of Meta Robots
<head>
<title>The Best Webpage on the Internet</title>
<meta name="ROBOTS" content="NOINDEX, NOFOLLOW">
</head>
In above example “NOINDEX, NOFOLLOW” tells robots not to include the given webpage and links in their indexes.
4. Rel="Nofollow"
An Example of nofollow
<a href=”http://www.example.com” title=“Example” rel=”nofollow”>Example Link</a>
simply "nofollow" tells search engines not to follow the link, but still some search engines still follow them for discovering new pages.
5. Rel="canonical"
When two or more copies of the exact same content appear on your website under different URLs.
<head>
<title>The Best Webpage on the Internet</title>
<link rel="canonical" href="http://www.example.com">
</head>
-XML sitemap
-RSS sitemap
2 Robots.TXT-With the help of robots.txt webmaster can easily allow or disallow the content of web page to the crawler.
Example Of robots.txt
#Robots.txt www.example.com/robots.txt
User-agent: *
Disallow:
# Don’t allow spambot to crawl any pages
User-agent: spambot
disallow: /
3 Meta Robots-An Example of Meta Robots
<head>
<title>The Best Webpage on the Internet</title>
<meta name="ROBOTS" content="NOINDEX, NOFOLLOW">
</head>
In above example “NOINDEX, NOFOLLOW” tells robots not to include the given webpage and links in their indexes.
4. Rel="Nofollow"
An Example of nofollow
<a href=”http://www.example.com” title=“Example” rel=”nofollow”>Example Link</a>
simply "nofollow" tells search engines not to follow the link, but still some search engines still follow them for discovering new pages.
5. Rel="canonical"
When two or more copies of the exact same content appear on your website under different URLs.
<head>
<title>The Best Webpage on the Internet</title>
<link rel="canonical" href="http://www.example.com">
</head>
Popular search engine protocols
Reviewed by I
on
December 27, 2013
Rating:
Reviewed by I
on
December 27, 2013
Rating: