!!

Welcome Guest

You are viewing this forum as a guest therefore You are Not Allowed to Post and Reply and You are not allowed to view links. Register or Login

collapse

* Search


* Recent Replies

What is a Feed and What is the Importance? by [Not Shown, Login to View]
[Jan 12, 2017]


May I Introduce myself by [Not Shown, Login to View]
[Jan 11, 2017]


how do i sort for jobs to place on a job website by [Not Shown, Login to View]
[Jan 09, 2017]


SEO for my wordpress news page by [Not Shown, Login to View]
[Jan 09, 2017]


powered by blogger by [Not Shown, Login to View]
[Jan 07, 2017]


How Often Should I Post New Articles? by [Not Shown, Login to View]
[Jan 04, 2017]


Suggested Structure of an Ideal Sales Page by [Not Shown, Login to View]
[Jan 01, 2017]


site review by [Not Shown, Login to View]
[Dec 31, 2016]


Blogger video bar configuration by [Not Shown, Login to View]
[Dec 16, 2016]


Picture permission from a website before using it. by [Not Shown, Login to View]
[Dec 11, 2016]


how to create news post without copying or editing content from another site by [Not Shown, Login to View]
[Dec 08, 2016]


Website Showcase - Drop Your Website Link(s) Here by [Not Shown, Login to View]
[Dec 06, 2016]


Expired domain parking and flipping by [Not Shown, Login to View]
[Nov 19, 2016]


adding a login page and registration page to a site by [Not Shown, Login to View]
[Oct 12, 2016]


Questions on website building. by [Not Shown, Login to View]
[Oct 08, 2016]


Question on links by [Not Shown, Login to View]
[Oct 08, 2016]


Implication of CBN Governor's statement by [Not Shown, Login to View]
[Oct 04, 2016]


Adsense did not accept my request by [Not Shown, Login to View]
[Aug 04, 2016]


What is Revenuehits by [Not Shown, Login to View]
[Aug 04, 2016]


Get Paid to Write an E-Book by [Not Shown, Login to View]
[Aug 02, 2016]


netcontacts real or scam by [Not Shown, Login to View]
[Jul 29, 2016]


How to Start Amazon affiliate marketing by [Not Shown, Login to View]
[Jul 29, 2016]


Can I use One website for Adsense, amazon and Information Marketing? by [Not Shown, Login to View]
[Jul 29, 2016]


Traffic generation for clickbank offer/product by [Not Shown, Login to View]
[Jul 22, 2016]


installing simple machine forum to once site by [Not Shown, Login to View]
[Jul 13, 2016]


Link not working by [Not Shown, Login to View]
[Jul 07, 2016]


My weblog wasn't accepted into adsense by [Not Shown, Login to View]
[Jul 05, 2016]


Guest Posting by [Not Shown, Login to View]
[Jun 28, 2016]


Social Networks Addresses - Drop Your Link by [Not Shown, Login to View]
[Jun 22, 2016]


65 Website Traffic & Link Building Techniques by [Not Shown, Login to View]
[Jun 22, 2016]

Author Topic: Crawler and indexing and robots.txt  (Read 607 times)

0 Members and 1 Guest are viewing this topic.

Offline [Not Shown, Login to View]

  • Level 1 (Infant)
  • *
  • Posts: 38
  • Respect: +4
  • Gender: Male
  • Location: Osun state (osogbo)
  • 1st Chamber
  • My Rank: 0
Crawler and indexing and robots.txt
« on: Mar 31, 2013 »
0
 Boss, what is the function of crawler and indexing? Also, how can i fill it in my blogger, bcos, i was warned not to play with it. So, to add the robots file in text format and select from 'custom robot header tags'. Thanks



Offline [Not Shown, Login to View]

  • Forum Guide
  • Level 3 (Regular)
  • ******
  • Posts: 337
  • Respect: +81
  • Gender: Female
  • Location: Port Harcourt
  • My Rank: 38
    • Chioma Andy's Blog
Re: Crawler and indexing
« Reply #1 on: Apr 01, 2013 »
0
if it is the one i know in Blogger, then i will advise that you do not temper with that area at all except you are well grounded in HTML.
Doing so will be putting your blog in danger.
for your daily scoop on Entertainment, News, and much more- You are not allowed to view links. Register or Login

Offline [Not Shown, Login to View]

  • Forum Guide
  • Level 3 (Regular)
  • ******
  • Posts: 397
  • Respect: +42
  • Gender: Male
  • Location: Port Harcourt, Nigeria
  • You Won't Shrink if you can think.
  • My Rank: 60
    • http://hotbusinesslink.com
Re: Crawler and indexing
« Reply #2 on: Apr 01, 2013 »
0
You are not allowed to view links. Register or Login
Boss, what is the function of crawler and indexing? Also, how can i fill it in my blogger, bcos, i was warned not to play with it. So, to add the rebots file in text format and select from 'custom rebot header tags'. Thanks



Addition to Chimummy have said, you don't have anything to do with the crawler and indexing in your  blog if you don't have any idea of HTML code.

or what exactly are you trying to do?
You are not allowed to view links. Register or Login

Offline [Not Shown, Login to View]

  • Forum Guide
  • Level 4 (Dedicated)
  • ******
  • Posts: 1055
  • Respect: +266
  • Gender: Male
  • Location: Ibadan, Oyo State of Nigeria
  • 1st Chamber
  • My Rank: 85
Re: Crawler and indexing
« Reply #3 on: Apr 03, 2013 »
0
You are not allowed to view links. Register or Login
Boss, what is the function of crawler and indexing? Also, how can i fill it in my blogger, bcos, i was warned not to play with it. So, to add the rebots file in text format and select from 'custom rebot header tags'. Thanks


@ Sunelpal3398 -  You should be guided accordingly.
Please don't  tamper with crawler and indexing if you are not
expert in HTML code.

Best regards,

Aare MacFally
Knowledge Is Power – Seek For It

Offline [Not Shown, Login to View]

  • Administrator
  • Level: Mentor
  • ******
  • Posts: 259520
  • Respect: +7989
  • Gender: Male
  • Location: Benin City
  • 18th Chamber
  • My Rank: 77
Re: Crawler and indexing
« Reply #4 on: Apr 12, 2013 »
0
Those settings and the robots.txt file are used to tell search engines what pages or links not to index. In most cases you do not need to change anything here so just as everyone else have suggested, do not change any of those settings.
Leave it for God, don't worry yourself. Leave it for God, don't cry no more.

 

Related Topics

  Subject / Started by Replies Last post
2 Replies
271 Views
Last post Feb 09, 2015
by [Not Shown, Login to View]
Google Indexing is Taking too long

Started by [Not Shown, Login to View] General Questions

3 Replies
427 Views
Last post Jan 26, 2016
by [Not Shown, Login to View]

TinyPortal © 2005-2011
SimplePortal 2.3.5 © 2008-2012, SimplePortal