Found no robots meta tag

Hi why cant bots find my sites , every tag checking site says this Found no robots meta tag

ive uploaded the txt file , why the problem

Do you know what the robots meta tag is and what it is used for?

Most all sites do not have robots meta tags, so that would be normal.

I am assuming the txt file your referring to is robots.txt. That is not the same thing as robots meta tags.

So do this. Remove the robots.txt file which most likely you did not create properly and instead of telling the robots what to crawl, it's most likely blocking them. Just remove it.

Then submit your site to the search engines. Hell you don't even need to do that. Just posting your sites url in a public forum like this one will point the search engines to your door.

If you do not know what a robots.txt file is, how to properly create one, how to use it, what it is for, what robots meta tags are, then it's best you do not play with any of them. You can do more harm then good.

Hi this is the command i use

User-agent: *
Disallow: 
Crawl-delay: 5
Sitemap: http://www.*******************/sitemap.xml

First of all, the bad bots will crawl your site even if you tell them not to.  Secondly, what do you want the robots.txt to do?  You can block the good bots that you don't want crawling your site using a robots.txt file; any bot that ignores the robot.txt file will crawl.  You can use the robots.txt file to keep the good bots from crawling certain portions of your site that you do not wish to be indexed.  If you want everything indexed and don't mind all the good bots crawling your site, then just remove the robots.txt file from your site.  If you want to block the bad bots, then robots.txt won't do it, you have to use other methods because they won't even bother to read your robots.txt file.

so its better just to remove it lol

 RE:

 No.... it's useful to to block the major search engines from indexing parts of your site  that have no business being crawled.  Bad bots can be blocked in .htaccess  See this page: http://perishablepress.com/2013-user-agent-blacklist/

Example robots.txt file is attached

What if your webserver does not use .htaccess?  I have set up whitelists; on Apache servers, allowing only the bots that I want to pass; however, bad bots sometimes claim to be good bots.  Still, better than nothing I suppose. Trying to fight the bad guys is a never ending battle.  I guess I need to read up on doing it with Nginx.  One has to worry about scrappers too as it could interfere with your Google ranking for your content to appear on multiple sites.

wish i had time to worry about such things as robots.txt .... but i still wouldn't

houstonlively  did you not add your sitemap url

Sitemap: http://www.*******************/sitemap.xml

 RE:

 You had time to read about other people worrying about it.... that's an even worse waste of time.

 RE:

 Then I guess you wouldn't be able to do it that way.  Let me know if there are any other dumb questions I can answer.,, I'm here for you.

Hate to sound like a broken record but if you want your site to rank quickly you really only need to worry about two things.

1. link building

2. content

Much of what you read on the web about seo is out dated ( unfortunately no one takes the time to go back and remove misinformation including me).

I can rank a new site (page 1) in 5 days for title...about a month for keywords. Without any meta tags at all .

1a. More on link building. Scares  peolple and sounds like a lot of work but it's easy. Invest a one time fee of 7 bucks at social monkey. The first day submit your dolphin url but spend the extra 4 bucks to submit 100 links after submitting to the 25 you get for 7 bucks. thats all you need to spend here .

    Then everyday go back and submit one url for free( one of your back links ). this begins to build your second level links. After 3 months you can resubmit your site url.

    As much as this helps ( and it does ). Each day take another minute or two to create one or two        "quality links". These can be as simple as posting a comment to high ranking blog. You will be amazed with your results after just a couple weeks.

2a. Content . Needing Good content is a bit of a lie. Should really be ORIGINAL CONTENT. Never copy content. The keywords you want to rank need to appear in the content of some of your pages even if they aren't assessable by your navigation but I would have to assume they will need to be eventually. Search engines are getting smarter all the time. DO NOT USE INVISIBLE CONTENT. If necessary to add some content containing keywords simple build a page that tells all about the features of your site.

When you do submit quick links do not add keywords that aren't in your content. when building your quality links a few a day are plenty. Don't build hundreds of links in one day, The search engines see this as spam. Trickle your links over a period of time as social monkey does.

10 minutes a day will have you jumping for joy in no time. ( these methods work for any site )

I guess I hit a nerve LOL.  You think the common answer is .htaccess but that is just Apache and Apache even with its latest updates is still an old plough horse when today a race horse is needed.  And many, including you, keep telling people to use blacklists even if the blacklist grows to be a huge arse file; Apache processes that file top down so a million entries to try and cover all the new and changing attacks won't work.

 RE:

 Hit a nerve? Don't give yourself too much credit.  Had you bothered to check the 2013 blacklist I referred to, you would have found out that it adds about 8KB to the size of .htaccess .... about the size of  single icon file.  Granted, you may be a bust person, and not have all millisecond for a page to load, but many people won't even notice the millisecond or two that 8KB adds to page load time.

Hi is this correct , having Disallow: / on its own blocks all robots from scanning your site. houstonlively in your txt file its repeated many times

User-agent: libWeb/clsHTTP
Disallow: /

User-agent: asterias
Disallow: /

User-agent: httplib
Disallow: /

User-agent: turingos
Disallow: /

User-agent: spanner
Disallow: /

 RE:

 Perhaps you should study this: http://www.robotstxt.org/robotstxt.html

Sorry i just seen it , makes perfect sense now. ;-)