Busting through the jargon web Part 3 - Crawling and Indexing
After a very busy summer working on websites I've at last found time to expand further on my jargon buster series looking at some common 'tech speak' terms that can often leave 'non-techies' feeling a bit bewildered and bamboozled.
Many of my customers with new websites that have just gone live ask me "Why can't I find my new website when I 'google' it?"
To help explain this, in this post I've chosen to focus on two terms which are important for everyone with a website, but particularly important to understand if you have a brand new website with a brand new domain name - Crawling and Indexing.
Naturally when you have a shiny new website the first thing you expect to be able to do when it goes 'live'/ 'launched' is find it by using the search bar on the internet. Search is now used as the main way to find websites and therefore most people try this straight away and feel alarmed when it doesn't pop up straight away. "Huh" they think, "I've paid all this money for a new website and I'm not showing up in search even when I type in the full domain name, There must be something wrong?"
Well I'd like to reassure you there is nothing wrong, it just takes time for search engines to realise that your website is there. Setting a website to 'live' is not the same thing as being able to find it in a search engine such as Google.
When your new website first goes live you will be able to find it by putting in the website address or URL such as https://uskvalleypromotions.com into your web browser’s address bar, but you can’t put in the name of your business, such as Usk Valley Promotions into a search engine or address bar and expect to find it the same day it is launched.
The address bar is located at the top of all web browsers and looks like this - it's where the full web address appears.
When you type in a Google Search box, you are searching the Google index, not the whole Internet.
Crawling and Indexing explained
Search engines send a bot or spider (web crawler) around the internet to collect information. Crawling is the process where the bot goes from website to website finding new information for a search engine. The spider must be able to crawl and read the information on your site.
Once your website is crawled, it has to be indexed. Indexing is the processing of the information gathered by the bot. Once web pages are processed, they are added to a search engine’s index if they are determined to be good quality content.
Rather than me waffle on about the mechanics of Search which will certainly make you glaze over, watch this video by Google's Matt Cutts which explains very well how search engines such as Google, Bing and Yahoo use 'crawlers/ spiders' and 'indexing' to find information on the web and return it back to you.
How long before my site gets indexed and starts appearing in Search?
Typically it takes around three weeks for a brand new website with a brand new domain name to be fully Indexed by Google and to begin appearing in Search. So when you have a new site patience is the key and for the first few weeks you will have to send out direct links to the site or type the full address in the address bar..
Busting through the jargon web series
This is part of a series of posts in which I will be sharing more basic explanations of these jargon loaded terms. Some of you reading this may know these terms already, some of you are highly likely to be more expert than me in understanding their application, but for those of you who ‘feel in the dark’ by search terminology, I hope this series will help you feel a little more empowered and less bamboozled! At the end of the series I will put everything into a downloadable PDF for you to access easily on my website.