Having a new website and not indexing it in the main search engines is like buying a new car and not fueling it. This is the same with websites if search engines do not index it. The idea is that you can have a checklist of the steps you need to take to add your website to the Google and other search engines’ index. You have to wait for the search engine’s robot (whether from Google, Yahoo or Bing) to go through your website to track its content and add it to its index.
Meaning of Tracking And Indexing
Tracking is the process by which the search bots go from site to site looking for new and updated information to inform the search engines about it. The bots travel through the links of the sites to gather new information.
Indexing is the process in which the information collected by the bots is analyzed and added (depending on its quality) to the search engine index relating to specific keywords. An indexed page is the one that appears when performing a search in the search engine for a specific term or by typing its URL.
Google bot is a search software that Google sends to collect information from websites through links. The Google bots read the contents on these sites and add what it finds back to the Google database.
Common Indexing or Search Engine Blocking Errors
Check if the page is indexed
The first thing you have to do is confirm if the page is in the indexes of the search engines. It can be indexed and yet give the impression that it is not. That’s why it’s good to be sure.
Check if you are blocking the search engine
If the page is not indexed, the reason may be that you are blocking the access and indexing. The blockade can be due to 2 reasons:
- A direct “disallow” in the robots.txt file: Access the robots.txt file at www.example.com/robots.txt. You should look for a “Disallow” directive that contains the path to the page that is not indexed. For example, if the path to access the page is www.example.com/category/page.html, there should be a “Disallow:/category/directive. This would mean that access to the page is denied to the search engine.
- A “noindex” parameter in the meta robots tag: To see the meta robots configuration, you have to access the code of the page in question by pressing CTRL + U in Firefox or Chrome browser. Examine the initial code section located between the labels, and if the “noindex” parameter appears inside of the tag, then the indexing has been blocked.
Factors That Favor the Indexability Of A Websites
The indexing factors are those that determine the ease or complexity by which the search bots find your site valuable to be indexed. A search bot invests a certain amount of time in tracking your website. To prevent these search bots from leaving your website, blog or online store without the information you want them to store, take note of the main indexing factors:
Friendly URLs
The easier the URLs are to read and understand, the better the bots can read them. It is highly recommended to include keywords in the URLs to enhance the relevance of the term within the page. Also, the user will also appreciate finding intuitive and well-constructed URLs that always indicate what to expect.
If you use WordPress, it is not necessary to be an expert or hire a WordPress consultancy to know how to activate friendly URLs. Just go in your administrator to “Settings> Permanent links” and select a link structure that is not the “Default” option. It is recommended that you choose the structure “Input name.” Once you save the changes, a .htaccess file will be created in the root of your WordPress installation, and all the URLs of your installation will change.
Breadcrumbs
The navigation through breadcrumbs is often overlooked in the design and development process of a web page. Some people may see it as unnecessary while others consider it too much trouble for the result to be achieved. You should know that breadcrumbs will significantly increase the usability of your website.
Breadcrumbs provide bots (and users) with an alternative method of navigation and allow you to see what the position is within the web structure and reduce the number of steps necessary to navigate to a higher level within a website. Here are some tips when using breadcrumbs on your site:
- Use the breadcrumbs at the top of the page: The most common and intuitive placement of breadcrumbs is at the top of the site. This allows users to find the links with ease and make use of them quickly.
- Use breadcrumbs consistently: If you decide to use breadcrumbs, you should make sure that you use them throughout the website.
- Style the breadcrumbs appropriately: You want your breadcrumbs to be visible but not the focal point. You’ll also want to make sure that your breadcrumbs are not part of the main content of the page but simply additional support. Find a middle ground where your breadcrumbs are not noticeable but can be seen when they are needed.
- Categorize the pages clearly: If you have pages that correspond to two or more categories, you may not want to use the breadcrumbs on your website. Trying to place a page in two or more categories will produce unclear breadcrumbs and a puzzle for the user. Make sure that your site has an organized hierarchy and that it is displayed correctly through the breadcrumbs.
- Separate each level correctly: Make sure that users can distinguish the difference between each level of breadcrumbs.
Web Domain
The domain is one of the most important SEO factors for Google. It is necessary to choose a domain as optimal as possible. Here are recommendations that will help index your domain quickly on search engines:
- Choose a domain that contains a keyword, and that is short and easy to remember.
- If you choose a domain that contains several terms, do not use strange characters and separate the hyphenated terms.
- The extension of the domain should be influenced by the users you want on the site.
