In Google search, DNS errors, network errors, and HTTP status codes play a significant role. The article will consider the most significant network errors, DNS, as well as the main 20 status codes. These issues below can be seen in the Google Search Console report.
The presence of a URL has a negative impact on Google search. Timeouts are handled by a robot called Googlebot. It also searches for possible DNS errors, analogues of 5xx. When a network error occurs, scanning slows down rapidly because the server cannot quickly cope with the received loads. Indexed URLs are removed from the Google index for several days. For all found errors, Search Console generates its own warning.
How do you debug network errors?
Network errors are fixed before Google starts crawling URLs. There may be situations where the status code is missing and cannot report a problem. This is because the server does not have time to respond to these errors, as a result, diagnostics become difficult. You can debug timeout problems in the following ways:
- View firewall log. You may have set a very wide set of blocking rules.
- Analyze network traffic. To do this, you can use tools such as Wireshark or tcpdump. They allow you to capture and diagnose TCP packets, look for possible anomalies that will indicate a server module or network component.
- If you can’t find suspicious signs, then you should contact the hosting support.
It should be understood that DNS errors mostly occur through incorrect configuration. To debug it, you need to look at the DNS records. First of all, you need to make sure that the CNAME and A records point to the correct IP address.
It is also worth making sure that the name servers point to the correct IP addresses of the site.
HTTP status codes
They are generated directly by the server when it forms a response to a client request. Each of the codes has its own meaning, but sometimes the results of queries can be the same. A certain group of status codes signals a redirect, but the results are identical.
GSC allows you to generate a 4xx–5xx message, and for unsuccessful redirects – 3xx. If you received a response from the server “2xx”, then the content can be used for indexing.
Status codes 2xx (success)
If, when looking at Google, it hits a blank page, an error notification appears, then Search Console will write “404”.
200 “success” – content can be submitted for indexing;
201 “created” – no guarantee in content indexing;
202 “accepted” – Googlebot waits for content for some time, and then passes the received information for indexing;
204 “no content” – Googlebot received the content and reported it to the indexing pipeline, the 404 error may not be correct.
3xx status codes (redirects)
301 “moved permanently” – The indexing pipeline treats the redirect as a “strong signal” and the target becomes canonical.
302 “found”, 303 “see other” – a redirect followed by Googlebot is perceived as a “weak signal”;
304 “not modified” – the indexing pipeline receives a message from Googlebot that the content has not changed since the previous crawl;
307 “temporary redirect” – similar to 302;
308 “moved permanently” – similar to 301.
4xx status codes (client errors)
400 “bad request” – Googlebot sends information to the indexing pipeline that there is no content (exception – 429);
401 “unauthorized” – remove the URL from the index if it has already been indexed;
403 “forbidden” – smooth decrease in scanning frequency;
404 “not found”, 410 “gone”, 411 “length required” – do not use status codes “401, 403” to limit the scanning speed;
429 “too many requests” – the server is overloaded.
Status codes 5xx (server errors)
500 “internal server error”, 502 “bad gateway”, 503 “service unavailable” – page crawl speed is reduced by Googlebotom in proportion to the number of some URLs. The Google pipeline can remove those URLs that produce errors.