How To Find EVERYTHING on the Internet

How To Find Everything On The Internet

Google is so simple… IF you want what everyone else wants. The world’s favourite search engine has made its fortune second-guessing every phrase you type into it. No matter what you actually want, Google will use its giant, virtual brain to make a judgement on what most people would want, and then give you that content. By default, Google doesn’t simply search for what you type. It searches for what it thinks you mean.

This is actually a huge problem, because whilst for very typical searches, most of the time Google will be right; for more unusual searches, it’s likely to be wrong a high proportion of the time. And not just wrong – very wrong.

In its default state, Google is driven at a sort of tabloidesque mass market where everyone’s looking for components of populist culture. Many users searching for more obscure info give up when Google doesn’t give them what they’re looking for, assuming the information is not on the Internet. But in reality, almost everything is on the Internet, somewhere.

HOW GOOGLE DECIDES WHAT’S IMPORTANT

Google ranks Web pages not only according to their own popularity, but also on their context, and on whether they match up with the most popular meaning of the keywords you’ve searched for.

For example, search for the keyword “Dating” on Google. Provided the search engine hasn’t previously associated you with any other type of dating, you’ll see a whole front page focusing purely on matchmaking services and love interest. In fact, I got almost to the bottom of Results Page 3 before I found anything to do with historical dating – and that was only one result, with things immediately returning to another solid list of matchmaking links until Page 6. So if you type “Dating” into Google, there’s no question about what the search engine automatically thinks you mean. It thinks you want to get hitched.

A lot of the Internet’s less widely searched information resides on low-ranking websites. Sites which may be inactive, and may not have been updated for a decade or more. Sites in which there was never what Google would interpret as widespread interest in the first place. Hardly anyone ever linked to the sites, and the pages only received very modest numbers of visits, even when first published.

So Google sees these sites as unpopular, and ranks them as very low priority. Often, indeed, these low-ranking sites have been left behind by progress too. Whereas current sites typically incorporate all the latest optimisation tricks to help them impress the search engines in technical terms, inactive sites might still be sitting there optimised for the Internet of the 1990s.

In situations where the context of a search is clear and singular (in other words, there’s only one thing you can possibly mean), you do still have a good chance of easily finding a unique result on a low-ranking site. The problems with ranking status begin when obscure searches are overwhelmed with information from unrelated popular topics, which just happen to share their keywords.

TOUGH SEARCH

For example, there was a UK band in the late 1970s, called King. They recorded for John Peel’s BBC radio show and the session is still in existence, very occasionally getting airplay. However, they didn’t release any records, and the project was exceptionally short-lived. Thus, finding online references to the band online could prove a challenge…

Search simply for King on Google, and of course you have no chance. You might think that searching for King band would get you there, but because there was another, much longer-lived, more popular and more famous UK band called King in the 1980s, you get them instead.

In fact, even if you type King band 1970s, you still don’t get any relevant results. King Crimson, yes, Jonathan King, yes – even Focus and and Bad Company, all on Page 1… But just King? Nope.

In fact, Wikipedia dominates the front page, even though some results don’t even look related to the search. Why? Because Google knows Wikipedia is extraordinarily popular, and the world’s favourite search engine loves showing you results from extraordinarily popular sites. If it’s a choice between Wikipedia with a vaguely or barely relevant match, and a little niche site with a precise match, Wikipedia is almost certain to win.

It’s understandable that people conclude, in these situations, that the information simply isn’t on the Web. But in almost all cases they’ll be wrong. There are in fact lots of references to the 1970s band King online. But how do you find them? Indeed, how do you find ANYTHING you’re really after when Google appears not to have it?…

EXPAND THE SEARCH TERM

One of the obvious ways is to expand the search term to be even more specific, and include more keywords. Anything you can think of associated with your sought after topic, whack it into the search box. It doesn’t have to make sense or be a recognisable phrase. Google is a machine. Just stuff it with words associated with what you want. Ideally, in relation to my own search, I could include the name of someone who was in the band. But I may not know who was in the band, in which case I’d still be stumped. So other than stuffing Google with as many related words as possible, what other options are there?

USE QUOTES

Some searches are based on exact phrases, where the words must appear as typed, in a specific order. By default, Google doesn’t search for phrases as typed. It interprets what it thinks you’re most likely to mean with a collection of words, and searches for the most popular match in relation to that. As we’ve seen, in some cases this gives completely irrelevant results. But you can force Google to search for exactly what you typed by enclosing your phrase in quotes. So instead of typing Here is my phrase, you’d type “Here is my phrase”. Google will then only return a result if it literally includes the exact, ordered sequence of words: Here is my phrase.

This is particularly useful when you’re searching for people’s names. For example, let’s say you’re trying to track down someone with a name like Melanie Spice. Type that into Google as is, and the search engine thinks you’re looking for one of the two singers called Melanie, in the world-famous band The Spice Girls. Certainly here in the UK, you won’t find anything relating to an actual person called Melanie Spice in the top few results pages. Melanie Spice gets completely obliterated by Melanie C and Melanie B. Neither Mel B nor Mel C is called Melanie Spice, so Google is delivering completely irrelevant results to the search you’ve entered.

BUT, even though the results are technically wrong, they will, in the majority of practical situations, actually be right. A lot more people entering the words Melanie and Spice will be after Spice Girls info, than searching for an actual person who’s literally called Melanie Spice. Google is providing the wrong result, but people want the wrong result, and in this case, two wrongs do make a right.

If, however, you instead type “Melanie Spice”, using quotes, you should start to see actual people by the name of Melanie Spice. There’s still a huge dominance of Spice Girls results, but now you do at least have some alternatives, which are likely to be in keeping with what you typed, as opposed to what Google assumes you probably mean.

Always try the careful use of quotes when your search is swamped with irrelevant results. And incidentally, the more words in your quote-surrounded phrase, the easier it will be to find. So with names, if you have a middle name or initial, use it, and keep everything in quotes. “Melanie J Spice”, for instance, will cut out ALL of the irrelevant Spice Girls stuff, and take Google straight to that specific person. A person you’d never realistically have found simply by typing Melanie Spice.

DATE RANGES AND GEOGRAPHICAL RESTRICTIONS

But when you don’t have names, and you can’t really unearth much using quotes, what else can you do? Well, you can employ Google’s Search Tools. Two particularly helpful tools Google offers are the ability to confine your results to your own country, and the ability to set a precise date range.

These options, accessed via the Search tools button at the top right of your initial results, can once again cut out huge volumes of irrelevant data.

Confining your search to your own country is pretty self explanatory – it cuts out stuff posted around the rest of the world, so you’re much more likely to get locally relevant articles.

But setting a date range is more complicated. What it does, is excludes anything which was actually posted outside of the period you set. So, if you’re looking for info on something that happened in 1998, set a date range of January 1998 to December 1998, and you’ll cut out a vast, vast, vast amount of stuff that’s been posted since. Click Search tools > Any time > Custom Range… and then set the actual dates, as shown in the illustration below…

Search Tools Date Range

You shouldn’t even get sites such as Wikipedia, Twitter, YouTube or Facebook showing up, because your date range predates their birth. It should, however, be stressed that some Web users will backdate posts for a variety of reasons, and that Google can be fooled by this. But the bulk of what you find should be original matter from the period in question, and you’ll have an almost infinitely better chance of quickly hitting your target.

RETURNING TO THE PLOT

So what about our search for the short-lived 1970s band, King? Original published matter relating to that subject predates the Internet itself, and the only online references will have been posted retrospectively. Can Google’s date range still help? It’s unlikely, but Web publishers will usually refer to dates, or at least years, in historical posts. So a good alternative would be to confine your search to your own country if necessary, then actually type a relevant year, in quotes, into the search box, along with your other search keywords.

My breakthrough in finding material about the band King, came when I combined a number of the above methods. I knew the group had done a John Peel session, so I typed “John Peel”, in quotes, along with the band name, and the year of the session, which was 1978. But note the use of quotes…

“King” “John Peel” “1978”

There are three sets of quotes – one for each separate piece of information. You wouldn’t surround the whole thing with one set of quotes unless you expect the entire phrase to appear verbatim within the posts you’re looking for. If words are going to be a separate entity in the article, they need their own, separate set of quotes. The above search term has finally netted me a result. You can see the relevant text highlighted in yellow below…

King 1970s Band

With such an obscure subject, the poster of that top result deserves a link. Anyone posting obscure, hard to find information should be supported. One of the reasons their posts are so hard to find is because people don’t link to them, and support them…

King – The Late 1970s Band

PROGRESSIVE SEARCHING

One of the best things about searching for information is that once you find one post, it’ll very often provide you with enough keywords to do a new, better search, and find much more. That was the case with the post I linked to above, which, importantly, gave me the names of all the band members. So I can now search for “King” with all the different band members’ names, once again, in quotes where necessary.

This is ‘progressive searching’ – gradually getting more and more key info, and each time using the new info to unearth more data still. Eventually I was able to find plenty of pages relating to the band. It’s all there, when you know where, and how, to look.

IMAGE SEARCH

Google Images

Google Image Search can be a good alternative to the standard Web Search. Pictures tend to jump out at you more than text results, and Google loads a lot more images onto a page than it does text links, so it’s a faster way of searching. The drawback is that in many cases an obscure article won’t contain a picture, or the picture may not have been indexed by Google. So Google Images generally affords a much narrower range of access than the text pages. That’s the trade-off.

SOCIAL MEDIA

Don’t rule out social media as a good source of information. On sites like Twitter, you may not find much info in a single, 140 character Tweet, but you may get a link, which could lead you to a wealth of detail, and you may find numerous Tweets by multiple users, which assemble into a sort of overview.

Twitter is particularly useful with absolutely up-to-the-minute issues. Has your Tumblr stopped working five minutes ago?… Google won’t have indexed any info on that in such a short time. But Twitter’s search works in realtime, so chances are that typing Tumblr down into the Twitter search box will tell you if others are experiencing the same problem.

For anything that’s happened within the past few hours I’d try social media before Google, although Google does have a Past hour option within its Search Tools date range selector.

Social media is also good with info that people try to manage or ‘sweep under the carpet’ on the search engines. Customer complaints would be a good example. It can be very difficult to work out if a business is a scam on Google, because companies with endemic problems will often use powerful SEO techniques to bury complaints beneath their own propaganda. In extreme cases they’ll even write posts keyworded with terms like “scam” and “ripoff”, but which end up actually recommending the company. MLM schemes often use that technique. The result is that you don’t find the REAL complaints.

But on social media there’s no such thing as SEO, and all the search results are chronological. So if people constantly complain about a business, you can find those complaints, regardless of how much the company itself spams the site with ‘false positives’. On Twitter, for instance, you can simply type…

to:@UsernameOfCompany (as in the username of the company or business)

…into the search box and read the results. This shows the business or user’s live, incoming public messages, so you’ll soon see if there’s a scam going on, or if there are other problems.

FORUMS AND MESSAGE BOARDS

Posting questions on specialist forums and message boards can sometimes work when you’re really struggling to find information, but these days you need to be very picky, and some forums are a complete waste of time. With almost all forums, information is provided by random, unpaid users who may be very knowledgeable, but equally, may know nothing. What’s worst about forums is that people will often try to answer questions they know absolutely nothing about, just for the sake of responding. Commonly, replies to questions will include phrases like: “I suspect”, “I think”, “probably”, and “I’m pretty sure”. It’s very unreliable, for the reasons outlined in my Help Forum Helpers post.

ANSWERS SITES

This is a broad group of sites that purports to specialise in answering specific questions. They range in nature from Web-search-based utilities like Ask.com, to user generated content (UGC) sites like Yahoo Answers. Some of these sites have rating systems which put pressure on members not just to answer, but to provide a good answer. The answers can, however, still be very hit and miss, because as with regular forums, people are often replying for reasons and motivations other than an enthusiasm for, and specialist knowledge of, the specific topic or sub-topic concerned.

In common with forums, these sites attract members whose goal is to answer as many questions as possible, and to do that the members will frequently get their answers from Google. So requesting info from these sites can effectively just be inserting a middleman between you and Google. You’re probably better off doing your own Google searches.

One answers site I have found to be impressive is Stack Exchange. It specialises in computer and Web programming, so it won’t answer general enquiries, but it does have a very effective system of motivation, and the quality of the best answers is typically very high indeed. So there are some effective resources out there – beyond the mighty Google.

OTHER METHODS

Google hasn’t become so huge without reason. If you know how to use it, it’s probably the best shot you have of finding most things on the Web.

However, it won’t always come up trumps. The European Right to be Forgotten legislation means that in certain cases, information can be removed on request from Google’s search index – by affected parties. The info is still online, but Web searches won’t locate it. Similarly, some publishers will deliberately block Google from crawling and displaying their content, for various reasons. I’ve got content online which I’ve deliberately kept out of the Google results. No major search engine will find it, but there are other ways it can be found – not least via links and search facilities on my other sites.

One option as regards this type of hidden content is to use site search functions rather than dedicated search engines. Particularly on big forums this can yield excellent results. Use Google to find the specialist sites and forums, then leave Google, go onto the specialist site, and search there. So many sites have their own search functions – often extremely thorough routines with advanced options. But users will most often go straight back to Google when they don’t find what they want in the first page they hit. It’s just a habit, albeit in many cases a bad one.

Another of my blogs has a really efficient site search function, and I promote it on every page. Most visitors, however, don’t use it, and would seemingly rather plod around the site paging through piles of stuff they don’t want, on the off chance of finding what they do. Either that, or they bounce straight back to Google.

But going to the other extreme, search facilities on social blogging platforms can be grim. A lot of them search only for tags, so if publishers don’t include your specific keywords when tagging a post, you won’t find anything. The likes of Tumblr and WordPress are not really pitched at Web-surfers anyway. They’re more about creating an ‘indigenous’ community which consumes its own product in an almighty series of you-read-mine-I’ll-read-yours deals.

CONCLUSION

The key to finding the most relevant content is in understanding all options available in the search routines – wherever you go on the Web. If you’re just typing words into a box and hitting enter, you’re probably missing more than you can imagine. Take time to explore the search functions themselves. Click their Advanced tabs and see what sort of extra tools you have available.

Ultimately, it amazes me that people would rather spend nine hours on Twitter asking vague acquaintances if they know the answer to this or that, than spend 15 minutes exploring the advanced functions on Google and unlocking an everlasting world of independence. Knowledge, as they say, is power. But the knowledge of how to gain knowledge, is the most powerful knowledge of all.

  Author: Bob Leggitt

Advertisements