It is 4:30 in the afternoon and you are winding down. It has been a long day and to your credit, you have put out a few fires, finished all but one of the tasks you set out to do, and you have sat in on 3 meetings that in your opinion really didn’t need to happen. You start to clean up, think about what you are going to do for dinner, “hmmmm, do I cook or go out?”
As you are pondering one of the day's more important decisions, your boss reaches out and asks you to find information on the African swallow for a meeting tomorrow morning. You look at him, nod your head reassuringly and say, “I will have it to you shortly.” In your thoughts, you are thinking what do I know about African swallows ugh?!
And so, your internet journey begins...
Searching the internet can be super frustrating. It is simply amazing how many Web pages are returned when performing a search. When Searching for the illustrious African Swallow depending on the search engine you use there are over 99 million results and that is just one search engine.
Data professionals spend 30% of their time every week searching for, governing, and preparing data, according to a study by the IDC. Despite the technological advances since the early 2000s, the problem of searching and gathering data has not changed significantly other than the fact there is a lot more data. In 2001 McKinsey and the IDC reported that knowledge workers spend two and a half hours a day searching for information. For this article, data on the internet can be grouped into 3 components: Open Public Data, Closed Public Data, and Private Data.
Open Public Data is really, open data and public data. “Open data” is the information that has been published on federal, state, or local government-sanctioned websites, this data is usually structured, machine-readable, open-licensed, and for the most part well maintained. “Public data” is the data that exists everywhere else, again this is information that is freely available on the web but can be difficult to use. It can be unstructured and unruly, and usage requirements are often unclear.
Closed Public Data is data that the public can access for free but generally is behind a barrier such as having the requirement of a free account to access or within a public database that is not crawled or only partially crawled by search engines. An example is most of the federal government contracts, grants, and other opportunities are not easily accessible or findable using a search engine and require you to use an account on one of the various platforms.
Private Data there are three types of private data business, customer, and individual all three are protected and are only available within a closed silo of information. As an example, your personal or company calendar is only available within your private network. This is not typically searchable and only shared if specifically enabled.
For us, average knowledge workers here are a few tips that can help remove the irrelevant, unhelpful, or incorrect results, and the time spent searching publicly available data. Here are a few tips to help:
1) Use specific keywords, or terms: Try and make your keyword as specific as possible, for example typing in African swallow will result in millions of pages but typing in Airspeed African swallow will return more specific results.
2) Try using the minus (-) operator or symbol: This will help narrow your search and bring back a lot less unexpected results. For instance, use African swallow -Monty -Python to remove all references and exclude results from the movie Monty Python and the Holy Grail.
3) Keep your search terms simple and direct: Search engines will use stop words. Stop words are common words typically used as conjunctions or prepositions. Words like (and, but, in of, the, a, on, etc.,) just to name a few. Most times stop words are ignored but searching for African swallow and the African swallow will have different outcomes.
4) Don’t Use Punctuation or suffixes: A couple of other things to note when keeping it simple and direct avoid punctuation it is usually ignored and always try to keep the word at its base or simplest form. Try not to use (-ing, -ed, or s). For example, search for swallow not swallows, or support vice supports, supported, supporting adding suffixes and plurals will change the search results.
5) Quotation Marks: Using quotation marks around a word or phrase will return only those exact words or phrases in that order. It is a great way to limit the number of results. Putting quotes around a word will remove all the variations of that word in the search. A word like competitor, if searched the results will come back for (compete, competition, competitors, etc.,) In quotes “competitor” will limit the pages returned to just competitor. When using a phrase united states Olympic competitor 1980, without quotation marks, your search will return results based on all of the words, with quotation marks the results will be results for that exact phrase.
6) Use more than one search Engine: There are several search engines available the most popular are Google, Bing, and Yahoo. Other search engines include DuckDuckGo, Wiki.com, Dogpile, Yippy, and many more. There are specific search engines as well like Google Scholar If you are looking for academic articles, metacrawler, which searches and aggregates web search results for a phrase or a Keyword of many different search engines (Google, Bing, Yahoo) all at once. All have their advantages and disadvantages and, wait for it…. you can search “the best search engines of 2021” to find out.
Keep in mind search engines are supposed to place the most relevant results at the top. There are now companies that are dedicated to improving a company’s products, or services visibility online. They constantly tweak the wording of a company’s or their own websites to match the most commonly used keywords. This process is known as Search Engine Optimization (SEO). As a result, the information listed at the top of your search may be the result of an exceptionally good SEO. This does not mean that the site will have the best content or information you are looking for.
Hopefully, with these simple tips and tricks, you will be able to find the right information for that meeting on African swallows first thing in the morning.
While accessing public data will be useful, you may need to research closed public data or private data. There are tools and resources to access these harder-to-reach data sets that we will explore in future posts.
Take a tour of Ratio Exchange and see how easy it is to search and find data and information on Federal contracts, grants, and other opportunities to accelerate your business's growth.