Moving beyond simple searches: How to dig deep with secondary research

Tips for secondary research:

• Use more than one search engine. Have you ever compared the search results from multiple search engines when using the same search terms? As part of some recent research, the results from Google and Bing were very different: only one result was common to both engines’ first page of results. However, one of the search results from Bing was actually very useful (and didn’t show up on Google’s first three pages of results).
• Lean on citations. Academic journals use references to link related content. Once you find a relevant paper, look at the other papers it cites: those may be applicable to your work and the author has already done the work to find topically-related articles. Google Scholar is great at extracting references and linking to the cited papers.
• Be patient with search results. Knowing the language used by an industry is critical to conducting secondary research; these terms are the key to getting to the ‘good stuff’. While having conversations with stakeholders is the fastest way to learn jargon, that isn’t always possible or timely. Starting with a reasonable guess and working with a search engine may pay off, but you may need to read past the first couple pages to tease out the best terms to use going forward.
• Consider using a research tool. Sifting through a lot of research material online can quickly become overwhelming. There are several tools that help researchers manage their sources by saving, categorizing, and tagging them. For example, Zotero is a browser plugin that will recognize content on the web (like a journal article or web content), allow you to save it to the cloud or your hard drive, and catalog the resource (author, date, source, tags, etc.).
• Ask the author for a copy. Sometimes you’ll find a reference to a journal article or paper that fits perfectly with your research but isn’t freely available. Reaching out to the author and politely asking for a copy is appropriate; most researchers want their work to be read and used! Platforms such as academia.edu and ResearchGate can help researchers share their work and may provide a way to reach out to the author. Be polite and be patient.

—

If you looked at the search engine queries from the Zeitspace office over the past year, you might see some interesting ones. Projects relating to yield management, ranking algorithms, and medical assessment tools will have that effect.

As a design and development consultancy, Zeitspace works on a range of projects. Having a team with wonderfully varied backgrounds and skill sets goes a long way to being able to hit the ground running. Our strong culture of learning means we don’t freak out when we encounter something we don’t know. In fact, we’re comfortable with not knowing.

As UX researchers and designers, many of us are used to talking to people in order to better understand what they need, why they might need it, and how our work could help. This type of learning is an example of primary research. Primary research is the way we come up with new information to help us answer questions.

But sometimes, the information already exists and we just need to find it. Secondary UX research is learning from what is already available. In some cases, secondary research can be quite straightforward and involve some time with a search engine (or a book). In other cases, the topic might be more complicated or farther from our areas of expertise. It is that second case — moving beyond simple searches — that we’ll talk about here.

Thankfully, people are quite good at capturing what they’ve learned through primary research and sharing it with others; we can thank books and the internet for that. Information can take the form of academic papers, blog posts, conference presentations, podcasts, videos, and, yes, even Twitter conversations. In fact, all those (and whatever else you can get your hands on) is fair game when it comes to secondary research.

Leaning on those sources of information can help you:

• develop a broad understanding of a problem space;
• learn about the surrounding business context (trends, leaders, etc.);
• use data triangulation, pulling and relating research from multiple sources;
• learn where to go for more in-depth information (when or if we need it);
• build credibility with stakeholders; 
• build confidence in your work

Secondary research activities

(Chart created by Matthew Reynolds)

Doing secondary research is a great way to ramp up quickly so you’re better prepared to design and develop something for clients. It is also a great way to make connections between different elements of product validation and development. Are there regulatory, social, cultural, or commercial trends we should know about? Who are the thought leaders for the space we’re exploring? What is the state of innovation for this industry and its key players? Where are the opportunities for differentiation or innovation? This broader situational awareness can help you mitigate risk and create a better solution.

Getting started with secondary research isn’t tricky, but can be helped along if you’ve learned about the problem from clients, those with lived experience, or subject matter experts. From those conversations, you will learn about jargon and important data sources (journals, community websites, conferences, individuals or companies, for example). During this stage, you should be building up a series of questions you feel are important. You’ll use these questions to test the assumptions you have about the problem and possible solutions.

Before you jump into a marathon search engine session, remember that an important goal is to figure out the edges of the problem space. As Abby Covert says in How to Make Sense of Any Mess: “The first step to taming any mess is to shine a light on it so you can outline its edges and depths.” That is one of the strengths of secondary research: it allows you to feel out the edges of the problem in a structured way.

Keeping that in mind, the first stages of secondary research are certainly exploratory. You should prioritize breadth over depth, at least until you’re confident with the ‘edges’ of the problem space. You may find yourself revisiting and revising your list of questions often, and that’s a good sign — it shows you’re testing your assumptions. Spend some time with your favourite search engines and shine a light on the topic.

You should move on to more specialized sources when you’ve learned a bit about the jargon and found some interesting resources to let you know you’re on the right path. While search engines are mostly wonderful, there is a fair bit of information that lives behind paywalls or the need for user accounts. This means that a lot of great information isn’t indexed consistently and may not show up in search engine results at all.

There is, undoubtedly, a gap between what is freely available from such resources and what is available to paying customers. For example, Canadian universities pay hundreds of millions of dollars each year to access academic journal databases. Until open access to academic publishing reaches a tipping point, paywalls will always guard some content. There is, however, a lot of useful content available freely.

Practical resources for secondary research

Academic databases index thousands of journals, representing millions of articles written over decades. If you or your organization is affiliated with a university, check to see if you’re allowed to use the online databases; these are incredible resources. Online alumni access is often limited, but can be an inexpensive or free way to test the waters. Some universities also allow the general public to use their physical libraries (with computers that can access research databases).

Many public libraries subscribe to journal article databases, news article databases, and learning platforms. Locally, the Kitchener Public Library subscribes to the GALE Academic OneFile library (with millions of articles from 17,000 journals, as well as transcripts from podcasts, NPR, and CNN, for example), ProQuest (newspaper articles, databases, etc.), Lynda.com/LinkedIn Learning, and much more.

Thankfully, not all journal articles are locked up behind paywalls. The Public Library of Science (PLOS) is a search engine with free access to a smaller body of journal articles and resources (about 215,000 articles with a focus on biological sciences).

The Directory of Open Access Journals (DOAJ) is a search engine that indexes open access, peer-reviewed journal articles only (more than 14,000 journals and 4.5-million articles).

JSTOR is a large digital library with “access to more than 12 million academic journal articles, books, and primary sources in 75 disciplines.” Unfortunately, of the more than 2,000 journals represented by JSTOR, only 52 of them are open access.

As much as we love a good peer-reviewed article, secondary research should consider more than academic journals. The Internet Archive aims to keep us honest by backing up large portions of the internet. Their Wayback Machine allows users to visit snapshots of websites over time, which is useful if a web resource has disappeared. Coverage isn’t universal, but it is incredibly powerful when it works. The Internet Archive also allows institutions to upload large bodies of content. Several Canadian universities have uploaded books and archives that are not easily found using Google, but quickly accessed using the Internet Archive search engine.

Google Scholar does two things well. First, it indexes multiple instances of the same paper. If you find an interesting paper behind a paywall, Google Scholar may point you to a version that is available freely (e.g., on the author’s university site). Second, references within a paper are shown as links to the cited papers; this allows you to quickly gather a bunch of related research.

If you aren’t able to find what you’re looking for using free platforms, you may want to consider paying for more specialized access. For example, DeepDyve is a service that caters to those who want access to academic journals. It isn’t cheap, and there are some significant restrictions on what you can do with the content you find.

The Association for Computing Machinery (ACM) is the largest professional group of computer scientists. You can pay to be a member and access the ACM Digital Library, a meticulously cataloged, online repository of hundreds of thousands of articles spanning decades of research.

Whether you use academic, public, or paid platforms to help with secondary research, the goals are the same: to ramp up, to deep dive, to contextualize, to gain confidence, to build credibility. In short, to learn how to make better solutions for people.

There is a whole world out there beyond your default search engine. Have fun!