Keyword search vs Abstract search…

The purpose of placing keywords in an abstract is to allow a search engine or another researcher to easily identify main topics in your research. For additional thoughts, see link and link.

As I continue digging through doctoral studies to identify patterns of concern or mistakes, I began reviewing studies from a University that uses a case study method for many students. I’ve identified problems in case studies here and here. I wanted to quickly see how many times the phrase “case study” appeared as a keyword or phrase. Using the R library tidyverse, and two commands (str_detect and table), I found only 4 instances in the keywords:

library(tidyverse)
str_detect(selected_university$keywords, "case study") %>% table()
.
FALSE  TRUE 
  233     4

However, when I searched for the same string in the Abstract, I found 215 instances.

str_detect(selected_university$abstract, "case study") %>% table()
.
FALSE  TRUE 
   22   215

This tells me that a specific research design is deemed not important enough to place as a keyword phrase. No problem.

Student Note: Don’t rely on keywords for finding similar types of research designs.

It also tells me that 90% of this University’s DBA graduates in 2019 used the same research design. Did I hear somebody say formulaic?

In writing about formulaic papers in organizational research, Alvesson and Gabriel wrote –

Formulaic papers are the products of a sequence of interrelated codified and standardized practices that involve formulaic research, a formulaic editorial process, formulaic reviewing, and more generally, formulaic mind-sets, that is, formulaic ways of thinking about what constitutes scholarship. Reliance on a formula is in itself not detrimental to quality, especially if the formula has yielded good results in the past. As we shall see presently, however, slavish adherence to formula renders researchers oblivious to potentially interesting possibilities that exist outside the formula,
eliminating the scope for serendipity and accidental discovery that have long been crucial factors in
scientific discovery and technological innovations
Alvesson & Gabriel, 2013, p. 247 (emphasis added)

I don’t have a problem with writing templates or standardized statistical approaches, but when 90% of a University’s doctoral studies relate to case study methodology, and issues have been identified in research from that University relating to the framing and execution of the case study method, what does that say about the quality of the formula?

Reference:

Alvesson, M., & Gabriel, Y. (2013). Beyond formulaic research: In praise of greater diversity in organizational research and publications. Academy of Management Learning & Education, 12(2), 245-263. https://doi.org/10.5465/amle.2012.0327

Examining influence in a qualitative study?

It’s very important to use the correct terminology associated with a research method and design. For example, the word influence is widely-associated with quantitative research. Influence can be measured by examining the change in the Y variable when the X variable is manipulated or involving a third variable (Z). Quantitative research is more scientific, less subjective, is repeatable, and can be generalized. Qualitative research is based on the knowledge and skill of the researcher. There are times when an experienced researcher will explore influence in a qualitative study but that is few and far between, generally related to specific disciplines (e.g., medical, social work), and is supported with significant academic research (see here, here, and here). I don’t recommend emerging scholars perform qualitative research. Besides skill, the time needed to complete a qualitative study is much longer than the time needed to complete a quantitative study.

Grant (2019) is an example of why emerging scholars shouldn’t do qualitative research. This emerging scholar explored the influence of leadership behaviors on two dimensions: employee engagement and collaboration (the organization is not germane to this discussion). To perform this study, the emerging scholar created a 7-item open-ended survey and distributed it anonymously to 10 people in an organization exceeding 3,800 people. The emerging scholar would interpret the responses and categorize them to answer the following two research questions –

What leadership styles and behaviors are being utilized at [organization]?
What is the influence of existing leadership styles and behaviors on employee engagement and collaboration?

Yin (2018) describes five situations where a single case study would be appropriate: critical, unusual, common, revelatory, or longitudinal (pp. 48-50). In addition, Yin describes two types of single case studies: holistic and embedded (pp. 51-53). When reviewing the dissertation, the researcher is attempting a build a common, holistic single-case study. Common because leadership is an everyday situation. Holistic because the organization appears to have a single purpose. However, a case study focuses on “how” or “why” a situation occurred (perhaps leadership style evolution); not “what” style is prevalent or which specific styles influence two outcomes. With an anonymous survey, there is no way to follow up with a participant to clarify their responses. To quote a colleague –

Who’s the researcher? Carnac the Magnificent?
Name withheld

As a result, the research method (QUAL) and design (case study) doesn’t appear to align with the research questions. The results of the study should be ignored. However, I wanted to discuss the themes identified by Grant –

A collaborative, or transformational, leadership style is present
Organizational leaders are engaging
Unfair hiring practices have become standard

First, are collaborate and transformational the same? They’re close, but I believe some scholars would say they’re different. Second, what does the organization’s hiring practices have to do with leadership in an organization? Plus, how can one generalize to an organization of 3,800+ from a sample of 10? Do the math: That’s a 95% CI of nearly 31 points! Even if 90% of the sample described an organization leaders as collaborative, as interpreted by the researcher, that means the 95% CI would between 60% and Inf. What are the other 40%? Non-collaborative?

Reference:

Grant, R. M. (2019). Investigating the influence of leadership behaviors on employee engagement and collaboration in a Federal organization (Doctoral dissertation). ProQuest Dissertations & Theses Global: The Humanities and Social Sciences Collection. (22615969)

Yin, R. K. (2018). Case study research and applications: Design and methods (6th Ed.). SAGE Publications.

Poorly titled study, poorly worded research questions, misapplied research design: Who’s at fault?

As I’m putting together a paper on the misapplication of case study research design or “non-case case studies”, I wanted to take a break and review a QUAN study. The title of this recently published dissertation interested me: Technical Workforce Shortage and Its Effect on Economic Development in the ECOWAS Region (Jordan, 2020). When the word effect is used in a study, I assume it’s a QUAN study since the term effect means a result of a change. What do I find? Another qualitative non-case case study.

Let me provide some background. ECOWAS stands for Economic Community of West African States. The purpose of the study was to explore perceptions of, and gain insight into, issues HR professionals face while hiring high-tech workers for multinational companies in the ECOWAS region. The emerging researcher performed semi-structured interviews with 10 human resource managers. Three research questions lead this inquiry (my comments in blue):

What are the issues HR Professionals faced while hiring high-tech workers for
multinational companies and societies in the ECOWAS region, due to the shortage of high-tech
human capital? (If the emerging researcher already knew there is a shortage of high-tech workers, why do the study? Also, aren’t “What?” questions associated with QUAN studies than QUAL?)

What strategies do HR Professionals employ to recruit and retain high-tech
workers for multinational companies and societies in the ECOWAS region? (See above)

How is the high-tech shortage problem in the ECOWAS region being addressed
currently and for the future? (Since the first two research questions address recruiting and retention, the first part of this research question [currently] has been addressed. The second part of the research question is speculation. The committee should have caught the double-barreled question involving speculation part and removed it.)

If the emerging scholar had adopted the two research questions along the lines I propose, and changed the research design from case study to descriptive, I’d be fine with the proposal. The title, which would normally be finalized after the study is complete, should not have had the word effect in it.

I have a lesser concern about the sample. The emerging scholar represents that each of the participants “were from global staffing and recruitment businesses…that focus on staffing for multinational companies in West African countries” (p. 124), and had “extensive knowledge and lived experiences related to the technical human capital shortage in West Africa” (p. 126). However, there was no evidence that the participants were currently active in recruiting high-tech workers or how long they have been active or inactive. I have knowledge of accounting and generally know how CPA firms recruit recent college graduates, but I haven’t been employed in a CPA firm since 1997. Is my knowledge the same as a PWC recruiter who is doing the job today in our current world and economy?

The emerging scholar identified some themes that align with her research questions (e.g., government infrastructure, lack of law enforcement and safety, brain drain). Unfortunately, some don’t make sense (e.g., Expatriate in charge of technology in Africa), some are mislabeled (recruitment strategies leads to an appropriate discussion about sign-on bonuses and local recruitment fairs; retention strategies leads to an appropriate discussion about merit bonuses, training and development, and knowledge transfer), and some are suggestions (which aligns with public opinion and is unrelated to this study).

From what I see, the emerging scholar did a satisfactory job in executing the study. The interview guide was adequate but items could have been omitted to not muddy the focus of the study; however, in my opinion the committee dropped the ball in helping the student with the title, properly frame the purpose of the study, developing research questions aligned with qualitative research, and, most of all, selecting the appropriate research method. In addition, communicating aspects about her sample, developing the interview guide, and communicating results must be borne by the student and the committee.

My advice to this emerging scholar would have been –

Name your study something like – “An inquiry into hiring and retaining high-tech workers in ECOWAS Member States“
Change your purpose statement to something like – “The purpose of this qualitative, descriptive study was explore challenges facing HR managers in hiring and retaining high-tech workers in ECOWAS member states.”
Research Questions –
- Why do HR Professionals have difficulty hiring high-tech workers for ECOWAS member state companies?
- How do HR Professionals retain high-tech workers the ECOWAS member state companies?

Reference:

Jordan, T. S. (2020). Technical workforce shortage and its effect on economic development in the ECOWAS Region (Doctoral dissertation). ProQuest Dissertations & Theses Global: The Humanities and Social Sciences Collection. (28090880)

N = 2 in a non-case study Case Study

A student recently referenced a dissertation that focused on strategies that could be used to promote a sustainable business beyond 5 years (Johnson-Hilliard, 2015). What struck my interest was the size of sample: 2!!! I get it. In qualitative research, it’s not about the number of participants per se but the depth data collection and analysis. As I read on, the student frames the study as a multiple case study. In a multiple case study, an N = 2 could be appropriate where two businesses are compared and contrasted. However, upon further reading, the novice researcher conducted semi-structured interviews with the owners of the business. Reviews of financial statements, market-related factors (e.g., location, competition), or marketing-related artifacts (clients acquired by quarter, advertising mediums, customer lifetime value analyses), dimensions associated with success in business literature, were not performed. She did mention that she reviewed each company’s business plan to “verify if they are on the right path to potential risks or rewards” (p. 54). Besides the Unit of Focus changing from business to business owner, I guess the researcher will also be able to ‘see the future’ regarding potential risks and rewards.

The overarching research question in this study was – What strategies do salon business owners need to succeed in business beyond 5 years? Next, let’s look at the the interview guide (My thoughts are in blue)

What strategies do you use to enhance growth of your business? (The novice researcher is requiring each participant to provide a list of strategies. Do they know what a strategy is? I suppose it’s easier to ask a participant to provide a list rather than dig through documents and transcripts to determine which strategies were key)
How important is having a strategy to you as a small business owner? (Wait! First, tell me what strategies (if any) you employ to enhance growth, then tell me their importance? Does that mean some strategies are not important? Which strategies didn’t enhance growth?)
How do you compete with larger salons? (The size of the competitors salon was just primed by the novice researcher. The participant is now instructed to ignore small- or equal-sized competitors, potentially next door, and discuss how they compete with larger competitors. Hopefully, this line of inquiry leads to a series of marketing-related themes)
What are the causes of negative challenges in salons? (What’s a negative challenge? It’s not defined by the researcher. I searched for the term in some academic literature and couldn’t find anything. If not defined by the researcher, who knows how this will be interpreted by the participants. Taking that into consideration, is the researcher now having the participants speak for the entire industry? Did the scope of the inquiry just change?)
What are some gains or losses of being a successful business owner? (Another swerve from the identification of success factors in operating a salon for 5+ years to entrepreneurship rewards and sacrifices)
What additional information can you provide to assist me in understanding successful salon operations? (A throw-away request for information. With no follow-up to any of the other five items, who cares at this point…)

Before I dig into the themes, I find it troublesome when no details are provided about each business. How long have they been in business? What is their revenue? How many employees? Where are they located in Savannah, generally speaking? How many clients do they see in an average week? What is the average sale? Nothing to tell the reader anything about the businesses so they can decide whether to ignore the study and its results or potentially apply it to different situations.

Now, the themes –

Theme 1: Key Strategies for Salon Owners to Succeed in Business beyond 5 years – Yes, the first theme was the research question. Regardless of the questionable title (Who reviewed this study?), the researcher listed three strategies: Education, Training, and Skills. This makes sense since one has to be licensed by the State of Georgia to practice and maintain a record of Continuing Education. But should a “key strategy” be to make sure you are licensed in the State? It would appear that licensure would be the entrance to the field.

Theme 2: Effective Strategies for a Successful Business – Again, a questionable title; however, three strategies were listed: Customer Service, Niche Marketing, and Technology. The novice researcher reported that P1 stated “she employs excellent customer service in her establishment to all customers” (p. 70). What does “excellent customer service” mean? Isn’t that a self-serving statement? Is somebody going to say they don’t provide excellent customer service? This is an example of a novice researcher “reporting” what people say and calling it a theme rather than a participant describing the customer interaction process and the researcher characterizing the level of customer service. Next, niche marketing. The novice researcher describes how P2 appeared to have a niche market in hair molds and pieces for clients that have lost their hair to cancer, etc. However, there is no reference to the % of sales attributed to this service. Finally, technology. P1 stated she doesn’t use technology to schedule appointments while P2 does. In a 50-50 situation, I don’t understand why this was included in the study.

Theme 3: Determination and Dedication – Both participants identified their own determination and dedication, in what could be described as “self-serving” statements, so the novice researcher “reported” it (pp. 72-73). I guess we’ll never know the components of determination and dedication the two business-owners displayed.

Theme 4: Professionalism – When reading the analysis, the comments made by the participants align with Theme 3. But I’m speculating that because both participants commented about “providing professional environment, service, and attitude” (p. 74), it appears this was another case of extracting words used by the participant and making it a theme.

Yin (2018) describes two types of multiple-case study designs: holistic and embedded. A holistic design focuses on a single unit of analysis, while an embedded design involves looking at multiple units (p. 48). Without an explanation on the size and complexity of the two businesses, its difficult to determine which would have been appropriate; however, simply asking the business owners their thoughts fails both design models.

I don’t know whether to place responsibility on the quality (or lack thereof) of this study on the student, chairperson, committee, or University (my alma mater). This is an example of how the peer-review process can fail an emerging researcher. Regardless, the results of this study should be ignored due to internal validity issues.

References:

Johnson-Hilliard, M. (2015). Small business sustainability in the salon industry (Doctoral dissertation). ProQuest Dissertations Publishing. (3736144).

Yin, R. K. (2018). Case study research and applications: Design and methods (6th Ed.). SAGE Publications.