This research method (qualitative) addresses “how” questions – rather than “how many” through the perspective of those studied – informants.
Gary, 2019, p. 31
Makes sense…qualitative studies are not about “how many” but the words used by participants to describe their experiences and the interpretation of those words by the researcher based on their worldview and theoretical framework.
So, why report this?
Figure 1. Thematic Coding (Gary, 2019, p. 64).
I guess the experiences are important when proposing the study; however, its important to “demonstrate why one should have confidence in the findings” (Hannah & Lautsch, 2011, p. 16). Hannah and Lautsch call his credentialing counting. Who cares if a theme was framed from the responses of 10/10 or 9/10 of participants? Isn’t the theme more important?
There are other problems with this research (e.g., 7 formal “questions” vs an interview guide, no research question of any kind to guide the study), but this counting issue just bugs me. I agree with Sutton (2017): put the numbers in the closet.
References:
Gary, M. E. (2019). Managing toxic leaders: An exploration of human resources management’s role in mitigating the impact of leader imposed toxicity on organization, individuals, and other stakeholders (Doctoral dissertation). ProQuest Dissertations & Theses Global: The Humanities and Social Sciences Collection (13897507)
Hannah, D. R., & Lautsch, B. A. (2011). Counting in qualitative research: Why to conduct it, when to avoid it, and when to closet it. Journal of Management Inquiry, 20(1), 14-22. https://doi.org/10.1177/1056492610375988
As part of an upcoming paper on the quality of doctoral research in DBA programs, I came across a study where the emerging scholar did one of my pet peeves: quantizing qualitative information outside of a mixed method research design. As a result, I employ techniques performed in a prior post to illustrate how this approach can be detrimental to the effect of a study.
Wagner (2019) explored the effectiveness of the Department of Defense’s Transition Assistance Program (TAP) as it related to California veterans. According to the author’s sources, California is home to over 1.8M veterans; 230,000 of those serving post-9/11. The emerging scholar used a questionnaire, rather than an interview guide, to collect data. From information collected via questionnaire (N = 10), the scholar coded participants responses into three levels: high, moderate, and low regarding the participants confidence in a service provided by the TAP. For example, relating to preparedness to transition from the military to the private sector (Appendix E, p. 121), the scholar coded responses into three categories –
High – Veteran utilized resources provided by TAP for financial health, relocation, and career search
Moderate – Veteran able to locate financial, networking, and relocation resources, but did not use
Low – Veteran did not know where to (sic) locate essential transition resources.
Once coded, two faculty reviewed the coding and confirming the ‘classification’ (pp. 52-53). Student Note: This type of step is needed for internal validity in some research designs.
I chose one item for illustration purposes, but the other items are similar in form and content. Participants were asked about drafting a basic resume (see pp. 53-57). From their responses, the emerging scholar classified them into the three levels of confidence and created a cross tabulation of the results. Based on these results, the scholar stated –
The majority of respondents (60%) had a high degree of confidence in drafting a basic resume and cover letter after participating in TAP.
Wagner, 2019, p. 53
Later, the scholar wrote –
High confidence was exhibited by participants who felt empowered by the TAP workshop and were capable of drafting a basic resume. Moderate confidence was demonstrated by veterans who obtained skills to create a basic resume while veterans with low confidence struggled to translate their military career to a basic resume or lacked focus
Wagner, 2019, p. 54 (emphasis added)
By simply describing and interpreting the responses, and not quantizing them into levels, the emerging scholar’s analysis may have had more influence on a reader. There are still questions about the depth of inquiry (e.g., questionnaire vs. in-depth interviews), but that’s hard to explore without obtaining transcripts. However, when quantizing comes into play a reader has to consider the writer’s level of confidence in a “majority of respondents (60%)” statement based on a sample size of 10.
Using the information in Table 3 of the study, I added 95% CI error bars that equate to an N = 230,000 and an n = 10 (CI = 31%; Figure 1) –
Figure 1. Confidence in Drafting a Resume after receiving TAP training with 95% CI = 31% (N = 10)
As one can see, each level’s confidence interval covers the other levels, and detracts from the effect of the study. Just for the record, there was no statistical difference between the three groups, X2(2) = 0.9722, p = .615, due to the small sample size.
This was not a qualitative study; this was merely a quantitative descriptive study. Using the author’s words, the analysis was based on data elicited from “17 formalized questions used during the interview process” (p. 89). Later, the author used the phrase ‘general consensus’ when describing how pre-2011 Veterans Opportunity Work Act participants felt that TAP was a “check in the box as part of out-processing” (p. 51), and TAP provided “adequate support to draft a basic resume and cover letter” (p. 93). Had the scholar simply reported the descriptive statistics based on a larger population, he may have had something; however, would the University had granted a doctorate for that level of rigor?
From a management perspective, if you were in a position to redirect the TAP program and read this study, would you act on these types of results?
Student Note: Make sure you clearly align your research question, research method, and research design. Also, make sure you speak to several faculty members at your university who perform research to get their view on your proposed study methodology. Some faculty focus on only one type of method and try to stuff every study into that mold…right or wrong. Some faculty focus on certain types of QUAN or QUAL. Others only know the method they performed when they did their study. Heck, they may have done their study incorrectly…Remember: It’s your study and it will become a public record.
Reference:
Wagner, J. D. (2019). Effectiveness of the Transition Assistance Process (TAP) in building career self-efficacy for California post-9/11 veterans (Doctoral dissertation). ProQuest LLC. (13865682)
Robinson (2020) interviewed eight recipients of the U.S. Department of Agriculture’s Supplemental Nutrition Assistance Program (SNAP) for the purposes of understanding healthy food choice options and program leadership. What struck my interest was the inclusion of the term Transformational Leadership in the title. It appears the emerging scholar was attempting to connect leadership behavior by Federal government program administrators to food choices made by recipients in the program. I don’t see it. Three things popped into my head –
Were any of the program leaders executing transformational leadership qualities?
What influence does USDA leadership styles have on the leadership styles of regional administrators?
If transformational leadership falls under the umbrella of organizational theory, are recipients of of aid considered part of an organization?
When exploring the themes (p. 55), I came to the conclusion this was simply another example of an oral survey in which participants gave answers to questions –
Participants need better guidance in making healthier food choices
Participants make food choices based on various factors
Federal leaders should hold regional leaders accountable for informing participants about benefits under the program
Participants should be explained the consequences of purchasing certain types of foods
Note that none of these themes had anything to do with transformational leadership and were more related to program evaluation gap analysis. I wonder if the participants could name any of the regional leadership they were supposed to follow?
Ignore the results of this study.
Reference:
Robinson, A. M. (2020). Exploring transformational leadership of the Supplemental Nutrition Assistance Program (SNAP) – A qualitative study. ProQuest Dissertations & Theses Global: The Humanities and Social Sciences Collection. (28031688)
As I’m putting together a paper on the misapplication of case study research design or “non-case case studies”, I wanted to take a break and review a QUAN study. The title of this recently published dissertation interested me: Technical Workforce Shortage and Its Effect on Economic Development in the ECOWAS Region (Jordan, 2020). When the word effect is used in a study, I assume it’s a QUAN study since the term effect means a result of a change. What do I find? Another qualitative non-case case study.
Let me provide some background. ECOWAS stands for Economic Community of West African States. The purpose of the study was to explore perceptions of, and gain insight into, issues HR professionals face while hiring high-tech workers for multinational companies in the ECOWAS region. The emerging researcher performed semi-structured interviews with 10 human resource managers. Three research questions lead this inquiry (my comments in blue):
What are the issues HR Professionals faced while hiring high-tech workers for multinational companies and societies in the ECOWAS region, due to the shortage of high-tech human capital? (If the emerging researcher already knew there is a shortage of high-tech workers, why do the study? Also, aren’t “What?” questions associated with QUAN studies than QUAL?)
What strategies do HR Professionals employ to recruit and retain high-tech workers for multinational companies and societies in the ECOWAS region? (See above)
How is the high-tech shortage problem in the ECOWAS region being addressed currently and for the future? (Since the first two research questions address recruiting and retention, the first part of this research question [currently] has been addressed. The second part of the research question is speculation. The committee should have caught the double-barreled question involving speculation part and removed it.)
If the emerging scholar had adopted the two research questions along the lines I propose, and changed the research design from case study to descriptive, I’d be fine with the proposal. The title, which would normally be finalized after the study is complete, should not have had the word effect in it.
I have a lesser concern about the sample. The emerging scholar represents that each of the participants “were from global staffing and recruitment businesses…that focus on staffing for multinational companies in West African countries” (p. 124), and had “extensive knowledge and lived experiences related to the technical human capital shortage in West Africa” (p. 126). However, there was no evidence that the participants were currently active in recruiting high-tech workers or how long they have been active or inactive. I have knowledge of accounting and generally know how CPA firms recruit recent college graduates, but I haven’t been employed in a CPA firm since 1997. Is my knowledge the same as a PWC recruiter who is doing the job today in our current world and economy?
The emerging scholar identified some themes that align with her research questions (e.g., government infrastructure, lack of law enforcement and safety, brain drain). Unfortunately, some don’t make sense (e.g., Expatriate in charge of technology in Africa), some are mislabeled (recruitment strategies leads to an appropriate discussion about sign-on bonuses and local recruitment fairs; retention strategies leads to an appropriate discussion about merit bonuses, training and development, and knowledge transfer), and some are suggestions (which aligns with public opinion and is unrelated to this study).
From what I see, the emerging scholar did a satisfactory job in executing the study. The interview guide was adequate but items could have been omitted to not muddy the focus of the study; however, in my opinion the committee dropped the ball in helping the student with the title, properly frame the purpose of the study, developing research questions aligned with qualitative research, and, most of all, selecting the appropriate research method. In addition, communicating aspects about her sample, developing the interview guide, and communicating results must be borne by the student and the committee.
My advice to this emerging scholar would have been –
Name your study something like – “An inquiry into hiring and retaining high-tech workers in ECOWAS Member States“
Change your purpose statement to something like – “The purpose of this qualitative, descriptive study was explore challenges facing HR managers in hiring and retaining high-tech workers in ECOWAS member states.”
Research Questions –
Why do HR Professionals have difficulty hiring high-tech workers for ECOWAS member state companies?
How do HR Professionals retain high-tech workers the ECOWAS member state companies?
Reference:
Jordan, T. S. (2020). Technical workforce shortage and its effect on economic development in the ECOWAS Region (Doctoral dissertation). ProQuest Dissertations & Theses Global: The Humanities and Social Sciences Collection. (28090880)
As part of a larger undertaking on exploring how case study research designs are applied by doctoral students and approved by faculty at some universities, I came across an interesting viewpoint on employee turnover. Onyenacho (2019) stated the following problem that led to her research –
Physician turnover is costly for outpatient health care executives (Fibuch & Ahmed, 2015). Outpatient health care executives lose 2 to 3 times the physician’s annual salary when replacing a physician (Shanafelt, Goh, & Sinsky, 2017; Shanafelt & Noseworthy, 2017), and lost revenue for physician replacement is $990,000 per full-time equivalent physician, with an organizational cost to replace a physician ranging from $500,000 to $1,000,000 (Shanafelt et al., 2017). The general business problem was that some outpatient health care executives experienced high physician turnover, which led to increased cost. The specific business problem was that some outpatient health care executives lacked strategies to reduce physician turnover.
Onyenacho, 2019, p. 3 (emphasis added )
My question: Does an individual experience turnover or does an organization experience turnover?
I suppose if a specific health care executive’s compensation or continued employment with an organization is connected to physician turnover rates within the organization, then the first statement would be true. But as I read on, I realized the emerging researcher is attributing or assigning physician turnover to a specific person in an organization rather viewing employee turnover as an organizational artifact. That’s a unique perspective.
I have an idea on why she phrased her problem statement that way. It relates to her selected research design, but that’s a different discussion.
Reference:
Onyenacho, M. A. (2019). Strategies outpatient health care executives use to reduce physician turnover (Doctoral Dissertation). PQDT Open.
Dr. James Ready 