Ethnicity: M = 1.26, SD = .529?

I understand that one has to “numericize” categories for quantitative analysis, but any student and faculty should understand the numbers mean nothing when compared (see Figure 1).

Figure 1: Barplot of Ethnicity (Race) with Distribution overlay (Deonarinesingh, 2019, p. 59).

A chairperson has to perform a lot of reading when reviewing a student’s dissertation. A committee member can hopefully help. But this type of error appeared on every chart in this study’s Chapter 4; regardless of the type of variables (e.g., categorical, interval). Did the faculty not know, or did they simply not read the study?

Student Note: Understand your variables and how best to display them. Don’t rely on your committee; they might not know or remember.

This study will return in a later post…stay tuned.

Reference:

Deonarinesingh, S. (2019). The effect of cultural intelligence upon organizational citizenship behavior, mediated by openness to experience (Doctoral dissertation). ProQuest Dissertations & Theses Global: The Humanities and Social Sciences Collection. (13880805)

Advertisement

N = 2 in a non-case study Case Study

A student recently referenced a dissertation that focused on strategies that could be used to promote a sustainable business beyond 5 years (Johnson-Hilliard, 2015). What struck my interest was the size of sample: 2!!! I get it. In qualitative research, it’s not about the number of participants per se but the depth data collection and analysis. As I read on, the student frames the study as a multiple case study. In a multiple case study, an N = 2 could be appropriate where two businesses are compared and contrasted. However, upon further reading, the novice researcher conducted semi-structured interviews with the owners of the business. Reviews of financial statements, market-related factors (e.g., location, competition), or marketing-related artifacts (clients acquired by quarter, advertising mediums, customer lifetime value analyses), dimensions associated with success in business literature, were not performed. She did mention that she reviewed each company’s business plan to “verify if they are on the right path to potential risks or rewards” (p. 54). Besides the Unit of Focus changing from business to business owner, I guess the researcher will also be able to ‘see the future’ regarding potential risks and rewards.

The overarching research question in this study was – What strategies do salon business owners need to succeed in business beyond 5 years? Next, let’s look at the the interview guide (My thoughts are in blue)

  1. What strategies do you use to enhance growth of your business? (The novice researcher is requiring each participant to provide a list of strategies. Do they know what a strategy is? I suppose it’s easier to ask a participant to provide a list rather than dig through documents and transcripts to determine which strategies were key)
  2. How important is having a strategy to you as a small business owner? (Wait! First, tell me what strategies (if any) you employ to enhance growth, then tell me their importance? Does that mean some strategies are not important? Which strategies didn’t enhance growth?)
  3. How do you compete with larger salons? (The size of the competitors salon was just primed by the novice researcher. The participant is now instructed to ignore small- or equal-sized competitors, potentially next door, and discuss how they compete with larger competitors. Hopefully, this line of inquiry leads to a series of marketing-related themes)
  4. What are the causes of negative challenges in salons? (What’s a negative challenge? It’s not defined by the researcher. I searched for the term in some academic literature and couldn’t find anything. If not defined by the researcher, who knows how this will be interpreted by the participants. Taking that into consideration, is the researcher now having the participants speak for the entire industry? Did the scope of the inquiry just change?)
  5. What are some gains or losses of being a successful business owner? (Another swerve from the identification of success factors in operating a salon for 5+ years to entrepreneurship rewards and sacrifices)
  6. What additional information can you provide to assist me in understanding successful salon operations? (A throw-away request for information. With no follow-up to any of the other five items, who cares at this point…)

Before I dig into the themes, I find it troublesome when no details are provided about each business. How long have they been in business? What is their revenue? How many employees? Where are they located in Savannah, generally speaking? How many clients do they see in an average week? What is the average sale? Nothing to tell the reader anything about the businesses so they can decide whether to ignore the study and its results or potentially apply it to different situations.

Now, the themes –

Theme 1: Key Strategies for Salon Owners to Succeed in Business beyond 5 years – Yes, the first theme was the research question. Regardless of the questionable title (Who reviewed this study?), the researcher listed three strategies: Education, Training, and Skills. This makes sense since one has to be licensed by the State of Georgia to practice and maintain a record of Continuing Education. But should a “key strategy” be to make sure you are licensed in the State? It would appear that licensure would be the entrance to the field.

Theme 2: Effective Strategies for a Successful Business – Again, a questionable title; however, three strategies were listed: Customer Service, Niche Marketing, and Technology. The novice researcher reported that P1 stated “she employs excellent customer service in her establishment to all customers” (p. 70). What does “excellent customer service” mean? Isn’t that a self-serving statement? Is somebody going to say they don’t provide excellent customer service? This is an example of a novice researcher “reporting” what people say and calling it a theme rather than a participant describing the customer interaction process and the researcher characterizing the level of customer service. Next, niche marketing. The novice researcher describes how P2 appeared to have a niche market in hair molds and pieces for clients that have lost their hair to cancer, etc. However, there is no reference to the % of sales attributed to this service. Finally, technology. P1 stated she doesn’t use technology to schedule appointments while P2 does. In a 50-50 situation, I don’t understand why this was included in the study.

Theme 3: Determination and Dedication – Both participants identified their own determination and dedication, in what could be described as “self-serving” statements, so the novice researcher “reported” it (pp. 72-73). I guess we’ll never know the components of determination and dedication the two business-owners displayed.

Theme 4: Professionalism – When reading the analysis, the comments made by the participants align with Theme 3. But I’m speculating that because both participants commented about “providing professional environment, service, and attitude” (p. 74), it appears this was another case of extracting words used by the participant and making it a theme.

Yin (2018) describes two types of multiple-case study designs: holistic and embedded. A holistic design focuses on a single unit of analysis, while an embedded design involves looking at multiple units (p. 48). Without an explanation on the size and complexity of the two businesses, its difficult to determine which would have been appropriate; however, simply asking the business owners their thoughts fails both design models.

I don’t know whether to place responsibility on the quality (or lack thereof) of this study on the student, chairperson, committee, or University (my alma mater). This is an example of how the peer-review process can fail an emerging researcher. Regardless, the results of this study should be ignored due to internal validity issues.

References:

Johnson-Hilliard, M. (2015). Small business sustainability in the salon industry (Doctoral dissertation). ProQuest Dissertations Publishing. (3736144).

Yin, R. K. (2018). Case study research and applications: Design and methods (6th Ed.). SAGE Publications.

When categorical variables and moderation analysis goes wrong…

I stumbled across a dissertation (Bosh, 2020), in which the student performed a moderation analysis using categorical variables. By performing a moderation analysis, a researcher is examining if the causal relationship between an independent variable (X) and the dependent variable (Y) changes upon the introduction of a moderating variable (M). To test for moderation, both X and M must be entered into the regression formula, to examine the main or simple effect, along with the interaction (X*M).

Y = i + aX + bM + cXM + e                      (1)

If the p-value of the moderating variable is statistically significant, then the main effects are ignored and the moderator becomes the focus. I have found moderation analysis can be confusing to students who don’t have a good grasp of statistics.

The student examined categorical variables as moderators. Categorical variables of three or more levels should be dummy-coded, since categorical variables with two levels are naturally dichotomous (0/1). This study had four categorical variables: Age, Gender. Marital Status, and Tenure (p. 110). The student references dummy coding but only in relation to Gender and Marital Status; two variables that are either naturally dichotomous (Gender) or artificially dichotomized in the study (Marital Status). No reference to dummy coding was made for Age and Tenure (p. 117). Student Note #1: When using categorical variables in regression, make sure you understand dummy coding.

In dummy coding, a researcher transforms a nominal or ordinal variable into k-1 variables (k refers to the number of levels). For each variable, a specific category is coded as a 1 and all other units are coded as 0. Age, for example, has three levels: 18-33, 34-49, and 50-65. Age would be dummy coded into two variables (Age1 and Age2), with 34-49 being represented by a 1 in Age1 and 50-65 being represented by a 1 in Age2. The base level, 18-33, would be represented as a 0 in both Age1 and Age2. Thus, Age1 and Age2 would be represented as deviations from the base level. For a great discussion of dummy coding, effect coding, and weighted effect coding, see Grotenhuis et al. (2017).

When the student begins testing hypotheses (p. 134), I know two variables are coded correctly and two that are questionable. However, upon inspection of the output, I note that there is no evidence that moderation was examined. In reviewing Table 22 in the study, the main affect of Job Satisfaction was used as a predictor of Affective Commitment in Model 1. However, the two poorly formed MVs of Age and Tenure were entered as a block in Model 2. Entering additional variables into a regression formula and examining the changes is not moderation analysis; it is simply a measurement of change in a model upon the inclusion of additional variables. The other two dichotomous variables, Gender and Marital Status, are entered as a block in the third model.

What does all this mean? Well, the student didn’t structure the moderation analysis properly. First, ordinal independent variables were not dummy-coded properly. Second, interaction was not examined. Could there be a moderating effect of these categorical variables? Maybe. We’ll never know. Technically, this is an example of a combined Type II and Type III error.

I reached out to Capella University via email to request the student’s email address so I could include his thoughts in this discussion; potentially working to perform a post-publication analysis of data. The University did not reply to my email nor to my follow-up phone call to the University’s FERPA representative. I also reached out to the student’s chairperson for comment. No reply.

Instructions to Students

Ignore the results of this study. However, the framework set by Bosh (2020) is ripe for replication. Simply cite the results of the study and the problems in the analysis as a reason for the need to replicate, and do the analysis correctly.

Reference:

Bosh, G. B. (2020). Explanatory relationships among employees personal characteristics, job satisfaction, and employee organizational commitment (Doctoral dissertation). ProQuest Dissertations Publishing. (27837234)

Grotenhuis, M., Pelzer, B., Eisinga, R., Nieuwenhuis, R., Schmidt-Catran, A., & Konig, R. (2017). When size matters: advantages of weighted effect coding in observational studies. International Journal of Public Health, 62(1), 163–167. https://doi.org/10.1007/s00038-016-0901-1