developing critical thinking skills in pharmacy students

•  Sign into My Research
•  Create My Research Account
• Company Website
• Our Products
• About Dissertations
• Español (España)
• Support Center

Select language

• Bahasa Indonesia
• Português (Brasil)
• Português (Portugal)

Welcome to My Research!

You may have access to the free features available through My Research. You can save searches, save documents, create alerts and more. Please log in through your library or institution to check if you have access.

Translate this article into 20 different languages!

If you log in through your library or institution you might have access to this article in multiple languages.

Get access to 20+ different citations styles

Styles include MLA, APA, Chicago and many more. This feature may be available for free if you log in through your library or institution.

Looking for a PDF of this document?

You may have access to it for free by logging in through your library or institution.

Want to save this document?

You may have access to different export options including Google Drive and Microsoft OneDrive and citation management tools like RefWorks and EasyBib. Try logging in through your library or institution to get access to these tools.

• More like this
• Preview Available
• Scholarly Journal

Developing Critical Thinking Skills in Pharmacy Students

No items selected.

Please select one or more items.

Select results items first to use the cite, email, save, and export options

You might have access to more details...

Try and log in through your library or institution to see if they have access.

Content area

To review the importance of and barriers to critical thinking and provide evidence-based recommendations to encourage development of these skills in pharmacy students. Critical thinking (CT) is one of the most desired skills of a pharmacy graduate but there are many challenges to students thinking critically including their own perceptions, poor metacognitive skills, a fixed mindset, a non-automated skillset, heuristics, biases and the fact that thinking is effortful. Though difficult, developing CT skills is not impossible. Research and practice suggest several factors that can improve one's thinking ability: a thoughtful learning environment, seeing or hearing what is done to executive cognitive operations that students can emulate, and guidance and support of their efforts until they can perform on their own. Teaching CT requires coordination at the curricular level and further to the more discrete level of a lesson and a course. Instructor training is imperative to this process since this intervention has been found to be the most effective in developing CT skills.

You have requested "on-the-fly" machine translation of selected content from our databases. This functionality is provided solely for your convenience and is in no way intended to replace human translation. Show full disclaimer

Neither ProQuest nor its licensors make any representations or warranties with respect to the translations. The translations are automatically generated "AS IS" and "AS AVAILABLE" and are not retained in our systems. PROQUEST AND ITS LICENSORS SPECIFICALLY DISCLAIM ANY AND ALL EXPRESS OR IMPLIED WARRANTIES, INCLUDING WITHOUT LIMITATION, ANY WARRANTIES FOR AVAILABILITY, ACCURACY, TIMELINESS, COMPLETENESS, NON-INFRINGMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Your use of the translations is subject to all use restrictions contained in your Electronic Products License Agreement and by using the translation functionality you agree to forgo any and all claims against ProQuest or its licensors for your use of the translation functionality and any output derived there from. Hide full disclaimer

Suggested sources

• About ProQuest
• Terms of Use
• Privacy Policy
• Cookie Policy

A Review of Developing Critical Thinking Skills in Pharmacy Students

• American Journal of Pharmaceutical Education 83(2):ajpe7033
• 83(2):ajpe7033

• University of North Carolina at Chapel Hill

• University of Oklahoma Health Sciences Center

• University of Texas at Austin

Abstract and Figures

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations
• Nutr Health

• Guillermo Isaac González-Rodríguez
• Maira Beatriz García-Híjar
• Jasveen Kaur

• AM J HEALTH-SYST PH
• Melissa S Medina
• Zhiquan Luo
• BMC Med Educ
• Jinxia Zhai
• Haisheng Zhang

• Frisca Ineke Dwi Rahayu
• Yulia Marfuah Hidayati
• Ebru SOLMAZ

• Jennifer D. Robinson
• AM J PHARM EDUC

• Priscilla Callahan-Lyon
• Belinda Ireland
• Richard S. Irwin

• Amir Hossein Hossein Pour
• Daniel Kahneman
• Christopher R. Huber

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

• Corpus ID: 269183111

REVIEW Developing Critical Thinking Skills in Pharmacy Students

• PhD Adam M. Persky , EdD Melissa S. Medina , P. M. Ashley N. Castleberry
• Education, Medicine

28 References

Strategies for teaching students to think critically, comparing the effects of problem-based learning and the traditional lecture method on critical thinking skills and metacognitive awareness in nursing students in a critical care nursing course., does college teach critical thinking a meta-analysis, improving student thinking: a comprehensive approach, moving from novice to expertise and its implications for instruction, energizing learning: the instructional power of conflict, instructional effects on critical thinking: performance on ill-defined issues, patient safety and quality: an evidence-based handbook for nurses, the road to excellence: the acquisition of expert performance in the arts and sciences, sports, and games, a systematic review of prevention and intervention strategies for smartphone addiction in students: applicability during the covid-19 pandemic, related papers.

Showing 1 through 3 of 0 Related Papers

ENHANCED BY  

Training student pharmacists to be strong critical thinkers.

PharmD students publish more than 70 clinical inquiry papers in peer-reviewed journals

By katie ginder-vogel.

During her Advanced Pharmacy Practice Experiences (APPE) rotation at the Winnebago Mental Health Institute, fourth-year PharmD student Marklie Munroe evaluated a medication that had recently been approved for the treatment of tardive dyskinesia, a movement disorder caused by psychiatric medications.

Munroe wanted to provide her preceptors, the practicing pharmacists mentoring her during her APPE rotation, with information about the safety and efficacy of the medication, as well as where the medication could fit into practice. Using what she’d learned at the University of Wisconsin–Madison School of Pharmacy, Munroe wrote a clinical inquiry article, a written response to a clinical question that is formatted to be easily digestible, in order to share the results of her analysis.

“My preceptor informed me there was an upcoming issue of the Pharmacy Society of Wisconsin’s journal focusing on mental health and encouraged me to submit the article,” says Munroe. “It was perfect timing.”

“Our curriculum’s focus on teaching students how to translate and communicate research through clinical inquiries is really a unique part of our PharmD program.” –Barry Gidal

Munroe’s peer-reviewed clinical inquiry, “Valbenazine in tardive dyskinesia,” was accepted by JPSW and was published in the mental health issue of the Journal of the Pharmacy Society of Wisconsin in the March/April 2018 issue.

Practicing pharmacists regularly conduct informal clinical inquiries on the job, by reading, analyzing, and summarizing existing resources, then presenting their opinions verbally to healthcare providers in response to their queries. If the inquiry is answering a common or important question, then pharmacists will draft an article for review and submission to a journal, like Munroe did. At the School of Pharmacy, PharmD students practice writing formal clinical inquiry articles to get them used to thinking in a structured, analytical way that supports logical conclusions.

“My clinical inquiry skills are invaluable to my practice,” Nate Menninga (PharmD ’17), now a resident at the William S. Middleton Memorial Veterans Hospital. As a third-year PharmD student, Menninga published a clinical inquiry about smoking cessation options for patients with mental health conditions.

“I strive to provide the best evidence-based medicine for my patients, who are both inpatient and ambulatory,” says Menninga. “And in order to provide this level of care, I must be able to effectively review the available literature, evaluate how the literature applies to patients, and effectively articulate and document my findings to justify my clinical decisions.”

The UW–Madison School of Pharmacy helps its PharmD students develop this crucial skill by teaching them to write evidence-based clinical inquiry review papers throughout their student careers. As a result, the School has an admirable record of producing high-quality evidence-based review papers that earn publication in peer-reviewed journals.

Over the past 12 years, School of Pharmacy students have had more than 70 clinical inquiry papers published in peer-reviewed journals.

“Our curriculum’s focus on teaching students how to translate and communicate research through clinical inquiries is really a unique part of our PharmD program,” says Barry Gidal , professor and chair of the Pharmacy Practice Division. “The students are learning how to really dissect and synthesize information, which is evident not only in their publishing successes but also in their development into excellent clinicians.”

Implementing clinical inquiry in the curriculum

In the mid-1990s, Connie Kraus , professor and vice chair of the Pharmacy Practice Division, and Denise Walbrandt Pigarelli , an associate professor in the Pharmacy Practice Division, implemented evidence-based inquiry into the PharmD curriculum.

“The School’s preceptors suggested making writing assignments more user-friendly,” says Kraus. “We opted to use a clinical inquiry, which is a written response to a critically appraised topic that is formatted in a specific way.”

In their written clinical inquiries, students are expected to provide an evidence-based response to a targeted question, like Munroe’s evaluation of the safety and effectiveness of valbenazine for treating tardive dyskinesia, or Menninga’s recommendations for the best pharmacologic smoking cessation options for patients with mental health conditions. Students need to provide specific information about patient populations and interventions, conduct an effective keyword search in PubMed and International Pharmaceutical Abstracts, and cite reliable sources such as systematic reviews/meta-analyses and clinical studies, to provide a recommendation and explain the strength of their recommendation.

“The ability to effectively search for information, read and interpret information, and convey it succinctly is the essence of critical thinking,” says Kraus. “In school, it is a written assignment to enable us, as teachers, to evaluate the student’s process. In practice, it is more likely to be a verbal process, but fundamentally the same. It gets down to how you present information, how you justify your response, how you express the way you did your search, and how confident you are in your answer.”

Clinical inquiry is in the School of Pharmacy’s DNA. UW–Madison’s Department of Family Medicine and Community Health is a founding member of the Family Physicians Inquiries Network (FPIN), a nationwide network of university family medicine departments, and Kraus has been a section editor for the FPIN journal, Evidence-Based Practice , since 2006 and a senior author since 2014.

“It’s part of the culture, and the School of Pharmacy has been an active player,” says Kraus.

A longitudinal focus

The School of Pharmacy embeds clinical inquiry and critical thinking skills over three years of the PharmD curriculum to help PharmD students develop and deepen that set of skills.

General instruction that helps PharmD students understand what level of evidence is needed for a systematic review, as opposed to a case report or expert opinion, takes place during students’ second year of pharmacy school. Also during their second year, the students take a foundational course in drug literature evaluation that has been a part of the curriculum for decades.

“The School provides an entire course on the interpretation of drug literature taught by experts in drug information and statistics,” says Menninga. “This course provides the foundation for students to understand the literature and critically evaluate studies for flaws.”

“The ability to effectively search for information, read and interpret information, and convey it succinctly is the essence of critical thinking.” –Connie Kraus

Assistant professor Amanda Margolis, an experiential learning coordinator in the School of Pharmacy’s Pharmacy Practice Division, co-coordinates the drug literature evaluation course.

“In drug literature, we teach students to consider incidence rates and statistical significance, identify flaws, and make sense of them in an article,” says Margolis. “We help them understand how limitations affect the validity of a trial—can you trust those results and apply them to practice?”

Walbrandt Pigarelli says students are building critical thinking skills and processes for solving medication-related problems generally.

“The skills utilized in writing clinical inquiries are essential to our future practice as pharmacists,” says Munroe. “Although we may not be writing a formal paper, we will complete the same assessments, like searching for quality articles, when formulating evidence-based recommendations.”

Clinical inquiries for publication

Kraus coordinates the Introductory Pharmacy Practice Experience (IPPE) program for third-year PharmD students and works with her third-year students to write one clinical inquiry that applies the lessons they learned in their second year to real questions asked by licensed pharmacists in practice. Students, like Menninga, are then invited to work with Kraus to publish their responses.

“During third year, we have a clinical inquiry unit that focuses on necessary skills, such as selecting databases, choosing appropriate search terms, and assessing validity,” explains Munroe. “This series of lectures presented by professors and librarians really set the foundation.”

Topics come from acute care settings, like hospitals, or from community pharmacies, and may be specialized or primary care questions. The students’ preceptors often have patient-related topics or questions about new drugs, like their cost-effectiveness, or how they compare to others in terms of efficacy or toxicity.

“We were taught to help our preceptors,” explains pharmacist Dimmy Sokhal (PharmD ’13), chief clinical officer at Hayat Pharmacy in Milwaukee, who published two clinical inquiries as a resident. “Now, as a preceptor, when I ask students to do it for me, I explain that if I’m calling a prescriber, I should be able to look at the CI (clinical inquiry) and give an evidence-based recommendation right away. For example, if you’re treating diabetes, the CI should cite specific American Diabetes Association guidelines, so the preceptor is prepared to answer questions.”

Kraus says the main thing she’s interested in is a student’s ability to respond to a question in the moment.

“If you are doing a clinical rotation and get involved in problem-solving, you have to be able to find good information quickly, read it, and turn it around to give people good advice,” says Kraus. “We consistently get informal feedback from preceptors and residency directors that our students are good at answering questions and have the confidence to jump in and say, ‘I can help you with that.’”

School of Pharmacy alumni who are now practicing pharmacists agree.

“After writing multiple CIs in school, now, if I have 30 minutes to call a doctor, I can pull an article or review, go through the meta-analysis and figure out where to focus,” says Sokhal.

Kraus, Margolis, and Walbrandt Pigarelli regularly invite students to publish particularly strong papers.

Our students who rotate through family medicine clinics have published many clinical inquiries in Evidence-Based Practice ; other students have published clinical inquires in JPSW ,” says Kraus. “More recently, our pharmacy residents, several of whom are also UW–Madison School of Pharmacy graduates, have also begun to publish clinical inquiries in American Family Physician and the Journal of Family Practice. ”

Gaining a competitive advantage

After students have the opportunity to write a formative review with Kraus in their third year, Walbrandt Pigarelli and Margolis reinforce students’ clinical inquiry skills in their fourth year. Fourth-year student pharmacists do eight clinical rotations and write five or six clinical inquiries.

“Doing assignments for three years produces a certain level of expertise. The rigor of our program has produced incredible results.” –Denise Walbrandt Pigarelli

“There’s a strong pedagogical reason for this method,” says Walbrandt Pigarelli. “Doing assignments for three years produces a certain level of expertise. The rigor of our program has produced incredible results.”

Margolis is involved in several residency programs, and at least two require residents to write a clinical inquiry based on UW–Madison School of Pharmacy’s clinical inquiry method.

“Residents who didn’t go to the UW–Madison School of Pharmacy say they wish they’d had it in pharmacy school,” says Margolis. “The early scaffolding of exposure and experience with clinical inquiry really helps.”

Menninga affirms this and says that requiring students to write multiple clinical inquiries throughout their time in pharmacy school ensures that they gain adequate experience and feedback to help them become an independent practitioner. “These skills will continue to be extremely useful as I continue to care for complex patients,” he says.

Learn more about the School of Pharmacy’s Pharmacy Practice Experiences .

UW–Madison Leading New Research Collaboration Aimed at Treating Lung Scarring Diseases

June 27, 2024

Amanda Margolis Earns AACP Emerging Teaching Scholar Award

June 12, 2024

Student News

The School’s Wisconsin Society of Pharmacy Students Named APhA-ASP Best Chapter Runner-Up

June 11, 2024

RESEARCH ARTICLE: Preparing students for pharmacy practice using critical thinking and reflection

• Kimberly B. Garza Auburn University, Alabama, United States
• Channing R. Ford Jacksonville State University, Jacksonville, Alabama, United States
• Lindsey E. Moseley Auburn University, Alabama, United States
• Bradley M. Wright Auburn University, Alabama, United States

Background: Social, behavioural, and administrative sciences are among the Accreditation Council of Pharmacy Education (ACPE) mandated foundational knowledge requirements for pharmacy school curricula. However, they are often taught in isolation, whereby students lose sight of their meaning and significance in the Pharmacists’ Patient Care Process (PPCP).     

Objective: The objective was to assess performance and critical thinking skills and explore student perceptions of a Longitudinal Patient Case assignment that integrates knowledge from multiple domains into a learning experience exploring how patients and pharmacists navigate the complexities of the US healthcare system.    

Methods: Student pharmacists developed their case throughout the term by incorporating course concepts, then submitted a narrated PowerPoint presentation. Quantitative data sources included overall scores, scores for each domain, midterm and final examination scores, and final course grades, analysed using descriptive statistics and Pearson’s correlation. Student reflections encouraged self-discovery and professional identity development and served as the qualitative data source. Thematic analysis occurred through a multi-phase approach.     

Results: All first-year student pharmacists (n = 153) participated. Scores on the narrated PowerPoint were significantly correlated with midterm and final exams and were moderately correlated with overall course grades. Themes that evolved from students’ self-reflections focused on project execution and learning/knowledge.     

Conclusions: Synthesising and applying content across a course in this manner enables students to make connections, think critically, and be creative.

Author Biographies

Kimberly b. garza, auburn university, alabama, united states.

Department of Health Outcomes Research and Policy, Auburn University Harrison School of Pharmacy

Channing R. Ford, Jacksonville State University, Jacksonville, Alabama, United States

Graduate Studies

Lindsey E. Moseley, Auburn University, Alabama, United States

Office of Student Success, Auburn University Harrison School of Pharmacy

Bradley M. Wright, Auburn University, Alabama, United States

Department of Pharmacy Practice, Auburn University Harrison School of Pharmacy

Accreditation Council for Pharmacy Education. (2015, February 2). Accreditation Standards and Key Elements for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy Degree ("Standards 2016"). https://www.acpe-accredit.org/pdf/Standards2016FINAL.pdf

Amin, M., Norgaard, L.S., Cavaco, A.M., Witry, M.J., Hillman, L., Cernasev, A., et al. (2020). Establishing trustworthiness and authenticity in qualitative pharmacy research. Research in Social and Administrative Pharmacy, 16(10), 1472-1482. https://doi.org/10.1016/j.sapharm.2020.02.005

Bubb, R. (2012). Assessing student learning. In W. Buskist, & V.A. Benassi (Eds), Effective college and university teaching: Strategies and tactics for the new professoriate (pp. 115-124). Thousand Oaks, CA: Sage Publications, Inc. https://doi.org/10.4135/9781452244006.n13

Bush, A.A., & Amechi, M.H. (2019). Conducting and presenting qualitative research in pharmacy education. Currents in Pharmacy Teaching and Learning, 11(6), 638-650. https://doi.org/10.1016/j.cptl.2019.02.030

Castleberry, A., & Nolen, A. (2018). Thematic analysis of qualitative research data: Is it as easy as it sounds? Currents in Pharmacy Teaching and Learning, 10(6), 807-815. https://doi.org/10.1016/j.cptl.2018.03.019

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillside, NJ: Lawrence Erbaum Associates

Garza, K.B., Moseley, L.E., Wright, B.M., & Ford, C.R. (2021). Evaluating the impact of a longitudinal patient case on the development of professionalism and professional identity. New Directions for Teaching and Learning, 89–101. https://doi.org/10.1002/tl.20471

Hamari, J., Shernoff, D.J., Rowe, E., Coller, B., Asbell-Clarke, J., & Edwards, T. (2016). Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning. Computers in Human Behavior, 54, 170-179. https://doi.org/10.1016/j.chb.2015.07.045

Joint Commission of Pharmacy Practitioners. (2014, May 29). Pharmacists’ patient care process. Retrieved November 4, 2021from https://jcpp.net/wp-content/uploads/2016/03/PatientCareProcess-with-supporting-organizations.pdf

Nerali, J., Telang, L., Telang, A., & Chakravarthy, P. (2016). The role of self-directed learning in problem-based learning: Health professions education. Archives of Medicine and Health Sciences, 4(1), 125-126. https://doi.org/10.4103/2321-4848.183364

O'Brien, B.C., Harris, I.B., Beckman, T.J., Reed, D.A., & Cook, D.A. (2014). Standards for reporting qualitative research: a synthesis of recommendations. Academic Medicine, 89(9), 1245-1251. https://doi.org/10.1097/ACM.0000000000000388

Paolini, A. (2015). Enhancing teaching effectiveness and student learning outcomes. Journal of Effective Teaching, 15(1), 20-33

Persky, A.M., Medina, M.S., & Castleberry, A.N. (2019). Developing critical thinking skills in pharmacy students. American Journal of Pharmaceutical Education, 83(2), 7033. https://doi.org/10.5688/ajpe7033

Piaget, J. (1950). Explanation in sociology. In J. Piaget (Ed.), Sociological studies (pp. 30-96). New York, NY: Routledge

Piaget, J., & Cook, M. (1952). The origins of intelligence in children. New York, NY: International Universities Press

Roberts, K., Dowell, A., & Nie, J.B. (2019). Attempting rigour and replicability in thematic analysis of qualitative research data: A case study of codebook development. BMC Medical Research Methodology, 19(1), 66. https://doi.org/10.1186/s12874-019-0707-y

Svinicki, M., & McKeachie, W. (2014). McKeachie's teaching tips: Strategies, research, and theory for college and university teachers. Belmont, CA: Wadsworth Cengage Learning

Vande Wiele, P., Morris, D., Ribière, V., & Ermine, J. (2017). Project based learning for professional identity: A case study of collaborative industry projects in marketing. The Independent Journal of Teaching and Learning, 12(2), 44-63. https://journals.co.za/doi/pdf/10.10520/EJC-c4e4139ef

Wright, B.M., Hornsby, L., Marlowe, K.F., Fowlin, J., Surry, D.W. (2018). Innovating pharmacy curriculum through backward design. TechTrends, 62(3), 224-229. https://doi.org/10.1007/s11528-018-0283-8

How to Cite

• Endnote/Zotero/Mendeley (RIS)

Developed By

Part of the PKP Publishing Services Network

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Am J Pharm Educ
• v.73(4); 2009 Jul 10

Assessment of Critical Thinking in Pharmacy Students

To determine whether changes occur over 1 academic year in pharmacy students' critical thinking skills and disposition to think critically.

First, second, third, and fourth-year pharmacy students completed the California Critical Thinking Skills Test (CCTST) and the California Critical Thinking Disposition Inventory (CCTDI) at the beginning and end of 1 academic year.

One hundred thirty-seven students completed the study. No significant changes occurred over the year in total scores on either instrument. However, scores in 3 of 12 subscale scores changed significantly and several significant correlations were found.

Pharmacy students' scores on 2 critical thinking instruments showed no major improvements over 1 academic year but most scores were above average. Some areas of possible weakness were identified. Additional studies comparing scores over a longer period of time (eg, admission to graduation) are needed.

INTRODUCTION

Critical thinking has been identified as an essential outcome of a pharmacy school education.

The Commission to Implement Change in Pharmaceutical Education in 1994, stated: “Entry level graduates must be able to examine issues rationally, logically, and coherently…since much of professional practice is problem-solving, students need to develop analytical skills to make decisions in both familiar and unfamiliar circumstances.” 1

Norris and Ennis have described critical thinking as “reasonable and reflective thinking that is focused upon deciding what to believe or do.” 2 Kurfiss added that “in critical thinking all assumptions are open to question, divergent views are aggressively sought, and the inquiry is not biased in favor of a particular outcome.” 3 The adequacy of critical thinking training in pharmacy schools' curricula has been questioned. An American College of Clinical Pharmacy (ACCP) White Paper called for renewed attention to outcomes such as critical thinking and their integration into the training of future pharmacists. 4

Many colleges and schools of pharmacy have responded by implementing changes to better promote desired outcomes. One noteworthy example has been the implementation of problem-based learning (PBL) into many curricula, 5 which some believe fosters critical thinking skills. 6 The ability to assess whether desired curricular outcomes are being achieved provides important feedback.

Consenting students enrolled in the Campbell University School of Pharmacy (CUSOP) completed assessment instruments designed to measure critical thinking skills and the disposition to use critical thinking. The primary objective of this study was to determine whether changes in critical thinking scores occurred between the beginning and end of the 2005-2006 academic school year. Secondary objectives included determining whether: (1) differences existed among the 4 student subject groups in scores, (2) correlations existed between instrument total scores and measures of achievement eg, selected Pharmacy College Admissions Test (PCAT) scores and grade point averages, and (3) the use of these instruments might be of value in the future for assessment. This study was significant in that it provided an evaluation of critical thinking using surrogate measures which had previously never been used at the School.

Campbell University is a private institution in North Carolina. The School of Pharmacy was established in 1986 as a doctor of pharmacy (PharmD) program. For analysis the students were placed in subject groups based on their year in the program, ie, first year (P1), second year (P2), third year (P3), or fourth year (P4).

This research was approved by the Campbell University Review Board, Copernicus Group IRB. Informed consent was required of all subjects. The study period was the 2005-2006 academic year. A prospective, pretest/posttest design was utilized. The population for this study were the CUSOP students.

Two previously validated instruments were used in this study: the California Critical Thinking Skills Test, 2000 (CCTST) and the California Critical Thinking Disposition Inventory (CCTDI). 7 - 11 These instruments have been used in several pharmacy studies. 12 - 17 The CCTST measures critical thinking skills and provides a total score and scores for 5 subscales (Analysis, Evaluation, Inference, Deductive Reasoning, and Inductive Reasoning). The CCTDI measures the willingness or disposition of an individual to engage in critical thinking. This instrument provides a total score and scores for 7 subscales (Truth-seeking, Open-Mindedness, Analyticity, Systematicity, Critical Thinking Self-Confidence, Inquisitiveness, and Cognitive Maturity). 8 - 12 , 14

The 2 instruments were administered initially to consenting P1, P2, and P3 students at the beginning of the fall 2005 semester. However, P4 students first completed the instruments in May 2005, immediately before the majority of these students left campus for the beginning of advanced pharmacy practice experiences (APPEs). The final administration of the instruments for all 4 student groups took place at the end of the spring semester 2006.

Due to scheduling conflicts, the instruments could not be administered at the same time; however, all 4 subject groups completed the CCTST and CCTDI within a 2-week period. Most subjects required 45-60 minutes to complete both instruments.

PCAT scores and school grades were obtained through the Office of the Dean with consent of the subjects. All PCAT scores used in this study were scaled scores, not percentiles. PCAT scores from test administration prior to the change in the PCAT norms were recalculated using formulas obtained from Harcourt Assessment (San Antonio, Texas) to represent the adjusted scales.

At CUSOP, the composite, biology, and chemistry PCAT scores are given emphasis by the Admissions Committee. As a result, these scores were used to study possible correlations with instrument total scores along with grade point average (GPA). SPSS 16.0 for Windows (SPSS, Inc., Chicago, IL) was used for data analysis. Because of the differences in times at which the subjects took the PCAT, correlation coefficients were calculated for each group separately rather than for all the subjects together. The predetermined alpha level was 0.05.

The baseline data of the participants are presented in Table ​ Table1. 1 . One hundred fifty-four students provided informed consent and began the study. The majority of subjects were female with no significant differences in gender distribution between subject groups and their respective classes (p<0.05). Overall participation in the study ranged from 51.3% of the P1 class to 24.7% of the P4 class. More P3 subjects had bachelor's degrees than would have been predicted from the number in their respective class. One hundred thirty-seven students completed both phases of the administration of the CCTST and CCTDI. Data from matched pairs were used for analysis.

Baseline Information on Pharmacy Students Participating in a Study to Determine Changes in Critical Thinking Over One Academic Year

a No significant differences ( p < 0.05) between groups and their respective classes in age or prior years of college.

b No significant differences between the P1, P2, and P4 groups and their respective classes in attainment of prior college degree ( p < 0.05). A significant difference was found between the P3 group and their respective class.

c Prior GPA for P1 students = prepharmacy GPA; prior GPA for P2, P3, and P4 students = pharmacy GPA immediately prior to study period. Cumulative GPA = cumulative pharmacy GPA at end of study period. No significant difference was found among the study groups or between the study groups and their respective classes in GPAs ( p < 0.05)

No significant differences were found between the subject groups and their respective classes in GPA prior to the study period or cumulative GPA at the end of the study period.

The possible ranges for instrument scores are reported in Table ​ Table2. 2 . Paired t test analyses are presented in Table ​ Table3 3 (total scores) and Table ​ Table4 4 (subscale scores). No significant changes in scores between the beginning and end of the academic year occurred in total scores for either instrument. Of 12 possible subscale scores, significant ( p < 0.05) changes occurred in 3 (Table ​ (Table3): 3 ): Inference (P3 subjects' scores decreased), Open-Mindedness (P4 subjects' scores decreased) and Self-Confidence (P2 subjects' scores increased). Inference is a subscale of the CCTST. Open-mindedness and Self-Confidence are subscales of the CCTDI.

Possible Ranges of Instrument Total Scores and Subscales 9 , 11

Abbreviation: CT = critical thinking

Changes in Pharmacy Students' Critical Thinking Total Scores a , b

Abbreviations: CCTST = California Critical Thinking Skills Test; CCTDI = California Critical Thinking Disposition Inventory

Changes in CCTST and CCTDI Subscale Scores

One-way ANOVA revealed no significant differences among the 4 subject groups in either the initial or final total test scores for the 2 instruments. However, 4 significant CCTST subscale differences were detected among groups and are noted in Table ​ Table4. 4 . No significant differences were found among subject groups in CCTDI subscale scores.

Several significant correlations are noted in Table ​ Table5. 5 . A consistent finding was correlation between composite PCAT and CCTST scores. Composite PCAT scores were correlated ( p < 0.05) with both initial and final CCTST total scores in the P1, P2, and P4 groups and with the initial CCTST total score in the P3 group. The CCTDI scores had fewer significant correlations.

Pearson Correlation Coefficients of Pharmacy Students' CCTST and CCTDI Scores With Demographic Variables

Although no significant changes occurred in total scores during the study, the majority of scores were favorable. CCTST total scores from subject groups were comparable to mean scores of 17-21 reported by others. 12 , 14 , 17 The mean CCTST score from an aggregate sample of 4-year college students (non-health science majors) was 16.8 (median 16). 11 Miller 12 and Phillips et al 14 both reported that median subject scores in their studies exceeded the college comparative group median of 16. For both CCTST initial and final scores, 70% of all CUSOP subjects scored greater than 16.

Miller 12 and Phillips et al 14 reported significant increases in CCTST total scores in their studies when the same cohorts of students were followed from admission to graduation. In our study, scores were compared before and after 1 academic year only. Miller 12 did present yearly data for several classes but no significant yearly changes were indicated. However, a trend toward higher scores was found as students proceeded through the curriculum.

The influence of a “ceiling effect” limiting changes in CCTST is a possible factor. Phillips et al 14 found that students who scored at or above 16 on an initial administration of CCTST did not experience a significant change in their subsequent score; however, those students who initially scored below 16 had a significant increase in their second score. The possibility of a limiting or ceiling effect in studies of medical students and residents also has been described. 18 , 19

Some similarities exist between Phillips et al 14 findings and the CUSOP study. The mean initial total score of CUSOP subjects whose scores were already at or above 16 decreased slightly over the study period: 22.3 vs. 21.9 ( p = 0.326, n = 104). Subjects whose pretest scores were below 16 had an increase in their second score, though the change was not significant: 13.5 vs. 14.5 ( p = 0.084, n = 33).

No changes in CCTDI total scores over 1 academic year were found with CUSOP subjects, though significant increases were found by Phillips et al 14 but not by Miller. 12 Over 75% of all CCTDI total scores at CUSOP were above a value of 280. Scores below 280 are considered indicative of overall weakness in willingness to use critical thinking. 9 CUSOP total scores were in the same range as mean scores, which have been reported by others. 12 , 14 , 17

CUSOP subscale scores overall were also comparable to scores reported in other studies and to comparative norms. 9 , 12 , 14 All but one of the CCTDI subscale mean scores were in the 40-49 range in this study and in others. 12 , 14 This range (40-49) is considered positive. The Truthseeking subscale scores though were in the 30-39 range for all 4 CUSOP subject groups. Truthseeking is the willingness to pursue the truth even if it might be different than one's own preconceived beliefs. 9 The 30-39 range is indicative of ambivalence.

Similar ranges for CCTDI subscales have now been found in at least 3 different pharmacy studies of students at different stages of a pharmacy curriculum. All but the Truthseeking scale have been in the 40-49 range. The Truthseeking scores might be “normal” for pharmacy students. Perhaps curricula are limited in the extent to which this trait can be influenced. Individual maturity and professional experience may be more significant factors in influencing this score. Or it may be an indication of curricular weakness in developing this trait. As more experiences with these instruments are published, a better picture can be obtained of what is reasonable to expect from students and from a curriculum. The availability of test scores from practicing pharmacists may also provide an additional view of critical thinking and a valuable source of data for comparison with students' scores. It is difficult to discern the significance of the subscale changes. The majority of subscale scores (9 of 12) did not change significantly, and most changes in scores that did occur were positive. The 3 changes occurred in 3 different subject groups. Future studies with these instruments hopefully can provide more insight.

If a presumption is that students' critical thinking scores as measured by these instruments should steadily improve each academic year, then a conclusion of this study might be that the CUSOP curriculum in some way may be deficient. This possibility cannot be ignored and continuous curriculum assessment is critical. However, similar research which found significant changes compared scores over a longer time period. 12 , 14

If the effect of a curriculum on critical thinking is cumulative, perhaps there may be delayed effects on scores from instruments such as the CCTST and CCTDI. Instrument scores after 1 semester or 1 academic year may not truly reflect influences on critical thinking by the curriculum experienced during that study period. A longer study period would allow more time for any influence of the curriculum to become apparent. The interactions between and among courses, faculty members, and students, including the school culture, may have an impact on critical thinking and other outcomes. These influences are difficult to control for and measure and may require a longer period of time to effect instrument scores.

A possible exception might be during times of significant curriculum change or for specific interventions such as PBL or critical thinking classes.

The CCTST was associated with several significant correlations. The correlations between the composite PCAT and CCTST total scores suggest that elements of critical thinking are indeed being measured by the composite PCAT. Miller also found similar correlations. 12 In the case of the correlations involving the P4 subjects, the PCAT was taken several years prior to this study. The relationship between the composite PCAT and CCTST scores seems to extend deeper into the pharmacy curriculum than just the initial first or second year. This may have implications in admissions decisions. The lack of many correlations with CCTDI is not unexpected because of the differences between the instruments. However, one significant finding was the correlation between P4 final GPA and their CCTDI total scores. Kidd and Latif 17 found that both CCTST and CCTDI were among a group of factors which seemed useful in predicting curricular success.

Some CUSOP subjects did have CCTST and CCTDI total scores below average or in a range indicative of possible weakness. Reports of the significance of low instrument scores on academic progression in pharmacy have not been found. Following these students to evaluate any type of difficulties encountered in the curriculum may provide valuable feedback on the meaning and value of the instrument scores. Overall our findings provide useful information for curriculum and admissions discussions. A picture of the CUSOP subjects was obtained that was not previously available.

Continuous use of these instruments has not taken place since the time of this study. However, we plan to administer the instruments again to new P1 subjects and follow this same cohort through the curriculum and retest again at least during their P4 year.

There is an expense associated with the use of these instruments; thus, yearly testing of every class may not be cost-effective or labor-effective for some schools.

Results of this study support the need for attention to the critical thinking abilities of CUSOP pharmacy students and have provided at least a snapshot of the extent of these skills in the subjects as measured by these instruments. This study provided a baseline to which future use of these instruments, as well as other measures of assessment, can be compared.

The use of the CCTST or the CCTDI is one part of an overall assessment plan which can present a unique picture of the students and the curriculum. These instruments along with other assessment measures may provide the triangulation necessary for the best possible evaluation of student critical thinking abilities and the effectiveness of a curriculum. As Henderson and Hawthorne 20 have described, a complete curriculum picture requires several “lenses” to view the curriculum through. CCTST and CCTDI can provide two of those additional lenses. Overall the results of this study have established a foundation which can serve as a starting point for continued research here.

Limitations

Several limitations are recognized. Sample size was small and a significant limiting factor. The subsequent low power may have impaired the detection of significant differences. This type of research had not been conducted at CUSOP prior to the study. Hopefully, continued educational research will result in students becoming more agreeable to participate in such studies. Because of IRB requirements students could not be required to participate; thus, self-selection may have biased the results. The P1 subject group was the only group that represented at least 50% of their respective class. Although a few differences among the groups were noticed (Table ​ (Table1), 1 ), differences in other characteristics, such as motivation, may have influenced the results and differentiated the groups. The influence of a ceiling effect limiting changes in scores might be possible. Future studies may provide more information regarding this influence.

A better picture of critical-thinking changes and influences of the curriculum might be seen with longer study periods. Significant changes in critical thinking scores with these instruments may require a longer study period. Future studies are being planned to compare scores at least between admission and graduation for a cohort of students.

The time needed to complete the instruments was longer than the typical 50-minute class period for many students. This may have contributed to the lack of willingness of some students to complete both phases of the study. The lack of incentives available to the students, as well as the pressure of school work in general, may have reduced willingness to participate. The experience of participating in this study in their early years in pharmacy school may have a positive influence on students' willingness to participate in future research.

CONCLUSIONS

There were no significant changes in P1, P2, P3, and P4 students' scores on the CCTST and CCTDI over 1 academic year. The majority of students' scores were still above average for college students/young adults and were comparable to scores of pharmacy students reported in other studies. Some student scores were below average and indicated possible weakness in the area of critical thinking. An understanding of the significance of both higher and lower scores on these instruments as students progress through the curriculum would be valuable.

ACKNOWLEDGEMENTS

Funding for this study was provided through the American Foundation for Pharmaceutical Education's (AFPE) New Investigator Program administered by the American Association of Colleges of Pharmacy (AACP). The assistance of Ms. Jean White, Ms. Phyllis Strickland, Dr. Don Meagher, and the student subjects of CUSOP is gratefully acknowledged.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Assessment of critical thinking in pharmacy students

Affiliation.

• 1 Campbell University School of Pharmacy, Buies Creek, NC 27506, USA.
• PMID: 19657499
• PMCID: PMC2720362
• DOI: 10.5688/aj730466

Objective: To determine whether changes occur over 1 academic year in pharmacy students' critical thinking skills and disposition to think critically.

Methods: First, second, third, and fourth-year pharmacy students completed the California Critical Thinking Skills Test (CCTST) and the California Critical Thinking Disposition Inventory (CCTDI) at the beginning and end of 1 academic year.

Results: One hundred thirty-seven students completed the study. No significant changes occurred over the year in total scores on either instrument. However, scores in 3 of 12 subscale scores changed significantly and several significant correlations were found.

Conclusion: Pharmacy students' scores on 2 critical thinking instruments showed no major improvements over 1 academic year but most scores were above average. Some areas of possible weakness were identified. Additional studies comparing scores over a longer period of time (eg, admission to graduation) are needed.

Keywords: California Critical Thinking Disposition Inventory (CCTDI); California Critical Thinking Skills Test (CCTST); assessment; critical thinking; curriculum.

PubMed Disclaimer

Similar articles

• Incorporation of an Explicit Critical-Thinking Curriculum to Improve Pharmacy Students' Critical-Thinking Skills. Cone C, Godwin D, Salazar K, Bond R, Thompson M, Myers O. Cone C, et al. Am J Pharm Educ. 2016 Apr 25;80(3):41. doi: 10.5688/ajpe80341. Am J Pharm Educ. 2016. PMID: 27170812 Free PMC article.
• Measuring change in students' critical thinking ability: implications for health care education. Bartlett DJ, Cox PD. Bartlett DJ, et al. J Allied Health. 2002 Summer;31(2):64-9. J Allied Health. 2002. PMID: 12040999
• Investigating the Correlation Between Pharmacy Student Performance on the Health Science Reasoning Test and a Critical Thinking Assignment. Nornoo AO, Jackson J, Axtell S. Nornoo AO, et al. Am J Pharm Educ. 2017 Mar 25;81(2):24. doi: 10.5688/ajpe81224. Am J Pharm Educ. 2017. PMID: 28381884 Free PMC article.
• Do scores on three commonly used measures of critical thinking correlate with academic success of health professions trainees? A systematic review and meta-analysis. Ross D, Loeffler K, Schipper S, Vandermeer B, Allan GM. Ross D, et al. Acad Med. 2013 May;88(5):724-34. doi: 10.1097/ACM.0b013e31828b0823. Acad Med. 2013. PMID: 23524925 Review.
• Shared responsibility for the education of pharmacy students. Turner C. Turner C. Am J Health Syst Pharm. 2018 Feb 15;75(4):223-229. doi: 10.2146/ajhp160708. Am J Health Syst Pharm. 2018. PMID: 29436468 Review. No abstract available.
• Trend of tendency to critical thinking among medical students in Tehran University of Medical Sciences, 2010-2015: A longitudinal study. Najafi M, Motlagh MK, Najafi M, Kashani AS, Ostadzar NF, Zarghi N, Shirazi M. Najafi M, et al. J Educ Health Promot. 2022 Jan 31;11:29. doi: 10.4103/jehp.jehp_1373_20. eCollection 2022. J Educ Health Promot. 2022. PMID: 35281392 Free PMC article.
• Psychometric properties of the critical thinking disposition assessment test amongst medical students in China: a cross-sectional study. Cui L, Zhu Y, Qu J, Tie L, Wang Z, Qu B. Cui L, et al. BMC Med Educ. 2021 Jan 6;21(1):10. doi: 10.1186/s12909-020-02437-2. BMC Med Educ. 2021. PMID: 33407421 Free PMC article.
• Using Focus Groups to Explore Evolving Perceptions of Student Pharmacists' Curricular Experiences. Moseley LE, Ford CR, Wilkins EB. Moseley LE, et al. Am J Pharm Educ. 2020 Jan;84(1):7122. doi: 10.5688/ajpe7122. Am J Pharm Educ. 2020. PMID: 32292187 Free PMC article.
• The Need for Palliative Care in Pharmacy Education. Pruskowski J, Patel R, Brazeau G. Pruskowski J, et al. Am J Pharm Educ. 2019 Jun;83(5):7410. doi: 10.5688/ajpe7410. Am J Pharm Educ. 2019. PMID: 31333268 Free PMC article.
• A Systematic Review of Entrepreneurship in Pharmacy Practice and Education. Mattingly TJ 2nd, Mullins CD, Melendez DR, Boyden K, Eddington ND. Mattingly TJ 2nd, et al. Am J Pharm Educ. 2019 Apr;83(3):7233. doi: 10.5688/ajpe7233. Am J Pharm Educ. 2019. PMID: 31065169 Free PMC article. Review.
• Penna RP. Alexandria, VA: American Association of Colleges of Pharmacy; 1994. The Papers of the Commission to Implement Change in Pharmaceutical Education; p. 18.
• Norris SP, Ennis RH. Evaluating Critical Thinking. Pacific Grove, CA: Midwest Publications; 1989. p. 3.
• Kurfiss JG. Washington, DC: The George Washington University, Graduate School of Education and Human Development; 1989. Critical Thinking: Theory, Research, Practice, and Possibilities. ASHE ERIC Higher Education Report No. 2; p. 2.
• American Association of Colleges of Pharmacy. A vision of pharmacy's future roles, responsibilities, and Manpower needs in the United States. Pharmacotherapy. 2000;20(8):991–1020. - PubMed
• Cisneros RM, Salisbury-Glennon JD, Anderson-Harper HM. A review of problem-based learning research in pharmacy: A call for future research. Am J Pharm Educ. 2002;66(1):19–26.

Publication types

• Search in MeSH

LinkOut - more resources

Full text sources.

• Europe PubMed Central
• PubMed Central
• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 31 August 2024

Development and validation of a higher-order thinking skills (HOTS) scale for major students in the interior design discipline for blended learning

• Dandan Li 1 ,
• Xiaolei Fan 2 &
• Lingchao Meng 3  

Scientific Reports volume  14 , Article number:  20287 ( 2024 ) Cite this article

Metrics details

• Environmental social sciences

Assessing and cultivating students’ HOTS are crucial for interior design education in a blended learning environment. However, current research has focused primarily on the impact of blended learning instructional strategies, learning tasks, and activities on the development of HOTS, whereas few studies have specifically addressed the assessment of these skills through dedicated scales in the context of blended learning. This study aimed to develop a comprehensive scale for assessing HOTS in interior design major students within the context of blended learning. Employing a mixed methods design, the research involved in-depth interviews with 10 education stakeholders to gather qualitative data, which informed the development of a 66-item soft skills assessment scale. The scale was administered to a purposive sample of 359 undergraduate students enrolled in an interior design program at a university in China. Exploratory and confirmatory factor analyses were also conducted to evaluate the underlying factor structure of the scale. The findings revealed a robust four-factor model encompassing critical thinking skills, problem-solving skills, teamwork skills, and practical innovation skills. The scale demonstrated high internal consistency (Cronbach's alpha = 0.948–0.966) and satisfactory convergent and discriminant validity. This scale provides a valuable instrument for assessing and cultivating HOTS among interior design major students in blended learning environments. Future research can utilize a scale to examine the factors influencing the development of these skills and inform instructional practices in the field.

Similar content being viewed by others

A meta-analysis of the effects of design thinking on student learning

Blended knowledge sharing model in design professional

Using design thinking for interdisciplinary curriculum design and teaching: a case study in higher education

Introduction.

In the contemporary landscape of the twenty-first century, students face numerous challenges that necessitate the development of competitive skills, with a particular emphasis on the cultivation of HOTS 1 , 2 , 3 , this has become a crucial objective in educational reform. Notably, it is worth noting that the National Education Association (NEA, 2012) has clearly identified critical thinking and problem-solving, communication, collaboration, creativity, and innovation as key competencies that students must possess in the current era, which are considered important components of twenty-first century skills 4 , 5 , 6 , 7 . As learners in the fields of creativity and design, students in the interior design profession also need to possess HOTS to address complex design problems and the evolving demands of the industry 8 , 9 .

Currently, blended learning has become an important instructional model in interior design education 10 , 11 . It serves as a teaching approach that combines traditional face-to-face instruction with online learning, providing students with a more flexible and personalized learning experience 12 , 13 . Indeed, several scholars have recognized the benefits of blended learning in providing students with diverse learning resources, activities, and opportunities for interaction, thereby fostering HOTS 14 , 15 , 16 , 17 . For example, blended learning, as evidenced by studies conducted by Anthony et al. 10 and Castro 11 , has demonstrated its efficacy in enhancing students' HOTS. The integration of online resources, virtual practices, and online discussions in blended learning fosters active student engagement and improves critical thinking, problem solving, and creative thinking skills. Therefore, teachers need to determine appropriate assessment methods and construct corresponding assessment tasks to assess students' expected learning outcomes. This decision requires teachers to have a clear understanding of students' learning progress and the development of various skills, whereas students have knowledge of only their scores and lack awareness of their individual skill development 18 , 19 .

Nevertheless, the precise assessment of students' HOTS in the blended learning milieu poses a formidable challenge. The dearth of empirically validated assessment tools impedes researchers from effectively discerning students' levels of cognitive aptitude and developmental growth within the blended learning realm 20 , 21 , 22 . In addition, from the perspective of actual research topics, current studies on blended learning focus mainly on the "concept, characteristics, mechanisms, models, and supporting technologies of blended learning 23 . " Research on "measuring students' HOTS in blended learning" is relatively limited, with most of the focus being on elementary, middle, and high school students 24 , 25 . Few studies have specifically examined HOTS measurement in the context of university students 26 , 27 , particularly in practical disciplines such as interior design. For example, Bervell et al. 28 suggested that the lack of high-quality assessment scales inevitably impacts the quality of research. Additionally, Schmitt 29 proposed the “Three Cs” principle for measurement, which includes clarity, coherence, and consistency. He highlighted that high-quality assessment scales should possess clear and specific measurement objectives, logically coherent items, and consistent measurement results to ensure the reliability and validity of the data. This reflects the importance of ensuring the alignment of the measurement goals of assessment scales with the research questions and the content of the discipline in the design of assessments.

The development of an assessment scale within the blended learning environment is expected to address the existing gap in measuring and assessing HOTS scores in interior design education. This scale not only facilitates the assessment of students' HOTS but also serves as a guide for curriculum design, instructional interventions, and student support initiatives. Ultimately, the integration of this assessment scale within the blended learning environment has the potential to optimize the development of HOTS among interior design students, empowering them to become adept critical thinkers, creative problem solvers, and competent professionals in the field.

Therefore, this study follows a scientific scale development procedure to develop an assessment scale specifically designed to measure the HOTS of interior design students in blended learning environments. This endeavor aims to provide educators with a reliable instrument for assessing students' progress in cultivating and applying HOTS, thus enabling the implementation of more effective teaching strategies and enhancing the overall quality of interior design education. The research questions are as follows:

What key dimensions should be considered when developing a HOTS assessment scale to accurately capture students' HOTS in an interior design major blended learning environment?

How can an advanced thinking skills assessment scale for blended learning in interior design be developed?

How can the reliability and validity of the HOTS assessment scale be verified and ensured, and is it reliable and effective in the interior design of major blended learning environments?

Key dimensions of HOTS assessment scale in an interior design major blended learning environment

The research results indicate that in the blended learning environment of interior design, this study identified 16 initial codes representing key dimensions for enhancing students' HOTS. These codes were further categorized into 8 main categories and 4 overarching themes: critical thinking, problem-solving, teamwork skills and practical innovation skills. They provide valuable insights for data comprehension and analysis, serving as a comprehensive framework for the HOTS scale. Analyzing category frequency and assessing its significance and universality in a qualitative dataset hold significant analytical value 30 , 31 . High-frequency terms indicate the central position of specific categories in participants' narratives, texts, and other data forms 32 . Through interviews with interior design experts and teachers, all core categories were mentioned more than 20 times, providing compelling evidence of their universality and importance within the field of interior design's HOTS dimensions. As shown in Table 1 .

Themes 1: critical thinking skills

Critical thinking skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. This discovery emphasizes the importance of critical thinking in interior design learning. This mainly includes the categories of logical reasoning and judgment, doubt and reflection, with a frequency of more than 8, highlighting the importance of critical thinking skills. Therefore, a detailed discussion of each feature is warranted. As shown in Table 2 .

Category 1: logical reasoning and judgment

The research results indicate that in a blended learning environment for interior design, logical reasoning and judgment play a key role in cultivating critical thinking skills. Logical reasoning refers to inferring reasonable conclusions from information through analysis and evaluation 33 . Judgment is based on logic and evidence for decision-making and evaluation. The importance of these concepts lies in their impact on the development and enhancement of students' HOTS. According to the research results, interior design experts and teachers unanimously believe that logical reasoning and judgment are very important. For example, as noted by Interviewee 1, “For students, logical reasoning skills are still very important. Especially in indoor space planning, students use logical reasoning to determine whether the layout of different functional areas is reasonable”. Similarly, Interviewee 2 also stated that “logical reasoning can help students conduct rational analysis of various design element combinations during the conceptual design stage, such as color matching, material selection, and lighting application”.

As emphasized by interviewees 1 and 2, logical reasoning and judgment are among the core competencies of interior designers in practical applications. These abilities enable designers to analyze and evaluate design problems and derive reasonable solutions from them. In the interior design industry, being able to conduct accurate logical reasoning and judgment is one of the key factors for success. Therefore, through targeted training and practice, students can enhance their logical thinking and judgment, thereby better addressing design challenges and providing innovative solutions.

Category 2: skepticism and reflection

Skepticism and reflection play crucial roles in cultivating students' critical thinking skills in a blended learning environment for interior design. Doubt can prompt students to question and explore information and viewpoints, whereas reflection helps students think deeply and evaluate their own thinking process 34 . These abilities are crucial for cultivating students' higher-order thinking skills. According to the research findings, most interior design experts and teachers agree that skepticism and reflection are crucial. For example, as noted by interviewees 3, “Sometimes, when facing learning tasks, students will think about how to better meet the needs of users”. Meanwhile, Interviewee 4 also agreed with this viewpoint. As emphasized by interviewees 3 and 4, skepticism and reflection are among the core competencies of interior designers in practical applications. These abilities enable designers to question existing perspectives and practices and propose innovative design solutions through in-depth thinking and evaluation. Therefore, in the interior design industry, designers with the ability to doubt and reflect are better able to respond to complex design needs and provide clients with unique and valuable design solutions.

Themes 2: problem-solving skills

The research findings indicate that problem-solving skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. This discovery emphasizes the importance of problem-solving skills in interior design learning. Specifically, categories such as identifying and defining problems, as well as developing and implementing plans, have been studied more than 8 times, highlighting the importance of problem-solving skills. Therefore, it is necessary to discuss each function in detail to better understand and cultivate students' problem-solving skills. As shown in Table 3 .

Category 1: identifying and defining issues

The research findings indicate that in a blended learning environment for interior design, identifying and defining problems play a crucial role in fostering students' problem-solving skills. Identifying and defining problems require students to possess the ability to analyze and evaluate problems, enabling them to accurately determine the essence of the problems and develop effective strategies and approaches to solve them 35 . Interior design experts and teachers widely recognize the importance of identifying and defining problems as core competencies in interior design practice. For example, Interviewee 5 emphasized the importance of identifying and defining problems, stating, "In interior design, identifying and defining problems is the first step in addressing design challenges. Students need to be able to clearly identify the scope, constraints, and objectives of the problems to engage in targeted thinking and decision-making in the subsequent design process." Interviewee 6 also supported this viewpoint. As stressed by Interviewees 5 and 6, identifying and defining problems not only require students to possess critical thinking abilities but also necessitate broad professional knowledge and understanding. Students need to comprehend principles of interior design, spatial planning, human behavior, and other relevant aspects to accurately identify and define problems associated with design tasks.

Category 2: developing and implementing a plan

The research results indicate that in a blended learning environment for interior design, developing and implementing plans plays a crucial role in cultivating students' problem-solving abilities. The development and implementation of a plan refers to students identifying and defining problems, devising specific solutions, and translating them into concrete implementation plans. Specifically, after determining the design strategy, students refine it into specific implementation steps and timelines, including drawing design drawings, organizing PPT reports, and presenting design proposals. For example, Interviewee 6 noted, “Students usually break down design strategies into specific tasks and steps by refining them.” Other interviewees also unanimously support this viewpoint. As emphasized by respondent 6, developing and implementing plans can help students maintain organizational, systematic, and goal-oriented problem-solving skills, thereby enhancing their problem-solving skills.

Themes 3: teamwork skills

The research results indicate that teamwork skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. This discovery emphasizes the importance of teamwork skills in interior design learning. This mainly includes communication and coordination and division of labor and collaboration, which are mentioned frequently in the interview documents. Therefore, it is necessary to discuss each function in detail to better understand and cultivate students' teamwork skills. As shown in Table 4 .

Category 1: communication and coordination

The research results indicate that communication and collaboration play crucial roles in cultivating students' teamwork abilities in a blended learning environment for interior design. Communication and collaboration refer to the ability of students to effectively share information, understand each other's perspectives, and work together to solve problems 36 . Specifically, team members need to understand each other's resource advantages integrate and share these resources to improve work efficiency and project quality. For example, Interviewee 7 noted, “In interior design, one member may be skilled in spatial planning, while another member may be skilled in color matching. Through communication and collaboration, team members can collectively utilize this expertise to improve work efficiency and project quality.” Other interviewees also unanimously believe that this viewpoint can promote students' teamwork skills, thereby promoting the development of their HOTS. As emphasized by the viewpoints of these interviewees, communication and collaboration enable team members to collectively solve problems and overcome challenges. Through effective communication, team members can exchange opinions and suggestions with each other, provide different solutions, and make joint decisions. Collaboration and cooperation among team members contribute to brainstorming and finding the best solution.

Category 2: division of labor and collaboration

The research results indicate that in the blended learning environment of interior design, the division of labor and collaboration play crucial roles in cultivating students' teamwork ability. The division of labor and collaboration refer to the ability of team members to assign different tasks and roles in a project based on their respective expertise and responsibilities and work together to complete the project 37 . For example, Interviewee 8 noted, “In an internal design project, some students are responsible for space planning, some students are responsible for color matching, and some students are responsible for rendering production.” Other interviewees also support this viewpoint. As emphasized by interviewee 8, the division of labor and collaboration help team members fully utilize their respective expertise and abilities, promote resource integration and complementarity, cultivate a spirit of teamwork, and enable team members to collaborate, support, and trust each other to achieve project goals together.

Themes 4: practical innovation skills

The research results indicate that practical innovation skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. This discovery emphasizes the importance of practical innovation skills in interior design learning. This mainly includes creative conception and design expression, as well as innovative application of materials and technology, which are often mentioned in interview documents. Therefore, it is necessary to discuss each function in detail to better understand and cultivate students' practical innovation skills. As shown in Table 5 .

Category 1: creative conception and design expression

The research results indicate that in the blended learning environment of interior design, creative ideation and design expression play crucial roles in cultivating students' practical and innovative skills. Creative ideation and design expression refer to the ability of students to break free from traditional thinking frameworks and try different design ideas and methods through creative ideation, which helps stimulate their creativity and cultivate their ability to think independently and solve problems. For example, interviewee 10 noted that "blended learning environments combine online and offline teaching modes, allowing students to acquire knowledge and skills more flexibly. Through learning and practice, students can master various expression tools and techniques, such as hand-drawn sketches, computer-aided design software, model making, etc., thereby more accurately conveying their design concepts." Other interviewees also expressed the importance of this viewpoint, emphasizing the importance of creative ideas and design expression in blended learning environments that cannot be ignored. As emphasized by interviewee 10, creative ideation and design expression in the blended learning environment of interior design can not only enhance students' creative thinking skills and problem-solving abilities but also strengthen their application skills in practical projects through diverse expression tools and techniques. The cultivation of these skills is crucial for students' success in their future careers.

Category 2: innovative application of materials and technology

Research findings indicate that the innovative application of materials and technology plays a crucial role in developing students' practical and creative skills within a blended learning environment for interior design. The innovative application of materials and technology refers to students' exploration and utilization of new materials and advanced technologies, enabling them to overcome the limitations of traditional design thinking and experiments with diverse design methods and approaches. This process not only stimulates their creativity but also significantly enhances their problem-solving skills. Specifically, the innovative application of materials and technology involves students gaining a deep understanding of the properties of new materials and their application methods in design, as well as becoming proficient in various advanced technological tools and equipment, such as 3D printing, virtual reality (VR), and augmented reality (AR). These skills enable students to more accurately realize their design concepts and effectively apply them in real-world projects.

For example, Interviewee 1 stated, "The blended learning environment combines online and offline teaching modes, allowing students to flexibly acquire the latest knowledge on materials and technology and apply these innovations in real projects." Other interviewees also emphasized the importance of this view. Therefore, the importance of the innovative application of materials and technology in a blended learning environment cannot be underestimated. As emphasized by interviewee 1, the innovative application of materials and technologies is crucial in the blended learning environment of interior design. This process not only enables students to flexibly acquire the latest materials and technical knowledge but also enables them to apply these innovations to practice in practical projects, thereby improving their practical abilities and professional ethics.

In summary, through research question 1 research, the dimensions of the HOTS assessment scale in blended learning for interior design include four main aspects: critical thinking skills, problem-solving skills, teamwork skills, and practical innovation skills. Based on the assessment scales developed by previous scholars in various dimensions, the researcher developed a HOTS assessment scale suitable for blended learning environments in interior design and collected feedback from interior design experts through interviews.

Development of the HOTS assessment scale

The above research results indicate that the dimensions of the HOTS scale mainly include critical thinking, problem-solving, teamwork skills and practical innovation skills. The dimensions of a scale represent the abstract characteristics and structure of the concept being measured. Since these dimensions are often abstract and difficult to measure directly, they need to be converted into several concrete indicators that can be directly observed or self-reported 38 . These concrete indicators, known as dimension items, operationalize the abstract dimensions, allowing for the measurement and evaluation of various aspects of the concept. This process transforms the abstract dimensions into specific, measurable components. The following content is based on the results of research question 1 to develop an advanced thinking skills assessment scale for mixed learning in interior design.

Dimension 1: critical thinking skills

The research results indicate that critical thinking skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. Critical thinking skills refer to the ability to analyze information objectively and make a reasoned judgment 39 . Scholars tend to emphasize this concept as a method of general skepticism, rational thinking, and self-reflection 7 , 40 . For example, Goodsett 26 suggested that it should be based on rational skepticism and careful thought about external matters as well as open self-reflection about internal thoughts and actions. Moreover, the California Critical Thinking Disposition Inventory (CCTDI) is widely used to measure critical thinking skills, including dimensions such as seeking truth, confidence, questioning and courage to seek truth, curiosity and openness, as well as analytical and systematic methods 41 . In addition, maturity means continuous adjustment and improvement of a person's cognitive system and learning activities through continuous awareness, reflection, and self-awareness 42 . Moreover, Nguyen 43 confirmed that critical thinking and cognitive maturity can be achieved through these activities, emphasizing that critical thinking includes cognitive skills such as analysis, synthesis, and evaluation, as well as emotional tendencies such as curiosity and openness.

In addition, in a blended learning environment for interior design, critical thinking skills help students better understand, evaluate, and apply design knowledge and skills, cultivating independent thinking and innovation abilities 44 . If students lack these skills, they may accept superficial information and solutions without sufficient thinking and evaluation, resulting in the overlooking of important details or the selection of inappropriate solutions in the design process. Therefore, for the measurement of critical thinking skills, the focus should be on cognitive skills such as analysis, synthesis, and evaluation, as well as curiosity and open mindedness. The specific items for critical thinking skills are shown in Table 6 .

Dimension 2: problem-solving skills

Problem-solving skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. Problem-solving skills involve the ability to analyze and solve problems by understanding them, identifying their root causes, and developing appropriate solutions 45 . According to the 5E-based STEM education approach, problem-solving skills encompass the following abilities: problem identification and definition, formulation of problem-solving strategies, problem representation, resource allocation, and monitoring and evaluation of solution effectiveness 7 , 46 . Moreover, D'zurilla and Nezu 47 and Tan 48 indicated that attitudes, beliefs, and knowledge skills during problem solving, as well as the quality of proposed solutions and observable outcomes, are demonstrated. In addition, D'Zurilla and Nezu devised the Social Problem-Solving Inventory (SPSI), which comprises seven subscales: cognitive response, emotional response, behavioral response, problem identification, generation of alternative solutions, decision-making, and solution implementation. Based on these research results, the problem-solving skills dimension questions designed in this study are shown in Table 7 .

Dimension 3: teamwork skills

The research results indicate that teamwork skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. Teamwork skills refer to the ability to effectively collaborate, coordinate, and communicate with others in a team environment 49 . For example, the Teamwork Skills Assessment Tool (TWKSAT) developed by Stevens and Campion 50 identifies five core dimensions of teamwork: conflict management; collaborative problem-solving; communication; goal setting; performance management; decision-making; and task coordination. The design of this tool highlights the essential skills in teamwork and provides a structured approach for evaluating these skills. In addition, he indicated that successful teams need to have a range of skills for problem solving, including situational control, conflict management, decision-making and coordination, monitoring and feedback, and an open mindset. These skills help team members effectively address complex challenges and demonstrate the team’s collaboration and flexibility. Therefore, the assessment of learners' teamwork skills needs to cover the above aspects. As shown in Table 8 .

Dimension 4: practice innovative skills

The research results indicate that practical innovation skills constitute a key core category in blended learning environments for interior design, which is crucial for cultivating students' HOTS. The practice of innovative skills encompasses the utilization of creative cognitive processes and problem-solving strategies to facilitate the generation of original ideas, solutions, and approaches 51 . This practice places significant emphasis on two critical aspects: creative conception and design expression, as well as the innovative application of materials and technology. Tang et al. 52 indicated that creative conception and design expression involve the generation and articulation of imaginative and inventive ideas within a given context. With the introduction of concepts such as 21st-century learning skills, the "5C" competency framework, and core student competencies, blended learning has emerged as the goal and direction of educational reform. It aims to promote the development of students' HOTS, equipping them with the essential qualities and key abilities needed for lifelong development and societal advancement. Blended learning not only emphasizes the mastery of core learning content but also requires students to develop critical thinking, complex problem-solving, creative thinking, and practical innovation skills. To adapt to the changes and developments in the blended learning environment, this study designed 13 preliminary test items based on 21st-century learning skills, the "5C" competency framework, core student competencies, and the TTCT assessment scale developed by Torrance 53 . These items aim to assess students' practice of innovative skills within a blended learning environment, as shown in Table 9 .

The researchers' results indicate that the consensus among the interviewed expert participants is that the structural integrity of the scale is satisfactory and does not require modification. However, certain measurement items have been identified as problematic and require revision. The primary recommendations are as follows: Within the domain of problem-solving skills, the item "I usually conduct classroom and online learning with questions and clear goals" was deemed biased because of its emphasis on the "online" environment. Consequently, the evaluation panel advised splitting this item into two separate components: (1) "I am adept at frequently adjusting and reversing a negative team atmosphere" and (2) "I consistently engage in praising and encouraging others, fostering harmonious relationships. “The assessment process requires revisions and adjustments to specific projects, forming a pilot test scale consisting of 66 observable results from the original 65 items. In addition, there were other suggestions about linguistic formulation and phraseology, which are not expounded upon herein.

Verify the effectiveness of the HOTS assessment scale

The research results indicate that there are significant differences in the average scores of the four dimensions of the HOTS, including critical thinking skills (A1–A24 items), problem-solving skills (B1–B13 items), teamwork skills (C1–C16 items), and practical innovation skills (D1–D13 items). Moreover, this also suggests that each item has discriminative power. Specifically, this will be explained through the following aspects.

Project analysis based on the CR value

The critical ratio (CR) method, which uses the CR value (decision value) to remove measurement items with poor discrimination, is the most used method in project analysis. The specific process involves the use of the CR value (critical value) to identify and remove such items. First, the modified pilot test scale data are aggregated and sorted. Individuals representing the top and bottom 27% of the distribution were subsequently selected, constituting 66 respondents in each group. The high-score group comprises individuals with a total score of 127 or above (including 127), whereas the low-score group comprises individuals with a total score of 99 or below (including 99). Finally, an independent sample t test was conducted to determine the significant differences in the mean scores for each item between the high-score and low-score groups. The statistical results are presented in Table 10 .

The above table shows that independent sample t tests were conducted for all the items; their t values were greater than 3, and their p values were less than 0.001, indicating that the difference between the highest and lowest 27% of the samples was significant and that each item had discriminative power.

In summary, based on previous research and relevant theories, the HOTS scale for interior design was revised. This revision process involved interviews with interior design experts, teachers, and students, followed by item examination and homogeneity testing via the critical ratio (CR) method. The results revealed significant correlations ( p  < 0.01) between all the items and the total score, with correlation coefficients (R) above 0.4. Therefore, the scale exhibits good accuracy and internal consistency in capturing measured HOTS. These findings provide a reliable foundation for further research and practical applications.

Pilot study exploratory factor analysis

This study used SPSS (version 28) to conduct the KMO and Bartlett tests on the scale. The total HOTS test scale as well as the KMO and Bartlett sphericities were first calculated for the four subscales to ensure that the sample data were suitable for factor analysis 7 . The overall KMO value is 0.946, indicating that the data are highly suitable for factor analysis. Additionally, Bartlett's test of sphericity was significant, further supporting the appropriateness of conducting factor analysis ( p  < 0.05). All the values are above 0.7, indicating that the data for these subscales are also suitable for factor analysis. According to Javadi et al. 54 , these results suggest the presence of shared factors among the items within the subscales, as shown in Table 11 .

For each subscale, exploratory factor analysis was conducted to extract factors with eigenvalues greater than 1 while eliminating items with communalities less than 0.30, loadings less than 0.50, and items that cross multiple (more than one) common factors 55 , 56 . Additionally, items that were inconsistent with the assumed structure of the measure were identified and eliminated to ensure the best structural validity. These principles were applied to the factor analysis of each subscale, ensuring that the extracted factor structure and observed items are consistent with the hypothesized measurement structure and analysis results, as shown in the table 55 , 58 . In the exploratory factor analysis (EFA), the latent variables were effectively interpreted and demonstrated a significant response, with cumulative explained variances of the common factors exceeding 60%. This finding confirms the alignment between the scale structure, comprising the remaining items, and the initial theoretical framework proposed in this study. Additionally, the items were systematically reorganized to construct the final questionnaire. Consequently, items A1 to A24 were associated with the critical thinking skills dimension, items B25 to B37 were linked to problem-solving skills, items C38 to C53 were indicative of teamwork skills, and items D54 to D66 were reflective of practical innovation skills. As shown in Table 12 below.

In addition, the criterion for extracting principal components in factor analysis is typically based on eigenvalues, with values greater than 1 indicating greater explanatory power than individual variables. The variance contribution ratio reflects the proportion of variance explained by each principal component relative to the total variance and signifies the ability of the principal component to capture comprehensive information. The cumulative variance contribution ratio measures the accumulated proportion of variance explained by the selected principal components, aiding in determining the optimal number of components to retain while minimizing information loss. The above table shows that four principal components can be extracted from the data, and their cumulative variance contribution rate reaches 59.748%.

However, from the scree plot (as shown in Fig.  1 ), the slope flattens starting from the fifth factor, indicating that no distinct factors can be extracted beyond that point. Therefore, retaining four factors seems more appropriate. The factor loading matrix is the core of factor analysis, and the values in the matrix represent the factor loading of each item on the common factors. Larger values indicate a stronger correlation between the item variable and the common factor. For ease of analysis, this study used the maximum variance method to rotate the initial factor loading matrix, redistributing the relationships between the factors and original variables and making the correlation coefficients range from 0 to 1, which facilitates interpretation. In this study, factor loadings with absolute values less than 0.4 were filtered out. According to the analysis results, the items of the HOTS assessment scale can be divided into four dimensions, which is consistent with theoretical expectations.

Gravel plot of factors.

Through the pretest of the scale and selection of measurement items, 66 measurement items were ultimately determined. On this basis, a formal scale for assessing HOTS in a blended learning environment was developed, and the reliability and validity of the scale were tested to ultimately confirm its usability.

Confirmatory factor analysis of final testing

Final test employed that AMOS (version 26.0), a confirmatory factor analysis (CFA) was conducted on the retested sample data to validate the stability of the HOTS structural model obtained through exploratory factor analysis. This analysis aimed to assess the fit between the measurement results and the actual data, confirming the robustness of the derived HOTS structure and its alignment with the empirical data. The relevant model was constructed based on the factor structure of each component obtained through EFA and the observed variables, as shown in the diagram. The model fit indices are presented in Fig.  2 (among them, A represents critical thinking skills, B represents problem-solving skills, C represents teamwork skills, and D represents practical innovation skills). The models strongly support the "4-dimensional" structure of the HOTS, which includes four first-order factors: critical thinking skills, problem-solving skills, teamwork skills, and practical innovation skills. Critical thinking skills play a pivotal role in the blended learning environment of interior design, connecting problem-solving skills, teamwork skills, and innovative practices. These four dimensions form the assessment structure of HOTS, with critical thinking skills serving as the core element, inspiring individuals to assess problems and propose innovative solutions. By providing appropriate learning resources, diverse learning activities, and learning tasks, as well as designing items for assessment scales, it is possible to delve into the measurement and development of HOTS in the field of interior design, providing guidance for educational and organizational practices. This comprehensive approach to learning and assessment helps cultivate students' HOTS and lays a solid foundation for their comprehensive abilities in the field of interior design. Thus, the CFA structural models provide strong support for the initial hypothesis of the proposed HOTS assessment structure in this study. As shown in Fig.  2 .

Confirmatory factor analysis based on 4 dimensions. *A represents the dimension of critical thinking. B represents the dimension of problem-solving skills. C represents the dimension of teamwork skills. D represents the dimension of practical innovation skills.

Additionally, χ2. The fitting values of RMSEA and SRMR are both below the threshold, whereas the fitting values of the other indicators are all above the threshold, indicating that the model fits well. As shown in Table 13 .

Reliability and validity analysis

The reliability and validity of the scale need to be assessed after the model fit has been determined through validation factor analysis 57 . Based on the findings of Marsh et al. 57 , the following conclusions can be drawn. In terms of hierarchical and correlational model fit, the standardized factor loadings of each item range from 0.700 to 0.802, all of which are greater than or equal to 0.7. This indicates a strong correspondence between the observed items and each latent variable. Furthermore, the Cronbach's α coefficients, which are used to assess the internal consistency or reliability of the scale, ranged from 0.948 to 0.966 for each dimension, indicating a high level of data reliability and internal consistency. The composite reliabilities ranged from 0.948 to 0.967, exceeding the threshold of 0.6 and demonstrating a substantial level of consistency (as shown in Table 14 ).

Additionally, the diagonal bold font represents the square root of the AVE for each dimension. All the dimensions have average variance extracted (AVE) values ranging from 0.551 to 0.589, all of which are greater than 0.5, indicating that the latent variables have strong explanatory power for their corresponding items. These results suggest that the scale structure constructed in this study is reliable and effective. Furthermore, according to the results presented in Table 15 , the square roots of the AVE values for each dimension are greater than the absolute values of the correlations with other dimensions, indicating discriminant validity of the data. Therefore, these four subscales demonstrate good convergent and discriminant validity, indicating that they are both interrelated and independent. This implies that they can effectively capture the content required to complete the HOTS test scale.

Discussion and conclusion

The assessment scale for HOTS in interior design blended learning encompasses four dimensions: critical thinking skills, problem-solving skills, teamwork skills, and practical innovation skills. The selection of these dimensions is based on the characteristics and requirements of the interior design discipline, which aims to comprehensively evaluate students' HOTS demonstrated in blended learning environments to better cultivate their ability to successfully address complex design projects in practice. Notably, multiple studies have shown that HOTSs include critical thinking, problem-solving skills, creative thinking, and decision-making skills, which are considered crucial in various fields, such as education, business, and engineering 20 , 59 , 60 , 61 . Compared with prior studies, these dimensions largely mirror previous research outcomes, with notable distinctions in the emphasis on teamwork skills and practical innovation skills 62 , 63 . Teamwork skills underscore the critical importance of collaboration in contemporary design endeavors, particularly within the realm of interior design 64 . Effective communication and coordination among team members are imperative for achieving collective design objectives.

Moreover, practical innovation skills aim to increase students' capacity for creatively applying theoretical knowledge in practical design settings. Innovation serves as a key driver of advancement in interior design, necessitating students to possess innovative acumen and adaptability to evolving design trends for industry success. Evaluating practical innovation skills aims to motivate students toward innovative thinking, exploration of novel concepts, and development of unique design solutions, which is consistent with the dynamic and evolving nature of the interior design sector. Prior research suggests a close interplay between critical thinking, problem-solving abilities, teamwork competencies, and creative thinking, with teamwork skills acting as a regulatory factor for critical and creative thought processes 7 , 65 . This interconnected nature of HOTS provides theoretical support for the construction and validation of a holistic assessment framework for HOTS.

After the examination by interior design expert members, one item needed to be split into two items. The results of the CR (construct validity) analysis of the scale items indicate that independent sample t tests were subsequently conducted on all the items. The t values were greater than 3, with p values less than 0.001, indicating significant differences between the top and bottom 27% of the samples and demonstrating the discriminant validity of each item. This discovery highlights the diversity and effectiveness of the scale's internal items, revealing the discriminatory power of the scale in assessing the study subjects. The high t values and significant p values reflect the substantiality of the internal items in distinguishing between different sample groups, further confirming the efficacy of these items in evaluating the target characteristics. These results provide a robust basis for further refinement and optimization of the scale and offer guidance for future research, emphasizing the importance of scale design in research and providing strong support for data interpretation and analysis.

This process involves evaluating measurement scales through EFA, and it was found that the explanatory variance of each subscale reached 59.748%, and the CR, AVE, Cronbach's alpha, and Pearson correlation coefficient values of the total scale and subscales were in a better state, which strongly demonstrates the structure, discrimination, and convergence effectiveness of the scale 57 .

The scale structure and items of this study are reliable and effective, which means that students in the field of interior design can use them to test their HOTS level and assess their qualities and abilities. In addition, scholars can use this scale to explore the relationships between students' HOTS and external factors, personal personalities, etc., to determine different methods and strategies for developing and improving HOTS.

Limitations and future research

The developed mixed learning HOTS assessment scale for interior design also has certain limitations that need to be addressed in future research. The first issue is that, owing to the requirement of practical innovation skills, students need to have certain practical experience and innovative abilities. First-grade students usually have not yet had sufficient opportunities for learning and practical experience, so it may not be possible to evaluate their abilities effectively in this dimension. Therefore, when this scale is used for assessment, it is necessary to consider students' grade level and learning experience to ensure the applicability and accuracy of the assessment tool. For first-grade students, it may be necessary to use other assessment tools that are suitable for their developmental stage and learning experience to evaluate other aspects of their HOTS 7 . Future research should focus on expanding the scope of this dimension to ensure greater applicability.

The second issue is that the sample comes from ordinary private undergraduate universities in central China and does not come from national public universities or key universities. Therefore, there may be regional characteristics in the obtained data. These findings suggest that the improved model should be validated with a wider range of regional origins, a more comprehensive school hierarchy, and a larger sample size. The thirdly issue is the findings of this study are derived from self-reported data collected from participants through surveys. However, it is important to note that the literature suggests caution in heavily relying on such self-reported data, as perception does not always equate to actions 66 . In addition, future research can draw on this scale to evaluate the HOTS of interior design students, explore the factors that affect their development, determine their training and improvement paths, and cultivate skilled talent for the twenty-first century.

This study adopts a mixed method research approach, combining qualitative and quantitative methods to achieve a comprehensive understanding of the phenomenon 67 . By integrating qualitative and quantitative research methods, mixed methods research provides a comprehensive and detailed exploration of research questions, using multiple data sources and analytical methods to obtain accurate and meaningful answers 68 . To increase the quality of the research, the entire study followed the guidelines for scale development procedures outlined by Professor Li after the data were obtained. As shown in Fig.  3

Scale development program.

Basis of theory

This study is guided by educational objectives such as 21st-century learning skills, the "5C" competency framework, and students' core abilities 4 . The construction process of the scale is based on theoretical foundations, including Bloom's taxonomy. Drawing from existing research, such as the CCTDI 41 , SPSI 69 , and TWKSAT scales, the dimensions and preliminary items of the scale were developed. Additionally, to enhance the validity and reliability of the scale, dimensions related to HOTS in interior design were obtained through semi-structured interviews, and the preliminary project adapted or directly cited existing research results. The preliminary items were primarily adapted or directly referenced from existing research findings. Based on existing research, such as the CCTDI, SPSI, TWKSAT, and twenty-first century skills frameworks, this study takes "critical thinking skills, problem-solving skills, teamwork skills, and practical innovative skills" as the four basic dimensions of the scale.

Participants and procedures

This study is based on previous research and develops a HOTS assessment scale to measure the thinking levels of interior design students in blended learning. By investigating the challenges and opportunities students encounter in blended learning environments and exploring the complexity and diversity of their HOTS, this study aims to obtain comprehensive insights. For research question 1, via the purposive sampling method, 10 interior design experts are selected to investigate the dimensions and evaluation indicators of HOTS in blended learning of interior design. The researcher employed a semi structured interview method, and a random sampling technique was used to select 10 senior experts and teachers in the field of interior design, holding the rank of associate professor or above. This included 5 males and 5 females. As shown in Table 16 .

For research question 2 and 3, the research was conducted at an undergraduate university in China, in the field of interior design and within a blended learning environment. In addition, a statement confirms that all experimental plans have been approved by the authorized committee of Zhengzhou University of Finance and Economics. In the process of practice, the methods used were all in accordance with relevant guidelines and regulations, and informed consent was obtained from all participants. The Interior Design Blended Learning HOTS assessment scale was developed based on sample data from 350 students who underwent one pre-test and retest. The participants in the study consisted of second-, third-, and fourth-grade students who had participated in at least one blended learning course. The sample sizes were 115, 118, and 117 for the respective grade levels, totaling 350 individuals. Among the participants, there were 218 male students and 132 female students, all of whom were within the age range of 19–22 years. Through purposeful sampling, this study ensured the involvement of relevant participants and focused on a specific university environment with diverse demographic characteristics and rich educational resources.

This approach enhances the reliability and generalizability of the research and contributes to a deeper understanding of the research question (as shown in Table 17 ).

Instruments

The tools used in this study include semi structured interview guidelines and the HOTS assessment scale developed by the researchers. For research question 1, the semi structured interview guidelines were reviewed by interior design experts to ensure the accuracy and appropriateness of their content and questions. In addition, for research question 2 and 3, the HOTS assessment scale developed by the researchers will be checked via the consistency ratio (CR) method to assess the consistency and reliability of the scale items and validate their effectiveness.

Data analysis

For research question 1, the researcher will utilize the NVivo version 14 software tool to conduct thematic analysis on the data obtained through semi structured interviews. Thematic analysis is a commonly used qualitative research method that aims to identify and categorize themes, concepts, and perspectives that emerge within a dataset 70 . By employing NVivo software, researchers can effectively organize and manage large amounts of textual data and extract themes and patterns from them.

For research question 2, the critical ratio (CR) method was employed to conduct item analysis and homogeneity testing on the items of the pilot test questionnaire. The CR method allows for the assessment of each item's contribution to the total score and the evaluation of the interrelationships among the items within the questionnaire. These analytical techniques served to facilitate the evaluation and validation of the scale's reliability and validity.

For research question 3, this study used SPSS (version 26), in which confirmatory factor analysis (CFA) was conducted on the confirmatory sample data via maximum likelihood estimation. The purpose of this analysis was to verify whether the hypothesized factor structure model of the questionnaire aligned with the actual survey data. Finally, several indices, including composite reliability (CR), average variance extracted (CR), average variance extracted (AVE), Cronbach's alpha coefficient, and the Pearson correlation coefficient, were computed to assess the reliability and validity of the developed scale and assess its reliability and validity.

In addition, exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) are commonly utilized techniques in questionnaire development and adaptation research 31 , 70 . The statistical software packages SPSS and AMOS are frequently employed for implementing these analytical techniques 71 , 72 , 73 . CFA is a data-driven approach to factor generation that does not require a predetermined number of factors or specific relationships with observed variables. Its focus lies in the numerical characteristics of the data. Therefore, prior to conducting CFA, survey questionnaires are typically constructed through EFA to reveal the underlying structure and relationships between observed variables and the latent structure.

In contrast, CFA tests the hypothesized model structure under specific theoretical assumptions or structural hypotheses, including the interrelationships among factors and the known number of factors. Its purpose is to validate the hypothesized model structure. Thus, the initial validity of the questionnaire structure, established through EFA, necessitates further confirmation through CFA 57 , 70 . Additionally, a sample size of at least 200 is recommended for conducting the validation factor analysis. In this study, confirmatory factor analysis was performed on a sample size of 317.

Data availability

All data generated or analyzed during this study are included in this published article. All the experimental protocols were approved by the Zhengzhou College of Finance and Economics licensing committee.

Hariadi, B. et al. Higher order thinking skills based learning outcomes improvement with blended web mobile learning Model. Int. J. Instr. 15 (2), 565–578 (2022).

Google Scholar  

Sagala, P. N. & Andriani, A. Development of higher-order thinking skills (HOTS) questions of probability theory subject based on bloom’s taxonomy. J. Phys. Conf. Ser. https://doi.org/10.1088/1742-6596/1188/1/012025 (2019).

Article   Google Scholar  

Yudha, R. P. Higher order thinking skills (HOTS) test instrument: Validity and reliability analysis with the rasch model. Eduma Math. Educ. Learn. Teach. https://doi.org/10.24235/eduma.v12i1.9468 (2023).

Leach, S. M., Immekus, J. C., French, B. F. & Hand, B. The factorial validity of the Cornell critical thinking tests: A multi-analytic approach. Think. Skills Creat. https://doi.org/10.1016/j.tsc.2020.100676 (2020).

Noroozi, O., Dehghanzadeh, H. & Talaee, E. A systematic review on the impacts of game-based learning on argumentation skills. Entertain. Comput. https://doi.org/10.1016/j.entcom.2020.100369 (2020).

Supena, I., Darmuki, A. & Hariyadi, A. The influence of 4C (constructive, critical, creativity, collaborative) learning model on students’ learning outcomes. Int. J. Instr. 14 (3), 873–892. https://doi.org/10.29333/iji.2021.14351a (2021).

Zhou, Y., Gan, L., Chen, J., Wijaya, T. T. & Li, Y. Development and validation of a higher-order thinking skills assessment scale for pre-service teachers. Think. Skills Creat. https://doi.org/10.1016/j.tsc.2023.101272 (2023).

Musfy, K., Sosa, M. & Ahmad, L. Interior design teaching methodology during the global COVID-19 pandemic. Interiority 3 (2), 163–184. https://doi.org/10.7454/in.v3i2.100 (2020).

Yong, S. D., Kusumarini, Y. & Tedjokoesoemo, P. E. D. Interior design students’ perception for AutoCAD SketchUp and Rhinoceros software usability. IOP Conf. Ser. Earth Environ. Sci. https://doi.org/10.1088/1755-1315/490/1/012015 (2020).

Anthony, B. et al. Blended learning adoption and implementation in higher education: A theoretical and systematic review. Technol. Knowl. Learn. 27 (2), 531–578. https://doi.org/10.1007/s10758-020-09477-z (2020).

Castro, R. Blended learning in higher education: Trends and capabilities. Edu. Inf. Technol. 24 (4), 2523–2546. https://doi.org/10.1007/s10639-019-09886-3 (2019).

Alismaiel, O. Develop a new model to measure the blended learning environments through students’ cognitive presence and critical thinking skills. Int. J. Emerg. Technol. Learn. 17 (12), 150–169. https://doi.org/10.3991/ijet.v17i12.30141 (2022).

Gao, Y. Blended teaching strategies for art design major courses in colleges. Int. J. Emerg. Technol. Learn. https://doi.org/10.3991/ijet.v15i24.19033 (2020).

Banihashem, S. K., Kerman, N. T., Noroozi, O., Moon, J. & Drachsler, H. Feedback sources in essay writing: peer-generated or AI-generated feedback?. Int. J. Edu. Technol. Higher Edu. 21 (1), 23 (2024).

Ji, J. A Design on Blended Learning to Improve College English Students’ Higher-Order Thinking Skills. https://doi.org/10.18282/l-e.v10i4.2553 (2021).

Noroozi, O. The role of students’ epistemic beliefs for their argumentation performance in higher education. Innov. Edu. Teach. Int. 60 (4), 501–512 (2023).

Valero Haro, A., Noroozi, O., Biemans, H. & Mulder, M. First- and second-order scaffolding of argumentation competence and domain-specific knowledge acquisition: A systematic review. Technol. Pedag. Edu. 28 (3), 329–345. https://doi.org/10.1080/1475939x.2019.1612772 (2019).

Narasuman, S. & Wilson, D. M. Investigating teachers’ implementation and strategies on higher order thinking skills in school based assessment instruments. Asian J. Univ. Edu. https://doi.org/10.24191/ajue.v16i1.8991 (2020).

Valero Haro, A., Noroozi, O., Biemans, H. & Mulder, M. Argumentation competence: Students’ argumentation knowledge, behavior and attitude and their relationships with domain-specific knowledge acquisition. J. Constr. Psychol. 35 (1), 123–145 (2022).

Johansson, E. The Assessment of Higher-order Thinking Skills in Online EFL Courses: A Quantitative Content Analysis (2020).

Noroozi, O., Kirschner, P. A., Biemans, H. J. A. & Mulder, M. Promoting argumentation competence: Extending from first- to second-order scaffolding through adaptive fading. Educ. Psychol. Rev. 30 (1), 153–176. https://doi.org/10.1007/s10648-017-9400-z (2017).

Noroozi, O., Weinberger, A., Biemans, H. J. A., Mulder, M. & Chizari, M. Facilitating argumentative knowledge construction through a transactive discussion script in CSCL. Comput. Educ. 61 , 59–76. https://doi.org/10.1016/j.compedu.2012.08.013 (2013).

Noroozi, O., Weinberger, A., Biemans, H. J. A., Mulder, M. & Chizari, M. Argumentation-based computer supported collaborative learning (ABCSCL): A synthesis of 15 years of research. Educ. Res. Rev. 7 (2), 79–106. https://doi.org/10.1016/j.edurev.2011.11.006 (2012).

Setiawan, Baiq Niswatul Khair, Ratnadi Ratnadi, Mansur Hakim, & Istiningsih, S. Developing HOTS-Based Assessment Instrument for Primary Schools (2019).

Suparman, S., Juandi, D., & Tamur, M. Does Problem-Based Learning Enhance Students’ Higher Order Thinking Skills in Mathematics Learning? A Systematic Review and Meta-Analysis 2021 4th International Conference on Big Data and Education (2021).

Goodsett, M. Best practices for teaching and assessing critical thinking in information literacy online learning objects. J. Acad. Lib. https://doi.org/10.1016/j.acalib.2020.102163 (2020).

Putra, I. N. A. J., Budiarta, L. G. R., & Adnyayanti, N. L. P. E. Developing Authentic Assessment Rubric Based on HOTS Learning Activities for EFL Teachers. In Proceedings of the 2nd International Conference on Languages and Arts across Cultures (ICLAAC 2022) (pp. 155–164). https://doi.org/10.2991/978-2-494069-29-9_17 .

Bervell, B., Umar, I. N., Kumar, J. A., Asante Somuah, B. & Arkorful, V. Blended learning acceptance scale (BLAS) in distance higher education: Toward an initial development and validation. SAGE Open https://doi.org/10.1177/21582440211040073 (2021).

Byrne, D. A worked example of Braun and Clarke’s approach to reflexive thematic analysis. Qual. Quant. 56 (3), 1391–1412 (2022).

Xu, W. & Zammit, K. Applying thematic analysis to education: A hybrid approach to interpreting data in practitioner research. Int. J. Qual. Methods 19 , 1609406920918810 (2020).

Braun, V. & Clarke, V. Conceptual and design thinking for thematic analysis. Qual. Psychol. 9 (1), 3 (2022).

Creswell, A., Shanahan, M., & Higgins, I. Selection-inference: Exploiting large language models for interpretable logical reasoning. arXiv:2205.09712 (2022).

Baron, J. Thinking and Deciding 155–156 (Cambridge University Press, 2023).

Book   Google Scholar  

Silver, N., Kaplan, M., LaVaque-Manty, D. & Meizlish, D. Using Reflection and Metacognition to Improve Student Learning: Across the Disciplines, Across the Academy (Taylor & Francis, 2023).

Oksuz, K., Cam, B. C., Kalkan, S. & Akbas, E. Imbalance problems in object detection: A review. IEEE Trans. Pattern Anal. Mach. Intell. 43 (10), 3388–3415 (2020).

Saputra, M. D., Joyoatmojo, S., Wardani, D. K. & Sangka, K. B. Developing critical-thinking skills through the collaboration of jigsaw model with problem-based learning model. Int. J. Instr. 12 (1), 1077–1094 (2019).

Imam, H. & Zaheer, M. K. Shared leadership and project success: The roles of knowledge sharing, cohesion and trust in the team. Int. J. Project Manag. 39 (5), 463–473 (2021).

DeCastellarnau, A. A classification of response scale characteristics that affect data quality: A literature review. Qual. Quant. 52 (4), 1523–1559 (2018).

Article   PubMed   Google Scholar  

Haber, J. Critical Thinking 145–146 (MIT Press, 2020).

Hanscomb, S. Critical Thinking: The Basics 180–181 (Routledge, 2023).

Sulaiman, W. S. W., Rahman, W. R. A. & Dzulkifli, M. A. Examining the construct validity of the adapted California critical thinking dispositions (CCTDI) among university students in Malaysia. Proc. Social Behav. Sci. 7 , 282–288 (2010).

Jaakkola, N. et al. Becoming self-aware—How do self-awareness and transformative learning fit in the sustainability competency discourse?. Front. Educ. https://doi.org/10.3389/feduc.2022.855583 (2022).

Nguyen, T. T. B. Critical thinking: What it means in a Vietnamese tertiary EFL context. English For. Language Int. J. 2 (3), 4–23 (2022).

Henriksen, D., Gretter, S. & Richardson, C. Design thinking and the practicing teacher: Addressing problems of practice in teacher education. Teach. Educ. 31 (2), 209–229 (2020).

Okes, D. Root cause analysis: The core of problem solving and corrective action 179–180 (Quality Press, 2019).

Eroğlu, S. & Bektaş, O. The effect of 5E-based STEM education on academic achievement, scientific creativity, and views on the nature of science. Learn. Individual Differ. 98 , 102181 (2022).

Dzurilla, T. J. & Nezu, A. M. Development and preliminary evaluation of the social problem-solving inventory. Psychol. Assess. J. Consult. Clin. Psychol. 2 (2), 156 (1990).

Tan, O.-S. Problem-based learning innovation: Using problems to power learning in the 21st century. Gale Cengage Learning (2021).

Driskell, J. E., Salas, E. & Driskell, T. Foundations of teamwork and collaboration. Am. Psychol. 73 (4), 334 (2018).

Lower, L. M., Newman, T. J. & Anderson-Butcher, D. Validity and reliability of the teamwork scale for youth. Res. Social Work Pract. 27 (6), 716–725 (2017).

Landa, R. Advertising by design: generating and designing creative ideas across media (Wiley, 2021).

Tang, T., Vezzani, V. & Eriksson, V. Developing critical thinking, collective creativity skills and problem solving through playful design jams. Think. Skills Creat. 37 , 100696 (2020).

Torrance, E. P. Torrance tests of creative thinking. Educational and psychological measurement (1966).

Javadi, M. H., Khoshnami, M. S., Noruzi, S. & Rahmani, R. Health anxiety and social health among health care workers and health volunteers exposed to coronavirus disease in Iran: A structural equation modeling. J. Affect. Disord. Rep. https://doi.org/10.1016/j.jadr.2022.100321 (2022).

Article   PubMed   PubMed Central   Google Scholar  

Hu, L. & Bentler, P. M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Equ. Model. Multidiscip. J. 6 (1), 1–55. https://doi.org/10.1080/10705519909540118 (1999).

Matsunaga, M. Item parceling in structural equation modeling: A primer. Commun. Methods Measures 2 (4), 260–293. https://doi.org/10.1080/19312450802458935 (2008).

Marsh, H. W., Morin, A. J., Parker, P. D. & Kaur, G. Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis. Ann. Rev. Clin. Psychol. 10 (1), 85–110 (2014).

Song, Y., Lee, Y. & Lee, J. Mediating effects of self-directed learning on the relationship between critical thinking and problem-solving in student nurses attending online classes: A cross-sectional descriptive study. Nurse Educ. Today https://doi.org/10.1016/j.nedt.2021.105227 (2022).

Chu, S. K. W., Reynolds, R. B., Tavares, N. J., Notari, M., & Lee, C. W. Y. 21st century skills development through inquiry-based learning from theory to practice . Springer (2021).

Eliyasni, R., Kenedi, A. K. & Sayer, I. M. Blended learning and project based learning: the method to improve students’ higher order thinking skill (HOTS). Jurnal Iqra’: Kajian Ilmu Pendidikan 4 (2), 231–248 (2019).

Yusuf, P. & Istiyono,. Blended learning: Its effect towards higher order thinking skills (HOTS). J. Phys. Conf. Ser. https://doi.org/10.1088/1742-6596/1832/1/012039 (2021).

Byron, K., Keem, S., Darden, T., Shalley, C. E. & Zhou, J. Building blocks of idea generation and implementation in teams: A meta-analysis of team design and team creativity and innovation. Personn. Psychol. 76 (1), 249–278 (2023).

Walid, A., Sajidan, S., Ramli, M. & Kusumah, R. G. T. Construction of the assessment concept to measure students’ high order thinking skills. J. Edu. Gift. Young Sci. 7 (2), 237–251 (2019).

Alawad, A. Evaluating online learning practice in the interior design studio. Int. J. Art Des. Edu. 40 (3), 526–542. https://doi.org/10.1111/jade.12365 (2021).

Awuor, N. O., Weng, C. & Militar, R. Teamwork competency and satisfaction in online group project-based engineering course: The cross-level moderating effect of collective efficacy and flipped instruction. Comput. Educ. 176 , 104357 (2022).

Noroozi, O., Alqassab, M., Taghizadeh Kerman, N., Banihashem, S. K. & Panadero, E. Does perception mean learning? Insights from an online peer feedback setting. Assess. Eval. Higher Edu. https://doi.org/10.1080/02602938.2024.2345669 (2024).

Creswell, J. W. A concise introduction to mixed methods research. SAGE publications124–125 (2021) .

Tashakkori, A., Johnson, R. B., & Teddlie, C. Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Sage Publications 180–181(2020).

Jiang, X., Lyons, M. D. & Huebner, E. S. An examination of the reciprocal relations between life satisfaction and social problem solving in early adolescents. J. Adolescence 53 (1), 141–151. https://doi.org/10.1016/j.adolescence.2016.09.004 (2016).

Orcan, F. Exploratory and confirmatory factor analysis: Which one to use first. Egitimde ve Psikolojide Olçme ve Degerlendirme Dergisi https://doi.org/10.21031/epod.394323 (2018).

Asparouhov, T. & Muthén, B. Exploratory structural equation modeling. Struct. Eq. Model. Multidiscip. J. 16 (3), 397–438 (2009).

Article   MathSciNet   Google Scholar  

Finch, H., French, B. F., & Immekus, J. C. Applied psychometrics using spss and amos. IAP (2016).

Marsh, H. W., Guo, J., Dicke, T., Parker, P. D. & Craven, R. G. Confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), and Set-ESEM: Optimal balance between goodness of fit and parsimony. Multivar. Behav. Res. 55 (1), 102–119. https://doi.org/10.1080/00273171.2019.1602503 (2020).

Download references

Acknowledgements

Thanks to the editorial team and reviewers of Scientific Reports for their valuable comments.

Author information

Authors and affiliations.

Faculty of Education, SEGI University, 47810 Petaling Jaya, Selangor, Malaysia

Department of Art and Design, Zhengzhou College of Finance and Economics, Zhengzhou, 450000, Henan, China

Xiaolei Fan

Faculty of Humanities and Arts, Macau University of Science and Technology, Avenida Wai Long, 999078, Taipa, Macao, Special Administrative Region of China

Lingchao Meng

You can also search for this author in PubMed   Google Scholar

Contributions

D.L. Conceptualized a text experiment, and wrote the main manuscript text. D.L. and X.F. conducted experiments, D.L., X.F. and L.M. analyzed the results. L.M. contributed to the conceptualization, methodology and editing, and critically reviewed the manuscript. All authors have reviewed the manuscript.

Corresponding author

Correspondence to Lingchao Meng .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Li, D., Fan, X. & Meng, L. Development and validation of a higher-order thinking skills (HOTS) scale for major students in the interior design discipline for blended learning. Sci Rep 14 , 20287 (2024). https://doi.org/10.1038/s41598-024-70908-3

Download citation

Received : 28 February 2024

Accepted : 22 August 2024

Published : 31 August 2024

DOI : https://doi.org/10.1038/s41598-024-70908-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Assessment scale
• Higher-order thinking skills
• Interior design
• Blended learning

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: Anthropocene newsletter — what matters in anthropocene research, free to your inbox weekly.

Online Study Skills Hub: Academic Writing

• Where do I start?
• Critical Thinking & Writing
• Editing & Proofreading
• Essay Planning: Structuring your work
• Dissertations & Theses
• Exam Preparation & Revision
• Finding Books & eBooks
• Finding Resources Step by Step Guide
• Logging in to Online Resources
• Request an Item
• Understanding Resource Lists
• Using eBooks
• Presentation Skills
• Reading & Note Taking
• Referencing: How & Why You Should do it This link opens in a new window
• Statistics & SPSS This link opens in a new window
• Time Management Skills
• Understanding Feedback
• Finding Information with AI
• AI & Academic Integrity

Introduction to Academic Writing

Academic writing at the university level is a formal style of writing that is used to communicate ideas, research, and arguments clearly and effectively. This type of writing is characterized by a structured approach, which often includes a clear introduction, a body that presents evidence and arguments, and a conclusion that summarizes the findings or arguments presented.

Key Features of Academic Writing:

Formal Tone and Style : Academic writing uses a formal tone, avoiding colloquial language, slang, or overly casual expressions. It often involves the use of third-person perspective, focusing on presenting information objectively and professionally.

Clarity and Precision : The language used in academic writing should be clear, concise, and specific. Writers should avoid ambiguity and vagueness, ensuring that their arguments and points are easily understandable.

Evidence-Based Arguments : A fundamental aspect of academic writing is the use of evidence to support claims and arguments. This often includes data from research studies, statistics, and citations from credible academic sources. Proper citation and referencing are essential to acknowledge the sources of information and avoid plagiarism.

Critical Thinking and Analysis : Academic writing encourages critical thinking, where students are expected to not only present facts but also analyze, interpret, and evaluate them. This involves questioning existing theories, forming hypotheses, and drawing reasoned conclusions based on evidence.

Structure and Organization : Effective academic writing is well-organized, with a logical flow of ideas. Common structures include essays, research papers, reports, and dissertations, each with its specific organizational pattern. For instance, essays typically have an introduction, body paragraphs, and a conclusion, while research papers include sections like literature review, methodology, results, and discussion.

Proper Referencing and Citation : Academic writing requires the use of proper referencing styles such as APA, MLA, or Chicago, to cite sources and give credit to original authors. This practice not only supports the writer's credibility but also helps readers locate the sources used.

Purpose of Academic Writing:

The primary purpose of academic writing is to contribute to the body of knowledge in a particular field, share research findings, engage in scholarly debates, and provide information that advances understanding. It is an essential skill for university students as it allows them to communicate effectively within the academic community and beyond.

Developing Academic Writing Skills:

Students can improve their academic writing skills through practice, feedback, and understanding the conventions of their particular discipline. Engaging with a wide range of academic texts, participating in writing workshops, and seeking feedback from peers and instructors can help in honing these skills.

Overall, academic writing is a critical component of university education that facilitates learning, critical thinking, and the advancement of knowledge. It plays a vital role in helping students articulate their ideas, engage with scholarly work, and contribute meaningfully to academic and professional fields.

Assessment Types

• << Previous: Where do I start?
• Next: Critical Thinking & Writing >>
• Last Updated: Sep 3, 2024 3:10 PM
• URL: https://library.roehampton.ac.uk/studyskills

FedEx and INJAZ UAE Collaborate to Nurture Over 1,000 Future Youth Leaders in the UAE

The entities continue advocating for the development of leadership and critical thinking skills among students in the region.

• Copy Link copied

Dubai, UAE, September 2, 2024: Federal Express Corporation, the world’s largest express transportation company, and INJAZ UAE, a member of Junior Achievement (JA) Worldwide, announced that their ongoing collaboration shaped the journeys of 1,172 students in the UAE during the 2023-2024 academic year. The achievement is part of a joint effort by both entities to enhance career readiness, financial literacy, and entrepreneurial skills among youth in the region.

FedEx and INJAZ UAE surpassed initial targets of 1,000 during the past year. The milestone was facilitated by 11 FedEx volunteers, who delivered a series of educational programs across seven academic institutions in the UAE.

As part of their outreach, the volunteers equipped students with essential skills such as resume building, interviewing techniques, prudent financial management, fundamentals of starting a business, and effective marketing strategies, among others.

Commenting on the success of the collaboration, Taarek Hinedi, vice president of FedEx Middle East and Africa Operations said, “Our work with INJAZ UAE reflects our commitment to empowering the next generation of leaders. By equipping students with essential workplace tools and knowledge, we invest in the region’s talent pool and prepare them to excel in their careers. Furthermore, our achievements over the past year are a testament to the hard work and dedication of our volunteers and the enthusiasm of the students.”

Razan Bashiti, Chief Executive Officer of INJAZ UAE added: “The support from FedEx has been instrumental in reaching our goals and expanding our impact. Together, we have made significant strides in preparing UAE’s youth for the workplace of the future. We look forward to continuing this collaboration and making a positive difference in the lives of many more students.”

The MENA region is home to almost 140 million young people aged between 10-24 years, representing a quarter of the population [1] . This youthful demographic represents a powerful opportunity to transform the future of economies across the region.

FedEx has been supporting JA Worldwide globally for more than 40 years, championing the development of technical, soft skills, and career-ready skills in youth to succeed in their professional journey.

About Federal Express Corporation

Federal Express Corporation is the world’s largest express transportation company, providing fast and reliable delivery to more than 220 countries and territories. Federal Express Corporation uses a global air-and-ground network to speed delivery of time-sensitive shipments by a definite time and date.

FedEx press releases are available here .

About INJAZ UAE

INJAZ UAE delivers cutting-edge, experiential volunteer-based programs in work readiness, entrepreneurship, and financial literacy. INJAZ UAE is dedicated to giving youth the knowledge and skills they need to plan their professional future and make smart academic and economic choices. INJAZ UAE’s programs help spark the learning in youth to experience the opportunities and realities of the global professional world in the 21st century. with schools, technical/ vocational centers and other partners, INJAZ UAE reached more than 120,000 youth since inception. Learn more about us at https://www.injazuae.org/ .

INJAZ UAE is a member of  JA Worldwide (JAW) regionally known as  INJAZ Al Arab (JA MENA), one of the world’s largest non-profit business education organizations, reaching over 17 million students a year in 115 countries. It serves as a nexus between the business community, educators and volunteers working together to empower young people to own their economic success.

[1] https://www.unicef.org/mena/media/23791/file/Young%20people%E2%80%99s%20health%20and%20well-being.pdf

Europe PMC requires Javascript to function effectively.

Either your web browser doesn't support Javascript or it is currently turned off. In the latter case, please turn on Javascript support in your web browser and reload this page.

Developing Critical Thinking Skills in Pharmacy Students

Citation metadata, document controls, main content.

Objective. To review the importance of and barriers to critical thinking and provide evidence-based recommendations to encourage development of these skills in pharmacy students.

Findings. Critical thinking (CT) is one of the most desired skills of a pharmacy graduate but there are many challenges to students thinking critically including their own perceptions, poor metacognitive skills, a fixed mindset, a non-automated skillset, heuristics, biases and the fact that thinking is effortful. Though difficult, developing CT skills is not impossible. Research and practice suggest several factors that can improve one's thinking ability: a thoughtful learning environment, seeing or hearing what is done to executive cognitive operations that students can emulate, and guidance and support of their efforts until they can perform on their own.

Summary. Teaching CT requires coordination at the curricular level and further to the more discrete level of a lesson and a course. Instructor training is imperative to this process since this intervention has been found to be the most effective in developing CT skills.

Keywords: critical thinking, metacognition, course design, problem solving, clinical decision making

INTRODUCTION

Critical Thinking (CT) is one of the most desired skills of a pharmacy graduate because pharmacists need to think for themselves, question claims, use good judgment, and make decisions. (1,2) It is needed in almost every facet of pharmacy practice because pharmacy students need to evaluate claims made in the literature, manage and resolve patients' medication problems, and assess treatment outcomes. (3) While pharmacy educators may agree that CT is an essential skill for pharmacy students to develop, it must be consistently defined because the definition determines how it is taught and assessed. (4) While many definitions of CT exist, (5) it is most commonly defined as automatically questioning if the information presented is factual, reliable, evidence-based, and unbiased. (2) In simpler terms, it is reflecting on what to believe or do. (6)

To operationalize the CT definition, six core CT skills have been proposed: interpretation, analysis, evaluation, inference, explanation, and self-regulation (directing one's actions automatically). (7,8) Interpretation includes understanding and communicating the meaning of information to others. Analysis includes connecting pieces of information together to determine the intended meaning. Inference is recognizing elements of information one has and using those elements to reach reasonable conclusions or hypotheses. Evaluation involves making a judgment about the credibility of a statement or information. Explanation includes adding clarity to information one shares so it can be fully understood by another person. Self-regulation is the ability to control one's own thoughts, behavior and emotions.

Besides the six core skills, CT is more than a stepwise process. It is a summation of attitude, knowledge, and knowledge of the CT process (Attitude + Knowledge + Thinking Skills = Critical Thinking). (9) All three components are necessary. First, individuals need an attitude that aligns with CT. This attitude includes a willingness to plan, being flexible, being persistent, willingness to self-correct, being mindful and a desire to reconcile information. (9) If the attitude is not there, it is unlikely that the individual will engage in the actual process. Second, CT requires knowledge or something to think about. The more knowledge the individual has, the better their process and answer. Thus, acquiring foundational, requisite knowledge is important in CT. The final part is the knowledge of the CT process. Knowing the steps and following them is key to success. Not following the steps can lead to incorrect answers. Skipping steps is one of the barriers to CT. When these three components are present, CT can occur at a deep level.

While CT is used often, it is important to differentiate CT from other processes. Problem solving, clinical reasoning and clinical decision-making are related higher-order CT skills and while the terms may be used interchangeably, there are distinguishing features. Problem solving is a general skill that involves the application of knowledge and skills to achieve certain goals. Problem solving can rely on CT but it does not have to. (10,11) The steps of identifying a problem, defining the goals, exploring multiple solutions, anticipating outcomes and acting, looking at the effects, and learning from the experience are all steps that can benefit from eliminating assumptions or guesses during the problem-solving process. (12) In comparison to general thinking skills, clinical reasoning and clinical decision-making depend on a CT mindset and are domain-specific skills that are used within pharmacy and other health sciences. (4) Clinical reasoning is the ability to consider if one's evidence-based knowledge is relevant for a particular patient during the diagnosis, treatment, and management process. (4,13) Clinical decision-making happens after the clinical reasoning process and is focused on compiling data and constructing an argument for treatment based on the interpretation of the facts/evidence about the patient. (14) Overall, the process of thinking like an expert by considering the evidence and making correct decisions about a patient to solve a patient's problems is a skillset that students should practice so it becomes automatic. See Figure 1 for a visual representation.

Barriers to Critical Thinking

There are several challenges to students thinking critically: perceptions, poor metacognitive skills, a fixed mindset, heuristics, biases and because thinking is effortful. The first barrier is students' perceptual problem--students believe they know how to solve problems, so often, they do not understand why they are being re-taught this skill. Educators teach students how to monitor their thinking and become better problem solvers by giving them a framework to be more thoughtful thinkers.

The next challenge is students' weak metacognitive skills. The relationship between CT and metacognitive skills has been noted in the literature. (15) Metacognition refers to an individual's ability to assess his/her own thinking and actual level of skill or understanding in an area. Metacognition helps critical thinkers be more aware of and control their thinking processes. (15) Students who are weak at metacognition jump to conclusions without evaluating the evidence, thinking they know the answer, which ultimately interferes with CT.

A third reason CT is difficult for students is that they may have a fixed mindset or a belief that their intelligence cannot change. (16) If students believe CT is an innate skill-set that occurs naturally, they may not invest the effort to develop it because they believe that no matter how hard they try, they will never get it.

Heuristics can get in the way of CT. Heuristics are our shortcuts to thinking--they are a strategy applied implicitly or deliberately during decision-making where we use only part of the information we might otherwise want or need. This results in decisions that are quicker and less effortful because the individual may be using the best single piece of data to make a more frugal approach. (17-19) In a classic study, participants were asked, "If a ball and bat cost $1.10, and the bat is $1 more than the ball, what was the cost of the ball?" (20) The most popular answer is $0.10, which is incorrect (the correct answer is the ball costs $0.05, the bat then is $1.05 or $1 more. If the ball was $0.10, the bat is only $0.90 more than the ball). We take cognitive shortcuts because thinking is effortful and if we can get a quick response that fits our current needs, we will do it. Kahneman referred to two systems of thought: System 1 and System 2. (19,21) System 1 is a fast decision-making system responsible for intuitive decision-making based on emotions, vivid imagery, and associative memory. System 2 thought processes is a slow system that observes System 1's outputs, and intervenes when "intuition" is insufficient. (21)

Another challenge that makes CT difficult for students is their inherent biases. One major bias is confirmation bias or the tendency to search for information in a way that confirms our ideas or beliefs. (22) Confirmation bias happens because of an eagerness to arrive at a conclusion, so students may assume they are questioning their assumptions when they are only searching for enough information to confirm their beliefs. (22) When we want to think critically, we want the evidence against our view to better inform our decision. See Appendix 1 for a list of cognitive biases that may affect our thinking.

CT is difficult and does not develop automatically. It takes practice and effort. Experts think critically without conscious thought, which makes it effortless. However, developing expertise is estimated to take 10 years or 10,000 hours (or more) of deliberate practice, so it is a time consuming activity. (14,23) In a study of thinking using the game Tetris, it was shown that initial game learning resulted in higher brain glucose consumption compared to individuals with experience playing and those watching someone play. (24) Similar results are seen when comparing experts to novices. Functional MRI studies show that experts use less of their brain to solve a problem than novices, partly because a problem for a novice is not a problem for an expert. (25) It is experience that has led to muscle memory and heuristics. Students do not have a lot of experience thinking critically and therefore, do not want to do it because it is difficult and time consuming; they would rather do things that are automatic and effortless.

Developing Critical Thinking Skills

Developing CT skills is difficult but not impossible. CT is a teachable skill and is often discipline-specific because it relies on discipline-specific knowledge. Research and practice suggest several factors that improve thinking: a thoughtful learning environment (eg, integration), seeing or hearing what is actually done to executive cognitive operations one is trying to improve (eg, model behavior), guidance and support of one's efforts until he or she can perform on their own (eg, scaffolding); (26) and prompting to question what is thought to be known (eg, challenging assumptions). (27) These are general, key points that instructors can do to help students develop CT skills.

Creating a thoughtful learning environment is not limited to just letting students make mistakes. Table 1 compares features of thoughtful classrooms to traditional classrooms that do not emphasize CT. The first piece of this thoughtful learning environment is helping students to integrate their knowledge. Integration allows students to build on previous experiences, provide developmentally appropriate opportunities for the individual to produce optimal performance, and lay a foundation for further development. By intentionally creating an environment that allows students to integrate previous and current knowledge, they can begin to evaluate how the concepts are related and make decisions on how to apply that knowledge to future, and likely different, situations. Integration can take many forms and does not necessarily mean courses need to be integrated or aligned in time. Integration can take the form of integrating the cumulative knowledge gained over the curriculum.

Modeling expert thinking is another way to help students see CT in action and begin to use these steps themselves. Instructors should verbalize their executive cognitive operations for students to hear or see when addressing a problem or issue that requires CT. No single step is too insignificant to point out. Learners are novice and assumptions should not be made that they understand or know how to perform a seemingly simple set in the thinking process. By watching the experts process information, learners begin to form those thinking skills as well.

Scaffolding is another general method that can facilitate development of CT skills. Scaffolding is a temporary support mechanism. Students receive assistance early on to complete tasks, then as their proficiency increases, that support is gradually removed. In this way, the student takes on more and more responsibility for his or her own learning. To provide scaffolding, instructors should provide clear directions and the purpose of the activity, keep students on task, direct students to worthy sources, and offer periodic assessments to clarify expectations. This process helps to reduce uncertainty, surprise and disappointment while creating momentum and efficiency for the student.

Thinking begins when our assumptions are violated. Driving to work requires little effort. We do it all the time and sometimes we may wonder how we got to work because our thoughts were elsewhere. On a daily basis, you assume your drive will be normal and unimpeded. Now imagine there is traffic. You move from auto-pilot to thinking mode because your assumptions were violated. When our assumptions are violated, we start to think and we see this thought process as early as a few weeks from birth. (28) In the classroom, we must identify and challenge students' assumptions. As an example from self-care instructors, when students are asked to recommend a product for cough associated with the common cold, any student pharmacist with community pharmacy experience may answer "dextromethorphan." This may be what they have seen in practice or what they received as a child from their parents. They have experience in this context. However, this answer is not supported by the guide-lines, (29,30) but the students will argue it is correct because of their experience. The cognitive dissonance--not expecting something to happen that you thought would--starts the cognitive thinking process. From an instructional standpoint, it may be important to initiate the critical thinking process by having students make predictions on outcomes and showing how their predictions may be correct or incorrect.

Developing CT requires a 4-step approach. (9) The first step is explicitly learning the skills of CT. The second is developing the disposition for effortful thinking. The third step is directing the learner to activities to increase the probability of application and transfer of skills. The final step is making the CT process visible by instructors making the metacognitive monitoring process explicit and overt. These four steps should be included both at the broad curricular level and down to the more discrete level of a lesson and a course.

Curriculum. College has shown to increase CT skills when CT is measured through standardized assessments of CT skills (four years of college = effect size of 0.6). (31) While part of this growth in college may be due to maturation and increase in knowledge, developing CT skills requires curriculum-level coordination. Just like a military action will fail if the individual units do not play their role, CT development will fail if individual units do not play their respective roles. One way to develop CT skills is to use a two-fold approach. (1,32) The first step is to have a course in the curriculum that teaches the general thinking skill process and starts to develop the dispositions. The second step is to have individual courses reflect that process within the context of the subject matter. Ideally courses have explicit learning objectives and make the thinking process equally as explicit; this is called the infusion method. Table 2 shows the effect sizes (difference in performance relative to the standard deviation) of these types of interventions. Typically effect sizes under 0.2 are considered small, over 0.4 are considered educationally significant, and over 0.7 are considered large. (33,34) To note, these effect sizes come from a variety of study types, durations and outcome measures. For example, one study in nursing used a standardized assessment of CT (California Critical Thinking Skills Test) to compare lecture to problem-based learning (PBL) in a pre/post design. (35) Examining pre-to-post changes, PBL showed an effect size of 0.42 whereas lecture was 0.010. When comparing the post-scores from PBL to lecture, the effect size was 0.44. Alternatively, undergraduates were placed in dyads across four different conditions outlined in Table 2: general, infusion, immersion and control. (36) The outcome was a rubric developed by the instructor and research team. Compared to control, the general (.46), infusion (1.1) and immersion (.97) all showed positive and moderate-to-large effect sizes. Relatively, infusion was better than general (.60) as was immersion (.49) with very little difference between infusion and immersion (.12). Although the effect sizes in Tables 2 and 3 should be interpreted with some caution as the context varies, they represent effects across a variety of disciplines, outcome measures and study designs, thus suggesting a more generalizable effect.

Courses. Within a course structure, collaborative learning (ie, peer teaching, cooperative learning) helps develop CT more than other activities. One meta-synthesis that attempted to integrate results from different but interrelated qualitative studies on critical thinking found an effect size of 0.41 for promoting CT skills when collaborative learning was used. (1,32) Collaborative learning provides feedback to learners and puts learners in a setting that challenges their assumptions and engages them in deeper learning to solve a problem. However, if learners receive minimal guidance, they may become lost and frustrated or develop misunderstandings and alternative understandings. (32,36) Students' CT improves most in environments where learning is mediated by someone who confronts their beliefs and alternative conceptions, encourages them to reflect on their own thinking, creates cognitive dissonance or puzzlement, and challenges and guides their thinking when they are actively involved in problem solving. This guided participation role may be implemented by learners in structured activities with the guidance, support, and challenge of companions. (26)

Lessons. Individual lessons should be designed with CT in mind by intentionally providing learners opportunities to engage in complex thinking. Appendix 2 offers a guide to developing these types of opportunities for students. The goals of the activities should be made clear and instructors should acknowledge that effortful thinking is required while recognizing that the learning environment allows students to make mistakes. Instructors should explicitly model their expert thinking and actively monitor how students are learning. Adjustments to teaching should be made reactively as instructors notice trends in student thinking. Providing enough time to think and learn during these activities is crucial. Expect novice students to take at least double the time it would take you as an expert to complete the activity. Appendix 3) provides a worksheet that students can use to develop their CT skills during an activity.

Instructors. While the curriculum structure can have a large effect, it relies heavily on the individual instructor. Instructor training has been found to be the most effective intervention in developing CT skills (Table 3). This training, however, must go beyond having students observe others think critically. This facilitation requires the appropriate material (eg, cases), facilitation skills and mentoring skills. (32) Appendix 4 provides a rubric to help instructors assess students' problem-solving skills on a problem-solving activity. Though difficult, instructors should often remain silent during the activity. When necessary, instructors can ask probing questions that require students to clarify, elaborate, explain in more depth or ask more questions, which are related to metacognition. Instructors can signal acceptance of the student's assertions by paraphrasing, providing a friendly facial expression, or writing responses for all to see. The key is to facilitate learning and not "do" the learning for the students.

Recommendations

A common model for the process of CT should be used in each pharmacy school curriculum. Ideally, a course should be required for all students early in the curriculum that addresses the definition, common model, and dispositions of CT and then provides an opportunity for students to actively practice these skills on general subject matter content. As students' knowledge of pharmacy specific content grows, courses need to explicitly use the process outlined in the general course with application to the subject specific content. The repetition of these skills in multiple courses or course series will help students practice this skill. Additionally, all instructors should learn the model taught to students and learn how to create and facilitate activities to encourage CT in their content areas.

While there may be many templates for CT, we propose a 4-step cycle: generation, conceptualization, optimization and implementation. (37,38) In the generation phase, learners identify the problem and find facts. This is followed by the conceptualization phase when learners define the problem and draft ideas that could explain the defined problem. In the optimization phase, learners evaluate and select an idea then design a plan. Finally, the implementation phase involves accepting the plan and taking action. The cycle restarts with finding a new problem. For example, during a patient encounter, a learner would enter the generation phase, find all the problems and facts (laboratory values, past medical history, etc.). Then the learner would define the problem(s) and generate ideas as to why the problems are occurring. For example, the patient is complaining of fatigue and the learner would have to come up with reasons why fatigue might occur (anemia, lack of sleep, pregnancy, poor diet). The learner then uses the facts to evaluate each potential cause and consider what further tests may be necessary to exclude some of the potential causes. After selecting the cause, the learner formulates a plan and decides his or her next action. Once the learner discovers the patient is anemic, the cycle restarts with treatment options. This cycle can be used along with the Joint Commission of Pharmacy Practitioners Pharmacists' Patient Care Process. (39)

Critical thinking skills (interpretation, analysis, evaluation, inference, explanation, and self-regulation) are important for health care providers, including pharmacists. While some students and instructors may think that CT skills are fixed, CT can be developed and augmented through a process of attitude alignment, absorption of knowledge, and learning new thinking skills. CT is also developed when one learns to combat potentially hazardous CT roadblocks such as bias, heuristics (thinking shortcuts), and simply not wanting to go through the effort of thinking on a higher level. Pharmacy educators can foster the development of CT skills in the wide scope of curricular design, in the narrowest interactions between professor and student, and everywhere in between. It is important to note that the methods described in this paper do not have to be added to an already compressed curriculum but rather can be used with existing materials to cover the content in a deeper and more meaningful way. By modeling expert thinking and using scaffolding techniques to support students' CT development, pharmacy educators can instill both the desire and the drive for students to begin thinking critically. Regardless, it is noteworthy to point out that teaching CT skills requires time and effort at the potential expense of other skills. Thus, gains in critical thinking during a PharmD curriculum may be a function of our need to develop a multitude of other skills like teamwork, empathy, adaptability, communication, and initiative. White boxes represent the thinking type while gray boxes provide descriptions of each type and show how the skills build upon each other.

(1.) Abrami PC, Bernard RM, Borokhovski E, et al. Instructional interventions affecting critical thinking skills and dispositions: a stage 1 meta-analysis. Rev Educ Res. 2008;78(4):1102-1134. (2.) Alfaro-LeFevre R. Critical Thinking, Clinical Reasoning, and Clinical Judgment: A Practical Approach. 6th ed. Philadelphia, PA: Elsevier; 2017.

(3.) Oderda GM, Zavod RM, Carter JT, et al. An environmental scan on the status of critical thinking and problem solving skills in colleges/schools of pharmacy: report of the 2009-2010 Academic Affairs Standing Committee. Am J Pharm Educ. 2010;74(10): Article S6.

(4.) Hughes R. Patient Safety and Quality: An Evidence-based Handbook for Nurses. Rockville, MD: Agency for Healthcare Research and Quality, US Department of Health and Human Services; 2008.

(5.) Oyler DR, Romanelli F. The fact of ignorance: revisiting the Socratic method as a tool for teaching critical thinking. Am J Pharm Educ. 2014;78(7):Article 144.

(6.) Ennis RH. Critical thinking asssessment. Theory Pract. 1993; 32(3):179-186.

(7.) Erstad W. Six critical thinking skills you need to master now. http://www.rasmussen.edu/student-life/blogs/college-life/critical-thinking-skills-to-master-now/. Accessed February 21, 2018.

(8.) Facione PA. Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction. Millbrae, CA: American Philosophical Association; 1990.

(9.) Halpern DF. Thought and Knowledge: An Introduction to Critical Thinking. 5th ed. New York, NY: Psychology Press; 2014.

(10.) Bransford Jd, Stein BS. The Ideal Problem Solver: A Guide for Improving Thinking, Learning, and Creativity. 2nd ed. New York, NY: W.H. Freeman; 1993.

(11.) Medina MS, Plaza CM, Stowe CD, et al. Report of the 2012-2013 academic affairs standing committee: revising the Center for the Advancement of Pharmacy Education (CAPE) Educational Outcomes 2013. Am J Pharm Educ. 2013;77(8):Article S8.

(12.) Hartman D. Strategies for critical thinking and problem solving. http://smallbusiness.chron.com/strategies-critical-thinking-problem-solving-12709.html. Accessed February 21, 2018.

(13.) Linn A, Khaw C, Kildea H, Tonkin A. Clinical reasoning: a guide to improving teaching and practice. Aust Fam Physician. 2012; 41(1-2):18-20.

(14.) Ericsson KA. The Road to Excellence: The Acquisition of Expert Performance in the Arts and Sciences, Sports, and Games. Mahwah, NJ: Lawrence Erlbaum Associates; 1996.

(15.) Medina MS, Castleberry AN, Persky AM. Strategies for improving learner metacognition in health professional education. Am J Pharm Educ. 2017;81(4):Article 78.

(16.) Dweck CS. Can personality be changed? The role of beliefs in personality and change. Curr Direct Psychol Sci. 2008;17(6):391-394.

(17.) Gigerenzer G, Gaissmaier W. Heuristic decision making. Ann Rev Psychol. 2011;62(1):451-482.

(18.) Martignon L, Hoffrage U. Fast, frugal, and fit: simple heuristics for paired comparison. Theory Decis. 2002;52(1):29-71.

(19.) Evans JS. Dual-processing accounts of reasoning, judgment, and social cognition. Ann Rev Psychol. 2008 2008;59:255-278.

(20.) De Neys W, Rossi S, Houde O. Bats, balls, and substitution sensitivity: cognitive misers are no happy fools. Psychon Bull Rev. 2013;20(2):269-273.

(21.) Kahneman D. Thinking, Fast and Slow. New York, NY: Farrar, Straus and Giroux; 2013.

(22.) Jonas E, Schulz-Hardt S, Frey D, Thelen N. Confirmation bias in sequential information search after preliminary decisions: an expansion of dissonance theoretical research on selective exposure to information. J Pers Soc Psychol. 2001;80(4): 557-571.

(23.) Persky AM, Robinson JD. Moving from novice to expertise and its implications for instruction. Am J Pharm Educ. 2017;81(9):Article 6065.

(24.) Haier RJ, Siegel BV, MacLachlan A, Soderling E, Lottenberg S, Buchsbaum MS. Regional glucose metabolic changes after learning a complex visuospatial/motor task: a positron emission tomographic study. Brain Res. 1992;570(1):134-143.

(25.) Jancke L, Shah NJ, Peters M. Cortical activations in primary and secondary motor areas for complex bimanual movements in professional pianists. Cog Brain Res. 2000;10(1-2):177-183.

(26.) Beyer BK. Improving Student Thinking: A Comprehensive Approach. Boston, MA: Allyn and Bacon; 1997.

(27.) Johnson DW, Johnson RT. Energizing learning: the instructional power of conflict. Educ Res. 2009;38(1):37-51.

(28.) Wynn K. Addition and subtraction by human infants. Nature 1992;358(6389):749-750.

(29.) Malesker MA, Callahan-Lyon P, Ireland B, Irwin RS. CHEST expert panel. Pharmacologic and nonpharmacologic treatment for acute cough associated with the common cold: CHEST expert panel report. Chest 2017;152(5):1021-1037.

(30.) Pratter MR, Brightling CE, Boulet LP, Irwin RS. An empiric integrative approach to the management of cough: ACCP evidence-based clinical practice guidelines. Chest. 2006;129(1 Suppl):222S-231S.

(31.) Huber CR, Kuncel NR. Does college teach critical thinking? A meta-analysis. Rev Educ Res. 2016;86(2):431-468.

(32.) Abrami PC, Bernard RM, Borokhovski E, Waddington DI, Wade CA, Persson T. Strategies for teaching students to think critically: a meta-analysis. Rev Educ Res. 2015;85(2):275-314.

(33.) Cohen J. A power primer. Psychol Bull 1992;112(1):155-159.

(34.) Hattie J. Visible Learning: A Synthesis of Over 800 Meta-analyses Relating to Achievement. London; New York, NY: Routledge; 2009.

(35.) Gholami M, Moghadam PK, Mohammadipoor F, et al. Comparing the effects of problem-based learning and the traditional lecture method on critical thinking skills and metacognitive awareness in nursing students in a critical care nursing course. Nurse Educ Today. 2016;45:16-21.

(36.) Angeli C, Valanides N. Instructional effects on critical thinking: performance on ill-defined issues. Learn Instruct. 2009;19(4):322-334.

(37.) Scheirer MA. Planning evaluation through the program life cycle. Am J Eval. 2012;33(2):263-294.

(38.) Romiszowski AJ. Designing Instructional Systems: Decision Making in Course Planning and Curriculum Design. New York, NY: Nichols Pub.; 1981.

(39.) Joint Commission of Pharmacy Practitioners. The pharmacists' patient care process. https://jcpp.net/patient-care-process. Accessed February 21, 2018.

(40.) List of biases in judgment and decision making. http://psychology.wikia.com/wiki/List_of_biases_in_judgment_and_decision_making. Accessed February 23, 2018.

Appendix 1. List of Biases That May Impact Critical Thinking (40)

Appendix 2. Generating Thoughtful Questions to Engage Students in Critical Thinking (26)

Select a topic

a. Rich enough detail, depth of detail, implications and interconnections and relationships inside and outside of area.

b. Open to diverse interpretation and methods of inquiry.

c. Capable of being entered at any variety of points.

d. Requires guidance of an instructor.

e. Is one that instructors are likely to spend lots of time on instead of rushing through it.

f. Contributes to the development of meaningful and significant key ideas, explanation, principles, concepts, and generalizations.

g. Can be learned about in the context of realistic problems.

h. Fits into the overall curriculum and course

Begin at the global level

i. Focuses on big picture.

j. Focuses on ill-defined rather than precisely delineated topics.

k. Requires students to pose and then answer numerous subordinate questions as they seek to define and probe the initial question and implications.

Word the question provocatively

1. Helps invite student engagement; questions that present unusual, unanticipated, or unconventional points of view bother people, agitate thinking, spark curiosity, and demand response.

Engage the students

m. Focus on the "non-present"--predicting or planning future conditions or events, reconstructing past events.

n. Have students reflect about questions before they attempt to answer them or before examining the answers they generate.

Appendix 3. Critical Thinking/Making an Argument Worksheet

Adapted from Halpern's Thought and Knowledge: An Introduction to Critical Thinking. (9)

1. State your conclusion.

2. Give 3 reasons (or some other number) that support your conclusion. Rate each reason (weak, moderate, strong, very strong).

3. Give 3 counterarguments (or some other number) that weaken your conclusion. Rate how much each counterargument weakens the conclusion (little, moderate, much, very much).

4. List any qualifiers (limitations on the reasons for or against).

5. List any assumptions.

6. Are your reasons and counterarguments directly related to your conclusion?

7. What is the overall strength of your argument? Weak, moderate, strong, very strong

Appendix 4. University of Oklahoma College of Pharmacy Problem Solving Rubric Created by Dr. Melissa Medina

Adam M. Persky, PhD, (a,b) Melissa S. Medina, EdD, (c) Ashley N. Castleberry, PharmD, MAEd (d)

(a) Eshelman School of Pharmacy, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina

(b) Associate Editor, American Journal of Pharmaceutical Education, Arlington, Virginia

(c) College of Pharmacy, University of Oklahoma Health Sciences Center, Oklahoma City, Oklahoma

(d) College of Pharmacy, University of Arkansas for Medical Sciences, Little Rock, Arkansas

Corresponding Author: Melissa S. Medina, Department of Pharmacy: Clinical and Administrative Sciences, College of Pharmacy, University of Oklahoma Health Sciences Center, PO Box 26901, 1110 N. Stonewall, CPB 125, Oklahoma City, OK 73190. Tel: 405-271-6484. Fax: 405-271-3830. E-mail: [email protected]

Submitted February 23, 2018; accepted May 21, 2018; published March 2019.

Caption: Figure 1. Schematic of Critical Thinking and its Relationship to Other Types of Thinking

White boxes represent the thinking type while gray boxes provide descriptions of each type and show how the skills build upon each other

Please Note: Illustration(s) are not available due to copyright restrictions.

Source Citation

Gale Document Number: GALE|A586026071

Related Subjects

• Critical thinking
• Decision making
• Medical students
•   Google Drive™
•  OneDrive™

Email Document

Required fields marked with *

Separate each e-mail address with a semicolon

• Subject Line:

Report an Issue

Submitted diagnostic report.

Timestamp: September 3, 2024 at 10:43:12 AM EDT

Ticket Number:

Reporter's Email:

Reporter's Name: