how to critically analyse psychological research

Rosie Psychology: Your online tutor

How to demonstrate critical evaluation in your psychology assignments

Thinking critically about psychology research

Critical thinking is often taught in undergraduate psychology degrees, and is a key marking criteria for higher marks in many assignments. But getting your head around how to write critically can sometimes be difficult. It can take practice. The aim of this short blog is to provide an introduction to critical evaluation, and how to start including evidence of critical evaluation in your psychology assignments.

So what does “critical evaluation” really mean?

Broadly speaking, critical evaluation is the process of thinking and writing critically about the quality of the sources of evidence used to support or refute an argument. By “ evidence “, I mean the literature you cite (e.g., a journal article or book chapter). By “ quality   of the evidence “, I mean thinking about whether this topic has been tested is in a robust way. If the quality of the sources is poor, then this could suggest poor support for your argument, and vice versa. Even if the quality is poor, this is important to discuss in your assignments as evidence of critical thinking in this way!

In the rest of this blog, I outline a few different ways you can start to implement critical thinking into your work and reading of psychology. I talk about the quality of the evidence, a few pointers for critiquing the methods, theoretical and practical critical evaluation too. This is not an exhaustive list, but hopefully it’ll help you to start getting those higher-level marks in psychology. I also include an example write-up at the end to illustrate how to write all of this up!

The quality of the evidence

There are different types of study designs in psychology research, but some are of higher quality than others. The higher the quality of the evidence, the stronger the support for your argument the research offers, because the idea has been tested more rigorously. The pyramid image below can really help to explain what we mean by “quality of evidence”, by showing different study designs in the order of their quality. 

Not every area of psychology is going to be full of high quality studies, and even the strongest sources of evidence (i.e., systematic reviews and/or meta-analyses) can have limitations! Because no study is perfect, it can be a good habit to tell the reader, in your report, (i) what the  design  of the study is that you’re citing, AND, (ii)  how  this affects your argument. Doing so would be evidence of critical thought. (See an example write-up below for implementing this, but do not copy and paste it!) 

But first, what do I mean by “design”? The design of the study refers to  how  the study was carried out. There are sometimes broad categories of design that you’ll have heard of, like a ‘survey design’, ‘a review paper’, or an ‘experimental design’. Within these categories, though, there can be more specific types of design (e.g. a  cross-sectional  survey design, or a  longitudinal  survey design; a  randomised controlled  experiment or a  simple pre-post  experiment). Knowing these specific types of design is a good place to start when thinking about how to critique the evidence when citing your sources, and the image below can help with that. 

Image source: https://thelogicofscience.com/2016/01/12/the-hierarchy-of-evidence-is-the-studys-design-robust/

In summary, there are various types of designs in psychology research. To name a few from the image above, we have: a meta-analysis or a systematic review (a review paper that summarises the research that explores the same research question); a cross-sectional survey study (a questionnaire that people complete once – these are really common in psychology!). If you’re not familiar with these, I would  highly suggest  doing a bit of reading around these methods and some of their general limitations – you can then use these limitation points in your assignments! To help with this, you could do a Google Scholar search for ‘limitations of a cross-sectional study’, or ‘why are randomised control trials gold standard?’. You can use any published papers as further support as a limitation.

Methodological critical evaluation

• Internal validity: Are the findings or the measures used in the study reliable (e.g., have they been replicated by another study, and is the reliability high)? 
• External validity: Are there any biases in the study that might affect generalisability(e.g., gender bias, where one gender may be overrepresented for the population in the sample recruited)?  Lack of generalisability is a common limitation that undergraduates tend to use by default as a limitation in their reports. It’s a perfectly valid limitation, but it can usually be made much more impactful by explaining exactly  how  it’s a problem for the topic of study. In some cases, this limitation may not be all that warranted; for example, a female bias may be expected in a sample of psychology students, because undergraduate courses tend to be filled mostly with females! 
• What is the design of the study, and how it a good or bad quality design (randomised control trial, cross-sectional study)? 

Theoretical critical evaluation

• Do the findings in the literature support the relevant psychological theories?
• Have the findings been replicated in another study? (If so, say so and add a reference!)

Practical critical evaluation

• In the real world, how easy would it be to implement these findings?
• Have these findings been implemented? (If so, you could find out if this has been done well!)

Summary points

In summary, there are various types of designs in psychology research. To name a few from the image above, we have: a meta-analysis or a systematic review (a review paper that summarises the research that explores the same research question); a cross-sectional survey study (a questionnaire that people complete once – these are really common in psychology!). If you’re not familiar with these, I would highly suggest doing a bit of reading around these methods and some of their general limitations – you can then use these limitation points in your assignments! To help with this, I would do a Google Scholar search for ‘limitations of a cross-sectional study’, or ‘why are randomised control trials gold standard?’. You can use these papers as further support as a limitation.

You don’t have to use all of these points in your writing, these are just examples of how you can demonstrate critical thinking in your work. Try to use at least a couple in any assignment. Here is an example of how to write these up:

An example write-up

“Depression and anxiety are generally associated with each other (see the meta-analysis by [reference here]). For example, one of these studies was a cross-sectional study [reference here] with 500 undergraduate psychology students. The researchers found that depression and anxiety (measured using the DASS-21 measure) were correlated at  r  = .76, indicating a strong effect. However, this one study is limited in that it used a cross-sectional design, which do not tell us whether depression causes anxiety or whether anxiety causes depression; it just tells us that they are correlated. It’s also limited in that the participants are not a clinical sample, which does not tell us about whether these are clinically co-morbid constructs. Finally, a strength of this study is that it used the DASS-21 which is generally found to be a reliable measure. Future studies would therefore benefit from using a longitudinal design to gain an idea as to how these variables are causally related to one another, and use more clinical samples to understand the implications for clinical practice. Overall, however, the research generally suggests that depression and anxiety are associated. That there is a meta-analysis on this topic [reference here], showing that there is lots of evidence, suggests that this finding is generally well-accepted.”

• Notice how I first found a review paper on the topic to broadly tell the reader how much evidence there is in the first place. I set the scene of the paragraph with the first sentence, and then the last sentence I brought it back, rounding the paragraph off. 
• Notice how I then described one study from this paper in more detail. Specifically, I mentioned the participants, the design of the study and the measure the researchers used to assess these variables. Critically, I then described  how  each of these pieces of the method are disadvantages/strengths of the study. Sometimes, it’s enough to just say “the study was limited in that it was a cross-sectional study”, but it can really show that you are thinking critically, if you also add “… because it does not tell us….”. 
• Notice how I added a statistic there to further illustrate my point (in this case, it was the correlation coefficient), showing that I didn’t just read the abstract of the paper. Doing this for the effect sizes in a study can also help demonstrate to a reader that you understand statistics (a higher-level marking criteria). 

Are these points you can include in your own work?

Thanks for reading,

Share this:

Leave a comment cancel reply.

• Already have a WordPress.com account? Log in now.
• Subscribe Subscribed
• Copy shortlink
• Report this content
• View post in Reader
• Manage subscriptions
• Collapse this bar

• Directories

Writing in Psychology

For most (if not all) your psychology assignments, you'll be required to critically analyse relevant psychological theory and research. If you're just starting out in psychology, you might not know what this involves. This guide will give you an idea of what it means to critically analyse research, along with some practical suggestions for how you can demonstrate your critical-thinking skills. 

What is critical analysis, and why is it important?

Critical analysis involves thinking about the merits and drawbacks of what you're reading. It doesn't necessarily mean tearing apart what you've read-it could also involve highlighting what an author or researcher has done well, and thinking through the implications of a study on the broader research area.

Critical analysis is extremely important in evaluating published research: Psychology studies often build on the limitations of others, and it's important to assess the merits of a study before accepting its conclusions. Furthermore, as a student, your critical analysis of the literature is a way of showing your marker that you've engaged with the field.

What makes critical analysis in psychology different, and how do I critically analyse the literature?

In psychology, critical analysis typically involves evaluating both theory and empirical research (i.e., scientific studies). When critically analysing theory , relevant questions include:

• Does the theory make sense (i.e., is it logical)?
• Can the theory explain psychological phenomena (i.e., what we actually observe in terms of people's behaviour), or does it leave some things unexplained?
• Have any studies been conducted to specifically test this theory, and if so, what did they find? Can we believe this study's conclusions?

In terms of evaluating empirical research , relevant questions include:

• Does the study's research question come logically from the literature the authors have reviewed?
• Are there any issues with the participant sample (e.g., not representative of the population being studied)?
• Do the measures (e.g., questionnaires) actually assess the process of interest?
• Have the appropriate statistical analyses been conducted?
• Do the authors make appropriate conclusions based on their findings, or do they go beyond their findings (i.e., overstate their conclusions)?

Before you critically analyse research, it's important to make sure that you understand what is being argued. We have some resources that can help you get the most out of your reading ( R eading strategies ), as well as some note-taking strategies ( N ote-taking ). The Cornell method might be especially useful, since it involves jotting down your own thoughts/opinions as you're reading, rather than simply summarising information.

As you get more practise critically analysing the literature, you'll find that it starts to feel more natural, and becomes something that you engage in automatically. However, as you're starting out, deliberately thinking through some of the questions in the previous section can help add structure to this process.

What does critical analysis look like?

After you've had a think about the merits and drawbacks of a published piece of work, how do you actually show that you've engaged in critical analysis? Below are some examples of sentences where critical analysis has been demonstrated:

• "Although Brown's (1995) theory can account for [abc], it cannot explain [xyz]."
• "This study is a seminal one in the area, given that it was the first to investigate...".
• "In order to clarify the role of [abc], the study could have controlled for...".
• "This study was a significant improvement over earlier efforts to investigate this topic because...".

What these statements have in common is that they are evaluative : They show that you're making a judgment about the theory or empirical study you're discussing. In general, your marker will be able to tell whether you have engaged in critical analysis by seeing if you've made such statements throughout your work.

Critical analysis in psychology: Some common pitfalls

"The sample size of the study was too small."

Your critiques need to have evidence behind them. Making statements such as this is fine, as long as you follow them up with your reasoning (in this case, on what basis have you decided that the study didn't have enough participants?).

" The study didn't look at participants of [this age/this gender/this ethnic group]."

Traditionally, the area of psychology has tended to focus on WEIRD (white, educated, industrialised, rich, and democratic) individuals. This is certainly an issue for the generalisability of research findings. However, if you make this type of statement, you can further demonstrate your critical-thinking skills by talking about why you think this is an issue for the particular topic you're researching: For example, how might the results of a study differ if a non-WEIRD participant sample had been recruited instead?

Being too critical.

Chances are that if a study is a highly cited one in your area, it probably has some merits (even if it's just that it drew attention to an important topic). You should always be on the lookout for strengths as well as limitations, be they theoretical (i.e., a cohesive, well-elaborated theory) or experimental (i.e., a clever study design).

Other assessments

Writing a creative piece

Writing a critical review

Writing a policy brief

Writing an abstract

Writing an annotated bibliography

Writing in Law

• ANU Library Academic Skills
• +61 2 6125 2972

• The Open University
• Accessibility hub
• Guest user / Sign out
• Study with The Open University

My OpenLearn Profile

Personalise your OpenLearn profile, save your favourite content and get recognition for your learning

About this free course

Become an ou student, download this course, share this free course.

Start this free course now. Just create an account and sign in. Enrol and complete the course for a free statement of participation or digital badge if available.

1 Important points to consider when critically evaluating published research papers

Simple review articles (also referred to as ‘narrative’ or ‘selective’ reviews), systematic reviews and meta-analyses provide rapid overviews and ‘snapshots’ of progress made within a field, summarising a given topic or research area. They can serve as useful guides, or as current and comprehensive ‘sources’ of information, and can act as a point of reference to relevant primary research studies within a given scientific area. Narrative or systematic reviews are often used as a first step towards a more detailed investigation of a topic or a specific enquiry (a hypothesis or research question), or to establish critical awareness of a rapidly-moving field (you will be required to demonstrate this as part of an assignment, an essay or a dissertation at postgraduate level).

The majority of primary ‘empirical’ research papers essentially follow the same structure (abbreviated here as IMRAD). There is a section on Introduction, followed by the Methods, then the Results, which includes figures and tables showing data described in the paper, and a Discussion. The paper typically ends with a Conclusion, and References and Acknowledgements sections.

The Title of the paper provides a concise first impression. The Abstract follows the basic structure of the extended article. It provides an ‘accessible’ and concise summary of the aims, methods, results and conclusions. The Introduction provides useful background information and context, and typically outlines the aims and objectives of the study. The Abstract can serve as a useful summary of the paper, presenting the purpose, scope and major findings. However, simply reading the abstract alone is not a substitute for critically reading the whole article. To really get a good understanding and to be able to critically evaluate a research study, it is necessary to read on.

While most research papers follow the above format, variations do exist. For example, the results and discussion sections may be combined. In some journals the materials and methods may follow the discussion, and in two of the most widely read journals, Science and Nature, the format does vary from the above due to restrictions on the length of articles. In addition, there may be supporting documents that accompany a paper, including supplementary materials such as supporting data, tables, figures, videos and so on. There may also be commentaries or editorials associated with a topical research paper, which provide an overview or critique of the study being presented.

Box 1 Key questions to ask when appraising a research paper

• Is the study’s research question relevant?
• Does the study add anything new to current knowledge and understanding?
• Does the study test a stated hypothesis?
• Is the design of the study appropriate to the research question?
• Do the study methods address key potential sources of bias?
• Were suitable ‘controls’ included in the study?
• Were the statistical analyses appropriate and applied correctly?
• Is there a clear statement of findings?
• Does the data support the authors’ conclusions?
• Are there any conflicts of interest or ethical concerns?

There are various strategies used in reading a scientific research paper, and one of these is to start with the title and the abstract, then look at the figures and tables, and move on to the introduction, before turning to the results and discussion, and finally, interrogating the methods.

Another strategy (outlined below) is to begin with the abstract and then the discussion, take a look at the methods, and then the results section (including any relevant tables and figures), before moving on to look more closely at the discussion and, finally, the conclusion. You should choose a strategy that works best for you. However, asking the ‘right’ questions is a central feature of critical appraisal, as with any enquiry, so where should you begin? Here are some critical questions to consider when evaluating a research paper.

Look at the Abstract and then the Discussion : Are these accessible and of general relevance or are they detailed, with far-reaching conclusions? Is it clear why the study was undertaken? Why are the conclusions important? Does the study add anything new to current knowledge and understanding? The reasons why a particular study design or statistical method were chosen should also be clear from reading a research paper. What is the research question being asked? Does the study test a stated hypothesis? Is the design of the study appropriate to the research question? Have the authors considered the limitations of their study and have they discussed these in context?

Take a look at the Methods : Were there any practical difficulties that could have compromised the study or its implementation? Were these considered in the protocol? Were there any missing values and, if so, was the number of missing values too large to permit meaningful analysis? Was the number of samples (cases or participants) too small to establish meaningful significance? Do the study methods address key potential sources of bias? Were suitable ‘controls’ included in the study? If controls are missing or not appropriate to the study design, we cannot be confident that the results really show what is happening in an experiment. Were the statistical analyses appropriate and applied correctly? Do the authors point out the limitations of methods or tests used? Were the methods referenced and described in sufficient detail for others to repeat or extend the study?

Take a look at the Results section and relevant tables and figures : Is there a clear statement of findings? Were the results expected? Do they make sense? What data supports them? Do the tables and figures clearly describe the data (highlighting trends etc.)? Try to distinguish between what the data show and what the authors say they show (i.e. their interpretation).

Moving on to look in greater depth at the Discussion and Conclusion : Are the results discussed in relation to similar (previous) studies? Do the authors indulge in excessive speculation? Are limitations of the study adequately addressed? Were the objectives of the study met and the hypothesis supported or refuted (and is a clear explanation provided)? Does the data support the authors’ conclusions? Maybe there is only one experiment to support a point. More often, several different experiments or approaches combine to support a particular conclusion. A rule of thumb here is that if multiple approaches and multiple lines of evidence from different directions are presented, and all point to the same conclusion, then the conclusions are more credible. But do question all assumptions. Identify any implicit or hidden assumptions that the authors may have used when interpreting their data. Be wary of data that is mixed up with interpretation and speculation! Remember, just because it is published, does not mean that it is right.

O ther points you should consider when evaluating a research paper : Are there any financial, ethical or other conflicts of interest associated with the study, its authors and sponsors? Are there ethical concerns with the study itself? Looking at the references, consider if the authors have preferentially cited their own previous publications (i.e. needlessly), and whether the list of references are recent (ensuring that the analysis is up-to-date). Finally, from a practical perspective, you should move beyond the text of a research paper, talk to your peers about it, consult available commentaries, online links to references and other external sources to help clarify any aspects you don’t understand.

The above can be taken as a general guide to help you begin to critically evaluate a scientific research paper, but only in the broadest sense. Do bear in mind that the way that research evidence is critiqued will also differ slightly according to the type of study being appraised, whether observational or experimental, and each study will have additional aspects that would need to be evaluated separately. For criteria recommended for the evaluation of qualitative research papers, see the article by Mildred Blaxter (1996), available online. Details are in the References.

Activity 1 Critical appraisal of a scientific research paper

A critical appraisal checklist, which you can download via the link below, can act as a useful tool to help you to interrogate research papers. The checklist is divided into four sections, broadly covering:

• some general aspects
• research design and methodology
• the results
• discussion, conclusion and references.

Science perspective – critical appraisal checklist [ Tip: hold Ctrl and click a link to open it in a new tab. ( Hide tip ) ]

• Identify and obtain a research article based on a topic of your own choosing, using a search engine such as Google Scholar or PubMed (for example).
• The selection criteria for your target paper are as follows: the article must be an open access primary research paper (not a review) containing empirical data, published in the last 2–3 years, and preferably no more than 5–6 pages in length.
• Critically evaluate the research paper using the checklist provided, making notes on the key points and your overall impression.

Critical appraisal checklists are useful tools to help assess the quality of a study. Assessment of various factors, including the importance of the research question, the design and methodology of a study, the validity of the results and their usefulness (application or relevance), the legitimacy of the conclusions, and any potential conflicts of interest, are an important part of the critical appraisal process. Limitations and further improvements can then be considered.

Clinical Psychology: Critical Appraisal and Evaluation Skills

• Getting Started
• Finding Books
• Finding Journals
• Finding Articles
• Evidence-based practice
• LinkedIn Learning
• Psychological Testing
• Images and Photographs
• Systematic literature searching
• Clinical trials
• Critical Appraisal and Evaluation Skills
• Useful Websites
• Study and research skills
• Referencing
• Training and Support
• News and Media Resources
• Viewing for your subject
• Decolonising Grey Literaure
• Generative AI
• Scientific Writing Centre
• Sustainability

Evaluating your results

Just because material has been published in a journal or appears on a wesbite, this does not mean it is suitable for your purpose.  The Open University, in its SAFARI guide to information skills , suggests the following criteria, which you can use for the web and for printed material:

P resentation - is the information presented in a clear and readable way? Are there relevant diagrams and photographs?  Is it written objectively or is it emotive?

R elevance - is it relevant and appropriate for your needs? Does it cover the countries or regions which interest you? Does it cover all aspects of your topic?

O bjectivity - is it balanced or is there some bias? Can you easily establish who the authors are and what their authority might be? Are there vested interests behind the website?  Is it trying to sell you something?

M ethod - how was the information gathered together? Are the methods clearly stated?  Ask yourself basic questions about sample size, use of control groups. questionnaire design etc.

P rovenance or Authority - who or what originated the information and are they reliable sources? Are the authors acknowledged experts in this area? What else have they published on this topic?  Do you they belong to well-known institutions? If you're looking at a journal article, is it from a peer-reviewed journal? If it is a website, did you find the link on a trusted site, such as NHS Evidence or a professional body or a university?

T imeliness - is the information up-to-date and can you tell if it has been superseded? Is it clear when the website was produced?

Critical Appraisal

Critical appraisal is the process of carefully and systematically examining research to judge its trustworthiness, and its value and relevance in an evidence-based context. This is of high relevance for medicine, biomedicine and health care contexts. Here are online courses, websites and checklists to help you appraise the material you have retrieved.

• Health Knowledge: Finding and Appraising the Evidence Online course from HealthKnowledge. Aimed at public health practitioners who are new to evidence based practice and critical appraisal. There are six modules including an overall introduction to critical appraisal, systematic reviews, economic evaluations and making sense of results.
• Critical Appraisal Checklists From SIGN. This is a basic course intended for those who need critical appraisal skills for their job or to further their professional education. You may also find it useful if you have not done critical appraisal for some time and want to revive your skills.
• Critical Appraisal Skills Programme (CASP) Oxford-based CASP runs workshops and provides a wealth of checklists and training materials on critical appraisal. Useful, downloadable checklists for different study types, such as systematic review or randomised clinical trials.
• Centre for Evidence-Based Medicine: Critical Appraisal CEBM based at Oxford University, provides worksheets, checklists and other tools to use for critical appraisal. The CEBM also provides other tools for EBM, such as likelihood ratios and the CATmaker, a computer-assisted critical appraisal tool.

Evaluating grey literature

• AACODS Checklist

Grey literature has not been through any sort of peer review process.  Therefore it is particularly important that you evaluate material very carefully to decide whether to use it.

The AACODS checklist is designed to enable evaluation and critical appraisal of grey literature.

A Authority

O Objectivity

S Significance.

It was prepared at Flinders University and there is a very helpful annotated checklist available.

• << Previous: Clinical trials
• Next: Useful Websites >>
• Last Updated: Aug 19, 2024 10:16 AM
• URL: https://lancaster.libguides.com/clinpsy

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 26 August 2024

Moral attitudes towards effort and efficiency: a comparison between American and Chinese history

• Amber X. Chen   ORCID: orcid.org/0000-0003-3359-9075 1 ,
• Shaojing Sun   ORCID: orcid.org/0000-0003-2189-5336 2 &
• Hongbo Yu   ORCID: orcid.org/0000-0002-3384-7772 1  

Humanities and Social Sciences Communications volume  11 , Article number:  1085 ( 2024 ) Cite this article

Metrics details

• Language and linguistics

In some cultures, merely exerting effort is considered virtuous, even when the effort is inefficient. Our study examines how this moral attitude towards effort (relative to efficiency) has evolved historically across two distinct sociopolitical and linguistic contexts: the People’s Republic of China and the United States, using natural language processing techniques. Specifically, two formal political corpora were used—the People’s Daily (1950–2021) and the Congressional speeches for the U.S. (1873–2011). We developed dictionaries for each concept based on pre-trained word embedding models in both languages. Moral attitudes towards effort and efficiency were calculated on a year-by-year basis as the cosine similarity between the dictionaries of these concepts and an existing dictionary of morality. We benchmarked the fluctuations of moral attitude towards inefficient effort against critical historical events in the two countries. Additional time series analysis and Granger tests revealed the association and potential directionality between the evolution of moral attitude towards inefficient effort and critical socio-cultural variables such as collectivism and cultural looseness. Our research sheds light on the historical and socio-cultural roots of moralization of effort and has implications for historical psychology research on moral attitudes.

Introduction

Consider this scenario: A newspaper company recently adopted an Artificial Intelligence (AI) system capable of producing articles indistinguishable from those written by human journalists. The sale of the newspaper is unaffected after the company relies entirely on the AI system. Consequently, the two journalists employed by the company now benefit from indefinite paid time off (PTO). During this period, one journalist continues to write two articles daily as before, while the other spends their days lounging on the couch and watching TV. Who would you instinctively consider more morally praiseworthy? Recent work on moralization of inefficient effort has suggested that across multiple cultures, people tend to morally praise the individuals who exert efforts even when the efforts produce no discernible outcomes (Amos et al., 2019 ; Bigman and Tamir, 2016 ; Celniker et al., 2023 ; Ess and Burke, 2022 ; Fwu et al., 2014 ). In laboratory settings, individuals tend to infer positive moral attributes from a mere cue of effort (Amos et al., 2019 ; Celniker et al., 2023 ). Utilizing vignettes across various domains—including paid employment and partner choice—in different cultures, this line of research demonstrates that individuals who exert greater effort, even in tasks that appear meaningless and completely replaceable by less effortful alternatives, are perceived as more moral and deserving of higher monetary reward. Such a preference for effort has behavioral consequences. For example, the demonstration of effort exerted by fundraisers has been shown to increase participants’ willingness to donate to the charity; participants were more likely to pay higher salaries to employees who exhibited more effort, even when the actual outcome or end product remains the same (Celniker et al., 2023 ).

The tendency to morally praise effort regardless of outcomes is observed not only in laboratory settings, but also seems to be a common theme in the workplace across many developed economies, where many high paying and prestigious jobs are deemed useless and meaningless (or “bullshit jobs”) even by the employees holding those positions (Celniker et al., 2023 ; Graeber, 2018 ). A sociology work suggests that wealthy people in the U.S. tend to use their “hard work” to morally justify their advantageous socioeconomic status, while intentionally or unintentionally downplaying or ignoring the structural privilege they have (Sherman, 2017 ).

Interestingly, the consequences of moralization of inefficient effort may differ at the societal level compared with the individual level. While the dissociation between effort and efficiency can lead to negative individual consequences, such as diminished learning outcomes (Kirk-Johnson et al., 2019 ) and a lack of sense of meaningfulness (Graeber, 2018 ), it can serve a strategic role at the societal level. By valuing effort irrespective of outcomes, societies might encourage persistence and foster resilience in the face of challenges and setbacks. Such work ethics can pave the way for long-term innovation (Kundro, 2022 ) and stimulate economic growth (Becker and Woessmann, 2009 ), even if immediate results are not always evident. In short, the moralization of inefficient effort, in its ambivalent nature and its relationship to the societal environment, is more than merely justifying one’s effortful actions but rather is linked to the evolution of moral systems and their intricate interplay with broader socio-cultural contexts.

Delving deeper into the moralization of inefficient effort, we could find its roots in a society’s history and culture. Historically, many cultures have celebrated virtues such as diligence, perseverance, and hard work, often rooted in religious and/or philosophical teachings (Fitouchi et al., 2022 ). For instance, the Protestant Work Ethic (PWE) emphasizes labor and discipline as symbols of personal virtue (Weber, 1905 ). Empirical studies have shown that societies with a strong emphasis on PWE tend to value individual effort and achievement, even in secular contexts (Furnham, 1984 ; Uhlmann and Sanchez-Burks, 2014 ). Similarly, Confucian values from East Asia stress the importance of industriousness and constant self-improvement as moral obligations (Hwang, 2012 ). For example, in societies influenced by Confucian values (e.g., China, Korea), the education system is designed to reward effort-driven practices such as intensive memorization for exams. Here, students might internalize the belief that effort is a desirable goal in and of itself, a belief often reinforced by parents and educators (Chen, 2023 ). These cultural norms may have shaped societies to prioritize the exertion of effort itself over its outcomes, viewing genuine hard work as intrinsically valuable, regardless of its direct utility or efficiency.

Furthermore, differences also manifest in the perception and pursuit of efficiency. In the Confucian value system, equality is prioritized over individual efficiency (Poznanski, 2017 ). In contrast, the PWE not only promotes hard work but also views efficiency and individual achievement as means to achieve personal salvation (Weber, 1905 ). For example, compared to American families, Japanese families often promote task involvement through interpersonal cooperation rather than competition, often avoiding evaluations based on individual performance (Holloway, 1988 ).

In both cultures, the moral underpinnings of effort and efficiency are not just abstract values. They influence educational practices, social judgments, and even economic policies. However, traditional survey and experimental methods have limited capacity in revealing the historical roots and evolution of these moral attitudes at the societal level (Atari and Henrich, 2023 ; Muthukrishna et al., 2021 ).

Here, we aim to trace the moral values of effort and efficiency within two distinct sociopolitical and linguistic contexts - the United States and the People’s Republic of China. We explore how these values evolved over time and how they interacted with key historical and economic trends. Our approach integrates natural language processing techniques and data from historical and contemporary sources, providing a robust and nuanced analysis of these complex moral constructs.

The historical and cultural perspectives

Rarely does any moral attitude remain static throughout history (Muthukrishna et al., 2021 ). Investigating the evolution of moral attitudes and the societal factors that influence and result from their changes can advance our understanding of cross-cultural and historical variations in human cognition. Studying these variations and their historical origins can provide insight into the fundamental aspects of human society, such as its values, ethics, and customs (Atari and Henrich, 2023 ; Schulz et al., 2019 ).

In the study of the evolution of moral attitudes, text corpora serve as an invaluable gateway into history, as it documents the collective minds of the human societies in the past (Muthukrishna et al., 2021 ). With the recent trend of applying computational linguistic models to historical text corpora, psychologists can reveal theoretically intriguing and hitherto neglected historical evolutions of various cultural and social constructs. For example, previous studies have identified a rise in individualism and the loosening of cultural norms within U.S. society over the past two centuries (Greenfield, 2013 ; Jackson et al., 2019 ). Adopting a similar approach, Choi and colleagues track the changes in the use of threatening language in American English books, and show that these changes correspond with actual threats in the U.S. history (Choi et al., 2022 ). Other studies have documented the surge of language use that is associated with cognitive distortion, such as overgeneralizing and mislabeling, in American English, Spanish, and German books since the 1980s, and shown that this pattern is not driven by linguistic shifts (Bollen et al., 2021 ).

This line of work typically relies on the frequency of a set of words (i.e., a dictionary) that represents the psychological construct in question in large language corpora (e.g., Google Book) across time (Michel et al., 2011 ). A second approach for studying historical psychology employs co-occurrence analysis. By rating language proximal to specific terms based on psychological constructs, such as positivity and stereotypes, researchers can gain insights into the increasing negativity of attitudes toward older adults across time and ambivalent age stereotypes across cultures (Mason et al., 2015 ; Ng and Chow, 2021 ). More recent research leverages a natural language processing (NLP) technique, word embeddings, to measure semantic similarities between psychological constructs of interests, and to examine the historical evolution of such semantic similarities. This approach is based on the distributional hypothesis from linguistics, which posits that the semantic contexts around words capture their meaning in terms of cultural associations (Sahlgren, 2008 ) and has been shown to align well with human implicit bias data (Caliskan et al., 2017 ). Existing research has demonstrated its usefulness in capturing historical dynamics of social group representation (Cao et al., 2024 ; Charlesworth et al., 2022 ), identifying polarized framing of immigrants between political parties’ speeches (Card et al., 2022 ), tracking the decline of gender and ethnic stereotypes in American historical corpora (Garg et al., 2018 ; Jones et al., 2020 ), as well as the consistency of gender stereotypes across child and adult corpora (Charlesworth et al., 2022 ).

Similar to the various psychological constructs reviewed above, cross-cultural diversity in moral attitudes in today’s world is likely a consequence of distinct historical evolution trajectories in different geographic, linguistic, and social contexts (Henrich, 2020 ). Existing NLP-based historical psychological research has largely been focused on the so-called Western, Educated, Industrial, Rich, and Democratic (WEIRD) societies, and has primarily relied on English texts. While there are exceptions that use non-English sources such as German and French (Baumard et al., 2022 ; Bollen et al., 2021 ; Martins and Baumard, 2020 ), the Eurocentric focus in previous research has constrained the exploration of broader cultural and historical diversity. Here, we examine and compare the historical evolution of moral attitudes toward effort and efficiency in modern history of China and the U.S., two societies significantly differ in their political and economic institutions, popular cultures, ideologies, and languages. This allows us to identify similar and distinct societal and cultural drivers behind the different evolution trajectories. The rationale for focusing on these two cultures and languages is twofold. From a theoretical perspective, China and the U.S. are known to differ in important political structures, cultural values, and social norms, such as collectivism/individualism and cultural tightness/looseness (we acknowledge that cultural and social norms also differ significantly within each country) (Talhelm and English, 2020 ). More importantly, the two cultures also differ in their dominant work ethic (Amos et al., 2019 ; Celniker et al., 2023 ; Zhang et al., 2012 ): while the U.S. work ethic is strongly influenced by Protestant Work Ethic (PWE) beliefs (industrious, ambitious, hardworking, intrinsically motivated), the Chinese work ethic reflects Confucianism values (e.g., hard work, diligence, frugality, and the love of education). It is thus theoretically interesting to examine how the evolution of moral attitudes towards effort and efficiency/productivity differs in these two distinct cultures (e.g., as the economic performance of the society fluctuates). From a pragmatic perspective, Chinese is one of the most widely used non-alphabetical texts, yet it is digitally disadvantaged and underrepresented in natural language processing research (Zaugg et al., 2022 ). The text corpus that we adopted in this study spans the entire history of the People’s Republic of China. Sharing the diachronic word embeddings based on this corpus will provide materials for future NLP-based historical psychology research.

Specifically, we set out to address three novel research questions and goals: (1) developing a computational linguistic tool targeting moral attitudes toward effort and efficiency expressed in Chinese and English texts; (2) characterizing the historical evolution trajectories of moral attitudes toward efforts and efficiency in modern Chinese and U.S. histories; and (3) understanding the social and cultural antecedents and consequences of the moral attitudes toward efforts and efficiency in both cultures.

Dictionary development

To capture the concepts of effort and efficiency in text data, we followed the procedure of several recent studies and developed dictionaries (i.e., word lists) representing the concepts of effort and efficiency. To this end, we employed a combination of pre-trained language models and human ratings, following the procedures of previous studies (Choi et al., 2022 ; Jackson et al., 2019 ; Lin et al., 2022 ). The process consisted of the following four main steps.

First, we identified English and Chinese seed words related to effort and efficiency from WordNet (Miller, 1995 ), a comprehensive lexical database and thesaurus.

Second, we computed the mean vector coordinates of the seed words using models pre-trained on datasets including Google News (Mikolov et al., 2013 ), Wikipedia (Bojanowski et al., 2017 ), and Twitter (Pennington et al., 2014 ) for English, and Renmin Web (people.cn), Wikipedia, and Weibo for Chinese (Li et al., 2018 ). This way, we located the seed words in high-dimensional semantic spaces and identified semantically similar terms. The selection of these pre-trained models was based on their coverage across diverse platforms, spanning traditional mass media, social media, and encyclopedism references. The rationale of using a more diverse set of corpora (as opposed to the corpora used for the main analyses) for dictionary development is to obtain a representative and generalizable linguistic indicator of the underlying constructs. The corpora used for the main analysis (i.e., People’s Daily and Congressional speeches) was chosen partly due to their well-preserved temporal information and historical continuity. However, these corpora are limited in the contents, genre, and scope. Therefore, we believe that basing our dictionary development on a more comprehensive linguistic source is more appropriate and useful for future research.

Third, we extracted the 50 words closest to the mean coordinates of the seed words in each pre-trained model. These words were considered closely related to the concepts. None-words, duplications, and very low frequency words were filtered out before the next step.

Finally, two native speakers of English and two native speakers of Chinese from the research team were trained to evaluate the remaining candidate words for conceptual relevance to the concept in question on a scale of 1 ( not at all relevant ) to 5 ( extremely relevant ). Then, we selected the words that had an average rating higher than 3. Within this selection, certain words were excluded despite meeting the rating criterion due to reasons such as being very similar to other terms in the dictionaries, a low frequency of occurrence, consisting of multiple tokens, or ambiguous interpretations. The inter-rater reliability for the relevance ratings, assessed using the Intraclass Correlation Coefficient (ICC), demonstrated good reliability for effort in both English (ICC = 0.88) and Chinese (ICC = 0.76) and moderate reliability for efficiency in English (ICC = 0.67) and Chinese (ICC = 0.51). We acknowledge that the ICC for the Chinese efficiency words was not great and therefore, the interpretations related to efficiency in the Chinese context should be taken with caution. The final dictionaries consisted of 10 words each for effort and efficiency in English, while the Chinese versions incorporated 13 for effort and 10 for efficiency. For the effort dictionary in Chinese, we include three additional words because four words positioned tenth in the relevance ratings which are all highly relevant to the concept and the overall cosine similarity between groups of words should not be significantly impacted, as the embedding process inherently normalizes these frequency effects. Some examples of the effort dictionary are “effort”, “toil”, and “perseverance”. The efficiency dictionary comprises words like “efficiency,” “productivity”, “effective”, “economical”, and “profitable”. Some examples of the Chinese effort dictionary are “努力”, “力求”, and “尽力”, and some examples of the Chinese efficiency dictionary are “效益”, “效率”, and “效用”. The complete list of candidate words and their ratings can be found in Supplementary Tables 1 – 4 .

Calculating the moral values of effort and efficiency concepts

To represent positive and negative moral values, we used the word lists (i.e., dictionaries) from the Moral Foundations Dictionary 2.0 (Frimer et al., 2019 ), which is based on the Moral Foundations Theory (Graham et al., 2013 ). Some examples of these words, which are rooted in moral foundations (care, fairness, loyalty, authority, and sanctity), include “compassion”, “equality”, “loyalty”, “respect”, and “purity”. The Chinese Moral Foundations Dictionary was adapted and translated from English to Chinese. The ability of this Chinese version to effectively capture moral values in Chinese texts has been validated in previous work (Chen et al., 2024 ; Garten et al., 2018 ). We acknowledge that various versions of the Moral Foundations Dictionary exist, both in English and Chinese (e.g., the Extended Moral Foundations Dictionary, (Hopp et al., 2020 ); the Chinese Moral Foundations Dictionary 2.0, (Cheng and Zhang, 2023 )). We chose MFD 2.0 over MFD 1.0 because the former offers a more comprehensive set of moral terms (Kennedy et al., 2021 b). MFD 2.0 contains five moral foundations, which is comparable to the Chinese moral foundations dictionary we used. We noted that different versions of Moral Foundations Theory have slightly different numbers and demarcation of the exact moral foundations (Atari et al., 2023 ; Graham et al., 2013 ; Hofmann et al., 2014 ). However, in this study, we did not test hypotheses specific to any single foundation. For the Chinese dictionary, the Chinese Moral Foundation Dictionary 2.0 is one of the most recent endeavors in creating a computational linguistic measure of morality in Chinese text (Cheng and Zhang, 2023 ). However, the researchers who developed this dictionary specifically argued that their dictionary did not distinguish between positively and negatively valenced moral words within each foundation, a feature that would be critical to our method (see below for details). Our choice of the dictionaries was based on the considerations of compatibility and ease of comparison between the English and Chinese analysis. We demonstrated the validity of the moral dictionaries in a validation analysis, where we benchmarked the fluctuations of the moral attitudes toward the U.S.S.R. in the Chinese and American corpora (see Dictionary Validation in the “Results” section).

Data collection and model training

To examine the societal attitudes toward effort and efficiency, we compiled two datasets from the U.S. Congressional speeches from 1873 to 2011 and People’s Daily from 1950 to 2021. The United States Congressional Record includes all speeches delivered on the floors of the United States House of Representatives and the United States Senate, encompassing a wide range of discussions and debates on various policy issues, reflecting the diverse views of American lawmakers. The content includes legislative activity, committee reports, member remarks, petitions, and memorials. We use the bound edition, a permanent, edited version of the daily records of congressional proceedings from the 43 rd to the 111 th Congress. This corpus has provided a reliable source to study sentiments and partisanship changes in the political context over time. For example, the congressional speeches have become increasingly positive and sentimental (Tucker et al., 2020 ). While the partisanship in the speeches remained stable and has become more polarized in recent decades (Enke, 2020 ; Gentzkow et al., 2019 ), parties not in power used more moral language (Wang and Inbar, 2021 ).

People’s Daily is the official outlet of the Chinese Communist Party and reflects a crucial perspective on the Party’s official stance on effort and efficiency in the context of their political and socio-economic environment. It features government policies, economic plans, societal issues, notable events, and official statements. Despite the structural changes in 1956, including the shift from traditional Chinese to simplified Chinese characters, the change in writing format from vertical to horizontal, and the expansion from 8 to 12 pages, the content and genre of the corpus remain stable over time. In that sense, this corpus has good temporal and genric continuity and covers almost the entire period of the People’s Republic of China.

The corpora were split by year and trained into word2vec models using python gensim package with its default hyperparameter settings. The U.S. Congressional speeches corpus has a mean token count of 13.4 million per year with a standard deviation of 6.8 million, ranging from 867,210 to 29,252,016 tokens. The People’s Daily corpus has a mean token count of 12.3 million, with a standard deviation of 5.9 million, ranging from 5,292,965 to 31,038,064 tokens. Year-by-year distribution of token counts across years is displayed in Supplementary Fig. 1 . We employed the following key model hyperparameters: a vector size of 100, which determines the dimensionality of the word vectors; a window size of 5, which defines the maximum distance between the current and predicted word within a sentence; a minimum count of 5, meaning that words occurring less frequently than this threshold were ignored; a negative sampling rate of 5, which specifies the number of “noise words” to be drawn for each target word during training; and a number of iterations (epochs) as 5. The models were trained using the Skip-gram architecture, which is effective for capturing the context of a word in a large corpus. Word2Vec models provided a robust framework for generating meaningful word embeddings, facilitating our subsequent analyses of moral values in the textual data.

Measuring moral values in text corpora

To assess the moral values of effort and efficiency in the above corpora, we adopted the word embeddings bias method, a natural language processing technique widely used in previous studies that quantifies changes in stereotypical bias throughout the U.S. history (Charlesworth et al., 2022 ; Garg et al., 2018 ). First, our analysis required dictionaries representing the concepts of interest, including effort, efficiency, and moral values, detailed in Dictionary Development section above.

Second, to measure bias in word embeddings, we computed the strength of association (or similarity) between the dictionary of a concept (e.g., effort) and the dictionary of the evaluative words (e.g., moral virtues). Specifically, we calculated the average cosine similarity between each concept (effort or efficiency) and the evaluative words, as shown in the formula below:

Here, Sim ( effort, virtue ) represents the average cosine similarity between the effort concept and positive moral values (virtue), while M and N denote the total number of words in the two dictionaries. We used the same approach to calculate the negative moral values (vice) of effort and efficiency. Moral attitude towards effort (or efficiency) is defined as the difference between the positive and negative moral values of effort (or efficiency). Hereafter, we use Effort and Efficiency to denote the moral attitudes towards effort and efficiency, respectively. Moral attitude toward inefficient effort is defined as the difference between the positive moral value of effort and the positive moral value of efficiency. Hereafter, we use Inefficient Effort to denote the moral attitudes towards inefficient effort. This operationalization construes Inefficient Effort as the bias in a society’s positive moral attitudes (i.e., virtue) toward effort relative to the positive moral attitudes toward efficiency. In other words, if a society values efforts precisely because of the outcomes the efforts produce (i.e., efficient effort), then the positive moral attitudes toward efforts should be comparable to the positive moral attitudes towards efficiency, in which case, the bias as we define it would be close to zero. On the other hand, if the bias is positive, then this implies that in such a society, efforts are valued over and beyond the outcome they produce. We refer to this additional positive moral value of effort as the moral value toward inefficient effort (i.e., efforts that do not produce outcomes). This “bias” approach is a standard practice in dictionary-based NLP work, such as stereotypes of groups (i.e., difference in semantic similarity between group names and positive/negative trait words (Charlesworth et al., 2022 , 2024 )), women’s occupation bias (i.e., difference in semantic similarity between occupation names and women/men words (Garg et al., 2018 )).

Other variables

To investigate the relationships between socio-cultural values and the moral values of effort and efficiency across time, we obtained metrices of several socio-cultural values, including individualism/collectivism, cultural tightness/looseness, and economic performance (i.e., per capita Gross Domestic Product, GDP).

Individualism/collectivism

We derived a text-based measure of year-by-year individualism and collectivism from the two countries based on the Google Ngram word frequency data, which provides insight into the prevalence of individualistic and collectivistic terms in published texts over time. The English and Chinese dictionaries are taken from existing studies in historical psychology (Greenfield, 2013 ; Zeng and Greenfield, 2015 ).

Tightness/looseness

Cultural tightness and looseness refer to the extent to which a culture adheres to social norms and tolerates deviant behavior (M. Gelfand, 2019 ; M. J. Gelfand et al., 2006 ; Jackson et al., 2019 ). Tight cultures have strict norms and little tolerance for deviations, whereas loose cultures are more permissive and flexible. Recent advances in research have facilitated the measurement of these constructs in textual content, examining the prevalence and context of norm-related words in various English corpora (Jackson et al., 2019 ). We translated the Chinese dictionary from the English version. A similar Google Ngram-based method as above was used.

We acknowledge that Google N-gram is only an imperfect reflection of the true cultural values of a society (Schmidt et al., 2021 ). We compared the cultural values (individualism, collectivism, looseness, and tightness) based on Google N-gram and another, generically more balanced English corpus (i.e., Corpus of Historical American English, COHA) using ARIMA regression. During the period of history we studied, all the four cultural values based on these two corpora were highly correlated (see Supplementary Methods for detail; Supplementary Fig. 2 ). We decided to use Google N-gram here because, to the best of our knowledge, it is still the most accessible and comprehensive corpus in Chinese. Using Google N-gram as the source of the cultural variables makes the results of the two countries more comparable. More efforts are needed to create more stable and balanced corpora in Chinese and other non-English languages.

GDP per capita

To control economic factors that may influence the attitudes toward effort and efficiency, we utilized GDP per capita data. This information was obtained from the Maddison Project, a comprehensive dataset that provides historical GDP per capita estimates for various countries (Bolt and van Zanden, 2020 ). By including GDP per capita in our analysis, we aimed to account for potential economic influences on the evolution of the examined concepts.

Statistical analysis

Descriptive analysis.

We first displayed the long-term trends in the moral values of effort and efficiency based on the U.S. Congressional speeches and People’s Daily .

Null models as baseline

To establish reference points for linguistic fluctuations over time and demonstrate the moral significance of the concepts, we ran 10,000 simulations. In these simulations, we substituted effort- and efficiency-related words with randomly selected words and extracted their semantic similarities with moral evaluative words (virtue and vice dictionaries, respectively). It yielded a distribution of the association between 10,000 random sets of words (the same size as the effort and efficiency dictionaries, respectively) and the moral evaluative words for each year, forming a 95% confidence interval for these random associations. We then compared these with the moral associations with the concepts we are interested in (Bollen et al., 2021 ; Jones et al., 2020 ).

Bayesian change point detection

To assess the convergent validity of the moral measures, we employed Bayesian Change Point Detection, a statistical technique designed to identify shifts in data patterns by estimating the probability of change points at each timepoint. This approach allows us to pinpoint the precise time points at which moral value shifts occur and examine their correspondence with key historical turning points in each country. Utilizing this method, we can establish a connection between our findings and the broader historical context. The bcp package in R was used for change point analysis (Erdman and Emerson, 2008 ).

Time series

To estimate the long-term trends of moral values associated with effort and efficiency in the U.S. Congressional speeches and People’s Daily , we applied the Autoregressive Integrated Moving Average (ARIMA) modeling, utilizing the auto.arima function from the forecast package in R (Hyndman and Khandakar, 2008 ), to the yearly association data. ARIMA is a time-series model that allows us to account for both the temporal dependencies and random fluctuations within the data, while controlling for GDP per capita.

Granger test

To further analyze the directionality between moral values and cultural changes, we employed Granger causality test to determine whether moral values of effort and efficiency precede cultural values, or vice versa. To ensure the stationarity of the data, we extracted the time series data from ARIMA models. To determine the optimal lag length for the Granger tests, we compared models with lags ranging up to 10 years and selected the one with the lowest Akaike Information Criterion (AIC). The Granger causality test was performed using the grangertest function from the lmtest package in R (Zeileis and Hothorn, 2002 ).

Dictionary validation

To validate the moral measures in the U.S. Congressional speeches and People’s Daily , we employed the embedding bias method to examine the moral values of the Union of Soviet Socialist Republics (U.S.S.R.) in the two corpora. It is expected that the evolution of moral attitudes toward the U.S.S.R. fluctuates in response to known events in the two countries diplomatic relationships with the U.S.S.R. As illustrated in Fig. 1A , the moral attitudes towards the U.S.S.R. in the U.S. Congressional speeches showed fluctuations in the 1920s and 1930s, with a prominent peak coinciding with the establishment of the U.S.-U.S.S.R. diplomatic ties in 1933, and remained relatively high during World War II, during which the U.S. and the U.S.S.R. were allies against Nazi Germany. However, the moral attitudes towards the U.S.S.R. quickly declined after the end of the war, when the tension between the Socialist Bloc and the West accelerated and the Cold War was in the corner. Of note, Winston Churchill’s Iron Curtain Speech was delivered in 1946 and the Gouzenko Affair (a high-profile U.S.S.R. spy at the Soviet Embassy in Ottawa) went viral in the media between 1945 and 1946. Moving forward, the moral attitudes towards the U.S.S.R. remained negative in the Congressional speech corpus, despite a brief uptick coinciding with Khrushchev’s 1959 visit to the U.S. The moral attitudes towards the U.S.S.R. in the People’s Daily also notably fluctuated in response to historical events, as shown in Fig. 1B . For instance, there have been marked declines since the 1960s, during which the moral attitudes dropped from positive to negative. This downturn reached its lowest point in 1971 following the Sino-Soviet border conflict in 1969. However, there was a resurgence in moral attitude towards the U.S.S.R. in 1980s along with the resumption of diplomatic relations, culminating in a peak when Soviet Communist leader Mikhail Gorbachev paid a state visit to China in 1989 for the Sino-Soviet Summit.

Historical events are marked with vertical dashed lines and annotated with black boxes.

Historical trends of moral attitudes toward effort, efficiency, and inefficient effort

We first showed the descriptive trajectories of the evolution of moral attitudes toward efforts, efficiency, and inefficient effort over the last seven decades of Chinese history and nearly 14 decades of U.S. history, in their respective text corpus. Figures 2 and 3 display the historical evolutions of moral attitudes towards effort, efficiency, and inefficient effort. The results were based on the average of all moral foundations in the dictionary. Results based on each moral foundation are shown in Supplementary Fig. 3 . All the years discussed below in this section were the first three highest probabilities determined by Bayesian Change Point Detection (see Supplementary Fig. 4 for the time series of posterior estimates) (Erdman and Emerson, 2008 ). This method was employed to detect substantial shifts in a data sequence, offering a probabilistic framework to identify the number and location of these changes by calculating the posterior probability of a change at each time point.

A Moral attitudes towards effort and efficiency. B Moral attitudes towards inefficient effort. The shaded regions denote the 95% confidence interval derived from a null model, comprising 10,000 sets of random words matched in length to the English effort/efficiency dictionaries (see the Null Model as Baseline section in Methods ).

A Moral attitudes towards effort and efficiency. B Moral attitudes towards inefficient effort. The shaded regions denote the 95% confidence interval derived from a null model, comprising 10,000 sets of random words matched in length to the Chinese effort/efficiency dictionaries (see the Null Model as Baseline section in Methods ).

As displayed in Fig. 2A , in the U.S. Congressional speeches, the moral attitude towards effort is predominantly positive. An upward surge occurred in 1957 when it increased beyond the predictions of the null model, a year of substantial change identified by Bayesian Change Point Detection. During a period of mostly uprisings in the moral attitude towards effort from 1964 to 2010, there is a marked decline in 1998, coinciding with the Asian Financial Crisis. The trajectory of efficiency mirrors this historical pattern, with a less significant increasing trend starting from 1964. Between 1964 to 2010, 1996 stands out as a year that exhibits an abrupt dip in its moral values. Together, before the 1960s, effort and efficiency appeared relatively indistinguishable from each other and overlapped with the predictions of the null models. Post-1960s, a discernible moralization of these terms emerges, with effort consistently receiving relatively more positive evaluations than efficiency. The trajectory of moral attitude toward inefficient effort (Fig. 2B ) is mainly positive but marked by fluctuations. Before the 1940s, the trajectory largely overlaps with the null model predictions. Since the 1940s, the trajectory has been consistently above the null model predictions.

An important feature of the U.S. Congress is its two-party system. Throughout history, the Republicans and the Democrats represent different social values and ideologies. A question thus arises - is the trajectory of moral attitudes we observed above equally present in the narratives of both parties? To explore this, we trained new embedding models separately on the speeches of Democratic and Republican legislators. We then calculated the cosine similarity between the moral dictionary and the effort/efficiency dictionaries as we did above (see Supplementary Methods for detail). The trajectories of moral attitudes towards effort, efficiency, and inefficient effort based on Democratic and Republican speeches are displayed in Supplementary Fig. 5 . In essence, the trajectories based on the two parties largely overlap with each other throughout the years we studied. Correlation analysis (after removing temporal dependency using ARIMA) showed that the trajectories based on the two parties are strongly and positively correlated (effort: r  = 0.31; efficiency: r  = 0.46; inefficient effort: r  = 0.46; all p s < 0.001). Moral attitudes towards effort and efficiency may also vary geographically at the state or even county level. Recent NLP-based work has begun to take into account finer-grained geographical variations in psychological outcomes (e.g., Atari and Henrich, 2023 ; Y. Chen et al., 2024 ; Lin et al., 2022 ). However, branching out to this question is beyond the scope of the present study. Since there is no equivalent partisanship in the Chinese corpus, we could not carry out a similar analysis there.

As shown in Fig. 3A , in People’s Daily , effort consistently demonstrates a stronger association with virtue words compared to vice words. Notably, the trajectory of moral attitude toward effort is consistently above the levels predicted by the null model. Conversely, while efficiency maintains its positive valuation across most data points, it consistently falls within the 95% confidence intervals of the null model except for the duration from the 1990s to 2000s, suggesting that the observed positive moral values is only distinguishable from mere random occurrences during this specific timeframe. This indicates that the moralization of efficiency may not be as distinct or robust as that of effort in this context. Importantly, in contrast to the U.S. Congressional speeches where both effort and efficiency underwent significant shifts in moral valuation across years, moral values of effort and efficiency in People’s Daily are more stable, as evidenced by the absence of any year where the Bayesian Change Point (BCP) detection probability exceeded 50%. This highlights a distinctive pattern between the two cultural contexts.

Regarding inefficient effort shown in Fig. 3B , its association with positive moral values has been consistently positive over the seven decades that we have data for. The trend for inefficient effort remained relatively stable until 1959, the year marked as a substantial change point identified by Bayesian Change Point Detection method. Between 1959 and 1978, we observed some fluctuations but gradually a downward trend emerged after 1978, paralleling China’s period of profound economic reform and transition to a more market-oriented economy, which emphasizes the importance of efficiency. The period from 1992 onward witnessed an even steeper decline in the positive moral attitudes towards inefficient effort, during which China’s economic reform and opening moved to a “fast track.” Notably, in 1992, the then paramount leader of the Chinese Communist Party, Deng Xiaoping, delivered a series of speeches that emphasized the importance of economic reform in the survival and thriving of the country and its people. In one of the speeches, he remarked that “whoever is against reform will be driven out of power,” putting pressure on the government and party leadership to carry out economic reforms and marketization (Chatwin, 2024 ).

One might argue that the construct of effort is related to other constructs in social-personality and cultural psychology. For example, a recent cross-cultural work examined the “difficulty-as-improvement” mindset (seeing difficulties and hardship as opportunities for self-betterment) in several cultures (Yan et al., 2023 ). In particular, this work showed that the “difficulty-as-improvement” mindset was associated with the Protestant Work Ethics, which is also related to how people view effort and efficiency. To ascertain the relationship between the moral attitudes towards effort and difficulty-as-improvement mindset during the period of times covered by our Chinese and American corpora, we calculated the cosine similarity between the difficulty word list and the improvement word list reported in Yan et al. ( 2023 ) for all the years (Supplementary Fig. 6 ). We then fit ARIMA models to the two time series data separately and obtained the residuals for each year. This is to remove the autocorrelation and temporal dependency of each time series. To estimate the statistical association, we test the correlation between the residuals for each year between the moral attitude towards effort and the “difficulty-as-improvement” mindset. In the U.S. Congressional speech corpus, these two time series were not significantly related ( r  = 0.092, p  = 0.285). In contrast, in the People’s Daily corpus, these two time series were positively related ( r  = 0.557, p  < 0.001). These results indicated that the relationship between the “difficulty-as-improvement” mindset and the moralization of effort is culturally dependent. Future research is needed to further delineate the modulatory role of specific cultural values in this relationship.

In a similar vein, another supplementary analysis showed that the moral attitudes towards effort and efficiency are largely independent of the sentiments associated with these constructs in both the U.S. context and the Chinese context (except for the association between positive sentiment and moral attitudes towards effort in the Chinese corpus; for detail, see Supplementary Methods and Supplementary Fig. 7 ).

Some of the words in the effort and efficiency dictionaries are missing from some of the years we studied here (see Supplementary Tables 5 and 6 for the information of missing words from the U.S. and the Chinese corpora, respectively). We therefore pruned the dictionaries by removing the words that are absent from more than 50% of the time across the period we studied. The temporal evolution patterns of the moral attitudes towards effort, efficiency, and inefficient effort were nearly identical to the patterns we reported in Figs. 2 and 3 (Supplementary Fig. 8 ).

ARIMA models

Next, we used a time series analysis (ARIMA model) to examine the association between the fluctuations of the moral attitudes toward effort, efficiency, and inefficient effort on the one hand, and four cultural values (i.e., individualism, collectivism, cultural looseness, and cultural tightness) on the other hand. We computed the mean vector coordinates for seed words using pre-trained models, which are trained on Google News (Mikolov et al., 2013 ), Wikipedia (Bojanowski et al., 2017 ), and Twitter (Pennington et al., 2014 ) for English, and Renmin, Wikipedia and Weibo for Chinese (Li et al., 2018 ), to represent seed words in high-dimensional semantic spaces and identify similar terms. As shown in Table 1 , during the period of U.S. history we examined here (1873–2011), positive moral attitude toward effort is positively correlated with Looseness ( b  = 0.03, 95% CI [0.01, 0.05], p  < 0.001), even after controlling for GDP per capita ( b  = 0.03, 95% CI [0.01, 0.04], p  < 0.001). This suggests that in the society with a higher tolerance of deviance (i.e., higher cultural looseness), we can expect a significant increase in the positive moral attitude towards effort.

For the moral attitude toward efficiency, collectivism is a significant predictor ( b  = −0.01, 95% CI [−0.02, −0.01], p  < 0.001). However, the significant relationship disappeared after controlling for GDP per capita ( b  = −0.01, 95% CI [−0.03, 0.00], p  = 0.099). This implies that as collectivist values decrease in U.S. society, the moral attitudes toward efficiency in the U.S. Congressional speeches increase, but this relationship appears to be dependent on the society’s economic performance. In contrast, Individualism is not in itself a significant predictor of the moral attitude toward efficiency ( b  = −0.01, 95% CI [−0.03, 0.01], p  = 0.195). Interestingly, however, when adjusted for GDP per capita, Individualism became a significantly negative predictor ( b  = −0.01, 95% CI [−0.03, −0.00], p  = 0.027). This suggests that when the economic performance is held constant, the more prevalent the individualist values are in U.S. society, the more negative the moral attitude toward efficiency is, as observed in Congressional speeches.

For the moral attitude toward inefficient effort, after controlling for GDP per capita, Looseness became a negative predictor ( b  = −0.06, 95% CI [−0.07, −0.05], p  = < 0.001). This suggests that in the relatively culturally looser periods in the U.S. history, inefficient effort in the U.S. congressional speeches becomes more morally negative statistically, possibly indicating a greater condemnation of wastefulness and inefficiency in their public discourse.

For People’s Daily (1950–2021), the effort models show that collectivism was a significantly positive predictor of effort after controlling for GDP per capita ( b  = 0.04, 95% CI [0.02, 0.06], p  < 0.001) (Table 2 ). The significant influences of collectivism on the moral attitudes towards effort reflect the intrinsic cultural appreciation for hard work and diligence that is deeply ingrained in Chinese culture (Heine, 2001 ; Leong et al., 2014 ).

In the basic efficiency models, Looseness emerged as a positive predictor of Efficiency before ( b  = 0.03, 95% CI [0.02, 0.05], p  < 0.001) and after controlling for GDP per capita ( b  = 0.05, 95% CI [0.00, 0.10], p  = 0.040). This association can be interpreted in light of the societal transformation, where flexible norms and behaviors (indicative of looseness) might lead to a favorable valuation of efficiency during China’s shift toward a market economy.

Most notably, in the inefficient effort models, Collectivism was a significantly positive predictor only before controlling for GDP per capita ( b  = 0.06, 95% CI [0.02, 0.10], p  = 0.005). Interestingly, Looseness emerged as a negative predictor of Inefficient Effort only after adjusting for GDP per capita ( b  = −0.15, 95% CI [−0.22, −0.08], p  = < 0.001). The negative impact of Looseness on moralization of inefficient effort observed in the People’s Daily echoes findings from the U.S. congressional speeches, suggesting a broader, cross-cultural trend. This pattern could imply that societies with more flexible norms and behaviors might value inefficiencies less and view them as counterproductive in rapidly changing environments.

Granger tests

Did cultural values lead to changes in the moral valuation of effort and efficiency, or did these moral values trigger shifts in cultural norms? To examine the existence and directionality of these relationships, we applied Granger analysis to the time series of cultural variables and moral attitudes.

Our findings, as shown in Table 3 , revealed that within the U.S. Congressional speeches, Individualism significantly influenced Effort ( F (9, 220) = 3.05, p  = 0.002) and Efficiency ( F (9, 220) = 2.56, p  = 0.008), without significant reverse causality from Effort ( F (9, 220) = 1.28, p  = 0.251) or Efficiency ( F (9, 220) = 0.61, p  = 0.785) back to Individualism. Similarly, Collectivism significantly influenced Effort ( F (10, 214) = 2.89, p  = 0.002) and Efficiency ( F (10, 214) = 3.12, p  = < 0.001), without notable reverse causality from Effort ( F (10, 214) = 0.86, p  = 0.571) or Efficiency ( F (10, 214) = 1.00, p  = 0.442) to Collectivism. In contrast, Inefficient Effort was a significant precursor of shifts in cultural values including Individualism ( F (9, 220) = 1.93, p  = 0.049) and Collectivism ( F (10, 214) = 3.30, p  = < 0.001). Meanwhile, cultural norms such as Tightness and Looseness did not show a causal influence on Inefficient Effort. In contrast, Inefficient Effort significantly predicted changes in Tightness ( F (10, 214) = 2.45, p  = 0.009).

The interplay between cultural variables and moral attitudes toward effort and efficiency showed different patterns in People’s Daily . Specifically, Individualism had a significant causal influence on Effort ( F (9, 82) = 2.07, p  = 0.042) with no significant reverse causality ( F (9, 82) = 1.19, p  = 0.314). Similarly, changes in Collectivism drove shifts in Effort ( F (9, 82) = 2.97, p  = 0.004) with no reverse relationship ( F (9, 82) = 0.82, p  = 0.599). Neither Looseness nor Tightness showed a significant causal influence on Effort. There was a significant impact of Effort on the change of Tightness ( F (9, 82) = 2.01, p  = 0.048). This suggests that increased valuations of effort might have some influence on adherence to norms and regulations. None of the cultural variables demonstrated a significant causal influence on Efficiency. However, intriguingly, Efficiency had a significant causal influence on Individualism ( F (9, 82) = 2.33, p  = 0.021).

In analyzing Inefficient Effort, a robust causal relationship was identified where Collectivism showed a causal relationship with Inefficient Effort ( F (9, 82) = 2.94, p  = 0.004) and vice versa ( F (9, 82) = 2.37, p  = 0.020) in the Chinese narrative. This bidirectional causality indicates a dynamic interplay between collective values and the moralization of inefficient efforts. Additionally, Looseness preceded the changes of Inefficient Effort ( F (2, 124) = 3.73, p  = 0.027) with no significant reverse causality ( F (2, 124) = 1.76, p  = 0.176). Complemented by the ARIMA models, the results indicated that collectivism positively influenced the moral values associated with effort. Furthermore, there was a positive bidirectional relationship between collectivism and the moralization of inefficient effort (Table 3 ).

For robustness checks, we reran the ARIMA models and Granger tests with the pruned effort and efficiency dictionaries (i.e., removing words that are missing from more than 50% of the years we studied) (Supplementary Tables 7 – 10 ). We also reran the ARIMA models and Granger Tests for the U.S. corpus using cultural variables derived from COHA (Supplementary Tables 11 and 12 ). The results remained qualitatively consistent.

In this study, we combined a computational linguistic approach (Atari and Dehghani, 2022 ; Caliskan et al., 2017 ; Garg et al., 2018 ; Hamilton et al., 2016 ; Kennedy et al., 2021 a; Xu et al., 2021 ) and historical text corpora in China and the U.S. (Atari and Henrich, 2023 ; Muthukrishna et al., 2021 ) to examine the historical trend of the moral attitudes towards effort and efficiency (Celniker et al., 2023 ). The two societies exemplify two distinctive cultures: the Protestant Work Ethic (PWE) predominant in the U.S. (Furnham, 1984 ; Uhlmann and Sanchez-Burks, 2014 ), and Confucian values that characterize China (Hwang, 2012 , 2015 ). By examining the historical trend of these moral attitudes in the U.S. and China, we offer insights into the evolving cultural narratives of hard work and productivity.

The U.S., deeply influenced by the PWE, has historically valued hard work as a moral virtue during the rise of capitalism. Yet, our study discerns a more intricate narrative. Up until the end of the 1940s, the moral attitudes toward effort and efficiency were indistinguishable, at least in the Congressional speech corpus, often overlapping with null models. However, marked increases in the positive moral values of effort and efficiency emerged starting from the 1950s and continued into the 1960s and 1970s, with effort surpassing efficiency in moral values. Intriguingly, during this period, the moral attitude toward inefficient effort starts to increase. This trend coincides with America’s post-World War II economic boom, material prosperity, and optimism and confidence. Directly relevant to the Congressional speech corpus we used, during this time, the Congress established the Council of Economic Advisors to encourage high employment, substantial profits, and low inflation. Within this context, pure effort, even if not always immediately efficient or productive, gained increasingly positive moral values. Future work is needed to ascertain the relationships between these different political and economic trends (zeitgeist) on the one hand, and the society’s moral attitudes towards efforts and efficiency on the other.

As for China, our findings depict a narrative that is different from that of the American one. The moral value of effort remains consistently positive and above the confidence interval of the null model throughout our period of observation. In contrast, the moral value of efficiency is mostly within the confidence interval of the null model, indicating that the concept of efficiency is not consistently and meaningfully associated with morality during this period of history. The lack of moral value associated with efficiency may be related to China’s long-standing adherence to Confucian values, which emphasize collective obligations irrespective of their outcomes. This philosophy is captured by sayings such as “Heroes are not judged solely by success or failure.” Indeed, Confucius himself is proud of being someone who “knows the impracticable nature of the times and yet will be doing in them” ( Analects ). A significant decline of positive moral attitude toward inefficient effort began at the turn of the 1960s, when China’s leadership became aware of the detrimental effects of unrealistic ideological zeal and rolled back some of the inefficient policies mandated during the Great Leap Forward. Another decline occurred in the early 1990s and coincided with the implementation of market reform policies. The zeitgeist of this period is best illustrated by the motto of the City of Shenzhen, one of the pioneering regions of China’s economic reform, “Time is money, efficiency is life.” This trajectory, perhaps, mirrors China’s transition to a rapidly modernizing and industrializing country.

The findings from our time series analysis reveal intriguing cultural insights. Both in the U.S. and in China, a consistent negative relationship was observed between cultural looseness and the moral attitudes toward inefficient effort. This suggests that when a society is less culturally loose, it will place more moral value on effort relative to efficiency. Another commonality emerging from the Granger tests is that it was individualism and collectivism, rather than cultural tightness or looseness, that preceded shifts in the moral values of effort. This indicates that while loosening cultural norms might influence attitudes toward work efficiency, the driving factors behind moral values of effort are rooted more in the individualistic or collectivistic tendencies of these societies.

Our results from the exploratory time series analysis (ARIMA and Granger tests), reveal distinct patterns of associations and precedence between cultural variables (i.e., individualism, collectivism, cultural tightness/looseness) and the moral attitudes towards effort and efficiency (i.e., effort, efficiency, and inefficiency effort) in the U.S. and China. Specifically, in the U.S., cultural looseness is positively correlated with moral values of effort and negatively with moral values of inefficient effort. Additionally, individualism is negatively correlated with moral values of efficiency. This suggests that in U.S. society, where cultural looseness prevails, there may be a greater emphasis on productive effort. Meanwhile, the change of individualism could be associated with the devaluation of efficiency. On the other hand, in China, collectivism upholds the moral importance of effort, even when its outcomes appear insignificant. This is evident from the positive and bidirectional relationship between collectivism and the moralization of inefficient effort. However, cultural looseness is negatively associated with and precedes positive moral value of inefficient effort, independent of the society’s economic performance. This might reflect Confucian values that emphasize individuals’ obligations to the group and social ideal, valuing effort to fulfill these obligations, regardless of its outcomes, is challenged by the shifting dynamics brought about by cultural looseness.

Furthermore, the results combining ARIMA models and Granger tests highlight distinct cultural dynamics in the U.S. and China. Cultural looseness is mainly positively associated with the moral values of effort in the U.S., while mainly positively correlated with moral values of efficiency in China, although no causal significance was detected in these relationships. Notably, both countries displayed a negative correlation between cultural looseness and the moral values of inefficient effort. Another distinct pattern emerged when examining collectivism. While in the US, collectivism showed a negative unidirectional influence on the moral value of efficiency, in China, collectivism had a positive driving impact on the moral values of effort and inefficient effort. This suggests different mechanisms may explain how cultural looseness and collectivism shape work ethics across the two cultures.

These findings not only underscore the intricate relationships between cultural dimensions and moral values but also highlight the divergent paths through which these associations manifest in different socio-cultural contexts. However, we acknowledge that our time series analyses are largely exploratory and descriptive. For the Granger tests, we noted that the analysis of mutual influence between cultural variables and moral attitudes without examining other complex external variables (such as politics, economics) is limited in revealing the mechanisms that underlie those changes in these variables. Future research is needed to more closely address the mechanisms underlying the relationship between societal and cultural variables and moral attitudes towards effort and efficiency.

One may notice that in some of the ARIMA models, individualism and collectivism have the same direction of an effect. While it is true that individualism and collectivism sometimes exhibit similar trends in the ARIMA analyses, there are no significant same-direction effects for any given analysis. It is not unexpected that these two cultural values sometimes may have similar effects on moral attitudes towards effort and efficiency. Our dictionaries of effort concept and efficiency concept do not specify the beneficiary of the effort or efficiency. In that sense, our analysis was agnostic of goals of effort or efficiency. For example, an effortful and/or efficient endeavor could be exerted to obtain collectivist goals (e.g., as a slogan during the “Great Leap Forward” movement in China goes, “Work hard, strive for the upper reaches, and build socialism as quickly, efficiently, and economically as possible”) or individualist goals (e.g., as exemplified by meritocracy narratives, “The American Dream I believe in is one that provides anyone willing to work hard enough with the opportunity to succeed” - U.S. Senator Tammy Duckworth). In sum, it is conceivable that individualism and collectivism values correlate with moral attitudes towards effort/efficiency in the same direction.

There are several limitations in the present research. First, our text corpora are exclusively drawn from U.S. Congressional speeches and Chinese newspapers. Both sources are formal, politically oriented narratives and texts. While these texts offer rich insights into their respective societies and times, future investigations should also diversify the source materials to include non-political content such as literature, local newspapers, magazines, and other popular media outlets, to capture a more comprehensive and representative understanding of work ethics. More broadly, before the widespread use of the internet and social media, a society’s written records (e.g., books, newspapers, magazines) were generally produced (and consumed) by political, economic, and/or cultural elites (Muthukrishna et al., 2021 ). Even the texts from more accessible online social media of our time do not reflect the entire society’s (offline) view faithfully (Robertson et al., 2024 ). This is a common challenge that most text-based historical psychology research faces and one should keep this in mind when conducting and interpreting such research.

Another limitation is the restricted time frame of our study. Although this period witnessed momentous societal shifts in both the U.S. and China, a deeper historical dive would no doubt unravel even richer evolutionary patterns. In the West, epochs such as the Renaissance, Age of Exploration, Reformation—which heralded the rise of the PWE—and the Industrial Revolution, and Age of Enlightenment likely have enormous influences on the moral values of effort and efficiency. Likewise, China’s transition from millennia of imperial rule to the republic regime at the turn of the 20th century, the New Culture Movement, World War II, and the Chinese Civil War, may have profoundly shaped the moral landscapes of effort and efficiency. Future research is needed to examine the evolution of moral attitudes towards effort and efficiency during these historical times.

Lastly, our moral evaluative words were based on the Moral Foundations Dictionary (MFD). This dictionary, while grounded in the Moral Foundations Theory (MFT) (Graham et al., 2013 ), only represents one of various computational linguistic measures of morality, such as the moral judgment dictionary from The Development and Psychometric Properties (Boyd et al., 2022 ; Brady et al., 2020 ) or other versions of the Moral Foundations Dictionary (Hopp et al., 2020 ). For example, the Extended Moral Foundations Dictionary (eMFD) was based on a more comprehensive, crowd-sourced annotation procedure instead of expert-driven content analysis (Hopp et al., 2020 , 2021 ; Weber et al., 2021 , p. 201). The present study does not aim to advocate one theoretical framework or tool over another but utilizes the MFD for its established efficacy in analyzing both English and Chinese texts. It is also worth noting that the field of computational linguistics has been burgeoning, with novel tools being developed for moral content analysis in textual data (Hoover et al., 2018 ; Vaisey and Miles, 2014 ). Future studies that integrate and compare these advanced tools will be able to bring about a richer, more nuanced exploration of morality from textual data.

Our language-based historical psychology approach to understanding moral attitudes towards effort and efficiency adds a cross-cultural and historical dimension to recent work on the perception and evaluation of work and effort (Graeber, 2018 ; Steger et al., 2013 ; Ward, 2024 ). Inefficient effort is work done for the sake of it being done instead of some useful products or outcomes, and therefore is an essential part of meaningless work. Our research provides insights that the moral outlooks of meaningful or meaningless work varies across cultures and times. Critically, in our own time, we are witnessing perhaps one of the most important revolutions in human history: with the rapid growth of artificial intelligence in various domains of work and life, the necessity and meaning of human work in its traditional sense is inevitably facing fundamental challenges. What are the unique roles of human effort and agency in future work? How will people’s moral attitudes towards effort and efficiency change when more human efforts can be replaced and improved by AI-powered automation? Future research is needed to ascertain these theoretically and practically important questions.

In sum, we investigated the evolving patterns of moral attitudes toward effort, efficiency, and inefficient effort in the recent history of China and the U.S., by combining an effective computational linguistic tool and historical text corpora. We found that the fluctuations of moral attitudes towards inefficient effort roughly corresponded to critical historical events or periods in both societies that mark significant social changes in world view and ideology. Further investigation through time series analysis suggests associations and possible influences of social and cultural factors, such as collectivism/individualism and cultural looseness, on the evolution of moral attitudes toward inefficient effort. Our study not only provides insights into the historical and sociocultural origins of the moralization of inefficient effort but also has implications for historical psychological investigations into the evolution of morality.

Data availability

Data and analysis code for reproducing the results reported in this paper can be accessed at https://osf.io/aghpk .

Amos C, Zhang L, Read D (2019) Hardworking as a heuristic for moral character: why we attribute moral values to those who work hard and its implications. J Bus Ethics 158(4):1047–1062. https://doi.org/10.1007/s10551-017-3725-x

Article   Google Scholar  

Atari M, Dehghani M (2022) Language analysis in moral psychology. In: Handbook of language analysis in psychology. The Guilford Press, p 207–228

Atari M, Haidt J, Graham J, Koleva S, Stevens ST, Dehghani M (2023) Morality beyond the WEIRD: How the nomological network of morality varies across cultures. J Personal Soc Psychol 125(5):1157–1188. https://doi.org/10.1037/pspp0000470

Atari M, Henrich J (2023) Historical psychology. Curr Direc Psychol Sci 32(2):176–183. https://doi.org/10.1177/09637214221149737

Baumard N, Huillery E, Hyafil A, Safra L (2022) The cultural evolution of love in literary history. Nat Hum Behav 6(4):4. https://doi.org/10.1038/s41562-022-01292-z . Article

Becker SO, Woessmann L (2009) Was Weber wrong? A human capital theory of protestant economic history*. Q J Econ 124(2):531–596. https://doi.org/10.1162/qjec.2009.124.2.531

Bigman YE, Tamir M (2016) The road to heaven is paved with effort: Perceived effort amplifies moral judgment. J Exp Psychol: Gen 145(12):1654–1669. https://doi.org/10.1037/xge0000230

Article   PubMed   Google Scholar  

Bojanowski P, Grave E, Joulin A, Mikolov T (2017) Enriching word vectors with subword information. Trans Assoc Comput Linguist 5:135–146. https://doi.org/10.1162/tacl_a_00051

Bollen J, ten Thij M, Breithaupt F, Barron ATJ, Rutter LA, Lorenzo-Luaces L, Scheffer M (2021) Historical language records reveal a surge of cognitive distortions in recent decades. Proc Natl Acad Sci 118(30):e2102061118. https://doi.org/10.1073/pnas.2102061118

Article   PubMed   PubMed Central   Google Scholar  

Bolt J, van Zanden JL (2020) Maddison style estimates of the evolution of the world economy. A new 2020 update (Maddison Project Database, version 2020) [Dataset]. https://www.rug.nl/ggdc/historicaldevelopment/maddison/releases/maddison-project-database-2020

Boyd R, Ashokkumar A, Seraj S, & Pennebaker J (2022) The development and psychometric properties of LIWC-22 . https://doi.org/10.13140/RG.2.2.23890.43205

Brady WJ, Crockett MJ, Van Bavel JJ (2020) The MAD model of moral contagion: the role of motivation, attention, and design in the spread of moralized content online. Perspect Psychol Sci 15(4):978–1010. https://doi.org/10.1177/1745691620917336

Caliskan A, Bryson JJ, Narayanan A (2017) Semantics derived automatically from language corpora contain human-like biases. Science 356(6334):183–186. https://doi.org/10.1126/science.aal4230

Article   ADS   PubMed   Google Scholar  

Chen Y, Chen AX, Yu H, Sun S (2024) Unraveling moral and emotional discourses on social media: A study of three cases. Inf Commun Soc 27(7):1295–1312. https://doi.org/10.1080/1369118X.2023.2246551

Card D, Chang S, Becker C, Mendelsohn J, Voigt R, Boustan L, Abramitzky R, Jurafsky D (2022) Computational analysis of 140 years of US political speeches reveals more positive but increasingly polarized framing of immigration. Proc Natl Acad Sci 119(31):e2120510119. https://doi.org/10.1073/pnas.2120510119

Celniker JB, Gregory A, Koo HJ, Piff PK, Ditto PH, Shariff AF (2023) The moralization of effort. J Exp Psychol Gen 152(1):60–79. https://doi.org/10.1037/xge0001259

Charlesworth TES, Caliskan A, Banaji MR (2022) Historical representations of social groups across 200 years of word embeddings from Google Books. Proc Natl Acad Sci 119(28):e2121798119. https://doi.org/10.1073/pnas.2121798119

Charlesworth TES, Ghate K, Caliskan A, Banaji MR (2024) Extracting intersectional stereotypes from embeddings: developing and validating the Flexible Intersectional Stereotype Extraction procedure. PNAS Nexus 3(3):pgae089. https://doi.org/10.1093/pnasnexus/pgae089

Chatwin J (2024) The Southern tour: Deng Xiaoping and the fight for China’s Future . Bloomsbury Publishing

Chen S-W (2023) Learning motivations and effort beliefs in Confucian cultural context: a dual-mode theoretical framework of achievement goal. Front Psychol , 14 . https://www.frontiersin.org/articles/10.3389/fpsyg.2023.1058456

Chen Y, Li S, Li Y, Atari M (2024) Surveying the dead minds: historical-psychological text analysis with contextualized construct representation (CCR) for Classical Chinese (arXiv:2403.00509). https://doi.org/10.48550/arXiv.2403.00509

Cheng CY, Zhang W (2023) C-MFD 2.0: developing a chinese moral foundation dictionary. Comput Commun Res 5(2):1. https://doi.org/10.5117/CCR2023.2.10.CHEN

Article   ADS   Google Scholar  

Choi VK, Shrestha S, Pan X, Gelfand MJ (2022) When danger strikes: a linguistic tool for tracking America’s collective response to threats. Proc Natl Acad Sci 119(4):e2113891119. https://doi.org/10.1073/pnas.2113891119

Enke B (2020) Moral values and voting. J Political Econ 128(10):3679–3729. https://doi.org/10.1086/708857

Erdman C, Emerson JW (2008) bcp: an R package for performing a bayesian analysis of change point problems. J Stat Softw 23:1–13. https://doi.org/10.18637/jss.v023.i03

Ess M, Burke SE (2022) Class attitudes and the American work ethic: praise for the hardworking poor and derogation of the lazy rich. J Exp Soc Psychol 100:104301. https://doi.org/10.1016/j.jesp.2022.104301

Fitouchi L, André J-B, Baumard N (2022) Moral disciplining: the cognitive and evolutionary foundations of puritanical morality. Behav Brain Sci , 1–71. https://doi.org/10.1017/S0140525X22002047

Frimer JA, Boghrati R, Haidt J, Graham J, Dehgani M (2019) Moral foundations dictionary for linguistic analyses 2.0. https://doi.org/10.17605/OSF.IO/EZN37

Furnham A (1984) The protestant work ethic: a review of the psychological literature. Eur J Soc Psychol 14(1):87–104. https://doi.org/10.1002/ejsp.2420140108

Fwu B, Wei C-F, Chen S-W, Wang H (2014) Effort counts: the moral significance of effort in the patterns of credit assignment on math learning in the Confucian cultural context. Int J Educ Dev 39:157–162. https://doi.org/10.1016/j.ijedudev.2014.07.010

Garg N, Schiebinger L, Jurafsky D, Zou J (2018) Word embeddings quantify 100 years of gender and ethnic stereotypes. Proc Natl Acad Sci 115(16):E3635–E3644. https://doi.org/10.1073/pnas.1720347115

Article   ADS   PubMed   PubMed Central   Google Scholar  

Garten J, Hoover J, Johnson KM, Boghrati R, Iskiwitch C, Dehghani M (2018) Dictionaries and distributions: combining expert knowledge and large scale textual data content analysis: Distributed dictionary representation. Behav Res Methods 50(1):344–361. https://doi.org/10.3758/s13428-017-0875-9

Gelfand M (2019) Rule makers, rule breakers: tight and loose cultures and the secret signals that direct our lives. Simon and Schuster

Gelfand MJ, Nishii LH, Raver JL (2006) On the nature and importance of cultural tightness-looseness. J Appl Psychol 91(6):1225. https://doi.org/10.1037/0021-9010.91.6.1225

Gentzkow M, Shapiro JM, Taddy M (2019) Measuring group differences in high‐dimensional choices: method and application to congressional speech. Econometrica 87(4):1307–1340. https://doi.org/10.3982/ECTA16566

Article   MathSciNet   Google Scholar  

Graeber D (2018) Bullshit jobs: a theory. Simon and Schuster

Graham J, Haidt J, Koleva S, Motyl M, Iyer R, Wojcik SP, Ditto PH (2013) Moral foundations theory. In Advances in Experimental Social Psychology (Vol 47, pp 55–130). Elsevier. https://doi.org/10.1016/B978-0-12-407236-7.00002-4

Greenfield PM (2013) The changing psychology of culture from 1800 through 2000. Psychol Sci 24(9):1722–1731. https://doi.org/10.1177/0956797613479387

Hamilton WL, Leskovec J, Jurafsky D (2016) Cultural shift or linguistic drift? comparing two computational measures of semantic change. Proc Conf Empir Methods Nat Lang Process Conf Empir Methods Nat Lang Process 2016:2116–2121

Google Scholar  

Heine SJ (2001) Self as cultural product: an examination of East Asian and North American selves. J Personal 69(6):881–906. https://doi.org/10.1111/1467-6494.696168

Henrich, J (2020) The WEIRDest people in the world: how the West became psychologically peculiar and particularly prosperous. Penguin UK

Hofmann W, Wisneski DC, Brandt MJ, Skitka LJ (2014) Morality in everyday life. Science 345(6202):1340–1343

Holloway SD (1988) Concepts of ability and effort in Japan and the United States. Rev Educ Res 58(3):327–345. https://doi.org/10.3102/00346543058003327

Hoover J, Dehghani M, Johnson K, Iliev R, Graham J (2018) Into the wild: Big data analytics in moral psychology. In: Atlas of moral psychology. The Guilford Press, p 525–536

Hopp FR, Fisher JT, Cornell D, Huskey R, Weber R (2020) The Extended Moral Foundations Dictionary (eMFD): development and applications of a crowd-sourced approach to extracting moral intuitions from text. 23

Hopp FR, Fisher JT, Cornell D, Huskey R, Weber R (2021) The extended Moral Foundations Dictionary (eMFD): development and applications of a crowd-sourced approach to extracting moral intuitions from text. Behav Res Methods 53(1):232–246. https://doi.org/10.3758/s13428-020-01433-0

Hwang K-K (2012) Foundations of Chinese psychology (Vol 1). Springer, New York. https://doi.org/10.1007/978-1-4614-1439-1

Hwang K-K (2015) Morality ‘East’and ‘West’: cultural concerns. Int Encycl Soc Behav Sci 15:806–810. https://doi.org/10.1016/B978-0-08-097086-8.64005-9

Hyndman RJ, Khandakar Y (2008) Automatic Time Series Forecasting: the forecast Package for R. J Stat Softw 27:1–22. https://doi.org/10.18637/jss.v027.i03

Jackson JC, Gelfand M, De S, Fox A (2019) The loosening of American culture over 200 years is associated with a creativity–order trade-off. Nat Hum Behav 3(3):3. https://doi.org/10.1038/s41562-018-0516-z . Article

Jones J, Amin M, Kim J, Skiena S (2020) Stereotypical gender associations in language have decreased over time. Sociol Sci 7:1–35. https://doi.org/10.15195/v7.a1

Kennedy B, Ashokkumar A, Boyd RL, Dehghani M (2021a) Text analysis for psychology: methods, principles, and practices. OSF. https://doi.org/10.31234/osf.io/h2b8t

Kennedy B, Atari M, Mostafazadeh Davani A, Hoover J, Omrani A, Graham J, Dehghani M (2021b) Moral concerns are differentially observable in language. Cognition 212:104696. https://doi.org/10.1016/j.cognition.2021.104696

Kirk-Johnson A, Galla BM, Fraundorf SH (2019) Perceiving effort as poor learning: the misinterpreted-effort hypothesis of how experienced effort and perceived learning relate to study strategy choice. Cogn Psychol 115:101237. https://doi.org/10.1016/j.cogpsych.2019.101237

Kundro, TG (2022). The benefits and burdens of work moralization on creativity. Acad Manag J , amj.2021.0273. https://doi.org/10.5465/amj.2021.0273

Leong FTL, Huang JL, Mak S (2014) Protestant work ethic, confucian values, and work-related attitudes in Singapore. J Career Assess 22(2):304–316. https://doi.org/10.1177/1069072713493985

Li S, Zhao Z, Hu R, Li W, Liu T, Du X (2018) Analogical reasoning on chinese morphological and semantic relations. In: Proceedings of the 56th annual meeting of the association for computational linguistics (Volume 2: Short Papers), 138–143. https://doi.org/10.18653/v1/P18-2023

Lin Y, Caluori N, Öztürk EB, Gelfand MJ (2022) From virility to virtue: the psychology of apology in honor cultures. Proc Natl Acad Sci 119(41):e2210324119. https://doi.org/10.1073/pnas.2210324119

Martins MdeJD, Baumard N (2020) The rise of prosociality in fiction preceded democratic revolutions in Early Modern Europe. Proc Natl Acad Sci 117(46):28684–28691. https://doi.org/10.1073/pnas.2009571117

Mason SE, Kuntz CV, McGill CM (2015) Oldsters and Ngrams: age stereotypes across time. Psychol Rep. 116(1):324–329. https://doi.org/10.2466/17.10.PR0.116k17w6

Michel J-B, Shen YK, Aiden AP, Veres A, Gray MK, Pickett JP, Hoiberg D, Clancy D, Norvig P, Orwant J (2011) Quantitative analysis of culture using millions of digitized books. Science 331(6014):176–182. https://doi.org/10.1126/science.1199644

Mikolov T, Sutskever I, Chen K, Corrado GS, & Dean J (2013). Distributed Representations of Words and Phrases and their Compositionality. Advances in Neural Information Processing Systems, 26. https://papers.nips.cc/paper_files/paper/2013/hash/9aa42b31882ec039965f3c4923ce901b-Abstract.html

Miller GA (1995) WordNet: a lexical database for English. Commun ACM 38(11):39–41. https://doi.org/10.1145/219717.219748

Muthukrishna M, Henrich J, Slingerland E (2021) Psychology as a historical science. Annu Rev Psychol 72(1):717–749. https://doi.org/10.1146/annurev-psych-082820-111436

Ng R, Chow TYJ (2021) Aging narratives over 210 years (1810–2019). J Gerontology: Ser B 76(9):1799–1807. https://doi.org/10.1093/geronb/gbaa222

Pennington J, Socher R, Manning CD (2014) GloVe: global vectors for word representation. Empirical Methods in Natural Language Processing (EMNLP) , 1532–1543. http://www.aclweb.org/anthology/D14-1162

Poznanski K (2017) Confucian economics: how is Chinese thinking different? China Econ J 10(3):362–384. https://doi.org/10.1080/17538963.2017.1370170

Robertson C, Rosario K. del, & Bavel JJV (2024) Inside the Funhouse Mirror Factory: how social media distorts perceptions of norms. OSF. https://doi.org/10.31234/osf.io/kgcrq

Sahlgren M (2008) The distributional hypothesis. Ital J Disabil Stud 20:33–53

Schmidt B, Piantadosi ST, Mahowald K (2021) Uncontrolled corpus composition drives an apparent surge in cognitive distortions. Proc Natl Acad Sci 118(45):e2115010118. https://doi.org/10.1073/pnas.2115010118

Schulz JF, Bahrami-Rad D, Beauchamp JP, Henrich J (2019) The Church, intensive kinship, and global psychological variation. Science 366(6466):eaau5141

Sherman R (2017) Uneasy street: the anxieties of affluence. Princeton University Press

Steger MF, Littman-Ovadia H, Miller M, Menger L, Rothmann S (2013) Engaging in work even when it is meaningless: positive affective disposition and meaningful work interact in relation to work engagement. J Career Assess 21(2):348–361. https://doi.org/10.1177/1069072712471517

Talhelm T, English AS (2020) Historically rice-farming societies have tighter social norms in China and worldwide. Proc Natl Acad Sci 117(33):19816–19824. https://doi.org/10.1073/pnas.1909909117

Tucker EC, Capps CJ, Shamir L (2020) A data science approach to 138 years of congressional speeches. Heliyon 6(8):e04417. https://doi.org/10.1016/j.heliyon.2020.e04417

Uhlmann EL, Sanchez-Burks J (2014) The implicit legacy of American protestantism. J Cross-Cult Psychol 45(6):992–1006. https://doi.org/10.1177/0022022114527344

Vaisey S, Miles A (2014) Tools from moral psychology for measuring personal moral culture. Theory Soc 43(3/4):311–332. https://doi.org/10.1007/s11186-014-9221-8

Wang S-YN, Inbar Y (2021) Moral-language use by U.S. political elites. Psychol Sci 32(1):14–26. https://doi.org/10.1177/0956797620960397

Ward S (2024) Choosing money over meaningful work: examining relative job preferences for high compensation versus meaningful work. Personal Soc Psychol Bull 50(7):1128–1148. https://doi.org/10.1177/01461672231159781

Weber M (1905) The protestant ethic and the spirit of capitalism. Unwin Hyman

Weber R, Mangus JM, Huskey R, Hopp FR, Amir O, Swanson R, Gordon A, Khooshabeh P, Hahn L, Tamborini R (2021) Extracting latent moral information from text narratives: relevance, challenges, and solutions. In Computational methods for communication science. Routledge

Xu A, Stellar JE, Xu Y (2021) Evolution of emotion semantics. Cognition 217:104875. https://doi.org/10.1016/j.cognition.2021.104875

Yan VX, Oyserman D, Kiper G, Atari M (2023) Difficulty-as-improvement: the courage to keep going in the face of life’s difficulties. Personal Soc Psychol Bull , 01461672231153680. https://doi.org/10.1177/01461672231153680

Zaugg IA, Hossain A, & Molloy B (2022) Digitally-disadvantaged languages. Internet Policy Rev , 11(2). https://doi.org/10.14763/2022.2.1654

Zeileis A, Hothorn T (2002) Diagnostic checking in regression relationships. R N 2(3):7–10

Zeng R, Greenfield PM (2015) Cultural evolution over the last 40 years in China: using the Google Ngram Viewer to study implications of social and political change for cultural values. Int J Psychol 50(1):47–55. https://doi.org/10.1002/ijop.12125

Zhang S, Liu W, Liu X (2012) Investigating the relationship between protestant work ethic and confucian dynamism: an empirical test in mainland China. J Bus Ethics 106(2):243–252. https://doi.org/10.1007/s10551-011-0993-8

Download references

Acknowledgements

This research is funded by UCSB ISBER Collaborative Research Initiative Grant (awarded to H.Y.). S.S. is supported by a China National Social Science Foundation grant (22BXW045). The authors would like to thank Ji Cao for curating the Chinese text corpus. We also thank Christina Kushnir and Annabel Dang for their evaluation of the words in the dictionaries, and Tianhao Qin for checking the analysis codes. Furthermore, we thank Yingyi Luo for constructive feedback on an earlier version of the manuscript.

Author information

Authors and affiliations.

Department of Psychological and Brain Sciences, University of California Santa Barbara, Santa Barbara, CA, USA

Amber X. Chen & Hongbo Yu

School of Journalism, Fudan University, Shanghai, China

Shaojing Sun

You can also search for this author in PubMed   Google Scholar

Contributions

A.X.C. and H.Y. designed the study, A.X.C. did the analyses, A.X.C, S.S., and H.Y. wrote the initial and revised manuscript.

Corresponding author

Correspondence to Hongbo Yu .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethical approval

Ethical approval was not required as the study did not involve human participants. Since there were no human participants involved in this study, the last author’s institution (University of California Santa Barbara) judged that this research did not need to complete a protocol with the Office of Research Application for the use of Human Subjects.

Informed consent

This article does not contain any studies with human participants performed by any of the authors. Therefore, an informed consent is not relevant.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary materials, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Chen, A.X., Sun, S. & Yu, H. Moral attitudes towards effort and efficiency: a comparison between American and Chinese history. Humanit Soc Sci Commun 11 , 1085 (2024). https://doi.org/10.1057/s41599-024-03603-3

Download citation

Received : 03 November 2023

Accepted : 15 August 2024

Published : 26 August 2024

DOI : https://doi.org/10.1057/s41599-024-03603-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies