Common elements of a scholarly article:
What is pop-sci?
Research is done by...
...by way of...
...communicated through...
...and organized in...
A Heirarchy of research information:
Source: SUNY Downstate Medical Center. Medical Research Library of Brooklyn. Evidence Based Medicine Course. A Guide to Research Methods: The Evidence Pyramid: http://library.downstate.edu/EBM2/2100.htm
Click image to enlarge
Scientific information has a ‘life cycle’ of its own… it is born as an idea, and then matures and becomes more available to the public. First it appears within the so-called ‘invisible college’ of experts in the field, discussed at conferences and symposia or posted as pre-prints for comments and corrections. Then it appears in the published literature (the primary literature), often as a journal article in a peer-reviewed journal.
Researchers can use the indexing and alerting services of the secondary literature to find out what has been published in a field. Depending on how much information is added by the indexer or abstracter, this may take a few months (though electronic publication has sped up this process). Finally, the information may appear in more popular or reference sources, sometimes called the tertiary literature.
The person beginning a literature search may take this process in reverse: using tertiary sources for general background, then going to the secondary literature to survey what has been published, following up by finding the original (primary) sources, and generating their own research Idea.
(Original content by Wade Lee-Smith)
It's important to make a considered decision as to where to search for your review of the literature. It's uncommon for a disciplinary area to be covered by a single publisher, so searching a single publisher platform or database is unlikely to give you sufficient coverage of studies for a review. A good quality review involves searching a number of databases individually.
The most common method is to search a combination of large inter-disciplinary databases such as Scopus & Web of Science Core Collection, and some subject-specific databases (such as PsycInfo or EconLit etc.). The Library databases are an excellent place to start for sources of peer-reviewed journal articles.
Depending on disciplinary expectations, or the topic of your review, you may also need to consider sources or search methods other than database searching. There is general information below on searching grey literature. However, due to the wide varieties of grey literature available, you may need to spend some time investigating sources relevant for your specific need.
Grey literature is information which has been produced outside of traditional publishing channels (where the main purpose of the producing body is not commercial publishing). One example may be Government publications.
Grey literature may be included in a literature review to minimise publication bias . The quality of grey literature can vary greatly - some may be reviewed whereas some may not have been through a traditional editorial process.
See the Grey Literature guide for further information on finding and evaluating grey sources.
In certain disciplines (such as physics) there can be a culture of preprints being made available prior to submissions to journals. There has also been a noticeable rise in preprints in medical and health areas in the wake of Covid-19.
If preprints are relevant for you, you can search preprint servers directly. Another option is to utilise a search engine such as Google Scholar to search specifically for preprints, as Google Scholar has timely coverage of most preprint servers including ArXiv , RePec , SSRN , BioRxiv , and MedRxiv . Articles in Press are not preprints, but are accepted manuscripts that are not yet formally published. Articles in Press have been made available as an early access online version of a paper that may not yet have received its final formatting or an allocation of a volume/issue number. As well as being available on a journal's website, Articles in Press are available in databases such as Scopus and Web of Science, and so (unlike preprints) don't necessarily require a separate search.
Conference papers are typically published in conference proceedings (the collection of papers presented at a conference), and may be found on an organisation or Society's website, as a journal, or as a special issue of journal.
In certain disciplines (such as computer science), conference papers may be highly regarded as a form of scholarly communication; the conferences are highly selective, the papers are generally peer reviewed, and papers are published in proceedings affiliated with high-quality publishing houses.
Conference papers may be indexed in a range of scholarly databases. If you only want to see conference papers, database limits can be used to filter results, or try a specific index such as the examples below:
Honours students and postgraduates may request an Interlibrary Loan of a conference paper. However, conference paper requests may take longer than traditional article requests as they can be difficult to locate; they may have been only supplied to attendees or not formally published. Sometimes only the abstract is available.
If you are specifically looking for statistical data, try searching for the keyword statistics in a Google Advanced Search and limiting by a relevant site or domain. Below are some examples of sites, or you can try a domain such as .gov for government websites.
Statistical data can be found in the following selected sources:
For a list of databases that include statistics see: Databases by Subject: Statistics .
If you are specifically looking for information found in newspapers, the library has a large collection of Australian and overseas newspapers, both current and historical.
See the newspapers webpage , or the Newspapers subject guide for comprehensive information on newspaper sources, as well as searching tips, online videos and more.
The Theses subject guide provides resources and guidelines for locating and accessing theses (dissertations) produced by Monash University as well as other universities in Australia and internationally.
An official website of the United States government
The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.
The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.
The PMC website is updating on October 15, 2024. Learn More or Try it out now .
Debora f.b. leite.
I Departamento de Ginecologia e Obstetricia, Faculdade de Ciencias Medicas, Universidade Estadual de Campinas, Campinas, SP, BR
II Universidade Federal de Pernambuco, Pernambuco, PE, BR
III Hospital das Clinicas, Universidade Federal de Pernambuco, Pernambuco, PE, BR
Jose g. cecatti.
A sophisticated literature review (LR) can result in a robust dissertation/thesis by scrutinizing the main problem examined by the academic study; anticipating research hypotheses, methods and results; and maintaining the interest of the audience in how the dissertation/thesis will provide solutions for the current gaps in a particular field. Unfortunately, little guidance is available on elaborating LRs, and writing an LR chapter is not a linear process. An LR translates students’ abilities in information literacy, the language domain, and critical writing. Students in postgraduate programs should be systematically trained in these skills. Therefore, this paper discusses the purposes of LRs in dissertations and theses. Second, the paper considers five steps for developing a review: defining the main topic, searching the literature, analyzing the results, writing the review and reflecting on the writing. Ultimately, this study proposes a twelve-item LR checklist. By clearly stating the desired achievements, this checklist allows Masters and Ph.D. students to continuously assess their own progress in elaborating an LR. Institutions aiming to strengthen students’ necessary skills in critical academic writing should also use this tool.
Writing the literature review (LR) is often viewed as a difficult task that can be a point of writer’s block and procrastination ( 1 ) in postgraduate life. Disagreements on the definitions or classifications of LRs ( 2 ) may confuse students about their purpose and scope, as well as how to perform an LR. Interestingly, at many universities, the LR is still an important element in any academic work, despite the more recent trend of producing scientific articles rather than classical theses.
The LR is not an isolated section of the thesis/dissertation or a copy of the background section of a research proposal. It identifies the state-of-the-art knowledge in a particular field, clarifies information that is already known, elucidates implications of the problem being analyzed, links theory and practice ( 3 - 5 ), highlights gaps in the current literature, and places the dissertation/thesis within the research agenda of that field. Additionally, by writing the LR, postgraduate students will comprehend the structure of the subject and elaborate on their cognitive connections ( 3 ) while analyzing and synthesizing data with increasing maturity.
At the same time, the LR transforms the student and hints at the contents of other chapters for the reader. First, the LR explains the research question; second, it supports the hypothesis, objectives, and methods of the research project; and finally, it facilitates a description of the student’s interpretation of the results and his/her conclusions. For scholars, the LR is an introductory chapter ( 6 ). If it is well written, it demonstrates the student’s understanding of and maturity in a particular topic. A sound and sophisticated LR can indicate a robust dissertation/thesis.
A consensus on the best method to elaborate a dissertation/thesis has not been achieved. The LR can be a distinct chapter or included in different sections; it can be part of the introduction chapter, part of each research topic, or part of each published paper ( 7 ). However, scholars view the LR as an integral part of the main body of an academic work because it is intrinsically connected to other sections ( Figure 1 ) and is frequently present. The structure of the LR depends on the conventions of a particular discipline, the rules of the department, and the student’s and supervisor’s areas of expertise, needs and interests.
Interestingly, many postgraduate students choose to submit their LR to peer-reviewed journals. As LRs are critical evaluations of current knowledge, they are indeed publishable material, even in the form of narrative or systematic reviews. However, systematic reviews have specific patterns 1 ( 8 ) that may not entirely fit with the questions posed in the dissertation/thesis. Additionally, the scope of a systematic review may be too narrow, and the strict criteria for study inclusion may omit important information from the dissertation/thesis. Therefore, this essay discusses the definition of an LR is and methods to develop an LR in the context of an academic dissertation/thesis. Finally, we suggest a checklist to evaluate an LR.
Conducting research and writing a dissertation/thesis translates rational thinking and enthusiasm ( 9 ). While a strong body of literature that instructs students on research methodology, data analysis and writing scientific papers exists, little guidance on performing LRs is available. The LR is a unique opportunity to assess and contrast various arguments and theories, not just summarize them. The research results should not be discussed within the LR, but the postgraduate student tends to write a comprehensive LR while reflecting on his or her own findings ( 10 ).
Many people believe that writing an LR is a lonely and linear process. Supervisors or the institutions assume that the Ph.D. student has mastered the relevant techniques and vocabulary associated with his/her subject and conducts a self-reflection about previously published findings. Indeed, while elaborating the LR, the student should aggregate diverse skills, which mainly rely on his/her own commitment to mastering them. Thus, less supervision should be required ( 11 ). However, the parameters described above might not currently be the case for many students ( 11 , 12 ), and the lack of formal and systematic training on writing LRs is an important concern ( 11 ).
An institutional environment devoted to active learning will provide students the opportunity to continuously reflect on LRs, which will form a dialogue between the postgraduate student and the current literature in a particular field ( 13 ). Postgraduate students will be interpreting studies by other researchers, and, according to Hart (1998) ( 3 ), the outcomes of the LR in a dissertation/thesis include the following:
A sound LR translates the postgraduate student’s expertise in academic and scientific writing: it expresses his/her level of comfort with synthesizing ideas ( 11 ). The LR reveals how well the postgraduate student has proceeded in three domains: an effective literature search, the language domain, and critical writing.
All students should be trained in gathering appropriate data for specific purposes, and information literacy skills are a cornerstone. These skills are defined as “an individual’s ability to know when they need information, to identify information that can help them address the issue or problem at hand, and to locate, evaluate, and use that information effectively” ( 14 ). Librarian support is of vital importance in coaching the appropriate use of Boolean logic (AND, OR, NOT) and other tools for highly efficient literature searches (e.g., quotation marks and truncation), as is the appropriate management of electronic databases.
Academic writing must be concise and precise: unnecessary words distract the reader from the essential content ( 15 ). In this context, reading about issues distant from the research topic ( 16 ) may increase students’ general vocabulary and familiarity with grammar. Ultimately, reading diverse materials facilitates and encourages the writing process itself.
Critical judgment includes critical reading, thinking and writing. It supposes a student’s analytical reflection about what he/she has read. The student should delineate the basic elements of the topic, characterize the most relevant claims, identify relationships, and finally contrast those relationships ( 17 ). Each scientific document highlights the perspective of the author, and students will become more confident in judging the supporting evidence and underlying premises of a study and constructing their own counterargument as they read more articles. A paucity of integration or contradictory perspectives indicates lower levels of cognitive complexity ( 12 ).
Thus, while elaborating an LR, the postgraduate student should achieve the highest category of Bloom’s cognitive skills: evaluation ( 12 ). The writer should not only summarize data and understand each topic but also be able to make judgments based on objective criteria, compare resources and findings, identify discrepancies due to methodology, and construct his/her own argument ( 12 ). As a result, the student will be sufficiently confident to show his/her own voice .
Writing a consistent LR is an intense and complex activity that reveals the training and long-lasting academic skills of a writer. It is not a lonely or linear process. However, students are unlikely to be prepared to write an LR if they have not mastered the aforementioned domains ( 10 ). An institutional environment that supports student learning is crucial.
Different institutions employ distinct methods to promote students’ learning processes. First, many universities propose modules to develop behind the scenes activities that enhance self-reflection about general skills (e.g., the skills we have mastered and the skills we need to develop further), behaviors that should be incorporated (e.g., self-criticism about one’s own thoughts), and each student’s role in the advancement of his/her field. Lectures or workshops about LRs themselves are useful because they describe the purposes of the LR and how it fits into the whole picture of a student’s work. These activities may explain what type of discussion an LR must involve, the importance of defining the correct scope, the reasons to include a particular resource, and the main role of critical reading.
Some pedagogic services that promote a continuous improvement in study and academic skills are equally important. Examples include workshops about time management, the accomplishment of personal objectives, active learning, and foreign languages for nonnative speakers. Additionally, opportunities to converse with other students promotes an awareness of others’ experiences and difficulties. Ultimately, the supervisor’s role in providing feedback and setting deadlines is crucial in developing students’ abilities and in strengthening students’ writing quality ( 12 ).
A consensus on the appropriate method for elaborating an LR is not available, but four main steps are generally accepted: defining the main topic, searching the literature, analyzing the results, and writing ( 6 ). We suggest a fifth step: reflecting on the information that has been written in previous publications ( Figure 2 ).
Planning an LR is directly linked to the research main question of the thesis and occurs in parallel to students’ training in the three domains discussed above. The planning stage helps organize ideas, delimit the scope of the LR ( 11 ), and avoid the wasting of time in the process. Planning includes the following steps:
The ability to gather adequate information from the literature must be addressed in postgraduate programs. Librarian support is important, particularly for accessing difficult texts. This step comprises the following components:
In addition, two other approaches are suggested. First, a review of the reference list of each document might be useful for identifying relevant publications to be included and important opinions to be assessed. This step is also relevant for referencing the original studies and leading authors in that field. Moreover, students can directly contact the experts on a particular topic to consult with them regarding their experience or use them as a source of additional unpublished documents.
Before submitting a dissertation/thesis, the electronic search strategy should be repeated. This process will ensure that the most recently published papers will be considered in the LR.
This task is an important exercise in time management. First, students should read the title and abstract to understand whether that document suits their purposes, addresses the research question, and helps develop the topic of interest. Then, they should scan the full text, determine how it is structured, group it with similar documents, and verify whether other arguments might be considered ( 5 ).
Critical reading and thinking skills are important in this step. This step consists of the following components:
The recognition of when a student is able and ready to write after a sufficient period of reading and thinking is likely a difficult task. Some students can produce a review in a single long work session. However, as discussed above, writing is not a linear process, and students do not need to write LRs according to a specific sequence of sections. Writing an LR is a time-consuming task, and some scholars believe that a period of at least six months is sufficient ( 6 ). An LR, and academic writing in general, expresses the writer’s proper thoughts, conclusions about others’ work ( 6 , 10 , 13 , 16 ), and decisions about methods to progress in the chosen field of knowledge. Thus, each student is expected to present a different learning and writing trajectory.
In this step, writing methods should be considered; then, editing, citing and correct referencing should complete this stage, at least temporarily. Freewriting techniques may be a good starting point for brainstorming ideas and improving the understanding of the information that has been read ( 1 ). Students should consider the following parameters when creating an agenda for writing the LR: two-hour writing blocks (at minimum), with prespecified tasks that are possible to complete in one section; short (minutes) and long breaks (days or weeks) to allow sufficient time for mental rest and reflection; and short- and long-term goals to motivate the writing itself ( 20 ). With increasing experience, this scheme can vary widely, and it is not a straightforward rule. Importantly, each discipline has a different way of writing ( 1 ), and each department has its own preferred styles for citations and references.
In this step, the postgraduate student should ask him/herself the same questions as in the analyzing the results step, which can take more time than anticipated. Ambiguities, repeated ideas, and a lack of coherence may not be noted when the student is immersed in the writing task for long periods. The whole effort will likely be a work in progress, and continuous refinements in the written material will occur once the writing process has begun.
In contrast to review papers, the LR of a dissertation/thesis should not be a standalone piece or work. Instead, it should present the student as a scholar and should maintain the interest of the audience in how that dissertation/thesis will provide solutions for the current gaps in a particular field.
A checklist for evaluating an LR is convenient for students’ continuous academic development and research transparency: it clearly states the desired achievements for the LR of a dissertation/thesis. Here, we present an LR checklist developed from an LR scoring rubric ( 11 ). For a critical analysis of an LR, we maintain the five categories but offer twelve criteria that are not scaled ( Figure 3 ). The criteria all have the same importance and are not mutually exclusive.
1. justified criteria exist for the inclusion and exclusion of literature in the review.
This criterion builds on the main topic and areas covered by the LR ( 18 ). While experts may be confident in retrieving and selecting literature, postgraduate students must convince their audience about the adequacy of their search strategy and their reasons for intentionally selecting what material to cover ( 11 ). References from different fields of knowledge provide distinct perspective, but narrowing the scope of coverage may be important in areas with a large body of existing knowledge.
2. a critical examination of the state of the field exists.
A critical examination is an assessment of distinct aspects in the field ( 1 ) along with a constructive argument. It is not a negative critique but an expression of the student’s understanding of how other scholars have added to the topic ( 1 ), and the student should analyze and contextualize contradictory statements. A writer’s personal bias (beliefs or political involvement) have been shown to influence the structure and writing of a document; therefore, the cultural and paradigmatic background guide how the theories are revised and presented ( 13 ). However, an honest judgment is important when considering different perspectives.
The broader scholarly literature should be related to the chosen main topic for the LR ( how to develop the literature review section). The LR can cover the literature from one or more disciplines, depending on its scope, but it should always offer a new perspective. In addition, students should be careful in citing and referencing previous publications. As a rule, original studies and primary references should generally be included. Systematic and narrative reviews present summarized data, and it may be important to cite them, particularly for issues that should be understood but do not require a detailed description. Similarly, quotations highlight the exact statement from another publication. However, excessive referencing may disclose lower levels of analysis and synthesis by the student.
Situating the LR in its historical context shows the level of comfort of the student in addressing a particular topic. Instead of only presenting statements and theories in a temporal approach, which occasionally follows a linear timeline, the LR should authentically characterize the student’s academic work in the state-of-art techniques in their particular field of knowledge. Thus, the LR should reinforce why the dissertation/thesis represents original work in the chosen research field.
Distinct theories on the same topic may exist in different disciplines, and one discipline may consider multiple concepts to explain one topic. These misunderstandings should be addressed and contemplated. The LR should not synthesize all theories or concepts at the same time. Although this approach might demonstrate in-depth reading on a particular topic, it can reveal a student’s inability to comprehend and synthesize his/her research problem.
The LR is a unique opportunity to articulate ideas and arguments and to purpose new relationships between them ( 10 , 11 ). More importantly, a sound LR will outline to the audience how these important variables and phenomena will be addressed in the current academic work. Indeed, the LR should build a bidirectional link with the remaining sections and ground the connections between all of the sections ( Figure 1 ).
The LR is a ‘creative inquiry’ ( 13 ) in which the student elaborates his/her own discourse, builds on previous knowledge in the field, and describes his/her own perspective while interpreting others’ work ( 13 , 17 ). Thus, students should articulate the current knowledge, not accept the results at face value ( 11 , 13 , 17 ), and improve their own cognitive abilities ( 12 ).
8. the main methodologies and research techniques that have been used in the field are identified and their advantages and disadvantages are discussed.
The LR is expected to distinguish the research that has been completed from investigations that remain to be performed, address the benefits and limitations of the main methods applied to date, and consider the strategies for addressing the expected limitations described above. While placing his/her research within the methodological context of a particular topic, the LR will justify the methodology of the study and substantiate the student’s interpretations.
The audience expects the writer to analyze and synthesize methodological approaches in the field. The findings should be explained according to the strengths and limitations of previous research methods, and students must avoid interpretations that are not supported by the analyzed literature. This criterion translates to the student’s comprehension of the applicability and types of answers provided by different research methodologies, even those using a quantitative or qualitative research approach.
10. the scholarly significance of the research problem is rationalized.
The LR is an introductory section of a dissertation/thesis and will present the postgraduate student as a scholar in a particular field ( 11 ). Therefore, the LR should discuss how the research problem is currently addressed in the discipline being investigated or in different disciplines, depending on the scope of the LR. The LR explains the academic paradigms in the topic of interest ( 13 ) and methods to advance the field from these starting points. However, an excess number of personal citations—whether referencing the student’s research or studies by his/her research team—may reflect a narrow literature search and a lack of comprehensive synthesis of ideas and arguments.
The practical significance indicates a student’s comprehensive understanding of research terminology (e.g., risk versus associated factor), methodology (e.g., efficacy versus effectiveness) and plausible interpretations in the context of the field. Notably, the academic argument about a topic may not always reflect the debate in real life terms. For example, using a quantitative approach in epidemiology, statistically significant differences between groups do not explain all of the factors involved in a particular problem ( 21 ). Therefore, excessive faith in p -values may reflect lower levels of critical evaluation of the context and implications of a research problem by the student.
12. the lr was written with a coherent, clear structure that supported the review.
This category strictly relates to the language domain: the text should be coherent and presented in a logical sequence, regardless of which organizational ( 18 ) approach is chosen. The beginning of each section/subsection should state what themes will be addressed, paragraphs should be carefully linked to each other ( 10 ), and the first sentence of each paragraph should generally summarize the content. Additionally, the student’s statements are clear, sound, and linked to other scholars’ works, and precise and concise language that follows standardized writing conventions (e.g., in terms of active/passive voice and verb tenses) is used. Attention to grammar, such as orthography and punctuation, indicates prudence and supports a robust dissertation/thesis. Ultimately, all of these strategies provide fluency and consistency for the text.
Although the scoring rubric was initially proposed for postgraduate programs in education research, we are convinced that this checklist is a valuable tool for all academic areas. It enables the monitoring of students’ learning curves and a concentrated effort on any criteria that are not yet achieved. For institutions, the checklist is a guide to support supervisors’ feedback, improve students’ writing skills, and highlight the learning goals of each program. These criteria do not form a linear sequence, but ideally, all twelve achievements should be perceived in the LR.
A single correct method to classify, evaluate and guide the elaboration of an LR has not been established. In this essay, we have suggested directions for planning, structuring and critically evaluating an LR. The planning of the scope of an LR and approaches to complete it is a valuable effort, and the five steps represent a rational starting point. An institutional environment devoted to active learning will support students in continuously reflecting on LRs, which will form a dialogue between the writer and the current literature in a particular field ( 13 ).
The completion of an LR is a challenging and necessary process for understanding one’s own field of expertise. Knowledge is always transitory, but our responsibility as scholars is to provide a critical contribution to our field, allowing others to think through our work. Good researchers are grounded in sophisticated LRs, which reveal a writer’s training and long-lasting academic skills. We recommend using the LR checklist as a tool for strengthening the skills necessary for critical academic writing.
Leite DFB has initially conceived the idea and has written the first draft of this review. Padilha MAS and Cecatti JG have supervised data interpretation and critically reviewed the manuscript. All authors have read the draft and agreed with this submission. Authors are responsible for all aspects of this academic piece.
We are grateful to all of the professors of the ‘Getting Started with Graduate Research and Generic Skills’ module at University College Cork, Cork, Ireland, for suggesting and supporting this article. Funding: DFBL has granted scholarship from Brazilian Federal Agency for Support and Evaluation of Graduate Education (CAPES) to take part of her Ph.D. studies in Ireland (process number 88881.134512/2016-01). There is no participation from sponsors on authors’ decision to write or to submit this manuscript.
No potential conflict of interest was reported.
1 The questions posed in systematic reviews usually follow the ‘PICOS’ acronym: Population, Intervention, Comparison, Outcomes, Study design.
2 In 1988, Cooper proposed a taxonomy that aims to facilitate students’ and institutions’ understanding of literature reviews. Six characteristics with specific categories are briefly described: Focus: research outcomes, research methodologies, theories, or practices and applications; Goals: integration (generalization, conflict resolution, and linguistic bridge-building), criticism, or identification of central issues; Perspective: neutral representation or espousal of a position; Coverage: exhaustive, exhaustive with selective citations, representative, central or pivotal; Organization: historical, conceptual, or methodological; and Audience: specialized scholars, general scholars, practitioners or policymakers, or the general public.
Literature Review is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.
Also, we can define a literature review as the collected body of scholarly works related to a topic:
The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic
A literature review is important because it:
All content in this section is from Literature Review Research from Old Dominion University
Keep in mind the following, a literature review is NOT:
Not an essay
Not an annotated bibliography in which you summarize each article that you have reviewed. A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.
Not a research paper where you select resources to support one side of an issue versus another. A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.
A literature review serves several purposes. For example, it
As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.
Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:
Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.
Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.
Historical Review Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.
Methodological Review A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.
Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"
Theoretical Review The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.
* Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147.
All content in this section is from The Literature Review created by Dr. Robert Larabee USC
Robinson, P. and Lowe, J. (2015), Literature reviews vs systematic reviews. Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393
What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California
Systematic review or meta-analysis?
A systematic review answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.
A meta-analysis is the use of statistical methods to summarize the results of these studies.
Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:
Not all systematic reviews contain meta-analysis.
Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review. More information on meta-analyses can be found in Cochrane Handbook, Chapter 9 .
A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies. It is a systematic review that uses quantitative methods to synthesize and summarize the results.
An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings. Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted. In that case, an integrative review is an appropriate strategy.
Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.
A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.
Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.
A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:
Given this, the purpose of a literature review is to:
Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.
Types of Literature Reviews
It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.
In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.
Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].
Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.
Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.
Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.
Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.
Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.
NOTE: Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.
Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews." Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.
I. Thinking About Your Literature Review
The structure of a literature review should include the following in support of understanding the research problem :
The critical evaluation of each work should consider :
II. Development of the Literature Review
Four Basic Stages of Writing 1. Problem formulation -- which topic or field is being examined and what are its component issues? 2. Literature search -- finding materials relevant to the subject being explored. 3. Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4. Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.
Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1. Roughly how many sources would be appropriate to include? 2. What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3. Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4. Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5. Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.
III. Ways to Organize Your Literature Review
Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.
Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.
Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:
IV. Writing Your Literature Review
Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.
Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.
V. Common Mistakes to Avoid
These are the most common mistakes made in reviewing social science research literature.
Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.
Break Out of Your Disciplinary Box!
Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.
Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.
Don't Just Review for Content!
While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:
When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.
Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.
When Do I Know I Can Stop Looking and Move On?
Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:
Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.
Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.
Chapter 5: The Literature Review
Following are a few acceptable sources for literature reviews, listed in order from what will be considered most acceptable to less acceptable sources for your literature review assignments:
A peer reviewed journal article is a paper that has been submitted to a scholarly journal, accepted, and published. Peer review journal papers go through a rigorous, blind review process of peer review. What this means is that two to three experts in the area of research featured in the paper have reviewed and accepted the paper for publication. The names of the author(s) who are seeking to publish the research have been removed (blind review), so as to minimize any bias towards the authors of the research (albeit, sometimes a savvy reviewer can discern who has done the research based upon previous publications, etc.). This blind review process can be long (often 12 to 18 months) and may involve many back and forth edits on the behalf of the researchers, as they work to address the edits and concerns of the peers who reviewed their paper. Often, reviewers will reject the paper for a variety of reasons, such as unclear or questionable methods, lack of contribution to the field, etc. Because peer reviewed journal articles have gone through a rigorous process of review, they are considered to be the premier source for research. Peer reviewed journal articles should serve as the foundation for your literature review.
The following link will provide more information on peer reviewed journal articles. Make sure you watch the little video on the upper left-hand side of your screen, in addition to reading the material at the following website: http://guides.lib.jjay.cuny.edu/c.php?g=288333&p=1922599
An edited academic book is a collection of scholarly scientific papers written by different authors. The works are original papers, not published elsewhere (“Edited volume,” 2018). The papers within the text also go through a process of review; however, the review is often not a blind review because the authors have been invited to contribute to the book. Consequently, edited academic books are fine to use for your literature review, but you also want to ensure that your literature review contains mostly peer reviewed journal papers.
Articles from professional journals should be used with caution for your literature review. This is because articles in trade journals are not usually peer reviewed, even though they may appear to be. A good way to find out is to read the “About Us” section of the professional journal, which should state whether or not the papers are peer reviewed. You can also find out by Googling the name of the journal and adding “peer reviewed” to the search.
Governmental websites can be excellent sources for statistical data, e.g, Statistics Canada collects and publishes data related to the economy, society, and the environment.
Material from other websites can also serve as a source for statistics that you may need for your literature review. Since you want to justify the value of the research that interests you, you might make use of a professional association’s website to learn how many members they have, for example. You might want to demonstrate, as part of the introduction to your literature review, why more research on the topic of PTSD in police officers is important. You could use peer reviewed journal articles to determine the prevalence of PTSD in police officers in Canada in the last ten years, and then use the Ontario Police Officers´ Association website to determine the approximate number of police officers employed in the Province of Ontario over the last ten years. This might help you estimate how many police officers could be suffering with PTSD in Ontario. That number could potentially help to justify a research grant down the road. But again, this type of website- based material should be used with caution and sparingly.
Research Methods for the Social Sciences: An Introduction Copyright © 2020 by Valerie Sheppard is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.
Run a free plagiarism check in 10 minutes, generate accurate citations for free.
Published on May 19, 2022 by Eoghan Ryan . Revised on May 31, 2023.
A tertiary source , also called a reference work, is a source that gives an overview of information gathered from primary and secondary sources but does not provide original interpretations or analysis. Examples include:
These sources types compile information from a wide variety of sources. They may also list, summarize, and index sources that provide original data or direct evidence (primary sources) and sources that describe or interpret this evidence (secondary sources).
What is a tertiary source, examples of tertiary sources, how to tell if a source is tertiary, how and when to use tertiary sources, practice questions, other interesting articles, frequently asked questions about tertiary sources.
There are three types of research sources:
You will mainly use primary and secondary sources, as these provide information that you can analyze or use to formulate your own ideas and arguments.
Tertiary sources do not provide original insights or analyses. Instead, they collect, index, and provide an overview of primary and secondary sources. This means that while you might use them to learn more about a topic you’re new to, you’re unlikely to cite them in your paper.
Tertiary sources provide a wide range of helpful information, including key terms, definitions , lists of relevant sources, and broad overviews.
The key difference between a tertiary source and a primary or secondary source is that the tertiary source does not provide any original insights or analysis.
But what constitutes a tertiary source depends on your research problem and how you use the source.
For example, while encyclopedias are typically considered tertiary sources, a research paper focusing on the development of encyclopedic writing since 1900 might use encyclopedia entries as direct evidence and therefore as primary sources.
To determine whether a source is tertiary, ask:
Although tertiary sources are often credible , they’re not typically attributed to a single author and don’t provide the specialized knowledge expected of scholarly sources . For these reasons, you likely won’t cite tertiary sources in your research paper, but you might still use them behind the scenes in your research.
Use tertiary sources in the beginning stages of your research process to:
This will lay the foundation for further research and direct you to helpful primary and secondary sources that you will engage with in more detail during the writing process .
The AI-powered Citation Checker helps you avoid common mistakes such as:
If you want to know more about ChatGPT, AI tools , citation , and plagiarism , make sure to check out some of our other articles with explanations and examples.
Plagiarism
A tertiary source may list, summarize , or index primary and secondary sources or provide general information from a variety of sources. But it does not provide original interpretations or analysis.
Some examples of tertiary sources include:
Primary sources provide direct evidence about your research topic (photographs, personal letters, etc.).
Secondary sources interpret and comment on information from primary sources (academic books, journal articles, etc.).
Tertiary sources are reference works that identify and provide background information on primary and secondary sources . They do not provide original insights or analysis.
What constitutes a tertiary source depends on your research question and how you use the source.
You usually shouldn’t cite tertiary sources as evidence in your research paper, but you can use them in the beginning stages of the research process to:
Use tertiary sources in your preliminary research to find relevant primary and secondary sources that you will engage with in more depth during the writing process .
If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.
Ryan, E. (2023, May 31). Tertiary Sources Explained | Quick Guide & Examples. Scribbr. Retrieved September 27, 2024, from https://www.scribbr.com/working-with-sources/tertiary-sources/
Other students also liked, primary vs. secondary sources | difference & examples, how to write a literature review | guide, examples, & templates, the basics of in-text citation | apa & mla examples, get unlimited documents corrected.
✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts
What is a literature review.
A literature review is an overview of the available research for a specific scholarly topic. Literature reviews summarize existing research to answer a review question, provide context for new research, or identify important gaps in the existing body of literature.
An incredible amount of academic literature is published each year; by some estimates nearly three million articles .
Sorting through and reviewing that literature can be complicated, so this Research Guide provides a structured approach to make the process more manageable.
A literature search is a systematic search of the scholarly sources in a particular discipline. A literature review is the analysis, critical evaluation and synthesis of the results of that search. During this process you will move from a review of the literature to a review for your research. Your synthesis of the literature is your unique contribution to research.
— those new to reviewing the literature
— those that need a refresher or a deeper understanding of writing literature reviews
You may need to do a literature review as a part of a course assignment, a capstone project, a master's thesis, a dissertation, or as part of a journal article. No matter the context, a literature review is an essential part of the research process.
What is the purpose of a literature review.
A literature review is typically performed for a specific reason. Even when assigned as an assignment, the goal of the literature review will be one or more of the following:
Reviewing the literature helps you understand a research topic and develop your own perspective.
Thanks to Librarian Jamie Niehof at the University of Michigan for providing permission to reuse and remix this Literature Reviews guide.
Journal of Translational Medicine volume 22 , Article number: 873 ( 2024 ) Cite this article
Metrics details
In the management of complex diseases, the strategic adoption of combination therapy has gained considerable prominence. Combination therapy not only holds the potential to enhance treatment efficacy but also to alleviate the side effects caused by excessive use of a single drug. Presently, the exploration of combination therapy encounters significant challenges due to the vast spectrum of potential drug combinations, necessitating the development of efficient screening strategies.
In this study, we propose a prediction scoring method that integrates heterogeneous data using a weighted Bayesian method for drug combination prediction. Heterogeneous data refers to different types of data related to drugs, such as chemical, pharmacological, and target profiles. By constructing a multiplex drug similarity network, we formulate new features for drug pairs and propose a novel Bayesian-based integration scheme with the introduction of weights to integrate information from various sources. This method yields support strength scores for drug combinations to assess their potential effectiveness.
Upon comprehensive comparison with other methods, our method shows superior performance across multiple metrics, including the Area Under the Receiver Operating Characteristic Curve, accuracy, precision, and recall. Furthermore, literature validation shows that many top-ranked drug combinations based on the support strength score, such as goserelin and letrozole, have been experimentally or clinically validated for their effectiveness.
Our findings have significant clinical and practical implications. This new method enhances the performance of drug combination predictions, enabling effective pre-screening for trials and, thereby, benefiting clinical treatments. Future research should focus on developing new methods for application in various scenarios and for integrating diverse data sources.
Combination therapy involves using drugs with distinct pharmacological mechanisms to minimize adverse effects and maximize therapeutic efficacy [ 1 ]. Nowadays, combination therapies of drug repurposing are commonly utilized in treating a spectrum of complex diseases, including cancer [ 2 , 3 ], cardiovascular diseases [ 4 ], and type 2 diabetes [ 5 ], among others [ 6 , 7 , 8 ]. These conditions are highly prevalent and impactful [ 9 , 10 ]. The need for combination therapies arises from the complex nature of these diseases, which involve multiple pathways and genes, making a one-gene, one-drug approach insufficient. Janumet is a combination therapy for type 2 diabetes that merges sitagliptin and metformin [ 11 ]. Sitagliptin boosts insulin secretion, while metformin lowers glucose production and enhances cellular uptake [ 12 ]. This combination effectively manages blood glucose levels and minimizes side effects compared to higher doses of single drugs.
While current combination therapeutic approaches have achieved increasing success and are becoming more crucial in the treatment of various diseases [ 2 , 13 , 14 ], the size of the pre-selected drug combination space is rapidly expanding with the growing number of drugs. Therefore, identifying drug combinations within this vast space that are effective in treatment remains a challenging endeavor. A significant proportion of drug combinations in widespread use today are constructed from clinical observations [ 15 , 16 , 17 ]. However, the drawbacks of relying solely on such discovered mechanisms include time-consuming, labor-intensive, and expensive. This implies that developers require a multitude of multifaceted resources to support the discovery process. Furthermore, clinical in vivo experiments, which can involve hundreds of patients and high costs per trial, may sometimes lead to unnecessary or even harmful treatments being administered to patients [ 18 , 19 ].
One strategy for conducting drug combination researches for in vitro trials is high-throughput screening, which enables the execution of a large number of tests within a reasonable timeframe and at a lower cost compared to clinical trials [ 20 , 21 ]. For instance, high-throughput screening can process up to 100,000 compounds per day, significantly accelerating the initial discovery phase [ 22 ]. However, the high-throughput screening approach is not yet suitable for effectively exploring the entire combination space, as it involves substantial infrastructure costs, and conducting large-scale experiments to test potential drug combinations is both time and cost-intensive. Therefore, there is an urgent need to develop powerful and efficient computational techniques to narrow down the pool of drug candidates for experimentally validated drug combinations in wet trials [ 23 , 24 ], thereby facilitating the identification of promising drug combinations.
Various computational methods have been proposed to predict drug combinations of drug repurposing [ 25 , 26 , 27 , 28 , 29 , 30 ]. Early studies adopt systems biology approaches, employing mathematical models to represent drug perturbations through biochemical reactions and kinetic parameters [ 31 , 32 ]. However, these approaches are often limited to few and well-studied signaling pathways, failing to extract information from the broader pharmacological space. Additionally, chemical biology network analyses have been used to predict effective drug combinations [ 33 , 34 , 35 , 36 , 37 ], but they may struggle to uncover underlying molecular mechanisms or extract information from a larger pharmacological context.
There are computational methods that utilize heterogeneous information of drugs to construct drug knowledge graphs such as multi-dimensional similarity networks, and train computational models to predict drug combinations [ 26 , 27 , 38 , 39 , 40 ]. For example, the PEA model [ 27 ] proposes the utilization of a Naive Bayes network for drug combination prediction. Nevertheless, this method assumes the independence of attributes within a designated classification, embodying an ostensibly simple yet notably strong assumption that proves inadequate in capturing the intricacies inherent in real-world scenarios. Another approach involves Gradient Boost Tree based on feature vectors extracted by restart random walk method [ 26 ]. However, its integration approach for drug-drug similarities is relatively crude, potentially resulting in incomplete utilization of information. Additionally, a recent approach named NEWMIN [ 40 ] has been proposed, which assigns different weights to various similarity features and employs the word2vec method to extract feature vectors for drug pairs. Subsequently, random forest is utilized for prediction. Overall, while each method has its strengths and limitations, integrating advanced computational approaches with comprehensive drug similarity knowledge graphs from heterogeneous information remains crucial for enhancing the accuracy and reliability of drug combination predictions.
In this paper, we propose a novel competitive approach called WBCP for predicting effective drug combinations. WBCP uses a novel weighted Bayesian strategy, that integrates multiple pieces of information from the drug knowledge graph to enhance predictive performance. In contrast to previous studies, our method stands out in several key aspects. First, we convert drug-drug similarities into similarities between query drug pairs and known drug combinations, extracting effective and interpretable features for downstream prediction tasks. Second, the WBCP method enhances Naive Bayes by designing a Bayesian model with attribute weighting and applying it to the likelihood ratios of features, refining the attribute independence assumption to better align with real-world data complexity. Third, it generates a support strength score (0–1), where higher scores indicate greater support for the drug pair belongs to the drug combination class, making the score both intuitive and meaningful. In terms of performance evaluation, our method WBCP has been comprehensively compared with other state-of-the-art methods [ 26 , 27 , 40 ] and several machine learning methods. The results, including the area under the receiver operating characteristic curve (AUC) and the area under the precision-recall curve (AUPR), among others, indicate that our proposed WBCP method is a competitive approach for predicting drug combinations and can play a significant role in the pre-screening of drug combinations.
WBCP method presents a novel algorithmic framework for predicting drug combinations through the integration of heterogeneous data. Initially, seven drug similarity networks are constructed from diverse sources data, encompassing information such as chemical structural, target, and side effects (Fig. 1 A). For each similarity network, similarities between query drug pairs and all known drug combinations are computed, then the maximum similarity value corresponding to the query drug pair serves as a feature for that query drug pair (Fig. 1 B). Subsequently, a weighted Bayesian method is used to amalgamate features into integrated likelihood ratio (ILR), a new statistical measure that evaluates the likelihood of drug combination. The ILR is then transformed into a support strength score (ranging from 0 to 1) using a sigmoid-like function (Fig. 1 C). The support strength score reflects the support strength for classifying the query drug pair to positive group over the negative group, with a higher score indicating greater support.
The workflow of WBCP. A . Constructing drug similarity networks: calculation of the drug similarity networks corresponding to each of seven types heterogenous data, encompassing Anatomical Therapeutic and Chemical (ATC) similarity, Simplified Molecular Input Line Entry System (SMILES) structure similarity, target protein sequence similarity, Gene Ontology (GO) semantic similarity, Kyoto Encyclopedia of Genes and Genomes (KEGG) pathways similarity, SIDER drug side effects similarity, and OFFSIDEs drug effects similarity. B . Constructing seven features for query drug pairs: for each of seven types heterogenous data, a feature of the query drug pair are defined as the maximum similarity between the query drug pair and all known drug combinations. The black dashed line in the figure represents the similarity between drug pairs and known drug combinations, while the red solid line indicates the maximum similarity between drug pairs and all known drug combinations. C . Predicting the drug combination: seven features are integrated to obtain the integrated likelihood ratio (ILR) using a novel weighted Bayesian method. ILR is then transformed to the support strength score for drug combination prediction. Support strength score ranges from 0 to 1, with higher values indicating greater support for the query drug pairs being categorized into the positive group compared to the negative group
In this study, seven types of drug similarity network are calculated by utilizing diverse types of information, encompassing Anatomical Therapeutic and Chemical (ATC) similarity, Simplified Molecular Input Line Entry System (SMILES) structure similarity, target protein sequence similarity, Gene Ontology (GO) semantic similarity, Kyoto Encyclopedia of Genes and Genomes (KEGG) pathways similarity, SIDER drug side effects similarity, and OFFSIDEs drug effects similarity. The data preprocessing includes standardizing data names and formats to ensure consistency across the same type of data. If there are missing data, the corresponding entries are removed.
ATC similarity: ATC code, obtained from the DrugBank database, is a WHO-established pharmaceutical coding system for drug classification based on drug action organ, therapeutic effects, pharmacological features, and chemical characteristics. The inverse document frequency (IDF) is calculated to discount ATC code with the following formula [ 41 ]:
where D represents the set of drugs and n t represents the count of drugs with corresponding ATC code t appears. The ATC similarity of a pair of drugs \(({\text{dA,dB}})\) is calculated as the cosine similarity of their corresponding IDF-weighted vectors \({\mathbf{A}}\) and \({\mathbf{B}}\) . The formula is as follows:
SMILES similarity: The SMILES structure is a simplified molecular input line entry system used to describe the structure of chemical molecules in a string format. The SMILES format structure are extracted from the DrugBank database, which are then transformed to SDFset (a median format for the further calculation of SMILES similarity) using “smiles2sdf” function located in the ChemmineR R package [ 42 ]. Then the atom pair sequences are obtained for drugs [ 43 ], and Tanimoto coefficient between drugs is used for the SMILES similarity measure with “cmp.similarity” function from the ChemmineR R package. The formula is as follows:
where \(a\) represents the atom pairs unique to drug dA, \(b\) represents the atom pairs unique to drug dB, \(c\) represents the atom pairs shared by both two drugs.
Target protein sequence similarity: The protein sequence data of drug targets is downloaded from Uniprot database ( https://www.uniprot.org/ ) [ 44 ], and sequence-constructed structural and physicochemical features have been proved valuable in inferring drug-drug similarity [ 27 ]. Commonly used structural and physicochemical descriptors are obtained from protein sequences. Based on sequence descriptors, the protein sequence similarity is calculated using “parSeqSim” function located in the protr R package [ 45 ].
GO semantic similarity: The GO term of drug targets indicate the biological progress related to corresponding drugs, which is constructed from Uniprot database [ 44 ]. The Jaccard’s coefficient is used to estimate the GO semantic similarity of drugs dA and dB, and the formula is as follows:
where \(\Gamma ( \cdot )\) represents the GO semantic representation of drugs.
KEGG pathways similarity: If two drugs act on the same pathway, then they may have a higher probability of synergistic and complementary effects [ 46 ]. KEGG DRUG contain graphical representation of chemical composition patterns, therapeutic classes, and drug development history. The drug targets enrich in particular KEGG pathways are used for inferring KEGG pathways similarity of drugs based on Jaccard’s coefficient.
SIDER similarity: Campillos et al. demonstrate that if two drugs have similar side effects they elicited, then they tend to share common drug targets [ 47 ]. Therefore, these two drugs are more likely to interact with each other and achieve better therapeutic effects. The side effects of drugs are obtained from SIDER database ( http://sideeffects.embl.de ) [ 48 ], and the SIDER drug side effects similarity between two drugs is calculated with the Jaccard’s coefficient.
OFFSIDES similarity: Offsides database ( http://PharmGKB.org ), as a complement of SIDER database, provides drug effects not already list on the drug’s package insert [ 49 ]. The adverse events may help to uncover potential drug-drug interaction between two drugs, which can identify novel drug combination. In this study, Jaccard’s coefficient is used to infer the OFFSIDES drug effects similarity between two drugs.
As the seven drug similarity networks contained distinct drug nodes, we establish a criterion: if a drug is present in all similarity networks, it is retained along with its connections. Consequently, each similarity network ultimately consists of 558 nodes.
In our study, the features utilized are derived from similarities between pairs of drugs, which are calculated based on seven types of drug-drug similarity. The similarity between a query drug pair (dA, dB) and a known drug combination (dA', dB') is determined from the drug-drug similarities S(dA, dA') and S(dB, dB') [ 50 ]. In the evaluation of drug pair similarities, considering the interchangeable nature of drug pairs (dA, dB) and (dB, dA), it is necessary to consider the similarity between the query drug pair (dB, dA) and the known drug combination (dA', dB'). Consequently, the similarity between the query drug pair (dA, dB) and the known drug combination (dA', dB') can be expressed as follows:
For the i-th drug similarity network, the corresponding \({\text{SP}}_{i} \, (i = 1,2, \cdots ,7)\) can be calculated, we define a feature \(f_{i}\) for the query drug pair (dA, dB):
Subsequently, for each query drug pair, we generate seven features \((f_{1} ,f_{2} , \cdots ,f_{7} )\) as illustrated in Fig. 1 B.
In the integration of seven features \((f_{1} ,f_{2} , \cdots ,f_{7} )\) for drug pairs, we employ a novel weighted Bayesian approach. A Bayesian network serves as a representation of the joint probability distribution among multivariate variables, simplifying the complexity by decomposing the joint probability into a series of manageable modules. Initiating our method with the Bayesian theorem:
by dividing these probabilities, we obtain
\(P( + )\) and \(P( - )\) are the prior probabilities that a drug pair occurs in the positive set and the negative set, and likelihood ratio \(\frac{{P\left( {f_{1} ,f_{2} , \cdots ,f_{7} | + } \right)}}{{P\left( {f_{1} ,f_{2} , \cdots ,f_{7} | - } \right)}}\) constitutes our focal point of interest, encompassing all seven features.
Nevertheless, the computation of the aforementioned likelihood ratio is not a straightforward task. This is because it involves multiple features, necessitating consideration of the relationships between each feature and the integration of the seven features to obtain a comprehensive result. To address the challenge of feature integration as described above, the Naive Bayesian algorithm emerges as a viable approach, recognized for its simplicity, effectiveness, and notable performance across various problem domains [ 51 ]. Under the assumption of Naive Bayesian,
However, the Naive Bayesian relies on the assumption of conditional independence between attributes, which is seldom achievable in practical applications.
Consequently, in this paper, inspired by the boosting naive Bayes model [ 52 ], we propose a novel feature-weighted Bayesian method aimed at mitigating this strong assumption. To begin with, calculate the ILR as follows:
where \(w_{i}\) refers to the weight corresponding to feature \(i \, \left( {i = 1, \, 2, \, \cdots , \, 7} \right)\) , \(P\left( {f_{i} | + } \right)\) and \(P\left( {f_{i} | - } \right)\) represent the probability density of a drug pair exhibiting the feature \(f_{i}\) in positive and negative, respectively.
Next, to better quantify the support for classifying the query drug pair to positive group over the negative group, we transform the ILR into a support strength score,
This score ranges from 0 to 1, with higher values indicating greater support for the query drug pairs being categorized into the positive group compared to the negative group.
First is the calculation of weights. For ease of computation, we first discretize the continuous variables. The discretization process can be carried out using hierarchical clustering [ 53 , 54 ], where the optimal choice of the number of discrete classes is determined by the profile coefficients [ 55 ]. Each discrete class is assigned its corresponding discrete value based on the mean of its results. For the feature \(f_{i} \, (i = 1, \cdots ,7)\) , the corresponding discretized variable is denoted as \(f_{i}^{dis}\) , with distinct values in the discretized data being \(\{ d_{1} , \cdots ,d_{{M_{i} }} \}\) , where \(M_{i}\) is the number of distinct values. The calculation of weights is based on the following formula, which has been modified inspired by the boosting naive Bayes model [ 52 , 56 ]:
where the class includes both a positive set and a negative set. By considering both positive and negative sets in the calculation, a composite result is obtained, reflecting the extent to which each feature influences the prediction classification. Additionally, since different features may have varying numbers of discrete values after discretization, normalization is applied to ensure the comparability of feature weights.
Additionally, the ILR of the seven features are constructed as a weighted product of the likelihood ratios generated by each individual feature. Here, the likelihoods \(P(f_{i} | + )\) and \(P(f_{i} | - )\) can be estimated using kernel density estimation [ 27 ] of Gaussian mixture functions due to the simple mathematical properties of Gaussian functions:
\(N_{ + }\) is the number of drug pairs belonging to the positive group, \(f_{j|i| + }\) representing the j-th value corresponding to the feature \(f_{i}\) in the positive drug pairs, \(b\) represents the bandwidth which is determined by the Silverman's rule of thumb [ 57 , 58 , 59 ]. Similarly, the probability \(P(f_{i} | - )\) can be calculated by the above method.
We obtain the drug combination datasets as benchmark data from the following three data resources: (1) DCDB 2.0 database [ 60 ]: This dataset comprises information from over 140,000 clinical studies and the U.S. Food and Drug Administration (FDA) Orange Book. It encompasses 1363 pairs of drug combinations involving 904 ingredient compositions. For our analysis, only drug combinations labeled as ‘Efficacious’ are utilized. (2) ASDCD [ 61 ]: ASDCD is a database specializing in antifungal synergistic drug combinations. From this resource, we gather 548 pairwise validated combinations. (3) A curated dataset of drug combinations [ 34 ]: This dataset integrates drug combination information from the DCDB 2.0 database, Therapeutic Target Database (TTD), and FDA Orange Book. All drug pairs in this dataset have been either FDA-approved or experimentally validated. This dataset serves as a valuable supplement to our previous two datasets.
These datasets are then merged, with redundant entries removed, to create a comprehensive drug combination dataset. Then, we compared the comprehensive drug combination dataset with those in our drug similarity network and took the intersection, resulting in a final set of 831 known drug combinations used in our study.
In this section, we display the experimental results of the novel WBCP method using 831 known drug combinations as positive samples, alongside an equal number of negative samples. To ensure diversity among negative samples, we employ the K-means algorithm to cluster all drug pairs excluding the known drug combinations. The optimal number of clusters is determined utilizing the “cascadeKM” function from the R package “vegan” [ 62 ], yielding 4 clusters as the optimal choice. Subsequently, an equivalent number of samples are randomly selected from each cluster to form the negative set. The positive and negative samples constitute our benchmark dataset.
The performance of WBCP is evaluated through tenfold cross-validation on benchmark. We conduct an overall performance comparison of WBCP with other methods firstly, then we analyze the performance of the feature extraction and prediction components of the WBCP scheme separately. For feature extraction, we employ visual analysis; additionally, for prediction, we evaluate multiple prediction methods based on the same feature vectors. The results indicate that the WBCP method exhibits a strong level of competitiveness compared to other methods in the context of drug combination prediction.
In this section, we compare WBCP method with other methods on the benchmark dataset. Distinct from WBCP, WBCP_NW represents a comparable scheme without considering weights, and WBCP_MI involves a different weight configuration, where the weights are determined by the mutual information between each constructed feature and the classification outcome in the training set. Additionally, we compare several recently proposed methods for predicting drug combinations based on knowledge graph similarity networks, including PEA [ 27 ], Liu et al. [ 26 ], and NEWMIN [ 40 ], and provide the results in Table 1 .
We conduct an evaluation based on six metrics, namely AUC, AUPR, accuracy, precision, recall, F1-score, for various drug combination prediction methods. These metrics serve as standard and effective evaluation measures. The average performances of each method on the selected metrics across the ten-fold cross-validation along with their corresponding standard deviations are calculated. The mean of the metrics indicates the method’s overall performance, while the standard deviation measures the model's variability and reliability.
As shown in Table 1 , WBCP demonstrates significantly superior performance compared to others across various metrices, including AUC, AUPR, accuracy, precision, and F1-score. Notably, WBCP exhibits a substantial improvement over other methods (AUC = 0.9188), followed by the WBCP_MI (AUC = 0.9170). While WBCP’s AUPR value (AUPR = 0.9174) is slightly lower than WBCP_MI (AUPR = 0.9177), all other performance metrics for WBCP surpass WBCP_MI. The results indicate that this method has overall advantages and is a competitive approach for predicting drug combinations.
In the structure of our WBCP method, the first part involves the extraction of features for drug pairs based on seven similarity networks, while the second part entails the prediction of drug combinations based on the extracted features. Comparable methods with similar structures include those proposed by Liu et al. [ 26 ] and NEWMIN [ 40 ]. For feature extraction, WBCP method involves the maximum similarity between query drug pair and known drug combinations, Liu et al. utilizes the restart random walk method, while NEWMIN employs the word2vec approach. By employing these three methods of feature extraction, we can derive three distinct types of drug pair features based on the benchmark dataset, designated as WBCP features, restart random walk features, and word2vec features.
Regarding the superiority or inferiority of features, we hypothesize that it may be attributed to the boundaries learned during the feature extraction process that differentiate between positive and negative drug pairs. To visualize the distribution of drug pairs’ feature vectors, we apply the t-distributed stochastic neighbor embedding (t-SNE) algorithm [ 63 ] to three distinct types of drug pair features. In the t-SNE plot, tight clustering of positive (or negative) set data indicates that the extracted features effectively capture the similarity within the same categories, while the distance or boundary between positive and negative set clusters suggests effective discrimination between the positive and negative categories.
Figure 2 A illustrates the spatial distribution of positive and negative drug pair features extracted by WBCP. These two independent classes appear somewhat distinguishable, with certain combination pairs even forming distinct clusters. Figure 2 B depicts the spatial distribution of restarted random walk features for the two classes of drug pairs. While negative samples are relatively concentrated, multiple clusters are formed among the positive samples. However, it can be observed that each cluster of positive samples contains negative samples, which may have some impact on classification. Figure 2 C shows the performance of word2vec features for the two classes of drug pairs. Both positive and negative pairs are uniformly distributed in space without clear boundaries or clusters, suggesting that these features may be somewhat generic. Relatively speaking, WBCP features may be a promising option for feature extraction in drug combination prediction. Overall, the features extracted by WBCP perform the best in the t-SNE plot, this performance is likely attributable to the fact that WBCP extracts drug combination features based on similarities with known drug combinations.
The t-SNE plots of three types of drug pair features in benchmark: A WBCP features, B restart random walk features, C word2vec features. The positive pairs are represented by yellow dots, while the negative pairs are represented by blue triangles. Evaluating feature effectiveness through the clustering of two categories in the t-SNE plot
Currently, there are various machine learning methods available for predicting drug combinations based on the constructed feature vectors of drug pairs. In this section, we evaluate the predictive and classification performance of various commonly used machine learning methods and the prediction parts of WBCP and WBCP_MI based on the simple and naive feature vectors in WBCP. The commonly used machine learning methods for prediction and classification include support vector machine (SVM), logistic regression, random forest, k-nearest neighbors (KNN), Naive Bayesian, AdaBoost, and Gradient Boost Tree (more details can be found in Supplementary note S4).
As depicted in Table 2 , based on the constructed feature vectors proposed in WBCP method, various machine learning methods are employed for drug combination prediction. Evaluation metrics encompass AUC, AUPR, and F1-score, the reasons for choosing these three metrics are that AUC and AUPR can assess the overall performance of the methods, while F1-score balances precision and recall, providing a comprehensive evaluation. We have also calculated other metrics, with the results presented in Supplementary Table S1. The evaluation results of WBCP and WBCP_MI are generally leading, which could be attributed to the consideration of attribute-weighted likelihood ratios. This suggests that for predicting drug combinations based on the same feature vectors or embedding vectors, our WBCP method may be competitive across various machine learning methods.
Based on the aforementioned analyses, we seek to validate further the prediction performance of our WBCP across different configurations of drug pair feature vectors. We conduct a comparative assessment of prediction and classification performance of different methods using two other types feature vectors mentioned in the feature visualization section, namely restarted random walk features and word2vec features, extracted from the benchmark dataset. Evaluation metrics encompass AUC, AUPR, and F1-score, we have also calculated other metrics, with the results presented in Supplementary Table S2.
The results displayed in Table 3 affirm the superior performance of the new method WBCP, notably excelling in AUC and AUPR metrics compared to traditional machine learning methodologies. Notably, when employing the restart random walk features, the SVM achieved the best F1-score performance. This disparity may be attributed to our approach of partitioning support strength scores using thresholds, which are determined by the optimal cutoff points derived from the ROC curves [ 64 , 65 ].
In this section, we employ WBCP method to integrate multi-dimensional similarity networks from heterogenous information to predict drug combinations. We do not directly use the benchmark dataset containing 1662 samples as the training dataset, but instead randomly select 750 pairs of known drug combinations as positive set, along with an equal number of negative samples from it as negative set. The prediction samples consist of all samples except for the training samples, containing a total of 153903 query drug pairs.
Next, we conduct two aspects of analysis on the prediction results: first, the prediction and classification results of the remaining 81 pairs of known drug combinations, excluding those in the training samples; second, we rank the support strength scores of all query drug pairs predicted in WBCP, and conduct corresponding literature validation for top-ranking predicted combinations to analyze the reliability of the WBCP prediction process. The literature validation involves searching relevant academic databases such as PubMed and Web of Science using specific keywords to find supporting literature for the drug combination. Keywords include drug names along with terms like ‘combination’, ‘synergy’, and ‘therapy’. For top-ranking drug pairs, if some have not yet been validated in the literature, this suggests that they are potentially valuable combinations for research and warrant further experimental investigation.
For the 81 pairs of known drug combinations in the prediction set, we analyze their support strength scores (more details in supplementary Table S3). Among them, 29 pairs of drug combinations (approximately 35.80%) have support strength scores exceeding 0.9, while 42 pairs (approximately 51.85%) have support strength scores exceeding 0.7. Next, we conduct literature validation for the top-ranking drug combinations predicted by WBCP method (more details in supplementary Table S4). Among the top 20 predicted drug combinations indicate in Table 4 , 13 combinations are supported by existing literature, while seven combinations lacked literature support, indicating potential novel drug combinations.
The drug pair with the highest predicted support strength score is palonosetron and prednisone. In a study involving patients undergoing radiotherapy and concurrent cisplatin treatment for antiemetic prophylaxis, they find that 57% of patients had no vomiting after 5 weeks of treatment, including those treated with palonosetron and prednisone for antiemetic therapy [ 66 ]. Palonosetron is a 5-HT3 receptor antagonist used to prevent and treat chemotherapy-induced nausea and vomiting, while prednisone is a corticosteroid used to treat inflammation or immune-mediated reactions, as well as endocrine or neoplastic diseases [ 67 , 68 ]. On one hand, palonosetron and prednisone can alleviate nausea and vomiting symptoms through different mechanisms, leading to enhanced antiemetic effects when combined [ 69 ]. On the other hand, as palonosetron targets chemotherapy-induced vomiting and prednisone addresses vomiting caused by other reasons, their combination can cover a broader spectrum of vomiting types. Additionally, prednisone may have anti-inflammatory and immunomodulatory effects, which can alleviate other chemotherapy-related discomforts such as pain and inflammation [ 70 ].
In addition to the combination of cancer treatment drugs with antiemetic drugs, another top-ranking prediction validated by literature involves two breast cancer drugs, goserelin and letrozole. As a promising treatment option for premenopausal women with metastatic breast cancer, a combination of gonadotropin-releasing hormone analogs and aromatase inhibitors may be preferable [ 71 ]. It has been indicated that the combination of goserelin and letrozole provides clinical benefits to most patients. Goserelin is a hormone antagonist commonly used to treat breast cancer, prostate cancer, and endometriosis [ 72 ]. It works by inhibiting the release of gonadotropin-releasing hormone (GnRH) from the pituitary gland, thereby suppressing the production of estrogen and progesterone by the ovaries, leading to the inhibition of cancer cell growth [ 73 ]. Letrozole is an aromatase inhibitor. It works by decreasing the amount of estrogen produced in the body [ 74 ]. In premenopausal women with hormone receptor-positive breast cancer, the combination of a GnRH agonist like goserelin along with an aromatase inhibitor like letrozole may be used to suppress ovarian function and lower estrogen levels, thereby slowing the growth of hormone-sensitive tumors [ 75 ]. Additionally, in women undergoing fertility treatment, the combination of goserelin and letrozole may be used to induce ovulation by suppressing the natural hormonal fluctuations that interfere with the ovulation process [ 76 ].
Another top-ranking validated combination involves two drugs used in the treatment of type 2 diabetes mellitus: chlorpropamide and pioglitazone. Combinations of sulfonylurea drugs with thiazolidinedione drugs have been widely reported to enhance glucose-lowering effects [ 77 ]. Chlorpropamide is an oral antidiabetic medication belonging to the sulfonylurea class. Its mechanism of action primarily involves stimulating insulin release and reducing hepatic glycogen synthesis, thereby lowering blood sugar levels [ 78 ]. Pioglitazone, on the other hand, is an oral antidiabetic medication classified as an insulin sensitizer [ 79 ]. It acts on the liver, adipose tissue, and muscles to enhance tissue sensitivity to insulin, increase glucose utilization by tissues, and lower blood sugar levels. The two drugs have different mechanisms of action: chlorpropamide primarily stimulates insulin release, while pioglitazone primarily enhances tissue sensitivity to insulin. They complement each other, and their combination can achieve better glucose-lowering effects [ 77 ]. Additionally, at appropriate doses, the combination of the two drugs can alleviate potential side effects that may occur with monotherapy.
Finally, apart from the drug pairs validated by literature, we conduct partial analysis on the predicted potential drug pairs. Among the newly predicted potential drug combinations, the top-ranking combination is ondansetron and palonosetron. Both drugs are used to address gastrointestinal issues and treat vomiting. Ondansetron is a serotonin 5-HT3 receptor antagonist primarily used to prevent nausea and vomiting in cancer chemotherapy and postoperative settings [ 80 ]. Palonosetron, also a serotonin antagonist, is used for prophylaxis or control of chemotherapy-induced nausea and vomiting, as well as postoperative nausea and vomiting [ 81 ]. It may be possible to increase the blocking effect on 5-HT3 receptors by combining these two drugs, thereby enhancing the antiemetic effect. The next combination is palonosetron and paroxetine. As mentioned earlier, palonosetron is a serotonin 5-HT3 receptor antagonist used for chemotherapy-induced nausea and vomiting, while paroxetine is a selective serotonin reuptake inhibitor mainly used to treat depression, anxiety, and other psychiatric disorders [ 82 ]. This potential combination may be based on several considerations. First, paroxetine is sometimes used to treat nausea and vomiting, particularly those associated with anxiety and depression. Second, in certain situations, there may be a need to enhance the antiemetic effect, especially for patients who do not respond well to standard treatments or require stronger antiemetic effects.
The focal point of drug combinations prediction is to discern effective drug combinations that collaborate synergistically in the treatment of diseases while minimizing adverse effects. The primary challenges in drug combination prediction arise from the vast scale of the search space for potential combinations. Moreover, incorporating heterogeneous information into drug combination prediction further amplifies complexity. One strategy to accomplish this objective is through the utilization of databases enriched with substantial information on drug combinations, coupled with a computational method designed for predicting therapeutic effects. In previous research, the integration between drug similarity networks may have been too coarse, resulting in incomplete utilization of information [ 26 ]. Additionally, some researchers have proposed computational methodologies based on relatively strong assumptions, which may be impractical within real-word scenarios [ 27 ]. Consequently, there exists considerable latitude for refinement and advancement in the computational approaches employed for drug combination prediction. We propose a novel approach, WBCP, tailored for drug combination prediction. The features constructed by the WBCP method are associated with a set of known drug combinations. This method integrates heterogeneous information related to multiple drugs and provides support strength scores describing the priority of predicting drug combinations. Additionally, our WBCP method is applicable for the large-scale prediction of drug combinations, offering both convenience and high practical utility.
WBCP is a specifically designed to accommodate diverse types of data. By integrating weak predictive features, such as chemical structure, target protein sequences, and side effects, within a unified framework, comprehensive feature augmentation is achieved (more details in supplementary note S3). The primary rationale lies in Bayesian methods transforming constructed features into a probabilistic framework to extract latent patterns. Additionally, our proposed inclusion of a novel weighting mechanism in the algorithm weakens the initial assumption of conditional independence between features, thereby enhancing the performance of integration-based computational approaches. Concretely, WBCP transforms the constructed features into the ILR as described Sect. “ Weighted Bayesian method for integrating features ”, the ILR is further converted into the support strength score, which ranges from 0 to 1. Higher support strength score indicate a greater level of support that the drug pair forms an effective drug combination. Traditional methods typically rely on a pattern of extracted feature vectors and strong assumptions made by machine learning predictions. In contrast, our method constructs maximum similarity between query drug pairs and a set of known drug combinations, and mitigating the strong assumptions previously imposed that did not align with real-world scenarios.
We conduct experiments on large-scale datasets, and the results demonstrate that WBCP outperforms other state-of-the-art methods. Our research reveals some notable findings, with one key observation emphasizing that integrating features by assigning individual weights to each feature, rather than assuming conditional independence, not only alleviates potential assumptions but also enhances the predictive capability of the model (more details in supplementary Table S5). Additionally, visual analysis of the features extracted by our method compared to others reveals that our newly constructed features, namely the maximum similarity feature between queried drug pairs and known drug combinations, exhibit blurred boundaries and clustering or cluster formation in spatial distribution, which is advantageous for distinguishing positive and negative samples. In the literature validation, many of the top-ranked drug combinations have been supported by previous studies demonstrating their efficacy through experiments or clinical use. For example, the combination of the two breast cancer drugs, goserelin and letrozole, has been validated [ 71 ]. In conclusion, compared to other computational methods, WBCP demonstrates greater capability in identifying drug combinations.
There are several key points to mention about this study. Utilizing drug characterization within the framework of molecular networks, along with target and side effect information and so on, in the prediction of drug combinations may offers a transparent comprehension and interpretability of the underlying biological mechanisms, facilitating further research. Additionally, we incorporate two sources of drug side effect information, one obtained from SIDER database and the other from Offsides database. The inclusion of the latter is particularly noteworthy, as it serves as a valuable supplement to the former, providing information on drug side effects not documented in drug manuals. Additionally, we explore the application of this method to the performance of drug combinations in various cell lines. The data is high-dimensional, containing not only drug information but also cell line information. The method's performance is presented in the supplementary note S5.
Secondly, our method exhibits exceptional scalability. On one hand, when predicting the effects of a drug pair with unknown outcomes for the first time, it requires only the computation of constructed features for the drug pair within a set of known drug combinations. Subsequently, the ILR is calculated and converted into a support strength score. On the other hand, beyond the seven features previously mentioned, numerous additional heterogenous information can characterize a drug. If any such information proves to be an effective complement, our method allows for the efficient calculation of its weight, facilitating its seamless integration into the overall process. This simplicity and effectiveness enhance the versatility of our method.
We posit that our method holds considerable potential across various domains, notably in drug development, large-scale clinical trial design, and the strategic guidance of in vitro experiments. The potential impact of our method is underscored by its capacity to inform and optimize the identification of promising drug combinations, providing researches with a robust tool for enhancing efficacy and resource efficacy in the design of expensive clinical trials. Moreover, the structured application of our method facilitates the strategic planning and execution of in vitro experiments, offering a systematic approach to enhance precision and mitigate costs associated with extensive trial initiatives.
In the drug combination prediction framework, there are still some directions for further exploration in future research. Firstly, we posit that the incorporation of additional clinical information, such as drug dosage, could enhance advantages. Given that one of the primary objectives of drug combinations is to mitigate drug side effects, some of which may result from overdosing, integrating more comprehensive clinical information may enhance the chances of success [ 83 ]. Secondly, Specifying the disease itself is another noteworthy direction for future research in predicting drug combinations. Since drug combinations are designed for specific indications, incorporating disease-specific information would offer a more nuanced perspective. This approach could be especially beneficial for complex conditions such as cancer or cardiovascular diseases, where tailored drug combinations are critical [ 2 , 10 ]. Lastly, the mode of drug delivery emerges as a pertinent element that can be considered in the future, as it has been shown that different drug dosage forms and delivery modes have a significant impact on drug efficacy [ 84 ].
This study proposes a computational method for predicting drug combinations, named WBCP. The WBCP algorithm integrates various data types, such as chemical structures, target protein sequences, and side effects, into a unified framework. The method integrates a Bayesian approach with a novel weighting mechanism to relax the assumptions of conditional independence between features, demonstrating competitive performance in comprehensive evaluations. WBCP method presents significant potential in drug development, clinical trial design, and in vitro experiment planning, optimizing the identification of promising drug combinations and thereby improving efficacy and reducing costs in these processes.
The datasets and codes analyzed during the current study are available in the website: https://github.com/YQHuFD/WBCP .
He B, Lu C, Zheng G, et al. Combination therapeutics in complex diseases. J Cell Mol Med. 2016;20(12):2231–40.
Article PubMed PubMed Central Google Scholar
Mokhtari RB, Homayouni TS, Baluch N, et al. Combination therapy in combating cancer. Oncotarget. 2017;8(23):38022–43.
Article PubMed Central Google Scholar
Ahmed F, Samantasinghar A, Soomro AM, et al. A systematic review of computational approaches to understand cancer biology for informed drug repurposing. J Biomed Inform. 2023;142:104373.
Article PubMed Google Scholar
Mangiafico S, Costello-Boerrigter LC, Andersen IA, et al. Neutral endopeptidase inhibition and the natriuretic peptide system: an evolving strategy in cardiovascular therapeutics. Eur Heart J. 2013;34(12):886–93.
Article CAS PubMed Google Scholar
The ACCORD Study Group. Effects of combination lipid therapy in type 2 diabetes mellitus. N Engl J Med. 2010;362(17):1563–74.
Butterfield LH, Najjar YG. Immunotherapy combination approaches: mechanisms, biomarkers and clinical observations. Nat Rev Immunol. 2024;24(6):399–416.
Samantasinghar A, Ahmed F, Rahim CSA, et al. Artificial intelligence-assisted repurposing of lubiprostone alleviates tubulointerstitial fibrosis. Transl Res. 2023;262:75–88.
Ahmed F, Ho SG, Samantasinghar A, et al. Drug repurposing in psoriasis, performed by reversal of disease-associated gene expression profiles. Comput Struct Biotec. 2022;20:6097–107.
Article CAS Google Scholar
Bray F, Laversanne M, Sung H, et al. Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. Ca-Cancer J Clin. 2024;74(3):229–63.
Samuel PO, Edo GI, Emakpor OL, et al. Lifestyle modifications for preventing and managing cardiovascular diseases. Sport Sci Hlth. 2024;20(1):23–36.
Google Scholar
Reynolds JK, Neumiller JJ, Campbell RK. Janumet™: a combination product suitable for use in patients with type 2 diabetes. Expert Opin Inv Drug. 2008;17(10):1559–65.
Daneshjou D, Mehranjani MS, Zadehmodarres S, et al. Sitagliptin/metformin improves the fertilization rate and embryo quality in polycystic ovary syndrome patients through increasing the expression of GDF9 and BMP15: a new alternative to metformin (a randomized trial). J Reprod Immunol. 2022;150:103499.
Tambuyzer E, Vandendriessche B, Austin CP, et al. Therapies for rare diseases: therapeutic modalities, progress and challenges ahead. Nat Rev Drug Discov. 2020;19(2):93–111.
Samantasinghar A, Sunildutt NP, Ahmed F, et al. A comprehensive review of key factors affecting the efficacy of antibody drug conjugate. Biomed Pharmacother. 2023;161:114408.
Jia J, Zhu F, Ma X, et al. Mechanisms of drug combinations: interaction and network perspectives. Nat Rev Drug Discov. 2009;8(2):111–28.
Al-Lazikani B, Banerji U, Workman P. Combinatorial drug therapy for cancer in the post-genomic era. Nat Biotechnol. 2012;30(7):679–92.
Asif A, Park SH, Manzoor Soomro A, et al. Microphysiological system with continuous analysis of albumin for hepatotoxicity modeling and drug screening. J Ind Eng Chem. 2021;98:318–26.
Idée J, Port M, Raynal I, et al. Clinical and biological consequences of transmetallation induced by contrast agents for magnetic resonance imaging: a review. Fund Clin Pharmacol. 2006;20(6):563–76.
Article Google Scholar
Sunildutt N, Parihar P, Salih ARC, et al. Revolutionizing drug development: harnessing the potential of organ-on-chip technology for disease modeling and drug discovery. Front Pharmacol. 2023;14:1139229.
Article CAS PubMed PubMed Central Google Scholar
Sun X, Vilar S, Tatonetti NP. High-throughput methods for combinatorial drug discovery. Sci Transl Med. 2013;5(205):205rv1.
Cokol M, Chua HN, Tasan M, et al. Systematic exploration of synergistic drug pairs. Mol Syst Biol. 2011;7:544.
Armstrong JW. A review of high-throughput screening approaches for drug discovery. Am Biotechnol Lab. 1999;17(1):26–8.
CAS Google Scholar
Liu H, Fan Z, Lin J, et al. The recent progress of deep-learning-based in silico prediction of drug combination. Drug Discov Today. 2023;28(7):103625.
Costello JC, Heiser LM, Georgii E, et al. A community effort to assess and improve drug sensitivity prediction algorithms. Nat Biotechnol. 2014;32(12):1202–12.
Gu J, Bang D, Yi J, et al. A model-agnostic framework to enhance knowledge graph-based drug combination prediction with drug–drug interaction data and supervised contrastive learning. Brief Bioinform. 2023;24(5):bbab285.
Liu H, Zhang W, Nie L, et al. Predicting effective drug combinations using gradient tree boosting based on features extracted from drug-protein heterogeneous network. BMC Bioinformatics. 2019;20(1):645.
Li P, Huang C, Fu Y, et al. Large-scale exploration and analysis of drug combinations. Bioinformatics. 2015;31(2):2007–16.
Ahmed F, Kang IS, Kim KH, et al. Drug repurposing for viral cancers: a paradigm of machine learning, deep learning, and virtual screening-based approaches. J Med Virol. 2023;95(4):e28693.
Ahmed F, Lee JW, Samantasinghar A, et al. SperoPredictor: an integrated machine learning and molecular docking-based drug repurposing framework with use case of COVID-19. Front Public Health. 2022;10:902123.
Sunildutt N, Ahmed F, Salih ARC, et al. Integrating transcriptomic and structural insights: revealing drug repurposing opportunities for sporadic ALS. ACS Omega. 2024;9(3):3793–806.
CAS PubMed PubMed Central Google Scholar
Ryall KA, Tan AC. Systems biology approaches for advancing the discovery of effective drug combinations. J Cheminformatics. 2015;7:1–15.
Chou T-C. Drug combination studies and their synergy quantification using the Chou-Talalay method. Cancer Res. 2010;70(2):440–6.
Tang J, Karhinen L, Xu T, et al. Target inhibition networks: predicting selective combinations of druggable targets to block cancer survival pathways. PLoS Comput Biol. 2013;9(9):e1003226.
Cheng F, Kovács IA, Barabási AL. Network-based prediction of drug combinations. Nat Commun. 2019;10(1):1197.
Ahmed F, Soomro AM, Chethikkattuveli Salih AR, et al. A comprehensive review of artificial intelligence and network based approaches to drug repurposing in Covid-19. Biomed Pharmacother. 2022;153:113350.
Ahmed F, Samantasinghar A, Ali W, et al. Network-based drug repurposing identifies small molecule drugs as immune checkpoint inhibitors for endometrial cancer. Mol Divers. 2024;1–17.
Ahmed F, Yang YJ, Samantasinghar A, et al. Network-based drug repurposing for HPV-associated cervical cancer. Comput Struct Biotec. 2023;21:5186–200.
Iadevaia S, Lu Y, Morales FC, et al. Identification of optimal drug combinations targeting cellular networks: integrating phospho-proteomics and computational network analysis. Cancer Res. 2010;70(17):6704–14.
Huang L, Li F, Sheng J, et al. DrugComboRanker: drug combination discovery based on target network analysis. Bioinformatics. 2014;30(12):i228-236.
Yu L, Xia M, An Q. A network embedding framework based on integrating multiplex network for drug combination prediction. Brief Bioinform. 2022;23(1):bbab364.
Kastrin A, Ferk P, Leskošek B. Predicting potential drug-drug interactions on topological and semantic similarity features using statistical learning. PLoS ONE. 2018;13(5):e0196865.
Cao Y, Charisi A, Cheng LC, et al. ChemmineR: a compound mining framework for R. Bioinformatics. 2008;24(5):1733–4.
Chen X, Reynolds CH. Performance of similarity measures in 2D fragment-based similarity searching: comparison of structural descriptors and similarity coefficients. J Chem Inf Comput Sci. 2002;42(6):1407–14.
UniProt Consortium. UniProt: the universal protein knowledgebase in 2021. Nucleic Acids Res. 2021;49(D1):D480–9.
Xiao N, Cao DS, Zhu MF, et al. protr/ProtrWeb: R package and web server for generating various numerical representation schemes of protein sequences. Bioinformatics. 2015;31(11):1857–9.
Dücker C, Brockmöller J. How precise is quantitative prediction of pharmacokinetic effects due to drug-drug interactions and genotype from in vitro data? A comprehensive analysis on the example CYP2D6 and CYP2C19 substrates. Pharmacol Ther. 2021;217:107629.
Campillos M, Kuhn M, Gavin AC, et al. Drug target identification using side-effect similarity. Science. 2008;321(5886):263–6.
Kuhn M, Letunic I, Jensen LJ, et al. The SIDER database of drugs and side effects. Nucleic Acids Res. 2016;44(D1):D1075–9.
Tatonetti NP, Ye PP, Daneshjou R, et al. Data-driven prediction of drug effects and interactions. Sci Transl Med. 2012;4(125):125ra31.
Song D, Chen Y, Min Q, et al. Similarity-based machine learning support vector machine predictor of drug-drug interactions with improved accuracies. J Clin Pharm Ther. 2019;44(2):268–75.
Rish I. An empirical study of the naive Bayes classifier. IJCAI 2001 workshop on empirical methods in artificial intelligence. 2001; 3(22):41–46.
Vidaurre D, Bielza C, Larrañaga P. Forward stagewise naïve Bayes. Prog Artif Intell. 2012;1:57–69.
Dash R, Paramguru RL, Dash R. Comparative analysis of supervised and unsupervised discretization techniques. Int J Adv Sci Technol. 2011;2(3):29–37.
Chmielewski MR, Grzymala-Busse JW. Global discretization of continuous attributes as preprocessing for machine learning. Int J Approx Reason. 1996;15(4):319–31.
Murtagh F, Contreras P. Algorithms for hierarchical clustering: an overview. Wires Data Min Knowl. 2012;2(1):86–97.
Ferreira JTAS, Denison DGT, Hand DJ. Data mining with products of trees. International symposium on intelligent data analysis. Berlin, Heidelberg: Springer; 2001. pp. 167–76.
R Core Team. R: A language and environment for statistical computing. 2021. https://www.R-project.org/ .
Andersson B, von Davier AA. Improving the bandwidth selection in kernel equating. J Educ Meas. 2014;51(3):223–38.
Chen S. Optimal bandwidth selection for kernel density functionals estimation. J Probab Stat. 2015;2015(1): 242683.
Liu Y, Wei Q, Yu G, et al. DCDB 2.0: a major update of the drug combination database. Database (Oxford). 2014;2014:bau124.
Chen X, Ren B, Chen M, et al. ASDCD: antifungal synergistic drug combination database. PLoS ONE. 2014;9(1): e86499.
Oksanen J, Simpson GL, Blanchet FG, et al. vegan: Community ecology package. 2022. https://CRAN.R-project.org/package=vegan .
Van der Maaten L, Hinton G. Visualizing data using t-SNE. J Mach Learn Res. 2008;9(11):2579–605.
Nahm FS. Receiver operating characteristic curve: overview and practical use for clinicians. Korean J Anesthesiol. 2022;75(1):25–36.
Habibzadeh F, Habibzadeh P, Yadollahie M. On determining the most appropriate test cut-off value: the case of tests with continuous results. Biochem Medica. 2016;26(3):297–307.
Ruhlmann CH, Belli C, Dahl T, et al. Palonosetron and prednisolone for the prevention of nausea and emesis during fractionated radiotherapy and 5 cycles of concomitant weekly cisplatin—a phase II study. Support Care Cancer. 2013;21(12):3425–31.
Piechotta V, Adams A, Haque M, et al. Antiemetics for adults for prevention of nausea and vomiting caused by moderately or highly emetogenic chemotherapy: a network meta-analysis. Cochrane Db Syst Rev. 2021;11(11):CD012775.
Shaikh S, Verma H, Yadav N, et al. Applications of steroid in clinical practice: a review. Int Sch Res Notices. 2012;2012(1):985495.
Fabi A, Malaguti P. An update on palonosetron hydrochloride for the treatment of radio/chemotherapy-induced nausea and vomiting. Expert Opin Pharmaco. 2013;14(5):629–41.
Glare P, Aubrey K, Gulati A, et al. Pharmacologic management of persistent pain in cancer survivors. Drugs. 2022;82(3):275–91.
Yao S, Xu B, Li Q, et al. Goserelin plus letrozole as first- or second-line hormonal treatment in premenopausal patients with advanced breast cancer. Endocr J. 2011;58(6):509–16.
Moore HC, Unger JM, Phillips K-A, et al. Goserelin for ovarian protection during breast-cancer adjuvant chemotherapy. New Engl J Med. 2015;372(10):923–32.
Limonta P, Moretti RM, Marelli MM, et al. The biology of gonadotropin hormone-releasing hormone: role in the control of tumor growth and progression in humans. Front Neuroendocrinol. 2003;24(4):279–95.
Haynes BP, Dowsett M, Miller WR, et al. The pharmacology of letrozole. J Steroid Biochem. 2003;87(1):35–45.
Lu Y-S, Wong A, Kim HJ. Ovarian function suppression with luteinizing hormone-releasing hormone agonists for the treatment of hormone receptor-positive early breast cancer in premenopausal women. Front Oncol. 2021;11:700722.
Zhou H, Cao D, Yang J, et al. Gonadotropin-releasing hormone agonist combined with a levonorgestrel-releasing intrauterine system or letrozole for fertility-preserving treatment of endometrial carcinoma and complex atypical hyperplasia in young women. Int J Gynecol Cancer. 2017;27(6):1178–82.
Scheen AJ. Drug interactions of clinical importance with antihyperglycaemic agents: an update. Drug Saf. 2005;28(7):601–31.
Osadebe PO, Odoh EU, Uzor PF. Oral anti-diabetic agents-review and updates. Br J Med Med Res. 2015;5(2):134–59.
Mudaliar S, Henry RR. New oral therapies for type 2 diabetes mellitus: the glitazones or insulin sensitizers. Annu Rev Med. 2001;52(1):239–57.
Ye J-H, Ponnudurai R, Schaefer R. Ondansetron: a selective 5-HT3 receptor antagonist and its applications in CNS-related disorders. CNS Drug Rev. 2001;7(2):199–213.
Muchatuta NA, Paech MJ. Management of postoperative nausea and vomiting: focus on palonosetron. Ther Clin Risk Manag. 2009;5(1):21–34.
Wagstaff AJ, Cheer SM, Matheson AJ, et al. Paroxetine: an update of its use in psychiatric disorders in adults. Drugs. 2002;62(4):655–703.
Powell JR, Cook J, Wang Y, et al. Drug dosing recommendations for all patients: a roadmap for change. Clin Pharmacol Ther. 2021;109(1):65–72.
Wen H, Jung H, Li X. Drug delivery approaches in addressing clinical pharmacology-related issues: opportunities and challenges. AAPS J. 2015;17(6):1327–40.
Download references
Thanks to all those who maintain excellent databases and to all experimentalists who enabled this work by making their data publicly available.
This research is supported by National Key R&D Program of China (2023YFF1205101).
Authors and affiliations.
State Key Laboratory of Genetic Engineering, Human Phenome Institute, Institute of Biostatistics, School of Life Sciences, Fudan University, Shanghai, China
Tingting Li, Long Xiao, Anqi Chen & Yue-Qing Hu
Department of Gastrointestinal Surgery, Renji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
Haigang Geng
Shanghai Center for Mathematical Sciences, Fudan University, Shanghai, China
Yue-Qing Hu
You can also search for this author in PubMed Google Scholar
T. L wrote the manuscript; Y. H and T. L designed the research; T. L performed the research; T. L analyzed the data; L. X, H. G and A. C contributed comments on the first manuscript.
Correspondence to Yue-Qing Hu .
Competing interests.
No potential competing interest is reported by the authors.
Publisher's note.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional file 1., rights and permissions.
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .
Reprints and permissions
Cite this article.
Li, T., Xiao, L., Geng, H. et al. A weighted Bayesian integration method for predicting drug combination using heterogeneous data. J Transl Med 22 , 873 (2024). https://doi.org/10.1186/s12967-024-05660-3
Download citation
Received : 20 May 2024
Accepted : 04 September 2024
Published : 28 September 2024
DOI : https://doi.org/10.1186/s12967-024-05660-3
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
ISSN: 1479-5876
5 things to know from this week’s big report on cannabis.
A new scientific report finds that the gap between federal and state regulations on cannabis is leading to emerging problems with public health Jim Mone/AP hide caption
More than half of all U.S. states have legalized cannabis, be it for medical purposes, recreational use, or both. The shelves of cannabis dispensaries offer an ever-widening array of gummies, drinks and joints.
Meanwhile, the federal government still considers most types of cannabis illegal.
A new report from the National Academies of Sciences, Engineering and Medicine, released this week, finds this disconnect between the states and the federal government is leading to fragmented policies, and risks to the public.
As states built new commercial markets for cannabis, they initially focused on regulating sales and revenue. “The consequence of that is the public health aspects were often given a backseat and we're now playing catch up for that,” says Dr. Steven Teutsch , chair of the National Academies committee that wrote the report on how cannabis impacts public health.
The report calls for federal leadership and national standards on cannabis quality and potency, to safeguard public health.
Here are five takeaways:
In 2022, more U.S. adults reported using cannabis than alcohol on a near-daily basis, according to the National Survey on Drug Use and Health. It was the first time that regular marijuana use surpassed regular alcohol use.
Regular cannabis use has skyrocketed in the past 30 years — from fewer than 1 million people reporting near-daily use in 1992, to more than 17 million in 2022.
Weed has gotten more accessible as it’s gained legal status in many states — around two-thirds of those 12 and older consider it to be “fairly easy” or “very easy” to obtain. And it’s also dropped in price, in terms of the price per unit of delta-9-tetrahydrocannabinol, or THC — the primary compound responsible for its psychoactive effects.
The concentration of THC in cannabis flower has increased over time. “I think most people are aware of the phenomenon that 'this is not your grand daddy's weed'... I hear this all the time," Staci Gruber, with the Harvard-affiliated McLean Hospital, told NPR in 2019 .
And while smoking dried cannabis flower is still how most people consume weed, there’s been a rise in cannabis edibles, vape oils and other products, says Dr. Yasmin Hurd , director of the Addiction Institute at Mt. Sinai and vice chair of the NASEM committee.
“There are now concentrates such as dabs, wax and shatter that contain very high concentrations of THC, even in the range of 60% to 90%,” she says. Hurd spoke at a press conference on Thursday announcing the report’s release.
Higher concentrations of THC make it more likely for people to take more than they intend to, which has contributed to more traffic accidents and hospital visits related to cannabis use, Hurd says.
While many states with legalized cannabis use have set limits on the amount of THC in gummies and other edibles, those rules often don’t apply to other cannabis products, according to the report.
Cannabis is classified as a Schedule I substance by the U.S. Drug Enforcement Administration, meaning that the federal government considers it to be a drug with high abuse potential, and no accepted medical use.
Thanks to the 2018 Farm Bill — which defined a subset of cannabis as hemp, and excluded it from the Controlled Substances Act — there’s been a boom in products containing hemp-derived chemicals. These include CBD and delta-8 THC , a psychoactive compound extracted and synthesized from CBD, and they can be sold in states that have not legalized cannabis.
These have evaded regulation, though some of these chemicals have been processed to increase their psychoactive properties. U.S. health officials from the CDC and FDA have warned about the health and safety risks.
The report recommends that Congress close this loophole, by specifying that all intoxicating forms of cannabis — including those derived from hemp — are subject to regulation.
Scientific research on the health effects of cannabis has advanced little in recent years, because there are huge barriers to studying the drug.
Since cannabis is classified as a Schedule I substance, researchers often can’t get it for studies. Even if they can, they contend with all kinds of tight regulations.
The White House Office of National Drug Policy isn’t allowed to study the impacts of legalizing cannabis, even though it’s already happened in many states.
Earlier this year, the DEA proposed reclassifying cannabis as a Schedule III drug, like ketamine — one with recognized medical uses, low to moderate potential for abuse, and fewer restrictions.
The report also recommends that Congress remove the restrictions on research for the Office of National Drug Policy.
People tend to think that cannabis is less dangerous once it’s been legalized, Hurd says.
But many people have not been fully informed of the potential harms. “The risks associated with THC consumption— psychosis, suicidal ideation, cannabis use disorder — those increase as the dose increases,” Hurd says.
More kids and young adults are now seeing pro-cannabis messages through advertising, and the cannabis industry lobby is increasingly influential — successful in swatting down efforts to limit THC concentration in Washington, for instance, or to limit pesticide use on cannabis farms in Colorado, according to the report .
“We really need to approach cannabis with a public health framework,” Dr. Pamela Ling, director of the UCSF Center for Tobacco Control Research and Education, wrote in an email after reviewing the report at NPR’s request.
“The good news is we don't have to start from scratch. We have models from best practices from tobacco control and alcohol that can be applied to cannabis — particularly regarding marketing restrictions, age restrictions, the retail environment, taxation, and ways to decrease youth access,” she says.
The report also recommends public health campaigns that describe the risks, especially for kids and young adults, those who are pregnant and the elderly. And it calls for training cannabis retail staff to talk knowledgeably about the risks and benefits to customers.
IMAGES
VIDEO
COMMENTS
Sources for literature review and examples. Generally, your literature review should integrate a wide range of sources such as: Books. Textbooks remain as the most important source to find models and theories related to the research area. Research the most respected authorities in your selected research area and find the latest editions of ...
Revised on May 31, 2023. Throughout the research process, you'll likely use various types of sources. The source types commonly used in academic writing include: Academic journals. Books. Websites. Newspapers. Encyclopedias. The type of source you look for will depend on the stage you are at in the writing process.
Scholarly, professional literature falls under 3 categories, primary, secondary, and tertiary. Published works (also known as a publication) may fall into one or more of these categories, depending on the discipline. See definitions and linked examples of primary, secondary, and tertiary sources. Differences in Publishing Norms by Broader ...
Primary sources provide raw information and first-hand evidence. Examples include interview transcripts, statistical data, and works of art. Primary research gives you direct access to the subject of your research. Secondary sources provide second-hand information and commentary from other researchers. Examples include journal articles, reviews ...
Generally, these sources are commenting on, analyzing, interpreting, or evaluating primary sources. These types of sources help researchers contextualize what's happening in their field, and they can contribute to the direction of primary research by identifying longer-term trends and implications. In college, many of the papers and articles ...
The term primary source is used broadly to embody all sources that are original. P rimary sources provide first-hand information that is closest to the object of study. Primary sources vary by discipline. In the natural and social sciences, original reports of research found in academic journals detailing the methodology used in the research, in-depth descriptions, and discussions of the ...
Examples of literature reviews. Step 1 - Search for relevant literature. Step 2 - Evaluate and select sources. Step 3 - Identify themes, debates, and gaps. Step 4 - Outline your literature review's structure. Step 5 - Write your literature review.
Tertiary Literature. Tertiary literature consists of a distillation and collection of primary and secondary sources such as textbooks, encyclopedia articles, and guidebooks or handbooks. The purpose of tertiary literature is to provide an overview of key research findings and an introduction to principles and practices within the discipline.
Finding sources (scholarly articles, research books, dissertations, etc.) for your literature review is part of the research process. This process is iterative, meaning you repeat and modify searches until you have gathered enough sources for your project. The main steps in this research process are:
Your literature review should contain the following information: The most pertinent studies and important past and current research and practices in the field; An overview of sources you have explored while researching a particular topic; An explanation to your readers as to how your research fits within a larger field of study.
Research for your literature review can be categorised as either primary or secondary in nature. The simplest definition of primary sources is either original information (such as survey data) or a first person account of an event (such as an interview transcript). ... In some instances, material may act as a secondary source for one research ...
Sources of information or evidence are often categorized as primary, secondary, or tertiary material. These classifications are based on the originality of the material and the proximity of the source or origin. This informs the reader as to whether the author is reporting information that is first hand or is conveying the experiences and ...
Generally, there are three basic types of information sources in research including primary, secondary, and tertiary. They are as follows: Primary Sources: Primary sources of information are first hand accounts of research or an event including original scholarly research results, raw data, testimony, speeches, historic objects or other ...
Secondary Source - These sources are translated, repackaged, restated, analyzed, or interpreted original information that is a primary source. Thus, the information comes to us secondhand, or through at least one filter. Here are some examples that are often used as secondary sources: All nonfiction books and magazine articles except ...
Primary source: Usually a report by the original researchers of a study (unfiltered sources) Secondary source: Description or summary by somebody other than the original researcher, e.g. a review article (filtered sources) Conceptual/theoretical: Papers concerned with description or analysis of theories or concepts associated with the topic.
Literature search. Fink has defined research literature review as a "systematic, explicit and reproducible method for identifying, evaluating, and synthesizing the existing body of completed and recorded work produced by researchers, scholars and practitioners."[]Review of research literature can be summarized into a seven step process: (i) Selecting research questions/purpose of the ...
A good quality literature review involves searching a number of databases individually. The most common method is to search a combination of large inter-disciplinary databases such as Scopus & Web of Science Core Collection, and some subject-specific databases (such as PsycInfo or EconLit etc.). The Library databases are an excellent place to ...
A sophisticated literature review (LR) can result in a robust dissertation/thesis by scrutinizing the main problem examined by the academic study; anticipating research hypotheses, methods and results; and maintaining the interest of the audience in how the dissertation/thesis will provide solutions for the current gaps in a particular field.
The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic. A literature review is important because it: Explains the background of research on a topic. Demonstrates why a topic is significant to a subject area. Discovers relationships between research studies/ideas.
A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories.A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that ...
5.3 Acceptable sources for literature reviews Following are a few acceptable sources for literature reviews, listed in order from what will be considered most acceptable to less acceptable sources for your literature review assignments: ... What this means is that two to three experts in the area of research featured in the paper have reviewed ...
Types of Medical Literature. Medical literature is often classified based on how far removed the information is from the original source. Primary Literature/Source Primary sources are original materials. It is authored by researchers, contains original research data, and is usually published in a peer-reviewed journal.
What is a tertiary source? There are three types of research sources: Primary sources: These provide direct evidence about the topic of your research question (e.g., newspapers, diary entries, and photographs).; Secondary sources: These interpret or analyze information from primary sources (e.g., books and journal articles).; Tertiary sources: These are reference works that list other kinds of ...
provides comparisons for your own research findings. A literature review surveys books, scholarly articles, and any other sources relevant to a particular. issue, area of research, or theory, and ...
A literature review is an overview of the available research for a specific scholarly topic. Literature reviews summarize existing research to answer a review question, provide context for new research, or identify important gaps in the existing body of literature.. An incredible amount of academic literature is published each year; by some estimates nearly three million articles.
WBCP method presents a novel algorithmic framework for predicting drug combinations through the integration of heterogeneous data. Initially, seven drug similarity networks are constructed from diverse sources data, encompassing information such as chemical structural, target, and side effects (Fig. 1A). For each similarity network, similarities between query drug pairs and all known drug ...
4. Research on cannabis is stifled. Scientific research on the health effects of cannabis has advanced little in recent years, because there are huge barriers to studying the drug.