validity concerns in research using organic data

• DOI: 10.1177/0149206319862027
• Corpus ID: 199151950

Validity Concerns in Research Using Organic Data

• Heng Xu , Nan Zhang , Le Zhou
• Published in Journal of Management 10 July 2019
• Computer Science

Figures from this paper

30 Citations

Fields of gold: scraping web data for marketing insights, methods, developments, and technological innovations for population surveys, fields of gold: generating relevant and credible insights via web scraping and apis.

• Highly Influenced

Open government data portals: Predictors of site engagement among early users of Health Data NY

Bringing context inside process research with digital trace data, new guidelines for null hypothesis significance testing in hypothetico-deductive is research, education as a resource for socio-economic development of russia: perceptions of social network users, surprises, conflicting findings, or questionable research practices a methodology for evaluating cumulative empirical analyses and replication studies, the great wave: the increasing demand for digital competence within the public sector, a psychometric view of technology-based assessments, 64 references, three eras of survey research.

• Highly Influential

A primer on theory-driven web scraping: Automatic extraction of big data from the Internet for use in psychological research.

Estimating the reproducibility of psychological science, geolocated twitter panels to study the impact of events, social media: a contextual framework to guide research and practice., big data and the challenge of construct validity, mining brand perceptions from twitter social networks, the reusable holdout: preserving validity in adaptive data analysis, creating social contagion through viral product design: a randomized trial of peer influence in networks, preserving statistical validity in adaptive data analysis, related papers.

Showing 1 through 3 of 0 Related Papers

Validity Concerns in Research Using Organic Data

• Work and Organizations

Research output : Contribution to journal › Article › peer-review

With the advent of computing technologies, researchers across social science fields are using increasingly complex methods to collect, process, and analyze data in pursuit of scientific evidence. Given the complexity of research methods used, it is important to ensure that the research findings produced by a study are robust instead of being affected significantly by uncertainties associated with the design or implementation of the study. The field of metascience—the use of scientific methodology to study science itself—has examined various aspects of this robustness requirement for research that uses conventional designed studies (e.g., surveys, laboratory experiments) to collect data. Largely missing, however, are efforts to examine the robustness of empirical research using “organic data,” namely, data that are generated without any explicit research design elements and are continuously documented by digital devices (e.g., video captured by ubiquitous sensing devices; content and social interactions extracted from social networking sites, Twitter feeds, and click streams). Given the growing popularity of using organic data in management research, it is essential to understand issues concerning the usage and processing of organic data that may affect the robustness of research findings. This commentary first provides an overview of commonly present issues that threaten the validity of inferences drawn from empirical studies using organic data. This is followed by a discussion on some key considerations and suggestions for making organic data a robust and integral part of future research endeavors in management.

Original language	English (US)
Pages (from-to)	1257-1274
Number of pages	18
Journal
Volume	46
Issue number	7
DOIs
State	Published - Sep 1 2020

Bibliographical note

• open science (e.g., transparency in research practices)
• replication studies
• research design
• research methods

Publisher link

• 10.1177/0149206319862027

Other files and links

• Link to publication in Scopus
• Link to the citations in Scopus

Fingerprint

• Social Networking Site Computer Science 100%
• Design Research Computer Science 100%
• Process Data Computer Science 100%
• Empirical Research Computer Science 100%
• Computing Technology Computer Science 100%
• Social Science Field Computer Science 100%
• Laboratory Experiment Computer Science 100%
• Scientific Evidence Computer Science 100%

T1 - Validity Concerns in Research Using Organic Data

AU - Xu, Heng

AU - Zhang, Nan

AU - Zhou, Le

N1 - Publisher Copyright: © The Author(s) 2019.

PY - 2020/9/1

Y1 - 2020/9/1

N2 - With the advent of computing technologies, researchers across social science fields are using increasingly complex methods to collect, process, and analyze data in pursuit of scientific evidence. Given the complexity of research methods used, it is important to ensure that the research findings produced by a study are robust instead of being affected significantly by uncertainties associated with the design or implementation of the study. The field of metascience—the use of scientific methodology to study science itself—has examined various aspects of this robustness requirement for research that uses conventional designed studies (e.g., surveys, laboratory experiments) to collect data. Largely missing, however, are efforts to examine the robustness of empirical research using “organic data,” namely, data that are generated without any explicit research design elements and are continuously documented by digital devices (e.g., video captured by ubiquitous sensing devices; content and social interactions extracted from social networking sites, Twitter feeds, and click streams). Given the growing popularity of using organic data in management research, it is essential to understand issues concerning the usage and processing of organic data that may affect the robustness of research findings. This commentary first provides an overview of commonly present issues that threaten the validity of inferences drawn from empirical studies using organic data. This is followed by a discussion on some key considerations and suggestions for making organic data a robust and integral part of future research endeavors in management.

AB - With the advent of computing technologies, researchers across social science fields are using increasingly complex methods to collect, process, and analyze data in pursuit of scientific evidence. Given the complexity of research methods used, it is important to ensure that the research findings produced by a study are robust instead of being affected significantly by uncertainties associated with the design or implementation of the study. The field of metascience—the use of scientific methodology to study science itself—has examined various aspects of this robustness requirement for research that uses conventional designed studies (e.g., surveys, laboratory experiments) to collect data. Largely missing, however, are efforts to examine the robustness of empirical research using “organic data,” namely, data that are generated without any explicit research design elements and are continuously documented by digital devices (e.g., video captured by ubiquitous sensing devices; content and social interactions extracted from social networking sites, Twitter feeds, and click streams). Given the growing popularity of using organic data in management research, it is essential to understand issues concerning the usage and processing of organic data that may affect the robustness of research findings. This commentary first provides an overview of commonly present issues that threaten the validity of inferences drawn from empirical studies using organic data. This is followed by a discussion on some key considerations and suggestions for making organic data a robust and integral part of future research endeavors in management.

KW - open science (e.g., transparency in research practices)

KW - replication studies

KW - research design

KW - research methods

UR - http://www.scopus.com/inward/record.url?scp=85068888854&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068888854&partnerID=8YFLogxK

U2 - 10.1177/0149206319862027

DO - 10.1177/0149206319862027

M3 - Article

AN - SCOPUS:85068888854

SN - 0149-2063

JO - Journal of Management

JF - Journal of Management

Robust Analytics Lab ⟨⟨⟨⟨⟨⟨⟨⟨⟨

Interdisciplinary research on the societal implications of AI and data analytics

Validity Concerns in Using Organic Data

An article co-authored by Heng Xu , Nan Zhang , and Le (Betty) Zhou was recently accepted for publication at the Journal of Management. In the article, we provide an overview of the common issues that threaten the validity of inferences drawn in empirical studies from analyzing organic data – that is, data that were not generated according to an explicit research design but were instead captured by digital devices or platforms; examples include contents and social interactions extracted from social networking sites, Twitter feeds, click streams, etc. Specifically, we discussed two main types of validity threats in the usage of organic data. One is caused by the opaqueness of the data generation process (to the researcher/practitioner), and the other by the opaqueness of the automated information-extraction algorithms.

Here is the link to the article.

12 Nov 2019

• publications

Lecture: Validity Concerns in Research Using Organic Data

When: 1:30 pm, June 6, 2019

Where: Room A-3A, Building No. 25, Weijin Road Campus

Lecturer: Nan Zhang

About the Lecturer:Dr. Nan Zhang is a Professor of IT and Analytics at the American University’s Kogod School of Business. Dr. Zhang is a world-renowned expert on database anddata analytics, having published over 100 research papers and served as a program director at the U.S. National Science Foundation (NSF) for both fields. According to csrankings.org, He is ranked among top 15 computer scientists in premier Database publicationsfrom 2009 to 2019 in the U.S. Before joining Kogod, Dr. Zhang was a professor of Information/Computer Science at Penn State, George Washington, and UT Arlington. His work has received several awards, including the NSF CAREER award in 2008, Best Paper Awardsfrom IEEE ICC 2013 and NAS 2010, and Best Paper Nominations from IEEE ISI 2015 and HICSS 2018/2019.

About the Lecture: The field of meta-science -- the use of scientific methodology to study science itself -- has examined various aspects of this robustness requirement for researchthat uses conventional designed studies (e.g., surveys, laboratory experiments) to collect and analyze data. Largely missing, however, are efforts to examine the robustness of empirical research using "organic data", namely data that are generated withoutany explicit research design elements and are continuously documented by digital devices (e.g., content and social interactions extracted from social networking sites, Twitter feeds, and click streams). Given the growing popularity of using organic data inbusiness research, it is essential to understand issues concerning the usage and processing of organic data that may affect the robustness of research findings. This paper first provides an overview of commonly present issues that threaten the validity ofinferences drawn from empirical studies using organic data. This is followed by a discussion on some key considerations and suggestions for making organic data a robust and integral part of future research endeavors in management research.

Organizer: College of Management and Economics

All students and staff of Tianjin University are welcome.

"); } else{ if(str_appendix!=""&link_appendix!=""){ document.write(""); } } "); } else{ if(str_appendix!=""&link_appendix!=""){ document.write(""); } }