Discussion

It might be helpful in this conversation to distinguish a bit between processes associated with evaluating credibility of what is usually meant by "literature" (e.g. journal articles, book chapters, written word), and data sets (often numeric based, direct from research or compiled). Research data and written articles are related types of information in scientific research and not always completely separable, but they have tended to be distinguished in their paths to initial publication, how they are made discoverable by indexing services and available in libraries, how they are evaluated before and after publication and how they are used. As Kristin suggests, these lines are blurring somewhat with new modes of handling data and publishing in the digital environment. However, it is still useful for everyday research and also helpful when evaluating sources to be aware of these distinctions. For example, this Module 1 is primarily focusing on searching, evaluating and organizing written word literature; how to manage data sets are further considered in Module 3 and in other lectures throughout the course. I posted separately on evaluating the secondary literature databases discussed in this Module. The "data" in some of these sources primarily consist of citations and abstracts of articles, such as the Web of Science; other databases reprint numeric and process data from articles and other sources, such as Reaxys. The older types have traditionally been indexed by human professionals in part as another layer of review, such as Chemical Abstracts. More recent databases may be extracted by algorithm, such as PubChem, which reprints data from other compilations "as is" with a full disclosure of the source for the user to consider directly in their own evaluation process. As Bob mentions, databases may not pick up on the types of human sourced errors highlighted in "Retraction Watch", a blog that focuses primarily on data published within articles. Another vote for researchers to review data, and the rationale published with it either in a more traditional analysis or a "data paper", before using in their own work. A few other approaches to data evaluation related to chemistry in particular that are worth mentioning briefly as they will not be otherwise covered are crystallographic data and materials property data. The crystallographic data research community has formulated over time a process for peer review of these data through a robust standard file format that enables automated validation checks as well as human expert review. The data are concurrently published in a sustainable repository that supports both open retrieval of individual data sets through the original publication and subscription based analysis software (Cambridge Crystallographic Database, <a href="http://www.ccdc.cam.ac.uk/pages/Home.aspx">http://www.ccdc.cam.ac.uk/pages/Home.aspx</a>). The National Institute of Standards and Technology employs rigorous data evaluation strategies for various types of materials properties reported from various sources based on binary assessment of acceptable vs. non-acceptable for re-use in industry applications. An early version of a decision tree for ceramics materials is the NIST Interactive Data Evaluation Assessment Tool that identifies several levels of evaluated data, including certified, validated, qualified, commercial, typical, research and unevaluated (IDELA, <a href="http://www.ceramics.nist.gov/IDELA/IDELA.htm">http://www.ceramics.nist.gov/IDELA/IDELA.htm</a>). The more recent NIST ThermoData Engine employs similar types to dynamically evaluate data based on large compilations of experimental data (<a href="http://trc.nist.gov/tde.html">http://trc.nist.gov/tde.html</a>), the latest effort in a long history to systematically collect and evaluate data for re-use.