Home
Todd Suomela's Library tagged → View Popular, Search in Google
1. pass/fail first semester
2. pre-accepted article
3. replication audit
4. a frequent flier system for journals
-
One idea is to develop a points system for each journal analogous to frequent flier miles. Each review would earn a scholar a certain number of points with bonuses awarded by editors for especially timely or high quality reviews. The author could then cash in those points when they submit to that journal in order to request a rapid review of their own manuscript. The journal would in turn offer those points to reviewers who review the manuscript quickly, helping to speed it through the process.
"The Roy Rosenzweig Center for History and New Media at George Mason University created PressForward to explore and produce the best means for collecting, screening, and drawing attention to the vast expanse of scholarship that is currently decentralized across the web or does not fit into traditional genres such as the journal article or the monograph."
"The decision highlights a tension between the need for rigorous peer review of scientific research and the desire to provide as much data as possible to the scientific community.
"More data, in and of itself, is always a good thing -- if there aren't adverse effects," said Maunsell, who is also a neuroscientist at Harvard University. But peer review was becoming less effective because many reviewers failed to evaluate the supplemental materials, which the journal wasn't even required to provide, he explained. "We were taking a hit on peer review for something that wasn't formally our responsibility." "
The core service of Faculty of 1000 (F1000) identifies and evaluates the most important articles in biology and medical research publications. The selection process comprises a peer-nominated global 'Faculty' of the world's leading scientists and clinicians who rate the best of the articles they read and explain their importance.Launched in 2002, F1000 was conceived as a collaboration of 1000 international Faculty Members. Although the name stuck, the remit of our service continues to grow and the Faculty now numbers more than 10,000 experts whose evaluations form a fully searchable resource identifying the best research available. Faculty Members and their evaluations are organized into over 40 Faculties (subjects), which are further subdivided into over 300 Sections.
"This paper presents the first meta-analysis for the inter-rater reliability (IRR) of journal peer reviews. IRR is defined as the extent to which two or more independent reviews of the same scientific document agree."
"Which academic articles get published in the more prestigious journals is a pretty random process."
-
The fundamental problem is that the “we need a journal” approach is stuck in the printed page paradigm. To get negative results published we need to reduce the barriers to publication much lower than they currently are, while at the same time applying either a pre- or post-publication filter.
-
But the key is a low barriers for authors and a robust business model that can operate even if the service is perceived as being low impact. Without these you are creating a lot of work for yourself, and probably a lot of grief.
The 'Black Box' of Peer Review
-
The result is How Professors Think: Inside the Curious World of Academic Judgment (Harvard University Press), which aims to expose what goes on behind the closed doors where funds are allocated and careers can be made. For those who have always wondered why they missed out on that grant or fellowship, the book may or may not provide comfort. Lamont describes processes in which most peer reviewers take their responsibilities seriously, and devote considerable time and attention to getting it right.
She also finds plenty of flaws -- professors whose judgment on proposals is clouded by their own personal interests, deal making among panelists to make sure decisions are made in time for panelists to catch their planes, and an uneven and somewhat unpredictable efforts by panelists to reward personal drive and determination over qualities that a grant program says are the actual criteria.
The Association for Psychological Science latest issue of Perspectives on Psychological Science has a few interesting articles about the research and publishing side of psychology.
Anonymous peer review has served as the bedrock of research dissemination in scientific psychology for decades and has only sporadically been questioned. However, other disciplines, such as biomedicine and physics, have found the traditional peer-review system to be wanting and have begun to test and try alternative practices. In this article, we survey criticisms of the traditional peer-review system and describe several alternatives in the interests of facilitating discussion and debate. We also consider why the natural sciences tend to employ fewer reviewers and have lower rejection rates than do the social sciences. Our two recommendations are that a serious discussion of problems and alternatives to peer review should be started at all levels of psychology and that a science of research communication should be a priority, with psychologists as part of its advance guard because of their relevant substantive and methodological knowledge.
Selected Tags
Related Tags
Top Contributors
Groups interested in peer-rev...
-
Open Source Journals
Items: 2 | Visits: 9
Created by: Nathan Brouwer
-
Scholarly Communication
Articles and sites on aspect...
Items: 61 | Visits: 2
Created by: Kay Cunningham
Diigo is about better ways to research, share and collaborate on information. Learn more »
Join Diigo