JVP: Methodology
Journal Valuation Project and Database.
We view the database as a work in progress that will evolve to incorporate new data and new methodology as it becomes available. In general, however, we found it difficult to improve upon the basic methodology employed by Prof. Barschall. Most importantly, like Barschall, we rely on an analysis of text density and line counts to provide an indicator of the entire content delivered by a journal in a year. More specifically, we calculate the maximum amount of text that would appear in each journal, given the format and fonts used, if all pages were completely filled with text. Such an approach is clearly more valid in the case of text-rich journals than in those where pictures, diagrams or formulas may dominate, but thus far we have found no practical way to take these differences into account. The text analysis is accomplished as follows:
- We locate three pages from three separate issues of the journal in the target year where the number of lines on the page can be easily counted. Ideally, we find pages with at least one unbroken column of text, but in some cases the number of lines must be estimated by measurement of the largest text blocks we can find on a page.
- From each of the three pages we select a five-line block of text in which the number of characters and spaces are counted. If a five line block of text cannot be found, five separate text-only lines will be used. Recognizing the rapidly expanding role of electronic journals--and taking a cue from a 1998 Cornell University study-- we use the electronic versions of the journals and Microsoft Word to do the actual counting. In cases where no electronic version exists (or exists but cannot be manipulated in this fashion), we photocopy the pages and count the characters manually. For each journal, the three pages and a worksheet with other data are retained for a record.
- The total number of characters for the year (including spaces between words) is then determined by a formula that multiplies the number of characters in an average line by the number of columns, then by the maximum number of lines found on a page, and then by the number of pages published per year. Only numbered pages of scholarly content are counted. Index pages and other extraneous matter are excluded from the page count where possible. To calculate cost per thousand characters we used the institutional subscription price published in an early print issue. If this could not be found we used the price from the EBSCO LibrarianŐs Handbook. If these sources failed us, we used the price paid by the UW-Madison.
- To introduce an evaluative dimension, we have also continued Prof. Barschall's reliance upon ISI's Impact Factors: cost per thousand characters divided by the Impact Factor. Impact Factors suggest the relative importance of a journal as reflected by how many times its articles are cited within a recent two-year period. Much has been written about precautions one must take in using Impact Factors to evaluate journals, but the consensus is that, if used carefully, they provided the best indicator available to us now. Ideally, this measure of use could also be compared to local use data.
For more about the work of Prof. Heinz Barschall, see http://barschall.stanford.edu/.