Articles
spacer

D-Lib Magazine
October 2000

Volume 6 Number 10

ISSN 1082-9873

Measuring the Impact of an Electronic Journal Collection on Library Costs

A Framework and Preliminary Observations

 

Carol Hansen Montgomery, Ph.D.
Dean of Libraries
Drexel University
[email protected]
<http://www.library.drexel.edu/facts/staff/dean.html>

Red Line

spacer

I. Introduction

Much has been written about the economic impact of electronic publishing on publishers. There also has been considerable discussion of the cost of subscribing to electronic publications. This paper addresses another important organizational impact triggered by the migration to electronic journals that has heretofore received little attention in the literature: the changes in the library's operational costs associated with shifts in staffing, resources, materials, space and equipment.

In 1998 the W.W. Hagerty Library of Drexel University made migration to an electronic journal collection as quickly as possible a key component of its strategic plan. If a journal is available electronically, only the electronic version is purchased whenever possible1. The sole exceptions are (1) when the electronic journal lacks an important feature of the print version (e.g., equivalent visuals) and (2) when the journal is part of the browsing collection (e.g., Scientific American and Newsweek). With the year 2000 renewals, Drexel's journal collection consisted of 800 print only subscriptions and 5000 electronic journals; in 2001 the library will subscribe to about 300 print only journals and over 6000 electronic journals. A dramatic change in staff workload is the most immediate impact on library operations, but space, equipment, and even supply needs are affected. Some of the aspects of this transformation were obvious and predictable; others were not. This paper describes the changes experienced so far in the Drexel Library.

A common assumption is that converting library journals to digital format will ultimately improve library service and lower cost, but this is yet to be proven. Understanding the total costs associated with the library model for delivering digital information has now become a requirement for library survival since in the digital world, as opposed to print, the library has many viable competitors. Our goal is to develop a framework for assessing the shifts in personnel and costs that can be used for planning and budgeting at Drexel and provide guidance to other academic libraries who are not yet so far down this path.

II. Background

It is so common that it is almost universal for journal cost analyses to use subscription costs and ignore the operational costs associated with a journal collection. For example, White and Crawford [1] undertook a cost-benefit analysis to determine whether acquiring Business Periodicals Online (BPO), a full-text database, was more cost-effective for supplying articles than obtaining articles in the database through interlibrary loan (ILL). They found that the out-of-pocket costs (ILL transaction costs versus the BPO subscription costs) were similar, but the level of service was much greater with BPO.

Hawbaker and Wagner [2] also compute only subscription costs when comparing the costs of print subscriptions to online access of full-text. They conclude that, for a full-text business database, the University of the Pacific's library can offer more than twice as many journals for a 15 percent increase in expenditures.

The Electronic Libraries Programme (eLib) [3], funded by the Joint Information Systems Committee (JISC) in the United Kingdom, consists of a series of major library projects investigating issues of digital library implementation and integration. The emphasis is on "working libraries." The guidelines for evaluation of the eLib projects call for "modelling of functional, cost, organisational and technical variables" as one of the desired components. [4] Pricing models for electronic journal subscriptions, licensing agreements, and infrastructure requirements to provide access are themes that are explored in these projects. Each project tends to be fairly focused in terms of the range of digital content and services offered or the topic addressed. Many deal with organizational and management models that can serve as the basis for scaling up initiatives.

JSTOR [5], which builds journal backfiles, does address building-related costs. One of the JSTOR objectives is "To reduce long-term capital and operating costs of libraries associated with the storage and care of journal collections." By guaranteeing online availability of backfiles, JSTOR not only makes these files more accessible, but also allows libraries to discard old journal runs without decreasing service to their users.

In an analysis completed several years ago Tenopir and King [8] report the average cost of acquiring and maintaining a print journal collection to be $71 per title in academic libraries and $81 for special libraries. These are 1998 figures adjusted for inflation.

Odlyzko [6] also focuses on non-subscription costs. In a 1999 article he points out additional factors to consider in evaluating the impact of journal growth on libraries:

Journal subscription costs are only one part of the scholarly information system....internal operating costs of research libraries are at least twice as high as their acquisition budgets. Thus for every article that brings in $4,000 in revenues to publishers, libraries in aggregate spend at least $8,000 on ordering, cataloging, shelving, and checking out material, as well as on reference help. The scholarly journal crisis is really a library cost crisis. If publishers suddenly started to give away their print material for free, the growth of the literature would in a few years bring us back to a crisis situation.

Odlyzko�s analysis shows that the library's non-subscription (i.e., operational) costs are on average double the subscription costs. His figures are derived from the Association of Research Libraries (ARL) statistics [7]. His is a macro level measurement that does not take into account, for example, the different processing costs for books and journals or library costs unrelated to the collections which might cause the non-subscription figure to be over-estimated. On the other hand, ARL statistics do not report the considerable costs associated with constructing and maintaining library buildings, a factor which if added to Odlyzko�s number would lead to a higher estimate of non-subscription costs. Even if off by a factor of two, Odlyzko's estimate is astounding to consider, and points out the importance of looking at how these operational costs shift in the transition to an electronic model.

III. Development of Drexel's Electronic Journal Collection

In the spring of 1998 only one full-text journal collection was accessible via the Drexel Library�s web site and database access was limited to text-based systems. That summer the web site was completely re-designed and by the fall more than 20 databases and several collections of full-text journals were available. The total number of print journals was 1,500 titles at the time. For 1999 and 2000 the number of print only journal subscriptions was reduced to 1,100 and about 800 respectively. Some of the reductions were made because we had subscribed to an electronic counterpart; the other journals were not renewed primarily on the basis of low use. During the fall of 1998 through 1999, and into 2000, electronic subscriptions were sought out aggressively and added as they became available, bringing the fall 2000 total to over 6,000 unique electronic titles. Print only renewals for 2001 will be about 300.

The selection/ordering/acquiring process is far more complex for electronic journals than for print journals. The methods for purchase often include buying packages of titles or services, many with value-added features. Reviewing and negotiating proper terms for e-journal licenses is a major aspect of the complexity. Additionally, new variables must be considered (e.g., graphics, linking options, comparability with the corresponding print publication, web interface functionality, and other value-added features).

When the publisher's policy is to require purchase of the print journal in order to obtain access to the electronic journal, we attempt to negotiate a discount for the e-journal only. This has met with limited success so far, but does have the advantage of educating publishers about our needs. Because of the added cost of receiving, processing, binding and storing the print issues, we do not retain print journals even if we have paid for them. Some of these journals are never shipped by the publishers; others are retained by our serials vendor for their back issue file; and still others arrive and are given to our back issue jobber.

Drexel's approach to back files of print journals will seem cavalier, if not totally irresponsible, to those concerned with the archival role of libraries. Our position is that archival storage in most subject areas is not part of the mission of the Drexel Library. On a national -- even international -- basis archiving of old, little-used materials would be much more cost effective if done centrally or in a few places for redundancy. This is true of both electronic and print formats. We are willing to make the leap of faith that this will happen, and are ready to pay the cost of access to the archived materials when they are needed. There are numerous well-qualified national and international organizations addressing this issue, including the Research Library Group and OCLC.

IV. Impact on Library Staffing and Other Costs

Here we discuss changes in each area of the library's operations with particular attention to the changes in staffing patterns and shifts in costs. Table 1 summarizes these operational effects.

 

Table 1. The Transition from Print to Electronic Journals: Changes in Staffing and Other Costs

Table showing operational effects on subscription costs.

A PDF version of Table 1 is available here for those who wish to print the table.

spacer

Infrastructure

Systems

The impact of electronic journals on the infrastructure is outlined in Table 1. While space is the most important requirement for the print format, networks, computer hardware/software and systems staff are required to provide access to electronic resources. These items are rapidly becoming key components of a well-functioning operation in all academic institutions, as they are essential for so many other reasons. None of the library systems are used for electronic journals exclusively since we provide access to many other applications.

The Webmaster easily spent 30 percent of his time on electronic journal access during the start-up period. He maintains the entire Library web site, which initially included over 200 static HTML pages listing e-journals by title and by subject. When it became clear that maintaining this many continually changing static HTML pages was a major burden, the Webmaster developed an e-journal maintenance database using MySQL and PERL scripts to manage the lists and deliver them to the web dynamically.

Space Utilization

The chief impact of print journals on infrastructure is managing the physical space for growth of the collection over time. The transition to electronic journals essentially eliminates space concerns -- no more trimming the collection, converting to microfilm, or moving it to a remote location to make space for new volumes. Eventually, because of retrospective conversion efforts like JSTOR, we will be able to reclaim journal storage space for other purposes. The cost savings, both on a capital and annual basis, are considerable. Estimating $100 per square foot [9], the minimum cost for library buildings in large urban centers, the 20,000 square foot space currently occupied by the Drexel journal stacks would cost $2 million to construct. Estimating annual maintenance costs at $12 per square foot, the cost of maintaining the space occupied by the library�s journal collection is approximately $240,000 per year.

Subscription Costs

Budget allocations reflect the decision to shift from print to electronic subscriptions. Purchase decisions are based on two processes. First, we undertook a major initiative to analyze our current print and electronic holdings prior to renewal of subscriptions for the coming year. Secondly, throughout the year we invest significant staff resources to keep current with all e-journal offerings from vendors, publishers and consortia within the scope of our collection and initiate negotiations for pricing and packages tailored to our needs. In particular, we seek out electronic equivalents of current print holdings and replace the print with the electronic version of the title unless the title meets the exception criteria. Eventually, we expect to have a browsing collection of fewer than 100 titles.

As a result of these efforts Drexel's total journal subscription costs will be approximately $636,000 for 2001. See Table 2 for the breakdown. Aggregator subscription costs are difficult to calculate since they are part database, part electronic journals. With a "best guess" allocation of the cost of these services, we are spending or expect to spend a total of $600,000 for electronic journals.

Table 2. Subscription Costs FY2001

Category

# of Titles

Amount

Print only subscriptions 300 $36,000
Electronic subscriptions 2800 $550,000
Aggregator/databases*
with full-text contnet
3500 $45,000
Total E-Journals 6300
Unique Titles
$595,000
*These are services such as ProQuest and Wilson Select which are part database, part journal full-text. Half the cost of these services has been allocted to electronic subscriptions. The count is of unique titles.

On a raw per title basis the e-journal subscription dollar has superior purchasing power when the aggregators' titles are included. Our print only journal subscriptions now cost an average of $120 per title while e-journals are $95 per title. This difference is far greater when one considers that nearly all the electronic journals come, even when a subscription is first entered, with several years of backfiles. And, the electronic subscriptions include many titles that cost several thousand dollars in print, while the 300 print journals consist mainly of humanities and social science publications along with some popular titles, all of which are low cost historically. The increased value of electronic journals is even more evident when coupled with use statistics, since our figures show that electronic journals are used more heavily than print counterparts. See Montgomery and Sparks [10].

Administration/Management

Academic library directors have always paid considerable attention to electronic journals. Journal costs usually take most of the materials budget in science and technology libraries. Faculty often have strong feelings about particular titles which they do not hesitate to make known. Traditionally, the decision to subscribe to a new journal has required careful consideration because of the implication that the subscription is a long term commitment. And, for the last two decades as prices escalated so dramatically, directors became increasingly involved in both advocating for additional funding to pay for journals and overseeing the time-consuming annual journal evaluation processes and cost-cutting measures. Electronic journals raise new issues that require the director's involvement to an even greater extent. Activities that are new or escalated for a director who makes a major commitment to electronic journals include:

  • communicating and obtaining institutional funding and support,
  • joining consortia and other "buying clubs,"
  • negotiating and reviewing contracts,
  • determining and revising strategies for e-resource acquisition,
  • building a library staff with the appropriate skills, and
  • managing the change in budget allocation.

Drexel created a new position, Electronic Resources Manager (ERM), to provide a focal point for integrated development of all electronic resources. This position crosses the traditional departmental functions of management, systems, technical services and reference. The person in this position shares the responsibility of keeping up-to-date on the availability of new electronic resources with the Information Services (IS) librarians who do collection development. The ERM initiates contacts with vendors to negotiate favorable pricing and packaging and arranges trials for each new service considered for purchase. She also reviews licenses and contracts and spends some effort negotiating appropriate amendments and corrections to these documents. For example, one of our goals is to always provide remote access to content we make available to our users; some contracts do not allow this. The ERM also interacts with consortia for purchase of electronic resources and evaluates the cost/benefits of going with a particular group offer. Once the purchase decision is made, IP information is communicated to vendors and content changes are made on our web site. She also collaborates closely with the Webmaster in designing and populating our e-journal database. Finally, gathering and organizing use statistics for electronic resources is a major aspect of her responsibilities. A recent ARL SPEC Kit [11] describes these activities and the various ways large academic research libraries have structured themselves to deal with them.

It is always more difficult and time-consuming to manage change than to maintain the status quo. The amount of time spent managing and overseeing this transition is substantial, when one includes major ongoing efforts to restructure workflow, and reorganize staff to respond to the migration to electronic journals.

Technical Services

In the Technical Services Department, the transition to e-journals has had a direct impact on the day-to-day work of each staff member. Changes in workflow and procedures are dramatic, with very large shifts in costs. It is clear that the significant reduction in print titles has directly decreased workload for tasks related to the print format. Less time is needed to check in print issues, claim non-arrivals, replace missing pages, and prepare and receive bindery shipments. Also, direct costs for cataloging new print titles and maintaining existing MARC records (OCLC charges) have been reduced. Bindery fees are also reduced accordingly.

Offsetting the decrease in activity levels and costs related to the print format is a large increased workload for both the serials acquisitions and cataloging functions related to providing access to electronic journals. Updating the e-journal maintenance database that now creates our e-journal lists is a major new task. The e-journal collection is much more volatile than a print collection: links break, coverage changes, and sometimes the electronic journals themselves are available through a new distributor. An advantage of electronic distribution that creates extra work is that we are not bound to calendar-year-only subscriptions, so journals are added continuously and sometimes discontinue during the year.

Another activity that has greatly affected Technical Services is an expanded review process for journal renewals that includes the IS librarians (each represents the various colleges in the University). During the past two years we have evaluated every journal title -- print and electronic -- before it is renewed. The coordination and tracking of the renewal decisions has increased significantly.

Not only has the format of materials shifted, but the volume of materials has increased more than three-fold. We are now managing over 6,000 journal titles as opposed to 1,500 titles two years ago. Unfortunately, we are not always able to switch existing staff to e-journal tasks. In the process of "re-engineering" the entire department, we upgraded two positions, added one temporary position, and replaced one position. We now require detail-oriented support staff who have advanced computer skills and who can adjust to continuous changes in procedures and methods as our environment evolves.

Circulation/Access/Stack Maintenance

Staffing

Obviously, shelving decreases when journals are no longer physically stored in the library. Bound journal re-shelving has been reduced by 40 percent and re-shelving of current journal issues is down over 20 percent over the past two years. At Drexel collection of print journal re-shelving statistics is only partly automated. Shelvers track use by title as they shelve bound volumes and current issues. Fewer journals to shelve also translates to less time collecting print re-shelving statistics.

Electronic Use Statistics

In theory, it is easier to collect use statistics and richer, more accurate demographic and search information for electronic journal usage because data collection can be automated and expanded. In reality, at this time it is very difficult and labor intensive to obtain useful and comparable title-by-title use data for electronic journals and compile them in a way that is helpful for making management decisions. Activity measures and, in particular, comparable activity measures across journal vendor services are frustratingly difficult to come by. Mercer [12] describes the problems encountered in trying to collect and analyze the vendor information so as to use it for service evaluation and decision-making. Among the statistics reported are session length, number of searches, journal title hits, page hits, types of pages hit, top XX titles accessed each month, "turnaways," form and type of articles downloaded, and number of unique IP addresses using a service or journal title.

Since the data for print volumes are not strictly comparable, they must be interpreted carefully. Our print statistics represent volumes or issues re-shelved rather than actual articles copied or read, while the e-journal statistics below represent articles accessed which may or may not have been read. The print use data is somewhat under-reported because, even when asked not to, users re-shelve journals after they look at them. Even so, from preliminary data we can say confidently that our users are accessing the electronic journals in numbers far exceeding our print collection.

Photocopying

Since our statistics have decreased so dramatically for print journal usage, it is only logical that photocopier use would also decrease since this is one of the primary uses of our library photocopiers. Photocopy use has decreased about 20 percent since electronic journals were introduced.

Reserve

Circulation of reserve materials, which had been steady at about 30,000 items per year, dropped by 50 percent during the 1999/2000 academic year. What portion of the e-resources used are electronic journals and what are other e-resources is an open question. We do expect this trend to continue for the print reserve format, particularly when our electronic reserve module is fully functional later this year. It appears that not only are students using fewer reserve materials, but our faculty also are placing fewer items on reserve. With respect to staffing impact, we have reorganized some of the work assignments in this department due to the reduced workload, and upgraded the reserve room supervisor position in anticipation of the electronic reserves activity.

Document Delivery/Interlibrary Loan (DD/ILL)

Our expectation with the implementation of electronic journals was that we would see a significant decrease in user requests for journal articles via our DD/ILL services. In fact, "borrowing" photocopies of journal articles from other libraries increased by 16 percent in FY1999-2000. The library's document delivery service, which provides copies of articles from the Drexel Library collections free of charge to faculty and distant learners, delivered 1122 articles from the electronic journal collection in this same time period. The majority of these articles are for faculty who presumably are not aware of the ready accessibility of e-journals, or who either cannot or choose not to retrieve the articles themselves. At the moment, it is not possible to measure the net impact of the electronic journals on the DD/ILL department volume because too many other factors are influencing use of the service. Research activity has grown dramatically at Drexel in the past two years, and the provision of over 100 web-based databases likely has increased demand for articles. Our prediction is that ultimately we will see a decrease in net requests for this service as our users become increasingly self-sufficient and as electronic content continues to expand.

Reference Information Services (IS)

Reference services are nearly always affected by any significant change in library resources. At Drexel the Information Services/Reference staff are responsible for materials selection in addition to the usual functions of answering questions, teaching classes, and performing public relations functions such as promoting the availability of services. So they are involved in several stages of the "life cycle" of electronic journals at Drexel. They share responsibility for identifying e-journal candidates for purchase, evaluating potential purchases, helping students and faculty use the e-journals effectively, incorporating information about them in their classes, and helping publicize them to their constituencies.

Reference Service

Some interesting trends are occurring at the reference desk. Questions decreased in 1999/2000 by about eight percent although more of the transactions that do occur turn into "teaching" opportunities for those users who are less self-sufficient. Staff observe that, in particular, students using the web-enabled computers in the "hub" near the reference desk, are increasingly self-sufficient.

Instructional Program

Offsetting the decrease in reference questions is the amount of time IS staff are spending on instruction and outreach activities to make faculty and students aware of the library's resources and services. Workshops and teaching sessions have increased. Vendor presentations are more frequent. IS librarians are engaging in greatly expanded public relations by personal visits and presentations, email updates to departments, exhibits and other activities. Another effort that has also expanded is the preparation of both online and printed documentation to help users understand how to use electronic journals.

Staffing

The electronic journal option and new processes have most certainly increased the workload for selecting journals. We do expect that over time this increase will level off as the collections and offerings stabilize in the electronic environment. No new staff positions have been added in the IS department but there has been significant turnover and, again, we carefully screen new hires for expanded computer skills and experience using, selecting and promoting electronic resources. A lot of the increased journal evaluation work comes in the summer, a time when many of the other activities of the department are reduced. So far the staff have been able to handle the additional work at current staff levels.

V. Discussion

Drexel is probably farther along in the transition to an all electronic journal collection than most, if not all, academic libraries in the United States. A late 1997/1998 survey of ARL and non-ARL academic libraries found that just 29 and 33.5 percent, respectively, had cancelled print journals in favor of electronic access in the previous 12 months [13]. Fifty-one percent of the ARL libraries and 40 percent of the non-ARL libraries had not cancelled print subscriptions in favor of electronic and declared that they will not in the future. Their reluctance was attributed to the enormous change required in academia to relinquish print.

This has not been a problem for Drexel. Faculty and students have embraced the transition almost universally. Organizational readiness, important in any successful organizational change, has been critical to the ability of the Drexel Library to move so rapidly to a new model. The most important factors have been: (1) a highly computer literate faculty and student body; (2) programmatic emphases in science, technology and business, areas where publishers have been quickest to provide e-journals; (3) the existence of a high speed ubiquitous network; (4) general dissatisfaction with the print journal collection; (5) a supportive administration that provided a significant increase in funding; and (6) a strong and growing distance education program.

This description of the Drexel experience should be useful to others because our transition is indicative of what most academic libraries will eventually experience. There are accredited academic institutions that are functioning with completely digital libraries, i.e., they never had a print library. Examples are Jones International University [15] and the University of Phoenix. [16] Other libraries have created large electronic journal collections - e.g., the University of California system [17] and most, if not all, large research libraries - but they are maintaining large print collections concurrently. The approach Drexel is implementing - substituting electronic for print - will be the typical scenario in most academic libraries because it will be necessary to make electronic collections affordable.

Preliminary cost comparisons for processing print versus electronic journals indicate that the electronic collection is substantially more expensive to maintain. We estimated staff costs by allocating percentages of individual staff members' time to the various tasks and projects described in this paper using the functional cost analysis approach of Abels, Kantor and Saracevic [14]. The amount of time spent per task was determined by interviewing staff and supervisors to analyze the impact for each area and by reviewing library statistics and other records. Then, we computed the annual cost in salaries using indiviuals� rate of pay. The result indicated that there are substantial costs in maintaining an electronic journal collection that more than offset the savings from eliminating the clerical chores associated with maintaining a print journal collection. While fewer staff are needed the new staff are more skilled, and therefore more highly compensated. Likely, as the electronic journal publishing industry and related service industries mature, the change process will become easier, and thereby less costly, for libraries.

Drexel's per title subscription costs are lower for electronic journals. While this is a function of our selection process and the particular "deals" we have been able to obtain, we suspect that the majority of academic libraries will have the same experience, particularly if they purchase a large number of titles through aggregator collections. Since use is much higher for e-journals the cost benefit is even greater. We plan further analysis to refine our calculations of operational costs, and subscriptions costs that include factors such as backfiles and use data in order to come up with good estimates of "real" per title costs that include all factors of operational costs, and subscriptions costs that include all factors 2

There are many areas where improvements made by publishers and vendors could decrease the library workload.

Of particular value would be:

  • Better information about the existence of electronic journals, and their characteristics.
  • Standards for presentation of use data by vendors.
  • Easier methods of providing access to electronic journals, either through cataloging or in list form.
  • An assured solution to archiving.

As the entire electronic publishing "system" matures, we anticipate that these improvements will come.

VI. Notes

1 Some publishers continue to insist that print journals must be purchased to gain access to the electronic equivalent, i.e., bundling. We have developed strategies to discontinue receiving and storing the print copies.

2 The Drexel Library recently received a one-year, $128,000 grant from the Institute of Museum and Library Services to study these costs and develop models for measurement.

VII. References

[1] White, G. W. and Crawford, G. A. "Cost-Benefit Analysis of Electronic Information: A Case Study." College and Research Libraries, 59:6 (November 1998): 503-10.

[2] Hawbaker, A. C. and Wagner, C. K. "Periodical Ownership Versus Full-text Online Access: A Cost-Benefit Analysis." Journal of Academic Librarianship, 22:2 (March 1996): 105-9.

[3] eLib: The Electronic Libraries Programme. 10 October 2000 <http://www.ukoln.ac.uk/services/elib/>.

[4] Evaluation of the Electronic Libraries Programme: Guidelines for eLib Project Evaluation. 10 October 2000 <http://www.ukoln.ac.uk/services/elib/papers/tavistock/evaluation-guide/intro.html>

[5] JSTOR: The Need. 10 October 2000 <http://www.jstor.org/about/need.html>.

[6] Odlyzko, A. M. "Competition and Cooperation: Libraries and Publishers in the Transition to Competition Electronic Scholarly Journals." Journal of Electronic Publishing, 4(4) (June 1999): 163-85.

[7] (ARL) Association of Research Libraries, Statistics and Measurement Program. 10 October 2000 <http://www.arl.org/stats/index.html>.

[8] Tenopir, C. and King, D. W. "Towards Electronic Journals: Realities for Scientists, Librarians, and Publishers." Washington, DC : Special Libraries Association, 2000, p. 216.

[9] Fox, B. L. "Library Buildings 1999: Structural Ergonomics." Library Journal, 124:20 (December 1999) 57-67.

[10] Montgomery, C and Sparks, J. "Framework for Assessing the Impact of an Electronic Journal Collection on Library Costs and Staffing Patterns." March 2000. <http://www.si.umich.edu/PEAK-2000/montgomery.pdf> (6 October 2000).

[11] Blieler, R. and Plum, T. Networked Information Resources. SPEC Kit 253. Washington, DC. Association of Research Libraries. December, 1999.

[12] Mercer, L. S. "Measuring the Use and Value of Electronic Journals and Books. Issues in Science and Technology Librarianship, Winter 2000. 10 October 2000 <http://www.library.ucsb.edu/istl/00-winter/article1.html>.

[13] Shemberg, M. and Grossman, C. "Electronic Journals in Academic Libraries." A comparison of ARL and Non-ARL Libraries. Library Hi Tech, 17 (November 1, 1999): 26-45.

[14] Abels, E. G., Kantor, P. B., and Saracevic, T. "Studying the Costs and Value of Library and Information Services: Applying Functional Cost Analysis to the Library In Transition." Journal of the American Society for Information Science, 47 (March 1996):217-27.

[15] Jones International University. 10 October 2000 <http://www.jonesinternational.edu>.

[16] University of Phoenix Online Collections. 12 October 2000 <http://www.apollolibrary.com/>.

[17] California Digital Library. 10 October 2000 <http://www.cdlib.org>.

Copyright© 2000 Carol Hansen Montgomery
spacer
spacer

Top | Contents
Search | Author Index | Title Index | Monthly Issues
Previous Article | Next Article
Home | E-mail the Editor

spacer
spacer

D-Lib Magazine Access Terms and Conditions

DOI: 10.1045/october2000-montgomery