Go to

 

 
Go to Balanced Scorecard home

2001-02 Balanced Scorecard Metrics

The Balanced Scorecard is a management technique designed to provide a view of an organization from four perspectives: user, internal processes, financial, and future/learning potential. This document explains how the University of Virginia Library is defining each of these four areas by providing the details of the metrics we are using in determining our success in each perspective.

Each of the metrics described below contains two targets. Target1 indicates complete success in achieving the metric; Target2 indicates partial success. Management Information Services will compile and report the results of all metrics on a regular basis.


USER PERSPECTIVE: How well is the library meeting the needs of our users?

  1. PROVIDE EXCELLENT SERVICE TO USERS OF THE UNIVERSITY OF VIRGINIA LIBRARY

    METRIC I. OVERALL RATING IN STUDENT AND FACULTY SURVEYS.
    Target1: We should attain a score of at least 4 (out of 5) from each of the major constituencies (undergraduates, graduates, humanities faculty, social science faculty, science faculty).
    Target2: Score of at least 3.9 from each of the major constituencies.
    Method: The University Library has conducted extensive surveys of our clientele in three of the past four years. A final question in each survey asks the respondent to "rate your overall satisfaction..." with the Library using a 1 to 5 scale. This metric will consider the two latest surveys on record: currently the 2000 faculty survey and the 2001 student survey.
    Separate Tallies: The ratings for faculty and students will be kept and reported separately.

    METRIC II. CUSTOMER SERVICE RATING IN STUDENT AND FACULTY SURVEYS.
    Target1: Customer services should receive a score of at least 4.25 (out of 5) from each of the major constituencies: faculty, graduate students, and undergraduates.
    Target2: Score of at least 4.0 from each of the major constituencies.
    Method: As part of its series of surveys, the University Library now asks respondents to rate three aspects of customer service: speed, courtesy, accuracy. The question refers very generally to customer service and thus could be applied to quality of service received at any service point: reference, circulation, e-centers, email, phone, correspondence, etc. Scoring is on a scale of one to five, five being "most satisfactory." The three scores will be combined into one composite number.

    METRIC III. ARL INDEX RANKING.
    Target1: The U.Va. Library should rank in the top 20.
    Target2: The U.Va. Library should remain ranked in the top 25.
    Method: The Association of Research Libraries (ARL) includes 112 of the largest academic libraries in North America. Each year ARL uses principal component analysis to aggregate five data categories: volumes held, volumes added (gross), current serials, total library expenditures, total professional and support staff. For each library this analysis produces the "ARL Library Membership Index." In 1999/2000 (the latest year available) the University of Virginia Library ranks 23rd.
  2. EDUCATE USERS IN THE SKILLS NECESSARY TO FULFILL THEIR INFORMATION NEEDS

    METRIC IV. OVERALL INSTRUCTION RATING IN USER EDUCATION PARTICIPANT SURVEY.
    Target1: An overall rating of 4 on a scale of 1 to 5.
    Target2: An overall rating of at least 3.8.
    Method: Evaluation will ascertain how well the training received correlates to the needs of the audience. A brief survey of 3-4 questions developed by Management Information Services (MIS) and the User Education Committee will be distributed via e-mail to a sample of participants of course-related user education classes at the end of the semester during which instruction is received. The survey will include an overall rating question.

    METRIC V. ATTENDANCE AT SPECIAL COLLECTIONS SESSIONS.
    Target1: Increase by 20% annually the number of informational, instructional, and outreach sessions and the number of persons attending sessions in Special Collections.
    Target2: Increase by 10% the number of sessions and the number attending.
    Method: Special Collections will count and record the number of sessions and persons attending each fiscal year.
  3. SELECT, BUILD, MAINTAIN, AND PRESERVE HIGH QUALITY COLLECTIONS THAT REFLECT THE NEEDS OF THE LIBRARYÃŒS USERS AND SUPPORT THE UNIVERSITYÃŒS MISSION

    METRIC VI. CIRCULATION OF NEW MONOGRAPHS.
    Target1: 60% of all newly cataloged monographs (excluding reference books and other non-circulating materials) should circulate to U.Va. patrons within two years of the time of cataloging. Newly acquired or newly digitized electronic monographs will be included as part of this target.
    Target2: 50% of all new titles should circulate (or be used) within the first two years.
    Method for print material: A program will extract data from the SIRSI records. We expect to retrieve the data retrospectively, enabling us to examine the two-year circulation of print monographs acquired in 1999.
    Method for digital material: The procedures for tallying use of digital monographs are not yet established; we expect to use WebTrends to obtain the data, but it may not be possible to do so retrospectively.
    Separate Tallies: The numbers for print material and digital materials will be kept and reported separately.
  4. PROVIDE CONVENIENT AND TIMELY ACCESS TO THE LIBRARY'S COLLECTIONS

    METRIC VII. TURNAROUND TIME FOR PROCESSING AND FILLING USER REQUESTS.
    Target1: Satisfy the turnaround targets 90% of the time.
    Target2: Satisfy the turnaround targets 80% of the time.
    Categories:
    Searches: Notify patron of results of search within 4 days of request.
    Recalls: Process requests within 1 day.
    ILL: Fill requests in 7 days.
    LEO: Fill requests in 7 days.
    Toolkit: Fill requests by agreed upon date.
    New Books: Fill requests in 7 days.

INTERNAL PROCESS PERSPECTIVE: How do the library's internal processes function to efficiently deliver library collections and services?

  1. ACQUIRE, CREATE, ORGANIZE, PRESERVE AND DELIVER INFORMATION RESOURCES IN A TIMELY, EFFICIENT, AND ACCURATE MANNER

    METRIC VIII. PROCESSING TIME FOR ROUTINE ACQUISITIONS.
    Target1: 90% of in-print books from North America will be processed within one month.
    Target2: 80% of in-print books from North America will be processed within one month.
    Method: This metric will calculate the elapsed time from ordering to delivery at the destination library. Certain classes of materials will be excluded: user requests, non-book materials, and orders for Special Collections, Law, and Health Sciences. The methodology will rely on sampling SIRSI records. For most Alderman and Clemons materials, we will measure electronically from the order date or received date to the cataloging date. For materials in other libraries, we will add an average in-transit time.

    METRIC IX. TURNAROUND TIME FOR SHELVING BOOKS.
    Target1: Shelve 90% of items within 4 hours of their return to the home libraryÃŒs circulation desk with an average accuracy rate for each library of 98%.
    Target2: Shelve 90% of items within 6 hours of their return to the home libraryÃŒs circulation desk with an average accuracy rate for each library of 98%.
    Method: Use the methods currently in place for measuring re-shelving turnaround time in Alderman, Clemons, Fine Arts, Government Documents, and SEL; a one week sample can be done each semester in the smaller libraries. Quality control checks should be done twice a month in each unit.
  2. USE RESOURCES IN THE MOST INNOVATIVE, EFFICIENT, AND EFFECTIVE WAY POSSIBLE

    METRIC X. STAFF SURVEY RATING OF INTERNAL SERVICE ACTIVITIES.

    Target1: Composite rating of 4.0 of all internal service activities in annual staff survey, with no units rated below 3.5.
    Target2: Composite rating of 3.5, with no units below 3.0.
    Method: The University Library will conduct a staff survey once a year. As part of the survey, each staff member will have the opportunity to rate the performance of each internal service unit or activity (e.g., Administration, Budget, Business Services, Communications and Publications, Interlibrary Services deliveries, Library Human Resources, Library Technology Services, Management Information Services) on a 1 to 5 scale.
  3. DEVELOP WITHIN THE LIBRARY A CULTURE OF ASSESSMENT

    METRIC XI. PERCENTAGE OF DEPARTMENTS WITH AN ASSESSMENT PLAN.
    Target1: 100% participation.
    Target2: 80% participation.
    Method: Our goal should be to increase the quality and quantity of assessment activities as we move toward a culture of assessment. To further this goal, each department or other unit that reports to the University Librarian or an Associate University Librarian should develop an assessment plan to coordinate assessment activities within their unit. MIS will assist unit managers as they develop plans and will monitor and tally the results for this metric.
  4. CONTINUOUSLY IDENTIFY, REVIEW, AND IMPROVE THE PROCESSES THAT HAVE THE GREATEST IMPACT

    METRIC XII. PROCESS IMPROVEMENT.
    Target1: Complete at least one major process improvement each year.
    Target2: Complete at least one major process improvement every two years.
    Method: Each year the Library seeks to identify major activities, review relevant policies and procedures, and make changes that will improve efficiency. By July 1 the Associate University Librarian for Organizational Development will send a message to all library staff asking them to submit ideas for library processes that could be improved using new technology or more efficient methods. The Administrative Group will consider the proposals and decide on the process improvement project for the year. The AUL for Organizational Development will monitor this metric. (Examples: The shelving and 3-C timeliness initiatives both improved library services.)

FINANCE PERSPECTIVE: How well are the library's finances managed to achieve our mission?

  1. INCREASE THE FINANCIAL BASE THROUGH PRIVATE DONATIONS, OTHER EXTERNAL SUPPORT, AND INCREASED INSTITUTIONAL SUPPORT

    METRIC XIII. NON-STATE EXPENDITURES AS A PROPORTION OF TOTAL UNIVERSITY LIBRARY EXPENDITURES.
    Target1: The target for FY2002 will be 10%. The target will increase by one half of one percent each year, i.e., the target for FY2003 will be 10.5%, for FY2004 it will be 11% etc.
    Target2: The partial success indicator for FY2002 will be 9%.
    Method: Non-state expenditures include annual and unrestricted gifts, sponsored programs, grants, endowment and project income, and e-commerce. These expenditures will be compared with the total expenditures of the University Library. The Budget Director will work with the Director of Development to monitor this figure each year.

    METRIC XIV. LIBRARY BUDGET AS A PROPORTION OF THE UNIVERSITY BUDGET.
    Target1: The University Libraries should continue to account for at least 4% of the E&G (educational and general) share of the University budget.
    Target2: The University Libraries should account for at least 3.75% of E&G.
    Method: The total of the E&G budgets for all University of Virginia Libraries, including Health Sciences, Darden, and Law, is currently approximately 4% of the overall University E&G budget. The Budget Director will monitor this figure each year.
  2. PROVIDE RESOURCES AND SERVICES THAT HAVE A HIGH RATIO OF VALUE TO COST

    Metric XV. EXPENDITURES FOR CIRCULATING MONOGRAPHS AS A PROPORTION OF ALL MONOGRAPHIC EXPENDITURES.
    Target1: Monographs that circulate within the first two years should account for at least 70% of the monographic budget.
    Target2: The circulated items should account for at least 60% of the monographic budget.
    Method: A yearly study similar to the 1993-95 baseline study will be conducted. The study will track purchased monographs for two years to determine if a customer uses them.

    METRIC XVI. UNIT COST OF SERIAL USE (PAPER AND ELECTRONIC)
    Target1. 0% increase in unit cost each year.
    Target2. Less than 5% increase in unit cost each year.
    Method: Once each year survey use of a sample of 400 paper serials (both unbound current issues and bound back volumes) and of all electronic journals for which usage data are available. Data for bound volumes will be from SIRSI; data for e-journals from statistics regularly reported by vendors. For unbound paper serials the most rigorous approach to counting uses is by sequestering the serials and handing them out on demand. If this method is not feasible, approximations to rigorous counts can be estimated by asking users to leave serials for staff to re-shelve or through other methods to be decided upon.
    Separate Tallies: The numbers for print journals and electronic journals will be kept and reported separately.

    METRIC XVII. UNIT COST OF ONLINE DATABASES.
    Target1: 0% increase in unit cost each year.
    Target2: Less than 5% increase in unit cost each year.
    Method: Divide UVaÃŒs annual payment for each database by the annual usage number for that database. A single measurement can be provided by dividing the total cost of all databases (with reported usage statistics) by total uses. Usage is defined as either searches of or accesses to the database, whichever figure is available from the vendor. Searches are preferred if both figures are available. MIS currently has a database set up for collecting this data. We need to conduct an initial study of cost per use; the data from later years will be compared to the results of the initial study.

    METRIC XVIII. UNIT COST OF PROCESSING LIBRARY MATERIALS.
    Target1. Annual percent increase in unit cost to be equal to or less than annual percent increase in overall library operating budget.
    Target2: Annual increase in unit cost will be within 5% of the annual percent increase in overall library operating budget.
    Method: We will calculate the total expenditures and the number of transactions in each of three departments: Acquisitions, Cataloging, and Interlibrary Services. The unit cost is the expenditure divided by the number of transactions.
    Separate Tallies: The unit costs for Acquisitions, Cataloging, and Interlibrary Services will be tallied and reported separately.

    METRIC XIX. UNIT COST OF PUBLIC SERVICE TRANSACTIONS
    Target1. Annual percent increase in unit cost to be equal to or less than annual percent increase in overall library operating budget.
    Target2: Annual increase in unit cost will be within 5% of the annual percent increase in overall library operating budget.
    Method: Analogous to metric XVIII above. We will calculate the expenditures and the number of transactions for each of three functions: circulation, instruction, and reference. The methods of data collection will vary for three functions. Reference will be based on a two week sample; for circulation and instruction we will use data for an entire year.
    Separate Tallies: The unit costs for circulation, instruction, and reference will be tallied and reported separately.

LEARNING/FUTURE PERSPECTIVE: How well is the library positioned to ensure that goals are met in the future?

  1. FOSTER LEARNING AMONG ITS EMPLOYEES TO ENCOURAGE CREATIVITY, COOPERATION, AND INNOVATION

    METRIC XX. DEVELOP INDIVIDUALIZED AND DEPARTMENTAL LEARNING PLANS.
    Target1: 90% of library staff and departments have completed learning plans.
    Target2: 80% of library staff and departments have completed learning plans.
    Method: In FY2002, develop individualized learning plans that outline both developmental and job-skills training needs for each library employee so that 60% are completed. In addition, develop departmental training plans for each department and branch or unit so that 60% are completed by the end of the year. The targets will be revised for FY2003.
  2. RECRUIT, DEVELOP, AND RETAIN PRODUCTIVE, HIGHLY QUALIFIED STAFF

    METRIC XXI. RETENTION RATE OF COMMENDED EMPLOYEES.
    Target1: 95% of commended employees remain on the staff.
    Target2: 90% of commended employees remain on the staff.
    Method: Staff members who have received at least one extraordinary contributor write-up (or at least one "exceeds expectations" or "outstanding" rating in the old system) and faculty who have received at least one "extra-meritorious" or "highest" salary increase over the past three performance cycles are identified at the beginning of each fiscal year. The Library may also take other forms of recognition into consideration. At the end of the year, we determine what percentage of these employees still work at the library.
  3. PROVIDE FACILITIES THAT PROMOTE STAFF PRODUCTIVITY, ENCOURAGE LIBRARY USE AND ENSURE TOP QUALITY SERVICES

    METRIC XXII. RELIABILITY OF EQUIPMENT.
    Target1: Staff and user workstations (computers, scanners, order pickers, video, audio, microform, and copy equipment) shall function properly at least 95% of the time.
    Target2: Equipment shall function properly at least 90% of the time.
    Method: A random sampling of workstations will be surveyed throughout the year. A workstation is considered down if any core function of the workstation is unusable for more than 15 minutes (e.g. if the only printer for a computer workstation is inoperative for more than 15 minutes, the workstation is considered down, despite being able to perform other functions).

    METRIC XXIII. RENOVATION OF MAJOR STAFF AND PUBLIC AREAS.
    Target1: Three or more major renovations per year of which at least one is a staff area and one is a public area.
    Target2: Two major renovations per year.
    Method: Renovation of existing space allows the Library the opportunity to improve service and increase staff satisfaction and productivity and satisfaction. The Budget Director will solicit suggestions for renovations at the same time as budget requests are made. Administrative Group will decide which areas will be renovated.
  4. DEVELOP AN INNOVATIVE AND EFFECTIVE INFORMATION TECHNOLOGY INFRASTRUCTURE TO SUPPORT LIBRARY SERVICES AND VARIOUS INITIATIVES

    METRIC XXIV. EXPENDITURES FOR DIGITAL MATERIALS.
    Target1: Ensure that U.Va. is in the top quarter of all ARL libraries in our percentage of collections dollars expended on digital materials.
    Target2: Ensure that U.Va. is in the top third of all ARL libraries in our percentage of collections dollars expended on digital materials.
    Method: MIS will determine, and report, the percentage of the collections budget spent on digital materials at the end of the fiscal year.

    METRIC XXV. USABILITY TESTING OF WEBSITES.
    Target1: At least 75% of all new and revised websites receive heuristic evaluation and at least 25% undergo full or discount usability testing.
    Target2: At least 50% receive heuristic evaluation and at least 25% undergoing full or discount usability testing for the first year.
    Method: The Library's Web presence provides the key entry point for our users, making it the hub of technology infrastructure. Ease of use is a key ingredient to building a customer-orientated system. A web usability team will be responsible for assigning heuristic testing and arranging usability testing by library customers, both internal and external. This group will also tally and record the number of new websites receiving heuristic and usability testing and compare this number to the total number of new websites.

    METRIC XXVI. INCREASING THE QUALITY OF DIGITAL ACCESS.
    Target1: 100% of identified priority content will be produced per DCRT (Digital Contents Review Team) standards within six months.
    Target2: 90% of identified priority content produced per DCRT standards produced within six months.
    Method: Digital content will be produced according to priorities and standards set by DCRT for 2001/2002. Percentage completed is determined by comparing the amount DCRT identifies as priority content with the amount produced that meets standards as set by DCRT.


University of Virginia Library
PO Box 400113, Charlottesville, VA 22904-4113
ph: (434) 924-3021, fax: (434) 924-1431, library@virginia.edu

Text Version    |   Libraries   |   Depts./Contacts   |  U.Va. Home   |   ITC

Website Feedback   |   Search   |   Questions? Ask a Librarian   |   Hours   |   Map   |   Policies   |   Jobs

Tracking Opt-out    |   © by the Rector and Visitors of the University of Virginia

Federal Library Depository logo  This library is a Congressionally designated depository for U.S. Government documents. Public access to the Government documents is guaranteed by public law.