Go to

 

 
Go to Balanced Scorecard home

2003-04 Balanced Scorecard Metrics

The Balanced Scorecard is a management technique designed to provide a view of an organization from four perspectives: user, internal processes, financial, and future/learning potential. This document explains how the University of Virginia Library is defining each of these four areas by providing the details of the metrics we are using in determining our success in each perspective.

Each of the metrics described below contains two targets. Target1 indicates complete success in achieving the metric; Target2 indicates partial success. Management Information Services will compile and report the results of all metrics on a regular basis.


USER PERSPECTIVE: How well is the library meeting the needs of our users?

  1. PROVIDE EXCELLENT SERVICE TO USERS OF THE UNIVERSITY OF VIRGINIA LIBRARY

    METRIC U.1.a. OVERALL RATING IN STUDENT AND FACULTY SURVEYS.
    Target1: We should attain a score of at least 4.00 (out of 5.00) from each of the major constituencies: undergraduates, graduates, humanities faculty, social science faculty, science faculty.
    Target2: Score of at least 3.90 from each of the major constituencies.
    Method: The University Library conducts extensive surveys of our clientele approximately every other year. A final question in each survey asks the respondent to "rate your overall satisfaction..." with the Library using a 1 to 5 scale. This metric will consider the two latest surveys on record.
    Separate Tallies: The ratings for faculty and students will be kept and reported separately.

    METRIC U.1.b. CUSTOMER SERVICE RATING IN STUDENT AND FACULTY SURVEYS.
    Target1: Customer services should receive a score of at least 4.25 (out of 5.00) from each of the major constituencies: undergraduates, graduates, humanities faculty, social science faculty, science faculty.
    Target2: Score of at least 4.00 from each of the major constituencies.
    Method: As part of its series of surveys, the University Library now asks respondents to rate three aspects of customer service: speed, courtesy, accuracy. The question refers very generally to customer service and thus could be applied to quality of service received at any service point: reference, circulation, e-centers, email, phone, correspondence, etc. Scoring is on a scale of one to five, five being "most satisfactory." The three scores will be combined into one composite number. This metric will consider the two latest surveys on record.

    METRIC U.1.c. ARL INDEX RANKING.
    Target1: The U.Va. Library should rank in the top 20.
    Target2: The U.Va. Library should remain ranked in the top 25.
    Method: The Association of Research Libraries (ARL) includes 112 of the largest academic libraries in North America. Each year ARL uses principal component analysis to aggregate five data categories: volumes held, volumes added (gross), current serials, total library expenditures, total professional and support staff. For each library this analysis produces the "ARL Library Membership Index."
  2. EDUCATE USERS IN THE SKILLS NECESSARY TO FULFILL THEIR INFORMATION NEEDS

    METRIC U.2.a. OVERALL INSTRUCTION RATING IN USER EDUCATION PARTICIPANT SURVEY.
    Target1: An overall rating of 4.00 on a scale of 1 to 5.
    Target2: An overall rating of at least 3.80.
    Method: Evaluation will ascertain how well the training received correlates to the needs of the audience. A brief survey of 3-4 questions developed by Management Information Services (MIS) and the User Education Committee will be distributed via e-mail to a sample of participants of course-related user education classes at the end of the semester during which instruction is received. The survey will include an overall rating question.

    METRIC U.2.b. USE OF SPECIAL COLLECTIONS MATERIALS.
    Target1: Increase by 10% annually the number of materials used or circulated in Special Collections.
    Target2: Increase by 5% the number of materials used or circulated.
    Method: Special Collections will perform a SIRSI check-out or itemuse for materials used from the collections for any purpose (patron use, classes, reference, show & tell, on hold, etc.). A SIRSI report can be run at the end of the year to count the uses. Data for archives is collected manually and will be added to the SIRSI data.
  3. SELECT, BUILD, MAINTAIN, AND PRESERVE HIGH QUALITY COLLECTIONS THAT REFLECT THE NEEDS OF THE LIBRARY'S USERS AND SUPPORT THE UNIVERSITY'S MISSION

    METRIC U.3.a. CIRCULATION OF NEW MONOGRAPHS.
    Target1: 60% of all newly cataloged monographs (excluding reference books and other non-circulating materials) should circulate within two years of the time of cataloging. Usage of newly acquired or newly digitized electronic monographs will be measured separately.
    Target2: 50% of all new titles should circulate within the first two years. Usage of newly acquired or newly digitized electronic monographs will be measured separately.
    Method for print material: A program will extract data from the SIRSI records, enabling us to examine the two-year circulation of print monographs. Count items circulated to users only, not binding, etc.
    Method for digital material: A program will extract cataloging data from the SIRSI records. These will be compared to U.Va. usage statistics gathered both from local servers and from vendors.
    Separate Tallies: The numbers for print material and digital materials will be kept and reported separately.
  4. PROVIDE CONVENIENT AND TIMELY ACCESS TO THE LIBRARY'S COLLECTIONS

    METRIC U.4.a. TURNAROUND TIME FOR PROCESSING AND FILLING USER REQUESTS.
    Target1: Satisfy the turnaround targets 90% of the time.
    Target2: Satisfy the turnaround targets 80% of the time.
    Categories:
    Searches: Notify patron of results of search within 4 days of request.
    ILL: Fill requests in 7 days.
    LEO: Fill requests in 7 days.
    Toolkit: Fill requests by agreed upon date.
    New Books: Fill requests in 7 days.
    Method: Each category will require development of a separate methodology.
  5. METRIC U.4.b. REDUCE THE NUMBER OF RECALLS PLACED EACH YEAR.
    Target1: Reduce the number of recalls by 5%.
    Target2: Reduce the number of recalls by 0%.
    Method: Using data from the library's annual statistical report, compare the number of recalls placed in the current year to the number placed in the previous year.

INTERNAL PROCESS PERSPECTIVE: How do the library's internal processes function to efficiently deliver library collections and services?

  1. ACQUIRE, CREATE, ORGANIZE, PRESERVE AND DELIVER INFORMATION RESOURCES IN A TIMELY, EFFICIENT, AND ACCURATE MANNER

    METRIC I.1.a. PROCESSING TIME FOR ROUTINE ACQUISITIONS.
    Target1: 90% of in-print books from North America will be processed within one month.
    Target2: 80% of in-print books from North America will be processed within one month.
    Method: This metric will estimate the elapsed time from ordering to delivery at the destination library. Certain classes of materials will be excluded: user requests, non-book materials, out-of-print books, foreign acquisitions,and orders for Special Collections, Law, and Health Sciences. The methodology will rely on SIRSI records for Yankee and New York Times/Washington Post approval orders.

    METRIC I.1.b. TURNAROUND TIME FOR RESHELVING/REFILING.
    Target1: Accurately shelve/refile 85% of all materials by the end of the day following discharge regardless of where the item was discharged.
    Target2: Accurately shelve/refile 75% of all materials by the end of the day following discharge regardless of where the item was discharged.
    Method: On a random fifty dates, a sample of 20 items from all libraries will be selected from the uses and discharges of the previous day for checking in the stacks. The overall annual percentage is determined by dividing the number of items meeting the criteria by the total sample size.
  2. USE RESOURCES IN THE MOST INNOVATIVE, EFFICIENT, AND EFFECTIVE WAY POSSIBLE

    METRIC I.2.a. STAFF SURVEY RATING OF INTERNAL SERVICE ACTIVITIES.

    Target1: Composite rating of 4.00 of all internal service activities in annual staff survey, with no units rated below 3.50.
    Target2: Composite rating of 3.50, with no units below 3.00.
    Method: The University Library will conduct a staff survey each spring. As part of the survey, each staff member will have the opportunity to rate the performance of each internal service unit or activity (e.g., Administration, Acquisitions, Cataloging, Budget, Business Services, Communications and Publications, Interlibrary Services deliveries, Library Human Resources, Library Information Technology Services, Management Information Services) on a 1 to 5 scale.

    METRIC I.2.b. INTERNAL COMMUNICATIONS RATING.
    Target1: A score of 4.00 on internal communications in the annual staff work life survey, with no rating below 3.50 for a single question.
    Target2: A score of 3.75, with no rating below 3.25 for a single question.
    Method: Survey all staff anonymously on the Web each fall to ascertain their rating of internal communication in four areas: communication from administration, communication between departments, communication within departments, and an overall rating.
  3. CONTINUOUSLY IDENTIFY, REVIEW, AND IMPROVE THE PROCESSES THAT HAVE THE GREATEST IMPACT

    METRIC I.3.a. PROCESS IMPROVEMENT.
    Target1: Complete at least one major process improvement each year.
    Target2: Complete at least one major process improvement every two years.
    Method: Each year the Library seeks to identify major activities, review relevant policies and procedures, and make changes that will improve efficiency. By July 1 the Associate University Librarian for Organizational Development will send a message to all library staff asking them to submit ideas for library processes that could be improved using new technology or more efficient methods. The Administrative Group will consider the proposals and decide on the process improvement project for the year. The AUL for Organizational Development will monitor this metric. (Examples: The shelving and 3-C timeliness initiatives both improved library services.).

FINANCE PERSPECTIVE: How well are the library's finances managed to achieve our mission?

  1. INCREASE THE FINANCIAL BASE THROUGH PRIVATE DONATIONS, OTHER EXTERNAL SUPPORT, AND INCREASED INSTITUTIONAL SUPPORT

    METRIC F.1.a. NON-STATE EXPENDITURES AS A PROPORTION OF TOTAL UNIVERSITY LIBRARY EXPENDITURES.
    Target1: The target for FY2002, our initial year for Balanced Scorecard, was 10%. The target will increase by one half of one percent each following year, i.e., the target for FY2003 will be 10.5%, for FY2004 it will be 11.0% etc.
    Target2: The partial success indicator for each following year is .5% less than the Target1 indicator.
    Method: Non-state expenditures include annual and unrestricted gifts, sponsored programs, grants, endowment and project income, and e-commerce. These expenditures will be compared with the total expenditures of the University Library. The Budget Director will work with the AUL for Development to monitor this figure each year.

    METRIC F.1.b. LIBRARY EXPENDITURES AS A PROPORTION OF THE UNIVERSITY EXPENDITURES.
    Target1: The University Libraries should continue to account for at least 4.00% of the E&G (educational and general) share of the University expenditures.
    Target2: The University Libraries should account for at least 3.75% of E&G.
    Method: Total University E&G expenditures will be obtained from Institutional Assessment and Studies; Library expenditures will be obtained from Oracle. These two figures will be used to calculate the percent.

    METRIC F.1.c. AMOUNT OF LIBRARY DONATIONS THAT ARE UNRESTRICTED.
    Target1: Increase the amount of giving that is unrestricted by 10%.
    Target2: Increase the amount of giving that is unrestricted by 5%.
    Method: Use any funds donated (endowment, annual giving, planned gifts, etc.) for which the library may make the decision for its use and which could be used to replace state funds if needed. The baseline is FY 2002/03.
  2. PROVIDE RESOURCES AND SERVICES THAT HAVE A HIGH RATIO OF VALUE TO COST

    METRIC F.2.a. UNIT COST OF ELECTRONIC SERIAL USE
    Target1. 0% increase in unit cost each year.
    Target2. Less than 5% increase in unit cost each year.
    Method: Collect data annually from vendors for all electronic journals for which usage data are available. Cost data can be gathered from SIRSI. Divide the total cost by the total uses.
  3. METRIC F.2.b. UNIT COST OF ONLINE DATABASES.
    Target1: 0% increase in unit cost each year.
    Target2: Less than 5% increase in unit cost each year.
    Method: Divide UVa's annual payment for each database by the annual usage number for that database. A single measurement can be provided by dividing the total cost of all databases (with reported usage statistics) by total uses. Usage is defined as either searches of or accesses to the database, whichever figure is available from the vendor.

    METRIC F.2.c. UNIT COST OF PROCESSING LIBRARY MATERIALS.
    Target1. Annual percent change in unit cost will be equal to or less than annual percent change in overall library operating expenditures.
    Target2: Annual percent change in unit cost will be within 5% of the annual percent change in overall library operating expenditures.
    Method: We will calculate the total expenditures and the number of transactions in each of three departments: Acquisitions, Cataloging, and Digital Library Production Services. The unit cost is the expenditure divided by the number of transactions.
    Separate Tallies: The unit costs for Acquisitions, Cataloging (including Preservation), and Digital Library Production Services will be tallied and reported separately.

    METRIC F.2.d. UNIT COST OF PUBLIC SERVICE TRANSACTIONS
    Target1. Annual percent change in unit cost will be equal to or less than annual percent change in overall library operating expenditures.
    Target2: Annual change in unit cost will be within 5% of the annual percent change in overall library operating expenditures.
    Method: Analogous to Metric F.2.c. above. We will calculate the expenditures and the number of transactions for each of three functions: circulation, reference, and ILS. Data collection for circulation and reference will be based on a two week sample.
    Separate Tallies: The unit costs for ILS (including Instructional Scanning Services), circulation, and reference will be tallied and reported separately.

LEARNING/GROWTH PERSPECTIVE: How well is the library positioned to ensure that goals are met in the future?

  1. FOSTER LEARNING AMONG ITS EMPLOYEES TO ENCOURAGE CREATIVITY, COOPERATION, AND INNOVATION

    METRIC L.1.a. IMPACT OF TRAINING.
    Target1: A rating of 4.00 for overall satisfaction with training activities in the annual work-life quality survey, with no individual training measures below 3.50
    Target2: Overall rating of 3.75, with no measures below 3.25.
    Method: The University Library will conduct a work-life quality survey once a year. As part of the survey, each staff member will have the opportunity to rate his or her satisfaction with different aspects of training on a 1 to 5 scale.

    METRIC L.1.b. DIVERSITY OF STAFF.
    Target1: A net increase of at least 4 in faculty/staff diversity, with a net increase of at least 2 in faculty diversity.
    Target2: A net increase of at least 2 in faculty/staff diversity, with a net increase of at least 1 in faculty diversity.
    Method: Using EEO categories for diversity, Library HR will determine the net change in staff diversity from the previous year.
  2. RECRUIT, DEVELOP, AND RETAIN PRODUCTIVE, HIGHLY QUALIFIED STAFF

    METRIC L.2.a. JOB SATISFACTION AMONG LIBRARY EMPLOYEES.
    Target1: A score of 4.25 for overall job satisfaction in annual staff work-life survey, with no rating below 3.50 for a single question.
    Target2: A score of 3.75, with no rating below 3.25 for a single question.
    Method: The University Library will conduct a work-life quality survey once a year. As part of the survey, each staff member will have the opportunity to rate his or her satisfaction with the employment environment on a 1 to 5 scale.

    METRIC L.2.b. RETENTION RATE OF COMMENDED EMPLOYEES.
    Target1: 95% of commended employees remain on the staff.
    Target2: 90% of commended employees remain on the staff.
    Method: Staff members who have received at least one extraordinary contributor write-up (or at least one extra-meritorious or outstanding rating in the old system) and faculty who have received at least one extra-meritorious or highest salary increase over the past three performance cycles are identified at the beginning of each fiscal year. The Library may also take other forms of recognition into consideration. At the end of the year, we will determine what percentage of these employees still work at the library.

    METRIC L.2.c. COMPARE STAFF SALARIES (FACULTY AND CLASSIFIED) TO PEER GROUPS.
    Target1: Library faculty salary rating in the upper 40% of all ARL institutions and 60% of classified employees above the mean salary for similar jobs in the State of Virginia.
    Target2: Library faculty salary rating in the upper 50% of all ARL institutions and 50% of classified employees above the mean salary for similar jobs in the State of Virginia.
    Method: Faculty salary rating determined by annual ARL statistical reports. Library HR and/or MIS will compare salary information from the most recent report from the Bureau of Labor Statistics to detmerine which employees exceed the mean salary for comparable jobs in the state. If Watson Wyeth data become available, the Library will switch to that data source.
  3. PROVIDE FACILITIES THAT PROMOTE STAFF PRODUCTIVITY, ENCOURAGE LIBRARY USE AND ENSURE TOP QUALITY SERVICES

    METRIC L.3.a. RELIABILITY OF EQUIPMENT.
    Target1: Staff and user workstations (computers, scanners, video, audio, microform, copy equipment, etc.) shall function properly at least 95% of the time.
    Target2: Equipment shall function properly at least 90% of the time.
    Method: A random sampling of 20 workstations will be surveyed every other week. A workstation is considered down if any core function of the workstation is unusable when checked (e.g. if the only printer for a computer workstation is inoperative, the workstation is considered down, despite being able to perform other functions).

    METRIC L.3.b. RENOVATION OF MAJOR STAFF AND PUBLIC AREAS.
    Target1: Three or more major renovations per year of which at least one is a staff area and one is a public area.
    Target2: Two major renovations per year of either staff or public areas.
    Method: Renovation of existing space allows the Library the opportunity to improve service and increase staff satisfaction and productivity. The Director of Business Services will solicit suggestions for renovations at the same time as budget requests are made. Administrative Group will decide which areas will be renovated.
  4. DEVELOP AN INNOVATIVE AND EFFECTIVE INFORMATION TECHNOLOGY INFRASTRUCTURE TO SUPPORT LIBRARY SERVICES AND VARIOUS INITIATIVES

    METRIC L.4.a. EXPENDITURES FOR DIGITAL MATERIALS.
    Target1: U.Va. should be in the top quarter of all ARL libraries in our percentage of collections dollars expended on digital materials.
    Target2: U.Va. should be in the top third of all ARL libraries in our percentage of collections dollars expended on digital materials.
    Method: MIS will consult ARL supplementary statistics for this data.

    METRIC L.4.b. USABILITY TESTING OF LIBRARY WEBSITES.
    Target1: At least 75% of all websites under development or review should receive heuristic evaluation and at least 50% of all such sites should receive usability testing.
    Target2: At least 50% of all websites under development or review should receive heuristic evaluation and at least 25% of all such sites should receive usability testing.
    Method: The web usability team will be responsible for assigning heuristic testing and arranging usability testing by library customers, both internal and external. This group will also tally and record the number of new websites receiving heuristic and usability testing and compare this number to the total number of websites under development or review.
  5. METRIC L.4.c. INCREASING THE QUALITY OF DIGITAL ACCESS.
    Target1: 100% of identified priority content will be produced per DCRT (Digital Contents Review Team) standards.
    Target2: 90% of identified priority content will be produced per DCRT standards.
    Method: Percentage completed is determined by comparing the number of projects DCRT identifies as priority content with the number of projects produced that meets standards as set by DCRT.
  6. DEVELOP WITHIN THE LIBRARY A CULTURE OF ASSESSMENT.

    METRIC L.5.a. DEVELOP A CULTURE OF ASSESSMENT WITHIN THE LIBRARY.
    Target1: 75% of respondents score 12 or more affirmative responses on the ARL Culture of Assessment IQ.
    Target2: 50% of respondents score 12 of more affirmative responses on the ARL Culture of Assessment IQ.
    Method: MIS will use ARL's Culture of Assessment Institutional Quotient instrument to survey department heads and associate university librarians.


University of Virginia Library
PO Box 400113, Charlottesville, VA 22904-4113
ph: (434) 924-3021, fax: (434) 924-1431, library@virginia.edu

Text Version    |   Libraries   |   Depts./Contacts   |  U.Va. Home   |   ITC

Website Feedback   |   Search   |   Questions? Ask a Librarian   |   Hours   |   Map   |   Policies   |   Jobs

Tracking Opt-out    |   © by the Rector and Visitors of the University of Virginia

Federal Library Depository logo  This library is a Congressionally designated depository for U.S. Government documents. Public access to the Government documents is guaranteed by public law.