Go to

 

 
Go to Balanced Scorecard home

2004-05 Balanced Scorecard Metrics

The Balanced Scorecard is a management tool designed to provide a view of an organization from four perspectives: user, internal processes, financial, and learning/growth potential. The user perspective measures service or satisfaction with service. Internal processes looks at workflow efficiency; finance, at cost effectiveness. Learning/growth tells us how well prepared we are to meet new challenges and initiate new programs. This document explains what the University of Virginia Library wants to know in each of these four areas and the specific metrics we are using to inform us about each perspective.

Each of the metrics described below contains two targets. Target1 indicates complete success in achieving the metric; Target2 indicates partial success. Management Information Services will compile and report the results of all metrics annually.



USER PERSPECTIVE: How well is the library meeting the needs of our users?

  1. PROVIDE EXCELLENT SERVICE TO USERS OF THE UNIVERSITY OF VIRGINIA LIBRARY
    [Guiding Principles: Providing superb service that meets our users' needs and surpasses their expectations.]

    METRIC U.1.a. OVERALL RATING IN STUDENT AND FACULTY SURVEYS.
    Target1: A score of at least 4.00 (out of 5.00) from each of the major constituencies: undergraduate students, graduate students, humanities faculty, social science faculty, science faculty.
    Target2: A score of at least 3.90 from each of the major constituencies.
    Method: The University Library conducts extensive surveys of our clientele approximately every other year. A final question in each survey asks the respondent to "rate your overall satisfaction…" with the Library using a 1 to 5 scale. This metric will consider the two latest surveys on record.
    Separate Tallies: The ratings from faculty and students will be kept and reported separately.

    METRIC U.1.b. CUSTOMER SERVICE RATING IN STUDENT AND FACULTY SURVEYS.
    Target1: A score of at least 4.25 (out of 5.00) from each of the major constituencies: undergraduate students, graduate students, humanities faculty, social science faculty, science faculty.
    Target2: A score of at least 4.00 from each of the major constituencies.
    Method: As part of its series of surveys, the University Library asks respondents to rate three aspects of customer service: speed, courtesy, competence. The question refers very generally to customer service and thus could be applied to quality of service received at any service point: reference, circulation, e-centers, email, phone, correspondence, etc. The scoring scale is 1 to 5, with 5 being "most satisfactory." The three scores will be combined into one composite number. This metric will consider the two latest surveys on record.
  2. EDUCATE USERS IN THE SKILLS NECESSARY TO FULFILL THEIR INFORMATION NEEDS
    [Guiding Principles: Providing access to great collections that are easy to find, easy to use, and that inspire learning and scholarship. Providing superb service that meets our users' needs and surpasses their expectations.]

    METRIC U.2.a. OVERALL INSTRUCTION RATING IN USER EDUCATION PARTICIPANT SURVEY.
    Target1: An overall rating of 4.00 on a scale of 1 to 5.
    Target2: An overall rating of at least 3.80.
    Method: Evaluation will ascertain how well the training received correlates to the needs of the audience. An evaluation form will be provided to each participant of course-related user education classes at the end of the session. This evaluation will include an identically worded overall rating question. The wording of this question will be approved by User Services Council. Scores for this question will be entered into a database and an average rating from all respondents calculated by Management Information Services.
  3. SELECT, BUILD, MAINTAIN, AND PRESERVE HIGH QUALITY COLLECTIONS THAT REFLECT THE NEEDS OF THE LIBRARY'S USERS AND SUPPORT THE UNIVERSITY'S MISSION
    [Guiding Principles: Taking responsibility for realizing our vision through effective stewardship of collections, people, funds, and facilities.]

    METRIC U.3.a. CIRCULATION OF NEW MONOGRAPHS.
    Target1: 60% of all new monographs (excluding reference books and other non-circulating materials) circulate within two years of the time of cataloging.
    Target2: 50% of all new titles circulate within the first two years.
    Method: A program will extract data from the SIRSI records, enabling us to examine circulation of print monographs over a two-year cycle. Only items circulated to users (not binding, etc.) will be counted.

    METRIC U.3.b. USE OF SPECIAL COLLECTIONS MATERIALS.
    Target1: Increase by 10% annually the number of materials used or circulated in Special Collections.
    Target2: Increase by 5% the number of materials used or circulated.
    Method: Special Collections will perform a SIRSI check-out or itemuse for materials used from the collections for any purpose (patron use, classes, reference, show & tell, on hold, etc.). A SIRSI report will be run at the end of the year to count the uses. Data for archives is collected manually and will be added to the SIRSI data.
  4. PROVIDE CONVENIENT AND TIMELY ACCESS TO THE LIBRARY'S COLLECTIONS
    [Guiding Principles: Providing access to great collections that are easy to find, easy to use, and that inspire learning and scholarship.]

    METRIC U.4.a. TURNAROUND TIME FOR SEARCHES, LEO, AND INSTRUCTIONAL SCANNING (LOCAL RESOURCES).
    Target1: Satisfy the turnaround targets 90% of the time.
    Target2: Satisfy the turnaround targets 80% of the time.
    Method:
    Searches: Search for requested item within 1 day and notify patron of results within 4 days of request. Staff will tally data as searches are performed, including: the number of searches requested, the number of requests processed, how quickly the search was performed and how quickly the patron was notified. This data will be reported to Management Information Services each month.
    LEO: Fill requests in 7 days. A sample of LEO requests will be drawn from Sirsi by Management Information Services. Turnaround time from patron request to delivery will be measured.
    Instructional Scanning: Fill requests by agreed upon date. Instructional Scanning Services will provide a report in their database to track how quickly scanning requests are filled.

    METRIC U.4.b. TURNAROUND TIME FOR NEW BOOK AND ILL REQUESTS (EXTERNAL RESOURCES).
    Target1: Satisfy the turnaround targets 75% of the time.
    Target2: Satisfy the turnaround targets 50% of the time.
    Method:
    New Books: Fill requests in 7 days. A sample of in-print US titles requested by University-affiliated patrons will be drawn from the request database by Management Information Services. Turnaround time from patron request to availability will be measured.
    ILL: Fill requests in 7 days. A sample of patron requests will be drawn from the ILLiad database by Management Information Services. Turnaround time will be measured from patron request to notification that the book or article is available.
  5. METRIC U.4.c. REDUCE THE NUMBER OF RECALLS PLACED EACH YEAR.
    Target1: Reduce the number of recalls by 5%.
    Target2: Reduce the number of recalls by 0%.
    Method: Using data from the library's annual statistical report, compare the number of recalls placed in the current year to the number placed in the previous year.

INTERNAL PROCESS PERSPECTIVE: How do the library's internal processes function to efficiently deliver library collections and services?

  1. ACQUIRE, CREATE, ORGANIZE, PRESERVE AND DELIVER INFORMATION RESOURCES IN A TIMELY, EFFICIENT, AND ACCURATE MANNER
    [Guiding Principles: Building diverse staff, collections, services and physical spaces. Taking responsibility for realizing our vision through effective stewardship of collections, people, funds, and facilities. Being accountable for performance.]

    METRIC I.1.a. PROCESSING TIME FOR ROUTINE ACQUISITIONS.
    Target1: Process 90% of in-print books from North America within one month.
    Target2: Process 80% of in-print books from North America within one month.
    Method: This metric will estimate the elapsed time from ordering to delivery at the destination library. Certain classes of materials will be excluded: user requests, non-book materials, out of print books, foreign acquisitions, and orders for Special Collections, Law, and Health Sciences. The methodology will rely on SIRSI records for Yankee and New York Times/Washington Post approval orders.

    METRIC I.1.b. TURNAROUND TIME FOR RESHELVING/REFILING.
    Target1: Accurately shelve/refile 85% of all materials by the end of the day following discharge regardless of where the item was discharged.
    Target2: Accurately shelve/refile 75% of all materials by the end of the day following discharge regardless of where the item was discharged.
    Method: On a random hundred dates, a sample of 10 circulating books, bound journals, and documents from all libraries will be selected from the uses and discharges of the previous day for checking in the stacks. Each sample's results will be weighted according to the overall number of circulations for that day. The overall annual percentage is calculated by dividing the number of items meeting the criteria by the total sample size.
  2. OPERATE IN THE MOST INNOVATIVE, EFFICIENT, AND EFFECTIVE WAY POSSIBLE
    [Guiding Principles: Being accountable for performance.]

    METRIC I.2.a. INTERNAL COMMUNICATIONS RATING.
    Target1: Positive scores (4 or 5) on internal communications statements from 80% of respondents in the biennial staff work-life survey.
    Target2: Positive scores (4 or 5) on internal communications statements from 60% of respondents in the biennial staff work-life survey.
    Method: Survey all staff anonymously on the Web every other spring, alternating with the Internal Customer Service Survey, to ascertain their rating of internal communication in three areas: communication from administration, communication between departments, and communication within departments.

    METRIC I.2.b. USABILITY TESTING OF LIBRARY WEBSITES.
    Target1: At least 90% of all websites under development or review receive usability testing.
    Target2: At least 80% of all websites under development or review receive usability testing.
    Method: The web usability team will be responsible for assigning heuristic testing and arranging usability testing by library customers, both internal and external. This group will also tally and record the number of new websites receiving heuristic and usability testing and compare this number to the total number of websites under development or review.
  3. CONTINUOUSLY IDENTIFY, REVIEW, AND IMPROVE THE PROCESSES THAT HAVE THE GREATEST IMPACT
    [Guiding Principles: Taking responsibility for realizing our vision through effective stewardship of collections, people, funds, and facilities.]

    METRIC I.3.a. PROCESS IMPROVEMENT.
    Target1: Complete at least one major process improvement each year.
    Target2: Complete at least one major process improvement every two years.
    Method: Each year the Library seeks to identify major activities, review relevant policies and procedures, and make changes that will improve efficiency. By July 1 the Associate University Librarian for Organizational Development will send a message to all library staff asking them to submit ideas for library processes that could be improved using new technology or more efficient methods. The Administrative Group will consider the proposals and decide on the process improvement project for the year. The AUL for Organizational Development will monitor this metric. (Examples: The shelving and 3-C timeliness initiatives both improved library services.).

    METRIC I.3.b. STAFF SURVEY RATING OF INTERNAL SERVICE ACTIVITIES.
    Target1: A composite rating of at least 4.00 in the annual internal services survey, with no unit rated below 3.50.
    Target2: A composite rating of at least 3.50, with no unit rated below 3.00.
    Method: The University Library will conduct an internal services survey every other spring, alternating with the Worklife Survey. As part of the survey, each staff member will have the opportunity to rate the performance of each service unit or activity (e.g., Administration, Acquisitions, Cataloging, Education Library, Finance, Facilities, Geostat, Interlibrary Services, Library Human Resources, Library Information Technology Services, Management Information Services, Special Collections) on a 1 to 5 scale. The composite rating will be based on the overall rating given to each unit.
  4. PROVIDE FACILITIES THAT PROMOTE STAFF PRODUCTIVITY, ENCOURAGE LIBRARY USE AND ENSURE TOP QUALITY SERVICES
    [Guiding Principles: Building diverse staff, collections, services and physical spaces. Taking responsibility for realizing our vision through effective stewardship of collections, people, funds, and facilities.]

    METRIC I.4.a. RELIABILITY OF EQUIPMENT.
    This metric is suspended for FY05 while a new method is developed.
    Target1:
    Target2:
    Method:

    METRIC I.4.b. RENOVATION OF MAJOR STAFF AND PUBLIC AREAS.
    Target1: Three or more major renovations per year including at least one staff area and one public area.
    Target2: Two major renovations per year of either staff or public areas.
    Method: Renovation of existing space allows the Library the opportunity to improve service and increase staff satisfaction and productivity. The Director of Facilities will solicit suggestions for renovations at the same time as budget requests are made. Administrative Group will decide which areas will be renovated.

    METRIC I.4.c. RELIABILITY OF SERVERS.
    Target1: All library servers will be operational and accessible at least 98% of the time.
    Target2: All library servers will be operational and accessible at least 95% of the time.
    Method: This metric will be measured based on information generated by an Information Technology server report.

FINANCE PERSPECTIVE: How well are the library's finances managed to achieve our mission?

  1. INCREASE THE FINANCIAL BASE THROUGH PRIVATE DONATIONS, OTHER EXTERNAL SUPPORT, AND INCREASED INSTITUTIONAL SUPPORT
    [Guiding Principles: Providing access to great collections that are easy to find, easy to use, and that inspire learning and scholarship. Providing superb service that meets our users' needs and surpasses their expectations. Building diverse staff, collections, services and physical spaces. Taking responsibility for realizing our vision through effective stewardship of collections, people, funds, and facilities.]

    METRIC F.1.a. NON-INSTITUTIONAL (PRIVATE) EXPENDITURES AS A PROPORTION OF TOTAL UNIVERSITY LIBRARY EXPENDITURES.
    Target1: Increase non-institutional expenditures as a proportion of total University Library expenditures by one half of one percent each year (11.5% for FY2005).
    Target2: Increase the proportion by half a percentage point less than the Target1 indicator (11.0% for FY2005).
    Method: Non-institutional expenditures include annual and unrestricted gifts, sponsored programs, grants, endowment and project income, and e-commerce. These expenditures will be compared with the total expenditures of the University Library. The Director of Financial Services will work with the Associate University Librarian for Philanthropy to monitor this figure each year. The target for FY2002 (the baseline) was 10.0%; thus Target 1 is 10.5% for FY2003, 11.0% for FY2004, etc.

    METRIC F.1.b. LIBRARY EXPENDITURES AS A PROPORTION OF THE UNIVERSITY EXPENDITURES.
    Target1: The University Libraries should continue to account for at least 4.00% of the E&G (educational and general) share of the University expenditures.
    Target2: The University Libraries should account for at least 3.75% of E&G.
    Method: Total University E&G expenditures will be obtained from Institutional Assessment and Studies; Library expenditures will be obtained from Oracle. These two figures will be used to calculate the percent.

    METRIC F.1.c. AMOUNT OF DEVELOPMENT RECEIPTS THAT ARE UNRESTRICTED OR MINIMALLY RESTRICTED.
    Target1: Using FY 2001 as a baseline, increase unrestricted (or minimally restricted) giving by 10% each year.
    Target2: Using FY 2001 as a baseline, increase unrestricted (or minimally restricted) giving by 5% each year.
    Method: Include any funds donated (endowment, annual giving, planned gifts, etc.) for which the library may make the decision for its use and which could be used to replace state funds if needed. The baseline of FY 2000/01 was $232,000, meaning that Target1 for FY05 is $340,000, and Target2 is $282,000.
  2. PROVIDE RESOURCES AND SERVICES THAT HAVE A HIGH RATIO OF VALUE TO COST
    [Guiding Principles: Taking responsibility for realizing our vision through effective stewardship of collections, people, funds, and facilities. Being accountable for performance.]

    METRIC F.2.a. UNIT COST OF ELECTRONIC SERIAL USE
    Target1. No increase in cost per use each year.
    Target2. Less than 5% increase in cost per use each year.
    Method: Collect data annually from vendors for all electronic journals for which usage data are available. UVa.'s annual cost data can be gathered from SIRSI. Divide the total cost by the total uses.
  3. METRIC F.2.b. UNIT COST OF ONLINE DATABASES.
    Target1: No increase in unit cost each year.
    Target2: Less than 5% increase in unit cost each year.
    Method: Divide UVa's annual payment for each database by the annual usage number for that database. Calculate an overall unit cost measurement by dividing the total cost of all databases (with reported usage statistics) by total uses. Usage is defined as either searches of or accesses to the database, whichever figure is available from the vendor.

    METRIC F.2.d. UNIT COST OF INTERLIBRARY LOAN TRANSACTIONS
    Target1. Annual percent change in unit cost will be equal to or less than annual percent change in overall library operating expenditures.
    Target2. Annual change in unit cost will be within 5% of the annual percent change in overall library operating expenditures.
    Method: We will calculate the expenditures and the number of transactions or ILS.

LEARNING/GROWTH PERSPECTIVE: How well is the library positioned to expand our organizational capacity by creating an environment supportive of continuous personal and organizational development?

  1. FOSTER LEARNING AMONG ITS EMPLOYEES TO ENCOURAGE CREATIVITY, COOPERATION, AND INNOVATION
    [Guiding Principles: Cultivating a community where ideas are respected, collaboration is expected, and every individual feels welcome. Building diverse staff, collections, services and physical spaces. Enjoying the opportunity to be as curious, creative, and committed to the intellectual enterprise as the people we serve.]

    METRIC L.1.a. IMPACT OF TRAINING.
    Target1: Positive scores (4 or 5) on training statements from 80% of respondents in the biennial work-life survey.
    Target2: Positive scores (4 or 5) on training statements from 60% of respondents in the biennial work-life survey.
    Method: The University Library will conduct a work-life quality survey every other spring, alternating with the Internal Customer Service Survey. As part of the survey, each staff member will have the opportunity to rate his or her agreement with statements on different aspects of training on a 1 to 5 scale.
  2. RECRUIT, DEVELOP, AND RETAIN PRODUCTIVE, HIGHLY QUALIFIED STAFF
    [Guiding Principles: Building diverse staff, collections, services and physical spaces. Taking responsibility for realizing our vision through effective stewardship of collections, people, funds, and facilities. Being accountable for performance. Celebrating personal and organizational accomplishments.]

    METRIC L.2.a. JOB SATISFACTION AMONG LIBRARY EMPLOYEES.
    Target1: Positive scores (4 or 5) on job satisfaction statements from 80% of respondents in the biennial staff work-life survey.
    Target2: Positive scores (4 or 5) on job satisfaction statements from 60% of respondents in the biennial staff work-life survey.
    Method: The University Library will conduct a work-life quality survey every other spring, alternating with the Internal Customer Service Survey. As part of the survey, each staff member will have the opportunity to rate his or her agreement with employment environment statements on a 1 to 5 scale.

    METRIC L.2.b. RETENTION RATE OF EMPLOYEES.
    Target1: Retain 95% of employees.
    Target2: Retain 90% of employees.
    Method: Staff members who have received a satisfactory, or above, rating over the previous year are identified at the beginning of each fiscal year. At the end of the fiscal year, we will determine what percentage of these employees still work at the library.

    METRIC L.2.c. COMPARE STAFF SALARIES (FACULTY AND CLASSIFIED) TO PEER GROUPS.
    Target1:
    • A faculty salary ranking in the upper 40% of all ARL university libraries, and
    • 60% of classified salaries above the mean salary for similar jobs in the State of Virginia.
    Target2:
    • A faculty salary ranking in the upper 50% of all ARL university libraries, and
    • 50% of classified salaries above the mean salary for similar jobs in the State of Virginia.
    Method: Faculty salary rankings are published annually by ARL. Mean classified salary is calculated using salary information for comparable jobs in the state from the most recent report from the Bureau of Labor Statistics. If Watson Wyeth data become available, the Library will switch to that data source.

    METRIC L.2.d. DIVERSITY OF STAFF.
    Target1: A net increase of at least 4 in faculty/staff diversity, with a net increase of at least 2 in faculty diversity.
    Target2: A net increase of at least 2 in faculty/staff diversity, with a net increase of at least 1 in faculty diversity.
    Method: Using EEO categories for diversity, Library Human Resources will determine the net change in staff diversity from the previous year.
  3. DEVELOP AN INNOVATIVE AND EFFECTIVE INFRASTRUCTURE TO SUPPORT LIBRARY SERVICES AND VARIOUS INITIATIVES
    [Guiding Principles: Providing access to great collections that are easy to find, easy to use, and that inspire learning and scholarship. Providing superb service that meet our users' needs and surpasses their expectations. Enjoying the opportunity to be as curious, creative, and committed to the intellectual enterprise as the people we serve.]

    METRIC L.3.a. EXPENDITURES FOR ELECTRONIC MATERIALS.
    Target1: Rank in the top quarter of ARL libraries in percentage of collections dollars expended on electronic materials.
    Target2: Rank in the top third of ARL libraries in percentage of collections dollars expended on electronic materials.
    Method: MIS will consult ARL Supplementary Statistics for this data.

    METRIC L.3.b. ARL INDEX RANKING.
    Target1: Rank in the top 20 libraries included in the ARL Index.
    Target2: Rank in the top 25 libraries included in the ARL Index.
    Method: The Association of Research Libraries (ARL) includes 114 of the largest academic libraries in North America. Each year ARL uses principal component analysis to aggregate five data categories: volumes held, volumes added (gross), current serials, total library expenditures, total professional and support staff. For each library this analysis produces the "ARL Library Membership Index."

    METRIC L.3.c. FUNDING GENERATED BY RESEARCH AND DEVELOPMENT.
    Target1: Over a 3-year period, grant funding brought in by Digital Library Research & Development will match 100% of the unit's operating costs.
    Target2: Over a 3-year period, grant funding brought in by Digital Library Research & Development will match 80% of the unit's operating costs.
    Method: The annual allotment from outside grant awards for the current year and previous two fiscal years to Digital Library Research & Development will be summed and compared to the operating costs of the department (personnel and OTPS) over the same time period.

    METRIC L.3.d. DEVELOPMENT OF THE CENTRAL REPOSITORY.
    Target1: Increase the quantity of objects made available through the central repository by 15% annually.
    Target2: Increase the quantity of objects made available through the central repository by 10% annually.
    Method: Fedora-generated reports will track the number of objects added to the central repository each year (including new and migrated items). A baseline report will be run at the time of the Fedora implementation (August 2004), and percentages will be based on the baseline count for the first year.

    METRIC L.3.e. IMPLEMENTATION OF LIBRARY PLANNING INITIATIVES.
    Target1: Implement 90% of the initiatives approved through the planning process for the current fiscal year.
    Target2: Implement 80% of the initiatives approved through the planning process for the current fiscal year.
    Method: As part of the annual planning process, the Associate University Librarian for Organizational Development will review progress on the planning initiatives identified for the past year and determine how many have been implemented.

  4. DEVELOP AND MAINTAIN WITHIN THE LIBRARY A CULTURE OF ASSESSMENT.
    [Guiding Principles: Being accountable for performance.]

    METRIC L.4.a. DEVELOP A CULTURE OF ASSESSMENT WITHIN THE LIBRARY.
    Target1: 75% of respondents score 12 or more affirmative responses on the ARL Culture of Assessment IQ.
    Target2: 50% of respondents score 12 of more affirmative responses on the ARL Culture of Assessment IQ.
    Method: MIS will use ARL's Culture of Assessment Institutional Quotient instrument to survey department heads and associate university librarians each spring.


University of Virginia Library
PO Box 400113, Charlottesville, VA 22904-4113
ph: (434) 924-3021, fax: (434) 924-1431, library@virginia.edu

Text Version    |   Libraries   |   Depts./Contacts   |  U.Va. Home   |   ITC

Website Feedback   |   Search   |   Questions? Ask a Librarian   |   Hours   |   Map   |   Policies   |   Jobs

Tracking Opt-out    |   © by the Rector and Visitors of the University of Virginia

Federal Library Depository logo  This library is a Congressionally designated depository for U.S. Government documents. Public access to the Government documents is guaranteed by public law.