Go to

 

 
Go to Balanced Scorecard home

2006-07 Balanced Scorecard Metrics

The Balanced Scorecard is a management tool designed to provide a view of an organization from four perspectives: user, internal processes, financial, and learning/growth potential. The user perspective measures service or satisfaction with service. Internal processes looks at workflow efficiency; finance, at cost effectiveness. Learning/growth tells us how well prepared we are to meet new challenges and initiate new programs. This document explains what the University of Virginia Library wants to know in each of these four areas and the specific metrics we are using to inform us about each perspective.

Each of the metrics described below contains two targets. Target1 indicates complete success in achieving the metric; Target2 indicates partial success. Management Information Services will compile and report the results of all metrics annually.


USER PERSPECTIVE: How well is the library meeting the needs of our users?

  1. Provide excellent service to users of the University of Virginia Library

    Metric U.1.a.
    Overall rating in student and faculty surveys.
    Target1:
    At least 4.25 out of 5.0 from each of the major user groups: undergraduate students, graduate students, humanities faculty, social science faculty, and science faculty.
    Target2:
    A score of at least 4.0 from each of the major user groups.
    Method:
    The University Library regularly conducts user surveys of faculty and students. A final question in each survey asks the respondent to “rate your overall satisfaction…” with the Library. Data for this metric are compiled from responses to the latest faculty and student surveys on record.

    Metric U.1.b.
    Customer service rating in student and faculty surveys.
    Target1:
    At least 4.25 out of 5.0i from each of the user groups: undergraduate students, graduate students, humanities faculty, social science faculty, and science faculty.
    Target2:
    A score of at least 4.00 from each of the major user groups.
    Method:
    As part of its series of user surveys, the University Library asks respondents to rate three aspects of customer service: speed, courtesy, competence. The three scores will be combined into one composite number. Data for this metric will be compiled from responses to the latest faculty and student surveys on record.

  2. Educate users in the skills necessary to fulfill their information needs

    Metric U.2.a.
    Overall instruction rating in user education participant surveys.
    Target1:
    An overall rating of at least 4.0 out of 5.0i.
    Target2:
    An overall rating of at least 3.80.
    Method:
    Evaluation forms will be distributed to all participants of course-related user education classes at the end of each instructional session. All evaluation forms will include an identically worded question rating the overall instruction of that sessioni. Scores for this question will be entered into a database. The annual overall instruction rating will be calculated using the average score of each instruction session held that year.

    Metric U.2.b. Satisfaction rating with Library instruction in student and faculty surveys.
    Target1:
    A score of at least 4.00 out of 5.00i from each of the major constituencies: undergraduate students, graduate students, humanities faculty, social science faculty, science faculty.
    Target2:
    A score of at least 3.50 from each of the major constituencies.
    Method:
    As part of its series of surveys, the University Library asks respondents to rate their satisfaction with various forms of instruction in the use of Library resources and services. Scores for each form of instruction will be combined to generate an average score for instruction. Data for this metric will be compiled from responses to the latest faculty and student surveys on record.

  3. Develop high quality collections that reflect the needs of the Library’s users and support the University’s mission

    Metric U.3.a.
    Circulation of new monographs.
    Target1:
    60% of all new monographs (excluding reference books and other non-circulating materials) circulate within two years of the time of cataloging.
    Target2:
    50% of all new titles circulate within the first two years after cataloging.
    Method:
    A program will extract data from the SIRSI records documenting circulation of print monographs over a two-year cycle. Only items circulated to users (not binding, etc.) will be counted.

    Metric U.3.b.
    Use of Special Collections materials.
    Target1:
    Increase by 10% annually the number of materials used or circulated in Special Collections.
    Target2:
    Increase by 5% the number of materials used or circulated.
    Method:
    Special Collections will perform a SIRSI check-out or item use for materials used from the collections for any purpose (patron use, classes, reference, show & tell, on hold, etc). A SIRSI report will be run at the end of the year to count the uses and charges. Data for archives is collected manually and will be added to the SIRSI data.

  4. Provide convenient and timely access to the Library’s collections

    Metric U.4.a.
    Turnaround time for searches, LEO, and Instructional Scanning (local resources).
    Target1:
    Satisfy the turnaround targets 90% of the time.
    Target2:
    Satisfy the turnaround targets 80% of the time.
    Method:
    Searches: Search for requested item within 1 day and notify patron of results within 4 days of request. Staff will tally data as searches are performed, including: the number of searches requested, the number of requests processed, how quickly the search was performed, and how quickly the patron was notified. This data will be reported to Management Information Services each month.

    LEO:
    Fill requests in 7 days. Turnaround time from patron request to delivery will be measured by ILS.
    Instructional Scanning:
    Fill requests by agreed upon date. Instructional Scanning Services will provide a report in their database to track how quickly scanning requests are filled.

    Metric U.4.b.
    Turnaround time for new book and ILL requests (external resources).
    Target1:
    Satisfy the turnaround targets 75% of the time.
    Target2:
    Satisfy the turnaround targets 50% of the time.
    Method:
    New Books: Fill requests in 7 days. A sample of in-print US titles requested by University-affiliated patrons will be drawn from the request database by Management Information Services. Turnaround time from patron request to availability will be measured.

    ILL
    : Fill requests in 7 days. Turnaround time will be measured by ILS from patron request to notification that the book or article is available.

    Metric U.4.c.
    Reduce the number of recalls placed each year.
    Target1:
    Reduce the number of recalls by 5%.
    Target2:
    Reduce the number of recalls by 0%.
    Method:
    Using data from the library’s annual statistical report, compare the number of recalls placed in the current year to the number placed in the previous year.


INTERNAL PROCESS PERSPECTIVE: How do the library’s internal processes function to efficiently deliver library collections and services?

  1. Acquire, create, organize, preserve and deliver information resources in a timely, efficient, and accurate manner

    Metric I.1.a.
    Processing time for routine acquisitions.
    Target1:
    Process 90% of in-print books from North America within one month.
    Target2
    : Process 80% of in-print books from North America within one month.
    Method:
    This metric will estimate the elapsed time from ordering to delivery at the destination library. Certain classes of materials will be excluded: user requests, non-book materials, out of print books, foreign acquisitions, and orders for Special Collections, Law, and Health Sciences. The methodology will rely on SIRSI records for Yankee and New York Times/Washington Post approval orders.

    Metric I.1.b.
    Turnaround time and accuracy for reshelving/refiling.
    Target1:
    Accurately shelve/refile 85% of all materials by the end of the day following discharge regardless of where the item was discharged.
    Target2:
    Accurately shelve/refile 75% of all materials by the end of the day following discharge regardless of where the item was discharged.
    Method:
    On a random hundred dates, a sample of 10 circulating books, bound journals, and documents from all libraries will be selected from the uses and discharges of the previous day for checking in the stacks. Each sample’s results will be weighted according to the overall number of circulations for that day. The overall annual percentage is calculated by dividing the number of items meeting the criteria by the total sample size.

  2. Operate in the most innovative, efficient, and effective way possible

    Metric I.2.a. Internal communications rating.
    Target1: Positive scores (4 or 5) on 80% of responses to internal communications statements in the biennial staff work-life survey that relate to: communication from administration, communication between departments, and communication within departments.
    Target2: Positive scores (4 or 5) on 60% of responses to internal communications statements in the biennial staff work-life survey that relate to: communication from administration, communication between departments, and communication within departments.
    Method: The University Library will conduct a work-life quality survey every other spring, alternating with the Internal Customer Service Survey. As part of the survey, each staff member will have the opportunity to rate his or her agreement with statements on aspects of internal communication on a 1 to 5 scaleii. The responses from the communications section will be tallied in aggregate.

    Metric I.2.b. Conduct regular usability testing of library websites. (Suspended for this year to allow time for development of new metric focusing on more comprehensive and results-based usability testing)

  3. Continuously identify, review, and improve the processes that have the greatest impact

    Metric I.3.a.
    Process improvement. (Suspended for this year to determine better mechanism for identifying process to be improved and gauging success.

    Metric I.3.b.
    Staff survey rating of internal service activities.
    Target1
    : A composite rating of at least 4.00 in the annual internal services survey, with no unit rated below 3.50.
    Target2:
    A composite rating of at least 3.50, with no unit rated below 3.00.
    Method:
    The University Library will conduct an internal services survey every other spring, alternating with the worklife survey. As part of the survey, each staff member will have the opportunity to rate the performance of each service unit or activity (e.g., Administration, Acquisitions, Cataloging, Education Library, Finance, Facilities, Geostat, Interlibrary Services, Library Human Resources, Library Information Technology Services, Management Information Services, Special Collections) on a 1 to 5 scaleii. The composite rating will be based on the overall rating given to each unit.

    Metric I.3.c.
    Develop a culture of assessment within the library.
    Target1:
    90% of all responses on the ARL Culture of Assessment IQ are positive.
    Target2:
    75% of all responses on the ARL Culture of Assessment IQ are positive.
    Method:
    MIS will use ARL’s Culture of Assessment Institutional Quotient instrument to survey department heads and associate university librarians.

  4. Provide facilities that promote staff productivity, encourage library use and ensure top quality services

    Metric I.4.a.
    Response to equipment service requests.

    Metric I.4.a.1.
    Resolution of reported computing problems.
    Target1:
    90% of LSP requests are resolved within their target time.*
    Target2:
    80% of LSP requests are resolved within their target time.*
    Method:
    LITS will use the LibHelp Incident Database to track completion time for reported computing problems. *Target time for repair/fix requests is 48 hours; target time for update/install requests is 7 days. The metric statistic will reflect the proportion of all requests that are resolved within their targeted timeframe.

    Metric I.4.a.2.
    Resolution of reported problems with equipment provided by Printing and Copying Services (PCS).
    Target1:
    Average resolution time for printer/copier problems is within 1 business day.**
    Target2:
    Average resolution time for printer/copier problems is within 2 business days.**
    Method:
    PCS will provide MIS with information on the amount of time it takes to resolve problems for all printer/copier machines in the libraries. **Business days are defined as Monday-Friday 8-5.

    Metric I.4.b. Renovation of major staff and public areas.
    Target1: Spending a minimum of $100,000 in the current fiscal year, renovate or plan for renovation of at least one staff area and one public area.
    Target2:
    Spending a minimum of $75,000 in the current fiscal year, renovate or plan for renovation of at least one staff area or one public area.
    Method: The Director of Facilities will solicit suggestions for renovations at the same time as budget requests are made, and report on their completion at the end of the fiscal year. Renovations are defined as distinct projects costing at least $25,000 that improve service or increase staff satisfaction and productivity. Administrative team will decide renovation priorities.

    Metric I.4.c.
    Reliability of servers.
    Target1:
    Key production servers will be up at least 99.9% of the time.
    Target2:
    Key production servers will be up at least 99.0% of the time.
    Method:
    This metric will be based on information gathered from the Nagios telemetry tool. A server will be considered “up” if it can be located by the tool (“pinged”) and account access is available. This will measure whether the server is available, but not other factors affecting access such as software crashes or broken links. Key production servers are those servers with primary functions where even a short-term downtime would hamper the library’s ability to carry out its mission. Key servers in 2006-07 include scripta, virgo, viva, etext, fisher, orbis, (telemetry system).


FINANCE PERSPECTIVE: How well are the library’s finances managed to achieve our mission?

  1. Increase the financial base through private donations, other external support, and increased institutional support

    Metric F.1.a.
    Library expenditures as a proportion of university expenditures.
    Target1:
    The University Library will account for at least 2.50% of the total expenditures of the academic division of the University.
    Target2:
    The University Library will account for at least 2.25% of the total expenditures of the academic division of the University.
    Method:
    Total expenditures of the academic division of the University will be obtained from Institutional Assessment and Studies (IAAS); University Library expenditures will be obtained from Oracle. These two figures will be used to calculate the percent. When Balanced Scorecard results are released in the fall, the proportion may reflect expenditures from previous fiscal years and will be updated once current fiscal year data become available from IAAS. Note: The University now uses the GASB 34/35 accounting system.

    Metric F.1.b.
    Amount of development receipts that are unrestricted or minimally restricted.
    Target1:
    Using FY 2001 as a baseline, increase unrestricted (or minimally restricted) giving by 10% each year.
    Target2:
    Using FY 2001 as a baseline, increase unrestricted (or minimally restricted) giving by 5% each year.
    Method:
    Include any funds donated (endowment, annual giving, planned gifts, etc.) for which the library may make the decision for its use and which could be used to replace state funds if needed. The baseline of FY 2000/01 was $232,000, meaning that Target1 for FY07 is $411,000, and Target2 is $311,000.

  2. Provide resources and services that have a high ratio of value to cost

    Metric F.2.a.
    Unit cost of electronic serials.
    Target1
    : No increase in cost per use each year.
    Target2
    : Less than 5% increase in cost per use each year.
    Method:
    Collect data annually from vendors for all electronic journals for which usage data are available. UVa’s annual cost data can be gathered from SIRSI. Divide the total cost by the total uses.

    Metric F.2.b.
    Unit cost of Interlibrary Loan transactions.
    Target1
    : Annual percent change in unit cost will be equal to or less than annual percent change in overall library operating expenditures.
    Target2
    : Annual change in unit cost will be within 5% of the annual percent change in overall library operating expenditures.
    Method:
    We will calculate the expenditures and the number of transactions for ILS.

  3. Maintain sufficient levels of investment to remain a top-flight academic library

    Metric F.3.a.
    ARL index ranking.
    Target1:
    Rank in the top 20 libraries included in the ARL Index.
    Target2:
    Rank in the top 25 libraries included in the ARL Index.
    Method:
    Each year the Association of Research Libraries (ARL) uses principal component analysis to aggregate five data categories: volumes held, volumes added (gross), current serials, total library expenditures, total professional and support staff. For each library this analysis produces the “ARL Library Membership Index.”


LEARNING/GROWTH PERSPECTIVE: How well is the library positioned to expand our organizational capacity by creating an environment supportive of continuous personal and organizational development?

  1. Foster learning among its employees to encourage creativity, cooperation, and innovation

    Metric L.1.a.
    Impact of Staff Development.
    Target1:
    Positive scores (4 or 5) on 80% of responses to staff development statements in the biennial work-life survey.
    Target2:
    Positive scores (4 or 5) on 60% of responses to staff development statements in the biennial work-life survey.
    Method:
    The University Library will conduct a worklife quality survey every other spring, alternating with the Internal Customer Service Survey. As part of the survey, each staff member will have the opportunity to rate his or her agreement with statements on different aspects of staff development and training on a 1 to 5 scaleii. The responses from staff development section will be tallied in aggregate.

    Metric L.1.b.
    Value of Library-provided training.
    Target1:
    An overall rating of at least 4.0 out of 5.0i on training evaluations.
    Target2:
    An overall rating of at least 3.5 out of 5.0i on training evaluations.
    Method:
    Evaluation forms will be distributed to all participants of library-provided training sessions. All evaluation forms will include an identically worded question rating the overall instruction of that session on a scale of 1 to 5i. Management Information Services will calculate the annual overall instruction rating using the average score of each instruction session held that year.

  2. Recruit, develop, and retain productive, highly qualified staff

    Metric L.2.a.
    Job satisfaction among library employees.
    Target1:
    Positive scores (4 or 5) on 80% of responses to job satisfaction statements.
    Target2:
    Positive scores (4 or 5) on 60% of responses to job satisfaction statements.
    Method:
    The University Library will conduct a work-life quality survey every other spring, alternating with the Internal Customer Service Survey. As part of the survey, each staff member will have the opportunity to rate his or her agreement with job satisfaction statements.

    Metric L.2.b.
    Retention rate of employees.
    Target1:
    Retain 95% of employees.
    Target2:
    Retain 90% of employees.
    Method:
    Staff members who have received a satisfactory, or above, rating over the previous year are identified at the beginning of each fiscal year. At the end of the fiscal year, we will determine what percentage of these employees still work at the library.

    Metric L.2.c.
    Compare staff salaries to peer groups.
    Target1:

    • A faculty salary ranking in the upper 40% of all ARL university libraries, and
    • 60% of classified salaries above the mean salary for similar jobs in the State of Virginia.
    Target2:
    • A faculty salary ranking in the upper 50% of all ARL university libraries, and
    • 50% of classified salaries above the mean salary for similar jobs in the State of Virginia.
    Method: Faculty salary rankings are published annually by ARL. Mean classified salary is calculated using salary information from University and Library Human Resources. NOTE: availability of classified salary information varies from year to year. In years when no classified salary data are available, the metric will be based on faculty salary comparisons only and appropriately noted.

    Metric L.2.d.
    Diversity of staff
    This metric will be redeveloped after a review of University policies and targets for faculty, staff, and student diversity. Our intent is to develop a new methodology and targets in time to calculate the metric for FY2006-07 results.

    Metric L.2.e. Attitudes toward Diversity.
    Target1:
    Positive scores (4 or 5) on 80% of responses to diversity statements in the biennial work-life survey.
    Target2:
    Positive scores (4 or 5) on 60% of responses to diversity statements in the biennial work-life survey.
    Method:
    The University Library conducts a worklife quality survey every other spring, alternating with the Internal Customer Service Survey. As part of the survey, each staff member is asked to rate his or her agreement with statements on different aspects of workplace diversity on a 1 to 5 scale.ii The responses from the diversity section will be tallied in aggregate.

  3. Develop an innovative and effective infrastructure to support library services and various initiatives

    Metric L.3.a.
    Expenditures for electronic materials.
    Target1:
    Rank in the top quarter of ARL libraries in percentage of collections dollars expended on electronic materials.
    Target2:
    Rank in the top third of all ARL libraries in percentage of collections dollars expended on electronic materials.
    Method:
    MIS will consult ARL Supplementary Statistics for this data.

    Metric L.3.b. Funding generated by research and development.
    Target1: Over a 3-year period, external funding brought in by Digital Library Research & Development (DLR&D) will match the institutional funding provided for the unit.
    Target2:
    Over a 3-year period, external funding brought in by DLR&D will equal 90% of the institutional funding provided for the unit.
    Method: The annual allotment from external funding for the current year and previous two fiscal years brought in by DLR&D will be summed and compared to the institutional investment in the department (state-covered personnel and OTPS) over the same time period.

    Metric L.3.c.
    Development of the central repository.
    Target1: Increase the quantity of objects in the central repository by 15% annually.
    Target2: Increase the quantity of objects in the central repository by 10% annually.
    Method: Fedora-generated reports will track the number of objects added to the central repository each year (including new and migrated items). A baseline report will be run at the time of the Fedora implementation (August 2004), and percentages will be based on the baseline count for the first year.



University of Virginia Library
PO Box 400113, Charlottesville, VA 22904-4113
ph: (434) 924-3021, fax: (434) 924-1431, library@virginia.edu

Text Version    |   Libraries   |   Depts./Contacts   |  U.Va. Home   |   ITC

Website Feedback   |   Search   |   Questions? Ask a Librarian   |   Hours   |   Map   |   Policies   |   Jobs

Tracking Opt-out    |   © by the Rector and Visitors of the University of Virginia

Federal Library Depository logo  This library is a Congressionally designated depository for U.S. Government documents. Public access to the Government documents is guaranteed by public law.