An Open Letter on When Metrics Replace Judgment

A public library plans to remove a collection whose borrowing figures are low, until a librarian discovers the people using it rarely take its books home. Told through an imagined collection manager's letter, this story examines what measurements reveal, what they overlook, and why accountable judgment must interpret evidence rather than surrender decisions to a single score.

An Open Letter on When Metrics Replace Judgment printed in dark serif lettering on vintage parchment.

Dear members of the committee who asked me to let the numbers decide,

The first book was not borrowed at all. It lay open on a wooden reading stand, its thick pages held gently beneath a pair of cloth weights. Every Tuesday an older man visited the library, read a few paragraphs from it, and wrote words into a small exercise book. In our circulation report the book was a failure. It had not left the building once in three years. Under the revised collection policy, it was scheduled for removal. The man arrived one Tuesday to find the shelf marked with a yellow sticker, and asked me whether the library had decided that his language was no longer worth keeping.

I was the collection manager at an invented city library. The man, the book, the library, and the committee in this letter are fictional. I do not offer them as evidence of a specific institution’s conduct. I want to describe a temptation I once found reassuring: to believe that a decision becomes fair when it can be made by applying a consistent number. Numbers can reveal patterns our preferences conceal. They can also conceal uses that our counting methods were never designed to recognize.

The library had a real problem. Its shelves were full, new books arrived every month, and the budget could not grow without limit. Keeping every title forever would have left little room for fresh material, including works requested by young readers. We needed a thoughtful way to decide what to repair, replace, store, or remove. Our circulation figures showed which books went home regularly. I helped build the report because my colleagues needed evidence rather than the strength of whoever argued longest in a meeting.

The new policy identified books that had not circulated in several years and placed them on a review list. Most were suitable for reconsideration. Some were obsolete editions, damaged duplicates, or titles for which demand had genuinely disappeared. The system saved time by bringing likely candidates together. I was pleased with the first results. Then, under pressure to meet a storage deadline, we shortened the review. A book with a low borrowing score began to look not merely eligible for consideration but already found wanting.

The man at the reading stand was named Hafiz. He had learned to read his family’s language as a child but had used it little during his working years. His hands were stiff, and the library’s large, stable desk made reading easier than holding the heavy volume at home. He did not borrow the book because he feared damaging it and had no suitable place to store it. He used it every week. The computer registered none of those visits as circulation.

When I asked the desk staff, they told me that others used the collection in similar ways. Students compared script forms. Parents showed children familiar stories. A young translator consulted a historical dictionary and then returned it to its stand. Those were not always long visits, and the staff had no reason to record each one. Our data was accurate about loans and incomplete about use. We had allowed the word circulation to expand silently into the word value.

I thought of the argument that efficient methods can exclude what they fail to count. The reporting system did exactly what it was built to do. Its defect lay in what we demanded from the result. Borrowing data could tell us how often a volume went through the lending process. It could not by itself tell us what happened when the volume stayed inside the library. We were about to replace a judgment with a proxy and call the replacement objectivity.

One trustee said that anecdotal stories should not overturn comprehensive statistics. I agreed that one memorable visitor should not automatically determine the fate of an entire collection. The difficulty was that our supposedly comprehensive statistics did not measure the behavior he represented. A number can be precise and still answer the wrong question. To defend the collection responsibly, I needed more than Hafiz’s moving explanation. I needed to investigate its use, costs, and alternatives.

We arranged a short observation period with appropriate privacy safeguards. Staff recorded aggregate use of the reading stands without identifying visitors or tracking their reading habits. We invited voluntary comments about what the collection made possible. We examined whether some books could be held in a smaller reference area and whether neglected titles genuinely belonged elsewhere. I was careful not to invent popularity where it did not exist. Some volumes remained almost untouched even when in-room use was counted.

The results were mixed. The old dictionary received regular attention; several duplicate volumes did not. A collection of rare local pamphlets was consulted infrequently but was difficult to replace. Other books could be transferred to a partner library with better preservation facilities. We learned that we needed several kinds of judgment, not one heroic exception. The revised review was more demanding than deleting everything below a threshold. It was also a more faithful response to the library’s actual purpose.

A junior analyst asked whether our metrics were now useless. I told her no. They were more useful because we had stopped asking them to do work beyond their competence. A good measurement should sharpen a question, reveal surprises, and invite investigation. It should not prevent the person accountable for a public decision from explaining what else matters. We could still use circulation to identify candidates. We simply could not use it as the whole verdict.

The distinction became clearer when we looked at a different part of the building. Children’s picture books circulated constantly, and their high use justified replacement copies. A local history reference atlas rarely left its desk because borrowing was restricted to protect fragile pages. An accessibility guide was consulted by a smaller group of people whose needs were not represented by volume alone. Comparing all three through one score would have made a useful number behave like an authority it had never earned.

I remembered the danger of treating automated classifications as independent judgments. Our policy was not especially complex. It used a threshold chosen in a meeting by librarians and trustees. That simplicity did not make it free of values. We had decided to reward loans because loans were easy to count. We had decided the consequences of a low count would be removal because shelf space was scarce. The computer had merely carried those decisions forward.

We held a meeting with readers and staff. Some wanted the collection preserved exactly as it was. Others argued that unused books occupied space that could become a children’s study area. Both concerns deserved attention. It would have been unfair to treat Hafiz as an emblem that ended every discussion. He himself suggested moving the reading stand to a quieter corner if that would make room for new shelves. He cared about being able to use the book, not about controlling the entire library.

A trustee asked how we would prevent favoritism if professional judgment returned to the process. That was a serious challenge. A librarian’s preferences can be narrow, and a moving story can distort priorities. We proposed written review criteria, records of decisions, and an opportunity for staff and users to point out relevant uses that the initial data missed. The committee would still make difficult choices. It would explain them rather than pretending the formula had relieved it of responsibility.

Our first revised list removed several damaged duplicates and retained a smaller language reference area. The historical dictionary remained available on its reading stand. We improved signs so people knew that in-room consultation was welcome, and we recorded aggregate reference use without turning ordinary reading into surveillance. We set a date to review the arrangement again. The decision could change with new evidence, but the next committee would know why it had been made.

The experience changed my relationship with the dashboard. I still examine it each month. I watch for unexpected declines and rising requests. I no longer describe the numbers as speaking for themselves. Data never walks into a meeting and declares what a library owes its community. People choose what to measure, how to interpret changes, and which harms they are willing to accept. Measurement can discipline those choices. It cannot make them unnecessary.

This is not an argument for following intuition without evidence. I have seen cherished collections remain untouched for years because staff were unwilling to question their own attachments. I have seen resources allocated to the loudest visitors while others remained unknown. A useful measure can challenge both habits. Judgment becomes stronger when it is required to face unwelcome facts. It becomes weaker when it treats the limits of a dataset as the limits of reality.

In the discussion of governance by default, I recognize what happened in our library. A threshold introduced as an aid to review quietly became the decision itself. Nobody had formally voted that borrowing was the only form of value. But the routine began operating as though that question had been settled. The more often we followed the report automatically, the less natural it felt to ask what the report omitted.

Hafiz came to the library on the Tuesday after the committee’s decision. He found the book on a freshly repaired stand. He did not make a speech. He placed his exercise book beside it and began copying a paragraph. Later he asked whether the other readers had lost anything so that his book could remain. I explained which duplicate volumes had been moved and how the space had been rearranged. He nodded, satisfied not by a promise that his needs would always win, but by the evidence that they had genuinely been considered.

I sometimes think about the yellow sticker that started the argument. It was designed to make our work easier. A bright mark helped staff find titles for review without searching a long spreadsheet. There was nothing cruel about the paper or its color. The cruelty would have begun if we treated the sticker as sufficient explanation to someone who had relied on the book. A useful procedure becomes unjust when it silences the questions that should accompany its results.

So to every committee that governs through scores, I ask you to preserve two habits. First, define precisely what a number measures and what it does not. Second, make the person using that number answerable for the consequences. If a decision seems unfair, the reply should not end with “the data said so.” Show the reasons, the competing needs, and the evidence considered. Then be willing to correct what the original measurement could not see.

The library still has a crowded shelf and a limited budget. Neither problem disappeared. We made a difficult allocation, and there will be others. But I can now explain why some low-circulation books stay and some leave without hiding behind a metric or asking readers to accept my preference as law. That explanation is slower than a threshold. It is also the work I was hired to do.

With respect for the numbers that help us see, and the judgment that must remain responsible for what we do,

A librarian who learned that counting is not the same as deciding

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Scroll to Top