Kyle Banerjee's Blog

Despite the fact that people like libraries and say nice things about them, we’ve had our butts handed to us on a platter over the past few years.

We position ourselves as information professionals. However, according to a report from the Pew Internet and American Life Project, people turn to the Internet, professionals (doctors, financial experts, etc.), friends, family, colleagues, newspapers, magazines, government agencies, TV, and radio before they ask for help from librarians.

The reason people like libraries but don’t really use the information we provide is simple. We recognized sometime back that the morgue-like library atmosphere of yesteryear just wouldn’t cut it. To address this problem, we devoted increasingly large portions of libraries to social spaces. Coffee shops, snack bars, and other amenities became the norm rather than exception. Patrons like what we did.

Unfortunately, we haven’t yet done the same for information services. We may provide access to online resources, but we’re missing the point. People want their information in a social context. That’s why they consult their colleagues, family, and friends. That’s why they go to bulletin boards and a million social networks like Facebook when they need help. That’s why they like online services that know who they are and what interests them.

We won’t provide what patrons need, but we actively encourage them to turn elsewhere.  We tell them they should subscribe to Meebo — a service that keeps track of passwords and conversations across all kinds of systems. We meet with users in social networks that track who our patrons associate with as well as what they say. The web sites we use and recommend to track everything patrons do, down to their physical location. We use insecure email for all kinds conversations that include private data.

At the same time that everyone else not only expects but demands we share information, we go backwards. We pretend we protect patron privacy when we impose the most ridiculous barriers to transmitting any kind of information that would be useful to system integration. Meanwhile, the rest of the world moves forward.

Like most other people, I think it’s a good idea to be nice to other people. I also like it when people are nice to me (which is practically always). However, there are times when kindness strikes fear into my heart.

Last week, I heard the words I dread most — “How ya doin’? Everything OK?” This may sound like friendly banter, but experience has taught me that people say this only under three circumstances:

1. When they’re checking in on you
2. When you’re styling
3. When you’re falling apart

If one of first two possibilities don’t clearly apply, most likely the third does, and you just don’t realize it yet.  I heard these words just minutes before I succumbed to heat exhaustion on the Ultimate Road Ride last year (I thought I was fine until I almost blacked out on the bike). This year, I heard them again towards the end of the Shasta. This time, I was smart enough to realize people were telling me something.

So when I heard this question in a work context, I was worried. Under normal circumstances, I tell people I’m happy if I fail only 90% of the time. If you look at the great advances made throughout history, it’s pretty clear that progress is what you get when you screw up so many times that you finally learn something useful.

However, sometimes the stakes are high, and the success through iterative failure model can’t be used. I have a project at work that’s like that right now (even if some specific components of it might be improved best by rapid, iterative experimentation). So when more than one person asked me if I was OK in the same week — I had no reason to believe they were checking in on me because I work with them all the time — I took it as a sign of crappy work at best and an early sign of meltdown at worst.

My prescription was to pound out a threshold ride (held the ol’ HR at 170+ for a solid hour maxing out at 184 ), knock back a few beers, and get my head screwed back on. This past week has been much better. No one has been asking except to check in. Good…

I try to keep some contact with the real world, but every now and then something happens which makes me realize how insulated my environment is. Last night was one of those times.

Shirley and I went to an authentic Russian restaurant where we decided to do everything Russian style. At the next table, 4 business travelers watched flabbergasted as we washed down an excellent meal with 10 shots of vodka. This may sound like a lot, but it is very tame by Russian standards.

The travelers asked what we did for work, and when we mentioned that we were librarians, they were in shock. They made it clear that we looked like no librarians they’d ever seen, and it never occured to them that a librarian would know anything about vodka, let alone drink the stuff. They decided that they liked us and had a new opinion of the profession on that basis alone. They invited us to knock down a few shots after dinner. We accepted and had a great night with them.

I’m glad we helped our new friends see librarians in a more positive light, but I find it disconcerning that this change of heart was based on our sense of fun and an ability to put away more high octane hooch than they could. While I find it amusing that they assumed I was trying to improve the Dewey Decimal system when I mentioned that I worked with library systems, that is also a dead giveaway that they didn’t realize we actually provide a useful service.

As a profession, we celebrate the fact that we are viewed as meek, socially inept dorks by the rest of the world. We cultivate the image of the spinster with a bun in her hair. If we want to show how hip we are, we shave off a few pounds and a few years. Then we add a bit of makeup, an updated hairstyle, and a higher energy level. It’s the same idea with a little sex appeal.

Why do we play to such stereotypes? I suppose it makes people feel warm and fuzzy, but so does the image of the milkman who lost his job a long time ago. The internet has changed the way people interact with information, and we must make it clear we’ve adapted our services accordingly unless we want to relegate ourselves to irrelevance.

We can get you any book or article held by just about any library regardless of where you live whether or not it’s in digital form. We can get you electronic articles from well respected journals that would cost you a fortune to download (assuming you could find them at all). No other outfit, including Google and Amazon, can do this. I can’t help but think that if we want long term success and continued funding, we’re much better off encouraging people to focus on what we do rather than on anachronistic images.

For the past few days, I’ve been at the Innovative Users Group annual meeting. At the IUG meeting, roughly 1600 dweebs from around the world discuss library systems, particularly those produced by Innovative Interfaces Inc (III).

I’ve been a fixture at these meetings for a number of years, but many people acted surprised to see me at this one. I’m coordinating the migration of a resource sharing system used by 35 academic libraries in Oregon and Washington away from a III product to something else. They assumed my work would put me in an awkward position.

My project is not helping relations with the company. However, the situation was largely unavoidable. We have had a long and productive association with III, but their needs have evolved with time as have ours. After intense negotiations failed to result in a mutually acceptable contract, our board of directors unanimously decided that the strategic direction we were headed required a resource sharing system based on new architecture.

Right now, the library world is in a period of total upheaval. Just a few years ago, the whole idea of a library was to provide a controlled, centralized environment to centrally acquire, process, and preserve materials that people needed. More and more, the most important function libraries perform is helping people find and get electronic and physical materials that are owned and maintained by others. While libraries will continue to buy books and journals, these materials will play a shrinking role over time.

The reason we’re changing our resource sharing system right now is because of differences in opinion as to what library services will look like in view of a the new way people want to use libraries and information. No one knows what will work best in the end, so a little disagreement is healthy.

Our relationship with III is entering a new phase, but we continue to have many common interests. The member libraries in our consortium rely on III products for mission critical tasks. Even if we were unable to reach an agreement for a new contract, we are still interested in solving the same problems and can learn from each other.

Just as I still regard workmates from former workplaces as colleagues, I think the same about the many III staff I’ve worked with over the years. That is why I was glad I was able to attend the meeting and share ideas with others as I always have.

Of all the conferences I’ve attended over the years, my favorite by far is code4lib. It’s like no other. The level of passion and sense of community is far greater than you’ll see elsewhere. The spirit of invention, willingness to work hard, and desire to help others reminds me where the bar for excellence really is. Plus it’s just plain fun.

Organizers for other events should be taking notes. Long talks at code4lib last 20 minutes. Short talks last 5 minutes. This means that you get exposed to great ideas fast and if a presentation is a dud, you’re quickly in the clear. Presenters assume that if you’re interested in something, you’ll follow up on your own so spoonfeeding is unnecessary. As a result, you don’t lose time repeatedly reviewing the basics.

Barely over 200 people went this year, but it seems like every time there’s something interesting going on, people from code4lib are involved. Just since last week, I’ve met with totally separate groups to plan for LOCKSS throughout our consortium, discuss uploading our digital collections metadata to the Talis platform for experimental purposes, and discuss specifications that will improve interoperability between library systems with vendors and the DLF ILS task force. Participation from code4lib participants was disproportionate at all of these venues even though the conference has no actual connection.

In other words, if you’re into library technology, code4lib is really a great place to learn, meet interesting people, and get involved with some pretty cool projects.

One of the things that’s always made me nuts is how quick people can be to thwart others from doing anything new based on little more than vague fears.

Recently, a school asked a university library where a colleague works if they could download catalog records. The school library was hoping to avoid manually keying in all this info themselves since that is time consuming (i.e. expensive).

The university library said no. Lame excuses were given as to why fulfilling the request was impossible, but I suspect that the real reasons were related to emotional discomfort rather than an actual problem. The fact of the matter is that the university runs a publicly accessible Z39.50 server (and intends to continue doing so), it has the legal right to distribute their records even before you consider the fact that they were created at public expense, the system impact would be virtually nonexistent, and it would help the school. As a practical matter, the school didn’t need permission in first place and asking was really a courtesy.

Only library geeks even know what Z39.50 is, let alone know how to use it, so I’m scratching my head as to why an organization would host such a server if they don’t want people using it. It’s like buying a subscription to satellite TV and blocking out all the channels.

If we are going to wring our hands as we try to figure out how we will adapt our services for the future, we could start by not tossing up roadblocks unless there is a compelling reason. I don’t know how many projects I’ve seen sunk by nothing more substantial vague discomfort on the part of one or two individuals in gatekeeper positions. It seems like I’m constantly hearing people say something to the effect of “I’ll never try that again” because they were sent to bureaucratic and meeting hell for asking about doing things a different way.

This example might sound petty, but it’s stuff like this that really holds us back. I recently had a conversation with someone who expressed surprise at how easy it is for me and a few others to ask for favors. There’s no secret to it. Just look for excuses to help people rather than excuses for why you shouldn’t. If you make life easy and enjoyable for others, they like dealing with you and cut you slack when you need it. Otherwise, don’t be surprised if people treat you like you have a communicable disease whenever you come near…

I’m not sure how this wasn’t on my radar, but today I learned about a pilot project where the Library of Congress is putting photos on Flickr. For those of you who aren’t librarians, this might not sound like a big deal — even kids mount lots of photos on Flickr and the combined size of the high profile collections LC has added is smaller than many peoples’ personal photo collections.

However, LC’s willingness to work on Flickr represents a quantum leap forward because a key player in the library world is realizing that the ticket to success is to use the Internet like everyone else. Historically, librarians have collected, organized, and physically protected information. While this is a great strategy for physical materials, it’s not good model for resources that require massive technical, staff, and financial resources that we don’t have.

Just as practically all decent sized libraries depend on vendors for collection development, acquisitions, cataloging, and processing when physical resources are concerned, there is nothing wrong with doing the same for electronic resources. Besides, processing photos with Flickr is a heck of a lot easier than it is with CONTENTdm. Oh yeah, you also don’t need to drop $50K to get started and keep paying many thousands each year in maintenance fees.

I have heard fears expressed that this amounts to a loss of control over the information. I think that quite the opposite is the case. So long the information can be extracted (Flickr has a pretty decent API), it could be even safer than we could make it.

Although, if some joker changes the password for the account and gets flattened by a bus, there could be some real headaches. But everyone knows that the best way to keep electronic information safe is to maintain more than one copy.

Recently, I learned that Google is expanding its digitization program to include newspapers. Apparently they’re contacting libraries and content providers to see what resources and partnership opportunities are available. You’d think this would be a good thing, but a colleague told me that it created a squawk where she works. Upon learning what Google was up to, a number of people at her library started discussing how to stop Google by throwing up intellectual property barriers and other means.

This reaction is sad but all too common in the library community. Although we talk a good game for open access, many libraries adopt a very different stance when it comes to giving away useful things that they create. The vast majority of catalog records are created at public expense, yet these are copyrighted and cannot be legally used without paying substantial fees. Critical library tools are designed by librarians who work at public expense, but then are copyrighted and sold by the American Library Association or OCLC (a cooperative of libraries). We tell everyone else to mount their content on open web servers, but very few of our publications are distributed this way. When we set up digital archives consisting of local resources, we often put in access restrictions, even for low grade materials like student papers.

The thing that’s particularly vexing about the Google example at hand is that practically every library project has the explicit goal of being included in Google, and that lots of money and endless meetings are dedicated to this objective. I suspect my colleague’s library fears that successful efforts by Google may cause the grant fed cash cows grazing on library digitization turf to wander off.

Just as it is bad for the sport when an athlete intentionally trips the opponent he is unable to beat in a fair race, undermining others is a disservice to our users and our profession. The fact of the matter is that Google will do a much better job of digitizing and providing access than any library can — we simply do not have the resources and expertise to compete at this level.

Besides, institutions with a lot more resources than us have been unable to stop Google. Google has been sued for their indexing, how they crawl, advertise, and digitize — the publishers weren’t too happy when they started Google Books. We need to come to terms with reality, and that is that Google will continue to digitize things. That is probably good for most people. We should welcome anything that improves things for users, even if it forces us to learn new things and take our services in new directions.

Recently, Dorothea Salo delivered a blistering critique of library-created institutional repositories. Her basic point is that our IR service model is driven by a failed ideology, we ignore peoples’ real needs, our tools are hopeless, and we don’t deal with funding or other practical considerations in a realistic way. In addition, she charges us librarians with not eating our own dog food since we do not do what we try to convince others to do. In two words — we suck.

What has been interesting is the response to Dorothea’s comments. A lot of people are glad she’s speaking up.

I believe in emphasizing the positive, but I’m glad to see our profession might finally be getting ready for an open discussion on how we can move things forward. We do many good things, but our profession has a disturbing habit of pretending we are dizzy with success when we are reeling from spectacular failures.

The “how we done it good” articles and presentations that permeate our professional communications are not harmless. I once let a journal editor talk me into putting a positive spin on a method I used in one of my own projects and originally reported to be unworkable. A couple years later, I learned the article was required reading in a library school class — a student contacted me wanting more detail so she could copy my method. I’ve been asked to put a positive spin on sections of several other writing projects when my assessments of methods, services, or products were less than glowing.

We need to recognize our limitations. As a profession, we pretend we can solve more than we can. We cannot compete with companies like Google, Amazon, or a number of others on their own turf. If we make pathetic attempts to mimic these services, our users will simply ignore us. We must accept that some companies perform certain tasks far better than we will ever be able to and adjust our services accordingly.

At the same time, we need to not be afraid of taking risks or let past failures make us fearful of decisive action. The only way to succeed is to try. Most things worth shooting for are hard, so logically, there should be many failures. Rather than pretend they are successes, we should learn from them and move on.

The success of user created content in the blogosphere, Wikipedia, YouTube, and other venues has been welcomed as the democratization of information. However, I’ve been wondering if the success of these things isn’t because our information flows have become screwed up.

When formal channels can’t be used the way they should be, people seek alternatives. During the Communist era, the Soviet citizenry couldn’t find meat, toilet paper, and other essential goods in stores. To satisfy their needs, they turned to the shadow economy. I can’t help but notice that Americans now use a very similar method to find the information they need most.

Just as the planned economy failed the Soviet public, the channels we should be able to trust are failing us. Consequently, a growing number of people rely on the blogosphere, cocktail parties, and other places where the information and ideas are exchanged much more freely. Like their Soviet counterparts who knew better than to waste too much time looking in stores with empty shelves, these people largely ignore the mainstream media, the professional literature, and other traditional sources of information.

While many see this as a positive development, it reminds me of the old style Soviet economy when it was in its final throes. People all but gave up on the government stores and turned to bartering and the informal markets that sprung up everywhere to meet their needs. Eventually, the old system collapsed.

Just as Soviet citizens had to constantly hustle for butter and sugar that they knew couldn’t be found in stores, we turn to yahoos on the Internet and seek offline conversations for critical facts and honest assessments about people, methods, products, and organizations we can’t get from reports, presentations, articles, and conversations that appear on record. That we need to do this is symptomatic of inadequacies and problems in the formal channels.

If you think about it, a party is just a meeting with no agenda. Yet if you ask people if they want to go to a meeting, you will get a very different response than if you ask these same individuals if they want to go to a party — even if the same people are attending, they expect to talk shop the entire time, and you’re trying to tap them for designated driver duty.

Why is this? One explanation could be that the chances of learning something useful or interesting is far greater. This explanation also describes why people don’t pay attention to key reports, but they go out of their way to read blog postings on the same subject. Anyone who makes their living providing information services should be taking notice….

Yesterday, I took part in a conference call with one of our vendors. Our relationship has gone sour lately, and I was hoping this would be an opportunity for everyone to see things from the other side and start working better together. For most part, the discussions were very frank — a euphemism for saying that everyone was telling each other how it is.

That’s fair enough. While I believe that our vendors are business partners and we must work with them productively, our interests will not always align. This inevitably will result in frustration from time to time on both sides.

However, yesterday looked like it might be the beginning of the end. Multiple times, a high ranking representative on the other end referred to us “untrustworthy.”

I’ve heard similar accusations leveled at the company, but rather than get into a pissing contest over who is right, I think the more important issue is what happens when business partners don’t trust each other. Things are hard enough when everyone tries their best and presumes good intentions. As soon as you view your partner with suspicion, hedge your bets, and do the minimum to avoid being screwed yourself, you have a one way ticket to failure. That strategy is poison, and rarely leads to a good outcome. The discipline of game theory is practically dedicated to describing why this is the case.

The reality is that this vendor plays a key role in our operations, but we are a huge customer. Our relationship is somewhat like a couple navigating a major river in a canoe. If both people don’t work together, they wind up in the drink. It’s a dangerous situation, but probably survivable even if it’s pretty miserable in the short term.

However, there is a point when you have to ask yourself if you’re on the right path. I’ve stuck up for this company many times when I hear people razzing them. I think most people don’t appreciate how tough their position is.

On the other hand, this company has changed with time. Their outstanding strength has always been to provide powerful tools that let you do things they didn’t think up first. Now they seem more interested in keeping things locked up in proprietary structures that make it difficult to work in practical environments.

Nowadays, information comes from many sources and institutions need products from different vendors to work well together. Even Google doesn’t have the ability to do everything well, so they keep their focus on search. A small company that has far fewer resources will not be able to provide everything libraries need in a black box.

Whatever the case, I see storm clouds on the horizon. If things don’t get better soon, it will be a wild ride. Either we’ll be in a partnership that works much better, or we can find out what we’re really capable of when a long term partner pulls out. I’m looking forward to whatever happens.

One thing that I’ve always found strange about living in the information age is how little most of us know about anything. Sure, we get bombarded around the clock about the personal lives of movie stars and ramblings of pundits, but getting useful information is another matter entirely.

Right now, I’m thinking about Google’s nondisclosure agreement. To be offered a job there, you agree that you won’t mention or imply the name of Google, talk about anything that Google does, or use knowledge gained there at future places of employment. Agreements like this are pretty standard, but what bugs me is how many people accept this as normal.

If you can’t talk about what you do and what you know, at best you slow the learning process for those around you. At worst, you do real harm to people who make serious mistakes or cannot solve critical problems because you withheld information. Even if you care nothing about others or aiding progress, you hamstring your career because you might not be able to use your most marketable skill. It’s like learning a foreign language and then not being able to use it.

To get straight to the point, I think that most information is kept secret so those who claim to have an interest in the information can take advantage of others. Many vendors don’t allow libraries to disclose what they pay for products. The only logical explanation is that the customers who are getting ripped off would probably be mad if they found they’re paying much more than other customers. Keeping salaries secret protects the overpaid at the expense of the underpaid. My observation is that the vast majority of the people who hide information from their employers or employees do so with the intention of manipulating the system for their own purposes.

I understand that companies spend a lot of money creating information and that it can’t just be given away. I also understand the need for some information to be secret. However, there is a total lack of balance.

When someone needs to keep everyone else from knowing what they know in order to succeed, it makes me wonder what they really contribute. Withholding service and/or knowledge is not real work, even if the person in question occupies a linchpin position. However, as slaveowners, feudal barons, and others have discovered throughout history, it’s an easy way to live well off someone else.

The library community is always gaga over Google, and as much as I like their products, I think we need to keep an eye on these folks just like we would for anyone else. With about $150 billion in market capitalization as I write this, they effectively have more than $20 for every man, woman, and child on the planet or almost $140 for every internet user.

That’s really a lot for an outfit that makes the bulk of its money from people clicking on advertisements. It takes more than a company motto of “do no evil” to convince me that they’re that different from everyone else — particularly if they won’t let anyone who actually knows anything about them give their honest assessment of what’s actually going on.

For the past several months, I’ve been working on a book with Terry Reese. Basically, it discusses issues and technologies people need to be aware of if they’re thinking of starting a digital repository. We will finish this week. Even though the publisher has already accepted all of the chapters save two (which Terry and I are finishing), I can already see I have to add or modify a few sections that I was feeling pretty good about just last month.

Right now, I’m at a conference. One thing that absolutely blows my mind is that no matter how many skills and ideas I add to my bag of tricks, I always feel like I’m falling further and further behind.

Only a few years ago, a lone programmer could change the world with some code that was written in a couple hours (or days) using a handful of basic technologies. The days when the lone systems generalist could make a difference are ending. Skills that were once considered exotic are now essential for survival. The technologies, workflows, and services become increasingly sophisticated. Keeping up with the basics is like drinking out of a fire hose.

Our goal was to write something we’d want to read ourselves. Despite the fact that it’s on a technical topic, we were also hoping the ideas would be good for a few years. We’ll see how things go. In a worst case scenario, our book may help make some birds happy or prove useful for training puppies.