Having worked on a few incredibly frustrating experience management projects, I began to wonder if there are any great success stories out there. Are there better solutions available? Personal experience as well as research suggests that even the best-in-class products still require so much tweaking that it might make more sense to build something custom to begin with. Even when that appears to be the best path forward, building an experience management system can be a long, expensive, and complicated project. Maybe it's simply not worth doing at all.
First, let's distinguish between experience and expertise management. These two labels may be used interchangeably, but most often expertise refers to people and the knowledge and skills they possess. However, law firms tend to shy away from the expert label and use the softer-sounding experienced label instead. (Note: In ILTA's 2014 Technology Survey these two labels were mixed together but, even so, 77 percent of firms responding said they had no system for tracking experience/expertise. The larger firm size categories were more likely to have something in place and 72 percent of 700+ attorney firms did have a system.)
Tracking individual attorney expertise isn't easy either, but enough firms have tackled this problem by mining the work attorneys do and inferring experience rather than attempting manual capture in a database or similar vehicle. While that could yield more accurate results, it is rare that attorneys have the time or inclination to continually report and update their skills. Close enough results are often acceptable because expertise searching is often limited in exposure.
Experience, on the other hand, usually refers to a group's knowledge and skills, or more specifically the matter experience a firm has, and is used in pitches and proposals or for more general marketing purposes, such as in a firm's public website, Chambers, Thomson League Tables and other rankings or listings. Accuracy is more important here. Experience is the firm's positioning to the external world, clients, prospects, and competitors.
If you have not been involved in an experience management project yet, you might wonder what is so difficult about it. It's just a database, right? All you need is client, matter description, and a handful of other key pieces of information, right? Wrong. These are very complicated systems. The most common problems from a high level are listed below. Some of these will be discussed in greater detail.
- Product Selection—There just aren't that many to choose from. There are some newer market entrants, but they are not yet proven. Yet if you're not with a large firm, you may not have the resources to build a custom system.
- Vendor Support—Commercial systems are rarely used "out of the box," but customizations can make vendor support challenging because institutional project knowledge is lost over time, due to changes in personnel on either the vendor or firm side. Also, because there are no clear market leaders, the number of support personnel available may be minimal.
- Complexity—There are common fields needed for all matters, but these are often overlaid with unique and large numbers of fields that are practice or matter specific, resulting in very complex systems.
- Flexibility—Avoiding silos of spreadsheets or other "noncompliant" experience lists that practices used prior to implementing a firm-wide solution may not be possible if you aren't flexible enough to meet their needs with a central system. At the same time, you need to balance their needs with what the marketing department needs and the desire to "keep it simple."
- User Interface—Tend to be complicated due to the large number of fields and complex logic to only present relevant information (for example, if yes, then ask this additional question, if no, ask these two other questions).
- Classification Schemes—Getting agreement on classification schemes (e.g., matter types, industry, area of law) can be very challenging.
- Data Conversion and Normalization—Moving from one system to another (even when the system is a bunch of Excel spreadsheets) may require complex logic and scripting for data conversion. In some cases it may outweigh the effort, meaning rekeying all of the historical information into the new system.
- Integration—Pulling data from accounting and other systems in an attempt to automate but still requiring "exceptions" to this data, or pushing experience information back into other systems like a firm's website. Straight integration is not very difficult but if you need to include complex business rules, handle exceptions, or transform data, it can get complicated very quickly.
- Time—Getting attorney information on matters to be profiled, made far worse if buy-in is not obtained in advance.
- Laterals—Incorporating incoming laterals' experience into the system as well as worrying about departed attorneys.
- Setting Thresholds—Determining what constitutes a matter worthy of capturing, which may vary from practice to practice, and building workflows/alerts to let people know it's time to "profile a matter."
Complexity vs. Flexibility vs. Keeping It Simple
I've yet to see an experience system that doesn't have 100 or more fields, at least 30 or more fields for each type of matter above the common core fields. Even if you've seen the output of these systems (typically in pitches, proposals, and ranking tables) most people have no idea how difficult it is to make that possible. Aside from the difficulty imposed by fields pertaining to unique types of work, more complexity is introduced by the practice area/group classifications that firms use on their public websites, which do not match how things are viewed and organized internally. This means the experience database has to have a way to map the values between one schema and another. And what happens when any one of these schemas changes? Suddenly you have records that indicate a practice area that has now been split into two (even worse than when two groups are merged). What do you do? Sometimes there is a need to keep these historical classifications too.
Speaking of classification, I once spent two years on an area of law reclassification project related to an experience database, two fun-filled years of meetings and huge, gnarly spreadsheets just to get agreement and buy-in from every practice that the new schema was correct. It amounted to a single field in the database. This story is not unique. If you find anyone who worked on an experience management implementation who still has hair, they probably missed some important stuff!
Even when all of that classification and agreement has been achieved, you encounter the next issue, the User Interface (UI). With so many fields to fill in, particularly for work that spans more than one practice or matter type, it commonly requires several screens' worth of fields, or perhaps worse, one page that scrolls on and on and on. Trying to get matter details from an attorney is a daunting task. Present them with an ugly, complicated UI, and you're even less likely to gain their buy-in. Many firms don't even give attorneys access to the system, partly due to this issue.
Are There Any Happy Stories?
I spoke with numerous firms and my suspicions were confirmed: there are very few case-study-worthy stories around experience management. In asking people what the hardest part about their EM project is/was, one commented:
"The technology is the easy part. I'm really rethinking how much information we should capture. Do we really need 100 fields per record? Since every field is searchable, do we really need to separate everything? We are trying to strike a balance between fewer fields and ensuring data consistency and quality. For instance, we use the concept of tagging by using a keyword pick-list, which users can add to on the fly. This allows them to use their own lingo, be flexible to support any kind of information, but by tag sharing we ensure some amount of data consistency and reduce the proliferation of fields. That also helps keep the UI cleaner."
This sounds like a good approach to me—flexible yet comparatively simple. It seems like we haven't applied the best technology to experience management. If we can infer the expertise of an attorney by analyzing his or her work product documents, time entries, and bios, why can't we do something similar for matters to infer firm experience? There are definitely some core fields you will always need, but beyond that, it just gets too complicated. I would love for a vendor to develop and/or apply auto-classification technology, or e-discovery techniques, to pull out the core concepts around sets of documents and time entries, applying thesauri and/or legal taxonomies, to alleviate the need for so much manual profiling. There is a golden opportunity here to make many firms happy! In fact, it might be possible to abandon traditional experience management system approaches entirely to an almost completely automated data mining-type solution.
Back to Contents