Meaningful benchmarking in a public sector context

Benchmarking definition

What is benchmarking in a public sector context?  Benchmarking typically involves collecting information (and insights) on the delivery models and practices of similar programs or services in other jurisdictions or the private sector.  It is one of the most effective ways of identifying new and innovative approaches to program or service delivery.

The benchmark programs or services need not be identical but they should have some common characteristics or features.  For example, organizations may have a similar regulatory mandate or provide similar citizen services.  Seeking new approaches to service delivery may entail looking beyond organizations that provide the same service.  Some managers will maintain their services are unique—benchmarking is one way to stretch the boundaries and look beyond.

Benchmarking may also involve performance and cost comparisons with organizations in the same sector (e.g., facilities management).  A number of sector collaborative efforts and groups exist in this regard.

Benchmarking also has a secondary benefit of helping to develop long term collaborative relationships between public sector agencies delivering similar programs or services.

 Key challenges of the benchmarking process

Benchmarking does have its challenges.  For example:

• Clarifying the objectives of the benchmarking and comparative analysis. It is easy to lose focus as to why you are doing the benchmarking in the first place unless it is meant to be exploratory in nature.  The type of benchmarking done will influence your plan of action.  For example, benchmarking costs or performance will entail a very different focus from benchmarking service delivery methods.

• The high volume of information available and the need to identify information that is relevant to the study. One can become overwhelmed by the large amount of information available through the web.  Unfortunately, a lot of this information will be superfluous and not particularly useful to answering the key questions pursued by the benchmarking.  One needs to be able to decipher which information is most relevant.  The information collected also needs to be well organized based on key dimensions or criteria to be compared.

• Availability of critical information. Although a lot of information exists, information is often not readily available to address those questions that are of specific interest to the benchmarking exercise.  For example, cost and performance information can be difficult to obtain.  The level of resources devoted to specific activities may only be available in a very general way.

• Seeking the participation of other jurisdictions depending on their priorities and their level of interest in the subject matter. A commitment to share the results of the benchmarking with all participants is an incentive for the benchmark organizations to participate and devote the necessary time.  However this often is not enough.  Some gentle arm twisting may be required.

• The elapsed time required to conduct the work given the time required to seek the participation of the benchmark organizations, their availability for interviews and data collection, and the level of detail required of the analysis.

• The sensitivity of the participating organizations to sharing information with each other or with the client organization sponsoring the study.

So what can one do to avoid these pitfalls?  Establishing a knowledgeable benchmarking team and following some key steps can help mitigate these issues.

Establish an experienced project team

There can be a tendency with benchmarking to assign less experienced people to collect the information as it is perceived to be mainly an exercise of data collection.  Unfortunately, the result can be a lot of information that is not very useful and sure to be shelved.  It is critical that the people conducting the benchmarking research have an in-depth understanding of the sector in which the agencies operate, and are able to discern which information is most relevant. 

Benchmarking entails extensive analysis and synthesis of the information collected, and cannot be simply be an exercise of copying and pasting information regurgitated from an agency’s web site.  The greatest risk is that the results of the benchmarking will not be used because the information is deemed to be interesting but not very useful for decision-making.

Some key steps in the benchmarking process

Review industry/sector trends

It is always a good idea to do a high level review of industry/sector trends or a literature review at the outset of the benchmarking study.  The extensive information that already exists on the web will help to identify sector leaders and examples of common or best practices.  This information will also help to scope out and validate the objectives of the benchmarking exercise.

Identify potential benchmark organizations

The selection of the potential benchmark organizations is a critical step and needs to be reviewed on an iterative basis throughout the exercise.  Obtaining the commitment of targeted benchmark organizations in other jurisdictions will depend on their willingness/interest to participate given their priorities, and their openness or potential reluctance to share information or provide operating and cost information.  Existing contacts or relationships between the client organization sponsoring the benchmarking and potential benchmark agencies in other jurisdictions can help.

Comparisons with agencies in other countries will often generate the most different delivery approaches, although one must be careful to best understand the context in which these agencies operate.  Comparisons with jurisdictions in the same country provide the benefit of greater comparability in terms of the clientele and context but the delivery approaches may not be as radically different. 

Depending on the nature of the program or service, it may be possible to do comparisons with the private sector.  Private firms may be more reticent about sharing information; however firms that have experimented with new and innovative methods of delivery will provide valuable insights.

Prepare summary profiles of the benchmark organizations

The preparation of summary profiles of the benchmark organizations and validation of these profiles with the organizations during the interviews is an excellent way of obtaining the engagement of the participating organizations.  These profiles also help to clarify the purpose of the benchmarking and the key questions being addressed.  All the profiles should be prepared using the same common structure so as to later facilitate the comparisons between the organizations (see an example below of the profile contents).  Remember that you will also need to prepare a profile of the organization sponsoring the benchmarking so as to ensure comparability.

Conduct interviews with the benchmark organizations

Interviews are the most effective way to validate the organization profiles, address the information gaps, and obtain an in-depth understanding of the history of the delivery of the program or service in other jurisdictions, and more importantly, where it is heading.  The interviews also provide the context information to better understand why certain decisions or changes were made, and what worked or did not work as well.  For example, some countries have taken more risks in experimenting with new service delivery methods in the public sector, and in some cases have reverted back to previous delivery methods given problems encountered.  

The interviews also help to develop a long term working relationship between agencies.  Often, more than one interview is required with an organization, and there may be extensive follow-up and discussion.  This is all part of the learning experience.

Do comparisons based on key dimensions or criteria

There are many ways of presenting the benchmarking results.  Typically this involves a comparison between the benchmark organizations based on selected dimensions or criteria that are critical to the delivery of the program or service.  These dimensions and criteria may in the end be one of the key findings of the exercise, and will need to be adjusted and revised continuously so that they best reflect the strategic capabilities commonly required to deliver the program or service.  Generic examples of these dimensions and criteria are shown in the chart below.

It is also useful to group the benchmark organizations by type of delivery model or other distinguishing features or characteristics.  The benchmarking should help to identify those key characteristics or features that enable public agencies to be successful in their particular domain. 

Identify common/best practices and opportunities for improvement  

Ultimately, the benchmarking should lead to the identification of common/best practices and specific opportunities for consideration by the organization leading the benchmarking initiative.  Comparing the existing practices of the organization conducting the benchmarking to the most common or best practices of the benchmarking organizations under each dimension/criteria is an effective way of identifying gaps or alternative approaches.

There is always a tendency to identify leading or “world class” best practices.  The distinction between common (i.e., practices that are prevalent within the sector) and best practices is not always clear.  In practice, the application of specific practices must always be considered within the context of each organization. 

Implicit in these comparisons is establishing which delivery model each organization falls into and which delivery model makes most sense given the environment or context within which the organization is operating.

Distribute the results of the benchmarking to the participants

Finally, it is critical that participants receive a detailed report of the results of the benchmarking that is helpful to them in their decision-making and reflects their contribution.  In doing so, one needs to be careful to protect the confidentiality of the information where commitments have been made. 

Benchmarking tools and templates

The benchmarking guide and tools on the Public Management Toolkits site provide a detailed description of the benchmarking process as well as templates on how to complete a successful benchmarking exercise.  These can be modified depending on your requirements.  The tools have been tailored to specific functions and include examples of best practices.  A generic benchmarking tool is also available.