Expertise-based ranking of experts: An assessment level approach

The quality of a formal decision is influenced by the level of expertise of the decision makers (DMs). The composition of a team of DMs can change when new members join or old members leave, based on their ranking. In order to improve the quality of decisions, this ranking should be based on their demonstrated expertise. This paper proposes using the experts' expertise levels, in terms of ‘the ability to differentiate consistently’, to determine their ranking, according to the level at which they assess alternatives. The expertise level is expressed using the CWS-Index (Cochran–Weiss–Shanteau), a ratio between Discrimination and Inconsistency. The experts give their evaluations using pairwise comparisons of Fuzzy Preference Relations with an Additive Consistency property. This property can be used to generate estimators, and replaces the repetition needed to obtain the CWS-Index. Finally, a numerical example is discussed to illustrate the model for producing expertise-based ranking of experts.