ba AN INTEGRATED APPROACH USING DEMATEL , ANP AND TOPSIS FOR EVALUATING RESEARCH & DEVELOPMENT PROJECTS

Research and development (R&D) projects should be one of the main dimensions of universities for providing academic development. It is also a core performance indicator for monitoring and ranking universities. For these reasons, usually there are many projects submitted to the institutions that fund R&D projects. Funding institutions should evaluate the submitted projects in terms of multi-criteria and select the suitable ones among them. Therefore multi-criteria decision making techniques can be implemented as a useful tool for these kinds of problems. In this study an integrated approach which includes DEMATEL, ANP and TOPSIS methods is used for evaluating and ranking projects. The criteria are determined by taking the Turkish Scientific and Technical Research Institute’s (TÜBİTAK) project selection procedures into consideration. DEMATEL method is used in order to detect the cause and effect interaction among main criteria. Then ANP method is implemented for calculating the weights of each criterion. Finally, TOPSIS method is applied for ranking the projects.


Introduction
Research and development (R&D) projects should be one of the main dimensions of universities for providing academic development.Having R&D projects increases the quality of universities.It is also a core performance indicator for monitoring and ranking universities.Moreover, universities' long-term strategy of increasing R&D projects generates economic value for its country and the community.For these reasons, usually there are many projects submitted to the institutions that fund R&D projects.Funding institutions should evaluate the submitted projects in terms of multicriteria and select the suitable ones among them.
Evaluation is an essential tool that not only helps measuring projects' success, but also contributes to their success.Evaluation helps managers to plan, verify, and communicate what they aim to do, to decide how to allocate resources, to learn how best to modify or redesign programs and to estimate the project outputs, outcomes, and impacts (Ruegg, 2007).The main purpose while considering R&D-projects is to ensure that they are evaluated effectively, and to select ones which achieve the maximized benefit.Meade and Presley (2002) revealed three major themes for R&D project selection: (1) The need to relate selection criteria to corporate strategies.(2) The need to consider qualitative benefits and risks of candidate projects.(3) The need to reconcile and integrate the needs and desires of different stakeholders.Besides, the need of group decision making methods can arise in addition to these items.Because, R&D project selection involves multiple interrelated criteria and qualitative factors that are difficult to be measured by an individual expert.Similarly, limitations of existing R&D project selection identified by Chien (2002) are:  inadequate treatment of multiple, often interrelated, evaluation criteria;  inadequate treatment of project interrelationships among projects;  inability to handle non-monetary aspects; e.g.diversity among projects  no explicit recognition and incorporation of the experience and knowledge of R&D managers (i.e. the decision makers);  perceptions by R&D managers that the models are difficult to understand and use.
Hence, the multi-criteria decision making techniques can be implemented as a useful tool for these kinds of problems.This study aims to propose an evaluation approach based on a combined DEMATEL, ANP and TOPSIS methods in order to select R&D projects.In this study, the criteria are determined by taking the Turkish Scientific and Technical Research Institute's (TÜBİTAK) project selection procedures into consideration.DEMATEL method is used in order to detect the cause and effect interaction among main criteria.Then ANP method is implemented for calculating the weights of each criterion.Finally, TOPSIS method is applied for ranking the projects.
The structure of the rest of the paper is: In the next section, the evaluation framework is introduced and the techniques are explained.Then, implementation of the proposed integrated decision making method is detailed.Finally, some conclusions and discussions are given in the last section.
The study proposes an integrated approach in which DEMATEL, ANP and TOPSIS methods are used in a combined manner.The following three subsections clarify these methods.

The DEMATEL Methodology
DEMATEL method was developed by Gabus and Fontela (1972).It analyzes the influential status and strength between the factors and converts them into an explicit structural mode of a system (Lin and Wu, 2008).
The steps of DEMATEL technique are explained below; Step 1: Generating the direct-relation matrix: An evaluation scale of 0, 1, 2, 3, and 4 is used for influential comparison where 0 represents "no influence" while 4 represents "very high influence".A group of experts is asked to make pairwise comparisons between criteria.To compound all opinions from K experts, the direct-relation matrix A is calculated using Eq.
where the    is the score given by the k th expert indicating the influential level that factor i has on factor j.
Step 2: Normalizing the direct-relation matrix: The normalized direct-relation matrix M can be obtained by normalizing A using Eqs.( 2) and (3).
Step 3: Obtaining the total-relation matrix: The total-relation matrix T can be obtained by using Eq. ( 4), where I denotes the identity matrix.
Step 4: Compute the dispatcher group and receiver group: The vectors D and R represent the sum of rows and columns of matrix T respectively (Eqs. 5 and 6).D + R value indicates the degree of importance that the corresponding criterion plays in the entire system.The factor having greater value of D + R has more interrelationships with other factors.On the other hand, criteria having positive values of D -R are on the cause group and dispatches effects to the other criteria.On the contrary, criteria having negative values of D -R are on the effect group and receive effects from the other criteria.

𝐷 = ∑ 𝑡 𝑖𝑗 𝑛 𝑗=1
(5) Step 5: Set up a threshold value to obtain the causal diagram: Since the total-relation matrix T provides the information on how one criterion affects another, decision maker group should set up a threshold value in order to filter out some negligible relationships.

The ANP Methodology
ANP is the general form of analytic hierarchy process (AHP) and was proposed by Saaty (1996) to overcome the problem of interrelation among criteria or factors.It provides measurements to derive ratio scale priorities for the distribution of influence between factors and groups of factors in the decision (Saaty, 2001).The feedback structure does not have the top to bottom form of a hierarchy but looks more like a network, with cycles connecting its components of elements, which we can no longer call levels, and with loops that connect a component to itself (Saaty, 2005).
Through a supermatrix, whose entries are themselves matrices of column priorities, the ANP synthesizes the outcome of dependence and feedback within and between clusters of elements (Yang and Chang, 2012).The initial supermatrix must be transformed to a matrix in which each of its columns sums to unity.For this reason, this matrix must be normalized by the cluster's weight to get the column sums to unity.Hence, the weighted supermatrix is obtained (Saaty and Vargas, 1998).The supermatrix representation is given in Fig. 1.

Figure 1. The supermatrix representation
Pairwise comparisons between the criteria can be implemented according to dependency relationships which are obtained from DEMATEL approach in order to generate local weights assessing relative importance value using a scale of 1 (equal importance) to 9 (extreme importance).

The TOPSIS Methodology
The technique for order preference by similarity to an ideal solution (TOPSIS) was proposed by Hwang and Yoon (1981) and expanded by Chen and Hwang (1992).The main principle in TOPSIS method is that, in a graph, any chosen alternative should have the shortest distance from the ideal solution and the farthest distance from the negative-ideal solution (Opricovic and Tzeng, 2004).
The TOPSIS technique is implemented using the following steps (Triantaphyllou, 2000;Opricovic and Tzeng, 2004): Step 1. Calculate the normalized decision matrix: D is the decision matrix which refers to n alternatives that are evaluated in terms of m criteria.
R is the normalized decision matrix and rij is an element of R. The normalized value rij is calculated as: Then the R matrix is formed as follows: Step 2. Calculate the weighted normalized decision matrix: V is the weighted normalized decision matrix and vij is an element of V.The weighted normalized value vij is calculated as: where wi is the weight of the ith criterion and ∑   = 1  =1 . Then the V matrix is formed as follows: Step 3. Determine the positive-ideal and the negative-ideal solutions: The positive-ideal donated as  * and the negativeideal donated as  − alternatives are defined as: where  ′ is associated with benefit criteria, and  ′′ is associated with cost criteria.
* indicates the most preferable solution and similarly  − indicates the least preferable solution.
Step 4. Calculate the separation measure: The separation of each alternative from the positive-ideal solution and negativeideal solution are calculated using n-dimensional Euclidean distance method.The distances from the positive-ideal solution and negative-ideal solution can be calculated as follows: Step 5. Calculate the relative closeness to the ideal solution: The relative closeness of alternative   with respect to  * is calculated as follows: where 0 ≤   * ≤ 1.
If   =  * then   * is equal to 1 and if   =  − then   * is equal to 0.
Step 6. Rank the preference order: The best alternative can be now decided according to the preference rank order of   * .
Therefore, the best alternative is the one that has the shortest distance to the ideal solution.

Case study: R&D projects evaluation and selection
First of all, interactions among the main criteria are obtained by asking some academic experts using DEMATEL approach.
As an example the evaluation of one of the experts is given in Table 2. Similarly, evaluations of the rest of the experts are obtained and then averages of numbers are calculated using Eq. ( 1) in order to form initial direct-relation matrix (see Table 3).The normalized direct-relation matrix is obtained using Eqs.(2 and 3).After calculating the normalized direct-relation matrix, the total-relation matrix is obtained using Eqs.(4, 5, and 6).The total-relation matrix is shown in Table 4.The threshold value is determined as 0.55 by the experts.The values above the threshold are indicated in bold that give the cause and effect relationship among the main criteria.Those indicators are used in constructing pairwise comparison matrixes for ANP.For example C1 effects C2, as can be seen from Table 4, thus, pairwise comparison matrixes for sub-criteria of C2 are built for each sub-criterion of C1.As an illustrating example Table 5 gives the pairwise comparison matrix for one of the experts for sub-criteria of C2 in terms of criterion C11.
After taking the rest of the experts' evaluations, geometric average is calculated and then local weights are gained by following ANP approach (see Table 6).Similarly, the rest of the local weights are calculated based on the interactions obtained from the DEMATEL.Then, the unweighted supermatrix is formed for sub-criteria by placing the calculated local weights into the matrix in the proper places (see Table 7).Then, unweighted supermatrix is normalized to transform it to the weighted supermatrix in which each of its columns sums to 1.The power of the weighted supermatrix is taken until the values of each column are stabilized.These calculations are implemented using MATLAB software and the limit supermatrix is obtained which is given in Table 8.Any column of the matrix shows the weights of corresponding sub-criteria.
As a next step TOPSIS method is implemented after obtaining the local weights of each sub-criterion trough the limit supermatrix.TOPSIS method is used to score and rank the projects in terms of local weights and decision matrix which is built by scoring each project considering each criterion.The decision matrix is given in Table 9.Then, Table 9 is normalized by using Eq. ( 7) and multiplied by the weights obtained from ANP limit supermatrix, by using Eq.(8).The new table is called as the weighted normalized decision matrix.Then, the positive-ideal A * and the negative-ideal A − values are calculated by using Eqs.(9 and 10).Table 10 illustrates the weighted normalized decision matrix, and the values A * and A − .The separation or distances of each alternative from the positive-ideal solution and negative-ideal solution are calculated using Eq.(11 and 12).Then, the relative closeness of alternative A j is calculated using Eq. ( 13).Table 11 shows the overall results and the rank of each project.
As can be seen from Table 11, project 11 is found out as the best project because of having the greatest value of  * .The amount of the projects to be selected for funding can be determined depending on the allocated budget.Table 11 would be useful while considering project election.

Conclusion
This paper proposes an approach for evaluating and selecting suitable R&D projects to be funded by an institution, based on hybrid multi-criteria decision making methods.The proposed approach also depends on group decision making concept.Turkish Scientific and Technical Research Institute's (TÜBİTAK) project selection procedures are taken into consideration for determining main and sub-criteria.In order to obtain cause and effect interaction among main criteria which will be required for ANP method, DEMATEL approach is implemented to the main criteria.Based on those cause and effect interrelationships, the weights of each sub-criterion are calculated by applying ANP methodology.Finally projects are evaluated and ranked using TOPSIS method.The projects having the highest performance indicator values can be selected according to the institute's financial budget.The proposed approach is justified by applying to a case study.This approach can also be implemented to other similar multi-criteria and group decision making problems.
Habib et al. (2009)present a method for R&D project selection that allows for the consideration of important interactions among decision levels and criteria.The methodology uses the ANP for this evaluation.The research paper concludes with a case study describing the implementation of this

Table 9 .
Decision matrix for the projects

Table 10 .
The weighted normalized decision matrix, and positive and negative ideal solutions

Table 11 .
Final performance indices of the projects