Solving Constrained Optimization Problems with Sine-Cosine Algorithm

ABSTRACT


Introduction
Optimization can be defined as finding the most effective and highest achievable performance under the given limitations.Mathematically speaking, Optimization is finding the minimum or maximum of a function subject to the constraints.A set of values that satisfies all the constraints of an optimization problem creates a feasible solution.The optimization technique tries to find the optimum solution from all of these feasible solutions [1].
Optimization problems can be found in every area of life because all living things tend to do the best.For example, Birds fly in 'V' shape to reduce the energy consumption, another example fish moves in flocks to benefit from defense against predators [2].Optimization problems have been a topic since 1960's.In these years optimization problems have been tried to solve by classical mathematical methods.Deterministic methods have a great advantage that they find global optima.Unfortunately, they cannot solve all nonlinear problems.With classical optimization, only limited problems can be solved.The inadequacy of classical methods has forced scientists to search for new methods.To find the optimum, solution stochastic algorithms are developed.These algorithms sample the search space without exploring it thoroughly.Stochastic computation techniques have received a great deal of attention regarding their potential as optimization techniques for complex problems.As a result, the development of stochastic algorithms has begun.
It is impossible to develop one way to solve all the nonlinear problems.There are a lot of methods proposed.Stochastic algorithms can find promising solutions for difficult optimization problems, but there is no guarantee that optimal solutions can be reached all the time.Stochastic algorithms are good at solving most of the real world problems which are nonlinear and multimodal [1].
The general nonlinear programming problem is defined as follows [3]; Where   and ℎ   are constraints that are required to be satisfied.
The optimization problem is based on finding the optimum value of the objective function, and if there are no constraints on the variables, these problems are called unconstrained optimization problems.The solution of unconstrained optimization problems is easier than the constrained ones.However, most real-life problems are constrained.It is necessary to find the best value of a constraint optimization problem such as resource constraint, time constraint, cost constraint, design constraint according to these conditions.

Related Works
In 1994 Joines and Houck solve four test cases of constrained optimization problems with the genetic algorithm.They transform constrained optimization problems to the unconstraint optimization by using the penalty method.They aim at reaching the feasible solution of genetic algorithm by giving appropriate value according to the number of generations [4].In 1996 Michalewicz and Schoenauer present several constrainthandling techniques for optimization problems.The first one based on feasibility of solutions, the second one is penalty based, the third method makes a clear distinction between feasible and infeasible and fourthly hybrid methods.Moreover, they provide 11 test cases which we use in this study to make our experiments [3].
In 2002 Hu and Eberhart applied one of the constraint handling methods which based on preserving feasibility of solutions.They test particle swarm optimization algorithm on the same test cases [5].In 2005 Yeniay go over all constraint handling penalty based techniques for the genetic algorithm.He mentions their advantages and disadvantages.He emphasizes the importance of setting appropriate values of the penalty parameters [6].

Dealing with constraints
All the stochastic algorithms are directly suited to unconstrained optimization problems.Applying these algorithms to constrained optimization problems has always been a problem.In real life problems such as engineering design problems are constrained optimization problems and constraints has a great effect on the optimization performance [7].Fortunately, many constrained optimization algorithms can be transformed to the unconstrained case, often with the use of a penalty method.
Penalty function method is common because of its simple principle and easy implementation.We modify the objective function in such a way that it penalizes any violation of the constraints.
Penalty function method can be formulated as follows [8]; In minimization problems, we include the penalty function which adds a high cost to the objective function.In maximization problems, we subtract the penalty function from the objective function.
In this study, we implement two of the state-of-art algorithms that we mentioned to the well-known constraint test cases.In addition to these algorithms, we also implement Sine Cosine Algorithm that developed in 2016 [9].The above-mentioned algorithms were directly suited to unconstrained optimization test functions.
To deal with constraints we use the penalty function method due to its popularity and easy implementation [7].Penalty method adds the penalty term to the objective function for any violation of the constraints [8], [10].We aim to show the performance of Sine-Cosine Algorithm on constrained optimization problem and compare results with genetic algorithm and particle swarm optimization algorithm.Genetic algorithm mimics the biological evaluation.It starts with creating a population randomly.And population can be described as a group of individual solutions.In each iteration, the algorithm chooses some solutions from the current population as parents according to their fitness values to form a new generation.The new generation is created by applying genetic operators such as crossover and mutation.Next, these new generations are evaluated, and this process is going on until the termination condition is met.And the population evolves toward an optimal solution.Pbest is the best solution has achieved so far by a particle.Gbest is the best value obtained so far by any particle in the population.Next, the velocities of the particles are calculated using pbest and gbest values.Then particle positions are updated based on the velocities.Thus, the particles follows the best particle in the search space.[11]  As another population-based optimization algorithm sine cosine algorithm also starts with initializing random solutions called search agents.Sine cosine algorithm uses 4 variables to tune.These are r variables. decides that search agent is going to do whether exploration or exploitation.All stochastic algorithms are both exploration and exploitation but it is important to balance these. decides how far the solution's movement should be. assign a random weight and  decided whether sine or cosine formula is going to be used.

Sine Cosine Algorithm (SCA)
In every iteration, the solutions are evaluated by using the fitness function and the algorithm assigns the best solution obtained so far as the destination point.Next, the r variables are updated.
Search agent positions are updated based on the r variables and the best solution.Thus the potential solutions follow the best solution in the search space [9].

Experiments
We use global optimization toolbox in Matlab© which includes both genetic algorithm and particle swarm optimization algorithms with default values.And we implement Sine cosine algorithm with 30 search agents and 1000 iterations.The algorithms are run 30 times on the popular constraint optimization test problems that Michalewicz and Schoenauer first presented.We get these test problems from Kyoto University global optimization test problems web site [13].We implement penalty method to these test cases.
The test problems are given below;

Experimental Results
Results are shown in figures.The red lines represent the best-known values gathered in Kyoto University Global Optimization Website.
Sine cosine algorithm, genetic algorithm, and particle swarm optimization algorithm are represented by respectively by orange, pink and blue bars.In terms of accuracy mostly genetic algorithm gives the best results.SCA algorithm is outperformed by the others or results as the same.In G2 is maximization problem and in these experiments, any of the algorithms reach the maximum value gathered in Kyoto University Global Optimization Website so far.All the algorithms find the optimum value in G4 and G11 test cases.Like in these situations it is better to choose the algorithm which takes a shorter time.In G5 only PSO find the minimum optimal value.In G6, SCA and GA are both find the minimum value which updates the minimum value found up to now.Moreover, SCA is not good at solving problems like G7 but in the G8 test case, SCA outperforms both GA and PSO.In G9, G10, G13 test cases SCA is outperformed by the others.This graph demonstrates the time taken by the algorithm for each problem.In most of the cases, the slowest algorithm is a genetic algorithm.Except two of the cases, PSO is the fastest one.And SCA takes slightly more time than the PSO.Mostly both PSO and SCA finish under a second.

Conclusions and Discussion
In this study, we optimized well known constrained optimization problems with using the recently developed sine cosine algorithm.We deal with constraints with the help of penalty method.Then we compared SCA algorithm results with GA and PSO.In terms of accuracy GA gives the best results but in terms of speed PSO and SCA are faster.Although SCA gives better result in unconstrained optimization problems, it does not perform well on constrained optimization problems.But SCA is a new algorithm and it might give better results by making further improvement in the algorithm.Also, it might give better results when we change the method of handling constraints.

Figure 5 .Figure 7 .Figure 9 .Figure 11 .Figure 12 .
Figure 5. G6 Test Function(Minimization) Figure 6.G7 Test Function(Minimization) Particle Swarm Optimization is another population optimization technique developed by Eberhart and Kennedy in 1995.It starts with initializing the population of random solutions called particles.In PSO the particles have velocity values alongside the fitness values.In every iteration, gbest and pbest are selected.