COMPARISON OF METAHEURISTICS IN SOLVING THE KNAPSACK PROBLEM: AN EXPERIMENTAL ANALYSIS

Objective: Through statistical analysis using ANOVA, compare the obtained results and processing time of the metaheuristics Local Search, Tabu Search, and Genetic Algorithm programmed in Python language for application in the Knapsack Problem among the described instances. Method: The method used was modeling in order to compare randomly generated instances where the metaheuristics were programmed in Python language, inserted in Google Colaboratory, and executed in the cloud. Results and Conclusion: Analysis of Variance (ANOVA) was employed as there were three samples with paired instances to ensure conclusion validation. It was observed that, for the instances and interruption parameters of the metaheuristics used, the Genetic Algorithm generated more satisfactory results than the other metaheuristics. Research Implications: Provides relevant information about the effectiveness and performance of metaheuristic techniques, contributing to the evolution of the field of Operations Research by guiding the choice of approaches in practical applications and promoting collaboration and scientific replicability.


INTRODUCTION
Solving the Knapsack Problem has been the subject of numerous studies in the field of Combinatorial Optimization, aiming to find efficient and accurate solutions to the problem. In the quest for optimized solutions, metaheuristics have proven to be a promising approach, capable of finding solutions close to the global optimum in a timely manner and with reasonable computational cost.
Among the most commonly used metaheuristics for solving the Knapsack Problem, Simulated Annealing, Tabu Search, Local Search, Variable Neighborhood Search, Ant Colony, and Genetic Algorithms stand out. Each of these approaches has its own characteristics and advantages, but the choice of the best technique will depend on the problem's characteristics, such as instance size and complexity of the constraints.
In this article, we propose a comparison among some of the most commonly used metaheuristics for solving the Knapsack Problem, aiming to evaluate the efficiency of each of them in order to contribute to the selection of the most suitable technique for each case. Based on the conducted experimental analyses, relevant data is provided for researchers and professionals in the field to select the most appropriate technique for solving their Knapsack Problem instances.

Knapsack Problem
The Knapsack Problem aims to choose, among the available objects, which ones should be placed inside the knapsack in order to satisfy the constraint of the maximum weight capacity that can be carried while maximizing the total utility of the chosen objects. In other words, both the weight and value of each object need to be taken into consideration (BELFIORE and FÁVERO, 2012a).

Mathematical Formulation
According to Belfiore and Fávero (2012a), the mathematical formulation for the Knapsack Problem is as follows: It aims to determine, among the n possible objects, which ones should be carried. The objective is to find the optimal selection of objects to be placed in a knapsack with a maximum weight capacity given by , in order to maximize the total utility while considering the constraint that the knapsack's capacity cannot be exceeded. To do so, it is necessary to take into account that each object j has a utility value and a weight .

Heuristics and Metaheuristics
Heuristics are approximate techniques that aim to find acceptable solutions, even if they are not necessarily the best or optimal. They can be used in a wide range of problems, especially when dealing with complex situations that are difficult to solve or have large search spaces (GOLDBARG et al., 2015).
According to Belfiore and Fávero (2012b), the term "heuristic" comes from the Greek language and means "to discover." Heuristics can be developed based on expert knowledge, intuition, trial and error, and they are designed to handle highly complex problems or problems with very large search spaces. Although they do not guarantee optimality, heuristics offer a practical and efficient approach to problem-solving. However, when sacrificing optimality, more computationally efficient resolution methods should be obtained (GOLDBARG et al., 2015).
As for metaheuristics, Belfiore and Fávero (2012b) state that: The metaheuristic represents a combination of search procedures with higher-level strategies, including intensification and diversification, aiming to escape local optima Among the computational techniques are the following, as described by Goldbarg et al. (2015): 1. Local Search: is a metaheuristic that focuses on finding optimal solutions within a local search space. It seeks improvements by iteratively exploring neighboring solutions of the current solution. Its objective is to find the local optimum, even if it is not the global optimum. Local Search performs local movements, such as swapping or perturbing the current solution to evaluate its effect on the objective function. The algorithm continues until no further local improvements can be obtained.

Tabu Search:
is a heuristic search technique that seeks optimized solutions in a search space by prohibiting certain actions or search moves during the selection of solutions and exploration of neighborhoods. The term "tabu" comes from the Tongan language, spoken in Tonga, Polynesia, and means "something forbidden or sacred," although it does not perfectly represent the proposal of the approach. Tabu Search works by using variable configurations, which can be both feasible and infeasible solutions.
The goal is to find the best possible result while avoiding revisiting or repeating previously visited solutions.
3. Genetic Algorithm: Genetic Algorithms (GAs) and Memetic Algorithms (MAs) are based on the biological mimicry of multicellular sexual reproduction. These algorithms use the process of genetic variation within a population through reproduction and mutation. In reproduction, typically two parents (but more can be used) come together to produce one or more offspring, following the sexual reproduction paradigm. Although they use the form of sexual reproduction, genetic algorithms resemble more the chromosomes obtained in prokaryotic evolution, which mainly occurs through cloning with mutation, rather than chromosomes in sexual reproduction. In the classical mimicry of genetic algorithms, individuals are represented by chromosomes, and the reproduction process involves the mixing of genetic material from two or more individuals.

MATERIALS AND METHODS
For this study, six randomly generated instances were analyzed. The metaheuristics used were: LocalSearch, TabuSearch, and Genetic Algorithm, which were implemented in the Python language and executed in the Google Colaboratory cloud environment. It is important to note that the experiments were conducted on a computer with an Intel(R) Core(TM) i3-10110U CPU @ 2.10GHz/2.59 GHz processor, 12 GB of RAM, and Windows 11 operating system.
The analysis was based on the obtained results, including the interruption parameters of each metaheuristic method, as well as the optimal solution. To analyze the data, Analysis of Variance (ANOVA) was used as there were three samples with paired instances, ensuring the validation of the conclusions.

RESULTS AND CONCLUSIONS
In TABLE 1 presents the instances, the optimal solution for each instance, and their respective solutions calculated using the metaheuristics described in this study. To perform the ANOVA, the deviations of each metaheuristic from the optimal solution were calculated, including the mean, standard deviation, and variance (TABLE 2). The null hypothesis (H0) is defined, assuming that the distributions follow the same normal distribution, and the alternative hypothesis (H1) is that there is at least one distribution with a different mean from the others. By performing operations with the deviation columns (TABLE 2), the parameters described in TABLE 3 are observed. Now, as can be observed in the previous table, F = 2.766909 and critical F = 3.68232. Therefore, F < critical F, and thus, the p-value is above the significance level (0.05), leading to the acceptance of the null hypothesis for these instances, heuristics, and the mathematical model programmed for the used heuristics.
However, considering the comparison of processing time (TABLE 4) among the metaheuristics, a significant difference is observed in the time required to obtain the solutions described in TABLE 1. Applying the same analysis method described earlier (ANOVA), the following results are obtained and presented in TABLE 5. Using the values described as "Deviation" in the previous table, with the null hypothesis and alternative hypothesis as described in the ANOVA calculation for the results obtained through the heuristics, the Analysis of Variance was recalculated, resulting in the values described in TABLE 6. The calculated value of F is greater than the critical F value (Table 6), and the p-value is lower than the significance level (0.05), indicating the rejection of the null hypothesis. This demonstrates that one of the metaheuristics is more suitable than the others.
Considering that the Genetic Algorithm metaheuristic has an average of 0.210, which is the lowest among the employed methods, it can be concluded that it is the most recommended for optimizing the instances using the programmed models.
Based on the conducted experimental analysis, it is concluded that the Genetic Algorithm is the best metaheuristic for solving the Knapsack Problem with the observed instances. Although other metaheuristics have also shown good results, the Genetic Algorithm has demonstrated more robustness and stability compared to the others. It should be noted that the selection of the best technique depends on the problem's characteristics, such as size and complexity of the constraints.
Therefore, the use of the Genetic Algorithm for solving the Knapsack Problem proves to be a good option. This metaheuristic allows for an efficient solution and can be applied in various problem configurations. The results obtained in this study contribute to the selection of the best technique and provide insights for future research in the field of Combinatorial Optimization.