FIDOOP – FIM: DATA SEGREGATION USING FREQUENT ITEM SETS MINING AND MAP REDUCE ALGORITHM

Abstract

Hadoop is an open-source platform of the MapReduce programming model. As data size is increasing gradually day by day, the improvement and security of data structure in Hadoop have become a critical issue. So far, algorithms have lacked of mechanisms like data distribution, fault tolerance, load balancing and input-output overhead. Hence, in order to overcome these discrepancies, the most effective the new method is the FiDoop method using a Map Reduce programming model and FIM algorithm. FiDoop includes the Frequent Item set Ulta metric Tree rather than conventional FP-trees which avoid the necessity to build conditional pattern based on compressed storage. In prior techniques such as pfp, Fidoop and Fidoop-HD the execution time was considerably increasing when the number of records increased. In our proposed system, firstly, the input/output overhead is minimized by scanning the database twice. Secondly, FIUT matric which is an outcome of clustering improves in the partitioning of database and significantly reduce the search space. Later MapReduce plays main act in parallel mining process; mappers separately decay item sets while its reducers make tiny ultra-metric trees to be individually mined. Finally, Fidoop - FIM based on FIM algorithm highly reduces the execution speed of downloading as the number of records increases in size.