Methods of Determining the Number of Clusters in a Data Set and a New Clustering Criterion

dc.contributor.authorYan, Mingjinen
dc.contributor.committeechairYe, Keyingen
dc.contributor.committeememberPrins, Samantha C. Batesen
dc.contributor.committeememberSpitzner, Dan J.en
dc.contributor.committeememberSmith, Eric P.en
dc.contributor.departmentStatisticsen
dc.date.accessioned2014-03-14T20:19:52Zen
dc.date.adate2005-12-29en
dc.date.available2014-03-14T20:19:52Zen
dc.date.issued2005-11-28en
dc.date.rdate2006-12-29en
dc.date.sdate2005-12-06en
dc.description.abstractIn cluster analysis, a fundamental problem is to determine the best estimate of the number of clusters, which has a deterministic effect on the clustering results. However, a limitation in current applications is that no convincingly acceptable solution to the best-number-of-clusters problem is available due to high complexity of real data sets. In this dissertation, we tackle this problem of estimating the number of clusters, which is particularly oriented at processing very complicated data which may contain multiple types of cluster structure. Two new methods of choosing the number of clusters are proposed which have been shown empirically to be highly effective given clear and distinct cluster structure in a data set. In addition, we propose a sequential type of clustering approach, called multi-layer clustering, by combining these two methods. Multi-layer clustering not only functions as an efficient method of estimating the number of clusters, but also, by superimposing a sequential idea, improves the flexibility and effectiveness of any arbitrary existing one-layer clustering method. Empirical studies have shown that multi-layer clustering has higher efficiency than one layer clustering approaches, especially in detecting clusters in complicated data sets. The multi-layer clustering approach has been successfully implemented in clustering the WTCHP microarray data and the results can be interpreted very well based on known biological knowledge. Choosing an appropriate clustering method is another critical step in clustering. K-means clustering is one of the most popular clustering techniques used in practice. However, the k-means method tends to generate clusters containing a nearly equal number of objects, which is referred to as the ``equal-size'' problem. We propose a clustering method which competes with the k-means method. Our newly defined method is aimed at overcoming the so-called ``equal-size'' problem associated with the k-means method, while maintaining its advantage of computational simplicity. Advantages of the proposed method over k-means clustering have been demonstrated empirically using simulated data with low dimensionality.en
dc.description.degreePh. D.en
dc.identifier.otheretd-12062005-153906en
dc.identifier.sourceurlhttp://scholar.lib.vt.edu/theses/available/etd-12062005-153906/en
dc.identifier.urihttp://hdl.handle.net/10919/29957en
dc.publisherVirginia Techen
dc.relation.haspartProposal-Face.pdfen
dc.rightsIn Copyrighten
dc.rights.urihttp://rightsstatements.org/vocab/InC/1.0/en
dc.subjectGap statisticen
dc.subjectMulti-layer clusteringen
dc.subjectDD-weighted gap statisticen
dc.subjectCluster analysisen
dc.subjectWeighted gap statisticen
dc.subjectNumber of clustersen
dc.subjectK-means clusteringen
dc.titleMethods of Determining the Number of Clusters in a Data Set and a New Clustering Criterionen
dc.typeDissertationen
thesis.degree.disciplineStatisticsen
thesis.degree.grantorVirginia Polytechnic Institute and State Universityen
thesis.degree.leveldoctoralen
thesis.degree.namePh. D.en

Files

Original bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
Proposal-Face.pdf
Size:
938.17 KB
Format:
Adobe Portable Document Format