Bagging
Home > Ensembles
Bagging (Breiman, 1996), a name derived from “bootstrap aggregation”, was the first effective method of ensemble learning and is one of the simplest methods of arching [1]. The meta-algorithm, which is a special case of the model averaging, was originally designed for classification and is usually applied to decision tree models, but it can be used with any type of model for classification or regression. The method uses multiple versions of a training set by using the bootstrap, i.e. sampling with replacement. Each of these data sets is used to train a different model. The outputs of the models are combined by averaging (in case of regression) or voting (in case of classification) to create a single output. Bagging is only effective when using unstable (i.e. a small change in the training set can cause a significant change in the model) nonlinear models.
[1] Arching (adaptive reweighting and combining) is a generic term that refers to reusing or selecting data in order to improve classification.
Seminal Paper
BREIMAN, Leo, 1996. Bagging predictors , Machine Learning 24(2):123140. [Cited by 2460 ] (239.89/year)
Links
Bibliography
ABE, N. and H. MAMITSUKA, 1998. Query Learning Strategies Using Boosting and Bagging . Proceedings of the Fifteenth International Conference on …. [Cited by 46 ] (5.57/year)
ABE, N. and H. MAMITSUKA, 1998. Query learning using boosting and bagging. Proceedings of the 15th International Conference on Machine …. [Cited by 6 ] (0.73/year)
ALKOOT, F.M. and J. KITTLER, 2003. Population bias control for bagging k-NN experts . Proceedings of SPIE. [Cited by 4 ] (1.23/year)
ANTHONY, W.S. and D.J. HERBER, 1991. Moisture transfer of cotton bales covered with experimental bagging. Proceedings of the Beltwide Cotton Conferences, National …. [Cited by 9 ] (0.59/year)
ARAKAWA, O., 1988. … : changes in anthocyanin synthesis during maturation as affected by bagging and light quality. Journal of the Japanese Society for Horticultural Science. [Cited by 11 ] (0.60/year)
ARAKAWA, O., 1991. … anthocyanin accumulation in apple fruit as affected by cultivar, stage of fruit ripening and bagging. Journal of Horticultural Science. [Cited by 8 ] (0.52/year)
B?HLMANN, P. and B. YU, 2002. Analyzing bagging . The Annals of Statistics. [Cited by 67 ] (15.75/year)
BAGGING, R., A. Bagging ring shall be constructed per Drawing No. BAG-RING-101. The maximum roughness for all surfaces in …. [Cited by 1 ] (?/year)
BAUER, E. and R. KOHAVI, 1999. An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants . Machine Learning. [Cited by 576 ] (79.40/year)
BENTLEY, W.J. and M. VIVEROS, 1992. Brown-bagging Granny Smith apples on trees stops codling moth damage. California Agriculture. [Cited by 5 ] (0.35/year)
BORRA, S. and A. DI, 2002. Improving nonparametric regression methods by bagging and boosting . Computational Statistics and Data Analysis. [Cited by 11 ] (2.59/year)
BREIMAN, L., 1996. Bagging Predictors . Machine Learning. [Cited by 2460 ] (239.89/year)
BREIMAN, L., 1998. Out-of-bag estimation . Unpublished manuscript. [Cited by 54 ] (6.54/year)
BREIMAN, L., 1998. HALF&HALF BAGGING AND HARD BOUNDARY POINTS . [Cited by 15 ] (1.82/year)
BREIMAN, L., 2001. Using Iterated Bagging to Debias Regressions . Machine Learning. [Cited by 30 ] (5.71/year)
BREIMAN, L., 2001. Random Forests . Machine Learning. [Cited by 457 ] (86.97/year)
BREIMAN, L., lip;. Using adaptive bagging to debias regressions. Tec. rep., Statistics Department, University of California &h. [Cited by 35 ] (?/year)
BRIEM, G.J., J.A. BENEDIKTSSON and J.R. SVEINSSON, 2001. Boosting. Bagging and Consensus Based Classification of Multisource Remote Sensing Data . Multiple Classifier Systems. Second International Workshop, …. [Cited by 6 ] (1.14/year)
BRIEMAN, L., 1996. Bagging predictors. Machine Learning. [Cited by 25 ] (2.44/year)
BRYLL, R., R. GUTIERREZ-OSUNA and F. QUEK, 2003. Attribute bagging: improving accuracy of classi?er ensembles by using random feature subsets . Pattern Recognition. [Cited by 20 ] (6.14/year)
BUGANIC, R.D., M.C.C. LIZADA and M.B. DE, 1996. DISEASE CONTROL IN PHILIPPINE'CARABAO'MANGO WITH PREHARVEST BAGGING AND POSTHARVEST HOT WATER … . V International Mango Symposium 455. [Cited by 4 ] (0.39/year)
BUHLMANN, P. and B. YU, 2000. Explaining bagging . Preprint. [Cited by 14 ] (2.24/year)
BUJA, A. and W. STUETZLE, 2000. Smoothing effects of bagging . Preprint. AT&T Labs-Research. [Cited by 8 ] (1.28/year)
BUJA, A. and W. STUETZLE, 2000. The effect of bagging on variance, bias, and mean squared error. Preprint. AT&T Labs-Research. [Cited by 9 ] (1.44/year)
BYLANDER, T., 2002. Estimating Generalization Error on Two-Class Datasets Using Out-of-Bag Estimates . Machine Learning. [Cited by 7 ] (1.65/year)
CHAN, J.C.W., R. DEFRIES and C. HUANG, 2001. Enhanced algorithm performance for land cover classification from remotely sensed data using bagging … . IEEE Transactions on Geoscience and Remote Sensing. [Cited by 5 ] (0.95/year)
CHAWLA, N., et al. , 2001. Bagging is a small-data-set phenomenon . International Conference on Computer Vision and Pattern …. [Cited by 9 ] (1.71/year)
CHAWLA, N.V., et al. , 2003. Distributed learning with bagging-like performance . Pattern Recognition Letters. [Cited by 17 ] (5.22/year)
CHEN, S.X. and P. HALL, 2003. Effects of bagging and bias correction on estimators defined by estimating equations . Statistica Sinica. [Cited by 7 ] (2.15/year)
COUNCIL, M.C., P. BAG and M. CITY, Regional Options for Food Waste Composting . manukau.govt.nz. [not cited] (?/year)
DERBEKO, P., R. EL-YANIV and R. MEIR, 2002. Variance Optimized Bagging . ECML 2002. [Cited by 5 ] (1.18/year)
DIETTERICH, T.G., 2000. … of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization . Machine Learning. [Cited by 404 ] (64.59/year)
DOMINGOS, P., 1997. Why does bagging work? A Bayesian account and its implications . Proceedings of the Third International Conference on …. [Cited by 35 ] (3.78/year)
DRAPER, B. and K. BAEK, 1998. Bagging in Computer Vision . IEEE International Conference on Computer Vision and Pattern …. [Cited by 4 ] (0.48/year)
DRUCKER, H., et al. , 1997. Support vector regression machines . Advances in Neural Information Processing Systems. [Cited by 152 ] (16.42/year)
DUDOIT, S. and J. FRIDLYAND, 2003. Bagging to improve the accuracy of a clustering procedure . Bioinformatics. [Cited by 56 ] (17.21/year)
DUTRA, I.C., et al. , Proceedings of the 12th International Conference on …. An empirical evaluation of bagging in inductive logic programming . [Cited by 7 ] (?/year)
EMBRECHTS, M.J., et al. , Bagging neural network sensitivity analysis for feature reduction in QSAR problems. Proceedings of 2001 INNS-IEEE International Joint Conference …. [Cited by 6 ] (?/year)
ESPOSITO, R. and L. SAITTA, 2003. Monte Carlo Theory as an Explanation of Bagging and Boosting . Proceeding of the Eighteenth International Joint Conference …. [Cited by 11 ] (3.38/year)
FALLAHI, E., et al. , 2001. Influence of nitrogen and bagging on fruit quality and mineral concentrations of ???BC-2 Fuji ‘ …. Hort Technology. [Cited by 7 ] (1.33/year)
FAN, X. and J.P. MATTHEIS, 1998. Bagging'Fuji' apples during fruit development affects color development and storage quality . HortScience. [Cited by 5 ] (0.61/year)
FINER, N.N., et al. , 2001. Comparison of methods of bag and mask ventilation for neonatal resuscitation . Resuscitation. [Cited by 24 ] (4.57/year)
FISCHER, B. and J.M. BUHMANN, 2003. Bagging for path-based clustering . Pattern Analysis and Machine Intelligence, IEEE Transactions …. [Cited by 10 ] (3.07/year)
FRIEDMAN, J.H. and P. HALL, 2000. On bagging and nonlinear estimation . Preprint. [Cited by 31 ] (4.96/year)
FRIEDMAN, J.H., T. HASTIE and R. TIBSHIRANI, 2000. Additive logistic regression: a statistical view of boosting (With discussion and a rejoinder by the … . Ann. Statist. [Cited by 639 ] (102.16/year)
GEMAN, S., E. BIENENSTOCK and R. DOURSAT, 1992. Neural networks and the bias/variance dilemma . Neural Computation. [Cited by 1009 ] (70.78/year)
GENCAY, R. and M. QI, 2001. … derivative securities with neural networks: Bayesian regularization, early stopping, and bagging . Neural Networks, IEEE Transactions on. [Cited by 15 ] (2.85/year)
GLASS, C., et al. , 1993. Nurses'ability to achieve hyperinflation and hyperoxygenation with a manual resuscitation bag during … . Heart Lung. [Cited by 10 ] (0.75/year)
GRANDVALET, Y., 2004. Bagging Equalizes Influence . Machine Learning. [Cited by 7 ] (3.10/year)
HA, K., S. CHO and D. MACLACHLAN, 2004. Response models based on bagging neural networks . Journal of Interactive Marketing. [Cited by 5 ] (2.22/year)
HALL, L.O., et al. , Comparing Pure Parallel Ensemble Creation Techniques Against Bagging . ieeexplore.ieee.org. [Cited by 7 ] (?/year)
HENDERSON, J.C. and E. BRILL, 2000. Bagging and boosting a treebank parser . Proceedings of the first conference on North American …. [Cited by 19 ] (3.04/year)
HESKES, T., 1997. Balancing between bagging and bumping . Advances in Neural Information Processing Systems. [Cited by 26 ] (2.81/year)
HOFMAN, P.J., et al. , 1997. Bagging of mango (Mangifera indica cv.‘Keitt') fruit influences fruit quality and mineral …. Postharvest Biology and Technology. [Cited by 9 ] (0.97/year)
HOTHORN, T. and B. LAUSEN, 2002. Bagging combined classifiers . Classification, Clustering and Data Analysis: Recent …. [Cited by 4 ] (0.94/year)
HOTHORN, T. and B. LAUSEN, 2003. Bagging tree classifiers for laser scanning images: a data- and simulation-based strategy . Artificial Intelligence in Medicine. [Cited by 16 ] (4.92/year)
HOTHORN, T. and B. LAUSEN, 2003. Double-bagging: combining classi?ers by bootstrap aggregation . Pattern Recognition. [Cited by 26 ] (7.99/year)
HOTHORN, T. and B. LAUSEN, 2005. Bundling classifiers by bagging trees . Computational Statistics and Data Analysis. [Cited by 11 ] (8.77/year)
HOTHORN, T., et al. , 2004. Bagging survival trees . Statistics in Medicine. [Cited by 20 ] (8.87/year)
IBA, H., 1999. Bagging, boosting, and bloating in genetic programming . Proceedings of the Genetic and Evolutionary Computation …. [Cited by 19 ] (2.62/year)
JONES, A., et al. , 1992. Peak expiratory flow rates produced with the Laerdal and Mapleson-C bagging circuits. Aust J Physiother. [Cited by 8 ] (0.56/year)
JONES, A.Y.M., R.C. HUTCHINSON and T.E. OH, 1992. Effects of bagging and percussion on total static compliance of the respiratory system . Physiotherapy. [Cited by 24 ] (1.68/year)
JU, Z., 1998. Fruit bagging, a useful method for studying anthocyanin synthesis and gene expression in apples . Scientia Horticulturae. [Cited by 10 ] (1.21/year)
KITAGAWA, H., K. MANABE and E.B. ESGUERRA, 1991. BAGGING OF FRUIT ON THE TREE TO CONTROL DISEASE . Frontier in Tropical Fruit Research 321. [Cited by 5 ] (0.33/year)
KUBO, Y., et al. , 1988. … of 4 apple cultivars grown in the Southwest of Japan, with special reference to fruit bagging. Journal of the Japanese Society for Horticultural Science. [Cited by 20 ] (1.10/year)
KUHLBUSCH, T.A.J., S. NEUMANN and H. FISSAN, 2004. Number Size Distribution, Mass Concentration, and Particle Composition of PM1, PM2. 5, and PM10 in … . Journal of Occupational and Environmental Hygiene. [Cited by 6 ] (2.66/year)
KUNCHEVA, L.I., M. SKURICHINA and R.P.W. DUIN, 2002. An experimental study on diversity for bagging and boosting with linear classifiers . Information Fusion. [Cited by 14 ] (3.29/year)
LEE, H.K.H. and M.A. CLYDE, 2004. Lossless Online Bayesian Bagging . The Journal of Machine Learning Research. [Cited by 7 ] (3.10/year)
MACLIN, R. and D. OPITZ, 1997. An empirical evaluation of bagging and boosting . Proceedings of the Fourteenth National Conference on …. [Cited by 100 ] (10.81/year)
MAKI, D.G., C. ALVARADO and C. HASSEMER, 1986. … control measure: a comparative study of surface contamination with single-and double-bagging. . Infect Control. [Cited by 11 ] (0.54/year)
MAO, J., I.B.M.A.R. CENTER and C.A. SAN, 1998. A case study on bagging, boosting and basic ensembles of neuralnetworks for OCR . Neural Networks Proceedings, 1998. IEEE World Congress on …. [Cited by 22 ] (2.67/year)
MARDIN, C., et al. , 2003. New glaucoma classification method based on standard HRT parameters by bagging classification trees. Journal of Glaucoma. [Cited by 12 ] (3.69/year)
MCDONALD, R.E., L.A. RISSE and C.R. BARMORE, 1990. Bagging chopped lettuce in selected permeability films. HortScience. [Cited by 10 ] (0.62/year)
MYERS, W.R. and M.J. PEACH…, 1983. … MEASUREMENTS ON A POWERED AIR-PURIFYING RESPIRATOR MADE DURING ACTUAL FIELD USE IN A SILICA BAGGING … . Annals of Occupational Hygiene. [Cited by 7 ] (0.30/year)
NISHIMURA, H., et al. , 1999. … character recognition using HMM by multiple directional feature extraction and voting with bagging … . 5th Int. Conference on Document Analysis and Recognition. [Cited by 15 ] (2.07/year)
OPITZ, D.W. and R.F. MACLIN, 1997. An empirical evaluation of bagging and boosting for artificialneural networks . Neural Networks, 1997., International Conference on. [Cited by 8 ] (0.86/year)
OZA, N.C. and S. RUSSELL, 2001. Experimental comparisons of online and batch versions of bagging and boosting . Proceedings of the seventh ACM SIGKDD international …. [Cited by 9 ] (1.71/year)
OZA, N.C., 2005. Online bagging and boosting . Systems, Man and Cybernetics, 2005 IEEE International …. [Cited by 12 ] (9.56/year)
POGGIO, T., et al. , Bagging regularizes . cimvl.knu.ac.kr. [Cited by 5 ] (?/year)
QUINLAN, J.R. and B. BAGGING, C4. 5 In Proceedings of the Thirteenth National Conference on Artificial Intelligence, 1996. Available on line at: http://www. cse. unsw. edu. au/quinlan. [Cited by 1 ] (?/year)
QUINLAN, J.R., 1996. Bagging, boosting, and C4. 5 . Proceedings of the Thirteenth National Conference on …. [Cited by 456 ] (44.47/year)
QUINLAN, J.R., 1996. Boosting, bagging, and C4. 5. Proceedings of the Thirteenth National Conference on …. [Cited by 32 ] (3.12/year)
RIZZOLI, A., et al. , 2002. Geographical information systems and bootstrap aggregation (bagging) of tree-based classifiers for … . J Med Entomol. [Cited by 13 ] (3.06/year)
SAKAKIBARA, H., et al. , 1995. Overhead work and shoulder-neck pain in orchard farmers harvesting pears and apples. . Ergonomics. [Cited by 21 ] (1.87/year)
SILBERGLEIT, R., et al. , 1996. Sudden Severe Barotrauma from Self-Inflating Bag-Valve Devices . The Journal of Trauma: Injury, Infection, and Critical Care. [Cited by 7 ] (0.68/year)
SKURICHINA, M. and R.P.W. DUIN, … Systems (Proceedings Second International Workshop MCS 2001 …. Bagging and the random subspace method for redundant feature spaces . [Cited by 21 ] (?/year)
SKURICHINA, M. and R.P.W. DUIN, 1876. The Role of Combining Rules in Bagging and Boosting . Advances in Pattern Recognition (Proceedings Joint …. [Cited by 8 ] (0.06/year)
SKURICHINA, M. and R.P.W. DUIN, 1998. Bagging for linear classifiers . Pattern Recognition. [Cited by 35 ] (4.24/year)
SKURICHINA, M. and R.P.W. DUIN, 2002. Bagging, Boosting and the Random Subspace Method for Linear Classifiers . Pattern Analysis & Applications. [Cited by 44 ] (10.34/year)
SKURICHINA, M., L. KUNCHEVA and R.P.W. DUIN, Proc. Int. Workshop on Multiple Classifier Systems (LNCS …. Bagging and boosting for the nearest mean classifier: Effects of sample size on diversity and … . [Cited by 16 ] (?/year)
STRAZDIENE, E., V. DAUKANTIENE and M. GUTAUSKAS, 2003. Bagging of Thin Polymer Materials: Geometry, Resistance and Application . Materials Science (Med?iagotyra) ISSN 1392. [Cited by 4 ] (1.23/year)
TADESSE, T., et al. , 2001. Relative humidity around the fruit influences the mineral composition and incidence of blossom-end … . Journal of Horticultural Science & Biotechnology. [Cited by 4 ] (0.76/year)
TSYMBAL, A. and S. PUURONEN, 2000. Bagging and boosting with dynamic integration of classifiers . Proceedings of the International Conference on Principles of …. [Cited by 11 ] (1.76/year)
TYAS, J.A., et al. , 1998. Fruit canopy position and panicle bagging affects yield and quality of ‘Tai So'lychee. Scientia Horticulturae. [Cited by 5 ] (0.61/year)
WANG, H., O. ARAKAWA and Y. MOTOMURA, 2000. Influence of maturity and bagging on the relationship between anthocyanin accumulation and … . Postharvest Biology and Technology. [Cited by 9 ] (1.44/year)
WOLPERT, D.H. and W.G. MACREADY, 1998. Combining stacking with bagging to improve a learning algorithm . Machine Learning. [Cited by 17 ] (2.06/year)
YU, C. and D.B. SKILLICORN, 2001. Parallelizing Boosting and Bagging . Techincal Report Department of Computing and Information …. [Cited by 5 ] (0.95/year)
ZEMKE, S., 1999. Bagging imperfect predictors. Proceedings of Artificial Neural Networks in Engineering. [Cited by 7 ] (0.96/year)
ZHANG, X., Y. LI and K.W. YEUNG, 2000. Mathematical Simulation of Fabric Bagging. Textile Research Journal. [Cited by 5 ] (0.80/year)
ZHENG, Z. and G.I. WEBB, 1998. Multiple boosting: A combination of boosting and bagging . Proceedings of the 1998 International Conference on Parallel …. [Cited by 6 ] (0.73/year)