Perbandingan Kinerja Metode Metropolis-Hastings dan Gibbs-Sampling dalam Penaksiran Parameter Model Regresi Logistik Multinomial

Abstract

Parameter estimation in multinomial logistic regression models often faces challenges when sample sizes are limited or when predictor variables exhibit high multicollinearity, which can reduce estimation accuracy and stability. These conditions highlight the need for a Bayesian analysis approach capable of producing reliable and stable parameter estimates while maintaining prediction quality even under nonideal data conditions. This study aims to compare the performance of two Markov Chain Monte Carlo (MCMC) methods, namely Metropolis-Hastings (MH) and Gibbs Sampling with Polya-Gamma augmentation (Gibbs-PG). MH utilizes a symmetric Gaussian proposal distribution to efficiently explore the parameter space, whereas Gibbs-PG employs Polya-Gamma augmentation to reduce autocorrelation and improve mixing. The data include the Iris Dataset (three classes, four predictors) and four simulated datasets with varying sample sizes (n = 50 and n = 500) and multicollinearity levels (ρ = 0.2 and 0.95). Performance evaluation considers estimation stability (standard error and confidence interval width), sampling efficiency (Effective Sample Size, ESS), and classification accuracy for category prediction. The results indicate that Gibbs-PG produces more stable estimates, with standard errors 30–50% lower than MH under high multicollinearity conditions. In terms of sampling efficiency, Gibbs-PG achieves ESS values between 15,000 and 16,000, substantially higher than MH (27–88), indicating lower autocorrelation and more efficient mixing. Classification accuracy reaches 96.67% for the Bayesian methods using Gibbs-PG, outperforming Maximum Likelihood Estimation (MLE), which achieves 93.33%. Overall, Gibbs-PG proves to be the most reliable method for precise and stable parameter estimation, especially for datasets with high multicollinearity or limited sample sizes, while MH remains a computationally lighter alternative.