--- title: "Estimating heterogeneous panels with csdm" author: "Joao Claudio Macosso" output: rmarkdown::html_vignette bibliography: "`r system.file('REFERENCES.bib', package = 'csdm')`" csl: "`r system.file('apa.csl', package = 'csdm')`" vignette: > %\VignetteIndexEntry{Estimating heterogeneous panels with csdm} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include=FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>", warning = FALSE, message = FALSE ) library(csdm) ``` ## Scope `csdm` estimates heterogeneous panel models in which unobserved common factors may generate dependence across units. The package implements: - mean group (MG) estimation [@PesaranSmith1995]; - static common correlated effects (CCE) [@Pesaran2006]; - dynamic CCE (DCCE) [@ChudikPesaran2015a]; and - a cross-sectionally augmented ARDL fit with implied adjustment and long-run parameters. These estimators share a mean-group structure: a separate regression is fitted for every eligible unit and the unit-level coefficients are averaged. CCE-based models augment those regressions with cross-sectional averages that proxy the latent common-factor space. The package adopts parts of the model structure used by Stata's `xtdcce2` [@Ditzen2018], but it does not claim complete command or option parity. Pooled restrictions, estimation weights, CS-DL, CS-ECM, alternative fit-level covariance estimators, and prediction on new data are currently unavailable. ## Data and sample The bundled `PWT_60_07` data contain 93 countries observed annually from 1960 through 2007. The variables follow the original `xtdcce2` example: - `log_rgdpo`: log real output; - `log_hc`: log human capital; - `log_ck`: log physical capital; and - `log_ngd`: log population growth plus a 5% break-even investment rate. The 93 values of `log_ngd` in 1960 are missing because its construction uses a growth rate. The worked example starts in 1970 and uses 15 countries to keep the vignette quick to build. ```{r data} data(PWT_60_07, package = "csdm") keep_ids <- unique(PWT_60_07$id)[1:15] dat <- subset(PWT_60_07, id %in% keep_ids & year >= 1970) dim(dat) range(dat$year) ``` The examples below are levels regressions. Calling them growth regressions would require a differenced dependent variable, which is not part of the formula used here. ## Estimators Let $i=1,\ldots,N$ index units and $t=1,\ldots,T$ index time. A heterogeneous panel model is $$ y_{it}=\alpha_i+\boldsymbol{\beta}_i'\mathbf{x}_{it}+u_{it}. $$ ### Mean group MG estimates the equation separately for each unit and averages the identified unit coefficients: $$ \widehat{\boldsymbol{\beta}}_{MG} =\frac{1}{N_e}\sum_{i\in\mathcal E}\widehat{\boldsymbol{\beta}}_i, $$ where $\mathcal E$ is the common set of eligible units and $N_e$ is its size. The reported covariance is the cross-unit sample covariance of the coefficient vectors divided by $N_e$. Inference uses a large-$N$ normal approximation. MG permits slope heterogeneity but does not model common-factor dependence. ### Common correlated effects CCE augments each unit equation with cross-sectional averages: $$ y_{it}=\alpha_i+\boldsymbol{\beta}_i'\mathbf{x}_{it} +\boldsymbol{\gamma}_i'\bar{\mathbf z}_t+e_{it}, $$ where $\bar{\mathbf z}_t$ commonly contains the averages of the dependent variable and regressors. Under the CCE assumptions, these averages span the relevant common-factor space and allow consistent estimation of the unit slopes [@Pesaran2006]. Adding averages does not by itself guarantee that the factor space is adequately represented; rank, dimensions, and the choice of averages still matter. ### Dynamic CCE DCCE adds dynamics and lags of the cross-sectional averages: $$ y_{it}=\alpha_i +\sum_{p=1}^{P}\phi_{ip}y_{i,t-p} +\sum_{q=0}^{Q}\boldsymbol{\beta}_{iq}'\mathbf{x}_{i,t-q} +\sum_{s=0}^{S}\boldsymbol{\delta}_{is}'\bar{\mathbf z}_{t-s} +e_{it}. $$ `csdm_lr(type = "ardl", ylags = P, xdlags = Q)` controls model lags, and `csdm_csa(lags = S)` controls lags of the averages. Dynamic mean-group estimates can have short-$T$ bias; CCE augmentation does not remove that bias. ### CS-ARDL output `model = "cs_ardl"` fits the same unit-level levels ARDL used by DCCE and then transforms its coefficients. For unit $i$, define $$ d_i=1-\sum_{p=1}^{P}\phi_{ip},\qquad \varphi_i=-d_i,\qquad \boldsymbol{\theta}_i= \frac{\sum_{q=0}^{Q}\boldsymbol{\beta}_{iq}}{d_i}. $$ The package reports $\varphi_i$ as the adjustment coefficient and $\boldsymbol{\theta}_i$ as the long-run ratio before computing their mean-group summaries. Ratios are undefined when $d_i$ is numerically zero. The implementation also stores AR-root diagnostics, but it does not automatically discard unstable units. These transformations do not fit a separate error-correction model and do not establish cointegration. The `levels` component is the fitted levels ARDL coefficient vector, rather than the complete transformed short-run ECM vector. ## Specifying and fitting models The formula contains the contemporaneous economic regressors. Two specification objects add CCE terms and dynamics: - `csdm_csa(vars, lags)` selects variables whose cross-sectional averages enter the model and their maximum lags; - `csdm_lr(type, ylags, xdlags)` selects lags of the dependent variable and regressors. With `vars = "_all"`, averages are constructed from the evaluated response and economic model-matrix columns, excluding intercepts and unit trends. An explicit character vector instead refers to numeric columns in `data`. `vars = "_none"` turns off CCE augmentation where the chosen estimator permits it. ```{r specifications} form <- log_rgdpo ~ log_hc + log_ck + log_ngd csa_vars <- c("log_rgdpo", "log_hc", "log_ck", "log_ngd") static_csa <- csdm_csa(vars = csa_vars) dynamic_csa <- csdm_csa(vars = csa_vars, lags = 3) ardl_1_0 <- csdm_lr(type = "ardl", ylags = 1, xdlags = 0) ``` ### MG ```{r mg} fit_mg <- csdm( form, data = dat, id = "id", time = "year", model = "mg" ) summary(fit_mg) ``` ### CCE ```{r cce} fit_cce <- csdm( form, data = dat, id = "id", time = "year", model = "cce", csa = static_csa ) summary(fit_cce) ``` ### DCCE ```{r dcce} fit_dcce <- csdm( form, data = dat, id = "id", time = "year", model = "dcce", csa = dynamic_csa, lr = ardl_1_0 ) summary(fit_dcce) ``` ### CS-ARDL ```{r cs-ardl} fit_cs_ardl <- csdm( form, data = dat, id = "id", time = "year", model = "cs_ardl", csa = dynamic_csa, lr = ardl_1_0 ) summary(fit_cs_ardl) coef(fit_cs_ardl, component = "long_run") ``` The four calls use the same levels formula, so their coefficient meanings are directly comparable only after accounting for their different dynamic and CCE terms. In particular, a contemporaneous coefficient in DCCE is not a long-run effect. ## Samples, missing values, and identification `csdm()` requires unique, nonmissing unit-time keys. By default, numeric time indexes advance in steps of one; set `time_step` when the intended grid differs. Lags respect gaps rather than treating the previous observed row as the previous period. `subset` is evaluated before model construction. The supported missing-value policies are `na.omit`, `na.exclude`, and `na.fail`. Unit regressions must retain positive residual degrees of freedom, and every economic coefficient must be identified after projecting out the CCE terms. Units that fail these checks are listed with reasons in `fit$units`. At least two eligible units are required. For `csa = csdm_csa("_all")`, the source sample for cross-sectional averages is the complete base formula sample after subsetting and before dynamic lag trimming. It therefore need not equal the final set of fitted observations. Set `fullsample = TRUE` to calculate each average from all finite observations of that variable in the selected sample. This is useful when the averaging variables have different missing-value patterns and mirrors the `fullsample` option used in `xtdcce2`. ## Inference and R model methods Standard methods expose the fitted model and its sample: ```{r methods} coef(fit_cce) sqrt(diag(vcov(fit_cce))) nobs(fit_cce) head(model.frame(fit_cce)) head(residuals(fit_cce, format = "long")) head(fitted(fit_cce, format = "vector")) ``` For CS-ARDL, `coef()` and `vcov()` accept `component = "levels"`, `"adjustment"`, `"long_run"`, or `"all"`. Every component uses a common eligible-unit sample for its coefficients and covariance. Exact algebraic relationships can make the combined `"all"` covariance singular. The package also supplies `tidy()`, `glance()`, and `augment()` methods: ```{r tidy, eval=FALSE} broom::tidy(fit_cce, conf.int = TRUE) broom::glance(fit_cce) broom::augment(fit_cce) modelsummary::modelsummary( list(MG = fit_mg, CCE = fit_cce, DCCE = fit_dcce), statistic = "std.error" ) ``` ## Residual cross-sectional dependence Let $e_{it}$ denote the fitted residual and $T_{ij}$ the number of overlapping finite observations for units $i$ and $j$. The classical statistic implemented by `cd_test()` is $$ CD=\sqrt{\frac{2}{N(N-1)}} \sum_{i2\sqrt{\log(N)/T}\right\}. $$ The threshold is applied to the ordinary residual correlation, without a $\sqrt{T}$ multiplier. The enhancement is nonnegative, so CDw+ is not a second independent random-sign test. CD* applies the Pesaran-Xie bias correction after removing `n_pc` principal components [@PesaranXie2021]. Its approximation requires a nondegenerate bias-correction denominator; a numerically computable result alone does not establish that the asymptotic assumptions are suitable. ```{r cd-tests} cd_test(fit_mg, type = "CD") cd_test(fit_cce, type = "all", seed = 42) ``` Periods with no finite residual for any retained unit are outside the effective residual sample and are removed automatically. Partially observed periods are handled as follows: - classical CD remains pairwise-complete under the default `na.action = "pairwise"`; - CDw and CDw+ require a balanced matrix and otherwise error; - CD* requires a balanced matrix and otherwise returns `NA` with a warning. To evaluate all diagnostics on one common time sample, use: ```{r cd-balanced, eval=FALSE} cd_test( fit_cce, type = "all", seed = 42, na.action = "drop.incomplete.times" ) ``` A rejection concerns the residual dependence targeted by the selected test; it does not by itself identify the source of misspecification. A non-rejection does not prove residual independence, particularly in small samples or weak-power settings. ## Practical limits - Large-$N$ normal inference does not correct short-$T$ dynamic bias. - CCE validity requires enough informative averages and suitable factor and loading conditions. - Long-run ratios require a stable, nonzero AR denominator and an economically defensible long-run interpretation. - Reported mean-group fit statistics summarize unit regressions; they are not pooled-regression goodness-of-fit measures. - Saved fits from older releases should be refitted because corrected samples, transformations, and covariance calculations can change results. ## References ::: {#refs} :::