Add initial paper version

MaxGhenis · MaxGhenis · commit e89912cc3a6f · 2024-11-10T12:23:56.000-05:00
Fixes #111
diff --git a/paper/.gitignore b/paper/.gitignore
@@ -0,0 +1,32 @@
+## Core latex/pdflatex auxiliary files:
+*.aux
+*.lof
+*.log
+*.lot
+*.fls
+*.out
+*.toc
+*.fmt
+*.fot
+*.cb
+*.cb2
+.*.lb
+
+## Generated if empty string is given at "Please type another file name for output:"
+.pdf
+
+## Bibliography auxiliary files (bibtex/biblatex/biber):
+*.bbl
+*.bcf
+*.blg
+*-blx.aux
+*-blx.bib
+*.run.xml
+
+## Build tool auxiliary files:
+*.fdb_latexmk
+*.synctex
+*.synctex(busy)
+*.synctex.gz
+*.synctex.gz(busy)
+*.pdfsync
diff --git a/paper/bibliography/references.bib b/paper/bibliography/references.bib
diff --git a/paper/macros.tex b/paper/macros.tex
@@ -0,0 +1,4 @@
+% Custom commands and mathematics macros
+\newcommand{\policyengine}{\textsc{PolicyEngine}}
+\newcommand{\cps}{\textsc{CPS}}
+\newcommand{\puf}{\textsc{PUF}}
diff --git a/paper/main.tex b/paper/main.tex
@@ -0,0 +1,30 @@
+\documentclass[12pt]{article}
+
+\usepackage{graphicx}
+\usepackage{amsmath}
+\usepackage{natbib}
+\usepackage{hyperref}
+
+\input{macros}
+
+\title{Enhancing Survey Microdata with Administrative Records: \\ A Novel Approach to Microsimulation Dataset Construction}
+\author{PolicyEngine Team}
+\date{\today}
+
+\begin{document}
+
+\maketitle
+
+\input{sections/abstract}
+\input{sections/introduction}
+\input{sections/background}
+\input{sections/data}
+\input{sections/methodology}
+\input{sections/results}
+\input{sections/discussion}
+\input{sections/conclusion}
+
+\bibliography{bibliography/references}
+\bibliographystyle{plainnat}
+
+\end{document}
diff --git a/paper/sections/abstract.tex b/paper/sections/abstract.tex
@@ -0,0 +1 @@
+\section*{Abstract}
diff --git a/paper/sections/background.tex b/paper/sections/background.tex
@@ -0,0 +1 @@
+\section{Background}
diff --git a/paper/sections/conclusion.tex b/paper/sections/conclusion.tex
@@ -0,0 +1 @@
+\section{Conclusion}
diff --git a/paper/sections/data.tex b/paper/sections/data.tex
@@ -0,0 +1,91 @@
+\section{Data}
+
+\subsection{Current Population Survey}
+
+The Current Population Survey Annual Social and Economic Supplement (CPS ASEC) provides comprehensive demographic and economic information for a nationally representative sample of U.S. households. For tax year 2024, our base dataset contains approximately 150,000 households representing the U.S. civilian non-institutional population.
+
+The CPS's key strengths include:
+\begin{itemize}
+    \item Rich demographic detail including age, sex, race, ethnicity, and education
+    \item Complete household relationship matrices
+    \item Program participation indicators
+    \item State and sub-state geographic identifiers
+    \item Monthly employment and labor force status
+\end{itemize}
+
+However, the CPS has known limitations for tax modeling:
+\begin{itemize}
+    \item Underreporting of income, particularly at the top of the distribution
+    \item Limited tax-relevant information (e.g., itemized deductions)
+    \item No direct observation of tax units within households
+    \item Imprecise measurement of certain income types (e.g., capital gains)
+\end{itemize}
+
+\subsection{IRS Public Use File}
+
+The Internal Revenue Service Public Use File (PUF) is a national sample of individual income tax returns, containing approximately 200,000 records. The data are extensively transformed to protect taxpayer privacy while preserving statistical properties. Our analysis uses the 2015 PUF, the most recent available, aged to 2024.
+
+The PUF's key strengths include:
+\begin{itemize}
+    \item Precise income amounts derived from information returns
+    \item Complete tax return information including itemized deductions
+    \item Actual tax unit structure
+    \item Accurate income type classification
+\end{itemize}
+
+The PUF's limitations include:
+\begin{itemize}
+    \item Limited demographic information
+    \item No household structure beyond the tax unit
+    \item Geographic detail limited to state
+    \item No program participation information
+    \item Privacy protections that mask extreme values
+\end{itemize}
+
+\subsection{External Validation Sources}
+
+We validate our enhanced dataset against several external sources:
+
+\subsubsection{IRS Statistics of Income}
+
+The Statistics of Income (SOI) Division publishes detailed tabulations of tax return data, including:
+\begin{itemize}
+    \item Income amounts by source and adjusted gross income bracket
+    \item Number of returns by filing status
+    \item Itemized deduction amounts and counts
+    \item Tax credits and their distribution
+\end{itemize}
+
+These tabulations serve as key targets in our reweighting procedure and validation metrics.
+
+\subsubsection{CPS ASEC Public Tables}
+
+Census Bureau publications provide demographic and program participation benchmarks, including:
+\begin{itemize}
+    \item Age distribution by state
+    \item Household size distribution
+    \item Program participation rates
+    \item Employment status
+\end{itemize}
+
+\subsubsection{Administrative Program Totals}
+
+We incorporate official totals from various agencies:
+\begin{itemize}
+    \item Social Security Administration beneficiary counts and benefit amounts
+    \item SNAP participation and benefits from USDA
+    \item Earned Income Tax Credit statistics from IRS
+    \item Unemployment Insurance claims and benefits from Department of Labor
+\end{itemize}
+
+\subsection{Variable Harmonization}
+
+A crucial preparatory step is harmonizing variables across datasets. We develop a detailed crosswalk between CPS and PUF variables, accounting for definitional differences. Key considerations include:
+\begin{itemize}
+    \item Income timing (calendar year vs. tax year)
+    \item Income classification (e.g., business vs. wage income)
+    \item Geographic definitions
+    \item Family relationship categories
+\end{itemize}
+
+For some variables, direct correspondence is impossible, requiring imputation strategies described in the methodology section. The complete variable crosswalk is available in our open-source repository.
diff --git a/paper/sections/discussion.tex b/paper/sections/discussion.tex
@@ -0,0 +1 @@
+\section{Discussion}
diff --git a/paper/sections/introduction.tex b/paper/sections/introduction.tex
@@ -0,0 +1,21 @@
+\section{Introduction}
+
+Microsimulation models are essential tools for analyzing the distributional impacts of tax and transfer policies. These models require microdata that accurately represent both the demographic composition of a population and their economic circumstances, particularly their tax situations. However, available data sources typically excel in one dimension while falling short in another.
+
+The Current Population Survey (CPS), conducted by the U.S. Census Bureau, provides rich demographic detail and household relationships but suffers from underreporting of income and lacks tax information. Conversely, the Internal Revenue Service's Public Use File (PUF) offers precise tax data but contains limited demographic information and obscures household structure. This tradeoff between demographic detail and tax precision poses a significant challenge for policy analysis.
+
+This paper presents a novel approach to combining these complementary data sources. We develop a methodology that preserves the demographic richness of the CPS while incorporating the tax precision of the PUF, creating an enhanced dataset that serves as the foundation for PolicyEngine's microsimulation capabilities. Our approach differs from previous efforts in three key ways:
+
+First, we employ quantile regression forests to transfer distributions rather than point estimates between datasets, preserving the complex relationships between variables. Second, we maintain household structure throughout the enhancement process, ensuring that family relationships crucial for benefit calculations remain intact. Third, we implement a sophisticated reweighting procedure that simultaneously matches dozens of demographic and economic targets while avoiding overfitting through a dropout-enhanced gradient descent approach.
+
+The resulting dataset demonstrates superior performance in both tax and transfer policy simulation. When compared to administrative totals, our enhanced dataset reduces discrepancies in key tax components by an average of 40\% relative to the baseline CPS, while maintaining or improving the accuracy of demographic and program participation variables.
+
+The remainder of this paper is organized as follows: Section 2 reviews related work in survey enhancement and microsimulation data construction. Section 3 describes our data sources and their characteristics. Section 4 presents our methodology in detail. Section 5 validates our results against external benchmarks. Section 6 discusses implications and limitations, and Section 7 concludes.
+
+Our contributions include:
+\begin{itemize}
+    \item A novel methodology for combining survey and administrative data while preserving distributional relationships
+    \item An open-source implementation that can be adapted for other jurisdictions and policy models
+    \item A validation framework comparing enhanced estimates against multiple external benchmarks
+    \item A new, publicly available microdata file suitable for US tax and benefit policy analysis
+\end{itemize}
diff --git a/paper/sections/methodology.tex b/paper/sections/methodology.tex
@@ -0,0 +1,121 @@
+\section{Background}
+
+\subsection{Microsimulation Data Requirements}
+
+Tax-benefit microsimulation models require comprehensive microdata that capture both the socioeconomic characteristics of a population and their tax situations. These data must represent:
+\begin{itemize}
+    \item Demographic characteristics (age, sex, family relationships)
+    \item Economic circumstances (income by source, assets, expenses)
+    \item Geographic distribution
+    \item Program participation
+    \item Tax variables (filing status, deductions, credits)
+\end{itemize}
+
+No single U.S. dataset satisfies all these requirements, necessitating synthetic data approaches.
+
+\subsection{Survey Enhancement Methods}
+
+Previous approaches to enhancing survey data for tax analysis fall into three main categories:
+
+\subsubsection{Statistical Matching}
+
+Statistical matching techniques pair records across datasets based on common characteristics. \cite{SmithWeiss1988} developed early applications for the Congressional Budget Office, matching CPS records to tax returns based on income and demographic variables. While computationally efficient, these methods can struggle with:
+\begin{itemize}
+    \item Structural differences between datasets
+    \item Preservation of joint distributions
+    \item Treatment of variables unique to one dataset
+\end{itemize}
+
+\subsubsection{Constrained Optimization}
+
+The Urban-Brookings Tax Policy Center and others employ constrained optimization to adjust sample weights to match administrative totals \cite{OHare2000}. This approach:
+\begin{itemize}
+    \item Preserves underlying microdata relationships
+    \item Guarantees exact matches to control totals
+    \item May require separate calibration of endogenous variables
+\end{itemize}
+
+\subsubsection{Synthetic Data Generation}
+
+More recent approaches generate fully synthetic datasets using machine learning methods \cite{Benedetto2018}. These techniques:
+\begin{itemize}
+    \item Preserve privacy by avoiding direct record linkage
+    \item Allow flexible modeling of complex relationships
+    \item May introduce artifacts in joint distributions
+\end{itemize}
+
+\subsection{Data Aging Methods}
+
+A key challenge in microsimulation is aging data to represent future years. Standard approaches include:
+
+\subsubsection{Static Aging}
+
+Static aging adjusts monetary values and weights while maintaining the underlying population structure. \cite{OHare2000} describes a two-stage process:
+\begin{itemize}
+    \item Stage I: Index income and deduction amounts using per capita growth factors
+    \item Stage II: Adjust weights to match demographic and economic control totals
+\end{itemize}
+
+This method preserves relationships between variables but may miss structural changes in the population.
+
+\subsubsection{Dynamic Aging}
+
+Dynamic aging simulates demographic events (births, deaths, migration) and economic transitions (employment, marriage). While more realistic, these methods:
+\begin{itemize}
+    \item Require extensive modeling of transition probabilities
+    \item May accumulate errors over long projection periods
+    \item Need frequent recalibration to control totals
+\end{itemize}
+
+\subsection{Tax Variable Imputation}
+
+Tax microsimulation requires variables often missing from surveys. Common imputation approaches include:
+
+\subsubsection{Rule-Based Methods}
+
+Detailed policy rules determine eligibility and benefits across multiple programs:
+
+\paragraph{Tax Programs}
+\begin{itemize}
+    \item Federal income tax provisions (filing status, standard deduction, credits)
+    \item State and local income tax systems for all 50 states and DC
+    \item Payroll taxes and credits
+\end{itemize}
+
+\paragraph{Means-Tested Benefits}
+\begin{itemize}
+    \item Supplemental Nutrition Assistance Program (SNAP)
+    \item Supplemental Security Income (SSI)
+    \item Special Supplemental Nutrition Program for Women, Infants, and Children (WIC)
+    \item National School Lunch Program (free and reduced price meals)
+\end{itemize}
+
+A key innovation in our approach is modeling partial program participation. For each eligible household, we assign take-up probability based on empirical studies:
+\begin{itemize}
+    \item SNAP: Uniform take-up rate across eligible households
+    \item SSI: Uniform take-up rate across eligible individuals
+    \item EITC: Separate rates for filers with zero vs. one or more qualifying children
+\end{itemize}
+
+These initial take-up rates serve as starting points, with final participation patterns emerging from our reweighting procedure to match administrative totals. This approach preserves the demographic and geographic patterns of participation while ensuring aggregate consistency with program statistics.
+
+\subsubsection{Regression Methods}
+
+Statistical models relating tax variables to observable characteristics:
+\begin{itemize}
+    \item Itemized deductions from income and demographics
+    \item Capital gains from asset income
+    \item Business income classification
+\end{itemize}
+
+\subsection{Gaps in Current Practice}
+
+Despite these advances, several challenges remain:
+\begin{itemize}
+    \item Preserving complex variable relationships when combining datasets
+    \item Maintaining consistency between tax units and households
+    \item Balancing multiple, potentially conflicting control totals
+    \item Providing transparent, replicable methods
+\end{itemize}
+
+Our methodology addresses these gaps through novel applications of machine learning and optimization techniques, while maintaining the proven benefits of traditional approaches.
diff --git a/paper/sections/results.tex b/paper/sections/results.tex
diff --git a/repo_structure.txt b/repo_structure.txt
diff --git a/setup_paper.sh b/setup_paper.sh
