This tutorial shows how to fit a simple regression model (that is, a linear regression with a single independent variable) using Stata. The details of the underlying calculations can be found in our simple regression tutorial. The data used in this post come from the More Tweets, More Votes: Social Media as a Quantitative Indicator of Political Behavior study from DiGrazia J, McKelvey K, Bollen J, Rojas F (2013), which investigated the relationship between social media mentions of candidates in the 2010 and 2012 US House elections with actual vote results. The replication data in Stata format can be downloaded from our github repo.
In this example, we will assess the relationship between the percentage of social media posts that mention a Congressional candidate and how well the candidates did in the next election. The variables of interest are:
vote_share
(dependent variable): The percent of votes for a Republican candidatemshare
(independent variable): The percent of social media posts for a Republican candidate
Both variables are measured as percentages ranging from zero to 100.
Data Visualization
It is always a good idea to begin any statistical modeling with a graphical assessment of the data. This allows you to quickly examine the distributions of the variables and check for possible outliers. The following code returns a histogram for the vote_share
variable, our outcome of interest.
label var vote_share "Vote Share"
label var mshare "Tweet Share"
histogram vote_share, freq kdensity
We start off labeling our variables so that the figure displays informative labels rather than the variable name. The freq
option requests that the yaxis show the frequency with which the binned values occur in the data, the default is to show the density. The kdensity
option provides a kernel density line, which is a smoothed version of the histogram. The variable’s values (xaxis) fall within the range we expect. There is some negative skew in the distribution.
We can do the same thing for our independent variable.
histogram mshare, freq kdensity
We again see that the values fall into the range we expect. Note that there are also spikes at zero and 100. These indicate races where a single candidate received either all of the share of Tweets or none of the share of Tweets.
It is also helpful to look at the bivariate association between the two variables. This allows us to see whether there is visual evidence of a relationship, which will help us assess whether the regression results we ultimately get make sense given what we see in the data. The syntax to use is the following.
graph twoway (scatter vote_share mshare, msize(vsmall)) ///
(lfit vote_share mshare)
Here we are looking at a scatterplot of our observations, and we’ve also requested the best linear fit (i.e. the regression line) to better see the positive relationship. The msize
option makes the dots in the figure “very small”, which arguably looks a little nicer given the number of observations. The ///
allows the code to span multiple lines if we are running this from a do file. There is a clear, positive association between these variables.
Running the Regression
The following syntax runs the regression.
regress vote_share mshare
This returns:
Source  SS df MS Number of obs = 406
+ F(1, 404) = 141.17
Model  30146.6233 1 30146.6233 Prob > F = 0.0000
Residual  86273.9002 404 213.549258 Rsquared = 0.2589
+ Adj Rsquared = 0.2571
Total  116420.523 405 287.458083 Root MSE = 14.613

vote_share  Coef. Std. Err. t P>t [95% Conf. Interval]
+
mshare  .2685346 .0226011 11.88 0.000 .2241041 .3129651
_cons  37.04239 1.345062 27.54 0.000 34.39819 39.68658

The box at the top left provides us with an ANOVA table that gives 1) the sum of squares (SS
) for the model, often called the regression sum of squares, 2) the residual sum of squares, and 3) the total sum of squares. Dividing the SS
column by the df
(degrees of freedom) column returns the mean squares in the MS
column. These values go into calculating the \(F\)statistic, \(R^2\), adjusted \(R^2\), and Root Mean Square Error shown in the top right of the output.
Looking at the top right, we see that the number of observations used to fit the model was 406. The \(F\)statistic tests the null hypothesis that the independent variable does not help explain any variance in the outcome. We clearly reject the null hypothesis with \(p < 0.001\), as seen by Prob > F = 0.0000
. The Rsquared
value tells us that the independent variable explains 25.89% of the variation in the outcome. The adjusted \(R^2\) provides a slightly more conservative estimate of the percentage of variance explained, 25.71%. The Root MSE
is the square root of the residual MS from the top left table, \(\sqrt{213.549258} = 14.613\). This value gives a summary of how much the observed values vary around the predicted values, with better models having lower RMSEs. It is also used in the formula for the standard error of the coefficient estimates, shown in the next table.
The final table tells us the results of the regression model. For each increase of one on the mshare
variable, the vote share increases by 0.269. The standard error tells us how much sampletosample variability we should expect. Dividing the coefficient by the standard error gives us the \(t\)statistic used to calculate the \(p\)value. Here we see that the mshare
coefficient estimate is easily significant, \(p < 0.001\). The 95% confidence interval for the coefficient is also presented.
The bottom line in the table gives us the estimate for the intercept in the simple regression equation. This is the vote share we expect when Tweet share equals zero. Here we see that the predicted value is 37.04, which coincides with what we saw above in the scatterplot, and has a 95% confidence interval of [34.398, 39.687]. The estimated constant value is significantly different from zero, \(p < 0.001\), though this test is of less interest to us compared to assessing the significance of the regression line slope.
Fun Facts about Simple Regression
In a simple regression only (that is, when there is just a single independent variable), the \(R^2\) is exactly equal to the Pearson correlation between the two variables. To see this, run:
pwcorr vote_share mshare, sig
We use the pwcorr
(pairwise correlation) with the sig
option to get a \(p\)value with the estimate.
 vote_s~e mshare
+
vote_share  1.0000


mshare  0.5089 1.0000
 0.0000

The correlation between Tweet share and vote share is 0.5089. If we square this, we get
\[ 0.5089^2 = 0.2589, \]
which is the same as the \(R^2\) value from the regression.
Also in simple regression only, the model \(F\)test is the same as the test for the single independent variable. A \(t\)statistic with \(k\) degrees of freedom is equal to an \(F\)statistic with 1 and \(k\) degrees of freedom. When there are no other predictors in the model, the square root of \(F\) will equal the \(t\) for our coefficient,
\[ \sqrt{141.17} = 11.88. \]
For more detailed information on where these numbers come from, consult our simple regression tutorial.