Load the lars package and the New York Housing dataset (as in the attached file for download). Next, load the glmnet package that will be used to implement LASSO.
Save the MEDV as y and only use the first 12 variables (i. e., columns in New York Housing dataset) as x. While x is a set of independent variables, y is the dependent variable which is a quantitative measure of the median housing values. (Hints: can assign matrix format to the variable x).
Generate separate scatterplots with the line of best fit for all the predictors in x with y on the vertical axis.
Regress y on the predictors in x using OLS (Ordinary Least Square Regression). We will use this result as benchmark for comparison.
Use the glmnet function to plot the path of each of x's variable coefficients against the L1 norm of the beta vector. This graph indicates at which stage each coefficient shrinks to zero.
Use the cv. glmnet function to get the cross validation curve and the value of lambda that minimizes the mean cross validation error.
Using the minimum value of lambda from the previous exercise, get the estimated beta matrix.
To get a more parsimonious model we can use a higher value of lambda that is within one standard error of the minimum.
Use this value of lambda to get the beta coefficients. Note that more coefficients are now shrunk to zero.
Please refer to Example 6.1 and 6.2 of Chapter 6 in our textbook for details regarding how LASSO works so you know how to explain your results in this assignment.