Ridge Regression is a technique which is specialized to analyze multiple regression data which is multicollinearity data. The term multicollinearity refers to high co-relations between more than two predicted values. In this phenomenon,one predicted value in multiple regression models is linearly predicted with others to attain a certain level of accuracy. Ridge regression is a regularization technique. Regularization is a process of adding objects to prevent overfitting. Regularization technique are as follows: 1.penalize the magnitude of coefficients of features. 2.minimize the error between the actual and predicted observations. Ridge regression is used to create a "parsimonious model" in the following way: 1.the number of predicted variables are more than the no. of observations. 2.the dataset has multicollinearity.
In this, penalty equivalent is added the square of the magnitude of coefficients. 'lamda' is turning factor that controls the strength of penalty term. Accuracy and error are based on lamda .
if lamda=0,then the coefficients are same as simple linear regression. if lamda=infinity,then the coefficients will become zero if 0<lamda<infinity,then the minimization objective= least square objective(lso)+lamda(sum of square of coefficients)
Lasso regression is one of the reguralization technique . It is also used to create "parsominious models" in the presence of the large no. of features. The large no. of features may cause overfitting or computational challenges. Lasso regression adds a factor of the sum of the absolute value of the coefficients the optimization objective. 'lamda' is turning factor that controls the strength of penalty. > if lamda=0,we get same coefficients as simple linear regression >if lamda=infinity,all coefficints shrinks to zero >if 0<lamda<infinity,then 1.fitting a linear model of y on x 2.shrinking the coefficients.
Lasso regression is able to perform variable selection in the linear model.
Elastic net regression combines the power of ridge and lasso regression into one algorithm. It means this is used to remove weak variables altogether as with lassoor to reduce them to close to zero as with ridge. In addition to set lamda value the elastic net allows us to tune the alpha parameter. >if alpha=0 ,then it corresponds to ridge regression >if alpha=1,then it corresponds to lasso regression >if 0<alpha<1,then it is to optimize the elastic net.
Effectively this will shrink some coefficients and set some to 0 for sparse selection.