Correlation is the relationship between different phenomena represented by the values ​​of two or more random variables. In this case, correlation means that based on knowing the value of one variable, with a certain probability, it is possible to predict the value of another variable, since these values ​​appear in a certain relationship. The degree to which two values ​​are consistent can be represented graphically in a scatter diagram, or by means of a correlation coefficient.

Regression analysis is an extension of correlation analysis and is one of the most commonly used statistical techniques today. Regression analysis is a set of analytical techniques used to better understand the interrelationships between observed phenomena, expressed in the form of collected data. As a final result, the analysis produces a regression equation, but all the results obtained in the process can provide valuable information about the observed phenomena and their environment.

At its core, regression analysis involves two or more variables that are somehow related to each other. One of the variables is of particular interest because the purpose of the analysis is to explain changes in it through changes that occur simultaneously in the other variables in the model. This variable is called the dependent or criterion variable and is denoted by Y. The other variables serve to explain or predict the value of the dependent variable. These variables are called independent or predictor variables and are denoted by X. Typically, a model uses multiple independent variables, so they are denoted by X1, X2, ... Xp, where p denotes the total number of independent variables in the regression model. Predictor variables are also called covariates, regressors, or factors. Although it is most commonly used, the name independent variable is the least representative of reality, because these variables are rarely independent of each other.

The relationship between the dependent and independent variables is expressed in the form of one or more equations, collectively called a regression model. The actual relationship between the dependent and independent variables can be approximated by the following regression model:

Y = f(X1, X2, ..., Xp) + ε,

where ε is random error which represents the difference between the approximation and the actual value of the dependent variable Y, and the function f describes the relationship between the dependent and independent variables.

Regression models can be divided according to several criteria. Below we list some of the divisions.

Regression models can be classified according to the number of independent variables into:

  • simple regression, where there is one dependent and one independent variable and

  • multiple regression, where there is one dependent but multiple independent variables.

According to the type of dependent variable, regression models can be divided into:

  • models with continuous dependent variable,

  • models with categorical dependent variable which is not dichotomous, but takes more than two values ​​(categories) and

  • models with dichotomous dependent variable which represent a special case of models with categorical dependent variable, because the dependent variable can take only two values.

According to the type of relationship between the dependent and independent variables, regression can be:

  • linear regression, which is characterized by a linear relationship with respect to the model's parameters (coefficients), regardless of the degree of the independent variables, and

  • nonlinear regression, which can be:

    • quadratic regression,

    • polynomial regression,

    • exponential regression, etc.

According to the number of dependent variables, a regression model can be

  • univariate regression model, i.e. a model with one dependent variable and

  • multivariate regression model in which multiple dependent variables occur, which is why it consists of multiple regression equations.

In addition to allowing for the assessment of the degree of correlation of phenomena, regression analysis also provides the probability of the occurrence of the calculated correlation. It is used for testing hypotheses in studies and experiments, but also for predicting the future movement of the phenomenon being observed.

The concept of regression is easy to understand and is implemented in almost every statistical package, and it allows for the examination of the functional dependence between variables, and as such it lies at the basis of many modern statistical techniques. Therefore, the application of correlation and regression analysis can be found in almost all academic fields or applied science today. The following examples illustrate these applications across various disciplines:

  • Economics - forecasting consumption, forecasting stock price movements, etc.,

  • Psychology - the influence of intelligence on an individual's achievement, the influence of an individual's upbringing and cultural values ​​on their sense of happiness, etc.,

  • Agriculture - how to predict the amount of milk production based on knowledge of a set of other data,

  • History - how to estimate the age of an object based on other known characteristics of the object,

  • Politics and demography - forecasting regional migration rates based on local unemployment rates, average income, and political stability indices,

  • Environment - the connection between water quality and land use, i.e. whether increased land use leads to pollution of watercourses, etc.