Basic relationship

As the initial step, we present the results of stationarity tests, which tell us whether we should analyze the original series or some of their transformations. In Table 2, we show the results of the ADF and KPSS tests (see the Methods section for more details) for the original as well as the logarithmic series and their first differences. The outcome is quite straightforward, as we do not reject unit roots for either of the original series (or their logarithmic transformation for the Google searches; we do not examine the logarithmic transformation for the unemployment time series as these are already in the percentage representation). Further testing, which is not reported here, shows no cointegration relationship between unemployment and the search query series, so we need to proceed with the first differences of the series. For most of the cases, we support the stationarity of the first differences. In the analysis, we further proceed with the first differences of the unemployment rate and the first logarithmic differences of the Google searches. We opt for this specification because the combination of percentage representation and logarithmic transformation allows for a straightforward interpretation as an elasticity, i.e., as a proportional relationship.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Stationarity testing.

Augmented Dickey-Fuller test (ADF) for a presence of unit root and KPSS test for stationarity are used. *, ** and *** stand for statistical significance at 10%, 5% and 1% levels, respectively. Number of lags for the tests is based on the Akaike Information Criterion (AIC) selection.

https://doi.org/10.1371/journal.pone.0127084.t002

For the very basic relationship between the unemployment rate and the intensity of job-related searches on Google, we study the equation (1) where ΔUR_t and Δlog(GI)_t represent the first difference of an unemployment rate at time t and the first logarithmic difference of the Google searches at time t, respectively, for a given country, and ε_t is an error term.

The elasticity between Google searches and the unemployment rate from Eq 1 is estimated at 0.686 (with the p = value of 0.008), 0.185 (0.125), 0.331 (0.216) and 0.606 (0.001) for the Czech Republic, Hungary, Poland and Slovakia, respectively, with the heteroskedasticity and autocorrelation consistent (HAC) standard errors. The proportional relationship thus varies across the analyzed countries, but it remains positive for all four countries and statistically significant for two out of the four (at the 1% significance level). Specifically, the relationship is very strong for the Czech Republic and Slovakia, with values above 0.6. This result shows that the changes in the unemployment rate are well projected into the online search queries for vacancies and job-related terms. Studying the connection between these two variables thus appears to be promising and worth further utilization and investigation.

Nowcasting

Macroeconomic time series, such as unemployment rates, have a special property that is not present for financial series or other series in natural sciences—they are available with a pronounced lag. This lag occurs due to data processing and collection, which usually take several months; even after this period, there are sometimes corrections to the reported values. This characteristic makes a series that is available immediately without any lag and that is strongly correlated with the variable of interest very useful for forecasting the present value of the variable without waiting several months. This type of forecasting of the present is usually referred to as “nowcasting”.

In the previous section, we showed that Google searches for job-related terms are related to the unemployment rate, which makes search queries potentially useful for the nowcasting of unemployment. As a nowcasting model, we consider (2) where the unemployment rate is assumed to be available with a three month lag (please refer to the Methods section for more details about the data collection). We again consider the differenced series due to the stationarity issues discussed above. As a base model, we use the model specification in Eq 2 without the Google terms so that the competing model is defined as (3) For both models, we consider a maximum lag L which we set as L = 3, 6, 12. This way, we are able to comment on the quality of models with regards to the amount of information taken into consideration. The upper bound is set to 12 months as the unemployment series are usually strongly cyclical.

The results of the nowcasting models are summarized in Table 3. In the table, we show the adjusted R² (${\bar{R}}^2$) as a measure of the models’ quality controlling for the number of explanatory variables. We observe that for all countries, the inclusion of the Google series strongly enhances the model. The most promising results are reported for the Czech Republic where the models improve strongly regardless the number of lags taken into consideration. For the other three countries, we observe that the base models with 3 and 6 lags are very weak, even reaching negative values of ${\bar{R}}^2$. A strong seasonal (annual) pattern in the unemployment rates is thus visible here. The Google series are thus evidently useful for the in-sample modeling of the series, which is supported by a statistical significance of the online searches for all countries and regardless the maximum lag used. However, it is the out-of-sample performance that eventually matters.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Nowcasting summary (in-sample).

The whole analyzed period 01/2004-12/2013 is covered here. Model in Eq 2 is used here with varying maximum lag L. Joint significance of variables is a simple F-test based on heteroskedasticity and autocorrelation consistent (HAC) standard errors (p-values are reported in the brackets). Adjusted coefficient of determination ${\bar{R}}^2$ controls for the number of independent variables used in the model.

https://doi.org/10.1371/journal.pone.0127084.t003

We divide the analyzed period into two—a training (fitting) period and a nowcasting period. The model parameters are fitted on the data between 01/2004 and 12/2011 (96 observations). Nowcasting performance is then evaluated on the series between 01/2012 and 12/2013 (24 observations). The “Google model” (Eq 2) is compared to the base model (Eq 3) using the Diebold-Mariano test [29] (see the Methods section for more details). In Table 4, the resulting statistics are summarized. For the Czech Republic, the model using Google searches is on average outperforming the base model for each lag selection. For the maximum lag of 3 and 12 months, the difference is statistically significant. Similar results are reported for Hungary, for which the Google specification outperforms the base model for all lag selections as well. The difference is statistically significant for lags up to 3 and 6 here. For Poland, we find statistical significance only for the maximum lag of 6 months, and for Slovakia, the base model even outclasses the “Google model”. The results are thus quite diverse.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Nowcasting summary (out-of-sample).

The period between 01/2004 and 12/2011 is used for model fitting and the rest of the period between 01/2012 and 12/2013 is used for the forecasting comparison. Diebold-Mariano test described in the Methods section compares the “Google model” defined in Eq 2 to the base model defined in Eq 3 with a null hypothesis of no difference of forecasting accuracy versus the alternative of the “Google model” being more accurate.

https://doi.org/10.1371/journal.pone.0127084.t004

Data

The monthly unemployment data for the Czech Republic, Hungary, Poland and Slovakia have been obtained from the Eurostat database (http://ec.europa.eu/eurostat/). The basis for the unemployment measurement among EU countries is the EU Labour Force Survey (EU LFS)—a continuous and harmonized household survey that, in accordance with EU legislation, is conducted in each member state. The monthly data from Eurostat are estimates based on the results of EU LFS. Because there are no legal obligations that the EU countries deliver monthly data, these data are often interpolated/extrapolated using national surveys or registered unemployment data.

Eurostat defines an unemployed person as someone aged between 15 and 74 without work during the reference week who is available to start working within two weeks and who has actively sought employment at some time during the last four weeks. In our analysis, we use the general (both sexes, 15–74 years old), raw (not seasonally adjusted) unemployment rate. We use these data because we do not know the method used to make the seasonal adjustment and because the Google data are also not seasonally adjusted.

The Google search queries data have been downloaded from the Google Trends webpage (http://www.google.com/trends/). As the languages of the studied countries differ, we have looked for various terms. As Czech, Polish and Slovakian are all Slavonic languages, the searched words are very similar or even the same. For Czech, we searched for “práce” and “prace” (i.e. both with and without diacritics), for Polish “praca” and for Slovakian “práca” and “praca” (again both with and without diacritics) but also the Czech “práce” and “prace”, which turn out to be very frequently searched for by Slovakians. For Hungarian, we used the terms “állás” and “munka”. These all are equivalents for the English “job” and “work”. Other related words have not passed throught the Google threshold or only incomplete series are available. As the Google Trends engine allows to compare up to five terms for a given setting (in our case a country and a time frame), we can use more series for each country. In the cases of more searched queries (the Czech Republic, Hungary and Slovakia), we sum the series together and rescale them to 100. This can be done as if multiple series with the same spatial and temporal characteristics are obtained from the engine, these share a common scale.

The weekly series obtained from the Google Trends site have been transformed to monthly series on the basis of the number of days in the month. All series, both the unemployment rate and the Google searches, are studied between January 2004 and December 2013 (120 observations). The dataset is provided in the S1 Dataset.

Stationarity

A stochastic process {x_t} is stationary if for every collection of time indices 1 ≤ t₁ < t₂ < t_m, the joint probability distribution of (x_t1, x_t2, …, x_tm) is the same as the joint probability distribution of (x_t1+h, x_t2+h, …, x_tm+h) for all integers h ≥ 1 [30]. To test for stationarity, we utilize the Augmented Dickey-Fuller (ADF) test [31] and the KPSS test [32]. The tests have opposite null hypotheses and thereby provide a complementary pair, which is commonly used for stationarity testing.

In the ADF procedure [31], the OLS regression is run on to perform the test, where α₀ and γt are an intercept and a time trend, respectively, and p represents the lag order. The null hypothesis under which the series contains a unit root is found for against the alternative The ADF test statistics are then computed as the usual t-statistics, which, however, follow a more complicated distribution under the null hypothesis.

The null hypothesis of the KPSS test [32] is opposite to that of the ADF test, i.e., the KPSS test has the null hypothesis of stationarity. The test is based on the OLS regression of the series {x_t}: where α₀ and γt again represent an intercept and a time trend, respectively, and ξ_i are independent and identically distributed random variables with zero mean and a unit variance. The null hypothesis of stationarity is found for against the alternative The KPSS test statistic is defined as where S_t is the partial sum of the residuals and ${\widehat{\omega }}_T^2$ is an estimator of the spectral density at frequency zero.

Nowcasting accuracy

To compare the forecasting accuracy of the proposed models, we utilize the Diebold-Mariano test [29] based on absolute errors. An absolute error is simply defined as a_i = ∣f_i − y_i∣ where f_i stands for a nowcast value and y_i is an observed real value. We do not use also popular squared errors here as these are usually applied to magnify higher errors. However, the nowcasting errors in our case are always lower than unity which makes the squared errors counterintuitive so that we avoid using them. Diebold and Mariano [29] propose a test to compare the predictive accuracy of two competing forecasts. Let ${\{{\epsilon }_t^1\}}_{t_0}^T$ and ${\{{\epsilon }_t^2\}}_{t_0}^T$ be the sequences of forecast error losses from two competing forecasting measures by a particular loss function (absolute errors a_i in our case). The null and alternative hypotheses are then stated as The Diebold-Mariano test assesses the accuracy based on the loss differential which is equal to zero under the null hypothesis. The Diebold-Mariano statistic is then where $\bar{d}$ is the mean loss differential and ${\widehat{LRV}}_{\bar{d}}$ is a consistent estimate of the asymptotic (long-run) variance of $\sqrt{T}\bar{d}$ defined Under the null hypothesis, the testing statistic goes to a standard normal distribution so that $S\stackrel{A}{\sim }N(0,1)$[29].