Conceived and designed the experiments: RF ADF. Performed the experiments: RF. Analyzed the data: RF. Wrote the paper: RF ADF.
The authors have declared that no competing interests exist.
Theory predicts a close structural relation of formal languages with natural languages. Both share the aspect of an underlying grammar which either generates (hierarchically) structured expressions or allows us to decide whether a sentence is syntactically correct or not. The advantage of rule-based communication is commonly believed to be its efficiency and effectiveness. A particularly important class of formal languages are those underlying the mathematical syntax. Here we provide brain-imaging evidence that the syntactic processing of abstract mathematical formulae, written in a first order language, is, indeed efficient and effective as a rule-based generation and decision process. However, it is remarkable, that the neural network involved, consisting of intraparietal and prefrontal regions, only involves Broca's area in a surprisingly selective way. This seems to imply that despite structural analogies of common and current formal languages, at the neural level, mathematics and natural language are processed differently, in principal.
In a remarkable but controversially discussed paper
As closer examination reveals that examples of hierarchically organised data or information are abundant in everyday life. A familiar form of it is already apparent in simple equations or algebraic expressions, even if one usually does not perceive them as such when dealing with them. However, what they have in common with (natural) languages is the fact that the formation of the hierarchy in mathematical expressions is not arbitrary, but obeys strict rules, rules which not only apply to their generation but also to their interpretation (e.g., calculation). These rules, however, do not necessarily follow the principles of natural languages.
Here, we looked at the neural base of mathematics from this novel perspective, with Mathematical Logic as the obvious “language-mathematics interface”. We also added a new aspect, namely by also including the case where the processor (i.e., the human brain) encounters an “almost” well-defined structure, which is tantamount to error detection during interpretation.
The question of what the sources of mathematical thinking at the neural level might be has already been raised
However, in light of the above discussion and also from a modern standpoint which focuses on structures, objects and relations, the “number” approach not only inevitably falls short in recognising the essence of the cognitive roots of mathematics but also in relating it to other fundamental cognitive domains, such as language, for example.
Therefore, we designed an experiment using functional magnetic imaging (fMRI) to investigate the syntactic processing of abstract mathematical formulae and termini, written in a standard first-order language. The stimuli items used (see
The grammatical/generative part of the syntax starts with an alphabet, out of which termini and formulae are (recursively) built, thereby yielding arbitrary hierarchical expressions. For assumed processing steps see (Supplemental Information:
We predicted to find activation beyond those areas known to support number processing i.e., the intraparietal region.
Under the hypothesis that first-order languages, which are by definition formal languages, share a neural representation with other formal languages, we expect activation in the left inferior frontal gyrus (IFG), in particular Broca's area, as this region has been found to activate during the processing of syntactic hierarchical structures in artificial grammars (e.g.,
As responses were given only after stimulus presentation in a delayed mode, only the percentage of correct responses and no reaction time data could be analysed. Overall, performance data from the present fMRI experiment showed that the 24 participants answered correctly on an average of 88% (standard deviation (SD) of 0.12) on the 50 hierarchical items compared to 96% (SD 0.05) on the 50 non-hierarchical ones. This difference was significant for the two-tailed paired
A subgroup of 12 participants, however, showed no significant difference in performance with respect to the two types of problems. For this sub-group, the mean correct answers for hierarchical formulae was 94% (SD 0.068) and 95% (SD 0.062) for the lists, for the two-tailed paired
Therefore, all results are reported for the entire group of 24 participants. On average these 24 participants performed correctly with an average of 86% (SD 0.12) on the correct hierarchical items and 89% (SD 0.13) on the incorrect hierarchical items. This difference was not significant for the two-tailed paired
When comparing the fMRI data of the entire group for the processing of correct hierarchical structures to those of correct flat structures (see
FMRI data are mapped onto a reference brain (single subject), where areas differing significantly in activation are coloured red to yellow and correspond to values with
AREA | Talairach co-ordinates | ||||||
left | right | ||||||
mFG, BA 10 | −38 | 52 | −3 | – | – | – | |
mFG, BA 10 | – | – | – | 31 | 49 | −9 | |
IFG, BA 45 | −47 | 19 | 6 | – | – | – | |
IFG, BA 47 | −38 | 37 | −3 | – | – | – | |
MFG, BA 6 | −41 | 10 | 51 | – | – | – | |
MFG, BA 6 | – | – | – | 34 | 16 | 45 | |
SFG, BA 6 | −23 | 19 | 60 | – | – | – | |
MTG, BA 21 | −65 | −53 | 3 | – | – | – | |
MTG, BA 22 | – | – | – | 55 | −50 | 0 | |
AG, BA 39 | – | – | – | 37 | −74 | 33 | |
Inf. parietal lobule, BA 40 | −53 | −41 | 45 | – | – | – | |
Inf. parietal lobule, BA 40 | – | – | – | 43 | −47 | 42 | |
Precuneus, BA 7 | −5 | −65 | 45 | – | – | – | |
Cuneus, BA 18 | – | – | – | 13 | −77 | −18 |
Activation maxima (uncorrected) of the contrast: “hierarchical correct vs. list correct”. Abbreviations: AG: angular gyrus, BA: Brodmann area, IFG: inferior frontal gyrus, mFG: medial frontal gyrus, MFG: middle frontal gyrus, MTG: middle temporal gyrus, SFG: superior frontal gyrus.
When we compared the incorrect list items to the correct ones for the entire group (see
FMRI data are mapped onto a reference brain (single subject), where areas differing significantly in activation are coloured red to yellow and correspond to values with
AREA | Talairach co-ordinates | ||||||
left | right | ||||||
mFG, BA 10 | −14 | 67 | −3 | – | – | – | |
mFG, BA 8 | −5 | 40 | 36 | – | – | – | |
IFG, BA 47 | −44 | 19 | 0 | – | – | – | |
IFG, BA 47 | – | – | – | 22 | 10 | −15 | |
SFG, BA 6 | −11 | 19 | 63 | – | – | – | |
MFG, BA 6 | – | – | – | 34 | 7 | 48 | |
AG, BA 39 | – | – | – | 37 | −59 | 36 | |
AG, BA 39 | −38 | −56 | 33 | – | – | – | |
Cingulate G, BA 31 | −5 | −35 | 39 | – | – | – | |
MTG, BA 22 | −53 | −41 | 3 | – | – | – |
Activation maxima (uncorrected) of the contrast: “flat incorrect vs. flat correct” expressions. Abbreviations: AG: angular gyrus, BA: Brodmann area, IFG: inferior frontal gyrus, G: gyrus, mFG: medial frontal gyrus, MFG: middle frontal gyrus, MTG: middle temporal gyrus, SFG superior frontal gyrus.
Further, the comparison of the correct hierarchical condition with the baseline condition and also the comparison of the correct list condition with the baseline condition, revealed no overlap with the cytoarchitectonically defined Broca's area (
At a macroscopic level the present experiment found significant inferior frontal, middle frontal and parietal activation for the processing of the syntax of first order logic (“mathematical syntax”) of correct hierarchical structures compared to correct non-hierarchical structures as represented by the mathematical expressions used.
To understand the activation observed, it is necessary to identify the main processing modules needed to accomplish the given task successfully. One would expect processing to rely on a tangible neural network involving several modules which interchange information and interact as time elapses. The modules should grant visual decoding (“reading”) of the visually presented stimuli, allow mental transformations of the formulae/termini (visuospatial working memory), retrieval and application of the rules underlying the proper generation of the syntax of first-order logic, and finally preparation of the response.
The first processing step (i.e., reading) is necessary for both conditions (formulae and termini) and, therefore should not show up in a direct comparison between conditions, as was indeed the case. The observed bilateral parietal activation surrounding the entire intraparietal sulcus (IPS) replicates part of a neural network previously found in (arithmetic or algebraic) calculation tasks
It is assumed (
Further, the dorsolateral part of the prefrontal cortex, (DLPFC, BA 46 and BA 9) is assumed to be engaged in organisation of material to be remembered in encoding interactions, and, during retrieval interactions, in monitoring and verifying retrieved information. This is in line with the activation observed, as a model of processing hierarchical formulae that assumes both more memory resources for more material to be remembered and also more verification steps at each node in the hierarchical compared to the list condition.
The observation of activation in the left IFG (in BA 45/47) as a function of the processing of the syntactic hierarchy in abstract formulae is novel for two reasons. First, activation in the left IFG, as observed during arithmetic tasks in previous studies, which are notably by definition semantic and not syntactic, has been attributed to general working memory
Second, outside the domain of arithmetic calculations, processing activation in the left IFG has been found in a number of studies on language, with different sub-regions reflecting different aspects of language processing. Activation in the more posterior portion of the IFG ( i.e., in BA 44 and posterior portion of 45) has been observed for the processing of hierarchical sentence structures as compared to flat structures in German
Thus, the activation in the anterior prefrontal cortex, i.e. BA 45 and BA 47 has been observed, in the context of studies investigating “semantic processing” in language
The comparison of incorrect versus correct list items revealed significant activations in the left frontopolar cortex (FPC), bilaterally in the ventrolateral prefrontal cortex (VLPFC; BA 47), the anterior cingulate cortex (ACC; BA 32/8), the bilateral angular gyrus (BA 39), the bilateral middle frontal gyrus (BA6) and the left medial temporal lobe (BA 22). The number of activation foci exceeded by far those found
The additional comparisons “hierarchy vs. list” and “correct vs. incorrect” (see Supporting Information,
When taking into account the brain activations across different studies, the combined data suggest a functional differentiation between more posterior and more anterior portions of the IFG, with more anterior portions being recruited the more complex the relation between elements in a structured sequence are. This assumption of such a graduation from more posterior to anterior IFG receives support from two perspectives, these being evolutionary neuroanatomy
Finally, the present neuroimaging data suggests that a formal ruled-based generation and decision process as in the form of a calculus is effective because it strives for an optimal balance between data compression and reliability, implemented at the neural level. This in turn permits humans to communicate complexly structured information and to phrase problems more easily in face of the limits of the human processing system.
Twenty-four participants gave their informed consent, after having read and signed the guidelines set out for fMRI studies at the Max Planck Institute for Human Cognitive and Brain Sciences. Specifically, we had 24 healthy, right-handed subjects (8 female, 16 male), who were German native speakers with normal or corrected to normal vision. The age range was from 21 to 31 years of age, (mean: 25.9 years, SD 2.6). Almost all participants were university students and all were part of the Institute's database of regular and general fMRI subjects.
We based our specific first-order language on an alphabet consisting of: variables:
The set of variables and constants was chosen randomly, whereas the selection of the other symbols followed more specific rules. The two variables denoted by Greek letters
Out of the symbols we built first-order formulae that were either syntactically correct or incorrect. The errors were violations of the well-defined building rules for terms and formulae in logic, and not just simple misprints.
An item for the visual presentation either corresponded to an entire formula, an entire list or the baseline picture. So, e.g.
There were 25 syntactically correct formula items (e.g. as the formula above), 25 incorrect formula items and correspondingly 25 correct list items (e.g. as the list above) and 25 incorrect list items, i.e. a total of 100 items (50 formulae and 50 lists) to be judged for their grammatical content.
The stimuli represented for the formulae either 1,2,3-binary-trees, i.e. trees with one node at the top, two at the second level and three at the third level, or a list consisting of five simple 1-trees, i.e., a “hedge” (cf.
Stimuli of the following four types (1,2,0111;1,2,1011;1,2,1101;1,2,1110) were provided to ensure that subjects could not use the same reading strategy during the experiment.
For the baseline image, a row of white-greyish circles was used on a very dark grey background.
The software packages used were LIPSIA
For registration purposes, two sets of two-dimensional anatomical images were acquired for each participant immediately prior to the functional imaging. An MDEFT and an EPI-T1 sequence were used. T1-weighted MDEFT images were obtained, with a non slice-selective inversion pulse followed by a single excitation of each slice. Anatomical images were positioned parallel to AC-PC.
The functional MRI were as follows; Axial slices: TR = 2 s, TE = 30 ms, alpha = 90°, 29 slices (29×4 mm = 11.6 cm, whole brain), 4 mm slice thickness (no gap), voxel volume: 3×3×4 mm3, 64×64 matrix, 19.2 cm FOV. There were 25 stimuli per condition (4 conditions+nullevent), presented with SOA = 7 s, with a total stimulation time of 27 minutes (25×5×13 seconds).
The data processing was performed using the software package LIPSIA
Subsequently co-registration of data was carried out. To align the functional slices with a 3D stereotactic co-ordinate reference system, a rigid linear registration with six degrees of freedom (3 rotational, 3 translational) was performed. The rotational and translational parameters were acquired on the basis of the MDEFT and EPI-T1 slices to achieve an optimal match between these slices and the individual 3D reference data set. This 3D reference data set was acquired for each subject during a previous scanning session. The MDEFT volume data set with 160 slices and 1 mm slice thickness was standardised to the Talairach stereotactic space
The statistical evaluation was based on a least-squares estimation using the general linear model for serially auto-correlated observations. The design matrix was generated with a synthetic haemodynamic response function and its first and second derivative. The model equation, including the observation data, the design matrix and the error term, was convoluted with a Gaussian kernel of dispersion of 4 s FWHM to deal with the temporal auto-correlation. Afterwards, contrast-images (i.e., estimates of the raw-score differences between the specified conditions) were calculated for each subject. Each individual functional data-set was aligned with the standard stereotactic reference space, so that a group analysis based on the contrast-images could be performed.
The individual contrast-images were first masked and the individual and masked contrast-images were then entered into a second-level random effects analysis (one-sample
The experiment was devised as a reading experiment. The 125 stimuli items (50 stimuli of hierarchical type, 50 stimuli of list type and 25 baseline stimuli) were presented as a whole to the participants, in a fully randomised order. The presentation of hierarchical, flat and baseline conditions were intermixed. A stimulus item, e.g. a formula, was visible as a whole for a fixed period of 7600 ms on the screen. Randomisation was done using the random number generator of the computer programme “Presentation”, and was done for each subject separately. There was one run per participant with no repetition of formulae or list items, but the baseline item was always the same. The subjects' task was to judge the syntactic correctness of each of the formula or list items shown (for examples and assumed processing steps underlying the judgement in the different conditions see the Supporting Information
The response had to be given for each stimulus by the participant after the stimulus item disappeared from the screen and a new screen indicated that the answer had to be given. The participant had 1700 ms to press the respective button, i.e., one for correct and one for incorrect. No feedback was given after the button press. For the baseline condition no answer was required. (For a schematic description of the experiment, see Supporting Information,
All presentation material, including the visibility of the stimuli, was previously tested in the scanner. All participants were carefully instructed before the actual test and also had a training session with a sample of similar stimuli presented on a laptop and with a button press device.
(0.02 MB DOC)
Assumed processing steps required to check the syntax of the hierarchical expression.
(0.73 MB EPS)
Assumed processing steps required to check the syntax of the list of expressions.
(0.73 MB EPS)
Broca's area (BA 44/45) (blue, green, lilac) and activations from contrast “hierarchy correct-baseline”. FMRI data are mapped onto a reference brain (single subject), where areas differing significantly in activation are coloured red to yellow and correspond to values with Z>3.09 (uncorrected). The cross hair is placed at (−44, 37, 1) in the Talairach co-ordinate system. Views: coronal y = 37, sagittal x = −44 and axial z = 1. The region marked in green, blue and lilac corresponds to the cytoarchitectonically defined Broca's area with a probability of at least 50% according to
(0.09 MB TIF)
Broca's area (BA 44/45) (blue, green, lilac) and activations from contrast “list correct-baseline”. FMRI data are mapped onto a reference brain (single subject), where areas differing significantly in activation are coloured red to yellow and correspond to values with Z>3.09 (uncorrected). The cross hair is placed at (−44, 37, 1) in the Talairach co-ordinate system. Views: coronal y = 37, sagittal x = −44 and axial z = 1. The region marked in green, blue and lilac corresponds to the cytoarchitectonically defined Broca's area with a probability of at least 50% according to
(0.09 MB TIF)
Broca's area is outlined in blue, and regions of activations with Z>3.09, are outlined in white for “hierarchy correct-list correct”, in yellow for “hierarchy correct-baseline” and in red for “list correct-baseline”. The cross hair is placed at (−50, 31, 24) in the Talairach co-ordinate system. Views: coronal y = 31, sagittal x = −50 and axial z = 24. The region marked in blue corresponds to the cytoarchitectonically defined Broca's area (50% according to
(0.09 MB TIF)
Schematic illustration (not to scale) of the sequence of screen contents with the respective duration of each phase, of the fMRI experiment. (isi = inter stimulus interval)
(0.33 MB EPS)
(0.01 MB DOC)
(0.01 MB DOC)
We gratefully acknowledge the indispensable assistance and help of D. Wilfing, A. Mempel, M. Naumann, S. Gutekunst and Th. Mildner during the fMRI measurements. Further, we thank Prof. von Cramon for discussion. R.F. acknowledges the help of J. Bahlmann to make the necessary first steps into the world of fMRI, and he acknowledges fruitful discussions with M. Grigutsch, J. Lepsien and K. Müller and M. Makuuchi.