Friday, October 28, 2022

The novelty test and the reproducibility criterion


In response to SAR's 2013 letter to the OECD, calling for a revision of the Frascati Manual in order to add artistic research as a new and separate scientific category (see here), the new 2015 Frascati manual made a negative but clear stance. I have reported on that briefly (see here), and want to elaborate a little.



Translating the position delineated in the manual (2.65-67) to the discourse in artistic research, we have: 

  • research on the arts, includes musicology, art-history, etc. 
  • research for the arts, relates to AR 
  • artistic expression, art itself 

(The manual limits the examples of research for the arts to the traditional sectors catering to artists, such as instrument builders, but it should be clear that artistic research targets artists as the beneficiaries of their research just as well.)

Of further interest are the novelty test and reproducibility criterion that are cited as fundamental to research. For art, these are simply considered not relevant: in a paragraph about artistic expression vs. research, it is stated that

2.67. [...] Artistic performances fail the novelty test of R&D as they are looking for a new expression, rather than for new knowledge. Also, the reproducibility criterion (how to transfer the additional knowledge potentially produced) is not met. [...]

The novelty test, although no written rule, is largely considered to be a priority in universities, at least in Europe. During the Bergen Conference of European Ministers Responsible for Higher Education, in May 2005, the qualifications signifying the completion of a third cycle (see here) are awarded to students who (my highlighting):

  • have demonstrated a systematic understanding of a field of study and mastery of the skills and methods of research associated with that field; 
  • have demonstrated the ability to conceive, design, implement and adapt a substantial process of research with scholarly integrity; 
  • have made a contribution through original research that extends the frontier of knowledge by developing a substantial body of work, some of which merits national or international refereed publication
  • are capable of critical analysis, evaluation and synthesis of new and complex ideas; 
  • can communicate with their peers, the larger scholarly community and with society in general about their areas of expertise; 
  • can be expected to be able to promote, within academic and professional contexts, technological, social or cultural advancement in a knowledge based society. 

Artistic knowledge is often difficult to determine. Certainly the "embodied" type of knowledge is problematic because of its tacit nature. Unless it is explicated in an efficiently communicable language, there is no use of it outside of theoretical perspectives. But even in the latter case, as in Knowing in Performing, the knowledge is pointed to, considered, and transferred... in writing, not even with an added CD or DVD. In practice, embodied knowledge cannot be considered as such by itself, either. When a dance teacher shows a student how to perform a certain movement, there will always be verbal explication accompanying the demonstration, if only to make sure that there is no inadvertant misinterpretation of some detail. It may be difficult to write efficiently about such knowledge (although a picture painting a thousand words suggests a promising start), but the entirely non-verbal manner is least (re)productive. No surprise the reaction in the Frascati manual, then, about the reproducibility criterion not being met. 

Despite the powerful method of one-to-one teaching so typical of performance pedagogy, reproducibility is considered to be against the point of being an artist. Indeed, what is heard/seen on stages and recordings is considered valuable, or at least valid, only in relation to the  individuality of the artist. Generally, a new recording is either produced with new repertoire, or with the same repertoire as before but in different interpretations. But this is no more than an assumption, and with globalisation, the idea that countless thousands of pianists would each be able to play the same repertoire in noticeably different ways has no right to keep evading scrutiny. Of course, reproducibility in research is really meant to be that of the applied method. From that perspective, AR is aimed at the peer-researcher or the peer-artist, who need to be able to trust the accuracy of the resulting insights. AR is not aimed at the audience, at least not directly. Hence the misconception that AR should always be audibly distinguishable from art, or that it would destroy the "magic" of making the art (as I was once accused of by a professional performer).

The problem of art vs. art-as-research vs. artistic research relates to the demarcation problem in science, including immunizing strategies and defense mechanisms. In art, and about art, one can express and claim anything. In this New York Times review of a concert by Yuja Wang, little makes sense, except that it can be appreciated as a verbal performance by the reviewer. Perhaps he wanted to transmit knowledge that he felt he acquired; it may even be exactly what Ms. Wang had intended to convey. But it is immune to being refuted, so it might as well be a review of another concert altogether. It is art, about art. No new knowledge or reproducibility needed to be expressed or checked. 

In artistic research, what constitutes 'new knowledge' can be seen to underly a division in the genres. Some visual artists have expressed less than emphatic feelings towards musicians who play music of long-dead composers, as if present-days painters would be considered proper artists if they painted à la Rubens. Similarly, the question can be asked whether artistic researchers who investigate, say, performance practices from the 18th century, can be considered to develop new knowledge if, in fact, demonstrably old knowledge is unearthed and reapplied to their practice. Practices in other disciplines, e.g. archeology, can provide answers to this concern. Neverthess, 'novelty' alone is often not enough: there has to be 'impact' as well. From the perspective of the Frascati manual, it goes tangentially deeper. A musician like Ton Koopman could be acknowledged as engaging in scholarship, which in turn may partly qualify as research. But the resulting 'new' expression in his performances would not qualify for research or development. Since the performance can be considered as a product, the discussion would not be whether it is research, but whether it is experimental development. And this development would principly not qualify because of the tests that should then apply. 

The Frascati manual is intended as guidance for measuring and reporting policies. Naturally, such guidance can become a proxy debating space, given the sensitivities and sustainability of some institutions and university departments. As such, the manual is meant to avoid that, for instance, artistic performance is taken into account as R&D expenditure, the latter of which can be witnessed here (scroll down to the before-last line of table 3). But, as much as the stance against acknowledging a new knowledge-type for AR is clear, a door is left open for accepting institutional AR: "Higher education institutions have, nevertheless, to be evaluated on a case-by-case basis if they grant a doctoral degree to an artist as a result of artistic performances. The recommendation is to adopt an “institutional” approach and only to take account of artistic practice recognised as R&D by higher education institutions as potential R&D". (2.67)

The first Frascati Manual ever was issued in the early 1960s, and six new versions have appeared in the half-century since, at a rate of about nine years per new issue. (See here for some history.) It may take a while for the OECD to reconsider the matter. It may take that same while before it makes the distinction between artistic performance and composition...