“The manufacturer of accessories, whose art, for the technical execution of spectacles, is more decisive than the poet’s”

 

algorithmic liveness is a critical overview of how transmission produces different effects of liveness (telepresence, acceleration, real time, contextual feedback etc). The project focus on the specifics of what will be called algorithmic liveness, in which the dynamic aspects of binary coding results in double binded real time processes: they happen both as a result of the capacity of digital machines to constantly recalculate transmitted audiovisual states, and of the possibility of live inputs altering their feedback chains.

A first iteration of the project was presented at the Humboldt University Medientheater, as detailed here.

As discussed by Ernst (2013a, 2013b), these tempor(e)alities have implications for the concepts of memory and archive (thus, how algorithmic processes reconfigures time perceived as past). This project will examine how they affect presence and performance (that is, how algorithmic processes reconfigures time perceived as present). This perceptions also refer to a long time understanding of technical languages: for example, when comparing the writings of Kracauer and Bazin on cinema, the former relates technical images with history (the camera as memory), whereas the bazanian ontology and its engagement in realism relates technical images with actuality (the camera as mediated presence).

One of the topics of research is the continuos examining of how different technological devices result in particular modes of real-time manipulation: how pioneering equipment and logical devices (the radio, the TV, the Clavilux, The Lumigraph, the MIDI controller, the synthesizer) were employed to create real time audiovisual sequences? How do current computers and software are used in live audiovisual performance? What are the differences and similarities among older and newer technologies?

As background and conceptual framework, the research also addresses a broader context in which the concept of eventuality helps to understand how contemporary society shifted towards technologies in which algorithmic durations transform all occurrences into real time processed calculations (thus contributing with one of the aspects of Ernst’s current research Temporalizing The Present, by examining shifting tempor(e)alities of audiovisual apparatus, as they converted from analogic to digital). Some early ideas related to this perspective are published in the article Designing Real-Time: On How Events Affect Audiovisual Narrative.

 

 

Contemporary society is defined by events that transmit a sensation of real time. For Douglas Rushkoff, in Present Shock, our “society has reoriented itself towards the present moment. Everything is live, in real time, and always active. This is not a mere acceleration, though many of our lifestyles and technologies have accelerated the pace in which we do things. It is more a diminishing of anything that is not happening now — and the kidnapping of daily life by everything that supposedly is”.

This process of present shock is a complexification of the cultural shock produced by 19^th centuries inventions such as the telephone and the gramophone. They allowed people to listen to voices of people spatially or temporality distant — as described in Die Scheinbarkeit des “Live” (Ernst, 2015) — starting a process in which mediated presence resulted in different modes of remote relations.

Despite the impression of newness that Rushkoff text might offer, this sensation of liveness has antecedents spread through centuries, beyond the more evident thread of telecommunications (e.g. ancient rituals, theatre) as well as the different modes of information transmission insinuated in the end of the previous paragraph (e. g. happenings, body art). This project will not focus on those, and not even on more recent examples of immediacy (from the landing on the moon, considered by Gene Youngblood a paradigmatic example of transmission, to Sports or Concerts broadcasted on TV or streamed online).

This inflation of presentness is a relevant background, but also a consequence of the raising interest in mediated formats of transmission — of which Brecht’s article The Radio as an Apparatus of Communication can be considered an early example of an engagement present also in experiments such as Welles’s War of the Worlds, Cage’s Imaginary Landscape n. 4, Galloway and Rabinowitz’s Satellite Arts Project or Paiks’s Good Morning Mr. Orwell, to name a few examples included in genealogies such as Popper’s Communication Arts (1997), Daniels’s Medien Kunst Netz (2005) or Bambozzi, Bastos and Minelli’s Chronology of Mobile Media (2010).

 

Another aspect of the research is mapping how processes of transmission, as well as computational devices (by means of their algorithmic operations), are the material devices that drive such reorientation towards new forms of liveness and presence. Computers can synthesise, within a single system, several modes of transmission, correspondent to previous ways in which telecommunications introduced human culture to radio, TV and satellite, among others technologies of liveness.

As already suggested, transmission was already a familiar aspect of experimental art, before this illusions of distant presence became trivial effects of daily life — for example, in VOIP calls in which someone in São Paulo can dial a friend in Tokyo as easily as he can call his neighbour on their building intercom. The experimental usages of transmission will be the main topic of this research, given that artists pioneering usages of transmission exemplify some of their more incisive semiotic possibilities — and, also, because experimental arts constitute an exemplary field for the understanding of discontinuities on the histories of communication, since they reveal how processes such as transmission cannot be described by narratives of progress or evolution.

Not only that: experimentalism is also closely related to liveness. In Experimenting with Media Temporality, Ernst explain how the “subject of experimentality as event touches a crucial figure of contemporary epistemology, especially when we take epistemology in its processual, time-based sense as defined by cybernetics”. For him, “such processual ontology is close to the essence of media technologies itself (because only when in operation is a medium in its medium state)”. Being devices that lose their essence when not functioning, media seek intermittent functionality as a means to remain mediatic. From that perspective, it should be obvious that contemporary media exacerbate the liveness triggered by previous telecommunication and algorithmic procedures. It is as if sustaining liveness was a method to diversify mediatic ecologies by means of continuous functionality.

The topic of liveness regained significance, resulting in new bibliographical explorations that will be summed on the following paragraphs. Not all existing materials will be covered, since they will be further examined during the development of the proposed research (which is wider than this particular project and includes a theoretical overview of discussed topics). Also, it should be highlighted that the main materials of the proposed project will be devices used for live editing (specially how functionalities of hardware and software dictates certain semiotic possibilities of audiovisual languages). This will be further developed on the topic “Materials and Methods”, where it will also be explained how the proposed project relates to Humboldt University’s Music and Media Department laboratories and researches.

Paul Auslander proposed the concept of liveness, on his well-known book about Performance in a Mediatized Culture, where he examines how the fact that “we are well into a period of cultural history defined by the domination of mediatized representations” affect “the situation of live performance”. Auslander understanding of liveness is based on analysis of different media, seeking for recurring procedures: “Although I have stated that the relationship between the live and the mediatized is one of competitive opposition at the level of cultural economy, I do not see that opposition as deriving from the intrinsic characteristics of live and mediatized forms, but rather, as determined by cultural and historical contingencies. Through an examination of live and mediatized performances, an examination that begins with the discussion of early television and theatre /…/ I will argue against intrinsic opposition and in favour of a view that both emphasizes the mutual dependence of the live and mediatized and challenges the traditional assumption that the live precedes the mediatized” (Auslander, 2008: LOC 350 of 463).

This assumption can be related to the concept of presence, such as discussed in Fecchine’s Televisão e Presença [Television and Presence]. On her research, Fecchine questions “what kind of text is a direct transmission, that does not let itself be captured without its ‘exterior’, and the conditions in which its own communication happens”? Drawing form semiotics (specially the later works of Greimas such as Du Sens), she considers presence a form of mediation that operates on the level of the senses. On that context in which a gesture is already transmitting concepts, she describes a kind of presence that is not immediately resulting of the sensory, being the result of mediated transmission. In TV, instead of bodily proximity, spectators decode the implied risks of a live link being discontinued or the possibility of remote connection to distant places as potential openness. Fecchine analyses a number of live news broadcasts as examples of what she defines as an “in act enunciate”, showing how live TV produces semiotic events that make one “facing a TV screen to feel himself in the presence of the real world” (Fecchine, 2008).

From that perspective, that share some aspects with Auslander’s provocative statement that live does not precedes mediatized, the current attention to real time is a result of the fact that contemporary technologies are more explicitly perceived as real time mediations, because they depend on user’s agency. Simultaneity seems to be more easily identified in processes that happen through shared actions, in which the passive position does not exist. This happens because the idea of presence is culturally associated with body participation: someone is present when his body certifies an occurrence or interferes in it. As stated by Gumbrecht, “a present thing must be tangible by human hands — what implies, inversely, that it can have immediate impact on human bodies”. But tangible does not necessarily means physical, as we currently learn from VOIP technologies as much as we have dreamed about on science fiction or debated on theoretical discourses about human and computer symbiosis — for example, on the lifelong research of Licklieder, as his avocation of the need for computer participation in the formulation and real-time thinking (Licklieder, 1990).

In Die Scheinbarkeit des “Live”, Ernst reinforces this mutation on the character of presence, on contexts of transmission, when he states that: “In der Livesendung ist die empfundene Gleichzeitigkeit keine Illusion, sondern – von einem verschwindenden elektronisch bedingten Delta-t abgesehen – übertragungstechnisch real”. But liveness, on technologies such as the radio or the TV produces ambiguity, as a result of their continuous transmission being perceived as present time. On the same article, Ernst addresses the perceptual consequences of such sonic and audiovisual continuous producing a physiological sensation of presentness diffused through air waves: “Weil der auditive und der optische Sinn es zeitphysiologisch für gegenwärtig hält, muß eine Fernsehsendung den Aufzeichnungshinweis auf der symbolischen (also schriftlichen) Ebene als Information ausdrücklich einblenden: “Diese Sendung wurde aufgezeichnet”.

To avoid this dubious areas, the following research will focus on experimental works that stress liveness by means of real time editing of audiovisual elements, as well as real time capture of data to be used in sequences altering and manipulation (be it sensors, microphones “listening” to an environment’s sound, computer vision or interfaces that detect changes of state). The discussion about the specifics of such kind of work is controversial. In Live Cinema — Practices and Language, Mia Makela states that “LIVE CINEMA is a recently coined term for realtime audiovisual performances. Even though the term is new, the art form has a long trajectory. It is not easy to discuss this creative practice though, as there does not seem to exist written theory or complete history available” (Makela, 2006).

But the recognition that Live Cinema is a new term with a long trajectory obliges a more considerate examining of the field. Live editing can be understood as a mutatis mutandi return to the beginnings of cinema. As a recent example of that approach, in Irreproducible: cinema as event, Balsom states that “to view cinema solely as a medium of recorded reproducibility is to take a very narrow view of film history”. According to the researcher, this perspective:

excludes the diverse exhibition practices of early cinema and the avant-garde. Moving pictures were live from their very beginnings. Even before the development of the photographic support, magic lantern shows, panoramas, and optical toys offered illusions of movement that depended on the agency of a human operator and incorporated significant degrees of variability from one performance to the next”.

As a matter of fact, beginnings and ends can only be arbitrarily pointed out, in this field in which, as observed by Daniels and Naumann, “a web of parallel narratives develops that has interlinks and stretches history with attendant bifurcations, but possesses no universal model in which each of the art forms, media, technologies, and media practices dealt with here has its explicit, historical, and systematically defined place”. A number of authors consider that this intricate web of relations reveal how the approach of cinematic and audiovisual cultures from defined perspectives results in partial, insufficient understandings of topics such as montage, projection or sound-image relations, as a result of placing narrative in the centre of cinematic experience.

From that perspective, exploring what a semioticist would call the more iconical aspects of cinema (in other words, elements such as rhythm and textures, that are usually considered more musical than cinematic) helps broadening critical perspectives of audiovisual cultures, also by means of referring to this ancient, not central in contemporaries culture, modes of comprehension of reality. A critical / archaeological overview of live audiovisual languages, specially on the current context in which the sensation of real time seem to be a dominant of culture, has a lot to contribute to understanding how audiovisual languages can be meaningful to a post-gutenbergian era in which people exchange images and sound through apps rather than notes on napkins.

Despite the fact that experimental usages of transmission are relatively well documented, this proposal aims to offer new perspectives of analysis, by focusing on the functionalities of transmission, rather than its histories. Even if several researches cover the appearance of real time effects on language and culture (and connected topics, e.g. the histories of color-sound relations or practices of spatial montage), there are no significant publications (at least as far as this research went) that propose an interdisciplinary analysis of the functional and material aspects of the devices that allowed such effects to appear (and such histories to constitute themselves). In others words, the histories and genealogies of recent technologies are more oriented towards discursive narrations of facts than media archaeological excavations of functionalities / logics of its devices.

On the other hand, media archaeological research conducted on material aspects of technology have focused on older examples such as media panoramas (Huhtamo, 2013), specific phenomena such as programmed obsolescence and circuit bending (Hertz and Parikka, 2012) or the sound of wetware circuitry (Miyazaki, 2015). Related research also tends to be more narrative than inquisitive about functionalities and materialities (Levin, 2000; Deveraux, 2001; Ikoniadou, 2014; Tinhorão, 2014 [1981]). This justifies using existing genealogical information about liveness as a map to seek information that allow a deeper understanding of how devices such as Thomas Wilfred’s Clavilux, Radios, Oskar Fischinger’s Lumigraph, TVs, Nam June Paik Paik-Abe Synthesizer or a MIDI controller either transmit or synthesize real-time information and how existing technologies allow to re-enact their processes (be it for archival or experimental purposes).

               

 

F U R T H E R   R E A D I N G

Auslander, Philip. Liveness — Performance in a Mediatized Culture. London: Routledge, 2008 [1999].

Basbaum, Sergio. Sinestesia, Arte e Tecnologia — Fundamentos da Cromossonia. São Paulo: Annablume, 2002.

Bambozzi, Lucas; Bastos, Marcus; Minelli, Rodrigo. Cronologia das Mídias Móveis [Chronology of Mobile Media], in:
Mediações, Tecnologia, Espaço Público — Panorama Crítico da Arte em Mídias Móveis. São Paulo: Conrad,
2010.

Bastos, Marcus. Designing Real Time: On How Events Affect Audiovisual Narrative, in: Aaron Marcus (Ed.). Design,
User Experience and Usability. Switzerland: Springer International, 2014.

HYPERLINK “http://link.springer.com/chapter/10.1007%2F978-3-319-07668-3_49” \l “page-1″http://link.springer.com/chapter/10.1007%2F978-3-319-07668-3_49#page-1

______________. Sincronias entre acontecimento e narrative [Synchronies between event and narrative], in: Limiares das

Redes: escritos sobre arte e cultura contemporânea. São Paulo: Intermeios, 2014b.

Byrne, Brendan; Carvalho, Ana; Vjam Theory — Collective Writings on Realtime Visual Performance. London: Realtime

Books, 2008.

Daniels, Dieter; Frieling, Rudolf (Ed.). Medien Kunst Netz: Thematische Schwerpunkte. Karlsruhe: ZKM/Springer, 2005.

Daniels, Dieter; Naumann, Sandra. Audiovisuology (Compendium) — An interdisciplinary Survey of Audiovisual

Culture. Köln: Walther König, 2009.

Daniels, Dieter; Naumann, Sandra. Audiovisuology 2 (Essays) — Histories and Theories of Audiovisual Media and Art.

Köln: Walther König, 2011.

Deveraux, Paul. Stone Age Soundtracks — The Acoustic Archaelogy of Ancient Sites. London: Vega, 2001.

Ernst, Wolfgang. Archives in Transition, in: Digital Memory and the Archive. Minneapolis / London: University of Minnesota Press, 2013a.

______________. Between Real Time and Memory on Demand, in: Digital Memory and the Archive. Minneapolis / London: University of Minnesota Press, 2013b.

______________. Die Scheinbarkeit des “Live”: Irriatationen der Gegenwartswahrnehmung durch präsenzerzeugende 

Medien, in:  MAP _ Media | Archive | Performance (http://www.perfomap.de/map3/kapitel4/scheinbarkeit/pdf-download).

Faulkner, Michael. VJ: Audio Visual Art and VJ Culture Culture. London: Laurence King, 2006.

Fechine, Yvana. Televisão e Presença: uma abordagem semiótica da transmissão direta. São Paulo: Estação das

Letras e Cores, 2008.

Gumbrecht, Hans Ulrich. Produção de Presença — O que o sentido não consegue transmitir. Rio de Janeiro:

Contraponto / Editora PUC-Rio, 2010. p. 13.

Huhtamo, Erkki. Illusions in Motion — Media Archaeology of the Moving Panorama and Related Spectacles.

Cambrige (MA): MIT Press, 2012.

Ikoniadou, Eleni. The Rhytmic Event. Cambridge (MA): MIT Press, 2014.

Levin, Golan. Painterly Interfaces for Audiovisual Performance. Cambridge (MA): MIT, 2010 [Master of Sciences in Media

Arts and Science]

Licklieder, J. C. R. Man-Computer Symbiosis, in: Taylor, Robert. In Memoriam: J. C. R. Licklieder (1915-1990). Palo Alto:

Digital Systems Research Cente, 1990.

Lund, Cornelia; Lund, Holger (2009). Audio.Visual. On Visual Music and Related Media. Stuttgart: Arnoldsche

Verlagsanstalt, 2009.

Michaud, Philipe-Alain. Aby Warburg e a imagem em movimento. Rio de Janeiro: Contraponto, 2013.

Miyazaki, Shintaro. Listening to Wetware Circuitry — Sonic Experimentations and Algorhytmics, in: Rohlf, Jan et al. CTM
2015 — Untune Magazine. Berlin: CTM, 2015.

Parikka, Jussi; Hertz, Garnet. Zombie Media: Circuit Bending Media Archaeology into an Art Method. Cambridge (MA):

Leonardo, vol 45, n 5, 2012

Popper, Frank. Communication Art, in: Art of the Electronic Age. New York: Thames and Hudson, 1993.

Punt, Michael. The Jelly baby on my knee, in: Eves, Frans; van der Velden, Lucas; and van der Wenden, Jan Peter (Eds).
The Art of Programming — Sonic Acts 2001 conference on digital art, music and education. Amsterdan: Paradiso /
Sonic Acts Press, 2002.