segunda-feira, 7 de junho de 2010

Augmented reality in art

Claude Monet — Impréssion Soleil Levant, 1872

Beyond representation


A witty article – “L’exposition des impressionistes” – written by the painter, engraver and playwright Louis Leroy, for the satiric newspaper Le Charivari, named and consecrated at one go the most important aesthetic European movement of the last span of the nineteenth-century. The succession of incidents that led to the exhibition organised in 1874 by the Société Anonyme des Peintres, Sculpteurs et Graveurs, at the photographer Félix Nadar’s study, had begun eleven years before, when Edouard Manet saw the Salon de Paris of 1863 refuse his scandalous “Le déjeuner sur l’herbe.” This censorship by the French academy would lead the Emperor Napoleon III to decree the carrying out of a Salon des Refusés, to let the public judge the merit of the artistic creations of the so-called “refused” (works). By 1864 Manet would exhibit the first of a series of scandalous pictures for that epoch.

There are three elements in this story that I would like to explain short, as I am writing regarding a reunion of artists and professionals immersed in technologies of virtual realism of computational origin.

Curiously Manet refused to participate in the show that would be considered the first exposition of impressionist painters. On the other hand the tremendous criticism that Louis Leroy addresses against the paintings of the exposition mentions openly their lack of definition:

“Impression, I was sure of it. I also told myself, since I am impressed there must be there any impression – and what a liberty (freedom), what an easiness in this craft! A preliminary drawing for a wall paper has more definition than this sight of the sea.” (1)

Finally, the exhibition organised by Pissarro, Monet, Sisley, Degas, Renoir, Cézanne, Guillaumin and Berthe Morisot (the only woman painter belonging to this group), took place at Nadar’s studio, one of the most famous and inventive pioneers of photography that by then came off from his first experimental phase (Niépce, 1822, 1825, 1826; Niépce & Daguerre 1825-1829; Fox Talbot, 1834; Daguerre, 1839).

Nicéphore Niépce, heliografia (1825) Wikipedia

By then time exposure portraits, landscapes and city sights were already frequent, as well as object and machine images made in a mechanic way, i.e. through the direct action of day light over photosensitive materials. These surfaces chemically emulsified had and have the quality of retaining and fixing in image a certain exposition to the photons reflected by the illuminated objects. That that doesn’t reflect the light, because it lets it pass, or because it has an absorbent colour, is black, and that, that reflects the light in all its visible spectrum, is white. Between these two extremes there is a long range of grey. The outlines (contours) are abrupt transitions of state, shape, colour and luminosity. The line doesn’t exist. The grains do, like the pioneers of photography saw, when they understood the physics of the chemically emulsified and then sensitised particles.

Some impressionist painters, from Monet and Pissarro to Seurat, exactly understood this extraordinary important fact of perception. The points of primary colours congregate in spots, the colour and intensity transitions of which are understood as contours, volumes and lines – i.e. as images built along a complex, an interactive and ultra-rapid process of sensorial impression and of emotive and cerebral work.

Manet (1832-1883) refused the invitation from the younger rebel painters like Monet (1840-1926), Renoir (1841-1919), or Cézanne (1839-1906). Why? Only because they belonged to another generation? Nadar (1820-1910), who received the future “impressionists” at his photography study, was a decade older than Manet and a score older than Monet! So there must be another explanation.

This is my interpretation: owing to an extraordinary conjuncture, I think the behaviours of Manet, Nadar and of the artists of the Société Anonyme express the three founder movements of the modern culture of the second half of the nineteenth-century and of all the twentieth-century.

Nadar — Sarah Bernhardt (era como aqui a vemos...)

Manet represents the provocation and the urbanity of the new realistic programme announced by Goya (1746-1828), Géricault (1791-1824) and Courbet (1819-1877). Nadar is the leading figure of the surprising emergency of the technological realism, which, in spite of the innumerable falsifications, manipulations, and now special effects, goes on expanding like a sort of an absolutely facsimile speculation of reality – “ça a été” (Barthes, 1980). The Impressionists at last opened the door to an interminable formal analysis of the artistic practice, working and helping their successors work towards abstraction and later on accepting to welcome the iconoclastic traditions, originated in the Protestantism and even in the Zen Buddhism.

Curiously we are in the presence of three distinct typologies of realism: the critical realism, the technological realism and the analytic realism. While the first permits the integration of the technological and aesthetic acquisitions of the processes of figuration, representation and speculation in an essentially political narrative, and the second innovates without any compromise in a sort of noematic crescendo of the representation apparatus (the determinable X — that identity through time called upon by Husserl), the third, finally, sets up a “destructive” discipline in the art.

However, if we elect “Avatar”, the film produced by the writer and inventor artist James Cameron, to illustrate one of the latest examples of the technological realism, we fall into a paradox: the extreme measure of realism obtained through stereoscopic digital film techniques (Reality Camera System 1) and augmented reality systems, which instantly allow us to see the result of the graphic computation of processes that capture the real movements of cinematographic action (using the producer’s “virtual camera”), is after all good for creating a narrative universe of pure fantasy and propaganda.

To adjust our theoretic presumption, we need to turn to two new causes of the modern and contemporary paradigm of the manipulation of the communication and symbolic representation processes. The former is called illustration, caricature, comic, anime, Ukiyo-e, and the latter, propaganda, public relations, seduction and language games.

One of the important arts of the critical realism of the eighteenth and nineteenth centuries is illustration, above all the one practised through engraving techniques by Hogarth (1697-1764), Goya (1746-1828), Daumier (1808-1879), John Tenniel (1820-1914) and Toulouse-Lautrec (1864-1901), among others.

The explosion of the means of mechanical reproduction of writing and of image, of which lithography (Alois Senefelder, 1796) and photogravure (Niépce, Daguerre, Fox Talbot) were powerful instruments, associated to the true revolution of the transport systems, in operation by then, made possible the appearing of a new phenomenon: the proliferation and popularisation of the means of communication and art. The emerging of an urban mass-society aimed at a new paradigm of communication, new artistic production ways and a radical change of the nature of aesthetic reception. This was what happened, although under the form of a true growing synthesis between merchandise and pleasure.

The libertarian narrative of the French Revolution, associated with the optimistic and commercial pragmatism of the Industrial Revolution displaced the centre of the symbolic communication and figuration, of the cathedrals, of the churchyards, of the imperial saloons, to the city of speed, multitude and light. A new Pop realism would no doubt appear from such a cultural agitation.

Carl Jung (1875-1961), Sigmund Freud (1856-1939) and later his nephew Edward Bernays (1891-1995) are three among a brilliant group of pioneers, who raised the knowledge of the person’s behaviour, and above all the mass behaviour to unimaginable heights by the hand of wizards, who until that time guided the consciences of the faithful and of the subjects. Adam Curtis, in his prized documentary film of 2002 for BBC, “The Century of the Self” emphasises the importance of Bernays (the author of Propaganda, a book that is not much known nowadays), in the creation of the present and omnipresent system of Public Relations.

“If we understand the mechanism and motives of the group mind, is it not possible to control and regiment the masses according to our will without their knowing about it? The recent practice of propaganda has proved that it is possible, at least up to a certain point and within certain limits.” – (Edward L. Bernays, Propaganda, 1928).

“The engineering of consent is the very essence of the democratic process, the freedom to persuade and suggest.” – (Edward L. Bernays, The Engineering of Consent, 1947).
It seems so, that there is a very present realism, which didn’t exist when Impressionism appeared. For want of a better expression, let’s call it media realism. Why realism? Why not propaganda and manipulation?

If we think a little over the present publicity, at least the most creative (which “Postman Returns”, by PostPanic, is a good example), what do we have in common?

I would say, first we have a good story or a good anecdote, then seductive images, musical rhythm and at last a quasi-order in the shape of a tempting invitation or kind blackmail. The most important thing though is that the communication and the seductive shape have a precise objective here: to lead us to reality, or at least to an effective and immediate part of the surrounding reality. In the thicker and thicker and more complex labyrinth of the city, publicity is a vector of communication, information, social and cultural status. Because the urban and post-industrial obsolescence is enormous and the post-modern memory too volatile, realism, clearness, rhythm and humour – which is simultaneously an expression of critical realism and mnemonics – are crucial for an efficient way of commercial communication. The consumer needs help in the stream of material and virtual objects which flow into his choice possibilities. It is in this dialectics that the communicational intelligence becomes critical and needs a type of special creativity, lexical and dyslexical at the same time, where the qualia (and no more the aura) appears as indispensable. The commercial and informative propaganda is for the effect of this analysis the same reality, the media reality.

R. Crumb [in Wmagazine]

If we finally consider the worldly realism, that goes from William Hogarth (1697-1764) to Robert Crumb (1943-), and also through Hokusai’s Japanese pictures and the great influence these “images from the floating world” (Ukiyo-e) had in the nineteenthcentury Europe and went on having during all the twentieth-century, not only in Europe, but also in the United States of America, influencing decisively the emerging of the comic strips, of illustration bands published in the press, and the author’s editions and comics magazines, and are still continuing to influence such strong and global urban aesthetic movements like anime and manga, we can’t help registering here an important and powerful underground movement, without the educated preoccupations of the critical realism, properly so called, but nonetheless less perspicacious and contusing. In other words: that that distinguishes the worldly realism from the educated critical realism is the exaggerated sense of humour, the worship of mockery and provoking eroticism in opposition to the palatial game of shadows of the critical realism. Another important distinction derives from the audiences that each of these two realisms convenes. Manet’s public has never been the same that has been devouring Crumb’s heavy fantasies, although it certainly shares the taste for Hokusai’s pictures. The discreet production for an aristocracy of art appreciators is not to be mixed up with the mass production addressed to the urban crowds.

This short text, meant to isolate the core of the present electronic digital imagination, would need some more time and detail to avoid literal reading of the ideas expressed here. For example, how to explain Walt Disney – or Shrek, Hulk, T-1000, or Avatar – according to the different incarnations of realism that have been described? Where do the Teletubbies stay in this divagation?

The present electronic digital imagination lives somewhere at the intermittent contact point between technological realism, media realism and worldly realism. The skeletons, the hard cuirasses and the increasingly complex and hybrid grey matter of the digital world, form a kind of mutant techné, the applications of which demand increasing dedication and learning by the human race. From the initial realism, whose improvement already allows the digital world to make perfect illusions, we set out for a kind of augmented reality, or immanent artificiality, in which genesis and development, the interference of the collective of cognitive gods, producers, programmers and designers, who perform the creative process, will have a tendency to be dispersed and densified simultaneously into a finer and finer network —full of knots, levels of complexity and diversified degrees of interference, from where the new artificial life will start some day its development.

Meanwhile another paradox remains: the quicker the processors are, the more time and dedication are demanded from those who cause the creative processes.

The goal and the wish always meet a step ahead!


  1. In “Exhibition of the Impressionists”, Wikisource. And “Le 28 avril 1874, Louis Leroy écrivait dans un article intitulé “L’exposition des impressionistes”: “Impression, j’en étais sûr. Je me disais aussi, puisque je suis impressionné, il doit y avoir de l’impression là-dedans” — in Impressionisme.

[delivered for publication on April 3, 2010; published by Sines Digital on June 4, 2010]

Copyright © 2010 by António Cerveira Pinto

sexta-feira, 4 de junho de 2010

Mental thing

Universal Automata are sculptures printed in plastic from a 3d printer. The sculptures explore the creation of volumetric space through progressions of cellular automata algorithms. Andre Sier.

“Seeds rather than forests”

In the second half of 1997, I conceived an interactive, geo referenced map of my country that would connect a virtual navigation of a map of the territory to websites then exponentially popping on the Internet. The project would finally be presented at EXPO’98 under the name Portugal Digital.

For this purpose, I consulted and brought together several Portuguese institutions: the Instituto Superior Técnico, the Universidade Nova, the Centro Nacional de Informacão Geográfica and the Instituto Geográfico do Exército.

To compile and program the project, I received the help of Joaquim Muchaxo, one of a cluster of IT engineers who made the project viable in time for the big exhibition.

To calculate and visualise in real time all the processes that called up and compiled the data, it was necessary to buy a Silicon Graphics supercomputer, the SGI Onyx2 Reality Engine, with 4 GB of RAM and a 195 MHz processor.

In 1998, the cost of this machine was around 600,000 euros! Today, 13 years later, the same computing power costs no more than 3000 euros, i.e., 200 times less!

Measuring this technological revolution from another angle, for example, that of the virtual population of the Internet, we can confirm that there were 147 million users in 1998 while today the number has risen to 1,966,514,816, that is, it has increased 13- fold.

In little more than a decade, the technological revolution that was underway has led to a cognitive and sensorial fabric that is hybrid, digitally interactive, half-human, half-machine and whose degrees of freedom grant it an enormous linguistic and visual elasticity and a growing, even invasive, ubiquity. From a triangulation of communication satellites, this new superhuman skin is covering the planet with a film of wholly unexpected and transformative meta-reality.

Curiously, in 1999, one year after the presentation at EXPO ’98 of the unknown prototype of what, in 2005, would emerge as Google Maps (the fruit of another venture), André Sier, then a student at AR.CO, was presenting his first computational art project, 0 0 255, which, although inspired by the first-person shooting game Unreal, clearly deviated from the game’s ideology.

While computer and video games follow iconic and narrative models that stem from the imagination and from popular urban culture, not infrequently arising from the vast world of cartoon adventures, animated cinema, and sci-fi literature, the typical stripped-down nature of André Sier’s interactive worlds, while taking maximum advantage of computational engines, algorithms, libraries and available programming languages, clearly point to another cultural tradition: that of the essentialist and analytical aesthetics of one of the most important areas in nineteenth and twentieth-century modern art: the tendency towards abstraction.

Unlike the Jodis’ fantastic deconstructions of games such as Wolfenstein 3D, Quake, Jet Set Willy and Max Payne 2, André Sier follows a more constructivist approach. His distancing from what could be called entertainment, popular culture, commercial art, or the creative industries does not take place under a regime of divergence from this sort of alienated reality, which the Jodi’s hacker ideology so thoroughly distorts and scandalously exposes. Rather, it occurs as a construction of new possible worlds using the same genetic tools that industry uses for purposes as varied as warfare and popular agonistic culture.

Observing Sier’s work, as I have done for many years, I know that it is in itself a progressive record of sedimentation and generative expansion, accumulating strategies, algorithms, possibilities, designs, grammars, libraries, actors, environments and narratives, whether constituted or potential. The pieces evolve in series, precisely because they are worlds of autonomous possibilities which can iterate and gain in complexity, depth, definition and colour by way of automatic, aleatory, genetic and interactive processes, both endogenous and/or exogenous.

What makes the immersive worlds ordered by André Sier so fascinating is the intimate correlation, as it were, that exists between the intuitive drift of his oneiric constructions and the purely mental and logical techne that is rigorously pursued by someone who, in the circumstances of his own conscious creative process, cannot fail to be considered an artisan, or a technician, committed to master a language discipline to better tackle this matter, which invariably resists not only modelling but also the word and the final gesture that heralds the birth of a great work of art. In this case, the mass of the creation consists of zeroes and ones, or more precisely, binary combinatorial processes based on series of 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc., the activation of which depends on a bang — the discreet echo of a primordial “big bang” [Mark Whittle: Big Bang Acoustics].

Não Newtoniana (8x) from Andre Sier on Vimeo.
André Sier: continuum

The genetic revolution of products arising from instructions followed and algorithmic possibilities depends, from the start, on a strategic design, or, from the Deist perspective, a demiurge, or rather, that which is between God and the Realised Thing.

During the last century, the long analytical trend in modern art arrived at two apparently antithetical critical movements from which, it was then supposed, Western art would move inexorably towards a phase of revivalist and academic decadence (which was the case). These two movements were known as minimalism and conceptualism. They were two sides of the same coin: the phenomenological reduction of art as an object, or a thing in space-time, and as language.

Finally, a cosmopolitan cultural experiment, oscillating between logical mysticism and the voice of rhetoric, was born out of this dilettante phenomenology. However, things went well until Carl André, Donald Judd, Dan Flavin appeared in the minimalist camp, and Sol Lewitt, Joseph Kosuth and Dan Graham appeared in the conceptual camp. Tragically well!

In some sense, we can now say that the general trend towards abstraction that accelerated after analytical postimpressionism (particularly that of Monet and Seurat), cubism, suprematism, neoplasticism and abstract art at large, reached its end during the 1960s and 70s with the emergence and decline of minimalism and conceptual art, both of which were prisoners of a reductionism that was more metaphorical than genuinely intellectual.

However, they left a legacy which, today, artists like André Sier can legitimately revisit by invoking the philosophical and aesthetic acuity of European art’s inestimable heritage that the renaissance undoubtedly started, and which rationalism, positivism, and German idealism subsequently raised to levels of complexity and metaphysical robustness from which there could be no possible return to the religious narratives that dominated the sentiment and procedures of art for hundreds of thousands of years.

During the twentieth century, literature, fine arts and philosophy itself reached the degree zero of their respective constitutive and cultural paradigms. With forms being stripped to the most radical abstraction – the sort of return to geometry and logic that dominated the spirit of European and American intellectuals and artists from Monet to Roland Barthes – there remained the time in which to anatomise the processes by which several languages, authorial psychoanalysis, and the sociology of reception were generated. 

In 1936, the mathematician, logician and cryptologist Alan Turing had already published his description of a “mental experiment” called the “a(utomatic)-machine”, which would subsequently become known as the Turing machine. A “universal Turing machine” (UTM) is a machine that manages to simulate any other Turing machine (1948), and the Turing test is a way of assessing a machine’s ability to display intelligent behaviour.

During the Second World War, Turing was recruited by Winston Churchill to help the British Intelligence to decipher the coded messages from the German Navy, the encryption of which was carried out by two rotor-based electromechanical machines, the Enigma and the Lorenz (the latter being used strictly to encrypt the messages of the German high command). At the time, German submarines were responsible for sinking thousands of ships, particularly civil vessels which transported people, provisions, equipment and various materials (particularly for the war) between the American continent and wartime Europe. The German encryption machines, the origins of which dated back to the First World War (1914-18), seemed impossible for the Allied human cryptologists to break. It was then that Alan Turing, a member of the team of cryptologists working at Bletchley Park, also known as Station X, and his theories about computational numbers and automatic machines left an indelible mark on the procedures that led Tommy Flowers, the Post Office Electronics Engineer, to design and finally build a machine that was capable of emulating the coding operated by the rotors of the Lorenz and thus to decipher the messages of the German high command on the eve of the Allied landing in Normandy, known as D-Day.

Colossus Mark I and Colossus Mark II were therefore the first two electronic machines designed to digitally process information that were ever built for practical purposes, as well as being the absolute pioneers of modern day computers. This brief historical incursion is important if we are to understand the founding epistemological leap taken by what can properly be called the start of the postmodern era. In other words, the moment from which the understanding and human manufacturing of possible worlds moved, at least partially, from work that was merely human, physical and intellectual to the work of intelligent machines. Rather than painting forests or building worlds as Brian Eno said in a particularly elegant and poetic formulation, the postmodern creator, a sort of agnostic and post-industrial monist, devotes himself to sowing generative principles from which he expects new harmonic constellations to emerge – “seeds rather than forests.”

John Conway’s cellular automata (developed by Bill Koster and Stephen Wolfram, among others), Karl Sims’s genetic algorithms, and Craig Reynolds’s swarms are some of the paradigms of the new emerging culture in which André Sier is clearly located, along with many other contemporary, or rather post-contemporary, creators (to the extent that their creations are not “actual” but potential, incorporating past, present and potentially future states). Being among the youngest of the cognitive and computational Portuguese artists, André Sier is one of their most serious, original and remarkable representatives.

There is still a learning curve to be climbed regarding the dynamic reception of generative and interactive works that have been created outside of the strict disciplines of music, environments, and installations aimed purely at the ear. Responsibility for this cultural delay primarily falls upon the conservative inertia of the museum and gallerybased world of so-called “contemporary art.” While popular electronic culture has progressed at an exponential rate, as incontrovertibly attested by the sociological, economic and strategic importance of the games industry, the generative and cognitive arts in general remain encapsulated in a sort of “pre-artistic” limbo, as if they were strange beings which were not yet fully entitled to enter the “adult” world of art. This institutional delay will be overcome, probably after a big bang, which I believe lies around the corner. When we least expect it, the cognitive and generative arts will enter our neurones with the same apparent naturalness, speed and irresistible impregnation as an algorithm as revolutionary as that which led to the birth of Google. The preparatory work has been underway for a long time and the philosophically possible worlds of André Sier are surely part of the swarm that will produce the next big change in the τέχνη (techne).

Finally, in this brief introduction to the exhibition that André Sier had at the Museu de São Roque (Lisbon) I will leave you with some notions to remember when we see, hear, feel, perceive and interact with some of the pieces that make up
  1. The perceptive environment is multi-modal: space, object, sound, image, interaction, retroaction, ghost, connection, network, sharing, suspension, interval, continuation, potential.
  2. is not a finite world but a cosmogony of possibilities, computationally generated on digital foundations with various (32-bit and 64-bit) extensions. In this case, sentences like “I went to see André’s exhibition”, or “I liked Siers’s installations”, are incomplete and describe only the memory of a highly incomplete and ephemeral perception of the potential reality inscribed in the works of art on offer, the apprehension of which actually requires the apparently infinite time of games.
  3. Creatures imprinted and taken from the digital world of possibilities, inscribed or unleashed by human-machine interaction — a game, individual or collective, whether aleatory or built, shared, or simply an accumulation of possibilities — are the perceptive, sensorial and physical proof of a real emergence that is therefore much nearer to us than the merely fictional or simply virtual worlds of the prehistory of cognitive art.
Copyright © 2011 by António Cerveira Pinto