Tyson Frederick

Algorithmic Art Systems: Early Research in Creative AI Interaction

Peer-reviewed at the CADRE Laboratory for New Media and executed in public at the 2014 Bay Area Maker Faire, with the source beside the image.

Generative coordinate field from theta2, a Processing sketch, 2012

The Scholarly Problem

A deterministic system and a humanistic reading answer different questions. The system reports the rule it executed. The reading supplies a meaning the rule never stored. I took that gap as the problem of a three-year MFA at the CADRE Laboratory for New Media, and I built generative systems that keep the two answers separable.

The catalog, A Creative Process (May 2014), works from three instruments, and each instrument carries a limited warrant. I read the Communist Manifesto of Marx and Engels as a structural hypothesis: capital accumulates by turning interpretations into products, and alienation names the moment a maker meets that product as a foreign object. The Manifesto supplies the premise I tested, that a finished appearance can conceal the process which produced it. Bolter and Grusin, in Immediacy, Hypermediacy, and Remediation, supply the perceptual companion. Remediation spends its effort on feeling immediate. From Buddhist instruction I took śūnyatā in a working sense: an external object is empty of inherent meaning, and every label on it is imputed. The catalog argues that working sense. A scholarly edition of the Madhyamaka sources lies outside its claim.

Saturated media hides a premise inside repetition. Exposure arrives dressed as knowledge. Exposure and knowledge are different categories. Exposure establishes that a surface returned. Knowledge still requires an argument.

The systems themselves are rule machines. I wrote them as loops, stochastic variables, and vector arrays in Processing and ActionScript. The same failure mode meets a trained model. A reader treats a generated surface as a claim about the world. The surface records a procedure. The claim needs a warrant of its own. The prototypes hold that distinction where a committee and a public room can inspect it.

The Architecture & Pipeline

Five phases. Each phase closes on an artifact a reader can check.

Phase 1: Epistemic Deconstruction

I separated primary experience from secondary interpretation before I wrote a generative rule. Primary experience, here, is the operation: an index drawn from a table, a coordinate written to a buffer, a frame committed to the screen. Secondary interpretation is the story attached after the frame: urgency, authenticity, a market verdict, an authorial mood, an institutional blessing.

Alienation gives the structural account of that attachment. The producer loses the process. The consumer receives an appearance. I turned that account onto media saturation and named the hidden premise: repetition has already done the verifying. Remediation locates the same premise in the interface, which labors to feel unmediated. Śūnyatā blocks the last step. The reader makes the label. The object stores the operation, and the label arrives from outside it.

I kept each warrant inside its domain. A theory of capital explains a production relation. A theory of remediation explains a perceptual effect. A doctrine of emptiness explains why a label needs a maker. Together they identify the gap. Assigning a meaning to any single frame takes an argument the three instruments, by themselves, withhold. The prototypes exist to expose that remainder.

Artifact: A distinction between the operation a system performs and the meaning a reader imputes.

Phase 2: Steel-Manning Historical Frameworks

I stated each historical program at full strength before I named where it breaks. A steel man gives the claim its best warrant, so the failure belongs to the claim.

Modernism, at that strength, holds that a disciplined procedure can reach a form binding on every case. The procedure is a real achievement. The universal quantifier is a further claim, and it changes the category. A style is a generator with constrained variance. A truth that covers the domain has to survive the cases it would rather drop. The catalog names those cases. A harmony that cannot register sexism, unequal rights, or racism has left the standard it announced. The category error is the promotion of a method into a universal. Variance is the term the method defined away.

Postmodernism, at that strength, holds that the modern universal lacked warrant, and that excluded viewpoints stand as counterexamples. The objection is sound. The catalog then follows the next step: postmodernism specifies a new single viewpoint as the ideal. A single viewpoint still fails to scale across the variance that justified the revolt. Negating a false universal yields no decision procedure for the descriptions that remain. Historical narrative restates the conflict. A running algorithm shows a reader the difference between a rule and a reading. I needed a test that could fail in public.

Artifact: Two category errors, stated so a program can miss them.

Phase 3: Computational Prototyping (Processing & ActionScript)

I wrote the logic myself. Processing 2.0 sketches, in Java, and ActionScript 3.0 applications, in Flash, put loops, stochastic variables, and vector arrays between the premise and the frame. I ran the iterations at interactive speed so the corruption of meaning stayed visible across time. A still settles into one reading. A live loop keeps the rule present while that reading forms.

The MFA catalog documents the wider corpus in plates: theta2 and The Steps to Freedom, both Processing, 2012; Ladders and Chutes, ActionScript 3.0, 2013; Blue Rain, ActionScript 3.0, 2014; Our voices can't be heard, Processing, 2014. Three prototypes state the evaluation in its most direct form.

Bull vs. Bear (2012). I wrote a script that executes stochastic volatility inside a market simulation, and I used it to evaluate logical data flow. The settled output is a sequence of state, perturbation, and next state. The sequence has the shape of a verdict. A bullish or bearish reading treats the trace as if it had weighed evidence. Volatility is a perturbation rule. A warrant about the world is a separate argument. The prototype makes the hidden premise available for inspection: a coherent path presents itself as a conclusion.

What's Important to You (2012). I built a path-finding node graph to test associative meaning. An edge stores proximity between terms. A traversal can finish, and every step can be legal, while the path still lacks entailment. Entailment asks for an inference the edge leaves unstored. The graph holds the smuggled premise up to view. Linkage feels like reason because each node arrives already beside the last.

Iterations and San Francisco (2012–2013). I built generative coordinate-mapping systems to test visual noise and systemic rendering. Iterations, 2012, leaves the mechanism in the frame. The sketch opens a 1920×1080 stage at 80 frames per second, loads poll.txt, splits the first line on commas into an integer array, and on each pass selects a random index from that array against the coordinate tables declared in the source. The numerals record selections. San Francisco, in the same series, maps coordinates into a rendered field and poses the question at the scale of the picture. Density, overlap, and noise recruit the eye into a scene. The renderer commits vectors. I treated visual noise as a corruption channel. The system generates a field. The viewer processes the field into a scene. The scene takes the place of the rule. That replacement is how meaning corrupts: a secondary interpretation occupies the position of the primary operation and then speaks as the system.

Artifact: Three running tests, plus the catalog plates that surround them.

Processing sketch Iterations, 2012, source and rendered title in one frame

Phase 4: Academic Peer Review & Defense

I placed the outputs and the humanistic argument before the CADRE faculty. In April 2013 I defended the Advancement to Candidacy before G. Craig Hobbs, Shannon Wright, and Stan Welsh, Department of Fine Arts, San Jose State University. I answered three questions: whether the historical claims were sound, whether each program tested the claim I had attached to it, and whether the rule stayed distinct from the reading. In May 2014 the School of Art and Design accepted A Creative Process toward the MFA.

The defense narrowed the result to a standard. The algorithms hold a generative rule still. A reader can check an ideological reading against the operation that committed the frame. The faculty examined that standard. The candidacy and the degree record that the standard held.

Artifact: Advancement to Candidacy, April 2013. MFA catalog, May 2014.

Phase 5: Public Execution & Epistemic Transparency (Glass Box)

I carried the same distinction into a public room at the resolution the committee had examined. The closing form was a cyber-physical installation. Through OpenNI, a Kinect took gestural input, and the generative systems rendered a live field in the gallery, so a visitor's movement met the rule while it ran. At the 2014 Bay Area Maker Faire, the CADRE exhibition Litewall and Interactive Projects included this installation. The Faire awarded that exhibition an Editor's Choice.

I exhibited the raw source in the same room as the generative image. From that pairing I define good model behavior. A generative system behaves well when a visitor can recover the rule without a docent: which values the random draw produced, which coordinates the arrays fixed, which tokens the file supplied, and which meanings the reader imputed after the fact. This standard of transparency publishes the structural rule. An ideological reading has to survive contact with the rule. Where the reading runs past the procedure, the excess stands as interpretation, and the visible code is what marks it.

Artifact: A public room in which source and image share one view.

The Outcome

The research leaves a rubric for generative media. The same three tests apply to a trained model's output, because the failure mode lives in the reading.

Factual accuracy. Name the operation before naming the world. An index selection is a draw from a table. A coordinate is a position in a buffer. A step of stochastic volatility is a perturbation of state. Each sentence states a fact about the program. A fact about the world needs evidence the program leaves out.

Logical coherence. Test the inference the surface proposes. An edge records association, and entailment still wants its own warrant. A style records a constrained generator, and a universal wants coverage the generator defined away. The collapse of that universal records a failed quantifier, and any viewpoint offered in its place still needs a procedure if the viewpoint claims to scale. A loop can be valid, in the strict sense that it does what it specifies, while the interpretation attached to it stays unsound. The hidden premise, that the picture is the argument, stays outside the source.

Public transparency. Publish the rule with the output. The Maker Faire installation is that requirement in public form. Source and image share a room, and the secondary story meets the primary procedure in front of a visitor. I call the arrangement a glass box. A glass box resists ideological flattening because an imputation has to show itself as an imputation once the rule is readable. The rubric holds one separation: settled computational fact, humanistic interpretation, and a visible rule that lets a reader tell them apart.