Creative Coding Notes
Table of Contents
- Creative Coding History
- Modern Creative Coding Uses
- Graphics Concepts
- Creative Coding Environments and Libraries
- Communication Protocols
- Multimedia Tools
- Unique Displays and Touchscreens
- Hardware
- Other output options
- More resources
In addition to this list, check out these awesome lists:
- https://github.com/terkelg/awesome-creative-coding
- https://github.com/kosmos/awesome-generative-art
- https://github.com/danvoyce/awesome-creative-tech-events
- Open source, experimental, and tiny tools roundup by Everest Pipkin
- Graphics Programming Weekly
- Visual Programming Codex
- Generative Arts Newsletter
Creative Coding History
- Computer graphics & generative art roots
- Algorithmic Signs - Five pioneers of computer art in conversation
- John Whitney - Catalog (1961)
- Pioneer computer animator John Whitney Sr. in 1959, operating one of the first computer-graphics engines, a mechanical analog computer built largely from surplus World War II anti-aircraft guidance hardware. Source
- "How Computer-Generated Animations Were Made, Circa 1964"
- John Whitney - Lapis (1966)
- "The Incredible Machine" (1968)
- Jim Henson - Cyclia projection mapping prototype (1968)
- Doris Chase - Circles (1969-1970)
- Edward Ihnatowicz - "Senster" (1970)
- Norman McLaren - Synchromy (1971)
- Erkki Kurenniemi - DIMI Ballet (1971)
- John Whitney - Matrix III (1972)
- Ed Catmull & Fred Parke - first 3D rendered movie (1972)
- Manfred Mohr - Cube Transformation Study (1972)
- Ed Catmull & Fred Parke - Early CGI Facial Animation (1974)
- Manfred Mohr - Cubic Limit (1974)
- The Lost Art of Canada's Doomed Pre-Internet Web (1970s-80s) (Telidon)
- Carla's Island (1981)
- SIGGRAPH 1982 Art Exhibition Artist Interviews
- The Road to Point Reyes (1983)
- Calculated Movements by Larry Cuba (1985)
- Computer Music (Synthesizers, Synclavier) News Report w/Stanley Jordan (1986)
- Vincent John Vincent - Immersive Video Gesture Control (1986)
- Myron Krueger - Videoplace (1988) and more
- Samia Halaby
- A brief history of computer drawing
- William Latham - BioGenesis (1993)
- Karl Sims - Evolved Virtual Creatures (1994)
- Scott Snibbe - Motion Phone (1995)
- John Maeda (1990s)
- Romy Achituv & Camille Utterback - Text Rain (1999)
- Golan Levin and Zach Lieberman - The Manual Input Sessions (2004)
- Historic overviews
- The Recode Project - A historical reference for generative art
- A Brief History of Graphics
- "Why Love Generative Art?"
- 3D texturing history
- Some More Beginnings: On the History of Time-Based Media Exhibitions by Tina Rivers Ryan
- Poetic Science: The Origins of Digital Printmaking (IFPDA)
- Demoscene
- Flash era: interactive motion graphics
- GMunk
- Robert Penner (easing algorithms)
- Grant Skinner
- Physical computing
- Sega Motocross / Fonz (1976)
- Nintendo Power Pad 1 (1988)
- Nintendo Power Glove 1 (1990)
- VR (1977-present)
- Dance Dance Revolution (1998)
- Lists of modern projects
Modern Creative Coding Uses
- Data visualization
- Musical Instruments - Imogen Heap's musical gloves
- Math as music Simple Harmonic Motion
- Modern demoscene: Agenda Circling Forth
- iTunes visualizer / Winamp (Milkdrop)
- Music videos
- Modern dance - Unnamed Sound Sculpture
- Human movement analysis to art - Forms - Quayola and Memo Akten
- Generative jewelry - Nervous System
- Product design & development
- Fashion - Kinematics Dress
- Quadcopters
- 3D Printing - Probability Lattice
- Robotics + Projection mapping + 3D Motion graphics - Box
- Generative visuals - Algorithmic Abstractions
- Generative audio - The Ballad of the Psychotropic Robots & Wiki
- Science - Crowdsourced protein folding
- Interactive art installations & physical experiences
- Augmented reality
- Digital Art
- Computer-Aided Design
- Architecture
- Game development
- Mobile app development
- Web development
- Non-web UI development
- Touchscreen applications
- Projection Mapping
- Machine learning & AI
- Photogrammery
- Kinetic Sculpture
- Immersive spaces
- Body augmentation
Graphics Concepts
Basic Concepts
- Shapes
- Color
- Similarities between drawing environments
- Motion (frame-based vs. web rendering)
- Repetition (for() loops)
- Iteration
- Randomness
- OOP
- Physics / better motion
- Oscillation
- Basic trigonometry (movement & orientation)
- Nature / fractals / Recursion
- Higher level math
- Math as Code by Jam3
- Shaping functions
- 3D
- Data
Visual algorithms
Check out this nice visual list of some of the concepts below. And some code examples of another set of common algorithms, and also here.
Commons tools & concepts
- Anti-aliasing algorithms
- Barycentric vs. Bilinear
- Bézier Curves
- Bilinear sampling
- Blob detection
- Cellular autonoma
- Collision detection
- Computer vision (CV)
- Convolution
- Concave & convex hulls
- Delaunay Triangulation
- Diffusion Limited Aggregation
- Dilation / Erosion
- Dithering
- Easing & interpolation
- Edge detection
- Edge distance in Regular Polygons
- Emergence
- Feedback 2
- FXAA
- Genetic & evolutionary algorithms
- GPU architecture types explained
- Hardware acceleration
- Instancing
- Kalman Filters
- Light Bouncing
- Linear Interpolation
- Lissajous Curves
- Lorenz & Strange Attractors
- Map projections
- Marching Squares
- Metaballs
- MIP mapping (Explanation)
- Motion Blur
- Munching Squares
- Noise
- Perlin noise for smooth randomness, terrain, textures. More info
- Curl noise
- Fractal Brownian Motion
- Blue noise
- Outlines
- Random walkers
- Packing
- Parallax
- Particles
- Post-processing effects
- Poisson Fill
- Polygon expansion
- Reaction-diffusion
- Space-filling algorithms
- Strange Attractors
- Style Transfer
- Subdivision
- Text rendering
- Triangulation
- UV Coordinates
- Vectors
- Voronoi 2
- Wave Function Collapse
3D / spatial concepts
- Ambient occlusion (Slides) & 3d lighting
- Bump/Normal/Displacement maps
- Constructive Solid Geometry (CSG)
- Curves and Surfaces
- Displacement
- Dot Product / Explainer gif
- Half-Edge Data Structures
- Logistic Regression
- Marching cubes & Isosurfaces
- MatCap
- Meshes
- NURBS
- Optimization via quadtrees or Octrees
- Physically-Based Shading
- Projection (3D)
- Raytracing
- Raymarching
- Volumetric Rendering
- Quaternions
- Ray tracing
- Shadows
- Signed Distance Functions (SDF)
- Spherical mathematics, Quaternions and Euler rotation 1
- Sub Surface Scattering
- Superformula
- Voxels
Natural simulations
Some of these borrowed from @AmnonOwed @ this repo)
- Algorithmic Botany
- Biomimicry
- Boids / flocking & particle systems
- Emergence
- Evolutionary Computing
- Flocking
- Flow Fields
- Fluid dynamics & solvers
- Form-Finding
- Fractal Modeling
- Fractals & Mandelbulbs 4
- List_of_fractals_by_Hausdorff_dimension
- Genetic Algorithms
- Greeble
- Laplacian Growth
- Lindenmayer-Systems
- Morphogenesis
- Space Colonization
- Physics simulations
- Reaction-Diffusion
- Self-Organization
Articles
General
- "Advice for Creative Technologists"
- "Why Love Generative Art?"
- "Join the Creative Class"
- "The Brief History of Volumetric Filmmaking"
- The Missing Semester of Your CS Education
Algorithmic design
- On Meta-Design and Algorithmic Design Systems by Rune Madsen
- So you want to build a generator by Kate Compton
- On Generative Algorithms by Inconvergent
- Sketching with Math and Quasi Physics by kynd
- The Algorithm Thought Police
- Programming Design Systems
- Generative Possibility Space
- GenerativeHut tutorials
- Line by wblut
- What makes generative art hard?
- A Programmer's Introduction to Mathematics
- Thoughts on Software for the Visual Arts by Casey Reas
- What is Generative Art? by Amy Goodchild
Graphics Programming
- https://learnopengl.com/
- WebGL Guide
- Introduction to VR Graphics Performance
- Casual Introduction to Low-Level Graphics Programming
- Computational Geometry Tutorial
- Graphics in Plain Language
- A glossary
- A trip through the Graphics Pipeline
- Lighthouse3d.com: GLSL Tutorial – Core
- The Graphics Codex
- http://antongerdelan.net/teaching/
- Foundations of Game Engine Development Mathematics
- GPU Performance for Game Artists
- GPU Optimization for Game Dev
- Ray Tracing in One Weekend
- NVIDIA - GPU Gems 3
- A Beginner's Guide to Coding Graphics Shaders
- 3D Game Shaders For Beginners
- List of technical case studies on WebGL and creative development
- @mattdesl's workshop-webgl-glsl
- Starting with Shaders
- Real-time refraction
- Generative Art with CSS
- Crosshatch Automata
- Material Design on the GPU
- ShaderQuest
- Shadertoy for absolute beginners
- Intro to Shaders by Cyanilux
- Graphics Pipelines for Young Bloods
- The Jacob's Ladder of coding
- An aerial perspective on Generative Design
Audio programming:
- George E. Lewis: Why Do We Want Our Computers to Improvise?
- Generating music in the waveform domain
- https://ccrma.stanford.edu/~jos/
Math:
Machine Learning:
Video Series
Find this big list from Bruno Simon
- Computer Arts Society
- Coding Math
- The Coding Train
- The Art of Code
- TheChernoProject
- Inigo Quilez
- Code Parade
- Hopson
- The Art of Code
- Sebastian Lague
- Ben Eater
- Brackeys
- ThinMatrix
- Kristin Stock
- GDC
- ACM SIGGRAPH
- Ars Electronica
- Revision Festival
- Algorithmic Art
- Freya Holmér
- 3Blue1Brown
- David Bouchard - RTA838 - Generative Processes
- Two Minute Papers
- The Code Creative
- Lewis Lepton
- Brackeys
- Colorful Coding
- thedotisblack creative coding
- Yuri Artiukh
- TouchDesigner
Courses
- https://online-learning.harvard.edu/course/cs50-introduction-computer-science
- https://www.edx.org/course/computer-science-101
- https://teachyourselfcs.com/
- https://github.com/prakhar1989/awesome-courses
- https://github.com/shiv4nsh/awesome-computer-science-resources
- https://github.com/onlurking/awesome-infosec
- https://github.com/donnemartin/system-design-primer
- https://github.com/infoslack/awesome-web-hacking
Creative Coding Tools
Web-based
- Realtime graphics libraries
- Live coding
- Rendering tools
- SVG
- 3d models
- Audio
- Drawing recognition
- Voice-recognition
- Web Speech API demo (Chrome only)
- General tools
- Face trackers
- Body trackers
- Fonts
- Augmented/Mixed/Virtual Reality
- Hardware access
- Multi-user sync
- WebSockets
- WebRTC
- Other WebRTC servers
- CSS
GLSL
- Interactive tutorials
- Web-based coding playgrounds
- Other GLSL coding tools
- glslViewer
- KodeLife
- Snelly (pathtracing)
- glslViewer
- Raymarching
- Raymarching Demo
- Raymarching Demo 2
- Raymarching Intro
- Raymarching: more
- Ray Marching by Michael Walczyk
- Raymarching Workshop
- Inigo Quilez: LIVE Shader Deconstruction
- Raymarching Template
- Getting started with shaders: signed distance functions!
- SDF Tutorial 1 : box & balloon
- Simple Raymarching Example Scene
- Shader Coding: Ray Marching Tips & Tricks
- Volume Rendering for Developers
- Other
- F3 App
- glslb.in (defunct?)
- LWJGL Basics by @mattdesl
- shader-doodle by @halvves
Java
c++
Swift
Rust
Game Engines
Node-based
Computer Vision / Image recognition
- OpenCV
- OpenCV AI Kit
- YOLO
- PoseNet
- Openpose
- DensePose
- AlphaPose
- PoseTrack
- OpenTrack
- OpenDataCam
- BodyPix
- VIBE
- OpenPTrack
- OpenFace
- Snap Camera
- Consistent Video Depth Estimation
- Worldsheet
- Virtual Background
Machine learning
A nice roundup of browser-based tools built with ML tools, via @golan
- Deeplearn.js
- Runway
- Machine Learning for Artists
- ml5js demo from scratch
- Wekinator
- Mediapipe
- Teachable Machine by Google
- Spleeter
- Demucs
- DrumBot
- Synopsis
- Descript
- DAIN
- RiTa
- Imaginaire (GitHub)
- OpenAI Jukebox
- NeRF
- Eulerian Motion Fields
- Vocal Remover
- Reinforcement Learning Library w/JavaScript
- ProsePainter - info
- SIREN
- Text2Mesh
- Github Copilot
- Midjourney
- Dall-E 2
- Stylegan
- Open-Unmix
Virtual Reality
Augmented & Mixed Reality
Data processing
Communication Protocols
Local network & web communication:
Hardware device communication:
- MIDI
- Lighting
- Open Fixture library (database of lighting fixture info)
- ArtNet
- DMX
- Doug Fleenor Design products
- sACN
- Shared textures between apps
- Video streaming
- Audio & MIDI device sharing via software
- https://github.com/PipeWire/pipewire
- Jack Audio Connection Kit
- BlackHole (Soundflower replacement for OS X)
- MIDI-OX
Multi-protocol timeline tools & stage helpers
Multimedia Tools
Scriptable 3d tools
- Blender
- Houdini
- Cinema4d
- Meshlab
- Meshmixer
- Rhino & Grasshopper
- After Effects
- http://www.meshrepair.org/
- Modo
Sound
- CLI:
- Sound eXchange (cli conversion tool, a-la ffmpeg)
- Aubio
- WaoN
- Processing:
- openFrameworks
- Max
- WebAudio
- JUCE
- SOUL
- Faust
- ChucK
- Earsketch
- Supercollider
- Extempore
- Sonic PI
- ORCA
- Overtone
- Alda
- EarSketch
- TidalCycles
- Magenta
- VCV Rack (+ plugin docs)
- HICE
- Elementary Audio
Video + Image
- Glipse Editor
- ffmpeg
- imagemagick
- gifsicle
- HAP video format (GPU-accelerated video)
3d Scanning
- Hardware
- Apps
Motion Capture
Windows automation
- MultiMonitorTool
- Get info & control physical displays, and move apps between them
- nircmd
- A variety of deep Windows automation tools. Also can set an app to top of stack
- cmdow
- Control app windows - move, resize, hide, etc. Must use MultiMonitorTool to move betwee displays!
Physical Computing
Modern input devices
- Apple trackpad/touchscreen touch/gestures
- Mouse / tablet
- Cameras
- See @laserpilot's Guide to Cameras for Interactive Installations
- Webcams
- https://www.amazon.com/Logitech-Widescreen-Calling-Recording-Desktop/dp/B006JH8T3S/
- https://www.logitech.com/en-us/product/4k-pro-webcam?crid=34
- https://www.amazon.com/dp/B01N8QBO2G/ref=asc_df_B01N8QBO2G5276731/
- https://www.amazon.com/s/ref=nb_sb_noss?url=search-alias%3Doffice-products&field-keywords=ELP+Webcam
- https://www.e-consystems.com/
- http://www.cctvcameradvrs.com/3.7mm-f2.8-m12-mount-pinhole-cctv-camera-lens-ir-350nm-650nm
- https://www.axis.com/gb/en/products/axis-p12-series
- https://www.axis.com/gb/en/products/axis-f-series
- https://www.ptgrey.com/
- https://www.vivotek.com/
- https://www.theimagingsource.com/products/industrial-cameras/usb-3.1-monochrome/
- http://www.z-cam.com/
- http://www.marshall-usa.com/cameras/
- Depth cameras
- Comparisons:
- Products:
- Kinect
- Machine learning for MS system white paper
- OpenPTrack
- Xtion
- Zed
- Structure Sensor for iPad
- Orbbec
- Intel Realsense
- Framos
- Intel Euclid
- Xovis
- eys3D
- Mynt Eye
- Nuitrack (skeletal deletion middleware)
- K|Lens One
- Kinect
- CV cameras
- 360 cameras
- High-frame-rate cameras
- Bullet-time cameras
- Lidar
- Ouster OS-1
- Slamtec RPLidar
- High-end
- Radar
- Ultrasound vision
- Touch surfaces
- Accelerometer/position
- Sensors
- Unique controllers
- Leap Motion 1
- handUI for Unity
- Mimu Gloves
- Myo
- Gest
- Litho
- Brainwaves / EEG
- E-textiles
- Buttons / Encoders
- Touchscreen
- Bluetooth
- Voice recognition
- Home Automation
Unique Displays and Touchscreens
First, check out Survey of Alternative Displays and Guide to Projectors for Interactive Installations by @laserpilot.
Also, 3D Displays: Opportunities & Design Principles by Bluecadet
And this great survey of historic electronic display technologies
- Volumetric displays
- Transparent displays
- Transparent displays
- Odd screen sizes
- Outdoor screens
- LED floors
- Flexible displays
- Flexible LED tile
- Tiny high-res display
- Flipdots
- HUB75/SmartMatrix panels
Big Displays
- Watchout
- Matrox TripleHead2Go
- Ambery HDMI multi-screen layout
- Datapath FX4
- Alternative: Gofanco Prophecy
Art Frames
Media players
Hardware
Installation-friendly computers
- Maker
- Small
- Rugged
Machine management
Network inspection
Prototcol Extenders
- Audio over ethernet
- HDMI over ethernet
- HDMI over fiber
- MIDI over ethernet
- USB over ethernet
- USB over fiber
- Dmx over ethernet - female and male
- DMX over Wifi
- USB over Wifi
Wifi 5G Hotspots
Video capture
- El Gato Camlink
- Mokose
- MiraBox Capture Card
- Epiphan AV.io
- Inogeni
- Magewell SDI to UVC USB plus Marshall Electronics CV343
Other output options
- Projection mapping
- LED lights
- Addressable LEDs
- DMX control
- Lasers
- Robot arms
- Laser Cutters
- CNC routers
- Cuttle software
- Pen plotters
- 3d Printers
- Printers
- Spatial audio
- Augmented & Mixed Reality
- Embedded systems
- HTTP-controlled power strips
- Robotics
- Textiles & quilting
- Water Features
More resources
Websites
- Intro to programming @ hello.processing.org
- Introduction to Computational Media
- The Nature of Code by Daniel Shiffman
- Creative Applications
- Create Digital Motion
- Prosthetic Knowledge
- Flowing Data
- http://hackingforartists.com/
- http://hackingforartists.com/
- OpenProcessing
- WebGL Fundamentals
- THREE.js Fundamentals
- Discover THREE.js
- THREE.js Journey
Books
- The Nature of Code by Daniel Shiffman
- Form And Code by Casey Reas
- Generative Design
- Generative Art
- Code as Creative Medium by Golan Levin and Tega Brain
- Coding Art
- When the Machine Made Art
- 10 PRINT CHR$(205.5+RND(1)); : GOTO 10
- Interactive Installation Art & Design by Wang Chen
- Cybernetic Serendipity: The Computer and the Arts
- Interactive Art and Embodiment: The Implicit Body as Performance
- Liquid Spaces: Scenography, Installations and Spatial Experiences
- 3D Math Primer for Graphics and Game Development
- Generative Deep Learning: Teaching Machines to Paint, Write, Compose, and Play
- Multimedia: From Wagner to Virtual Reality
- A Little-Known Story about a Movement, a Magazine, and the Computer's Arrival in Art
- Creative Code: Aesthetics + Computation
- John Maeda - Nature
- Maeda @ Media
- Computer Graphics-Computer Art
- Digital Visions: Computers and art
- Digital Harmony: On the Complementarity of Music and Visual Art
- Drawing with Computers - Mark Wilson
- Computer in Art
- A Touch of Code: Interactive Installations and Experiences
- Video/Art: The First Fifty Years
- Sound Art: Sound as a Medium of Art
- Tweets:
Conferences
- ARS Electronica
- Eyeo Festival
Inst-int- Visualized
Resonate- SIGGRAPH
- Transmediale
- Hackaday Superconference
- OFFF
- Gray Area Festival
- NODE forum
- FITC
- Sonar +D
- kikk festival
- Festival Arttec
- New Media Castle
Videos
- The Art of Creative Coding (PBS)
- The Coding Train by Daniel Shiffman
- Eyeo & Inst-Int conference talks
- GROW Paris conference talks
- How To Draw With Code | Casey Reas
- Demographics: Behind the Scene
- Nature of Code video series
- Introduction to Computational Media
- Designing Generative Systems w/ P5.js
- Clouds documentary
- Indie Game The Movie
- Demoscene - The Art of the Algorithms
- Makin' Stuff Look Good in Unity
- Exploring Emergence | Andy Lomas
- Juice it or Lose it
- Paul Prudence @ Synergetica Studiolab
- "Abuse of An Algorithm Comes As No Surprise" by Marius Watz
- Creating Interactive Visuals for Large Audiences (Joel Pryde)
- The Limits of Learning, by Kyle McDonald @ Sónar+D
- Computational Craft by Gillian Smith
- Generative Art - Computers, Data, and Humanity
- Kate Compton: Creating generative art with Javascript
- Tim Holman - Generative Art Speedrun
- Generative Machines with Matt DesLauriers
- Maker resources
- Shader Basics, Blending & Textures: Shaders for Game Devs by Freya Holmér
- How to think about 3D rotations
Podcasts
People
- 1960s-1970s - early innovators
- Vera Molnár
- Manfred Mohr
- Frieder Nake
- Georg Nees
- Doris Chase
- John Whitney
- Ed Catmull (later founded Pixar)
- 1980s-2000s
- Yoichiro Kawaguchi
- Marius Watz :: @mariuswatz
- John Maeda - Modern creative coding pioneer
- Karl Sims - Early genetic algorithms & research
- Casey Reas - Co-built Processing. Explores emergence
- Jared Tarbell - Early 2000's generative art
- LIA - Particles & abstract a/v
- Zach Lieberman - Toolmaker & generative graphics
- Golan Levin :: @golan - Educator & modern algorithm pioneer
- Joshua Davis :: @JoshuaDavis - Educator & modern algorithm pioneer
- Recent
- Raven Kwok - Design via visual algorithms
- Olivia Jack - Live audiovisual coding
- Bees & Bombs - Seamless animated loops
- Jen Lewin - Installation art, kinect, LEDs
- Everest Pipkin - Conceptual art, tools
- Lauren McCarthy - Conceptual art, tools
- Sean Zellmer - Raymarching (3d)
- Saskia Freeke - Generative shapes
- Matt Deslaurier - Tools, techniques, execution
- Nervous System - Product design
- Reza Ali - Generative art & low-level tools
- Kyle McDonald - Machine learning & all new tech
- Rebecca Fiebrink - Machine learning-based tools
- Robert Hodgin :: @flight404 - Generative high-end animation
- Freya Holmér - Unity tools & explainer gifs
- I Am Robot and Proud - Minimal generative audio + visual
- Memo Akten :: @memotv - Lasers, performance, AI
- Sofia Crespo - Machine learning + graphics
- Tim Rodenbroeker - Generative poster design
- More:
- Mr. Doob :: @mrdoob
- Robert Hodgin
- Mario Klingemann :: @quasimondo
- Aaron Koblin :: @aaronkoblin
- Cedric Kiefer :: @CedricKiefer
- Dave Bollinger
- Nicolas Barradeau :: @nicoptere
- W:Blut :: @wblut
- Inear :: @inear
- Steven Witten :: MathBox
- Jer Thorp :: @blprnt
- Zach Gage :: @helvetica
- Chris O'Shea :: @chrisoshea
- Nervous System
- Felix Woitzel :: @Flexi23
- OutsideOfSociety :: @oosmoxiecode
- Micah Scott
- Women in creative technology