Hello! These are the materials for the computational photography workshop taught by Adam Ferriss (aferriss@gmail.com) at the Anderson Ranch summer of 2020.
- Schedule
- What is Computational Photograph
- Workshop Format
- What you'll get out of it
- Code of Conduct
- Setting up your environment
- In class demos
- Code Examples
- Resources
*The schedule may change depending on the pace and needs of the class!
July 27 - 31
The below times are in MST
9am - 11am Class time and demos
11am - 1pm Lunch
1pm - 4pm Class time and individual meetings
- Introductions
- Workshop Overview
- Setting up your environment
- p5 image function demos
loadImage()image()get()set()copy()
- exporting your work
- Pixel Part 1
- looping over pixel array
- accessing individual pixels
- pixels as data
- Pixels Part 2
- Kernel Convolution demo
- Shaders Part 1
- Shader Syntax
- Texture2D
- Shader functions
smoothstep(),step(),clamp(),mix()
- Shaders Part 2
- UV fun (wiggle, distort, displacement)
- Shader convolution
- Multipass rendering
- Feedback
- Going further
- Work Share / Critique
Computational Photography is actually a pretty broad term but for the purposes of this class, I am taking it to mean any method by which a photograph or digital image can be manipulated via code.
In the broader definition, it refers to imaging techniques that expand the capability of digital cameras / sensors through the use of software. A lot of this tech was made for our phones. Think of portrait mode on iPhones or Google's night sight on Androids. Newer techniques using machine learning have also been deployed for things like upscaling small images, removing noise and artifacts, or extracting 3d depth information from single photographs. We'll look at some of these new advances in the course of this workshop.
We'll be meeting on zoom every morning from 9am - 11am for live coding demos of new techniques. We'll have a short break for lunch and then return in the afternoon at 1pm. Depending on the day, we will likely do a combination of individual meetings and demos in the afternoon, ending at 4pm.
On Friday we'll all get together to share our work and critique.
My hope is that you take away some of the core techniques for learning how to manipulate images with code. Hopefully these are processes that will help further your own artistic practice and expand your tool set for artistic expression / ideation. What you produce here is up to you. If you want to make a suite of new tools for your own practice, great! If you want to just sit back and soak it in and experiment with the examples I provide, that's totally fine too!
I'll be leaving this repo up as a resource for you and the general public for as long as I can so hopefully it is something you can return to for guidance or help for as long as you need it.
I'm here to help you as much as I can so please don't hesitate to reach out if you need help or even just wanna chat (e-mail is great but we can talk about setting up a slack or discord if people are interested). Also reach out to your fellow workshop-mates. We're only together for a short time but we can all learn from each other during this week.
Lastly, I encourage you to consider the ways in which computational photography (and code based image making in general) might cause harm in the world (ex. face recognition used for surveillance by an authoritarian govt). What can we use these technologies for that will move the needle in the direction of racial, social, and ecological justice?
This code of conduct was borrowed from the p5.js code of conduct but the same tenets apply in this workshop.
-
Be mindful of your language. Any of the following behavior is unacceptable:
- Offensive comments related to gender identity and expression, sexual orientation, race, ethnicity, language, neuro-type, size, ability, class, religion, culture, subculture, political opinion, age, skill level, occupation, or background
- Threats of violence
- Deliberate intimidation
- Sexually explicit or violent material that is not contextualized and preceded by a considerate warning
- Unwelcome sexual attention
- Stalking or following
- Or any other kinds of harassment
Use your best judgement. If it will possibly make others uncomfortable, do not post it.
-
Be respectful. Disagreement is not an opportunity to attack someone elses thoughts or opinions. Although views may differ, remember to approach every situation with patience and care.
-
Be considerate. Think about how your contribution will affect others in the community.
-
Be open minded. Embrace new people and new ideas. Our community is continually evolving and we welcome positive change.
If you're just learning to code, or just want to get started quickly, I strongly recommend that you use the p5.js code editor which can be found at https://editor.p5js.com. Be sure to make an account, otherwise you won't be able to save your sketches or upload images. You can always download your code and move it to another environment later if you don't like the online editor.
However, if you'd like to branch out and explore other options, I recommend you download VSCode. Along with VSCode you may want to download some extensions. I've listed a few of the ones I use below in the resource section, but of particular note is the VSCode live share extension. This may be useful so that we can code together during one on one sessions. The live server extension is also really handy for spinning up a localhost server (which you will need to use images) from within VSCode.
If you opt to use VSCode, I've also provided a template p5.js project for you to start with. It also includes the typescript file so that code auto-completion will work out of the box. Just be sure to duplicate the template whenever you want to start a new sketch.
- Shadertoy to p5 conversion
- Posterized
- Pixelated
- Texture Rotation
- Single Pass Blur
- Bloom
- Bloom Buffers
- Motion Detector
- Load and Save Video
- Load Video
- Load Sound
- Video + Audio with a shader
- Slit Scan Video
- RGB Delay
- Feedback
- Vertex Shader
- Grid of Old Frames
I have created this collection of code example sketches for your reference. Most of them should be pretty well commented but if you've got questions about how anything is working, let me know!
Everything below is arranged from least complex to most complex
Drawing images at different sizes
Image chopper - Multiple Layers
Slicer-upper vertical and horizontal
Looping through pixels - multiple ways
Convolution - Emboss Feedback Brush
- Adam's Pixel Collection on the p5 editor
- p5js Pixel and Image functions:
- p5js Color Functions
- Image processing in p5js is a nice tutorial on getting started with pixel manipulation in p5.
- The Pixel Array - Coding Train
- Brightness Mirror - Coding Train
- p5js Shader functions:
- The Book of Shaders by Patricia Gonzalez Vivo and Jen Lowe is an introduction to writing shaders with interactive editors in the browser.
- p5js shaders is a write up of how to get started with shaders in p5js.
- Adam's shader examples is a repo I've created with a whole bunch of examples of how you might accomplish certain effects using shaders
- Shadertoy is an online fragment shader editor. There's a ton of knowledge here if you're willing to be patient and comb through other examples. People have done some truly mind blowing things on this site.
- Shader functions reference is a reference for GLSL shader functions
- p5js editor
- p5js.org reference is where you can find information about all of the functionality of p5.js.
- p5js forum is a good place to ask questions or talk to other people working with p5
- The Coding Train is an amazing series of videos hosted by Daniel Shiffman (and others) that teaches coding and extensively covers p5js / processing.
- MDN Javascript Docs. Probably my favorite site for getting information about features of javascript. Typically very well written with lots of examples.
- ES6 Features. ES6 is a modern flavor of javascript that adds some really handy new features to the language. This site is a little wrap up of all of them for quick reference.
- Nested Loops - Coding Train
- Arrays - Coding Train
- Arrays and Loops - Coding Train
- VS Code
- Workflow: Visual Studio Code
- Prettier is a great code style guide / auto formatter. It also helps bring everyone's code into the same style which makes it easier to debug when reading code other people have written.
- Shader Language Support is super helpful for syntax highlighting when working with GLSL or other shader languages.
- Live Server adds a localserver to VSCode so you can preview your sketches without having to do any work to set up a localhost. Just click the
Go Livebutton on the blue bar on the bottom right to start which should open up a web page at localhost:5500 - p5 Types is a typescript definitions file for p5.js. It is already included in the template I've prepared for you but if you plan to continue working in VSCode I highly recommend picking this up. This will give you autocomplete support when working with p5 in VSCode. You can just copy the
global.d.tsfile into your project folder and then on the first line of your sketch add:/// <reference path="./global.d.ts" /> - Bracket Pair Colorizer is a little addon that makes it a little easier to tell which brackets belong to which by coloring the opening { and closing } the same color
- Generative Design is an excellent book with tons of code examples for getting your feet wet with generative design. The book now comes with p5.js examples too!
- Getting Started with p5.js by Lauren McCarthy and Casey Reas is a great and quick primer for jumping into p5.js. I've often used this book for teaching and think it's one of the most friendly and approachable books for learning to code.
- Learning Processing by Daniel Shiffman is the book that I first read when learning to code. Super clear examples and covers just a tad of everything you need to get started with processing / p5.
- The Nature of Code is Daniel Shiffman's follow up to Learning Processing. It looks at patterns in nature as inspiration for processing sketches and goes into some deeper topics like AI and emergence behavior.