Processing Project – Final Code, Final Product

Here is the final code and the final piece I created:


import processing.video.*;

int cellSize = 15;
int cols, rows;

Capture video;

void setup() {
size(640, 480);
frameRate(30);
cols = width / cellSize;
rows = height / cellSize;
colorMode(RGB, 255, 255, 255, 100);

video = new Capture(this, width, height);

video.start();

background(0);
}
void draw() {
if (video.available()) {
video.read();
video.loadPixels();

for (int i = 0; i < cols; i++) {

for (int j = 0; j < rows; j++) {

int x = i*cellSize;
int y = j*cellSize;
int loc = (video.width - x - 1) + y*video.width; // Reversing x to mirror the image

float r = red(video.pixels[loc]);
float g = green(video.pixels[loc]);
float b = blue(video.pixels[loc]);

color c = color(r, g, b, 75);

pushMatrix();
translate(x+cellSize/2, y+cellSize/2);

rectMode(CENTER);
fill(c);
noStroke();
rect(0, 0, cellSize+1, cellSize+1);
popMatrix();
}
}
}
}

Screen Shot 2015-01-26 at 06.35.07

Processing Project – Evaluation

What I have learnt

This assignment has taught me the basics of Processing. Furthermore, I have learnt how to use a space effectively to put across an interactive piece that I have created. This has taught me valuable skills in terms of spacial awareness, the limitations of my project and how users interactive with certain things. I also learnt the, through my research, just how much possibilities there are when using interactive design. Processing has allowed me to think more creatively and has given me an insight on how creative designers think when creating a digital environment.

What I achieved

It is my belief that I have achieved what I deem to be a good interactive Processing piece that is accompanied by a solid and interesting idea as the driving force behind it. I was also very pleased at how my Dorset poster came out as it was a chance to showcase my Photoshop skills.

What went wrong

I spent too long going through many ideas and it took a while for me to get settled with an idea. Moreover, I think I could of used a more complicated Process for my piece and make it more interactive with the users. This would’ve allowed me to have more freedom with the space and also, I think more people would of stopped and interacted with it.

What I would do next time

If I had a similar task in the future, I would set aside a lot of my spare time into trying to learn the software being used. Although I did this to some degree, I could of applied myself much more and as a result, my Processing piece could of been a lot better.

Processing Project – User Testing and User Feedback

In the last post I showed my final design for my Processing piece, the next step for the iterative design process is to put the piece I have made in the foyer of Weymouth house to see how the public react to it. The space I will be displaying my project in is essentially a hallway where people walk through to get to their lectures or do work in the media labs. This space is a good space to display an interactive piece of Processing because, the people walking through do media related subjects and will be interested in the displayed piece. Relating back to the Dorset poster assignment, the Processing piece works in the exact same way of trying to find the best place to put it and also, gauging how people react to it when they see it.

Testing in the space

space

This is the view of when you walk into the foyer. It is an open space where many different forms of media are displayed. From assessing how people act in the space from my Dorset assignment, I have learnt that the people walking through do engage with the things around them. I decided to put my interactive piece nearer the entrance so people will instantly see it as they walk in and wonder what it is.

2015-01-26 04.14.38

Above is where I had my interactive piece displayed. It was right near the entrance. I saw one person walk in and interact with the piece to see what it was about.

2015-01-26 04.15.33

He then went closer to it to see what it did.

Jan 26, 2015 04_33.2

From this we can gather two things. He saw the interactive piece from a distance and then he was attracted to it and gave it a closer look. The piece did exactly what I hoped it would do which is to display the webcam as a retro theme. Another person was leaving and it caught his attention:

Jan 26, 2015 04_33

 

From these two people’s reaction, we can gather that the display took their interest. However, one of the main drawbacks of the code I used which wasn’t an issue on my computer was that the size of the screens of the foyer were bigger than my computer screen. This caused a problem because when I tried to change the sizing of the piece, it did not work. So maybe this could be why they got closer to the screen to properly see what was going on. To find out what they really thought, I stopped and asked them. Luckily they were compliant and gave me some feedback.

Feedback

The feedback I gained from these two people, Alex and Callum, was very constructive and they had some valid points about my project.

Callum said that:

“I liked the positioning of the piece as it caught my attention as soon as I walked in. However, I was disappointed I couldn’t interact with it anymore than I already was.”

Alex said that:

“I thought it was a game to begin with, that’s why I got closer to see if I could interact with it. However, I could not interact with it in that way, which was a bit of a shame. Although saying this, when I first saw it, it sparked my memory of being younger and playing old games. Also, now I have a craving to play my old games!”

Overview of testing and feedback

The testing in the foyer was overall a good experience. Although I had already seen people view something in the foyer, this being my Dorset poster, I haven’t seen how people interact with something in the space. However, I would of preferred to use a space that was a bit smaller as I think my point would of got across a lot easier if it was in a smaller environment as opposed to the one it was in for this project. In conclusion, I am pleased with how my interactive piece was perceived by the public.

 

 

 

Processing Project – Design

Now that I have my media theory idea sorted, I can now focus on designing an interactive Processing piece. From my initial ideas, I know that I will be using a webcam for this piece. The difficult thing now is thinking what I will actually change about the webcam to fit the theory I have chosen to use. Retro gaming is something that is close to my heart and that I engage and interact with on almost a daily basis and I believe that more people should do the same. Although, most people use advanced technologies for gaming nowadays such as, smartphones, next generation consoles and pc’s.

I think these digital technologies have too much control over the gaming industry. The basis of my Processing piece is to get people thinking about the old games they used to play and maybe to play them instead of games developed for digital technologies. I will do this by making the webcam look like it’s a retro game by having the pixels in the webcam bigger. This will hopefully trigger peoples brains into thinking its a retro themed interaction, which it is.

Creating 

Now how do I go about creating what I have envisioned? From the past initial ideas I have had, creating the webcam was easy as I used the past code. The difficult part was creating it so every pixel was a rectangle. I looked through different examples on how to achieve this and I finally found out how to accomplish my vision. The code I used for my final piece is below:


import processing.video.*;

int cellSize = 15;
int cols, rows;

Capture video;

void setup() {
size(640, 480);
frameRate(30);
cols = width / cellSize;
rows = height / cellSize;
colorMode(RGB, 255, 255, 255, 100);

video = new Capture(this, width, height);

video.start();

background(0);
}
void draw() {
if (video.available()) {
video.read();
video.loadPixels();

for (int i = 0; i < cols; i++) {

for (int j = 0; j < rows; j++) {

int x = i*cellSize;
int y = j*cellSize;
int loc = (video.width - x - 1) + y*video.width; // Reversing x to mirror the image

float r = red(video.pixels[loc]);
float g = green(video.pixels[loc]);
float b = blue(video.pixels[loc]);

color c = color(r, g, b, 75);

pushMatrix();
translate(x+cellSize/2, y+cellSize/2);

rectMode(CENTER);
fill(c);
noStroke();
rect(0, 0, cellSize+1, cellSize+1);
popMatrix();
}
}
}
}

The outcome of this was:

Screen Shot 2015-01-26 at 06.35.07

 

As you can see, the outcome shows a pixelated environment. This was exactly what I envision it to turn out like as it is an homage to retro gaming and, in my opinion, if a participant was in this frame, they would think it was to do with retro gaming as well.

Processing Project – Media Concept

Since my initial ideas were ruled out and not that viable for my final idea, I decided to focus on deciding what my media concept was going to be then focusing on designing my Processing piece. The brief suggests to focus the Processing final piece on an idea or concept that is relevant to 21st century experience. A part of digital media that I find interesting is the history of it and the transition from what it once was to what it is today. The segment of digital media history I find most fascinating and one that relates to me the most is gaming.

Gaming has become one of the biggest digital technologies in the world. In an article about statistics of the gaming industry in 2014, Andy Cresswell shows that, “there are 1.2billion gamers worldwide”(Andy Cresswell, 2014). This figure is mind blowing. Thats basically 1/7 of the world plays games. Many of these games are new games which involve the use of digital technologies. What my concept focuses on is the smaller minority in the 21st century who play retro games.

By retro games I mean games that were released in the 70s and 80s, games before digital technologies. Surprisingly, there is quite a big community of people who still play these games in favour of new games. I for one am one of these people. What I want to do with this idea is to apply a retro theme to a piece of Processing to make people in the 21st Century aware of retro side of technology.

In the next few weeks, I will be developing a piece of Processing which matches this concept and conveys what I want it to convey.

 

References

Andy Cresswell, 2014. http://www.performancepsu.com/statistics-facts-gaming-industry-infographic/

 

Processing Project – First ideas

After looking at the different ways I can used sound and video in Processing, I decided to mainly focus on developing and creating Processing pieces using video rather than sound. This is because, I feel that in the space provided for us to put the interactive pieces, sound will not work very well as it is normally a busy environment where many people will be creating sounds. On the other hand, with video, there is no sound being produced only visual, so the user can easily identify what it is they’re doing and what the interactive piece actually does.

Webcam

Over the Christmas holidays, I looked at various different ways to use webcam my basis for my Processing piece. One of the main drawbacks of using webcam in this project is that i’ve never used webcam in Processing. This means I will have to research and learn the code for simple webcam functions as a basis to create from that. I chose to develop my idea based around webcam as the user can see themselves and instantly know it is interacting with them. I looked into how easy it was to input a webcam into Processing. Surprisingly, it is far simpler than what I first interpreted. I thought it would be a difficult process to input. My first idea was just playing around with the webcam to begin with. So my first Processing idea was just using a simple webcam input function. The code I used for this was:

import processing.video.*;

Capture video;

void setup() {
 size(320, 240);
 println(Capture.list());

 video = new Capture(this, 320, 240);
 video.start();
}

void captureEvent(Capture video) {

 video.read();
}

void draw() {

 image(video, 0, 0);
}

The outcome of this code is:
processingexample1

As you can see, the sketch has produced a simple webcam. Now that I know how to input a webcam, I thought I should play around with the different functions that Processing has to offer.

With Processing, there is a lot that I can change with the webcam. Face recognition is one of these functions. With face recognition, a library has to be downloaded in order for Processing to recognise the code being used to make the face recognition work. This library is openCV. I looked into using the openCV library as a basis of my Processing piece. I ventured into the possibility of using face recognition in my Processing project. With this in mind, I made a simple face recognition sketch. The code I used for this was:


import gab.opencv.*;
import processing.video.*;
import java.awt.*;

Capture video;
OpenCV opencv;

void setup() {
size(640, 480);
video = new Capture(this, 640/2, 480/2);
opencv = new OpenCV(this, 640/2, 480/2);
opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE);

video.start();
}

void draw() {
scale(2);
opencv.loadImage(video);

image(video, 0, 0 );

noFill();
stroke(0, 255, 0);
strokeWeight(3);
Rectangle[] faces = opencv.detect();
println(faces.length);

for (int i = 0; i < faces.length; i++) {
println(faces[i].x + "," + faces[i].y);
rect(faces[i].x, faces[i].y, faces[i].width, faces[i].height);
}
}

void captureEvent(Capture c) {
c.read();
}

The outcome of this code was:

Screen Shot 2015-01-25 at 19.36.39

As you can see, the code has recognised my face as an object. I thought this could be a basis for my project however, if I wanted to personalise it more and make it something relatable to a media concept which the brief suggests I do, I would have to really research this area of Processing and there is a risk that I wouldn’t get it right. It’s not necessarily i’m not confident in my ability, it’s just I would rather do something which relates to a media concept which I am interested in and also, it is difficult to change this type of Processing. Saying this, I haven’t decided on my media concept as of yet. This is because I am still playing around with the different interactive possibilities webcam brings in processing. Another Processing function using webcam I wanted to look at was Blob Detection.

Blob Detection

Although I didn’t want to use face recognition as my idea I still wanted to use webcam as the basis for my Processing project. When I went to the science museum, one of the interactive displays was using detection of whoever was in front of the screen then projecting them onto the screen. This sparked an idea. I looked into using the same detection with parts of the body. I wanted it so it would only detect peoples hands, I wanted to relate it to when people were younger and used to draw with their hands. However, I drastically underestimated how difficult this would actually be. I tested out blog detection just from the one provided in the library to gauge what it would turn out like. This was the result:

Screen Shot 2015-01-25 at 21.02.39

As you can see, the blog detection is detecting blobs (which it should) however, it isn’t detecting my hands. That is quite an ignorant thing to say as obviously it won’t just detect hands because I haven’t coded it at all and it is just doing the basic blog detection. I enquired about blob detection and whether it’s a viable to use for my project and I was advised to steer away from blob detection and my idea of hands being detected. This is because the code required for this to happen is extremely complex and I am simply not at that level in Processing terms to accomplish this. If I had more time with Processing, I would pursue this idea and make it my final idea however, I don’t have that luxury so I must look in a different area for my idea.

Kinect

One suggestion one of my peers suggested was using Kinect as it already detects people and objects in the view of the camera. This would resolve one of the issues I had with Blob Detection as I didn’t know how to make it so it detects single objects. Although I hadn’t properly looked into Kinect as a viable option for my Processing piece and I didn’t even realise that I could do it. So my next course of action was to rent out a Kinect camera from the IT services. I needed to download a library which makes Processing compatible with Kinect. This was SimpleOpenNI. However, when I tried to run the Processing library with one of the Kinect examples, an error message appeared:

Screen-Shot-2015-01-22-at-17.24.53

This in no way can be good. I looked up some of the error messages that were appearing and it meant that I didn’t have the right Kinect camera. The right Kinect camera is an old one rather than the newer one which is the one I tried to use. The IT services didn’t have the old Kinect cameras. This meant I couldn’t use Kinect as a basis for my Processing.

What’s next?

Now that I have scrutinised a few ideas and tested whether they are viable for my final idea, I know for certain I am doing webcam based interaction. It is somewhat disheartening that these ideas aren’t viable as my final idea because of the reasons stated above but it is also a positive that I have found these issues now rather than following these as the final idea and discovering these problems nearer the final deadline. I feel that the fact I haven’t chosen a specific media concept yet has hindered my progress for my ideas. My next course of action is to finalise what my media concept is then base my idea around this concept.

Processing Project – Research

With this project, the piece of Processing has to be interactive in some way with the user. This means that some of the things I have learnt in Processing aren’t that applicable to this task because the functions and coding I have learnt so far are non-interactive. However, some of the functions can be used and changed to be interactive.

Examples of Interactive Installations

There are many different forms of interactivity that can be created through Processing. I researched the different ways a user can interact with a piece of Processing and I will be evaluating the pros and cons of each and how they can be used in the space where I will be showing it.

Sound

History has shown that sound plays a pivotal part in human interaction and how humans live. We communicate by sound, engage with sound and have fun with sound. Since the pre-historic ages, sound has been used. Cavemen would use sounds to actually communicate as they had no written language and also, they created music with this. Although, as humans have progressed through time, the use of sound for communication has lessened due to there being written language. The introduction of the radio was a turning point in the history of technology. Created in 1895, the radio was first used to send radio signals, this date is the date of the first radio signal ever sent. However, the first radio created that played sound and as we know of radio was in 1917. This was the first ever music broadcast. Ever since then, radio has played an important role of a humans everyday life. Back before TV, radio was really the only form of media a person could interact with. With the rise of modern technology, sound has become much more than something you hear on the radio.

With advances in digital technologies in the modern age, interactive instalments have become increasingly more popular as a form of entertainment and also, it is some peoples jobs to make interactive instalments. One digital sound installation that uses sound as the main focal point of the installation is “The Cave of Sounds”. A description provided by the people who made the installation says that:

“The Cave of Sounds is a circular arrangement of eight bespoke digital musical instruments. Audience members are invited to experiment with the instruments together, explore the potential of creating music as a means to interact with new people and consider the intertwined history of music and technology.”(The Cave of Sounds, 2014)

The Cave of Sounds is a brilliant example of how sound can be a vocal point in a digital installation. This is because most digital installations focus on video and art. The interaction with sound in this installation doesn’t exactly relate to what I will be creating but, seeing that you can do something with sound is a positive. What I mean by ‘it doesn’t exactly relate’ is that they use objects however with what I will be creating, I will be limited to something that is only on a screen. Looking at this installation has given me some great insight to some of the positive aspects of using sound in my Processing piece, however, I will have to look into what I can actually do in Processing that uses sound as I have not done it yet in any of the workshops.

Video

Video came about around the same time that sound was making an appearance.

“The first commercially successful modern motion picture system was developed by Thomas A. Edison with his laboratory staff, notably his co-inventor William Kennedy Laurie (W. K. L.) Dickson, between 1888 and 1893.”(Kinolorber, 2012)

From this, cinema became commercially available to the public. The Cinema was vastly different to what we know of it today. Films were under a minute long and had no sound. Cinema was such a different experience to what people have experienced before that sound wasn’t an issue back then. The incorporation of sound came in the 1920s. This revolutionised cinema as it brought a whole new different dimension to video motion. Ever since then, technological advances have made these basic technologies like the first motion picture and cinema a much more immersive experience. This is due to the addition of colour in the world of video motion. After this, more development in technology came about making cinema and video motion an immersive experience. This was with the development of HD and more advanced screens to show these videos on. In a journal about Digital Cinema published in 2002, John Belton discusses the change in cinema for audiences,

“For audiences, it began in the realm of special effects-a field that is now dominated by computer-generated imagery. Then there was digital sound. Now we are seeing a very slow movement toward digital production using digital cameras and digital projection.”(John Belton, 2002)

Digital technologies have made a big impact on the video motion realm. In the modern age, people create digital installations, previously mentioned above, that are made for people to interact with and gain an experience whilst using the instalment. Also, a digital installation is normally created to put across a meaning and it normally has a purpose. I talked about sound being used in digital instalments however, video motion is the most popular kind of digital instalment. A popular interactive digital installation is Hermes 8 Ties().

This digital installation showcases the use of video motion as when participants walk across the floor with the writing on, the words are displayed in the shadow of the person on the wall next to the floor. This showcases just how versatile digital environments can be when using video. Although this type of thing won’t work in the space provided for the project I have to do.

Thinking of ideas

Now that I have done my research for this project, the next thing to do is to think of initial ideas for the project. The research has given me solid grounding in terms of thinking of different forms of interactive media and I will be comparing the pros and cons of using sound and using video for my ideas.

 

References

The Cave of Sounds. https://www.youtube.com/watch?v=xgBUu4hNz0k

http://www.kinolorber.com/edison/d1.html

Hermes 8 Ties. https://www.youtube.com/watch?v=GCIBVhOlj7Q

John Belton, 2002. Vol. 100, Obsolescence (Spring, 2002)