Decoding long term memories and emotions

Last week, our reading assignment for To Remember and To Forget included Cracking the Brain’s Codes and The Importance of Feelings. The lecture was a review of the anatomy of the brain and an exploration of emotions and memories through neuroscience, through film (we watched a clip of Her), and through smell.

Cracking the Brain’s Codes reveals some of the newer developments in understanding how we think, remember, and feel by decoding the brain. The brain’s coding works through neural spikes. Different spike patterns based on functions, senses, and perceptions create different meanings. What matters is the rates of spikes; they represent variations in brightness, speed, distance, orientation, color, pitch, etc. This is especially interesting to me because I was just starting to learn about and work with serial communication in Physical Computing. The analogy here is very obvious: different computers are like different neurons, to communicate with one another two computers must establish a common pulse rate, common voltage level, and common voltage meaning. Taeyoon had mentioned in class in the past that it’s very tempting to find similarities between the computer and the brain, but there are many differences that sets the two apart. However, with the little that I know of either subject, it’s very hard for me to not compare the two. Even the article stated similarities of the Brain’s code to computer code, e.g. ASCII, pixels, and vector are made up of different patterns of code like patterns of neurons. I wonder if there is something innately structured in our brain that makes this comparison inevitable. And after all, the computer is created by the human brain; shouldn’t the creation share some similarities with its creator? Personally, it’s very attractive to think that by learning how computers communicate and how to code it to do different things, I’m also learning how the brain works. Perhaps because this belief ameliorates some of my existential dread, I find it very motivating.

A common topic in both readings focus on the importance and difficulty of understanding how networks of neuron spikes work. This is important because any complex thoughts or emotions that we may have is dictated by the structuring and wiring of these networks, instead of one off spikes between neurons. For instance long term knowledge or skills, such as driving a car, is learned through rewiring network of small groups of neuron that encode things it encounter over and over again. The construction of sentences is another example. Each group of neuron means simpler concepts, such as specific words. If we can understand these networks and work backwards, think of what science fiction-like utopia/dystopia we would create. We would be able to learn new things without having gone through the repetition of experiences. Computers would be able to create and make complex decisions. We would be able to alter the way we perceive our existence.

The difficulty in understanding these networks lies in the complexity of the brain, the lack of technology for better observations, and what Cracking the Brain’s Codes author calls the “noise” in the signal. The latter triggered yet another comparison. Nate Silver also wrote about the noise and the signal, but around the topic of big data. This seems to be an ongoing theme in our contemporary world: more information but stagnant growth of understanding in how the information are connected to find meaning and use. Nate Silver’s solution is to combine computer processing power with human intuition and experience, for the nuance and complexities in the given field, plus a bit of good luck. I think this is probably encouraging for neuroscientists, because we are studying the human brain after all and we, more than anyone else, possess the intuition and experience of our own complexed and nuanced emotions. Perhaps working with people who are more connected with with their personal emotions and feelings, such as artists and poets, would aid in the quest for decoding the brain.

Runaway Emojicon: Taking small steps

This week I restructured some of the code in draw() to make the program more readable and added initializeGame() to allow the game to start over if the player loses.

With the restructuring, my attempt was to make draw() the table of content to the rest of the program and only include logic that starts, restarts, and stops the game. However, I think the overall structure is still very messy. For instance, do I need to have a function to draw the background and a function to move it? Also, should initializeGame() and play() be in the same function? I think being able to figure out some of these structural issues and feeling more comfortable and confident building these structures would bring my programming skills to the next level.

initializeGame() is not working very well right now, because play() is still running. So the obstacle objects end up repeating when the player loses and click to restart the game. Again, I think this is a structure issue.

For the next step, I would like to do a better job restructuring the whole program and add new levels.

Lab 4: Two-way serial communication

When reading multiple sensors a question that comes up is “which value corresponds with which sensor?” There are two ways to set sensor values in order: punctuation method, which involves punctuating each set of data uniquely, and call-and-response (handshake) method, which works very much like it sounds, sending one set of values at a time by setting the programs to call and respond to each other.

We tried each method in this lab, neither worked very well for me. And I have a lot of questions regarding the additional code.

This is what I’ve learned so far:

Punctuation method

To send data from the Arduino, we use Serial.print() and Serial.println() to send out each set of values, separated by commas Serial.print(“,”);.

To receive the values in Processing, we add myPort.bufferUntil(‘\n’); in setup() so that the string of values would be saved to the buffer and read together. In the serialEvent() function, the string is read with myPort.readStringUntil(‘\n’);. Then trim out any whitespace character (newline, carriage returns, spaces, tabs) with trim(stringx), split the values at each comma with split(stringx, ‘,’);, convert the values into integers and store them in an array, for instance int sensor[] = int(split(stringx, ‘,’));.

There are a few downsides to this method, namely the program may slow down if the buffer is full and that we can’t easily send binary values.

Call-and-response method

In the Arduino IDE, we setup a new function establishContact(). This function sends out a message until receives data from Processing. We check for data by checking whether Serial.available() is greater than 0 (true) or less then/equal to 0 (false). We call for this new function in setup(). Then, with an if statement, make sure that the strings of values in loop() would only send when Serial.available() is greater than 0.

In Processing, we check for contact with a new variable boolean firstContact = false;. In serialEvent() we let the program first check if the message from the Arduino is the right one with myString.equals(), then send back a character with myPort.write() to start receiving actual data. If firstContact is true, the code from the rest of serialEvent() would run, trimming, splitting, converting, and storing the values.

Issues: 

  • With the lab (make a simple DIY mouse), my digital input worked well, allowing the ball to appear and disappear from the screen. But with the ball fluttered about when I moved the accelerometer. Instead of moving on the x and y axis smoothly, the ball was all over the place.
  • Often the program didn’t even run. It displayed an error message.
Error message in Processing.

Error message in Processing.

Questions:

  • Why is the call-and-response method better for sending binary data, even though it’s also sending and receiving strings with commas, then trimming, splitting, and storing them in a buffer? Also, wouldn’t this also slow down the program if the buffer gets full?
  • Why do we need to call-and-respond between the programs?
  • Explain more about the new functions in Arduino IDE and Processing. e.g. Serial.available(), trim(), split(), myPort.bufferUntil(), myPort.readStringUntil(), myString.equals(), etc.
Values show up properly in the Arduino IDE serial monitor.

Values show up properly in the Arduino IDE serial monitor.

Digital input works. Issues with analog input values.

Digital input works. Issues with analog input values.