Nietzsche,History and Reinforcement Learning

In "On the advantages and disadvantages of history for life," Nietzsche inquires into what kind of History is most advantageous for life. In other words, as a society, what should we be recording to ensure our survival?

In this post, I will show how a lot of Machine Learning -- in particular Reinforcement Learning (RL)-- aligns well with Nietzsche's conception of "useful history" or "history for life." My hope is to convince you that it's worth inquiring into what insights from Nietzsche we can apply to computational problems in learning and cognition.

Before I go into the specifics, I'd like to make a point that Nietzsche does not really make explicit in his book. Everything Nietzsche says about history applies to memory when we move to the level of the individual. Another way of saying this is that history is a form of collective memory. So as philosophical or scientific problems, history and how it's organized within a society is analogous to memory and how it's organized within a mind. Like Riemann's geometry, a lot of what Nietzsche wrote provides insights into problems their creator could not foresee. In particular, Nietzsche's book on history seems to have a lot to teach us about how to build "machines that learn and think like people."

Whether we are talking about human or AI agents, these agents are constrained by memory capacity and the amount of information they can "integrate" or query at any time. Given these constraints, we might ask these questions from the perspective of an agent: "What kind of things should I remember? How much of them should I store? How and where should I store them, given how I'm going to use them and retrieve them (frequency, purpose, which other memories should be summoned with it, etc.)? What things should I discard and when?"

I was talking to Vishal about an Einstein quote: "Never memorize something that you can look up." A seemingly good piece of advice until we started taking it too seriously:
"- Is that person coming, the one you invited?
- I don't know; look it up!
- Where do you lookup where to look things up?"

Going back to Nietzsche; in his thin book (one of the main reasons why I picked it up in the first place), he writes that there are 3 types of history:
- Monumental
- Antiquarian
- Critical

The first one - monumental - is about remembering big achievements that are often done from within a "cloud" that ignores a lot of things about truth, history, the past.
The second one - antiquarian - is about preserving things for their own sake, for the sake of saving random bits and pieces of the past.
The third one - critical - is centered around evaluating and criticizing the past, with the hope to be better in the present/future.

An excess of monumental history blinds us to errors or unnecessary risks; a lack of it makes us take initiative less often.
An excess of antiquarian history turns history into a meaningless list of facts; a lack of it makes it too easy to invent pasts that never existed.
An excess of critical history prevents us from acting; a lack of it makes it too easy to repeat our mistakes.

A very Nietzschean problem poses itself very quickly: we often have to decide between truth and life. In other words, some true information is detrimental to our survival. Easy example: if the thought of the inevatibility of one's death was constantly on one's mind, their life would probably be harder to live. We find many examples in the Economics literature about how a lot of our biases are the product of evolution. This is another way of saying that life ends up dominating truth. I think Nietzsche is torn apart by the paradoxes around truth and life; the fault lines it creates run through so much of his work.

Going back to monumental, antiquarian, and critical history, I've mentioned that we can apply these histories to individuals. I'm sure you can think of a great achievement of yours (monumental), a memory or object you've kept for no particular reason (antiquarian), a mistake you made that you will never repeat (critical).

For instance, monumental history is memory of reward. Within a RL framework, once a child learns how to recognize different digits, the encouragement she gets when she gets it right (or the negative stimuli that comes with getting it wrong) "propagates back" and "builds-up" the representation of different digits. Remembering more intense rewards or punishments is important because they tend to be more important.

Another feature of monumental history that Nietzsche highlights is that the effect of a reward (aka achievement) can propagate between domains that can look very different -- for example, from surviving a storm to winning a marathon, or from impressing a friend to asking someone out. This is in spite (or because) of the fact that memories surrounding rewards are carried through very noisy and corrupting (turn subtitles on) processes (emotions, aggrandizement, time, "competition" from other memories, etc.). This is true because of the importance of confidence, but also because high-dimensional problems probably share common structure. This is connected to the promising domain of transfer learning.

Moving on to antiquarian history. We can think of the antiquarian as a process that decides to store (old) things at random. In other words, you're sampling at random from the distribution of the world/your senses/your thoughts and "recording" these samples somehow. This is what allows you to "feel" that someone's been in your room. This is what allows certain seemingly useless memories and thoughts to save you under pressure. Antiquarian history does not have a clear analog in RL. However, the importance of antiquarian history teaches us that we shouldn't only strive to remember what we think is going to be useful. Even though memory and processing are constrained, it's still worth it to leave some room for a more random and more exhaustive kind of sampling. Another way of saying this is that if you forget all the things that don't matter, you will find out that they matter!

Finally, critical history - our ability to notice our mistakes and learn from them - is very important. This type of history has a clear analog in RL that's called actor-critic. In this framework, the agent charged with making decisions is "split" into an actor and a critic. The actor acts a certain policy, and the critic evaluates and updates it based on how the world reacts to actions dictated by the policy. Unless you're double-majoring in Philosophy and Computer Science, this is probably not how you live your life. But you do find your approach to morality and decisions to be evolving over time after each failure and success (broadly understood).

Emma pointed me to Generative Adversatial Networks (GANs), which is an unsupervised Machine Learning technique. In GANs, there are two networks learning: a generative neural network and a discriminative neural network. The generative network learns how to "imitate" a given data distribution, and the discriminative network learns how to discriminate between real instances of the distribution and ones that are synthesized by the generative network. The model is done learning when none of the networks can make progress against the other one. For example, one network can learn to generate images of Jennifer Lawrence while the other one learns how to discriminate real images of J Law from generated images. The promising successes of GANs indicate that Nietzcsche was right: it's important to have a "critic within." More generally, splitting a single system into two or more separate entities that collaborate and compete with each other sometimes leads to more efficient solutions. I'm not a Freudian, but it seems obvious to me that there's a lot of "unconscious" processing that happens in our heads and our body -- information that circulates but that we are never "consciously" aware of. This seems like a feature, not a bug.

How we choose to remember our history is a difficult problem that touches into a lot of disciplines, from History to Communication Theory. It also raises many important practical and political questions around education, media, and urban planning (which are powerful memory transmission mechanisms). One of the most lucid thinkers we know, Friedrich Nietzsche, grappled with this problem under many different forms. Because of this, I believe that Nietzsche has a lot to teach us about how our perceptual apparatus and our thought-process function. In learning about these questions, we will gain insight into how to build more intelligent artificial agents.

Another interesting question is: how do we remember history. It seems like Nietzsche leaves out a lot of the historical transmission process out, in particular things that are associated with women and older people: education on social norms, emotional development, storytelling...

Since we're on the topic of history, I'll leave you with a Karl Marx quote I really like: "Men make their own history, but they do not make it as they please." I think this is true for both meanings of make: enact and record/transmit.

---

Reading Recommendations: please comment or send me any kind of work that this makes you think of! In exchange, I'll keep you updated about where this goes.

Google and Politicians

David Foster Wallace wrote an incredibly powerful essay on John McCain - I couldn't find a piece so here are some excerpts in the form of a summary.

The piece focuses a lot on the question of McCain's commitment to sincerity and whether it's manufactured. I think many people ask themselves that question about politicians - for example I feel that Walid Junblatt genuinely cares about the cause and ideal of March 14, but I always wonder.

In some ways, manufacturing sincerity becomes an obligation once you're past a certain level in politics. Some do it better than others.

And everyone gets stuck when manufacturing sincerity clashes with a certain political move.

We can say the same thing about big tech companies, Google is a case in point. People's attitudes toward Google are changing. It's becoming unsurprising to see an article with a title explaining "Why Google gets too much power over your private life," etc. This is the result of Google starting to be perceived as part of the establishment, of the intelligence apparatus, etc. These are political choices Google made that clashed with its previous "genuinely nice" persona.

Do people at large have a different attitude towards politicians who betray a particularly sincere-looking image and companies that do the same?

If so, why? Is the justification for that purely psychological or can a moral justification be made as well?

My guess would be that similar mechanisms are at play and it varies a lot, but maybe the ability to "bounce back" is easier for politicians because we are more used to that phenomenon with them.

We don't really know why we do what we do

Recently, I've been spending some time playing with and thinking about my baby cousin Raphaëlle, who is a little bit over a year old. One thing I noticed is that her attention jumps from one thing to another seemingly without necessarily going back to some main process that organizes all the other subprocesses.

For example, you might go pick a flower with her to distract her from her - at times inexplicable - crying. She'll pick the flower, smell it, and focus all her attention on it before deciding that the texture of the floor suddenly got really interesting, etc. She doesn't really know why she picked a tennis racket up or decided to put her hand in cold water.

In contrast, we seem to think that underlying our choices to give attention and importance to certain things is some kind of rational system that weighs all options and then somehow leads to a decision about what course of action makes most sense. I run in the mornings sometimes because running is good for me, and it's relaxing, etc etc. I chose the burger because I haven't had a burger in a while and the pizza here is not that good.

I think that this framework - in the way we think about it - is a complete myth. Rather than some kind of optimizing calculus, what drives our behavior seems to be mainly habit - under one guise or another - and some sort of propensity to rely on certain particular elements in particular situations. Very often, we do the things we do then reconstruct a rationale for it according to this mythical system.

In some sense, then, Raphaëlle's current mode of being is more honest, more authentic than ours insofar as it doesn't posit a nonexistent underlying logic and doesn't really worry about its own groundlessness.

Two lines of thought that seem worth pursuing from here are:

1. What's the source of our being and thinking of our lives in this way? I think it's definitely reinforced by some ideas coming out of the enlightenment (like the conviction that we can subject everything, including ourselves, to reason), but it seems like it's also natural for us to think that we have the freedom to orient ourselves and that the way we do orient ourselves comes from our choices.  I can't know because I can't really get out of my time or myself, and the distinction between nature and culture is artificial in many respects, but even after blurring that distinctions we're left with a lot of unanswered questions.

2. It's important to not fall into another extreme where we think every decision we make is completely reliant on factors that have nothing to do with our conscious selves, and where even important - say - moral choices lose their whole meaning because choosing becomes as significant as picking a number out of a hat. That is to say that this rational system is a myth but it is not entirely unfounded. How do you theorize our mode of being from the particular angle I'm tackling without falling into (A) the mythology of omnipotent, omniscient reason on one side or (B) a smug deconstructionism where all that matters is to show that we have no idea what we're doing so we can show that we're smarter than those who think they know what they're doing?

Taking the magic out of self-deception

Note: When I say “the mind” I mean the whole thing and when I say “consciousness” I mean our gaze inside our minds, sorry I don’t have enough space to elaborate on the distinction. And when I say model of the mind I mean a model of how our mind works with respect to what's visible and what's not and who/what is calling the shots, not anything deeper/broader.

The solution to many problems is hidden within the question being asked. Self-deception occurs when one (appears to?) lie(s) to oneself. We can formulate the question at hand as follows: how can our mind deceive itself? How can the one who is being lied to also know that she is being lied to, precisely because she is the one doing the lying?

(for a more complex formulation of the paradox(es) of self-deception I wrote a paper on this a couple of years ago which is why I'm thinking about it)

I think Freud was right to think that the mind that is “lying” and the mind that is being lied to are not exactly the same thing. He built a model based on the separation of selves. Think about it like a small opaque circle (the unconscious mind) inside a larger transparent circle (the transparent area being what's accessible to our conscious mind).

(this is veeeery simplified but it's all we need; for more, Wikipedia)

For Freud, self-deception occurs when the unconscious mind deceives the conscious mind. Unfortunately, Freud's underlying model of the mind has a lot of problems, especially the fact that the unconscious mind seems to have unexplained, scientifically ungrounded "magical" properties.* This is why his model of self-deception is somewhat convincing yet flawed.

Sartre has a different model: for him, the mind is translucent. This means there is only one consciousness, but its "field of vision" is limited. Think about a map (the whole field) and a smaller magnifying glass (the narrow field of vision of the consciousness) that can move around to show whatever area it wants to on the map.

(Sartre's model is explained and analyzed in detail in the paper I already linked above... I swear I'm not selling anything)

Sartre's underlying model of the mind is plausible, but from within this model he needs to complicate things a lot to make it  work (“It” being the explanation of self-deception within his model of the mind). It seems that the problem doesn’t require as much complexity as he uses. The main issue with Sartre’s approach is that the mind must also somehow hide from itself the motives and decisions related to its own deception while theoretically being able to “see everything.”

I think we should use a model of the mind that takes into account what we know about ourselves and how and why our minds function. It doesn't need to be fully explained, just more plausible and less mystical or optimistic than Freud's or Sartre's models. Obviously, it’s not very fair to attack Freud and Sartre for things they couldn’t have known, but I’m using them because I think their theories constitute great points of departure for us given how hard they worked hard on the problem with what they had.

Here are some remarks on how we can build a hypothetical model of the mind that might help us understand self-deception better:

1. There is no clear boundary between our mind and our body. Descartes and Christianity (I can't speak about other religions because I don't know them very well) have gotten us used to thinking about our mind as a completely separate entity that understands its own articulable motives and drives. However, a lot of the things that drive us to make decisions probably don't even make any sense at the level of language, they are better understood as chemical processes. In that sense, when we refer to unconscious processes these processes are not conscious for someone or something other than ourselves: they are just unconscious.

2. It follows that, in order to understand ourselves, a lot of what we do is look at our feelings and reverse engineer their underlying desires, thoughts, etc. We bring these semi-hypothetical underlying thoughts into the realm of language without ever confirming them. In that sense, the boundary between knowledge and interpretation is shifting too.
3. There are evolutionary reasons why we would have a tendency to want to believe things that make us happy - think about it from the "perspective" of a mental process that focuses on some basic heuristics like avoiding pain. We could have one interpretation of things (e.g., “I only stole the $20 bill because he was stupid for leaving it”) that ends up being favored and less questioned because one of the basic processes controlling where consciousness looks seeks to avoid pain.
4. Following from 3, self-deception would be less of a conscious decision and more of a mind-steering-us-towards-comfort thing. Kind of the same process that makes us forget embarrassing moments - an even better analogy is, in Harry Potter, the spell that makes Muggles remember appointments or things they have forgotten so they leave when they get too close to the Quidditch stadium.

With these remarks in mind - and much more research - we can start to build a theory of self-deception (I would be in favor of calling it self-deceptions because I suspect that we group multiple different phenomena under that name) that is somewhere in between Sartre and Freud and much more in line with both current scientific knowledge, of which I apologize to the reader for not having more, and common sense.




*: Although I think Freud’s theory of the unconscious mind is deeply flawed, I think psychoanalysis can bring a lot to the table. Maybe I’ll write a post about it soon.

New Blog!

My new blogpost ideas are very clearly divided into two categories:

• Things about Lebanese culture, politics, and society
• Things that are more philosophical in nature

So I made a new blog for the Lebanese stuff (acountryofstrangers.blogspot.com), and I'll keep more philosophical/random posts here.

Rachid Khalidi on Israeli Apartheid

On Monday, March 18, Rachid Khalidi, who is the Edward Said Professor of Modern Arab Studies at Columbia University, appeared on the Brian Lehrer show to talk about his last book, Brokers of Deceit. The book's thesis is that although the U.S. posed as an honest broker between the Israelis and the Palestinians, the U.S. has in effect prevented the emergence of a viable Palestinian state and used its power as a cover for the continued oppression of Palestinians.

There's a part of the interview (approximately from 5:08 to 6:45) where Khalidi explains why he uses the term "apartheid" to describe the situation in Palestine/Israel. I found the bit incredibly lucid and well-argued, so I thought it might be useful to put it in writing because few people will take the time to listen to a whole interview and might miss it.

Q: Why do you use that word [apartheid]? Is it to be provocative? Is it because you really think it's like South Africa? [...]

A: Authorities like Bishop Tutu, who should know a thing or two about apartheid, authorities like two Israeli prime ministers (Ehud Olmert and Barak) have [all] used the term [...] as has almost anyone who is acquainted with South Africa in any detail.

When you have:

• Two legal systems for people
• Movement restrictions on one people
• Segregation -- Jim Crow-like restrictions on one people

You can call it whatever you want -- if people are offended by "apartheid" they can call it Jim Crow segregation, they can call it whatever they please.

But you have to really avoid looking at reality to avoid seeing that you have a system which is grossly unequal and that's been erected in large part over the last 20 to 30 years when there was supposedly a peace process ongoing. [...]

We [the U.S.] bankroll this, we run diplomatic interference for this system all the while claiming that we are honest brokers in a peace process, which in fact I don't think we have been.

Portraying Insurgency: Justice's Stress + Jay Z and Kanye's No Church in the Wild

Remark: Sorry this is a really long one.

In this post, I argue that Stress by Justice focuses on the real difficulty for a general audience (centered at the middle class or a little bit below) to relate to anthropologically distant insurgents. In contrasts, Kanye and Jay Z's No Church in the Wild makes a much bigger effort to bridge that distance, which makes it less ethically and politically challenging.

Justice, the French electronic music legends have a very powerful song called Stress with a controversial music video by Romain Gavras. The music video portrays the progress a group of French that seem overwhelmingly black and Arab youth from the poor Paris suburbs through the city. The camera follows them while they vandalize, harass, and break everything that stands on their aimless path. Watching it is like a punch in the stomach, and you come out not really knowing what to think.

In their infamous Watch the Throne, Jay Z and Kanye have, in addition to the crazy Mercy*** and this one, a beautiful song called No Church in the Wild****. The religious undertones of this song and the way it describes the struggle of following religious values amidst the absurdity and cruelty of the real world are fascinating, but I will not go into them in this post. This song portrays an epic battle between what seem to be protesters and a faceless, nationless police armed with shields, batons and teargas guns. Somehow by the end of the music video, the fight is accompanied by green laser lights and an elephant (??).

No Church in the Wild clearly makes empathy with the protesters very easy. Their rage is relatable, especially given the slowmotion close-ups on their enraged or pained faces. In front of them, there's nothing to relate to because you can't even see people behind these masks. The protesters are riddled with symbols, a particularly powerful one is the protester that sets himself on fire, a clear reference to Mohammed Bouazizi's powerful gesture from December 2010.

However, in Stress, the unabashed violence is baffling and difficult to withstand. There is no explanation for it, and its victims seem to not deserve it at all. At the end of the music video, the cameraman is beaten up and yelled at "Does filming this get you off, you S.O.B?" The quote is from this good Time article about the music video. Although Justice's record label claims that the videoclip is intended as a parody of media portrayal of poor Parisian suburb youth, I think it's much more about the way this youth's violence is a challenge to the ethics and politics of a middle class audience as well as to leftist artists and intellectuals.* The violence is not a direct translation of an ideal of greater justice, it's a much more visceral reaction to oppression and the absence of hope.

I'll paste my thesis from above here, just because it sums up the point well and this is already long. Stress by Justice focuses on the real difficulty for a general audience (centered at the middle class or a little bit below) to relate to anthropologically distant insurgents. In contrasts, Kanye and Jay Z's No Church in the Wild makes a much bigger effort to bridge that distance, which makes it less ethically and politically challenging. What I mean by less ethically and politically challenging is that Kanye and Jay Z's music video embellishes the reality of insurgency in a way that makes it more relatable to the audience, but also reduces the extent to which the issues of institutionalized racism and economic inequality are more urgent and difficult than it seems.

I think both works are needed politically, but work of the kind Gavras** does is more original; it takes much more courage and begets much less recognition.

*: This reminds me of something Lucas Iberico Lozada (his face doesn't actually look like this) told me, not in those exact words: if something like a revolution happens it will look much more like the England Riots than anything else.

**: An equally perplexing and interesting work by Romain Gavras is the movie Our Day will Come (original title Notre Jour Viendra).

***: My good friend Tarek Soubra pointed out that Mercy is not in Watch the Throne, sorry about that.

****: Found out much later that Gavras also directed No Church in the Wild. Not really sure what it means for this post.