math

Years ago when I was an analyst I sat in a presentation where a data scientist gave a talk about an algorithm he developed and implemented within the slots franchise. The slots franchise was a portfolio of virtual casino like slots games that can be downloaded and played on your phone. Even now, years later, 10 out of the top 50 highest grossing games in the world are all slots games. Slots games in and of themselves make hundreds of millions of dollars in revenue on a quarterly basis. The gameplay isn’t very complicated, a user spends virtual in game currency to pull the lever of a slots machine. If the correct symbols line up, you get more in-game virtual currency. If they don’t, well, you loose the in-game currency you just spent. As you play and rack up wins you get bumped up to “high roller” tables with progressively costlier entry fees. The animations for progressively costlier tables get more intricate, making the wins feel more grand. While the gameplay itself is straightforward, the progression system built around the gameplay is robust with layered penalties for not playing on a daily basis. Pretty soon the daily currency you get isn’t quite enough to play at these high roller tables, so you either have to save up your currency for a chance to play or you can spend just a few real currency to get more virtual currency. Much like real world casinos, the mobile franchises employ psychologists & economists to ensure that the rewards and win rate are perfectly tuned to keep a player engaged. Unlike real world casinos, its probably worth mentioning that there is no way to convert virtual in-game currency into real world money. If you take the time to accrue millions upon millions of virtual currency you do so for the love of the game. Multiple millions of people log onto these slots game every day but very few people actually make the leap to paying real world money for virtual currency. Less than two percent of people pay on a somewhat regular cadence. The top 5-10% of the two percent (~.1% of the entire playerbase) make up the bulk of the entire games revenue. They spend tens of thousands of dollars per month buying virtual currency in slots games and experiencing those sweet, sweet winning animations.

The presentation focused on the .1%, otherwise known as the VIPs. The data scientist had built a fancy machine learning algorithm to predict when VIPs. players would “churn” or leave the game. Losing a single VIPs player meant loosing thousands of dollars of revenue on a monthly basis for the slots franchise. He went on to explain how a series of tests he ran with his algorithm utilizing the VIP management team. The VIP. management team consisted of between 5 – 10 customer service workers and was traditionally used to check up every once in a while with these VIPs. Essentially give them a call, see how they were doing, congratulate them on a recent win. Since a small group of people had to manage thousands of VIPs, not every VIP would get a call every day. Through testing, the algorithm was able to identify players with the highest risk of churning and put them at the top of the VIP call list. The algorithm worked and subsequent reachout worked. It prevented a meaningful amount of people from churning who otherwise would, and thus led to a few hundred thousand dollars of additional revenue over just a six month period. The presentation ended to applause, with many of the people in the room walking towards the data scientist to personally congratulate him on this success.

I remember sitting in the back of the room quietly in thought. I wish I could say that I saw through the complex model, the revenue results and saw the truth behind what was actually happening to the players. The whole goal of the game is to acquire as many users as possible in the hopes of finding users with the propensity to become gambling addicts. Mobile casino games offer the ability to take the key dopamine release provided by in real life gambling and make it accessible anytime, anywhere. The slots franchise spends a fortune on acquiring new players with the goal of netting VIPs. Although some VIPs are wealthy, the vast majority of them are not. The most common profile of a VIP was a female homeowner, between the ages of 55 and 60, with at least some college education and an annual household income of more than $55,000. In addition to paying large sums of money into the game, becoming a VIP involves spending hours upon hours of your day playing different slots machines. As they become and sustain their VIP status they get increasingly frequent calls from the VIP management team. Many of the VIPs would form deep personnel connections and relationships to the VIP management team. I knew a few people on the management team, and they would often talk about how many often the VIPs would comment that the management team were the only real conversations they would have on a consistent basis.

There were many reasons why people stopped paying into a specific slots franchise. One of the main reasons was they simply didn’t have the money anymore. The truth of what was happening to the players was that this algorithm would often times find them at their weakest moment. Their personal slots budget had run out, and they were no longer able to play the one thing that brought them some kind of dopamine release on a daily basis. At this moment, a person who they had developed a deep personal relationship would reach out to them, check in on them, and quietly mention that they were at risk of losing their VIP status. Loosing your VIP status meant no more in game bonuses and, more devastatingly, no more regularly scheduled calls from the management team. This would prompt a meaningful number of them to open their wallets, refinance their home, or borrow money in order to keep the VIP status going. I truly wish as I was sitting there I could have seen that this data scientist had developed a system in which to exploit the addiction and loneliness of people who were my mom’s age all to extract a few hundred dollars. But I didn’t. Instead as I was sitting there my head was buzzing with ideas of how we could probably just cut out the management team entirely by smartly connecting the VIPs to each other. Social pressure, after all, holds way more weight than the tepid words of an overworked 24 year old VIP manager in San Francisco. With the money the franchise saved by cutting the management team they could probably host a couple happy hours.

In Weapons of Math Destruction Cathy O’Neil does a great job illustrating how black box algorithms have enabled the large scale spread of unequal treatment across various columns in society. She demonstrates how in key industries and social functions algorithms do not offer a fabled bias free decision making system based on all powerful “math”. Instead, the use of complex algorithms serves to codify pre-existing discriminatory beliefs and practices in different, more opaque ways. O’Neil details the use of mathematical models to scale injustice across various realms including criminal justice, predatory loans, college admissions, and low wage employment. The key power of the all powerful algorithm is its mathematical density & opaqueness. It allows those who wield it to use it as a shield in order to mask clearly discriminatory and harmful outcomes.

Listen man, I’m a numbers guy. I put this model into production, rev goes up, I get a bonus. You want to know how it impacts people go talk to the “cares about people” guy.

That to me is the key takeaway from this book. With the continued advancement of data collection and compute power, data scientists are able to scale the rent seeking initiatives demanded by those in power to a level they could previously only dream of. Data scientists are starting to take over what was previously in the realm of ghoul ass management consultants. The ones who would storm into a business after a private equity hostile takeover and start slashing the business left and right until they can easily suck what little assets have been left. The ones who get called in by city governments to reinvigorate the city and offer innovative solutions such as buff up the police force, kick the undesirables out, and gentrify baby. This brings me to my biggest issue with the book as a whole: the solutions proposed to combat WMDs.

Although there are brief vignettes of solutions (or rather attempts at solutions) layered throughout the book the conclusion & afterword sections of the book lay out O’Neil’s key propositions.

1. Make data scientists take some kind of moral hippocratic oath
2. Make sure our representatives pass legislation which limits the damage that unfettered algorithms can cause

In my view, these solutions are pretty naive and shows a misunderstanding of why these algorithms are pursued in the first place. For the first suggestion, I can picture the conversation between the DS and his boss in the earlier anecdote.

DS: Hey boss I’ve discovered this algorithm basically preys on older, lonelier people and pushes them to spend beyond their means to keep playing. I don’t think we should be using this to convince them to come back into the game.

Boss: So you’re telling me you want to throw away hundreds of thousands of dollars in revenue?

DS: Well actually, I think we should instead be using this algorithm to do the right thing, to identify people who are clearly addicted and have to stop. We can reach out to them and make sure they get gambling addiction rehab.

Boss: You want to throw away hundreds of thousands of dollars in revenue and instead insure that our most valuable cash cows never pick up the game again?

DS: Well ya, it’s the right thing to do.

Boss: Dope I hear you. Hand your model off to our stats intern I’m transferring you to another team.

DS: Are you kidding me? But why?

Boss: Because I’m going to use the quarterly bonus from the increased rev to buy a beach house in Santa Cruz you idiot.

Of course, this conversation would never happen. The DS knows he would get a bonus himself, and he needs to make sure that he can cover his kid’s tuition at Athenian. Maybe I’m biased, but tech workers looking past their material circumstances to keep the greater good in mind is more unlikely than hitting a home run on any of the Big Three. Besides, their entire job is predicated on finding the most scalable way to get to the objective function (be it revenue, # of people off the streets, etc…) in the quickest, easiest way possible. Unless the driver behind objective function is eliminated, those tasked with developing the WMDs will never be able or willing to change their destructive effects. For the gaming example above, it is the existence of easily accessible free to play games that is the root of the problem, not the individual WMDs that are incorporated inside them. Transparency in these models showcasing the destructive effects will have no sway over those in power whose very paychecks depend on ignoring the destructive effects. If you truly want to address this particular issue, we as a society need to outlaw the concept of hyper exploitative free to play games. This holds true for every exploitative industry covered in this book, from the predatory loan department to the for profit college system.

For the second point, getting lost in technocratic debates about a particular algorithm’s “fairness” moves the source of power to change things away from the people being affected by them. In O’Neil’s words:

Challenging WMDs will require a movement of people who refuse to bow down to the algorithmic gods, who band together, collect evidence of their harm, and demand better laws from policy-makers. 

Nowhere in that solution is there room for the those being directly effected by the WMDs themselves to assert their point of view and challenge them directly. In fact, the only real substantial progress with regards to criminal justice has come in the past few months as everyday people have risen up and made their voices heard with the protests over George Floyd’s death. The Minnesota police department was disbanded and forced to abandon its use of Compstat to harass citizens not because of some pencil pushing data scientist showing some graphs, but because the citizens made their voices heard and burned down a damn police station. Likewise, unionization movements starting all across Amazon warehouses are the only fundamental organization standing in the way of increased workplace tracking and surveillance. Demonstrations and the organizing of worker power are the only real tools we as people have against the ongoing creep of WMDs into our everyday lives. My key point here is that change doesn’t come around waiting for some Ivy graduate DS who has seen the light to have a heart to heart with a few Congressmen. The power to combat the creep of WMDs comes from organizing in your own workplace and not being quiet in the streets when you see injustice happening. These solutions are hard, and they force us out of our comfort zone, but these are the only solutions that tackle the root cause of the proliferation of WMDs.

jung

There was maybe a time when I was a kid that I could call myself religious or “spiritual”, but those times are long gone. I haven’t really taken religion or the concept of spirituality seriously since I was a teenager. In essence I stopped thinking about being apart of something that was larger than who I was as an individual. In the place of that spirituality entered the need to set individual life goals and pursue them with all my energy. Recently, a break in my life presented me with an opportunity to take a hard look at those goals and ask the question of whether I wanted to keep going down that path.Over the past 6 years my goals basically centered around making that cash. I didn’t really care who I had to push past, I didn’t care about the larger contribution to society of my work, and I didn’t care who my work exploited. I did anything really, that would get me from associate data, to full product, to principal. And furthermore I would do anything to get that sweet, sweet director position. I looked at everything and realized the sheer meaninglessness of it all. At one point I was literally getting paid by Abu Dabhi royals working for a sociopath just so I could go to bars named Snug on the weekend and order 25 dollar cocktails with dehydrated orange peels. Recently I’ve had a newfound respect and curiosity in pursuing works of art and literature that help me explore the spirituality that I’ve abandoned long ago. Man and his Symbols was a timely read as it not only laid out the context for the modern condition, but also laid out a way to pursue connection with a spiritual self.

For me, Man and His Symbols correctly identifies that modern life pushes an individual away from seeking deeper spiritual meaning. Atleast on a personal level, I have noticed throughout time myself and those closest to me having moved away from having religion or spirituality be a part of life. My career so far has only accelerated this feeling. One thing I find valuable in the framing of what the unconscious actually is, is the idea of it containing on some level the learning of all of mankind. This to me was the most interesting thread throughout the whole book. Much of the later parts, especially parts 2, 4, and 5 go into this idea much deeper. They show that throughout cultures and times there are certain themes that propagate over and over. Part 5 in particular shows an individual case how a Swiss engineer can read an obscure Chinese text and gain a tremendous amount of meaning from it. It shows that these questions of finding meaning live within all of us, and have lived within our forefathers before us. I find a deep comfort in knowing that the questions and struggles I’m facing now aren’t purely and individual failing. That I’m not alone in all this.

The primary thesis of Man and his Symbols is that an individual can gain a deeper, more holistic understanding of themselves by coming closer to their “unconscious self”. Human minds consist of two basic elemental structures, the conscious and unconscious mind. Jung believed that the conscious mind was the rational, everyday mind that we use to go about our daily lives. Meanwhile, the unconscious was the product of collective experiences inherited in the genes and only reveals itself to the conscious mind through the form of dreams.

The skills and attributes that man acquires in order to survive and thrive in the modern world push him further and further away from his unconscious side. The modern world demands rationality & individualism while de-emphasizing faith & social understanding. Jung portends that this unbalanced pursuit of satisfying the conscious side leads to an imbalance in one’s self between the conscious and unconscious. This leads to feelings of purposelessness, alienation from others, and a deep seated loneliness. In order for man to reconcile the two halves of his self & move past his ennui he must seek to reconnect with his unconscious, listen to it, and find the balance once again between conscious and unconscious thought. Jung goes on to outline that the mechanism in which to listen to your unconscious is to pay attention and find meaning within dreams as dreams are a way for the unconscious to communicate to an individual. By looking into the symbols contained within these dreams you are able to derive deep seated learnings about your conscious’ weaknesses and use those learnings to alter your conscious life. Much of the rest of the book are his followers outlining how certain symbols have universal meanings and can be traced across vastly different cultures. By listening and acting on the messages in your dreams you let your unconscious mind become an integral part of your life, and help guide the actions you take in the real world. These guided actions will help you move past feelings of spiritual emptiness and live a happy and fulfilling life.

My key takeaway from Man and His Symbols was actually that it would be valuable to delve deeper into forms of art and literature that center around questions of faith and how to connect to something that is larger than yourself. I don’t entirely buy into the idea that the way to deeper spiritual fulfillment is an individual analysis of your personal dreams. I think that method of finding fulfillment is lacking, as it separates you from your fellow man and turns social spiritual experiences back into an individualistic exercise. At least for me, it’s kind of like saying the cure for feeling individualistic and selfish is to just turn further inward. As Man and his Symbols correctly identifies, the spiritual struggles we face are largely universal. It seems to me that these struggles of personal loss of connection can only really be solved with removing those individualistic impulses and exploring meaning with your fellow man.

Groceries

Yesterday we ordered groceries for delivery from Costco. An elderly Chinese woman who spoke broken English arrived at our house in the morning bringing everything we ordered. My mom and I helped her unload the hundred or so pounds of groceries from her car to our house. I unloaded the three bags of soil, and they were heavy and unwieldy. I didn’t get her name, but she was profusely thankful that we were helping her unload all those groceries from her car. The truth was I wanted her to feel anger at us, not this misplaced sense of deference. That would at least validate the level of shame I felt in being a part of a society where an older woman roughly the same age as my mother was doing physical, manual labour for a total compensation of $30. She’s going to stretch those $30 to pay for food for her family, to pay rent on her house, and to pay for all the gas she needs to move groceries from place to place. So how do we (the affluent) rationalize this situation, a situation where an elderly person is forced into manual labour to scrape by a living serving those who happen to be in a more financially secure position?

If you have your material conditions met enough you are not in daily stress or pain like the woman who has to lift hundreds of pounds of groceries all day in her old age to feed her family. But you do have to confront and reckon with the reality that many of your fellow men must go through these acts in order to survive. You do have to reckon with the fact that your lifestyle, your lack of political engagement, your lack of desire to seek out and create strong social bonds which you can leverage to create meaningful change directly contribute to this plight. So instead of embracing the pain of these decisions, we whole heartedly ignore them. We pursue the much easier task of embracing philosophies and material goods that don’t necessarily make us feel good, but lighten this burden. We slowly kill our humanity. As Samuel Johnson states “A man who makes a beast of himself gets rid of the pain of being a man”. These philosophies & consumption of material goods have to be ritualistically re-affirmed constantly. Any respite from these medications puts us in a spot where we have to actually confront the reality before us. So we desperately have to constantly signal that we are good people, and constantly find new toys to consume. These signals aren’t even for others, they are mostly to convince ourselves.

Myself and many of the affluent people I know have basically abandoned religion as a steadfast set of beliefs to live by. This abandonment isn’t necessarily a bad thing, and in fact should allow us to look up and put effort into seeking our shared truth as humanity. Try to understand the shared line that is tied throughout all humans, and then try to understand what actions will strengthen and bring us closer to that line. But instead, we don’t engage in this pursuit at all. We don’t engage in this pursuit and instead look around at our material realities and cobble together a series of beliefs that provide justification to our position in life. We abandon the essential pursuit of universal truths and instead subscribe to a hyper individualistic philosophy that naturally deadens any connections to others. This edifice of distraction we have created is a way to never having to actually confront these universal truths. You can live in this cardboard version of constructed reality and feel respite from the need to pursue that connection. Instead of the pursuit, you substitute a constant, humming anxiety as you know things aren’t right, but you refuse to actually do any action about it. In fact, you cannot take any sort of action, as any sort of action will break you out of this spell and force you to confront the truth you have been running away from. Facing the truth is unbearable. So instead you have to rebuild this edifice of false beliefs every single day, you have to do everything you can to batten down the hatches while the whole time you are miserable. It’s easier in the short term than confronting the fear.

The current system we’ve set up with deep inequalities pervading every aspect of life services nobody. For the people on the bottom, their material conditions aren’t met and they constantly life the stress of being on a cliff of destitution. The affluent who have their material conditions met are forced to reckon with this fact by consuming short term solutions that deaden their ability to feel. This alienates the groups from each other and severs any semblance of a cohesive society, a society that wants to sustainably build better and better futures. Right now we’ve abandoned this goal entirely, as life expectancy has declined in the wealthiest country in the history of the world. To grapple with this fact we must strip ourselves of our basic humanity and fill that space with philosophies that assuage us and material goods that give short term dopamine boosts. The tragedy of our current situation is that while the bottom suffer materially, those on top live through a profound emptiness of the soul that they cannot address.

My girlfriend works as a speech language pathologist. She deals with many children who are autistic, but have not had a formal diagnosis yet. The parents fall into two camps. They either summarily reject that their child is autistic and pretend that everything is fine, or they accept and buy into a program to do everything they can to get their skills up to what they can be. The parents that reject their child’s diagnosis live in a fantasy world where they have a deep, underlying sadness with every interaction with their own child. They put an inordinate amount of effort into acting like everything is fine and are exhausted physically and spiritually. The parents that choose to acknowledge the reality of the situation feel a profound sadness that their child is not what they imagined it to be. They find the strength to push through the sadness and fear to reach a strong sense of purpose and fulfillment in working to make their most precious being in the world the strongest they can be. All of us face this choice today in our everyday lives. We can either chose to reject the reality in which we live in and live as husks of ourselves or we can accept this reality and begin the work of understanding and building a better one.

Heroes

I woke up this morning and saw this article in the paper talking about Pandemic Pay Cuts to health care workers. The gist of it is that private for profit hospitals & private equity backed staffing companies are cutting hours and pay in an effort to keep their topline revenues intact. Gotta love living in a world where health care workers can’t pay their rent while the MBAs execs preserve their rev bonuses so they can pay off their third cabin in Tahoe.

The immediate reaction to this amongst everybody is reverting to “Oh Dear”-ism. Sit back, watch the situation unfold, and helplessly sigh soflty to oneself “Oh Dear, if only things could be different.” The next reaction is applauding health care workers in their personal lives, and mentioning how they are our true “heroes”. Being called a hero is a death knell in this country. The moment Lizzo goes on T.V. and dedicates a song to you the end is near.

The troops are heroes until they come back from war, get fucked up and end up homeless. We walk over them in San Francisco and do nothing beyond quietly curse how dirty the city has become.

The teachers are heroes for being stewards of our children, but we fight any increase in property tax tooth and nail so they won’t be able to raise children in the very communities they teach in. And god forbid the money from the affluent suburbs go to underdeveloped cities. We wouldn’t want our hard earned dollars going to people who would just waste it over in East Oakland, they don’t know the value of a good education anyways.

The civil rights leaders were heroes for fighting for marginalized and segregated communities. As soon as the first crop of leaders died or were killed, we promptly forgot every single one of their messages. We lionize MLK every February while living in greater segregation today than the year he died.

Hero worship is just another distillation of how the liberals sees the world. The intent is what matters, not the tangible actions taken. Just like the Catholics confess every Sunday, we elevate people jnto the vaunted status of heroes to sooth our own egos, to absolve us of the sins of ignorance and inaction.

The truth of the matter is that the mechanisms to exert any kind of political will reflecting the values you want to see in a society are non existent today. Our leaders in politics have convinced us that government is the problem, and our lives should be controlled by the gods of the free market. I don’t even have to waste time thinking about politics anymore except for an hour once every two years where I go and vote for a candidate that looks the most like me. I can instead entrust all the energy and political power that I have to some Ivy league legacy graduate bond trader who gets rewarded for making my 401k go up a consistent 3% every year by finding new ways to gut American industry and ship the jobs overseas. I, of course, would never be negatively effected by these changes. I was smart enough to become an engineer, a doctor, a marketing genius or a MBA grad using spreadsheets to determine the fates of all the people who aren’t as smart as me. Why would I want to invest my hard earned time & money into political parties, labor unions, or even community civic groups when I have no use for them anymore. If my individual desires are met under this arrangement, what use are these levers of political engagement that I have access to? It’s easier to let these levers wither and rot away.

And now I’m confronted with a dilemma. I’m a good person, and I can’t just stand by and watch those on the front lines of a pandemic get treated like trash. As an individual, if I wanted to ensure that health care heroes receive enough support to live a decent, stable life what can be done? What organization do I speak to? How do I get involved? The harsh reality is, under the current system there is no way I can make my voice heard. The halls of power are closed to me and there is no way to enter. The only option I have now is to sit back, watch the situation unfold, and helplessly sigh soflty to myself “Oh Dear, if only things could be different.” After giving up all avenues to exercise tangible action, the only thing left is to signal my intent as loudly as possible.

The task before us is to rebuild the levers of political control that have been dismantled over the past forty years. To do this, we must look primarily outside of the current two party system. Neither party is built to be responsive to the needs of “essential workers”, or anybody who trades their time for money. Both parties are thoroughly in the arm of the non-essential, the wealthy who hold massive amounts of capital in the form of land or investments and don’t have to work a single day in their life. Matt Taibbi illustrates this very clearly in his piece on the recent bailout bill.

” As we head into the second month of pandemic lockdown, two parallel narratives are developing about the financial rescue. In one, ordinary people receive aid through programs that are piecemeal, complex, and riddled with conditions. A law freezing evictions applies to holders of government-backed mortgages only. “Disaster grants” are coming more slowly and in smaller amounts than expected; small businesses were disappointed to learn from the SBA early last week that aid would be limited to $1000 per employee. A one-time “economic impact payment,” reportedly delayed so recipients could experience the thrilling visual of Donald Trump’s name on the check, might help make half a rent payment. Unemployment insurance amounts have been raised, so tip and gig workers can now be ineligible for $600 a week more than before! The cost of a coronavirus test might be free, but you test positive, you could up paying $50,000 or more in hospital costs even with insurance. And so on. 

Meanwhile, “relief” programs aimed at the top income levels were immediate, staggering in size and scope, and often appeared as grants rather than loans.”

Whether it be by joining mutual aid organizations, running for political office oneself or joining smaller, more responsive political parties such as the PSL or the DSA, we must mobilize to reclaim the power we have given away. We can then use this power to ensure that our everyday heroes are valued.

Liberal Spirituality

I’m surrounded by good liberals in the Bay Area. As of December 2019, these good liberals are paying rapt attention to every element of the ongoing impeachment process. When you tune into any news outlet the only thing that’s being talked about is the impeachment process. It seems like it’s quite literally the only thing that mainstream media and good liberal viewers can focus energy on.

If you take one step back, it’s painfully obvious that literally nothing meaningful will come out of this process. It doesn’t take a genius to see how this is going to play out. Like the Russia investigation and the hallowed Mueller report, the consequences of impeachment are going to lead to… nothing. The articles of impeachment are passed, and in a few months time Trump will be cleared by a Republican controlled Senate. The Democrats will say the Republicans are putting party over country, the Republicans will say that the “truth has won out”. Aside from the 2% of the US population addicted to prime time news (1.7M who watch MSNBC, 2.3M who watch FOX, 1M people who watch CNN) nobody will give a shit. Nobody will give a shit because their lives will not be meaningfully changed in any way. If the outcome is foreseen and the outcome will achieve nothing, then why has there been so much political energy by the Democrats devoted to this singular act? Why is this the only thing the mainstream media talk about? Why is this the only thing that good liberals care about?

The spiritual values of a good liberal are centered around controlling and maintaining the moral & cultural high ground. However, their moral underpinnings are so self contradictory that they cannot withstand any sustained scrutiny. They are for equal rights, but not to the extent where they’re serious about reparations or a massive overhaul of the prison industrial complex or dismantling the police state. They are anti war, but not to the extent where they want to dismantle the military industrial complex and the empire it sustains. They are pro environment but skeptical of the green new deal. Not because it’s unfeasible but because it’s disruptive to the status quo from which they benefit directly. They are against the predatory practices of capitalism but are against anything that could impact the value of their 401k. They are for greater income equality but not if that threatens their ability to eat sushi whenever they feel like it or purchase whatever they want from Amazon and have it at their house in a day. The maintenance of the moral & cultural high ground allows them to feel like good people while being banner men for the underlying exploitative economic system they are ingrained in.

There is a way to live within this contradictory system of maintaining a moral high ground without challenging the existing pillars of oppression laid threadbare by the current administration. That way is to focus on useless, purely performative symbolic shit that feels like it has weight.

So much attention & energy is being devoted to impeachment precisely because it will not meaningfully change anything material about people’s lives. Good liberals who are concerned with Russiagate, the Mueller report, and impeachment don’t actually want to change anybody’s lives in any meaningful, positive way. What good liberals truly want more than anything is a political institution which would provide the cultural lubricant necessary to help justify living under a status quo that has been economically & socially devastating to people within and outside of the United States. Everyone that is, except them.

This obsession with technocratic rule breaking by conservatives isn’t a new development, but has been deeply heightened with this current administration. This development has been a boon for politicians and media. Liberal politicians like my very own rep Eric Swalwell attach themselves to this issue and make it the key focal point of their campaign. This allows them to forgo any push for fundamental change that economically improve the lives of the American or global populace. MSNBC is breaking viewership records left and right. Good liberals desperately need a product which will satisfy their spiritual need of maintaining the high ground, and like good capitalists MSNBC is more than happy to fill this exact need. The funny thing is that this product isn’t even good, it’s like watching a person read a rulebook for hours at a time. Other spiritual grifters like Joel Osteen at least have music & the promise of an eternal afterlife.

The painful realization I’ve underwent in the past three years is that this administration isn’t corrupting the existing political & economic establishment, but merely exposing the truth of what it has always been. The administrations that existed before the current administration put an inordinate amount of effort in the building of cultural & social lubricant to justify the actions by the state, and therefore justify the existence and maintenance of the way our current economic and social relations have been set up. Previous institutions provided good liberals the spiritual fulfillment they craved while silently maintaining the devastating status quo. Trump has exposed our political systems for what they actually are and has removed the cultural lubricant entirely. By removing this cultural lubricant he hasn’t really changed any of government’s institutions, but has been able to remove the brakes and allow it to move faster and focus on what it has already been doing since its inception.

The thing is, for all my bluster I don’t think I’m really good either. I’ve shed my old good liberal spiritual philosophy and figuring out what my place in all this is now has been difficult. I’m not a good liberal anymore but… I’m not sure what I am now. I can call myself a leftist, but aside from reading a lot of leftist literature, my track record of being apart of actual change is pretty scant. It’s something that I’m actively struggling with right now. The question that I ask myself daily is how I can push to be apart of something that truly builds a better world for us all. It’s something that I look forward to writing as I figure out.

To end this already long ramble, an alternative to being a good liberal would be to seek to understand the historical context & underlying mechanisms that construct the world we live in. A great starting point is Listen, Liberal by Tom Frank. Among many other books, I credit this one with opening my eyes on systemic issues I was blind to in my good liberal days under the Bush & Obama administrations. Thomas Frank is a great starting point, and from there if you are still interested there are a bunch of good leftist podcasts to dive into as well.

• The Dollop is super funny and a great entry point into another lens in which to view historic events.
• The oft maligned Chapotraphouse is a pretty good round table discussion show on current events. I think it’s a good entry point. Think of it as the Jon Stewart Daily Show for leftism.
• Rev Left Radio is a great podcast on historic left movements with researched & balanced takes.
• Citations Needed is a great podcast taking a Manufactured Consent view of current media cycles

Calls on Schwab

Action:

Bought a 35$ call option for Schwab on 10/7/2019 for $285 which expires in January 2020 

Sold the call for $550 on 10/15 when Scwhab was around ~39.50, netting a ~95% gain.

Main reason:

1. Announcement of free trades and the corresponding large dip in price

Narrative:

On 10/01 Schwab announced that they are going to be giving commission free trades to their customers. These commissions make up roughly 3-4% of their revenue or 90-100M dollars of quarterly revenue. This announcement saw a corresponding sell-off of the stock and a 16% dip in stock price. 

I saw this and thought a few things:

1. The stock dip implied a bigger revenue hit than what actually happened
2. It’s competitors (Etrade, TD Ameritrade, etc…) will all follow suit and will be hit harder than Schwab as commissions are a larger part of their revenue stream. They would likely come out on top in terms of the “commission wars”
3. Having just finished Flash Boys, I figured that they wouldn’t just lose 3-4% of their revenue outright. They would likely be able to encourage higher transactions amongst their user base and then recoup the commission fees by selling the bundled transactions to high frequency trading firms ( payment for order flow)
   1. In fact, it is a huge part of RobinHood’s business model which was valued at ~6 billion recently

My gut told me that the stock would rebound quickly as the market would realize that killing commissions was a flashy thing to do, but would not have that high of an impact on the actual revenue stream of Schwab. This time, my gut may have been right.

Puts on Uber

• Action:
  • Bought 2 puts on Uber at two separate times
    • Bought a 36$ put option on 9/17/2019 for $540 which expires on 3/20/2020
      • Uber was at ~33$ at the time of purchase.
    • Bought a 37$ put option on 10/11/2019 for $870 which expires on 3/20/2020
      • Uber was at ~31$ at the time of purchase
    • Basically betting that Uber will go down below $30 dollars (around 28 – 29) sometime within the next ~6 months.

• Reasons:
  • Big points as to why are:
    • Core business model is trash
    • Passage of AB-5
    • Desparate analyst shilling 

• Core Business Model = 🗑️
  • Their actual business model is trash, which is elaborated in excruciating detail in a multi-part series by Hubert Horan of nakedcapitalism. My main man Scott Galloway called them out as well. These two data points are enough to convince me that Uber’s business model is essentially selling $1 for 75 cents.
  • I have no idea wtf Uber freight is actually doing and how it fits in any way to their core business model. In addition, there is another Softbank funded company in literally the same space (Flexport). The Glassdoor comments of this company make its work environment look like an Uber 2.0 and the softbank money does not bode well for its future. Hopefully they go public and I can write another post about how I bought put options for them.
  • UberEats is seen as a large growth driver, but Uber literally classifies UberEats as a “loss leader”. This is insane as nobody has proven that mass delivery is a viable business model and there are other massively funded companies competing directly with UberEats on this front (all at a loss). They are literally using a product that loses a ton of money to drive users to an offering that loses a little bit less than a ton of money. It makes no sense, but it fits in with their growth narrative so they keep pouring money into it.
    • A more nuanced look at their business model shows that they are selling $1 for 50 cents in the hope that some of those customers will convert to buying $1 for 75 cents.
• AB-5
  • AB-5 passed, and Uber stock actually went up. This seemed odd, as this has the chance of severely harming and/or killing Uber within California as they now have to pay their workers a living wage. The only way Uber is able to keep its growth metrics narrative up is to keep its existing massive user base in California which means they will hemorrhage more capital in either fighting the bill or paying their workers an actual livable wage. They cannot actually decrease prices to the end user because the only thing keeping their narrative even remotely viable is the fact that they are continuing to see massive amounts of growth. Increasing prices in one of their largest markets will shed users and kill this narrative. 
  • In addition to this bill passing in California, it’s already starting copycat bills in other large population states in the US such as NY. In the next few years we will likely see more gig worker solidarity popping up across the US, emboldened by the success in California.
• Desperate Analyst Shilling
  • Many analysts at large investment banks are shilling the shit out of the stock. Their analysis literally makes no sense and many times target stock prices literally look like they are pulled out of thin air. The larger narrative that large bank analysts, VCs, & the larger business world is trying to shape is that Uber won’t be profitable now, but will be in the next 5 – 10 years. They are comparing this to other non-profitable companies at IPO such as Amazon. The thing they miss is that Uber doesn’t have razor thin margins, Uber literally has negative margins with no way of getting that in the black. 
  • Gut here is telling me that analysts are desperate to offload this stock to retail investors so they are not left holding the bag.

Analysis of Tumors Relating to Breast Cancer

October has officially become Breast Cancer Awareness month, which is an annual campaign to increase awareness of the disease and has transformed the official colors of October from orange and black to a much more radiant pink. While most people are aware of breast cancer, many forget to take the steps to have a plan to detect the disease in its early stages and encourage others to do the same.

In my hopes of adding a little bit to the cause, I have done a report analyzing breast cancer data in order to determine the best method to classify a tumor as benign or malignant, and whether or not the best predictors for malignancy are also strong indicators for the recurrence of cancer. This report focuses on using multivariate statistical techniques as well as basic machine learning in order to come up with a sorting algorithm. I hope this report as well as the code is useful to my readership which is hitting the upper single digits lately! Stay with me folks, you’re going to be Orange Numbers hipsters in no time as we soon move on up to the big leagues and start competing with Grantland for views.

The full report can be found and downloaded here. And the code accompanying the report can be found here.

Thanks for reading!

Painting a Picture with Data

As a kid I always wanted a Roomba. A robot that could get rid of one of my most hated chores sounded like a godsend. But alas, my begging wasn’t enough to convince my parents and vacuuming became a staple on my chore list. Years later, I still haven’t gotten a Roomba, however I got my hands on what I consider to be the next best thing. A set of one hundred incredibly hard to read log files.

At this point you’re probably have two questions. What on earth does having these log files compare to having a vacuuming robot? And more importantly, now that you’re all grown up why don’t you just buy a Roomba and stop complaining?

Well, I’ll start by tackling the second question. Although I’ve made amassed a fortune in the tens of dollars, it’s hard for me to justify spending seven hundred dollars just to get out of 20 minutes of chores a week. How these log files relate to the Roomba has a slightly more complicated answer. When I was a kid, seeing the robot do its work at a friends house fascinated me. It looked so futuristic how it was able to  navigate furniture and objects with no external controls. I’ve always wondered how it was able to do that and these log files help uncover just that.

These files detail the movement of a robot as it moves around a test area searching for a target, avoiding obstacles. Figure 1 shows an example path through the test area. At each step, the robot moves and then “looks” around with a 2 meter long laser to determine what is currently around it. A robot can look in a circle, i.e. a viewing area of 360◦. At any angle, it can detect an object up to 2 meters from it.  Each log file corresponds to a different trial in which a robot navigated the same test area. The location of the target is different in each trial but the locations of the walls/barriers/obstacles are expected to be fixed.Using these files I will finally be able to see how robots use feedback from sensors in order to “paint a picture” of a room they are in and thus navigate it easier.

As I mentioned earlier, these log files are very hard to read in that they aren’t stored in an easy to read csv. For reference they can be found by going here. However, they are stored in a logical manner which can be read using a couple of steps and the string parsing properties of R. Lets start off by looking at a sample log file and seeing how it is laid out and what patterns it shows.

## Player version 2.1.3
## File version 0.3.0
## Format:
## – Messages are newline-separated
## – Common header to each message is:
## time host robot interface index type subtype
## (double) (uint) (uint) (string) (uint) (uint) (uint)
## – Following the common header is the message payload
0000000000.100 16777343 6665 laser 00 004 001 +0.000 +0.000 0.000 0.156 0.155
0000000000.200 16777343 6665 position2d 00 004 001 -00.040 +00.000 +0.000 +00.440 +00.380
0000000000.200 16777343 6665 position2d 00 001 001 -14.000 -07.000 +0.785 +00.000 +00.000 +00.000 0
0000000000.200 16777343 6665 laser 00 001 001 0001 -3.1416 +3.1416 +0.01740495 +2.0000 0361 1.838 0 1.807 0 1.778 0 1.749 0 1.723 0 1.697 0 1.673 0 1.650 0 1.628 0 1.607 0 1.587 0 1.568 0 1.550 0 1.533 0 1.517 0 1.501 0 1.486 0 1.472 0 1.459 0 1.446 0 1.434 0 1.423 0 1.412 0 1.402 0 1.392 0 1.383 0 1.375 0 1.367 0 1.359 0 1.352 0 1.346 0 1.340 0 1.334 0 1.329 0 1.324 0 1.320 0 1.316 0 1.313 0 1.310 0 1.307 0 1.305 0 1.303

Starting off we can see that comment lines start with ## and these provide a description of the contents of each line. Each data line starts with 7 values that correspond to variables named time, host, robot, interface, index, type and subtype. After these , the remainder of the line contains values in a format corresponding to the interface type, e.g. laser, position2d. I am interested in the lines with values position2d and 001 for the interface and type. Specifically, we are interested in the cases of a position2d line immediately followed by a laser line. Each of these pairs of lines tell us the location of the robot and what it sees at that location.

The data following the first seven values for the position2d lines contain information about the location. The eight and ninth values in the line are the x and y coordinates of the robot’s current position. There are also values for the robot’s yaw which will come in handy later. Don’t know what yaw is? Moving on, the laser lines have six additional values before listing the actual values of what the robot “sees” at each of the 361 angles it scans. These 361 values are the important values and come in pairs of range (of view) and intensity. We only care about the range and each value gives the distance the robot can see in the direction in which it is looking. The maximum value is 2 which means it saw nothing, but smaller values means it saw something at that distance.

There are 361 intensities as the robot starts at one position and then rotates the viewer in 1 degree increments. When it completes the entire circle, it has an extra observation corresponding to a repeated measurement at the initial point. The prior summary was gathered using information from here and guidance from here.

Now, our first task is to actually read in the data in a workable format, i.e. take the data contained in each log file and put it into a manageable data frame. Because there are a 100 distinct log files each with represent a different walk through of the test area, we will end up having a list containing 100 separate data frames. This took multiple steps as well as multiple functions. The code I used can be found here, but I’ll walk through it in case you’ve somehow managed to not have a text editor on your computer in the year 2014. First, I wrote a function named rmvHash that would remove all commented lines out of the log file. Then, the function splitLine splits each line into individual character elements. The function vectorize then goes through and makes sure all the numbers are not stored as “chars” and can be read and manipulated like numbers. The meaty function Reader uses all the above elements in order to parse through and store all relevant variables in a data frame. Finally, the function readLog puts all these functions together and spits out a usable data frame. The function readLog is then used to read and store the individual log files as data frames in a singular list.

Finally! We have the data and can begin making pictures of what the robot sees. Before we do that, lets take a look at some of the sample paths that the robot takes through the test area. Here are the paths shown by the first four log files where the green circle signifies the starting point. (Click on the pictures to enlarge):

Why do these paths widely differ? Well, as mentioned earlier this robot was specifically designed to look for a particular object. The object is placed in different places within the test area on each separate run. Therefore, the robot follows a path, using its laser to scan at each point along the way until it finds its target. When it believes it reaches it target, it stops.

Now that we have a plot of how the robot travels, we can finally get to seeing what the robot sees as it moves along! We can see if we can literally paint a picture of the room using the information that the robot gives us. In order to do this, I plotted all the points along the path where the laser scan returned a value less than two. This is because a scan at a particular point returning a value of two signifies that the robot did not encounter an obstacle at that point. A scan less than two meters means that the robot encountered an object. After coding this into a function and plotting, we get the following four pictures from the first four log files.

There we finally got it! A picture based solely on information given by the robot. First off, we can see that the robot is very good at finding the object it needs to find. Once it hits an object that looks like it has a curvature and an arc corresponding to a radius of .5 meters it stops. Also, the robot is very good at avoiding obstacles. As we see from the path’s above, the robot is able to see where the obstacles are and then successfully navigate around them. In fact, using its laser sensors, runs two and four paint startlingly accurate portraits of the obstacles contained within the room.

This exercise was very cathartic to me. It allowed me to see how a navigation robot (i.e. a Roomba) uses sensors in order to identify and avert obstacles. It also was a great exercise in getting data in a sub-optimal format, and then converting it into a manner that is ready for analysis. Finally, it truly did allow me to take a jumble of numbers and then turn them into real life pictures that conveyed relevant information. If you have any further questions about the code I used, or comments on this article please feel free to email me for further information.

Beating the Market

One thing that has always fascinated me was the use of mathematical models in finance. More specifically, the use of models in order to understand and profit from the stock market. To me, it seemed so incredibly weird and counter intuitive. The stock market is known for being unpredictable. There are a ridiculous number of outside events that can affect the prices of stocks on a day to day basis. In addition to factors that our out of our control, these models have to take into account the stupidity of people who think they are qualified enough to make decisions about how to invest. I am saying this from the standpoint of an economics major, whose knowledge of long-winded theories and simplified economic models arguably makes me far worse than the average person when it comes to investing. That being said, there was a strategy for buying and selling stocks developed at Morgan Stanley in the 1980s called pairs trading that intrigued me. Using the statistical analysis software R, I’m attempting to recreate and see the viability of this strategy using real stock market data.

It was one of the first strategies based on computer-intensive analysis of past stock performance. The strategy involves keeping tabs on the ratio between two highly correlated stocks (i.e. AT&T and Verizon). As you can see from the following graph below, the ratio seems to be fluctuating around a stable average (the dashed green line). (Click on picture to enlarge)

The main idea is that movement of the ratio away from its historical average represents an opportunity to make money. For example, if AT&T is doing better than it usually does, relative to Verizon, then we should sell AT&T and buy Verizon. This is called “opening a position”. Then, when the ratio returns to its average, we should buy AT&T and sell Verizon. This is called “closing the position”. The dashed red lines above are what we have designated as what constitutes a point “above” and “below” the ratio in order to determine when to open a position. Determining where the red lines are supposed to be is what analysts get paid the big bucks to do, and it is what I’ll be attempting to do on a salary of cold pizza.

In order to carry this out I carried out the following steps. First, I set up some rules. When the ratio moves above m + ks (the mean plus some constant k times the standard deviation) we sell a dollar worth of stock one and buy a dollar worth of stock 2. We then wait until the ratio is less than or equal to the mean, at which point we close the position by buying back however many shares of stock 1 we initially sold, and selling the shares of stock 2 we initially bought. When the ratio is less than m – ks I do the same thing, but reversing the roles of stock 1 and stock 2. Also, we wait until the ratio comes back up to be greater than or equal to the mean. There are a couple things to note. First, k represents how extreme the ratio needs to be before we open a position. In essence, setting the value of k allows us to control where the “red lines” shown earlier are supposed to be. Secondly, one dollar is just an arbitrary number chosen for simplicity. It allows us to work with stocks that have very different prices and to deal with fractions of stocks.

First I wrote a function to determine the closing and opening points of the graph, and a visualization is shown below: (Click on picture to enlarge)

 

As you can see from the graph above, the green circles represent when we open the trade, and the red circles represent where we close the trade. After getting the opening and closing points, I then wrote a function that would go through and calculate the summed profit of going through all these points. The total profit while using an initial value of k turned out to be 1.75. But I think we can do better! I have no idea what the optimal value of k is, but lets find out.

In order to optimize k, I first split the data into two parts, training and test. I sought to optimize the value of k using the training data, and then use that value on the test data in order to see how accurate I was. I then created a vector of 1,000 consequent values from 0 to 3.32(the maximum distance of the ratio from the mean). I then used these values in place of k and wrote a simple for loop to calculate and store the profits calculated using the trainingdata at each of these 1,000 values. The following graph shows the profit calculated at each value of k. (Click on picture to enlarge)

 

As you can see, the most optimal value of k in this scenario is contained between 0 and .5. After that there seems to be a steep drop-off. It turns out the value of k that produces the most profit for the training data would be .2364. Finally, we got our optimal value of k! Using this value of k and calculated profit for our test data, we end up getting an outstanding profit of….

-.1206

That’s right. After all that optimization on past data we actually ended up with a negative profit. How could this possibly be? It certainly wasn’t my code, as it is immaculate. And it definitely isn’t the math or the principles behind this strategy as they intuitively make sense. What could have possibly went wrong?

Well, it turns out an incredible number of things can go wrong. Which makes sense, otherwise this strategy would be used by every scrub with a computer looking to make a quick buck (like me). It also explains why Long-Term Capital Management (LCTM), a large hedge fund who used this strategy extensively,went defunct in 1998. The original proponents of this strategy Morgan Stanley? They lost 80% of their value during the financial crisis.

This strategy is built on two ideals. One, the theory that market irrationality will correct itself. Two, you should be able to predict the future using information from the past. The first theory is sound to build on, but there are some problems in execution. Financial institutions use algorithmic trading strategies that monitor for deviations in price, automatically buying and selling to capitalize on market inefficiencies.When the relatively same algorithm is used by funds carrying billions of dollars what happens? The moment a fund invests/divests in a pair of stocks the rest follow suit and this market inefficiency is corrected almost instantaneously. This removes any margin for profit. Thus, pairs trading became less advantageous as more groups started doing it. This leads to increased money being used in each transaction to try to get profit from increasingly small margins. As increased money is used in each transaction, firms are more susceptible to shocks in the marketplace and stand to lose more and more.

Secondly, how much of the past can you use to predict the future? In this exercise I used the entirety of data that Verizon and AT&T had in common. However, if I only used data from the past ten years in order to calculate and test out my k value, would I have gotten better results? Perhaps, but there is no guarantee that this value of k would be any more accurate or trustworthy. However, firms and people choose which data to look at to get an image of profitability, and then run with it in hopes of beating everybody else to the punch.

Really, its the same story over and over again. People think that they can superimpose this concept that everybody who is in the stock market behaves stably and rationally. The answer is that they don’t, and I really don’t know how many more market crashes we need to prove that using complex mathematical models to predict the market is not the way to go. Pairs trading opened the door to these models, and now institutions hire high level mathematicians to come up with algorithms to beat the market. My opinion is, it doesn’t matter how much math you shovel into these algorithms, they are always going to be susceptible to shock. The sheer complexity is a veneer that covers up an empty model that in reality gives as good of advice as a coin flip.

As much as I talk down on models such as this, it was a great exercise in coding. If you have any questions on the code I used to set up the various functions I mentioned then please feel free to email me. Also if you have any questions or comment don’t hesitate to contact me. Thanks for reading.