Coupling & Cohesion: How Musk Would Need To Rearchitect Twitter

Wired magazine published an article about why Musk’s Plan to Reveal The Twitter Algorithm Won’t Solve Anything.

Several of my non-programmer friends were interested in this, and we started chatting. Because the idea itself is obviously shockingly out of left field, I discovered this was a perfect opportunity to explain the properties of Coupling & Cohesion and why they matter.

Coupling & cohesion are defined in terms of a change you want to make to a system. In this case, Elon Musk would like to open source “the algorithm”, which he defines as all the bits of code that “make any changes to people's tweets, if they're emphasized or de-emphasized”.

Disclaimer

I want to be clear that nothing here is based on my experience with the Twitter code base. I wouldn't speak to any private information, and my experience was nearly a decade ago. Things have most certainly changed.

The information from current developers in the article is plenty for us to speculate about how coupled and cohesive the system is with regards to this particular change.

Cohesion

Wired magazine reports that there is “no single algorithm that guides the way Twitter decides to elevate or bury content”. Like many high-traffic platforms, Twitter uses a microservice architecture. According to the Wired article, multiple places in the code may promote or hide content, scattered across a multitude of services.

This is an example of low cohesion. We want to change all the code related to promoting or suppressing tweets at once. That code is scattered in many locations & mixed in with totally unrelated code. It isn't cohesive.

In order to understand all the ways a tweet might be promoted or hidden, every piece of the system involved in any of those would need to be open sourced and the reader would need to understand how those components interacted. This makes the change Musk wants expensive & error-prone.

To make the change easier, Twitter would need to rearchitect their system. This would involve moving all the related behavior together in one place. It would also involve separating any behavior in those components that isn’t about promoting or hiding a tweet. A service or a group of services that only handled promoting or hiding tweets would be high cohesion and possible to open-source.

Coupling

There are two sources of coupling: code that the code being changed relies on, and code that relies on the code being changed. (Anyone know better words to distinguish those two? Let me know, because that is a mouthful.)

Luckily for Twitter, from Wired’s description it sounds like they are mostly dealing with only one of those two kinds of coupling. If not much else depends on which tweets are promoted or hidden, it makes the change a lot easier.

Wired reports that the scattered pieces of code “perform a complex dance atop mountains of data and a multitude of human actions. Results are also tailored to each user based on their personal information and behavior.” That is to say, the code that promotes or hides tweets is highly coupled to many different parts of the current system.

This coupling could prevent Twitter from extracting the behavior into a cohesive unit. Even if the code was centralized, it would still require understanding code that had nothing to do with promoting or hiding tweets in order to understand what is happening. If it is particularly tightly coupled, it might even be impossible to separate without an intermediate step.

Reducing coupling is less straight-forward than increasing cohesion. Twitter would need to consider why those dependencies were needed & what the purpose the data served. They would then turn that understanding into an interface of some kind, with names that reflect that understanding. Twitter’s current data could then be swapped out for some other source of data that satisfied the same purpose. That would let the system be loosely coupled with respect to this change.

"Quality"

I want to be clear here that nothing I describe here is the result of Twitter’s code being “bad”. Twitter’s code is built in a way that would make this particular change hard, but all code makes some kinds of changes hard.

With respect to different changes, Twitter’s system is already highly cohesive and loosely coupled. Twitter grew revenue 16% yoy last quarter. It has recently made obvious strides in reducing harassment & abuse on the platform. All of that involved hundreds of engineers safely evolving running software.

Writing "good" code isn't enough. Even the best code in the universe isn’t loosely coupled and highly cohesive with regards to every possible change.

Conclusion

I don’t think this goal is useful or plausible or something that will happen even if the sale does go through. I’m just using it here as a concrete example of a sweeping change that someone might want to make to an existing system.

Its very absurdity is useful. It clearly demonstrates why You Aren’t Going To Need It is a powerful approach to managing coupling & cohesion.

It is impossible to predict what billionaire in a midlife crisis will get angry your company didn’t let him post Hitler memes in the wake of his breakup & decide to buy your company to make one specific change. Attempting to anticipate that eventuality would have been a massive waste of time & money.

But when something like that happens, regardless of what the system looks like today we can adapt it. And as long as the system solved the previous goals as gracefully as possible, supporting all the existing features plus this one new change will be as easy as we could make it.

Imagine that Twitter had guessed that a billionaire would get mad about an ad they showed him. They might have spent a similar amount of time & effort as this project will take today making the ad targeting logic cohesive & decoupled. The code still wouldn't be any more cohesive or loosely coupled with regards to the change that a billionaire actually wants. It would have cost a bunch of money to do, making all the other work over the years harder, and it still wouldn't make this change any easier.

Attempting to anticipate the future doesn't help us build systems that can adapt to it.

If Musk follows through on wanting to publish all the code that contributes to promoting or hiding a tweet it will cost Twitter a great deal of money. The result is unlikely to be particularly useful, especially when the greatest factor in whether a tweet is "promoted" is whether other human beings hit the “retweet” button.

But by employing these principles, by first increasing cohesion and reducing coupling with regards to the specific change, it would be possible.

Moderating Discussions Over Video

As many colleges move online, I realize I have a somewhat-unique experience: I shameless ripped off the pedagogy from my small liberal arts professors and have spent the last decade+ applying it at distributed tech companies.  I've facilitated video conversations with anywhere from three to fifty participants, both in the course of my work, as part of reading groups for specific texts, on social science topics like "gender and racial bias in tech" and as part of consciousness raising groups to help foment cultural change.

You all have the advantage that the students have already been interacting with one another and with you; that pre-existing trust makes it much easier.  And you are all going to be doing this for the first time, so you can figure it out together. The best advice I can give is saving five minutes at the end to talk about how you all feel the discussion just went, and if it isn't working in the middle of the class, just stop and have a conversation about what isn't working.

The greatest challenge moving to video is that it is easier for people to check out from behind a screen and not have it be obvious.  The advantage is that if they do, it isn't as disruptive.  I always treat any video meeting as opt-in, and then work to make it easy for people to do that opting.

Basic advice:

  • Switch to Speaker View: when someone starts speaking their video will pop up, similar to you glancing at someone who takes a breath
  • Have the other videos in gallery view, so you keep an I on if people are engaged
  • For up to 10 people I don't bother with raised hands.  For more than that some video tools have a built-in mechanisms: for the rest you can use chat.
  • Have one person, ideally not a participant, take collaborative notes so everyone else can pay attention.  If they are in the same room as you (ideal) they can also manage the queue of people who want to speak on a piece of paper so you can glance down and know who is next.
  • Encourage people to take paper notes, so their screens can just be the video.  Similar, have them use an e-reader/tablet or print out any readings.  The other advantage of this is that you will be able to see when they glance down.

Facilitating:

  • Have a way for people to submit things they'd like on the agenda before the call starts: that gives you more time to plan. I typically use Slack for this, but wherever this group of people is chatting works.
  • Establish your facilitation plan up-front and communicate it, even if it is the same as last week. Remember, we've just taken out all the turn-taking mechanisms people are used to using for in-person discussions, so replacing them with something explicit helps.
  • Cover the goal of the specific discussion as well, and frame any expectations ("if you haven't read X you are welcome to observe but not participate" is a common one for our architectural reviews, for example.)
  • Since you can't go "around the room", for smaller groups where you want to go round-robin, write down the list of names of participants on the line so you can call in them confidently.  I like using the same list to keep track of people's contributions later.
  • Plan the emotional arc of the conversation: this is the key to keeping people engaged.  
  • As much as possible, prompt for specific kinds of comments, rather than using open-ended questions.  The tendency for confident voices is amplified, and being more specific draws out people who aren't confident responding to general prompts.
  • People aren't getting the usual signals to stop talking, so don't be shy about interrupting. I like interrupting with clarifying questions: it draws people into the conversation, rather than driving them out, without letting them run away with the time.  Facilitation has to be pretty active to be effective.
  • Provide positive feedback on the process of how people are participating, as well as the comments themselves

Accessibility:

  • Use the best camera you can find, and try to ensure your internet connection is high-bandwidth
  • Use a microphone (I like https://www.amazon.com/Blue-Snowball-Condenser-Microphone-Cardioid/dp/B014PYGTUQ) or you phone headset if your computer's built-in microphone isn't great
  • Having someone share notes makes the information more accessible, because it is harder to take notes in the video format with a computer in front of you
  • If you have students with hearing loss, CART services can often integrate with video meetings
  • Using techniques like going around the room can help people who are thrown off by the loss of implicit turn-taking
  • If your students' equipment isn't as good, don't be shy about repeating questions

Beyond all of that, know that it isn't as different as it feels at first and it is absolutely possible.

Rails Quirk: A Period in the URL Can Break The Route

When using Rails routing I came across an odd bug: a URL query parameter was breaking the route.  A URL query parameter without a period? Everything works fine. A URL query parameter with a period? 404.

I eventually found the answer in an off-hand comment in a random blog post, and traced it back to the code.  So that next time I remember what is going on, I figured I'd throw the explanation up here. By default, Rails assumes anything after the period represents the format (see the Mapping class defined in rails/actionpack/lib/action_dispatch/routing/mapper.rb). Which if, for example, you are using the format to determine whether a request should be served by a frontend app can then break the route.

To address this, you have two options.  First, you can follow the suggestion I've seen elsewhere and define your own constraint:

get "*path", to: "react_frontend#show", constraints: { path: /.*/ }
But to address the root of the issue, you can also just tell Rails not to look for the format with a regex:
get "*path", to: "react_frontend#show", :format => false






Reflecting on Diversity, Inclusion and My Self-Alienation

Two and a half years ago I joined LTSE, with the goal of changing the incentives companies face to prioritize short-term profits over everything else.  In May, the SEC approved the creation of the Long Term Stock Exchange, making us one of only a handful of venues authorized to list publicly traded companies.

When this milestone happened I discovered that I still have an internalized voice that says, "if you prioritize hiring underrepresented developers, it means you are de-prioritizing success". I found that some part of me holds an insidious belief that places where I felt comfortable couldn't be "the best" companies.  By demanding representation, this voice said, I was asking a sacrifice of the company I was working for. I had gotten as far as believing that sacrifice was justified, even necessary for the sake of justice, but it was still something I was being granted.

That voice is wrong.

We've built an engineering team here that is racially diverse and gender-balanced. We say out loud that we aren't trying to hire "smart" developers: we are hiring skilled developers who believe in practicing their skills in order to improve. We don't believe in a "founder gene": our tools set out to make explicit the implicit knowledge those folks horde, so that more people with valuable ideas can successfully found companies. My experience here is so different than what I had experienced elsewhere.  I no longer fantasize about quitting the industry on a regular basis. I feel like I can recruit without worrying that I am selling harmful snake oil, and I feel empowered to support people the way they want to be supported instead of the way the industries says we should want to be supported. But some part of me distrusts this ease. Part of me still believed that feeling comfortable must mean something is wrong, and that it is unreasonable to want this comfort "at the expense" of the things that "really matter".

That part is also wrong.

I don't believe that our success here vindicates "diverse" teams any more than not succeeding at this ridiculously ambitious mission would mean "diverse" teams are a failure. This is not a magical Utopia, and I still react to things that happen here with the weight of all those other experiences I have had. But this weekend I found myself crying as some deep-seated clinching, this sense that my basic existence was an impediment to success, loosened a little.  It is not unreasonable to want a community or company that takes me into account. We can succeed with a diverse team, where being a feminist is part of the bar, where we expect "D&I" efforts to be effective, where people take parental leave and no one yells and work is expected to be sustainable. It may even turn out that all of those things make it easier, not harder, to do useful, productive, successful work, rather than just being what it takes for me to not quit.

The part that always told me things could be different?  That part was right.

We are going to be hiring a bunch over the next stage of this project. Many of the people reaching out and proactively raising their hands are people who take for granted that every company in the world has a place for them. Some of them will turn out to be great, but my goal in this next phase is to make sure that other people, candidates who wouldn't think to jump in just because the project had some success, feel invited to join as well. I want them to know this company is for them, in a way it is not actually for all these people who get to assume that every company is.

UX Patterns of Emotional Journey

When building transformative experiences for our users, we begin by identifying the emotion that motivates their engagement.  We then imagine how we want them to feel when we have provided for their need.  Finally, we are left to build something that we think can successfully transform the first into the second.

The only way to actually know if such a design works for a sufficient audience to support the product is to experiment and see, but there are some patterns of UX that can suggest things we might want to try.  None of these is a product all on its own: we also have to actually address a need people have in a way that provides some substantive value.  But since we can offer what we think of as value and still not have people walk away feeling better than when they walked in, this is a toolbox we can come back to to ensure that the actual value we provide is also giving people something they want.

There are many more of these possible: I look forward to hearing about the patterns you have discovered! If you are interested in reading more about the use of visuals and interaction in creating experiences, I highly recommend Understanding Comics and Reimagining Comics by Scott McCloud: they are an accessible entry point into the world of visual and interactive impact.  The Design Of Everyday Things and Emotional Design, by Don Norman, are also great starting points, as well as Theater Of The Oppressed, by Augusto Boal and Impro by Keith Johnstone.

Introductory Language Values

I was having a conversation with some people about languages to use to teach programmers.  I am not a teacher and it has been my impression that language choice matters a lot less than pedagogy when creating good programmers comfortable with all of the possible tools of software development.  That said, I still have opinions and here are the things I look for in an intro language:

* Some things should be SUPER easy, hard things should be possible, nothing has to be fast or maintainable
* Fast feedback cycles between writing a thing and seeing if it works, with easily-visible results.  We want to introduce students to the phenomenal magical power of coding, because that will provide intrinsic motivation to keep going when things are hard.
* Transparent: if you dig down, you can figure out why something does what it does (LISP, Smalltalk and JavaScript all have this property)
* Supports encapsulation and recursion: these are the hardest concepts for most students to grasp, so introducing them early and insisting students use them is valuable.
* Good code should be pretty, ugly code should be ugly: I don't care if it's possible to write terrible code, I care if readable code is obviously readable. Students need to be able to start developing an aesthetic sense of code quality right away, but shouldn't have to write clean code in order to get something working.
* No meaningful whitespace or magical numbers of characters: these are often confusing to people not used to working with computers because in other places we use language those things don't matter.
* Good IDE support: most students aren't going to be used to working on the command line, and introducing version control is a more gentle introduction. This helps with explorability and getting students over the hurdle of understanding that they aren't writing prose: they are building a world with it's own internal rules system.
* Easy unit testing: it's hard enough to teach unit testing even at its easiest, but it is incredibly valuable to start with it because it introduces the idea of interfaces and encapsulation in a particularly tangible way. It can also help students learn to evolve software in small, safe steps.
* Publicly-visible well-written code bases: reading code is just as important as writing code when learning.
* An active, supportive, anti-sexist community: I want students to be able to feel like they belong when they go looking on line for information about what they are doing.
* Doesn't try to be clever or optimize for experienced user productivity: ideally, I should be able to tell a story about the language in a single sentence. "Everything is an object" or "everything inside the parentheses gets run together" or "we send objects messages" are ways to bootrstrap a mental model of the language (consider, for example, how Bootstrap explains LISP to teach students algebra: http://www.bootstrapworld.org/materials/spring2017/courses/algebra/units/unit1/index.html#lesson_Brainstorming5496)

Note that many of these are different things than I look for in a production language. I want students to make mistakes that help them learn, so protecting them from those mistakes isn't useful or helpful.  They aren't going to be working on large code bases, so libraries, package management and scalability aren't important. No language is perfect on all of these dimensions, but some are definitely better than others.

Confidence Through Feedback, or Why Imposter Syndrome is the Wrong Metaphor

Imposter syndrome is often presented as a personal failing.  A lack of confidence, our wrong-headed beliefs not matching the reality of how competent we are, or worst of a flaw of our gender.  Just tell yourself you are wrong!  Imagine everyone else is just like you!  Have confidence in all parts of yourself except that part that tells you not to have confidence!

Unsurprisingly, these interventions are not often effective.  At best, they change behavior, frequently while making people who already feel bad about themselves feel worse.  At worst, they lead people to stop trying to improve the environment they have found themselves in.

I would like to offer an alternative story: imposter syndrome is a rational response to insufficient feedback. 

Email Template For Addressing Conference Gender Diversity

Someone I worked with had asked for recommendations when I noted the speaking lineup of a conference he was attending was exclusively men, and I figured I'd share the letter I came up with in case it is useful to others:

While I attended $CONFERENCE_NAME last weekend, I was disappointed to notice the oversights in your speaker line up leading to it being made up of nothing but men.  Perhaps your prioritization of people with their own books to sell led to inadvertent systematic discrimination, as you were reliant on the discriminatory publishing world and more generally on people without a significant non-work-related demands on their time (who are most likely to be either single men or men in non-egalitarian marriages.)  {Depending on your impression of the conference itself, something like: "Since I was also disappointed in how much of the conference devolved into the speakers plugging their own books, I am confident you could kill two birds with one stone by instead seeking out the most qualified speakers.” could fit here too.}

I wanted to convey that when trying to build a group from $COMPANY to attend this weekend, a woman who is normally excited to attend local conferences had no interest at all.  Without any women speaking, a code of conduct or even the barest token of effort towards diversity, there was no evidence that there would exist other women there or that the men involved see women as peers.  She expected that the weekend would, at best, be full of getting interrupted so men could explaining things she already understood, dudes hitting on her, men quizzing her about the alien experience of being "one of those", people assuming she was part of the conference organizing staff or from recruiting or some attendee’s wife, or simply ignoring her all together.  She also assumed that any complaints would be brushed off as disruptive to the existing exclusionary atmosphere that it appears the organizers have cultivated.

Going forward, if I see another line up of all men speaking in Boston, I will have to assume she’s right and you are actively working to run a conference to alienate women.  Since I’m not interested in that environment, this may be my last $CONFERENCE_NAME conference; I could instead have attended $OTHER_CONFERENCE_THAT_MADE_AN_EFFORT the week before where both I and the women I know would both have felt more welcome.

Now, the reasoning behind this approach.  I like trying to turn it into a contest between conferences, since the only eventual pressure to change will come through economic pressure.  It also circumvents the argument that it's not possible or there are no qualified women, without ever having to point out just how incredibly insulting that argument is.  Other conferences have worked hard to change the make up on their conventions: accepting speakers through blind proposals (rather than just inviting people they know of or their currently-non-diverse attendees recommend), advertising a code of conduct widely and enforcing it when it comes up, creating scholarships for women who want to attend but who’s companies won’t support it and seeking out and addressing feedback from women speaking and attending.  It’s not like this stuff is easy; it’s jut possible.

I did have one more recommendation for the guy I was talking with: 

If you want to be helpful while you are there, be your usual polite and outgoing and aware self and discuss the technical work of any women you do meet, especially listening to their ideas and learning about the work they are doing.  A good interaction or two can brighten up even the most awkward conference.

The Dyslexic Programmer

I am dyslexic, and these are my experiences.  They certainly won’t be universal, especially as there isn’t just one form of dyslexia [0].  To identify my strain of dyslexia, I read quite quickly (though only somewhat accurately) through pure pattern-recognition. I can look at, say, "word" and identify that the second letter is an ‘o’, but if I want to understand it as a concept I ignore the letters involved all together [1].  Essentially, I’ve memorized how each word in the English language looks as a complete entity. (I also, thankfully, have an excellent conceptual memory.) This approach, of knowing something is made up of individual parts but not needing to worry about what specifically those parts are unless absolutely necessary, extends to how I approach math, history, social sciences and fantasy world-building as well. 

I believe that this tendency to generalization is why I am able to jump between levels of abstraction quite easily.  The concept of emergence, and the specific cases of recursion and polymorphism, are obvious to me.  Everything in the universe is made up of component parts, interacting in ways that give rise to the meta-phenomenon we observe, like “matter” and “consciousness”, and I can keep that in mind without worrying particularly about what those components are.  It is odd to me when people consider things to be discrete, isolated wholes; it can be useful to talk about them that way, but I usually don’t actually believe it.

How is Gender Studies hard?

Someone somewhere, of course, was mocking Gender Studies majors for taking an easy course because they couldn't hack liberal arts.  I made the point that the Gender Studies classes I took were significantly more difficult than my computer science classes.  Someone then asked me why I thought that was, and I came up with an answer: