Reset your Twitter password

Twitter made a fundamental security blunder yesterday that affects almost every user on the service: it was accidentally logging passwords in plain text internally.

Yes, Twitter, one of the world's biggest social networks... made a fundamental infrastructure error. I've been trying to figure out how this happened, because it's such a basic mistake that you'd hope a company of Twitter's maturity wouldn't make it.

Here's my best guess, based on digging around: GitHub was also affected by the exact same issue just a few days ago. This implies that some sort of common open-source library caused the problem, or at least had an error that caused logs to be appended with passwords accidentally.

Both Twitter and GitHub use the bcrypt encryption library, specifically bcrypt-ruby, which is an industry standard tool for encrypting passwords. That library has been mostly inactive for  two years, until nine days ago when GitHub employees committed a small change bumping its dependency on Ruby.

It appears that by default, the library (or another dependency) logged out these passwords to help developers diagnose the library while in development, but it was not set to production mode and continued to save them in plain text.

Twitter says that 'to its knowledge' the passwords were never accessed by employees, but given that the same company until recently offered broad access to delete any account, I find it difficult to believe it actually is able to audit these properly.

So, the moral of the story? Check the open source libraries your application depends on, and audit your logs, because there's likely something in there you don't expect.

Oh, and change your password for Twitter today.


Facebook's "Eye of Sauron"

You might not know this, but most technology companies out there include a mode in their products for employees called "impersonation" which allows engineers or support people to log in as you with the click of a button.

As it turns out, until recently any Facebook employee was able to use such features to log in as any user on the service and see the social network through their eyes as if actually logged in as them (including reading private messages). 

This feature is commonpractice, and even included in some modern frameworks like PHP's Laravel. This is because it's often easier to jump into a user's account when they need help and figure out what's going wrong for them, than it is to try and get a reasonable explanation of the actual issue.

The problem is unaudited access, and controlling these features from day one, not later on. Some companies implement a fix for this that makes it only possible to access user accounts like this once the user approves it. Facebook, however, did not (I've also heard Twitter had features like this until recently, accessible by any employee). 

Not only had Facebook fired people for misusing these powers before, it even has a special set of rules about using impersonation tools and recently restricted access to only security personnel. Unfortunately, such tools were abused by that team as well as we learned this week.

In response to this, Facebook built out something called the "Eye of Sauron" (not joking) a few years ago which akerts  employees who had their accounts accessed by another employee to the activity, because they often did so to test new features. You and I, however, would not receive such an alert, raising a question: do we have the same right to know?

I had a realization yesterday that I thought such features were widespread knowledge but that most people do not understand that such tools exist. Impersonation doesn't require knowing your password, resetting an account or anything else, so you'd never know -- and at most startups, they're legitimately useful tools that help debug when something is wrong if controlled correctly.

There's something of a disconnect between what users understand about what technology is able to do and the reality, but also the expectations we have of the companies that build the platforms we use everyday. 

Impersonation tooling is something that won't go away, but should be managed from day one, not once someone gets caught abusing it.


Tab Dump

All we want to do is watch each other play games
A great piece from NYT about the rise of the free-to-play game Fortnite, and what it's doing to the e-sports industry almost overnight.

NPR and other shows acquire Pocket Casts app
I hope to write about this separately soon, but Pocket Casts,  the most popular Android podcast app, has been acquired by a group of podcast producers. Not only is this fantastic news, because we need competition against Apple in this space, it means the future of the industry is in the hands of those creating for it.

Microsoft sees huge Windows 10 adoption... from fear