A week's worth of ideas, in your inbox. No spam ever.


Who Is Better

‘What do I want to do with my life?’

This question trips a lot of people up. It still messes with me, from time to time.

But, if you think about it, it’s the wrong question.

School programmed us to think about the what. Our schedule was organised so that we gained skills in certain topics and subjects, just the right balance of each.

University only takes this idea further. You choose one area of knowledge and study that for three, four, or seven years.

When you enter the working world, you try to find a job that matches your what. You work anywhere that needs people with your particular skills.

And you end up dissatisfied and confused. You try to search for another what, but the pattern keeps repeating.

The problem is the question; we focus on the wrong W.

Some people don’t have a burning passion for their work, it’s just a job they do to pay the bills, and yet they still enjoy most of their days.

Why? The people.

They’ve lucked in to an environment filled with people they enjoy being around, people they enjoy serving.

Imagine if we were intentional about who we wanted to be around.

Imagine if we chose what to learn based on how much value it would bring to the people we care about the most.

Imagine how resilient we’d be to technological and political change if we focussed on the who instead of the what.

Our education would be different, our work would be different and our lives would be different.

All because we changed ‘What do I want to do with my life?’ into ‘Who do I want to serve with my life?’


Temporary Answers

Who are you?

We all tend to give temporary answers to this question.

‘I live in X and I work at Y’.

And that’s fine. Those details are helpful signposts and they make good introductions.

But our internal definitions of ourselves often follow the same pattern.

We focus on the temporary things and ignore the permanent ones.

And we feel adrift and off-course when the temporary things change.

Next time you ask yourself who you are (it happens more often than you’d think), give permanent answers instead.

‘I was born in A, my mum’s name is B, my kid’s name is C. I do D every day.’

These kind of details are difficult or downright impossible to change.

Yes, this means your recent promotion or raise won’t be a part of the story you tell yourself.

But it also means you’ll feel less lost when the world shifts around you.

That’s a worthwhile trade, isn’t it?


The Long Game

What’s a blog for?

It’s an important question.

Some people will say that a blog exists to harness the power of SEO, to get the message out, to build an audience.

But this is short term thinking. (Yes, even though it takes years to monetise a blog.)

If you write a blog under your actual name, like I do, then there’s a chance that it will outlive you.

There’s a chance that the ideas you put out there will land in someone else’s mind (or a bunch of someones) and percolate in the culture.

This week I tried to game the system. I said timely things that I hoped would catch people’s attention, all in the name of getting more eyeballs.

I said nothing new, I just contributed in my own small way to the worst parts of the internet: rancour, outrage, temporariness.

I regret it now, of course.

Those few days’ entries (which are just below, if you want to scroll) nearly made me quit my daily routine. I nearly abandoned the habit I’ve developed to become a better thinker and a better writer.

Luckily I remembered what a blog is for just in time.

A personal blog is the place to play the long game.

And the long game isn’t tied to whatever you are or whatever you’re working on right now. Like gut reactions, these things are just temporary.

Filling your corner of the web with temporary things is a waste of your space and a waste of your voice.

Do the tactical marketing if you have to, just be sure to put it somewhere else, somewhere less special, and don’t confuse it with the daily work of becoming better.

A blog is a place to improve yourself and the world, one thought at a time.


What Only You Can Build

There aren’t enough software businesses.

Notice how I didn’t say companies. I’m not talking about startups with venture-backing and hockey stick growth. I’m talking about individuals or small groups making helpful little tools and apps.

There’s so much in the culture that tells us software has to swallow the world. But it’s not true.

Building a software business doesn’t require tons of money or time. If you can make a few pieces of software, and get a hundred or so users for each, you can make a nice, completely independent, living.

Of course every developer has these goals, they’re always working on the next killer app. But I’ve seen these developers and engineers lose motivation time and time again.

Another company pops up doing their exact idea. The market moves and their funding falls through. They buy the garbage about being first, about being the quickest, about being the biggest.

We need different pieces of software for the same reason that we need different books about the same topic.

Software expresses opinions about how the world is and should be, and no two people have the same exact take.

We need more opinionated software.

If you’re writing a non-fiction book, you have to show in your proposal to publishers that other people have written books about your topic.

It’s how you show the publishers that people care about your subject matter. A proposal without this section won’t get bought.

But a book that’s lacking a unique angle won’t get bought either.

In software, people either build the same things as everyone else (that means they include every feature, every integration) or they get scared away at the first hint of competition.

Next time you have an idea for a web or mobile app, don’t shrink away from the task just because you might be outdone. There’s plenty of customers to go around.

Find your own angle, your own approach. Build what only you can build.


Explainability Is Not the End

A preprint uploaded to the Arxiv yesterday, courtesy of Denmark Technical University and UC Berkeley, introduces a novel method for penalising bias in deep neural networks.

Explainability has been a recent focus of many deep learning research teams and much progress has been made in this area (including invertible networks, for example) but there has been little in the way of techniques for using the insights gained to engineer better, more robust and less biased networks.

The preprint’s authors suggest a method that they call Contextual Decomposition Explanation Penalisation (CDEP) which allows data scientists to remove spurious correlations between features at training time.

The method involves adding explanation error terms to the model’s loss function. These error terms calculate the L1 norm between the beta terms from the Contextual Decomposition (CD) algorithm (which is used to calculate feature importance) and a target term (usually zero) for those features and feature interactions scientists want to remove. In this way, the model minimises the loss when it reaches peak accuracy and when its explainability metrics match the scientist’s assumptions.

As use of the CD algorithm only adds a small constant term to the training time of a DNN, the authors of the paper have found a much more computationally efficient way of explaining and updating these models than the current state of the art.

Several datasets are tested in the preprint (including the Stanford Sentiment Treebank dataset, the International Skin Imaging Collaboration dataset and ColorMNIST) and the methodology is shown to improve accuracy across the board.

However, this seeming breakthrough in neural network explainability and bias-removal does have one caveat.

Neural networks have become popular (and successful) precisely because they remove the need for feature engineering. This means that engineers and scientists without domain expertise can utilise them to learn arbitrary decision functions. It is not necessary, for example, to know anything about edge detection or image segmentation to train an object detection model.

Systems like CDEP, however, do require their user to have some understanding of not only the data (which could run to many millions of observations) but also the domain from which it’s drawn. Both pieces are needed to correctly calibrate a model and remove its spurious correlations.

As more people develop deep learning skills (with the aid of online and distance learning) and as more money is invested into startups using deep learning in every imaginable vertical, it is likely that we’ll see a shortage of the kind of knowledge needed to counteract the effects of model bias.

The models whose latent bias provoke outrage are rarely manifesting the bias of their engineers. Rather they are the result of the engineer’s ignorance toward the bias in their datasets.

CDEP doesn’t resolve this problem, but it as least allows those with awareness to efficiently and effectively remove bias from their models.

Before we can use deep learning as the magic bullet it claims to be, we need to have a culture shift around who should be engineering and programming these models. Techniques like CDEP emphasise the importance of specialist domain knowledge in a world where the latest and greatest techniques are available to everyone with an AWS account.


Something’s Wrong with TV

A paper accepted to the upcoming ACM SIGSAC Conference on Computer and Communications Security shows precisely how Over-the-Top streaming devices (such as Roku and Amazon’s Fire TV) use and misuse data.

The channels on these devices are nothing more than applications that send and receive requests over the usual internet protocols. This means that they can (and do) employ trackers for the purposes of advertising.

Anyone even fleetingly familiar with the state of online tracking will know just how much data can be transmitted to these advertising networks. That the same thing happens on television, however, might come as a surprise.

These devices and their nearly limitless selection of channels (which includes games and other applications) have been found by researchers at Princeton and the University of Chicago to transmit data such as movie / video title, movie / video category, IP address, and device ID to as many as 64 distinct tracking services.

Less surprising is the list of companies collecting this data. Over 99% of the top 1000 Roku channels send data to Google (via doubleclick.net) and channels on the Fire TV device send data to Amazon, Google and Facebook.

While the findings of this paper raise some privacy concerns, it has become commonplace to accept tracking and advertisement targeting in exchange for free services. So to understand the full impact television has on individual privacy, we also have to consider paid television.

An article published today on Wired UK details Sky’s recent agreement to allow Virgin Media and Channel 4 to distribute targeted advertisements from Sky’s AdSmart system.

The AdSmart system uses a host of explicit and implicit data to segment Sky customers by demographic and psychographic attributes.

By analysing your viewing patterns, Sky claim that they are able to determine your income level, the number and ages of people in your household, whether you own a pet, whether you’re an early adopter of new technologies, and whether you’re expecting a baby. And this is just the implicit attributes.

They also collect data from reward programmes such as the Boots card (which enables cosmetics companies to appeal to people who buy from their competitors) and Experian (which provides proprietary ‘Mosaic’ segment data based on 30 years of credit reporting).

This is a terrifying violation of privacy and only becomes more so when you realise that Sky (the originators of AdSmart) and Virgin Media (AdSmart’s newest customer) have a combined 81% of the UK’s paid TV market share (2015 estimate, source).

For many people this system will feel like a gross misuse of their private data. Not only because of the types and volume of data collected but because they have to pay to watch Sky and Virgin Media.

By collecting our data and using it to drive advertisement revenue, these companies are profiting from our custom twice over.

These two recent reports show that the most popular television delivery mechanisms are morphing into a dangerous hybrid that costs viewers their money and their data.

This new middle ground is a symptom of half-hearted attempts by both traditional and new TV to explore alternative business models. The problem is that it’s only the viewers who are suffering. And as long as that’s the case, there will be something very wrong with TV.


Pandora’s Box

‘90% of all the data in the world today has been created in the past two years.’ It wasn’t too long ago that this statistic from IBM made its way into every conference speech and investor presentation.

And now an investigation published this weekend in the New York Times has shown that this industry cliche has a dark side – child abuse imagery is following the same curve.

The purpose of the Times’ report was to highlight how ill-equipped the police and government are to deal with the exponential growth in the number of abuse cases. It describes, in detail, how the allocated budgets fall woefully short of what would be needed to meet the crisis head on and how money is, in many cases, siphoned off by other departments.

The report also hints at the technological causes of the phenomenon. The impact of camera phones, cloud storage and social media are all discussed in the piece. But what exactly is it about these technologies that has encouraged the proliferation of images and videos depicting child abuse?

Images and videos like those discussed in the report are singularly damaging and uniquely horrendous. But perhaps, if we’re going to explain the rapid increase in this material’s availability, it’s necessary to examine why every category of content is expanding on an internet built on the premise of sharing.

My earliest experience of the web, like many of my peers’, was using it to download poor quality MP3s. Applications like Napster had us (and the music industry) convinced that nobody would ever have to buy music again. The sharing network was a revelation.

It was possible, at that time, to be optimistic about the future of child abuse. It seemed reasonable to assume that the wider availability of material from older (though no less despicable) crimes would satiate their consumers’ needs and lead, eventually, to a decline in the total number of new cases. But the inverse has happened.

As the total amount of material has grown, so has the rate of production. And this is not unique to images and videos of child abuse.

Everything from illicit to perfectly legal sexual content, from niche electronic music to snuff films, from harmless prank videos to extremist political rhetoric has undergone the same growth. YouTube, to take just one example, adds 500 hours of footage to its library every single minute.

So what explains this growth? And how has a network built on sharing ended up prizing creation?

When the turn of the century’s Dotcom boom-and-bust put an end to many of the sharing platform businesses a second wave of tech companies erupted in their place.

These companies emphasised connection and placed individual users at the centre. They existed to share what you had to say, what you thought, what you made. Traditional media conglomerates no longer controlled the airways.

After the second wave companies fumbled around trying not to repeat the mistakes of the economics-less Web 1.0, they settled on an advertising based model and set about optimising for usage time.

Eventually, these designers and analysts iterated their way to addictive products that engendered social anxiety and depression. Suddenly, it seemed as if the whole world was obsessed with their friend count. Social media was born.

The primary benefit of this model (compared to that of the traditional media) is also it’s downfall – its topic-blindness. No matter how niche or abnormal your tastes, social media sites encourage you, perhaps even manipulate you, to connect and share.

The joys of chasing ever higher follower counts or ‘growing your network’ are available to everyone with an internet connection, no matter if you’re a YouTube prankster or a dark web pedophile.

The issue of child abuse is, of course, not solely due to social media. But social media has certainly played its part. And if we ignore this overarching trend towards exponential growth, we’re left with theories that fall short of a full explanation.

Anonymity, for example, is often blamed for abusive online behaviour. But anonymity fails to explain why so many (~66%) of the reported cases occur on Facebook messenger, a platform where the vast majority of users identify themselves truthfully.

The world’s largest technology companies are slowly realising the immensity of the problem they’re facing. They’ve allowed child abusers to run rampant on their platforms simply because they’ve allowed everyone to run rampant, all in the name of network effects and exponential user growth. They’ve opened Pandora’s box and it may well take the end of social media to close it.


YouTube’s False Equivalence

A paper accepted to this year’s RecSys conference series hints at a potential change in the way YouTube’s algorithms suggest videos to users.

In the paper’s introduction, the authors present a (perhaps purposefully) weak case for why an algorithmic change is necessary, suggesting that the current model contains an ‘implicit bias’ which causes a ‘feedback loop effect’.

Karen Hao from the MIT Technology Review has suggested that this paper may be Google’s attempt to rectify the widely reported problems with a recommendation engine that tends to suggest videos of a marginal political and / or conspiratorial nature.

The proposed change would take into consideration the position of a recommended video in the sidebar, and would discount the influence of videos which were positioned near the top, to reduce the feedback loop effect.

The authors of the paper acknowledge the difficulties of designing and evaluating systems which reduce implicit bias and openly state that asking users for explicit feedback on their recommendation engine ‘can hardly scale up’.

But can this simple change really be expected to collapse alt-right echo chambers? Or is something more fundamental amiss with YouTube’s system?

The deep learning based recommendation engine that was discussed in a 2016 paper (also presented at RecSys) shows that the YouTube algorithm trains over 1 billion parameters (on data which includes details about both the video being watched and the user doing the watching.)

Working out why certain videos are recommended, then, is an impossible task. This leaves Google with few options except trying almost random algorithm changes to see which ones positively impact the behaviour of their users.

It is important to remember, however, how positive behaviour changes are assessed. In Google’s eyes, a recommendation system is doing its job if it keeps the viewer engaged. And perhaps it’s this false equivalence that makes ousting the echo chambers so difficult.

In the absence of explicit feedback, the paper’s authors have chosen to measure user ‘satisfaction’ via Like and comment counts, forgetting that these metrics can be gamed by content creators (by simply asking for them, for example).

Relying on metrics around behaviours that are subject to variegate social pressures, viewer suggestibility, and creator manipulation will never lead to algorithms which represent the true intentions and desires of users.

As long as YouTube equates viewing time with algorithmic success (and does so inside of a black box), they’ll rightly suffer criticism for creating environments which reward radical attitudes.


Who Are Likes For?

Earlier this week, Facebook announced that it would trial making metrics, such as Likes and video views, private to other users. The trial will take place in Australia and will be the company’s first ever experiment in hiding the metric that it introduced a decade ago.

Like counts and similar social metrics have received criticism since their inception for creating competitive and extreme environments on social media sites. Many experts believe that their use pressurises users and lends weight to offensive (organic or paid) content.

In May, Instagram (a Facebook-owned photo sharing application) started testing the same thing for users resident in Canada.

When representatives from Facebook discussed the changes with journalists, they emphasised that the trial’s results would be assessed on the basis of user happiness and continued engagement, but it’s important to remember who the Like button is for.

On websites without Facebook’s extensive reach, the Like button serves as a way to measure the value of a piece of user submitted content and to rank and filter these contributions. This is the use which many experts object to, stating that it has caused an increase in anxiety and depression amongst younger users, in particular.

On Facebook, however, the Like button is additionally implemented as an advertising network (via the social sharing API) and allows Facebook to gather enormous amounts of information from those who use the feature on Facebook and third-party websites.

It has been shown that statistical models which use Like data as their only inputs can achieve high accuracy for personal attributes such as sexual orientation, race and intelligence. It goes without saying, then, that Like metrics are a central pillar of Facebook’s efforts as an advertising marketplace, allowing them to more narrowly identify users who would be receptive to their customers’ messages.

While Facebook is publicly claiming that this trial change is meant to increase the happiness of its users, it comes many years after calls for removing Likes began. Furthermore, Facebook emphasised that it would only consider extending the trial if users kept using the Like feature, which suggests that gathering these metrics may be more important to the company than any level of user happiness.

It’s also possible that this change, which will remove social pressure on users to Like what their friends, followers and connections Like, could benefit the signal to noise ratio and further increase Facebook’s targeting powers.

The results from the Instagram trial haven’t been made public yet, but I think it’s safe to assume that they have been fiscally positive given this decision from Facebook.

The trial has been lauded by those who have been lobbying to make social networks less anxiety-inducing, but is merely hiding the metric enough?

The Like may be a bonding and feedback mechanism but it’s also the crux of one of the world’s most powerful and wide-reaching advertising systems. If we’re going to build a less manipulative, surveillance-free internet, it may be necessary to throw the baby out with the bathwater.


Deep Learning is Not the End

Deep learning is often considered the pinnacle of a machine learning or data science education.

It’s the thing you learn when you’ve mastered everything else. And it holds this position for good reason.

The concepts are trickier to grasp, the frameworks are harder to master, and it’s more experimental than the statistical models we’ve used for decades.

But far too many people stop once they get there.

Deep learning is a fragile, impenetrable, energy-and-data-guzzling methodology. It is not the best we can do.

Everyone who uses deep learning owes it to the research community, and the planet, to treat the tools we’ve been given as a starting point and not a destination.

We desperately need new ways to make artificial intelligence more efficient, robust and explainable.

And these new methods will only surface once we recognise the shortcomings of what already exists.

So treat deep learning as it should be treated, as an inefficient beginning and not the end.