That Post-Academia Thing I Needed to Write.

Two and a half-years ago I jumped off the AI bandwagon, left a permanent position in academia to joined “the industry,” namely Zalando, a big European fashion ecommerce retailer.

On the other hand, ever since I left, AI really did explode, with NIPS 2017 selling out in  15 days, and absurdities happening like Intel AI booking a mainstream rapper for their NIPS party.

Already, people are joking that attending the poster session is the new cool:

At the same time, my citations have reached an all time high since I left academia according to Google Scholar.

Left At The Wrong Time?

So looking back, do I regret having jumped academia, at the wrong time? Not at all, although my thinking around why I did what I did and why it was the right thing has changed over time.

Looking back, I think the main reason for me to leave academia was that I didn’t think I fit in. This might look odd if you look at my okayish track record or have heard me speak, but the bottom line was that I had huge troubles getting my stuff published. Maybe I did it wrong, maybe I gave up before I learned how to write a great paper, but somehow there was a disconnect between what I considered exciting and what I could get published.

I think I always liked a simple idea well executed, to make something that is small and fast. In many ways, what we tried to do with streamdrill fit the bill very well: Take approximate counting algorithms, add secondary indices and you get a counting and trend detection engine that can be used in many ways. (The company since folded for many other reasons, but the core of the algorithm can be found on github.) Taking the same approach in academia invariably lead to the same kind of reviews: “too simple”, “has been done before”, “all parts have been known.” I tried to get a better understanding of the mismatch writing about explore vs. build.

This was a bad place to be in, essentially beating myself up for not being able to publish those five+ papers per year like some of my colleagues. If you look at my publication record, you’ll see that most of my later output came from collaboration with other people. These were a lot of fun, but seeing how these people did awesome work and just wrote the papers I wouldn’t want to or couldn’t only cemented my believe that this wasn’t it for me.

This was part of the story, the other part was that I had begun to see deeper, more systemic issues with the academic environment. I’ve been struggling to put these thoughts into writing many times over the past two years, and always stopped myself, afraid of what my old colleagues would see, whether people would perceive me as whining, etc.

Conflicting Metrics and Peer Review

To be quite honest, my general sentiment after leaving academia was that a Ph.D. is nice as an experience, but you should think hard whether it’s worth hanging around as a PostDoc. As a Ph.D. student you can fully focus on one topic for an extended amount of time, and it teaches you many valuable skills like self-management, or presentation, mostly by necessity (material for another post I guess), but after that it’s about succeeding in an environment which is not without challenges.

For example, you will be mostly evaluated based on publications and funding raised, but half or even more of your time will be spent on teaching. Worse yet, you may actually enjoy teaching. It is fun and fulfilling to master a topic fully, or to help other human beings along in their lives. But ultimately, you should rather be hunting for that project, or work your peers to push that new topic your PostDocs are working on.

I also think that peer review is really really a very strange setup. I understand where it comes from, and that it is very hard to otherwise judge the merit of a piece of work if only a handful people on the planet know what is going on. On the other hand, it’s a bit as if you have to ask a competitor what he thinks of a new feature. It’s as if Netflix had to ask Blockbuster whether they thought streaming video on the Internet is an idea worth taking to the customer or not.

By Academia’s Nature, Change is Hard

There are many more things, and people are aware (see this summary page by Yann Lecun on new ideas for peer review for example). However, as it turns out, academia is, by its very nature, very hard to change.

Let me explain this is more detail. Positively speaking, academia is an open, global system where people can collaborate to solve problems. Through publication, each progress is shared with the community, and in principle, anyone who is in the system can pick up the problem and try to advance the state of the art.

I’m speaking of the ideal case, of course, in reality not everyone can “just join”, because you need to be socially integrated into a community otherwise you will have a hard time based on merit alone to get anyone to listen to you. And then there are many other issues around open access, access to education, and so on.

But even in the best case, the very nature of academia is that it is a global, open system, and as such, it is also very, very hard to change. In the industry, each company can make up its own rules and ideas about how they want to operate, but in academia, it only works because we have comparable standards around the planet.

The Academic Global System

The academic global system consists of researchers working at research institutes like universities or other research institutes who are being funded through government agencies, or the industry. It is pretty clear that research per se does not work in an economically self-sustaining fashion, but there is a general understanding that down the line, research is an important driver for innovation. In order to have as many people as potential working on advancing the state-of-the-art, research results are published openly, with peer review working as a filter to make sure only correct and relevant results are published.

It is interesting to see that the system is inherently decentralized, and that also the steering of what is considered worthwhile happens at many places: the funding agencies decide what they want to fund, although often after consultation by experts in the industry.  Professors and other senior researchers set the course in many ways, by advising their group’s researchers on what to work, by participating in conference program committee members, or as editors to journals.

With this simplified overview, I wanted to show just how entangled the whole system is. There is nothing bad about organizing things as they are. It has served us well over at least the last century, when academia as an industry really kicked off.

But since everything is entangled, it is very hard for any part of the system to redefine the rules. As a software engineer, you could say the system lacks clearly defined interfaces and separation of concern, or at least in a way which gives the different entities a lot of freedom of how to work. As a researcher you cannot simply say “from tomorrow I will prioritize teaching”, because you will still be measure by publication output (I do envy those personalities who can just stick to what they love and ignore everyone else, but I’m not that person). You cannot simply say “I’ll stop publishing my work at conferences and journals and only post it on my blog and talk about it on social media”, as your output (and your money) will still be measured in terms of citations to peer reviewed publications.

I am not saying that things cannot change, but only very slowly. Ironically, especially in AI, I think a lot of the changes came from companies like Google, Facebook, and Amazon (who have also recruited vast amounts of researchers who used to work in academia lately). They are not open in the sense that they are relying on huge amounts of data they are not publishing, and at least in the past, they’ve often only published results long after they have been put into production (as in the case of most of their Big Data related papers). These companies are big enough and self-sufficient enough to change the rules internally, and one could argue that based on scientific output, they aren’t doing a bad job compared to publicly funded academia.

In any case, the insight that academia is slow to change was a big one for me, and I think ultimately this was one of the reasons I have decided to leave, and I haven’t regretted it ever since. To be honest, in the beginning I was rather happy to be “out”, or “in”, depending on your perspective. I have since started to appreciate my time in academia more, and I’ll hope to find time to write about that soon.