Lessons learnt building data pipelines — 1

It has been a few years since I embarked on what was then trendily called “data engineering”. That title — and the associated work — was an outgrowth of a tendency for computing systems to accumulate vastly more data than used to be the case before, and more importantly, try to extract business value by sifting through it. Nowadays […]

Influential books I read in the last 2 years

Either I’ve become way better at choosing which books to read, or am able to get more out of reading each individual book than I used to. In any case, a surprisingly large number of books have managed to contribute long lasting ideas to my mental repertoire over the last couple of years, compared to […]

NIPS 2017 is exciting even if you don’t work in AI

I found out about the just concluded NIPS 2017 conference through a mention on my Twitter timeline. I’d heard this was a premier venue for presenting research in artificial intelligence, so I clicked through to the #NIPS2017 hashtag. The superlatives being thrown around to describe this year’s edition of the event roused me enough to […]

An unbeatable perk of the Bay Area: its meetups and conferences

One unbeatable perk of living in the San Francisco Bay Area is the sheer number and variety of technical events you get to attend and the novelty of the ideas you get exposed to at such events. Whether it is a new framework or programming language, or an entirely new way of thinking about problems, […]

What if your text editor spoke Kafka?

If you work in enterprise software long enough, you may end up only looking at software that satisfies complex constraints: how often does it fail, what kind of security features does it have, does it work well with these other systems of ours, an so on. Not to forget the most common of them all: […]

Rocking it with Docker and Metaparticle

My friend Asad messaged me from the ongoing KubeCon 2017 that a keynote slide from Brendan Burns (one of the creators of Kubernetes) advocated that configuring your app for the cloud native world ought to become significantly simpler. That appealed to me instantly, since I too was wishing a few days ago that Docker and […]

Notes from Hadoop Summit 2016

This was my first time attending a Hadoop Summit (thanks to my employer Yahoo for sponsoring!). I had a very positive impression of the event. Talks covered a range of topics and speakers were available for questions and hallway discussions. There were many vendors showcasing their wares as well. I mostly attended talks related to […]