The Discovery page, the new start page in Spotify, is finally out to a fairly significant percentage of all users. Really happy since we have worked on it for the past six months. Here's a screen shot:
From: Erik Bernhardsson
I was browsing around on the Internet and the physics geek in me started reading about Fermat's principle. And suddenly something came back to me that I've been trying to suppress for many years – how I never understood why there's anything fundamental about the principal of least time.
From: Erik Bernhardsson
Just promoting Spotify stuff here: check out the Snakebite repo on Github, written by Wouter de Bie. It's a super fast tool to access HDFS over CLI/Python, by accessing the namenode directly over sockets/protobuf. Spotify's developer blog features a nice blog post outlining what it's useful for.
From: Erik Bernhardsson
We've recently started evaluating the New Relic monitoring service at World Singles and when you use their Java agent with your web application container, you can get a lot of information about what's going on inside your application (JVM activity, database activity, external HTTP calls, web transac...
The simple way to get featured on big data blog these days seem to be Build something that does 1 thing super well but nothing else Benchmark it against Hadoop Publish stats showing that it's 100x faster than Hadoop $$$ Spark claims their 100x faster than Hadoop and there's a lot of stats showing ...
From: Erik Bernhardsson
I picked up an issue of Foreign Affairs while flying back to NYC from SFO. It features this long interview with U.S. General Stanley McChrystal and I thought it was pretty interesting how striking some of the similarities are between fighting in a war and developing software.
From: Erik Bernhardsson
Annoy is a simple package to find approximate nearest neighbors (ANN) that I just put on Github. I'm not trying to compete with existing packages, but Annoy has a couple of features that makes it pretty useful.
From: Erik Bernhardsson
Elias Freider just talked about Luigi at PyData 2013: The presentation above is much better than one I put together a few weeks ago. In case anyone is interested I'll include it too:
From: Erik Bernhardsson
I recently came across this paper describing how they do ML at Twitter. TL;DR Their approach is pretty interesting. Everything is a Pig workflow and then they do everything as UDF's. This approach seems pretty interesting.
From: Erik Bernhardsson
This article from today in Mashable describes some of the fun stuff I get to work with: Erik Bernhardsson is technical lead at Spotify, where he helped to build a music recommendation system based on large-scale machine learning algorithms, mainly matrix factorization of big matrices using Hadoop.
From: Erik Bernhardsson
Slides from the talk. Slightly edited because (a) some of the slides make little sense taken out of context (b) Slideshare seem to have problem converting some of the stuff. Collaborative filtering at Spotify from Erik Bernhardsson
From: Erik Bernhardsson
From the NYC Machine Learning talk I had last week: Haven't looked at it yet except briefly. Unfortunately the quality isn't the best.
From: Erik Bernhardsson
The Economist just published an article called The best, the worst and the ugly.
From: Erik Bernhardsson
This was posted on the Twitter Engineering blog a few days ago: Dimension Independent Similarity Computation (DISCO) I just glanced at the paper, and there's some cool stuff going on from a theoretical perspective.
From: Erik Bernhardsson
Not sure how I managed to miss this, but I'm watching this Tumblr presentation and they talk about their projects named after Arrested Development topics: Gob, Parmesan, Buster, Jetpants, Oscar, George and Motherboy. Still, the best software project name is probably still Apple's BHA.
From: Erik Bernhardsson
Something that pops up pretty frequently is to implement time decay, especially where you have recursive chains of jobs.
From: Erik Bernhardsson
I'm shamelessly promoting my first major open source project.
From: Erik Bernhardsson
If you may still be using Dreamweaver or HomeSite+ (or its older brother, HomeSite, or its older uncle, ColdFusion Studio), you may find that the tag insight, tag help, tag completion, and other features are not recognizing newer ColdFusion tags and functions, if perhaps you have updated to later ve...
I just blogged about how the hidden gem "enable monitoring server" option in CF 9.
