How to listen along a user on Spotify

tl;dr “I dont want to read your article, I want to download the code” : Skip to this section of the article for the link to the Github repository.

Lately I started using a bit more thoroughly. For those who don’t know what is, it’s a webstite that allows you to “scrobble” (log) every song you listen to.
It’s cool because then you can have cool stats about your listening habits, but also discover new artists related to what you like (Spotify largely copied what was doing, with the artist/album/song radio and “daily mix” playlists).

The idea

I was browsing the website and looking at what my friends were listening to, when suddenly I thought that it would be cool to be able to listen to what another user is listening to, at the same time. It’s a cool way to discover new music while relying on someone’s tastes isntead of the Spotify algorithms, that unfortunately often brings you the same songs that you already know too well.

I started googling, but unfortunately didn’t find anything remotely close to what I was trying to do, except a very old website called “overhere” that apparently did the same thing, but is now long dead.

So I knew I had to do it myself.

The program

After a few moments researching the API and the Spotify API, I came to the conclusion it should be doable and reasonably easy to do.

I ended up doing it in Python, since it makes such little scripts easy and quick to code, and libraries were available to interact with the aforementionned APIs.

Basically, the program checks what a user (of your choice) is listening to on, and if he/she is listening to a song, it will search for it on Spotify, and play it, if available. At the end of the song, it does it again. Rinse and repeat.

And voilà, you are listening along a user, to the exact same songs, at the same time.

Where to get it ?

You can find the code on this Github repository :

Everything needed is explained in the file.

Have fun discovering new music 🙂

Python version of the API

In the previous articles How I reverse-engineered the API – Part 1 and Part 2 we saw how I proceeded to analyze and reverse engineer the Ask FM API so I could automate some tasks.

From there, I decided to implement a basic version of the API, with only a few functions I needed (a lot are missing, like liking an answer, following and unfollowing people, blocking people, etc.). After some time using this ugly code, some people asked me to share my work. I was reluctant to do so for multiple reasons :  First, I’m not sure about the law around reverse engineering, it’s always been a grey area to me. Also, I’m not sure the people who asked me for it would benefit from this code or these articles because they won’t understand anything and won’t know how to use it. And lastly, I didn’t want to share an easy “working as-is” executable, because it would allow script kiddies to spam on because of the absence of captchas when using this.

For these reasons (and for intellectual property matters), I voluntarily stripped off the API secret key from the Java code on the Github repository, otherwise might get angry at me for sharing such a thing. I might contact them and if they’re okay with it, I’ll add it back.

The code

Anyway, enough talk, here is the link to the github repository :

Hexalyse/pyAskFm – Basic python implementation of the API

Final word

As you can see, there are a lot of things that could be done to improve this code. If you’re interested in contributing, read the file on github, and feel free to send a Pull Request.

How I reverse-engineered the API – Part 2

In the second part, we’ll take a look at the requests we intercepted thanks to Fiddler, and see if implemented some sort of security (tokens, checksums, etc)

Understanding the request headers and parameters

Here is the request we’ll analyze :

POST /authorize HTTP/1.1
X-Api-Version: 0.8
X-Client-Type: android_3.8.1
Accept: application/json; charset=utf-8
X-Access-Token: .4WspnFnDpwQNevsbXIEExPDgJZDM
Accept-Encoding: identity
Authorization: HMAC a9f98b69f649e0c96240cc6e36980da96f308cea
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
User-Agent: Dalvik/2.1.0 (Linux; U; Android 6.0.1; GT-N7100 Build/MOB30R)
Connection: Keep-Alive
Content-Length: 186

json {"did":"84a2b70bfae4ae65","guid":"84a2b70bfae4ae65","pass":"password123lol","rt":"4","ts":"1471967146","uid":"JohnDoe"}

As we can see , there are lots of interesting headers here. I’ll not make you wait and explain directly what I deducted from my tests :

  • X-Client-Type: This header indicates the version number of the application used. Let’s just use the same as the one we captured.
  • X-Api-Version: This is the… API version, you guessed it. It is important because the parameters we need to send to the API change with each version and they are NOT retro-compatible ! This must be a nightmare to maintain for the developers.
  • X-Access-Token: Before you make any requests, you need to make a GET request to the address, with a few get parameters (the device id, the current timestamp and a third parameter we’ll explain later). Then you use this token in your authorization POST request. The token you get back from this request will be the token that identify you and keeps trace of your “session”.
  • Authorization: Aaaaah, this header is the one who gave me a headache. We can see it sends an HMAC, which is just a glorified checksum, used to verify the request. The application basically generates this checksum (HMAC) from the URL  and the parameters of the request. Then, when the server receive your request, it will compute the HMAC and compare it to the one you sent. If it’s the same, the request is valid. Otherwise, the server will send you a nasty “Invalid request” error message.
  • Other headers are simple and classical HTTP headers

So… the three first headers are simple to reproduce. But this HMAC Authorization header is a barrier : we need to find how this HMAC is generated. And for this, we’ll have to dwelve into the AskFM source code. We’ll look at it later. For now, let’s look at the request parameters.

Every POST request has only one parameter, called “json”, containing an URL-encoded version of all the parameters in JSON. What is funny, is that I discovered you must format this JSON parameter exactly like the application does, or else the request will be considered invalid (I’m really wondering how they parse it). The correct formatting is : the keys must be alphabetically sorted, and there must be no spaces or new lines. Everything must be on a single line. OK, why not…

Also, besides the parameters specific to each request, there are four parameters that will always be present :

  • “did”: The Android device ID, in hexadecimal
  • “guid”: The same ID (can it be different ? I’m not an Android expert)
  • “rt”: I discovered this is a counter that is incremented at each request made. Maybe they use it for rate analytics ? Or for an ugly rate limiting ?
  • “ts”: This is just the current timestamp (in seconds)

Decompiling the apk to find the HMAC generation routine

So now we only need one last thing : to know how this HMAC is generated. For this we’ll need to decompile the apk. You can do it with tools like APKTool, but there are websites like that allows you to send an APK and download the decompiled code. I used a software with a GUI but I don’t remember its name.

Anyway, now we have the decompiled “human readable”, somewhat-Java-ish code, we just need to go through the different packages name and find the ones that sounds useful. I did it months ago so I don’t exactly remember how I proceeded and how I ended up finding the code that was interesting. All I know is that the code we’re interested in is the package and the class is called Signature.

Here is the decompiled Java code :

As we can see, the code uses a private key (the HMAC secret key), which is stored somewhere else in the source code. I won’t tell you where since… I don’t remember. But it’s pretty easy to find.

UPDATE : I’ve been informed by a reader that the developers changed the way the HMAC secret key is generated in the newer versions of the app. It is no longer stored as plain-text in the code. It is instead generated via another Java class. If you want it, you can extract this class, import it in your own Android app project, compile it, run it, and you’ll have the key without spending too much time reversing the code.

From there, we can just replace this function call with the secret key and we have a Java class we can use to generate a valid HMAC 😀

Conclusion and source code

I will stop this article here, since all that is left is clean up the Java code or re-implement it in any language we want. Then we just have to spend a LOT of time analyzing the requests to each of the API endpoints we’re interested in, and implement it.

I chose to make a very basic implementation of the API with only the functions I needed for a few projects I had which needed some sort of automation (posting answer, sending questions, etc.), but I was too lazy to port the Java code to Python so I kept it in Java.

If you want to get the source code, you can get it on Github : Hexalyse/pyAskFm – Basic python implementation of the API

How I reverse-engineered the API – Part 1

In this first post, we’ll see how I managed to reverse engineer the API. In a subsequent post, I might give the (ugly) code I came up with, which implements some of the API features I needed for various bots and scripts.

If you don’t already know, it’s a social network where people can create profiles and can send each other questions… and answer it, obviously.

Unfortunately, their website has nasty limitations (Google reCAPTCHA) that prevented me to do what I wanted. But I noticed their application never ask for any captchas. So I guessed it was using a backend API, and if I can use it, I can code bots more easily to automate various tasks (like sending “Group Questions”, a widespread practice amongst users). I searched if their API was documented and if they offered developers access to it : they didn’t. I searched the web for people who would have implemented it in any language, and found nothing.

So it was my job to reverse engineer and implement (a basic version of) their API in Python. I will retrace my journey, step by step, of how I finally got to the point of intercepting API calls made from the Android app, to the servers.

Step 1 : Using Fiddler to sniff API calls made from the app

The first obvious step was to install an HTTP-proxy application to be able to intercept the HTTP requests and replies between the app and their servers. I decided to use Fiddler because it’s free and supports HTTPS traffic recording (via a MitM attack using a crafted SSL certificate, thus allowing to decode HTTPS data, then re-encrypt it and sending it to the other party). So I installed Fiddler, activated the HTTPS decoding option, and installed the root certificate they provide you on my Android phone. You need to do this step, or otherwise, the SSL certificate crafted by Fiddler won’t be accepted by your phone, since it won’t have the root certificate to validate the chain. Then you just have to configure your WiFi connection to use a proxy, and use the correct IP address and port to point to the computer where Fiddler is running.

Unfortunately, even after doing all this, the application didn’t seem to work when I was proxying the network connection. It kept telling me “No internet connection”. But the connection was good, because everything worked fine in a browser and I could even access the HTTPS mobile website in Firefox and see every requests made, decrypted in Fiddler.

So there was another problem. And suddenly it stroke me : they must be using some sort of certificate pinning. What this mean is, even if the certificate given by Fiddler to the application is valid, the application is waiting for a very specific certificate. Every others would be rejected. This is a very good practice, because it prevents such Man-In-The-Middle attack from potentials attackers who could then steal your credentials. And it also prevents people from reversing their API.

But I wasn’t done already.

Step 2 : Installing Xposed framework to disable certificate pinning

Because there is a way to bypass certificate pinning. If you have the Xposed framework installed on your phone (requires root access), there are plugins for it which sole purpose is to bypass certificate pinning. There are multiple of them, actually :

Yaaay \o/ We just need to install one of this, and we’ll finally be able to intercept traffic from the app !

Step 3 : Intercepting API calls, then trying to figure how it works

Let’s try sniffing traffic again : It works !

Here is an example of a request (the one sent when we login in the app) :

POST /authorize HTTP/1.1
X-Api-Version: 0.8
X-Client-Type: android_3.8.1
Accept: application/json; charset=utf-8
X-Access-Token: .4WspnFnDpwQNevsbXIEExPDgJZDM
Accept-Encoding: identity
Authorization: HMAC a9f98b69f649e0c96240cc6e36980da96f308cea
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
User-Agent: Dalvik/2.1.0 (Linux; U; Android 6.0.1; GT-N7100 Build/MOB30R)
Connection: Keep-Alive
Content-Length: 186

json {"did":"84a2b70bfae4ae65","guid":"84a2b70bfae4ae65","pass":"password123lol","rt":"4","ts":"1471967146","uid":"JohnDoe"}

So now we’ve captured traffic going between the AskFM app and their servers, we need to analyze it.

We’ll cover this in the part 2 of this article.