🌍 Where Are DEV Users Coming From?

Boris Jamot ✊ / - Nov 30 '18 - - Dev Community

As you may already know, dev.to has an API to fetch articles and users.

Let's give it a try with my favorite HTTP client, httpie:

http https://dev.to/api/users/1
Enter fullscreen mode Exit fullscreen mode
{
    "github_username": "benhalpern",
    "id": 1,
    "joined_at": "Dec 27, 2015",
    "location": "Brooklyn, NY",
    "name": "Ben Halpern",
    "profile_image": "https://res.cloudinary.com/practicaldev/image/fetch/s--DOU9qJSH--/c_fill,f_auto,fl_progressive,h_320,q_auto,w_320/https://thepracticaldev.s3.amazonaws.com/uploads/user/profile_image/1/f451a206-11c8-4e3d-8936-143d0a7e65bb.png",
    "summary": "A Canadian software developer who thinks he’s funny.",
    "twitter_username": "bendhalpern",
    "type_of": "user",
    "username": "ben",
    "website_url": "http://benhalpern.com"
}
Enter fullscreen mode Exit fullscreen mode

What I first noticed is that id seems to be auto-incremented.
After some tries, I quickly found out the total number of users on dev.to: around 119000.
It gives me the idea to scrape all the users public data and try to play with it.

Let's try with this command:

for i in {1..119000}; do http https://dev.to/api/users/$i >> users.dev.to.json && echo "," >> users.dev.to.json && echo "https://dev.to/api/users/$i"; done
Enter fullscreen mode Exit fullscreen mode

After a few hours running in the background, I finally got all the users in my JSON file.

I ended up by importing the JSON file in a mongodb collection using NoSQLBooster.

There isn't many information in the users' profile, but two of them worth having a look at them: joined_at and location. It could be very interesting to have a diagram showing the progression of registrations through time. Maybe I'll do it some day, but for now, let's compute some statistics on users' location.

I used two queries to compute the statistics. This one helped me to get the big trends in people's location:

db.getCollection("users.dev.to").aggregate(
    {"$group": {_id: "$location", count:{$sum:1}}}
).sort("-count")
Enter fullscreen mode Exit fullscreen mode

Then, I use another query to get the count of people per location:

db.getCollection("users.dev.to").find({"location": /\bdublin\b/i})
Enter fullscreen mode Exit fullscreen mode

The conclusions are the following:

Most of the users (90%) don't fill their location.
16.3% of the users who filled their location are from USA.
5.8% are from India.
4.9% are from UK.
4% are from Japan.
3.8% are from Germany.

Let's have a look at the full statistics below:

# country region or city #region or city #country %
1 USA 1850 16.32%
Incl New York 205
Incl CA 194
Incl NY 147
Incl San Francisco 137
Incl TX 119
Incl Seattle 117
Incl Los Angeles 107
Incl Chicago 101
Incl Washington 81
Incl FL 78
Incl PA 68
Incl MA 67
Incl AZ 38
Incl MI 34
Incl OH 30
2 India 660 5.82%
Incl Bangalore 107
Incl Delhi 103
3 UK 551 4.86%
Incl London 258
Incl England 35
Incl Scotland 33
Incl Wales 7
4 Japan 451 3.98%
Incl Tokyo 242
5 Germany 428 3.77%
Incl Berlin 128
6 Brazil 301 2.65%
7 France 294 2.59%
Incl Paris 98
8 Canada 257 2.27%
Incl Toronto 107
9 Nigeria 187 1.65%
Incl Lagos 97
10 Netherlands 154 1.36%
Incl Amsterdam 62
11 Spain 110 0.97%
12 Argentina 109 0.96%
13 Italy 102 0.90%
14 Indonesia 92 0.81%
15 Russia 89 0.78%
16 Mexico 86 0.76%
17 Philippines 85 0.75%
18 Poland 81 0.71%
19 Ukraine 75 0.66%
20 Portugal 63 0.56%
21 South Africa 57 0.50%
22 Pakistan 56 0.49%
23 Turkey 55 0.49%
24 Belgium 53 0.47%
25 Egypt 52 0.46%
26 Bangladesh 50 0.44%
27 Colombia 46 0.41%
28 Romania 45 0.40%
29 Kenya 45 0.40%
30 Switzerland 45 0.40%
31 Austria 43 0.38%
32 China 40 0.35%
33 Norway 38 0.34%
34 Ireland 33 0.29%
Total main locations (listed above) 6683 58.94%
Other locations (unlisted) 4656 41.06%
Total locations 11339 100.00%
<null> 90260 80.80%
<empty> 10108 9.05%
Total 111707 100%

Note: these numbers are one month old, but they give a good overview of the trends.

Thanks for reading!

. . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player