DIY clickstream framework – 1. Intro

We are going to create a clickstream framework from scratch. In all my years in analytics, I stumbled on clickstream data quite recently. What I saw honestly blew my mind. The ability to see what your users do on your website gives a powerful ability to influence and recommend them on the path you want. Very simply clickstream is a log of all clicks and navigations on your website

How can this help. Few use cases

1. Improve user experience – Understand drop-offs and time to convert from one stage to another
2. Recommended products – Basis user page views, you can recommend similar products.
3. Segment customers – Understand what they want and send them personalized messages

Instead of asking users to tell you through explicit signals like like/share/comment, you will use implicit signals to make your website better

There are ton of products available in the market which enable you to capture clicks data like Mixpanel, Pendo, clevertap, webengage. These are fully functional products and take time to install, and are quite expensive. Here am going to walk you through a simple DIY tool to capture clicks on your website

Ok lets get started

We will build our clickstream for a shopify or ecommerce website. The uniform structure of shopify allows us to create a one-size fits all clickstream framework for ecommerce websites

Here is the information I want to capture

1. Top navigation will indicate the categories and how the user navigates to product

2. Catalog page clicks to products. Indicates type of product user wants

3. Product Detail page is the changing room equivalent in the digital world. Clicks on size, material etc indicate user interest

4. Login/Logout

5. Searches on website

6. Form filling

For the sake of simplicity, will only keep website clicks in the scope. Most Ecommerce players anyway rely more on website for traffic attraction. App is mainly used for regular users. There are ways to add App data which we will possibly cover later on

---

Now for each element to be captured, we will need to get some metadata or click information which will help identify the clicks. E.g. how will you differentiate between the “Add to cart” button on different product pages? The following information will be helpful

1. Dom Element text
2. Dom Element class
3. Click time
4. URL
5. Guest token (This is a browser token similar to cookie which will help identify guest users. Most users will be guest users and this is an important feature to identify returning users and enable personalization for them)
6. Session identifier (a browser-server session)
7. User identifier (if logged in)

The project will be divided into

1. Capture the events through a javascript event listener inserted into a page or multiple pages
2. Create a database to store the events in raw form. We can use SQL database since the structure is known. For further enhancements, will consider noSQL
3. Build an analytics module using Django (that’s what I know best). This analytics module will feed data into a processed database. The processed database will contain
   • User affinity to product based on his browsing and transaction
   • Similarity between products based on NLP
   • User affinity to each other again based on product and transaction
   • Users will be both at guest and logged in level
4. The processed DB will enable us to share APIs which can give information like how related a user is to what kind of products etc
5. Finally the APIs can be plugged into a shopify plugin or directly be called by the javascript pixel embedded in the client code. This will help personalizing the website content if required
6. These APIs can also be used to personalize messages like product category etc

Now that we have laid down the skeleton of the framework, lets get around to implementing it