Mobile Dashboard in near real time

I chose Klipfolio as a perfect visualization tool for operation reporting. 

There is overview of working architecture:

Using Amazon Redshift Instance I collect  data from adjust. Adjust sends data to ourt local database via callbacks. Than using pentaho data integration as ETL tool I take data and load it to the redshift. Job in pentaho runs every 10 minutes and incrementally update last 7 days. Considering mobile applications we should considering long period, because some phones can send data to mobile tracker with long delayed, that's why I prefer to take 7 days and update them with each load.

Data Flow

Using Pentaho data integration I create 2 transformations and put them in one job.

Run Pentaho DI

Pentaho home path: /home/dasboard/Downloads/data-integration

in order to start pentaha use terminal and run spoon.sh

Transformation 1: stg_adjust

This transformation extract data from mob.adjust for last 7 days and send it to s3://s3/webanalys/adjust_data in gzip format.

Transformation 2: sql_adjust

This transformation run sql queries:

--copy data from file to stg.stg_temp_adjsut

copy stg.stg_temp_adjust from 's3://webanalys/adjust_data'
credentials
'aws_access_key_id=AKIAIFNNQI73NUWZ7BZQ;aws_secret_access_key=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'
delimiter '|'
timeformat 'YYYY-MM-DD HH24:MI:SS'
gzip;

--delete data from stg.stg_adjust in order to write new fresh data
DELETE FROM stg.stg_adjust where stamp>= (SELECT MIN(stamp) FROM stg.stg_temp_adjust);
INSERT INTO stg.stg_adjust (SELECT * FROM stg.stg_temp_adjust);

--optimaze stg_adjsut table
analyze stg.stg_adjust;
vacuum stg.stg_adjust;

--delete from temp table all data
DELETE from stg.stg_temp_adjust;
vacuum stg.stg_temp_adjust;

Job: jb_adjsuttoAWS

Kitchen job

Pentaha allow as to schedule jobs via kitchen. Using crontab I run my job. Moreover, kitchen can perform only shell script. As a result I put my job in shell script:

#!/bin/bash
/home/dasboard/Downloads/data-integration/kitchen.sh -file=/home/dasboard/Downloads/pentaho_di_files/jb_adjusttoAWS.kjb -level=Minimal

When we successful load data to redshift, we can build new klip using database connection in klipfolio.

Just choose postgres database and fill credentials. Unfortunately, klipfolio has one disadvantage, you should create separate query for almost every klip. 

My job runs every 10 minutes. As a result we get fresh data and can quickly check health of mobile applications and their performance.

Klips: Installs, Average Session duration in sec, registrations, daily active unique users, weekly active unique users, monthly unique users and installs by channels:

Adjust Implementation

Going deeper with mobile tracking, I start to look at new mobile trackers. And chose one of the leader - adjust. In order to implement adjust to our mobile apps, we should add SDK to the mobile app.

First of all we should read docs:

• Getting Started https://docs.adjust.com/en/getting-started/
• Event Tracking https://docs.adjust.com/en/event-tracking/
• Tracke Generation https://docs.adjust.com/en/tracker-generation/
• Integration with facebook https://docs.adjust.com/en/facebook/
• Google ad tracking https://docs.adjust.com/en/google/

For mobile developers we should send link on github:

• Github repo https://github.com/adjust/sdks

Then add our application to adjust by typing App ID:

In addition we should describe screens of app and events. If we want to send additional dimensions or attributes we should use callbacks. Because due to personal data law, adjust don't store customer data.

• Working with callbacks https://docs.adjust.com/en/callbacks/

In order to create callback receiver, we should tell to system administrated, what fields will come from adjust. We can learn about default placeholders - https://partners.adjust.com/placeholders/

For example:

Then we should create events in adjust interface:


It has default events and we add some custom, Every custom event has unique token which we will give to mobile developer. Attributes should be connected to event. For example, registration event has account number and account type attributes. There is callback URL on the screen. We should put there link of callback server and list of placeholders in format key={value}. Adjust will replace value.

When mobile developers successful add SDK to the app, they can emulate activity and we can open adjust dashboard in order to see new events. Adjust has two environments. We can test, using SANDBOX in order not to mix with real data.

Going to statistics tab we can look at on default and custom events:


It is interesting fact, but install in mobile tracker it is just first launch. There is no integration with stores.


In addition we can look on events across channels or look at trends:


In order to create acquisition channel, we should go to App tab and click at "New tracker", adjsut will create link. And we can easy add to this link placeholders and custom parameters in order to collect this data in database.




Also it allows us to use cohort menu. For example, we can quickly to build retention profile:


Finally, we can look at rank and rating of mobile app. Unfortunately, we can extract this information from adjust.

The most effective tool for creating dashboard

We have wonderful dashboard in our openspace. I highly reccomend to use this cloud software for consolidation of various KPIs, SLAs and etc.

Let's meet Klipfolio you can quickly to try demo and build your own clips based on your data.

It works very easy. Let's look foundation of klipfolio.

1. Connect your data
Automated data retrieval from 1000s of data services (with more added every day).
Klipfolio is backed by a simple, flexible data architecture. That means you can upload any data source to the dashboard, automate its retrieval, and use (and reuse) that data to build visualizations and dashboards for your team.

 Forget the confusing schemas and intimidating data warehouses. From uploading your data to using it to create visualizations, there is no step in between to complicate matters (and no need to hire a team of developers to get started). Just upload your raw data to start using it to create a visualization.

 Data and presentation are treated separately in Klipfolio which was harder for us, but easier and better for you. Here's why it's better: You can use a single data source to create multiple data visualizations, or use multiple data sources to create a single visualization. What you do with your data is entirely up to you.

By default, all web accessible resources are set to automatically refresh every 30 minutes, but you can boost that up to every minute for real-time data sources. If Klipfolio detects a change in your data, your dashboard will automatically update.

Of course, you can always leverage the Klipfolio API to automate the retrieval of any other data source.







2. Visualize your data
Bring the story of your data to life with custom visualizations.

Everyone loves a good story. Stories have the ability to teach memorable lessons, change behaviour, and inspire action. With Klipfolio Dashboard, you get all the tools you need to bring the story of your performance to life with clear, compelling visualizations.
And you can bet that it's a story that will motivate your data-driven team.






Klipfolio includes 15 different types of visualizations so you can choose a visualization that will have the maximum impact on your dashboard.