четверг, 17 сентября 2015 г.

Role of web analytics in the organization

This bundle of product can solve any marketing task and it is easy integrate with eachother. But the price is pretty expensive for the leader. Of cource, it is possible to use open source solutions and combine with cheap marketing solutions in order to create ZOO of tools but it all depends on marketing budget.

As analyst I should work with Web Analytics and I want to cover questio, why do we need web analytics tool and who should use it.

Basicaly web analytics tool can answer on questions like:
  • Which website promotions are getting visitors to click and convert?
  • Where are website conversions funnels breaking down and causing the organization to lose visitors?
  • Which products on the website are performing the best?
  • What is average time on site per visitor?
  • What pages does customer visit before checkout?
  • Which content or tools on the website are having the biggest impact on converting?
  • and many others
Standart implementation of web analytics tools consist of 5 steps:

  1. Identifying the business questions to be answered
  2. Finding the best way to answer those question via collected data
  3. Tagging our website
  4. Development of reports and dashboards
  5. Dive in reports and dashboards in order to find business insights 
We can define the following roles for using SiteCatalyst:
  • Head of Digital Intelligence  - often meet with top management team and communicate with his team in order to understand how things are
  • Web Analyst - work with data, invent hypotheses and check them, create segments, A/B test, reports and etc in order to produce valuable information
  • Developer - guys who can work with code
  • Marketers - they spend money on implementation of web analytics tool but haven't enouth time to attend training or even log on into account. In case if they successful log on they can't creat proper report.

Adobe Marketing Cloud Brief overview

As a Digital Analyts, I have to work with Adobe Marketing Cloud, it is a leader on the market.
According to Forrester Adobe is a leader
Adobe offers lots of producs for marketing. It calls adobe cloud marketing.
Analitycs - It’s the industry-leading solution for applying real-time analytics and detailed segmentation across all of your marketing channels. Use it to discover high-value audiences and power customer intelligence for your business. Here we can finde main web anaytics product Adobe SiteCatalyst. Morover it has mobile marketing services for mobile analytics.
Audience Manager - It’s a data management platform that helps you build unique audience profiles so you can identify your most valuable segments and use them across any digital channel.
Campaign - It's a set of solutions that help you personalize and deliver campaigns across all of your online and offline channels. In other words it is campaing management tool like IBM Unica.
Experience Manager - It’s a comprehensive content management solution for building websites, mobile apps and forms. And it makes it easy to manage your marketing content and assets.
Media Optimizer - It’s a programmatic ad-buying solution that helps you forecast the best mix of search, display and social ads based on your budget. It also automates the execution of your media plan and helps you find the best way to deliver relevant content to your high-value audiences.
Primetime - It’s a multiscreen TV platform that helps broadcasters, cable networks and service providers create and monetize engaging and personalized TV and film experiences.
Social - It’s the social management platform that ties every piece of data to your bottom line. Beyond just the likes and follows, manage the deep relationship between your customers’ sentiments and your business goals.
Target - It’s a personalization solution that makes it easy to identify your best content through tests that are easy to execute. So you can deliver the right experience to the right customer.


вторник, 15 сентября 2015 г.

Tealium Web Page Processing Order

To understand how Tealium’s Tag Management System (TMS) loads, we should look at how a
browser loads a web page.

1. First, the browser requests the web page’s HTML (Hyper Text Markup Language) document.
2. The browser begins to download and parse the HTML document starting at the top-left and
continuing down to the bottom-right. As the browser reads through the HTML it encounters various
elements that make up the content of the web page. These elements can be any images, links,
text, or scripts like utag.js.
3. The browser continues to download these elements as it encounters them while it downloads
the HTML document itself. In the case of regular scripts, the browser must stop downloading the
HTML and interpret the script before continuing on to the rest of the HTML. Smart scripts, like
utag.js, have a built-in halt that pauses the script and allows the browser to continue downloading
the HTML. The halt is released when the browser signals that the HTML has been fully
downloaded and parsed into the Document Object Model (DOM). This signal is typically called
‘DOM Ready’ (more on this later).
4. The browser begins to render the web page on the visitor’s screen.
Order of Operation

The utag.js operations flow in a very specific order and understanding that order can prevent errors in
your implementation.
1. Run Extensions scoped to “Pre-Loader” and set Privacy Manager cookie
2. Process Data Layer
3. Evaluate Load Rules
4. Run Extensions scoped to “All Tags” and the Privacy Manager Extension
5. Load Bundled Vendor Tags and Vendor Tags with their Wait Flag set to “Off”
6. Load Vendor Tags, their Extensions, and Mappings
7. DOM Ready and the Send Function
8. After Page Load: utag.view() and Advanced Configurations

Processing of utag.js:
Step 1: Pre-Loader Extensions & Privacy Manager Cookie
Pre-Loader extensions run first. Since Pre-Loader extensions run before everything else, it is vital
that they not rely on any of the Tealium elements (data layer, other extensions, etc.) that run later.
Attempting to do so could result in the page loading improperly, or not at all. The order in which PreLoader extensions are processed is based upon their order in Tealium iQ. The Tealium Privacy cookie is set at this point, if it does not yet exist.
Step 2: The Data Layer is Built
The utag_data object, also referred to as the Universal Data Object (UDO), passes data from your
web page to Tealium iQ. The variables and data contained in utag_data are combined with data from
the rest of the web page, 1st-party cookies, and the DOM to create the utag.data object. While the
utag.data object contains all the web page’s data, only the data you identify as data sources are used
in the Tealium iQ console for mappings.
Step 3: Load Rules
The Load Rules are evaluated. The Load Rules determine if particular Tags load on any given
web page. If a Load Rule determines that a Tag should not load on a web page, that Tag and any
extensions scoped to it do not load. At this point, all of the data on the page may not be available as
the browser has not yet finished parsing. The Load Rules are re-evaluated later, when the rest of the
page’s data is available, to make certain they evaluate correctly.
Step 4: Extensions Scoped to “All Tags” & Privacy Manager
Any extensions scoped to “All Tags” run. These extensions execute in the order in which they appear
in Tealium iQ. This scope indicates that the data that come from these extensions are available to all
the Tags in your profile.
The Privacy Manager Extension, if enabled, runs with the extensions scoped to “All Tags”. The
Privacy Manager provides your visitors a way to manage their privacy settings for the various
Tags in use on your site. The visitor is provided with the option to opt in or out (depending on your
configuration) of Tags, if they so choose. It is essential the Privacy Manager runs before any other
Tag in order to abide by local restrictions or laws regarding visitors’ privacy.
The first thing the Privacy Manager does is to check for the Tealium Privacy cookie to determine the
privacy settings. Depending on your visitor’s preferences, the Privacy Manager may prevent some or
all of your Tags from loading. The Privacy Manager saves the visitor’s privacy settings in the Privacy
Manager cookie for the next time he or she visits your site.
Step 5: Bundled Tags and Tags With the Wait Flag set to “No”
Two types of Tags load at this point: bundled Tags and Tags with their Wait Flag set to “No”. Bundling Tags means that all of the Tag requests are placed into one large request The Wait Flag is an advanced setting that allows you to choose whether or not the Tag waits for the DOM Ready signal before it begins loading.
These Tags will load based on the initial evaluation of Load Rules and visitor privacy settings set by
the Privacy Manager.
Step 6: Vendor Tags, their Extensions, and Mappings:
Depending on whether or not the Load Rules permit, the vendor Tag libraries load next. If any of
those Tags have extensions scoped to them individually, those extensions load along with the Tag.
After the Tag libraries and Tag-specific extensions are loaded, the page data variables you identified
as data sources are mapped to the vendor Tags’ destination variables. As discussed earlier, even
though the Tag libraries load, they don’t do anything until the DOM Ready signal is received.
Step 7: DOM Ready and the Send Function
When the browser sends the DOM Ready signal, DOM Ready-scoped extensions and Tag-specific
extensions execute and data source mappings are processed. The Send function collects the final
vendor Tag requests together and sends them to each of the vendor services. The order in which you
have placed your Tags in Tealium iQ is the order in which the data is sent to each vendor.
Step 8: After Page Load: utag.view() and Advanced Configurations

We can use tealium web companion  (tealium tools extension for Chrome), in order to look what is load on our page.


Tealium IQ objects

I mentioned about Tealium IQ in my previous post. It is real powerful tool in age of digital transformation where speed and consolidations are vital.

Let's look inside in order to meet IQ objects.
Data Layer which has Data Sources and Bundles.
Data Layer is a collection of the data from your web page that is critical to your digital marketing campaigns all centralized in to one location in Tealium iQ.  There is an example of data layer, where we have all our objects:
By configuring data points to use in the Data Layer, such data can be shared with marketing technology tags configured inside the Tealium Tag Management platform, but can also be used to fire digital marketing technologies. Data Layer is not the Data Object; the Data Object is a JSON, a universal format which can work with all web browsers and systems alike. The Data Object forms part of the Data Layer.
We have 6 types of data you can choose from to compose your Data Layer.  

We call these data types data sources and they can be broken down in to the following categories: Universal Data Object, JavaScript Page Variable, Meta Data, Cookies, Query String Parameters and DOM Data. 

UDO – The UDO is an acronym for Universal Data Object – Your UDO is a code based representation of all the business critical information you want to know about a page when it loads.  For example, you will want to know the pages name, any information that may be related to the site like its country or language or the site name.  You may want visitor related information or to be aware of certain events like which products are being viewed or any product related ecommerce data.  Any information that may be required or useful to report you can place in your UDO.
JavaScript Page Variables – These are variables on your page that are outside of your Universal Data Object that hold a value that can be changed.  They can be text, number or true/false Boolean values, or something like a carousel click or an event for link tracking.
 Meta Data – is used when you are referencing the content on your page.  Information on your page may include things like author name of a blog or keywords or a title of an article.
Cookies – Tealium only tracks first party cookies.  You will select this data source when you are referencing a value in a cookie. 
Query String Parameters – When a data (or a variable) is passed within the URLs query string, you can point Tealium to that data and collect it in your data layer.  The querystring is anything in the URL beyond the question mark.  The querystring is set up in name value pairs that are separated by an equal sign, and each pair is separated by an ampersand.  If you have ever seen a referrer code in your querystring set to a specific value, then you have seen a querystring parameter.  The name is the referrer code and the value is what it is equal to. 

DOM Data – Any data stored in your Document Object Model, or DOM,  is automatically available to you in Tealium Data Layer and does not have to identified in your Data Layer.
AudienceStream Attributes are treated as imported data sources in Tealium iQ. This means you cannot edit or delete them from Tealium iQ. If you want remove or modify an Attribute, you must do so from your AudienceStream profile. Moreover, you cannot send Attributes to a profile library, only to a regular profile.

Bundles - list of predefined objects in order to quickly build Data Layer based on the most commonly used Data Sources and Tealium best practices. There are Standard Bundles ranging across a variety of uses from E-Commerce to Mobile Apps and Provider Bundles from such services as Demandware and Wordpress. For example Customer Data Sources:
The Tags tab gives you the ability to load your Tags on your websites. We support over 600 hundred different Tags in our Tags marketplace. Adding and configuring your Tags through Tealium iQ can be as easy as a few points and clicks. In other word we can choose any vendor from the list in order to get predefine tag. For example, on screen below I can easy deploy, google analytics, mixpanel and kissmetrics in order to send to these systems data from data layer.

Load Rules - allow me define rules for particular tag. For example, we can run tag only on landing page or during checkout.

Data Mappings - the last step during setup of tag. It allows me to map tealium iq objects with 3rd system. For example, I can collect data through data layer and send it to Google Analytics.

Extensions gives us power to manipulate with data objects in order to use various functions and build new objects. For example, we can calculate new field, or set any value to cookie.

вторник, 1 сентября 2015 г.

AWS S3 logs in Splunk

With Splunk it is possible to solve any task where we have lot's of unstructured data and want quickly to insight value from data. Let's consider one of example in case of cloud technologies such as AudienceStream DMP This is amazing data management platform which services us for real marketing automatization.

One of capabilities of AS platform is enriching current customer database with external data. For example we can using machine learning and internal transaction data for  defining score for every customer who is online. We can calculate scoring via R or enterprise data mining platform such as SAS or SPSS and send result to Amazon S3. AS can connect S3 and ingest files. But how can we analyze this process? There are lots of S3 logs in our bucket with GET, PUT methods.  Splunk can easy connect AWS S3 via splunk app - Splunk Add-on for Amazon Web Service.

As usual we download this app and install. Than we have to connect to AWS account:
We only need copy paste Account Key ID and Secret Key.
After connecting to AWS account we should setup new data inputs. Let's go to AWS S3:
And click "Add New".
Than choose existing account, S3 host name (in my case it is default) and S3 bucket:
Moreover we can specify index and white/blacklist and some other options. When we finish, Splunk will update input.conf file which we can access any time and adjust. There are all possible parameters of S3 input
When we finish, splunk begin to ingest logs. We can easy search them by typing name of index, in my case it is "s3logs":
 By default splunk didn't extract any field from S3 logs. But we can easy find structure of log and
using field extractor create all fields:
We have all information in order to create report, which shows us hourly upload by AS for the last 48 hours.

index=* sourcetype=aws:s3 earliest=-48h method_detail="REST.GET.OBJECT" clientip = 107.14.21.138 object_name=*.csv | timechart span=1h count by object_name

We filter by filetype, method and ip of AS.

Moreover, we can use Splunk CLI and automatically run search via crontab and extract to flatfile in order to visualize and update report every 5 minutes.

There is shell script for linux:

#!/bin/bash
# File: test_action.sh 
# Description: To output saved search result
#

SPLUNK_HOME="/Applications/Splunk"
OUTPUT="test_output.log"
USER=admin
PASSWORD=changeme
$SPLUNK_HOME/bin/splunk search 'index=* sourcetype=aws:s3 earliest=-48h method_detail="REST.GET.OBJECT" clientip = 107.14.21.138 object_name=*.csv | timechart span=1h count by object_name' -auth ${USER}:${PASSWORD} > ${OUTPUT}






Adjsut vs Appsflayer

As a part of digital transformation we are focusing on mobile applications and try to acquire new users into mobile phone. As you know there are several strategy can exist:

  1. Web Only
  2. Web 1st, mobile 2nd
  3. Mobile 2st, web 2nd
  4. Mobile 1st 
Just look at this chart from ComScore:
It is clear that all companies should focus on digital and don't forget about mobile consumers/clients.

One of important option, that I want from mobile tracker - integration with facebook, tweeter and advertising networks. As a result, I got 2 finalists: Adjsut and Appsflyer.

There is short comparison, why Adjsut is a winner:


1. SDK: Adjust SDK is open source resource, it helps to see how is the interaction with the application and what information is read from the application. So, this mean that all data is only your and not transferred to third parties. Adjsut support all major platforms such as Windows Phone, Unity, Adobe Air, Cordova and soon Unreal, Cocos2DX and Xamarin.

2. Data privacy: Adjust is a certificated that guarantees complete privacy of data during transmission between servers. Adjsut decided to make an open source code is because they want that their customers to see what's going on with their data. So all data is encrypted using SSL and licensed using the best Software. Adjust transmits all the data you and thus encodes it on our side with SSL method that does Appsflyer. So as Appsflyer stores data in raw format, on its side that does not correspond to the data privacy and you do not know what they do with them.

3. Reporting: Adjust provides real time reporting and data filtering to 4 levels of this skill in the structure of the campaign, from the traffic and the resource sub campaign. Compared with Appsflyer they only have 2 leveles (resource and campaign). Appsflyer cohort analysis very limited in retention analysis may see only the last 12 days retention. Adjust cohorts no limit viewing of data and especially retention, customers can select the dynamic by various filters, period, segments and apply certain events to the analysis of a cohort.

4. Retargeting: Adjust retargeting supports all reattribution on any platform on the market as Criteo, Facebook, Twitter, TapCommerce, etc. AppsFlyer only supports reattribution for Facebook.

5. Data export: You can download the whole data with adjust to csv format, and you can choose what data you want to upload. Appsflyer provides only 3 pre-formulated report and not one of them does not disclose evena on install date. Export date of cohorts have also not possible.

6. Callback parameters: Adjust as the same may send an unlimited number of parameters in the callback to server, and support for custom parameters (for example the login user on different devices). Appsflyer not support custom settings at all. In order to look at full list of defaul parameters, you can visit https://partners.adjust.com/placeholders/

7. Deeplinking: Deeplinking adjust offers to both, reengagments and user acquisition campaigns. In our case, there is no difference, the application has already pumped or not we support deeplinking and new users. Appsflyer deeplinking provides only for applications that have already been uploaded to the phone, and always carries a gun on the Side and not a specific page in the application.

8. Pricing: Adjust's possible to look like the first but even then they pay for install not make them cheaper, the more it touches the very customers that lead many campaigns on Facebook, Twitter and iAd. In this case adjust may be up to 4 times cheaper than Appsflyer. Appsflyer billing goes once a week, and adjust at once a month, and note that the price of install much higher than adjust.


  • Adjust prices:




  • Appsflyer prices:







9. Retention: The ability to look at retention users for any period of time. Adjsut haven't restrictions and Appsflyer retention may see only until the 12th day.

10. Creatives: One of adjust levels of this skill has a tracking creatives and their optimization. Appsflyer does not provide the ability to track and analyze creatives.

11. Support: Adjsut provide a full support service and documentation + as the works for a support Appsflyer in Israel, it means that they have a weekend on Friday when we have workday. :)

12. API: In case if you want work with API, then Adjust can offer only starting from Business Pro.