Tealium Web Page Processing Order

To understand how Tealium’s Tag Management System (TMS) loads, we should look at how a

browser loads a web page.

1. First, the browser requests the web page’s HTML (Hyper Text Markup Language) document.

2. The browser begins to download and parse the HTML document starting at the top-left and

continuing down to the bottom-right. As the browser reads through the HTML it encounters various

elements that make up the content of the web page. These elements can be any images, links,

text, or scripts like utag.js.

3. The browser continues to download these elements as it encounters them while it downloads

the HTML document itself. In the case of regular scripts, the browser must stop downloading the

HTML and interpret the script before continuing on to the rest of the HTML. Smart scripts, like

utag.js, have a built-in halt that pauses the script and allows the browser to continue downloading

the HTML. The halt is released when the browser signals that the HTML has been fully

downloaded and parsed into the Document Object Model (DOM). This signal is typically called

‘DOM Ready’ (more on this later).

4. The browser begins to render the web page on the visitor’s screen.

Order of Operation

The utag.js operations flow in a very specific order and understanding that order can prevent errors in

your implementation.

1. Run Extensions scoped to “Pre-Loader” and set Privacy Manager cookie

2. Process Data Layer

3. Evaluate Load Rules

4. Run Extensions scoped to “All Tags” and the Privacy Manager Extension

5. Load Bundled Vendor Tags and Vendor Tags with their Wait Flag set to “Off”

6. Load Vendor Tags, their Extensions, and Mappings

7. DOM Ready and the Send Function

8. After Page Load: utag.view() and Advanced Configurations

Processing of utag.js:

Step 1: Pre-Loader Extensions & Privacy Manager Cookie

Pre-Loader extensions run first. Since Pre-Loader extensions run before everything else, it is vital

that they not rely on any of the Tealium elements (data layer, other extensions, etc.) that run later.

Attempting to do so could result in the page loading improperly, or not at all. The order in which PreLoader extensions are processed is based upon their order in Tealium iQ. The Tealium Privacy cookie is set at this point, if it does not yet exist.

Step 2: The Data Layer is Built

The utag_data object, also referred to as the Universal Data Object (UDO), passes data from your

web page to Tealium iQ. The variables and data contained in utag_data are combined with data from

the rest of the web page, 1st-party cookies, and the DOM to create the utag.data object. While the

utag.data object contains all the web page’s data, only the data you identify as data sources are used

in the Tealium iQ console for mappings.

Step 3: Load Rules

The Load Rules are evaluated. The Load Rules determine if particular Tags load on any given

web page. If a Load Rule determines that a Tag should not load on a web page, that Tag and any

extensions scoped to it do not load. At this point, all of the data on the page may not be available as

the browser has not yet finished parsing. The Load Rules are re-evaluated later, when the rest of the

page’s data is available, to make certain they evaluate correctly.

Step 4: Extensions Scoped to “All Tags” & Privacy Manager

Any extensions scoped to “All Tags” run. These extensions execute in the order in which they appear

in Tealium iQ. This scope indicates that the data that come from these extensions are available to all

the Tags in your profile.

The Privacy Manager Extension, if enabled, runs with the extensions scoped to “All Tags”. The

Privacy Manager provides your visitors a way to manage their privacy settings for the various

Tags in use on your site. The visitor is provided with the option to opt in or out (depending on your

configuration) of Tags, if they so choose. It is essential the Privacy Manager runs before any other

Tag in order to abide by local restrictions or laws regarding visitors’ privacy.

The first thing the Privacy Manager does is to check for the Tealium Privacy cookie to determine the

privacy settings. Depending on your visitor’s preferences, the Privacy Manager may prevent some or

all of your Tags from loading. The Privacy Manager saves the visitor’s privacy settings in the Privacy

Manager cookie for the next time he or she visits your site.

Step 5: Bundled Tags and Tags With the Wait Flag set to “No”

Two types of Tags load at this point: bundled Tags and Tags with their Wait Flag set to “No”. Bundling Tags means that all of the Tag requests are placed into one large request The Wait Flag is an advanced setting that allows you to choose whether or not the Tag waits for the DOM Ready signal before it begins loading.

These Tags will load based on the initial evaluation of Load Rules and visitor privacy settings set by

the Privacy Manager.

Step 6: Vendor Tags, their Extensions, and Mappings:

Depending on whether or not the Load Rules permit, the vendor Tag libraries load next. If any of

those Tags have extensions scoped to them individually, those extensions load along with the Tag.

After the Tag libraries and Tag-specific extensions are loaded, the page data variables you identified

as data sources are mapped to the vendor Tags’ destination variables. As discussed earlier, even

though the Tag libraries load, they don’t do anything until the DOM Ready signal is received.

Step 7: DOM Ready and the Send Function

When the browser sends the DOM Ready signal, DOM Ready-scoped extensions and Tag-specific

extensions execute and data source mappings are processed. The Send function collects the final

vendor Tag requests together and sends them to each of the vendor services. The order in which you

have placed your Tags in Tealium iQ is the order in which the data is sent to each vendor.

Step 8: After Page Load: utag.view() and Advanced Configurations

We can use tealium web companion  (tealium tools extension for Chrome), in order to look what is load on our page.

Tealium IQ objects

I mentioned about Tealium IQ in my previous post. It is real powerful tool in age of digital transformation where speed and consolidations are vital.

Let's look inside in order to meet IQ objects.

Data Layer which has Data Sources and Bundles.

Data Layer is a collection of the data from your web page that is critical to your digital marketing campaigns all centralized in to one location in Tealium iQ.  There is an example of data layer, where we have all our objects:

By configuring data points to use in the Data Layer, such data can be shared with marketing technology tags configured inside the Tealium Tag Management platform, but can also be used to fire digital marketing technologies. Data Layer is not the Data Object; the Data Object is a JSON, a universal format which can work with all web browsers and systems alike. The Data Object forms part of the Data Layer.

We have 6 types of data you can choose from to compose your Data Layer.  

We call these data types data sources and they can be broken down in to the following categories: Universal Data Object, JavaScript Page Variable, Meta Data, Cookies, Query String Parameters and DOM Data. 

UDO – The UDO is an acronym for Universal Data Object – Your UDO is a code based representation of all the business critical information you want to know about a page when it loads.  For example, you will want to know the pages name, any information that may be related to the site like its country or language or the site name.  You may want visitor related information or to be aware of certain events like which products are being viewed or any product related ecommerce data.  Any information that may be required or useful to report you can place in your UDO.

JavaScript Page Variables – These are variables on your page that are outside of your Universal Data Object that hold a value that can be changed.  They can be text, number or true/false Boolean values, or something like a carousel click or an event for link tracking.

 Meta Data – is used when you are referencing the content on your page.  Information on your page may include things like author name of a blog or keywords or a title of an article.

Cookies – Tealium only tracks first party cookies.  You will select this data source when you are referencing a value in a cookie. 

Query String Parameters – When a data (or a variable) is passed within the URLs query string, you can point Tealium to that data and collect it in your data layer.  The querystring is anything in the URL beyond the question mark.  The querystring is set up in name value pairs that are separated by an equal sign, and each pair is separated by an ampersand.  If you have ever seen a referrer code in your querystring set to a specific value, then you have seen a querystring parameter.  The name is the referrer code and the value is what it is equal to. 

DOM Data – Any data stored in your Document Object Model, or DOM,  is automatically available to you in Tealium Data Layer and does not have to identified in your Data Layer.

AudienceStream Attributes are treated as imported data sources in Tealium iQ. This means you cannot edit or delete them from Tealium iQ. If you want remove or modify an Attribute, you must do so from your AudienceStream profile. Moreover, you cannot send Attributes to a profile library, only to a regular profile.

Bundles - list of predefined objects in order to quickly build Data Layer based on the most commonly used Data Sources and Tealium best practices. There are Standard Bundles ranging across a variety of uses from E-Commerce to Mobile Apps and Provider Bundles from such services as Demandware and Wordpress. For example Customer Data Sources:

The Tags tab gives you the ability to load your Tags on your websites. We support over 600 hundred different Tags in our Tags marketplace. Adding and configuring your Tags through Tealium iQ can be as easy as a few points and clicks. In other word we can choose any vendor from the list in order to get predefine tag. For example, on screen below I can easy deploy, google analytics, mixpanel and kissmetrics in order to send to these systems data from data layer.

Load Rules - allow me define rules for particular tag. For example, we can run tag only on landing page or during checkout.

Data Mappings - the last step during setup of tag. It allows me to map tealium iq objects with 3rd system. For example, I can collect data through data layer and send it to Google Analytics.

Extensions gives us power to manipulate with data objects in order to use various functions and build new objects. For example, we can calculate new field, or set any value to cookie.

AWS S3 logs in Splunk

With Splunk it is possible to solve any task where we have lot's of unstructured data and want quickly to insight value from data. Let's consider one of example in case of cloud technologies such as AudienceStream DMP This is amazing data management platform which services us for real marketing automatization.

One of capabilities of AS platform is enriching current customer database with external data. For example we can using machine learning and internal transaction data for  defining score for every customer who is online. We can calculate scoring via R or enterprise data mining platform such as SAS or SPSS and send result to Amazon S3. AS can connect S3 and ingest files. But how can we analyze this process? There are lots of S3 logs in our bucket with GET, PUT methods.  Splunk can easy connect AWS S3 via splunk app - Splunk Add-on for Amazon Web Service.

As usual we download this app and install. Than we have to connect to AWS account:

We only need copy paste Account Key ID and Secret Key.
After connecting to AWS account we should setup new data inputs. Let's go to AWS S3:

And click "Add New".
Than choose existing account, S3 host name (in my case it is default) and S3 bucket:

Moreover we can specify index and white/blacklist and some other options. When we finish, Splunk will update input.conf file which we can access any time and adjust. There are all possible parameters of S3 input
When we finish, splunk begin to ingest logs. We can easy search them by typing name of index, in my case it is "s3logs":

 By default splunk didn't extract any field from S3 logs. But we can easy find structure of log and
using field extractor create all fields:

We have all information in order to create report, which shows us hourly upload by AS for the last 48 hours.

index=* sourcetype=aws:s3 earliest=-48h method_detail="REST.GET.OBJECT" clientip = 107.14.21.138 object_name=*.csv | timechart span=1h count by object_name

We filter by filetype, method and ip of AS.

Moreover, we can use Splunk CLI and automatically run search via crontab and extract to flatfile in order to visualize and update report every 5 minutes.

There is shell script for linux:

#!/bin/bash
# File: test_action.sh 
# Description: To output saved search result
#

SPLUNK_HOME="/Applications/Splunk"
OUTPUT="test_output.log"
USER=admin
PASSWORD=changeme
$SPLUNK_HOME/bin/splunk search 'index=* sourcetype=aws:s3 earliest=-48h method_detail="REST.GET.OBJECT" clientip = 107.14.21.138 object_name=*.csv | timechart span=1h count by object_name' -auth ${USER}:${PASSWORD} > ${OUTPUT}

Adjsut vs Appsflayer

As a part of digital transformation we are focusing on mobile applications and try to acquire new users into mobile phone. As you know there are several strategy can exist:

1. Web Only
2. Web 1st, mobile 2nd
3. Mobile 2st, web 2nd
4. Mobile 1st 

Just look at this chart from ComScore:

It is clear that all companies should focus on digital and don't forget about mobile consumers/clients.

One of important option, that I want from mobile tracker - integration with facebook, tweeter and advertising networks. As a result, I got 2 finalists: Adjsut and Appsflyer.

There is short comparison, why Adjsut is a winner:

1. SDK: Adjust SDK is open source resource, it helps to see how is the interaction with the application and what information is read from the application. So, this mean that all data is only your and not transferred to third parties. Adjsut support all major platforms such as Windows Phone, Unity, Adobe Air, Cordova and soon Unreal, Cocos2DX and Xamarin.

2. Data privacy: Adjust is a certificated that guarantees complete privacy of data during transmission between servers. Adjsut decided to make an open source code is because they want that their customers to see what's going on with their data. So all data is encrypted using SSL and licensed using the best Software. Adjust transmits all the data you and thus encodes it on our side with SSL method that does Appsflyer. So as Appsflyer stores data in raw format, on its side that does not correspond to the data privacy and you do not know what they do with them.

3. Reporting: Adjust provides real time reporting and data filtering to 4 levels of this skill in the structure of the campaign, from the traffic and the resource sub campaign. Compared with Appsflyer they only have 2 leveles (resource and campaign). Appsflyer cohort analysis very limited in retention analysis may see only the last 12 days retention. Adjust cohorts no limit viewing of data and especially retention, customers can select the dynamic by various filters, period, segments and apply certain events to the analysis of a cohort.

4. Retargeting: Adjust retargeting supports all reattribution on any platform on the market as Criteo, Facebook, Twitter, TapCommerce, etc. AppsFlyer only supports reattribution for Facebook.

5. Data export: You can download the whole data with adjust to csv format, and you can choose what data you want to upload. Appsflyer provides only 3 pre-formulated report and not one of them does not disclose evena on install date. Export date of cohorts have also not possible.

6. Callback parameters: Adjust as the same may send an unlimited number of parameters in the callback to server, and support for custom parameters (for example the login user on different devices). Appsflyer not support custom settings at all. In order to look at full list of defaul parameters, you can visit https://partners.adjust.com/placeholders/

7. Deeplinking: Deeplinking adjust offers to both, reengagments and user acquisition campaigns. In our case, there is no difference, the application has already pumped or not we support deeplinking and new users. Appsflyer deeplinking provides only for applications that have already been uploaded to the phone, and always carries a gun on the Side and not a specific page in the application.

8. Pricing: Adjust's possible to look like the first but even then they pay for install not make them cheaper, the more it touches the very customers that lead many campaigns on Facebook, Twitter and iAd. In this case adjust may be up to 4 times cheaper than Appsflyer. Appsflyer billing goes once a week, and adjust at once a month, and note that the price of install much higher than adjust.

• Adjust prices:

• Appsflyer prices:

9. Retention: The ability to look at retention users for any period of time. Adjsut haven't restrictions and Appsflyer retention may see only until the 12th day.

10. Creatives: One of adjust levels of this skill has a tracking creatives and their optimization. Appsflyer does not provide the ability to track and analyze creatives.

11. Support: Adjsut provide a full support service and documentation + as the works for a support Appsflyer in Israel, it means that they have a weekend on Friday when we have workday. :)

12. API: In case if you want work with API, then Adjust can offer only starting from Business Pro.

Mobile metrics vs Web metrics

Mobile analytics is a logical continue of web analytics. In other words it is part of Digital Intelligence. Let's try to build connection for our daily web analytics metrics in case with mobile analytics.
In terms of mobile we should focus on behavior and actions.

We can divide metrics into 4 groups:

1. Metrics to evaluate Awareness 
2. Metric to Evaluate Engagement
3. Metrics to Evaluate Long-Term Value (ROI driving metrics)
4. Using App Analytics to do Smart Marketing (grow your happy user base)

1. Metrics to evaluate Awareness 

• Site traffic (web) = downloads (in apps)
• Number of people who install your app
• Helps evaluate SEO, ASO and acquisition campaigns
• Does not show you long term value of users
• Visitors (web) = Users (apps)
• People interactively using your app (not just visiting it)
• Tracking and understanding users helps you: Build segments, Elicit deeper engagement, Launch targeted app marketing)
• Web Pages = Screens
• User navigates between app screens
• Responsive to touch, shorter than a webpage, functional or instructional
• Pageviews = Sessions
• Counted from app open to app close
• One uninterrupted period of interaction in which a user is actively using your app

2. Metric to Evaluate Engagement

• Monthly Unique Visitors = Monthly Active Users (MAUs)
• Number of people who have opened your app at least once within the last 30 days
• Look for trends (going up or down)
• Target Audience = User Segments
• Groups of users (based on shared characteristics or actions)
• Essential for personalizing app marketing campaigns and sending targeted messages
• Time on Screen = Time on Screen

• How much time someone spends on a screen (Also measure number of screens per session)
• Investigate which screens 
  • User visit most
  • Spend the most time on 
  • Navigate away from quickly
• Time on Site = Session Length

• Period of time between app open and close
• Closely related metrics 
  • Number of sessions (how frequently are users returning to your app?)
  •  Session interval (how much time that passes between sessions?)

In addition, there are the interaction-focused app metrics

• Monthly active users
• User segments
• Time on screen
• Session Length

3. Metrics to Evaluate Long-Term Value (ROI driving metrics)

• Conversions=Events
• Actions taken by users in app
• Key events move people towards "ultimate" in-app action
• Examples: Checkout completed, article read, video played
• Bounce Rate=Funnel Drop-Off Rate

• A funnel is a series of key events that lead to "ultimate" in-app action
• Look at drop-offs (and conversion rate) at each step
• Cohorts=Cohorts

• Segments of users engaging with your app, based on the date of their first visit
• Groups of people who first downloaded your app in X month, and what they went to do later
• Highlights most engaged and valuable users over time
• Can split out cohorts by
  • Device
  • Segment
  • Campaign
  • Custom dimensions
• Monitor changes to retention as you launch new app updates
• Revenue per visitor = User Lifetime Value (LTV)

• Represents how much each user is worth in their lifetime
• Capture financial worth over time in terms of loyalty and evangelism
• Primary app revenue metric (various formulas)

4. Using App Analytics to do Smart Marketing (grow your happy user base)

• Email Marketing=Push&In-App Messaging
• Puch Notifications
  • For re-engaging latent users
  • Delivered to home screen
• In-app messages:
  • For moving users through key app funnels
  • Rich design capabilities
• Content Marketing=Context Marketing

• Highly targeted, relevant and personalized app marketing
• Tailor push and in-app messages to take both in-app and outside-of-app behavior

Overview of leading tag managment platfotm Tealium IQ

In this post I want to cover Tealium. Our company really focus on digital transformation and as a result we want to be flexible and get more speed in all marketing processes. In other words, tealium gives us great power for all marketing activities.

By default Tealium IQ is a tag management system. 

What is Tag? 

Tags are an efficient method to exchange data between your site, your visitor and your digital marketing vendors.

These clever little bits of code sit on your website and allow you to connect from the digital world with your customer in the physical world. 

Tags allow us to gather knowledge.  To understand the likes, dislikes, habits and behaviors as our visitors walk through our website. 

In our day to day lives its hard to see the effectiveness of a billboard or sign in a shopkeepers window, but in the digital world, thanks to tags, we can see which offers we make to our customers convert and we are able to improve our marketing strategies with the knowledge we gather. 

Typically, the workflow for adding a tag to your website begins with the marketing department deciding to add a particular tag, contacting the tag vendor for their implementation guide, communicating with IT to get the tag hard coded on to the site and then working with the tag vendor to ensure that the exchange of information is working.  Simple enough?  Not always.

Tags have a dark side.  Getting a tag hard coded on to your website can take weeks, depending on the length of your organizations dev cycle and how busy your IT department may be.  Working with multiple tag vendors, running multiple campaigns can also be time consuming and complex. 

But most importantly, once you have invested in implementing a tag, you are then relying heavily on the data you’re receiving from your tag vendor to guide and shape your digital marketing campaigns. 

How clean is the data you are basing your decisions on?  And are your tags performing on your page? Or, are they lowering your site performance? 

What is Tag Management?

This is where Tag Management steps in.  Tag Management is about getting more of what you want and less of what you don’t want. 

What you want is cleaner data, faster.  What you don’t want is lengthy delays in deploying tags and the “one sizes fits all” solution your tag vendors provide. 

The Tealium iQ interface allows you to manage the process of cleaning up your data before sending it to your tag vendors with points and clicks.  It also allows you to decide when and where you want your tags to load on your website. 

You most likely have an array of tag vendors ranging from site analytics and affiliate marketing to multivariate testing and retargeting tools. This can quickly get complex and difficult to manage without a tag management system that standardizes your data and centralizes you digital marketing campaigns in one easy to use interface. 

Tealium iQ allows marketers to be more self-sufficient when it comes to managing their tags and to removing the IT bottleneck that can occur when reaching out to a development team for implementing or updating a tag in the source code.  On the flip side, TiQ also works as a time saving tool to development teams. 

Tag Management is just what it says it is.. it is the ability to manage your tags instead of your tags and tag vendors managing you. 

What is Data Centric Tag Management?  Simply put, it’s an agile and efficient data exchange system for managing your tags.  Instead of tags being content on your page that grabs data and sends it off to your vendors, tag management begins with identifying the data points that are available on your page and are valuable to your digital marketing campaign. 

Instead of multiple tags hard coded on to your webpage, now you have a single tag, the Tealium Tag, through which the data flows in to the User Interface, and from there you can begin asking yourself questions.  Questions like “When and where do I want these tags to fire?” and “Which data points do I want exchanged with each one of my tag vendors?”   

Through Data Centric Tag Management, miraculous things begin to occur. 

First, you get to standardize your data.  When your data flows in to Tealium iQ first, before being sent to your tag vendors.  Here in Tealium iQ it is identified as a data source, which you name, identify and describe.

From there, the data source is mapped out to your tag vendors, providing a standard data collection across multiple platforms.  Your data collection will improve because a data-centric tag management system is a simplified method from translating your data on the page to the different formats and syntax that is used by your various vendors.  

Now, you can control which data points flow to which vendors, allowing you to determine who you are sharing your data with. 

All of this occurs with fewer IT resources since you are stepping outside of dealing with JavaScript code, which is better suited for marketers, while still allowing advanced JavaScript editing capabilities for the more technical users of Tealium iQ. 

Remember, with data centric tag management, you are getting cleaner data, faster, with fewer resources and improving the collection and distribution of your data!   

That’s the benefit of Tag Management!

Tealium is a cloud application. It uses some of the biggest and fastest delivery networks on the planet to deliver technology tags to site visitor web browsers. 

We can visit one of Tealium IQ clients and quickly at client's data layer. For example, let's visit http://www.cathaypacific.com/ . If you have Ghostery plagin into you browser you can quickly identify list of trackers of this web site:

One of trackers is Tealium. There are lot's of business use cases and advantages of using Tealium IQ:

• Data Driven Marketing
• Single source of customer profiles to power cart and browse abandonment, as well as align customer messaging across channels
• Fast deployment of tags
• Meet rapidly changing business requirements
• Maximize its marketing technology investments
• and many many others

SlackIT with Splunk

Splunk has announced cool new app SlackIT. You can easy download it.

Wait, do you know what is slack? It is the best in class team communication service. Do you have a team? Then you definitely should use slack.

Ok, let's connect splunk and slack.

First, we should download new app in splunk. It is easy. Moreover we haven't do any addition actions in splunk interface, except some actions with slack.conf file. But before we start, lets go to slack and add new integration:

Let's choose channel, which will get result from splunk search

In addition, we can choose any person, who will get result. Then click on "Add Incoming WebHook Integration" and copy Webhook URL, which will send json result from splunk to slack.

We have done with slack. Let's start to configure slack.conf file, which we can find in slackit app:

• url: Webhook URL , for example: https://hooks.slack.com/services/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
• username: the username that appears in slack
• channel: the channel that will receive the search results, channel can be specified by slackit search arg or person
• icon: the icon that appears in slack.
• allow_user_set_slack_url: enable this option if you're allowing the users send results to different # slack teams
• send_message_even_no_results: disable option to send message without result

[config]
url = https://hooks.slack.com/services/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y
username = gold_unicorn
channel = our-team
icon = https://www.splunk.com/content/dam/splunk2/images/icons/favicons/mstile-150x150.png
allow_user_set_slack_url = 0
send_message_even_no_results=0

Save file. That's all. Now we can easy send result from splunk to the salck channel. Let's try to do it.

As you see on screen, I've just added slackit command. And what is the result? All my team in channel saw this result:)


PS As a bonus you send any message via terminal using the same Webhock URL:

curl -X POST --data-urlencode 'payload={"channel "#our-team", "username": "webhookbot", "text": "What's up!", "icon_emoji": ":ghost:"}' https://hooks.slack.com/services/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y