1. Captivate Downloads from CDN Processor
Captivate begins its analytics process by downloading logs from its Content Delivery Network (CDN) processor. These logs contain a wealth of information about each request made to the CDN, including details about partial requests.
2. Parsing and Constructing Analytic Entries
The system meticulously parses each line of the log file. During this phase, partial requests are matched to their counterparts to ensure that each download is fully reconstructed. This comprehensive reconstruction is essential for accurate analytic entries.
3. Collecting Detailed Information for Analytics
For each analytic entry, the system collects crucial data to enhance the quality of analytics visuals and graphs. This data includes:
• User Agent: Identifies the browser or client software used.
• User Device: Specifies the type of device making the request.
• Geolocation: Determines the geographical location of the user.
These details are integral to creating insightful and precise analytic entries.
4. Handling IP Addresses
The system manages IP addresses based on their version:
• IPv4: The entire IP address is used.
• IPv6: The address is truncated to the first 64 bits.
This standardization aids in consistent tracking and analysis.
5. Validating Against IAB Parameters
Each analytic entry is validated against several IAB (Interactive Advertising Bureau) parameters to ensure accuracy and relevance:
Byte Range Request Check: Ensures the request is not just a small byte range (e.g., 0-1 bytes).
Unique Listener Identification: Generates a unique hash combining the IP address and user agent. This hash helps in identifying unique listeners.
Repeat Listener Check: Determines if the same device has accessed the content within a rolling 24-hour period. For instance, if an entry was recorded at 1 PM, the system checks for matching entries from 1 PM the previous day to the current time.
Download Completeness: Verifies that enough data has been downloaded to cover at least one minute of audio, or the entire audio file if it is less than one minute long.
Server Provider Exclusion: Ensures downloads do not originate from known server providers such as Google Cloud or Amazon Web Services, these are cached automatically on a daily basis.
Blacklist Verification: Checks against a blacklist of IP addresses known to be associated with malicious activities.
User Agent Validation: Confirms that the request comes from a valid user agent and not a bot.
6. Storing Validated Entries
Entries that pass all IAB validation checks are inserted into the Captivate analytics storage engine. This storage is optimized for quick retrieval and detailed analysis, ensuring that only high-quality data is used for analytics.
7. Automated Traffic Spike Monitoring
Captivate’s system performs a self-audit every 24 hours to monitor for any unusual spikes in analytics traffic. If significant anomalies are detected, the system alerts the team through established communication channels for further investigation.
This automated monitoring ensures the integrity of analytics data and helps identify potential issues promptly.
By following these detailed steps, Captivate ensures its analytics data is accurate, reliable, and useful for generating insightful reports and visualizations. This rigorous process helps in tracking listener behaviour and in maintaining the quality and trustworthiness of the analytics provided to clients.