Jot notes from #meshU on web metrics

Here’s a dump from the monitoring talk at meshU. Slides are available. Again, formatting is a mess, this time due to OpenOffice.

Watching websites
Presentation by bitcurrent

• Used to work for a company called coradiant
• Roughly one third of the audience monitors their website for uptime
• Startup 101: New idea -> Execution -> Feedback (did it work?) -> Money left? Yes: New idea. No: End.
• Do we understand our users? Is it easy and intuitive? Unfortunately usually this led to trial and error
• Internet makes us make mistakes faster, which isn’t necessarily bad
  • Most startups don’t succeed in doing what they set out to do
    • Amazon was just a bookstore
    • eBay sold Pez
    • F5 made hurricane modeling software

• Mistake speed is critical for the “money left?” stage

• Different sites need different metrics
  • Collaborative sites? Check comment rates, how engaged they are, how loyal they are
  • Ad supported? Click rate, buy rate, continued tweaking of campaigns
  • Which are you?

• Want to see what users do what we want
  • Enrolment, purchases, invitations, stickiness

• Want to see if the app is fast and reliable
  • Uptime, latency, SLAs, functionally correct, well maintained

• Understand our visitors
  • Intentions (we know what they want to do)
  • motivations (we know why they came to our site)

• Want to see if the app is easy to use
  • Easy to learn, clear and fast to use

• Our eyes: Synthetic tester, analytics receiver, passive capture, survey site
• Four big questions: What did they do? Could they do it? Why did they do it? How did they do it?
• What did they do
  • Domain of web analytics
  • Acquisition, usage and referral are important
  • Acquisition: what attracts them best?
  • Referral: what affiliates did they go to and why?
• Previously this was done by parsing access_logs, now people use AJAX to feed data back to the server. Example: Google Analytics.
• Set up goals and funnels for Google Analytics if you use GA
  • Much better to see what sites give you better conversion rates rather than bulk of traffic
• Don’t just look at numbers, look at comparisons to your site’s previous rates as well as to othe rsites
• Check out Clicky.com which gives better individual user look
• Also check out Mint
• Five levels of metrics: Check the slides, useful info but too much to copy
• These client side analytics have a problem with expert users. That is, users very familiar with your site. They tend to click through too fast, before the javascript at the end of the page has finished loading.
• What could go wrong? Check the slide for this too.
• Rather than “Oh? Something broke? We can give you a credit.” use “Excuse me, we’ve noticed that a proxy a x.y.z.a is interfering with the use of our site, here’s a credit and by the way your IT team can fix this by…”
• Use pingdom.com for testing
• Passive capture is the most accurate yet painful way to debug. Wireshark is free, TrueSight AIM is expensive but cool.
• TrueSight can tell you things like how many user sessions were interrupted by an outage
• Found that synthetic tests are often much worse than TrueSight, don’t show you the outliers very well
• Voice of customer: example of hotel booking site. Low conversion rate. Popped up survey, survey said that most users were just browsing to see the price and whether it was worth travelling at that point. Hotel booking site created new sidebar saying “Just browsing? Sign up for price drop alerts.” Conversions went way up.
• Also interesting: Clicktale. Lots of mouse and keylogging.
• Useful graphs: 80, 85, 90, 95^th percentile for load time over transactions per second
• Problem: AJAX! Standard off the shelf tools don’t see a prolonged pipeline containing 10 results as 10 results. Sees 1 very long request.
• Question period: Only question is security and how off-the-wire sniffers can read SSL. By giving them the private key.