dgms code

Pivotal talk: data-informed product decisions

22 Oct, 2014
1 minute read

http://www.meetup.com/Pivotal-Labs-Tech-Talks/events/213571202/

Marc Abraham. Slides

  • First thing is to ask the right questions
  • Book - Lean Analytics
  • Articulate Assumptions about the benefits of your product,
  • Define Hypothesis: how will we know (in terms of hard metrics) if our assumptions are validated? What does success look like?
  • To start with, focus on one key metric
  • Behavioural plan - what do you want users to do - both to achieve user needs and business needs
  • You have a live product:
    • Is it meeting the hypothesis

  • Product retrospectives: how are we doing with the product - what does the data tell us about how well it’s going?

  • Data-Driven vs Data Informed approaches
    • Data Driven:
      • Focus on one metric - collect lots of data (A/B, Multivariate tests etc. and understand whether what you’re doing is impacting that metric)
  • Data Informed
    • Data might not tell you whether it’s a good product idea full stop: might be a leap of faith
      • When you actually have an MVP, then you can start gathering data
    • Plurality is the key: data is often just one aspect in data decisions
    • Talk - Adam Osiri (?) on Facebook’s data-informed approach
    • Data is important, bit other factors come into play: resources, competition, regulation, brand.
    • You can’t replace intuition or creative ideas with data

  • 5 key points:

    1. Focus on the right questions
    2. Data can’t replace intuition
    3. Listen to the data and act accordingly
      • This is hard
      • e.g. data might say that your idea or your variation isn’t actually effective.
    4. Build and launch with data in mind
      • Include analytics in the user stories
      • Start thinking about your assumptions and hypotheses at the beginning of the product life cycle
      • as soon as you start working on the product you should be working with the analytics team/building data in from the ground up.
    5. Be clear about your hypotheses, sample size and timing.
      • how large a sample do you need before you can claim that your results are significant.
  • Final statement “Embrace the data - don’t fear it”