Week 4 Discussion Topic: Planning Data Collection

Goal for today: Plan out and practice doing interviews for the passwords assignment, which is due two weeks from today.

Situation

Imagine you work for a company that makes a wide variety of technologies (e.g. Microsoft), and is designing a new password manager. You need to figure out what the company should focus its time and attention on building. The company has asked you for user-driven design ideas and suggestions. You need to do some data collection — collect data about users and uses of password managers to try to help figure out how to make a good one. Then the company would like you to create a prototype interface.

Your goal in today’s class is to come up with a plan for collecting data. Who will you talk to? What will you ask them? What will you do to make sure it is ethical?

Contextual interview with a user

The assignment asks you to do 2 interviews: one with a user of a password manager, and one with a non-user. You’ll probably want to plan them separately, assuming that you have different questions for the two groups.

For the password manager user, you’ll want to plan a contextual inquiry: go where the user is, watch what they do, and then talk to them, using the master/apprentice model.

See if you can come up with a way to watch them use their password manager. Ideally, you’ll want to watcht them do multiple things with the password manager – e.g. log into multiple websites, change their password on a website, add a new website/password to their manager, login using 2FA using the password manager, etc. You (obviously) won’t be able to observe ALL of those things at once. But can you come up with something to look for or ask for that will allow you to observe more than one of them? (Would something like role play help here?)

Make a detailed plan. Your plan should answer these questions:

  • Who? Whow will you talk to? How will you find someone like that?
  • What? What will you ask them to do?
  • Where? Where will you watch this?

Write down your plan. Be sure to be specific: if there are things you want to know, write them down as questions. If there are categories of things you want to look for, create a spot to put the answers so you know to look for them.

Ethics

You will need to put some time into thinking about the ethics of this data collection. Start by thinking: What are the ethical issues that you see arising?

Here is a good place to start:

  • Informed consent: How will you 1) make sure the participant is fully informed and understands what they are being asked? And how will you document that they have agreed (consented) to participate?
    • What happens if they withdraw or don’t want to answer something in the middle of the study?
  • Minimal Risk: What are some potential risks that the participant might face? For example, are you asking the participant to do anything dangerous?
    • This is a study about passwords; are you going to learn your participant’s passwords? That seems like a big risk to them. What can you do to minimize that risk?
    • You should be able to do this study in a “minimal risk” study. Minimal risk is no more risk that normal, everyday life. Justify why your study is or is not minimal risk.
  • Deception: Does this study involve deception? If so, do you have a debriefing and a reaffirmation of consent planned?
  • Recruiting: How will you recruit your participant? Can you make sure that you aren’t coercing them or unduly influencing them to participate in the study?
    • If you are recruiting someone you know, does your existing relationship affect their willingness to participate?
    • Are you ensuring that a wide range of people potentially benefit from having their voices heard through your study?
  • Act with Integrity: Make sure everything you do has integrity.
    • Do you have any conflicts of interest? Does the participant?
    • Are you using AI for anything? Have you disclosed that fact? Is everyone involved OK with that?
    • How will you handle it if you learn of something illegal?

As you think about these questions, go back and revise / edit your plan.

Non-user interview

Next, think about the non-user interview portion of the assignment. You want to talk to a non-user of password managers – someone who uses password but doesn’t (yet) use a password manager. What might be different about this?

  • What is your goal with this? Why talk to a non-user if you are creating a new tool for users?
  • Can you do a contextual inquiry for this portion?
  • Do you need to ask different questions? Different focus?
  • Are the ethics different? Is the risk higher or lower for non-users than for users?

Write down a second plan – or a modification of the existing plan – for this 2nd interview.

Swap and provide feedback

Find another group, and swap your written plans. Use this opportunity to get feedback from others. What do they like about your plan? What issues do they see? Do they have any suggestions that might make it better?