Assignment: Duo MFA w/ PIN
For your first assignment, you are to do a “quick and dirty” evaluation of a new(-ish) security technology: Duo MFA with PIN code.
UW-Madison uses Duo multi-factor authentication to log in. Whenever you log into a university system, you not only have to enter your password (something you know, factor 1), but you also have to prove that you have your mobile phone (something you have, factor 2). You prove that you have your mobile phone by having an app on the phone (Duo), and when you try to log in, the app asks if it is really you logging in.
Recently, Duo made a change to their app (or, rather, UW-Madison turned on a feature that Duo has had for a while now) – instead of just clicking a button to say “Yes, it is me logging in”, the login page displays a 3-digit random number, and you have to enter that number correctly in the Duo app. Duo and UW-Madison claim that this makes the login system more secure and protects against attacks better.
Because this makes logging in more difficult, they also simultaneously made a second change. Previously, Duo would rememeber you for 1 day, and you would have to re-do the second factor (click the “Yes, it is me logging in” button) if more than 24 hours had gone by since you last did it. UW-Madison changed this timeframe to 7 days, so you only have to enter the 3-digit number once a week instead of once a day.
For this assignment, your job is to quickly evaluate the usability of this change. How well does this change work for users of the system? Does it make it easier to log in, or harder? Are users OK with this change, or are they upset and complaining? Are there times when users cannot successfully log in when they would have been able to in the past? What changes do users have to make to use this system? Overall, if we assume that it does provide more security, is the cost – how much harder it is for users to log in – worth it?
Collect Data
To answer these questions, you will do a quick and dirty usability evaluation.
The goal of this assignment is to get started learning about technology from other people. One way that we do this is by observing users using technology to accomplish some goal or task. We want to understand how they really use the technology – not just how they use it in a lab or in artificial circumstances – so we need to go to them.
-
You should find someone who is a user of the UW-Madison Duo MFA. This person should be someone that you do not already know much, and definitely someone that you’ve never talked to about security or logging in before. I recommend asking a friend-of-a-friend or a random person in a lobby.
-
Arrange to watch them use that technology. Go to their home; meet them in a coffee shop; use screen-sharing technologies. It doesn’t really matter, as long as you can clearly see what they are doing and how they are using the technology. Since this is a login technology, you might need to ask them to logout1. Since Duo only asks them to use the 3-digit PIN once per week, you might need to wait until they will be asked, or ask them to log in via a different device (e.g. library computer) than they normally do.
-
Ask them to use the technology to do something that they care about. It is up to them to choose what they do: check their email, log into Canvas, fill out a timesheet, etc. The technology use shouldn’t be instantaneous; it should take them a minute or a couple of minutes to do what they are trying to do. But don’t give them a task; let them come up with one. This will help you see what they do and how they use the technology. Note: No one sits down “to log in”; instead, they sit down to do something else (access Canvas, fill out a timesheet, lookup classes, etc.) and are asked to login as part of doing that something else. Don’t ask them to go log in. Instead, you will need to ask them what kinds of things they do that need logins, and then ask them to do that.
-
Watch them do it. Don’t ask any questions. Definitely don’t tell them what to do. Don’t say anything. Don’t answer questions (at least, until you are done with the assignment). Don’t take notes. Just watch.
-
Ask yourself: What did you learn? What did you notice about how they use the technology? (You can take notes once you are done.)
-
Only after they are done doing what they set out to do, you can start talking to them about this and asking them questions. Ask them if this was a pretty typical use of the login technology, or if something was different about this time. Ask them about their opinion about the technology – do they like it or not? Ask them if there times the technology works well for them, and if there are times when the technology doesn’t work well for them.
The whole interaction shouldn’t take too long – only 5-10 minutes with your participant. (Hopefully; logging in shouldn’t take long, but sometimes it does…)
Evaluate
Once you are done with the participant, use these questions to reflect on what you learned and evaluate the technology.
-
What was the user trying to do? Identify the goals the user was trying to accomplish. Did they accomplish them during your observations?
-
What technology was used, and what did they do with it? Describe the technology. What make/model was used? They the user use any other technologies? How did the user use the technology?
-
What worked well? What worked poorly? Did the technology(ies) help or hinder the user? How? What part(s) of the technology worked? What part(s) didn’t?
-
What surprised you about the way they used technology? The user is not like you. How did they use the technology differently than you would have, or differently than what you expected? Note: if you weren’t surprised by anything, you probably weren’t paying close enough attention to what the user was doing.
Write up a short (1 page) report that describes what you learned and answers these questions. Submit it on Canvas.
Due: Tuesday, Sept 16 before class
-
While logging into wisc.edu’s NetID service is pretty easy, it turns out logging out is not so easy.
This page has more information about why. Currently, the easiest way is to ask the user to visit https://login.wisc.edu/logout, and then ask them to close and re-open their web browser. This process will not log them out of any service they currently have open in another window, but hopefully will be enough that they have to login to somewhere they go to. ↩