Usability evaluation of travel booking sites

I worked with a partner to conduct a usability evaluation and expert review of two travel booking websites. We conducted usability testing with 8 participants on CheapOair.com and 8 participants on Kayak.com from June 24-28, 2020.

Methodology

  • Between-subjects study (participants tested CheapOair or Kayak; not both)
  • Evaluations were conducted using CheapOair's and Kayak’s live, public-facing websites
  • Sessions were conducted both in-person and remotely using Zoom
  • Two moderators followed the same facilitator's guide and used an interrupted task-based protocol

Participants

My partner and I conducted usability evaluations with 16 total participants:

  • 8 CheapOair evaluators; 8 Kayak evaluators
  • 9 men; 7 women
  • Various ages (ranging from late 20s to mid 60s)
  • All had traveled in recent memoryThose who had traveled since the COVID-19 outbreak in March 2020 indicated that they drove rather than flew
  • Had not used the site they were evaluating in past 3 months

Many participants mentioned familiarity with travel booking websites:

  • For flights: Expedia, Google Flights, Hipmunk (now defunct), Hopper, Momondo, Orbitz, Priceline, Skyscanner, specific airlines (Allegiant, Delta, Southwest)
  • For accommodations: Airbnb, Priceline, VRBO
  • For rental cars: Turbo

 

Rating severity of usability problems

The following 1-3 rating scale was used to rate the severity of usability problems:

1 = Cosmetic problem only: need not be fixed unless extra time is available on project
2 = Minor usability problem: fixing this should be given low priority
3 = Major usability problem: important to fix, so should be given high priority

 

Rating effort to address usability problems

The following 1-3 rating scale was used to rate the perceived effort required to address usability problems:

1 = Minor effort (e.g., minor fixes to styling or simple interactions)
2 = Moderate effort (e.g., changes to page structure or more complex interactions)
3 = Major effort (e.g., incorporating new data sources, significant changes to IA or interactions)

Expectations

In general, researchers expected Kayak to outperform CheapOair. A few reasons include:

  • CheapOair has many calls to action on their homepage; Kayak has two
  • CheapOair displays alternate dates and airports in search results by default; Kayak sticks closer to user’s entered search parameters
  • CheapOair's search button does not provide obvious user feedback; Kayak's search button provides subtle (but more obvious) feedback
  • CheapOair has duplicative options for limiting flights by number of connection(s); Kayak provides clear separation between search parameters and filters

Findings

  • In reality, tasks across the two sites proved equally difficult for participants
  • Uncovered nearly the same number of usability issues with Kayak (20) as with CheapOair (21)
  • Participants were most comfortable performing tasks when the steps necessary to accomplish the tasks were similar to other travel booking sites they had used in the past

© 2022 Kristen Marie Byers