Overview
My Role
From January to May 2023, I led a team of 5 product designers responsible for improving the user experience of picoCTF. My role included:
Defining the design strategy: I closely collaborated with stakeholders through weekly stand-ups to define the design direction for picoCTF, taking into account the needs of our target audience, business objectives, and the competitive landscape of the cybersecurity education market.
Developing the design system: I created a design system with a visual component library, color scheme, and other design elements to ensure consistency throughout the platform.
Leading the design team: I managed the team of designers, ensuring that all design work aligned with the overall design strategy and met high-quality standards.
Collaborating with developers: I worked with the tech lead to create React components using Storybook, effectively integrating our design system into the larger tech architecture of the platform.
Problem
Are you interested in cybersecurity but don't know where to start? With the highly technical nature of the subject, the field can seem overwhelming. This is why picoCTF exists, one of the largest online cybersecurity education and competition platforms with over 36,000 users. CTF stands for “Capture The Flag🚩, “a type of cybersecurity competition where participants compete to solve challenges. Designed for high school and college students interested in cybersecurity, picoCTF provides resources and support to help them learn.
However, by conducting user research, we found that many users feel unmotivated when using picoCTF. The main reasons are knowledge barriers and a lack of structured guidance and progress tracking. New users often struggle to navigate the platform and find the right resources to help them learn. Without a clear sense of progress, they may lose motivation and give up on learning about cybersecurity.
User Interviews
How might we leverage intrinsic and extrinsic motivation to promote and sustain users’ interest in learning cybersecurity?
Solution
To address the identified problems, our solution is to redesign the picoGym, which is the practice space for users to solve challenges. The redesigned picoGym will provide a structured learning experience for users to explore cybersecurity topics. It helps remove knowledge barriers and provides guidance. It also features a more engaging user experience, including game-like elements to increase motivation.
Guided Learning Modules
At pico, our practice space is called “picoGym,” so we name the guided learning modules “workout plans” to keep this fitness metaphor. These plans provided a structured way to guide users through various cybersecurity topics and can be tailored to different skill levels.
New Reward System
It allows users to earn badges when completing all the challenges within a workout plan. By providing clear goals and rewards, the system encourages users to engage more deeply with pico. It also allows users to track their achievements, providing users with recognition for their efforts.
Personal Progress Tracking
Users earn points by completing challenges, and more points allow them to reach higher levels. We implemented a daily contribution chart, similar to GitHub. It displays a calendar that shows a visual representation of a user’s activity over time, with each day color-coded based on the user’s activity level. This progress tracker provides incentives for users to continue learning and practicing.
Improved UI & Organization
We improved the UI and organization of picoGym, with a focus on creating a clean and modern look, by updating color schemes and other design elements. We redesign the challenge list from a modular design to a table design. The table layout allows for a more organized presentation of challenges, with less eye movement required for user to navigate. We made the website more accessible by checking the WCAG to improve accessibility for all users.
Research
Competitor Analysis
We first conducted a competitor analysis to understand the competitive landscape better and identify potential opportunities and gaps in the market.
| Feature | PicoCTF | CyberStart | Hacker101 | CTF365 | TryHackMe |
|---|---|---|---|---|---|
| Student-focused | ✓ | ✓ | × | × | ✓ |
| Gamification | ✓ | ✓ | ✓ | ✓ | ✓ |
| Comprehensive resources | ✓ | ✓ | ✓ | ✓ | ✓ |
| Video tutorials | × | × | ✓ | × | ✓ |
| Progress tracking | ✓ | ✓ | ✓ | × | ✓ |
| Structured learning paths | × | ✓ | × | × | ✓ |
| Personalized learning | × | ✓ | × | × | ✓ |
| Events/competitions | ✓ | ✓ | × | ✓ | × |
| Mobile app | × | ✓ | × | × | × |
| Community/forum | × | ✓ | ✓ | ✓ | ✓ |
| Accessibility options | × | ✓ | × | × | ✓ |
We found that
Most cybersecurity education platforms primarily cater to a general audience and lack a specific focus on high school and college students interested in cybersecurity as a career path.
Many platforms offer a wide range of challenges and competitions, but they often lack a structured learning path, making it difficult for beginners to navigate and progress effectively.
The gamification aspect of cybersecurity education platforms is generally present but may not be tailored to the preferences and engagement levels of high school and college students.
Progress tracking and performance analytics are not consistently provided, making it difficult for users to monitor their growth and identify areas for improvement.
Therefore, we aim to develop a comprehensive cybersecurity education platform that caters specifically to students and novices. Our target users will benefit from a structured curriculum, tailored practice challenges, and the ability to track their personal growth. We aim to provide a user-friendly and engaging learning experience that empowers learners to develop essential cybersecurity skills and knowledge at their own pace.
Stakeholder Map
We created a stakeholder map that help identify and prioritize key stakeholders involved or affected by the project.
Here are some insights:
Diverse User Base: We have a diverse user base. This suggests that picoCTF has the potential to cater to a wide range of learners with varying levels of expertise and interests in cybersecurity.
Industry Engagement: The presence of Cybersecurity Professionals and Human Resources as stakeholders indicates the potential for industry engagement and mentorship opportunities within the picoCTF platform. This can provide learners with real-world insights, networking opportunities, and access to expert guidance.
Financial Sustainability: As an NPO, picoCTF relies on sponsorships and funding to support its operations. It is important to consider the practicality of implementing new features or initiatives and strike a balance between technical feasibility and business viability.
Interviews & Affinity Diagramming
To better understand how users navigate the website and identify potential problems, I advocated conducting user interviews before diving into solutions. We ran 10 think-aloud studies with new users, during which they were given specific tasks within the picoCTF platform while verbalizing their thoughts, impressions, and any difficulties they encountered. We ran another 8 interviews with current users to gather their feedback, experiences, and suggestions.
The insights from the user interviews were organized through affinity diagramming. I also invited developers to participate in our affinity diagramming session, so that we could brainstorm solutions based on our findings.
Here are some findings from this session:
Learning Needs: User interviews revealed that there is a strong demand for structured learning materials and resources in cybersecurity. Users expressed a desire for clear and concise explanations, step-by-step guidance, and practical examples to enhance their learning experience.
Motivation and Engagement: Users highlighted the importance of motivation and engagement in the learning process. They expressed a desire for interactive elements, gamification features, and rewards to make the learning experience more enjoyable and encourage their continued participation.
Progress Tracking: Users expressed a need for a comprehensive progress-tracking system that allows them to monitor their growth, track completed challenges, and visualize their achievements. The ability to see their progress and improvement over time was deemed essential for motivation and a sense of accomplishment.
Accessibility and Ease of Use: Users emphasized the importance of a user-friendly interface and intuitive navigation. They expressed the need for a platform that is accessible to users of varying technical backgrounds, ensuring that beginners and novices can easily navigate and engage with the content.
Personas
We gained valuable insights and identified 2 types of users within our target audience from the interviews. One group demonstrated extrinsic motivation, being primarily driven by external factors such as rewards, achievements, and recognition. The other group exhibited intrinsic motivation, being internally motivated by their genuine interest and passion for cybersecurity.
User Journey Map
To visualize and better understand the user experience, we have created a journey map that captures the various phases users go through when engaging with picoCTF. This user journey map allows us to empathize with our users and identify opportunities for improvement.
Design
Ideation
With our research findings and personas in mind, I facilitated a collaborative brainstorming session. The goal was to generate a wide range of ideas without initially considering feasibility. We grouped similar ideas by affinity diagramming to create a structured overview of the ideas generated and their connections.
We reviewed the ideas, created storyboards of selected ideas, and conducted speed-dating sessions with users to evaluate their effectiveness in improving the user experience. We had 10 participants, all of whom were current users of picoCTF. We collected their feedback and analyzed it to identify core functionalities.
Besides user desirability, we discussed the ideas with the tech lead and the feasibility of implementation. We also talked to the product manager about the business viability. For example, creating a recommendation system for challenges within a non-profit organization like picoCTF poses challenges due to technical feasibility and business viability considerations. It requires robust machine-learning algorithms that are too complex for our developers to implement, and the associated cost of developing and maintaining the system is high.
Therefore, we had to compromise and determine the most appropriate approach that aligns with the organization's mission and resources while balancing financial sustainability. We settled on the following core functionalities:
Guided learning modules to learn and practice
Personal progress tracking to monitor learning progress
Reward badges to recognize achievements
Refined Design Goals
Our design solution is to redesign the picoGym. We refined our design goals and matched the features we hoped to implement to the goals we aimed to achieve:
Elevate Motivation
Create an engaging learning experience that sparks curiosity and excitement about cybersecurity.
Incorporate gamification elements such as challenges, levels, and rewards to motivate users to participate and progress actively.
Provide clear and meaningful goals for users to strive towards, encouraging a sense of achievement and accomplishment.
Increase Long-term Engagement
Offer a variety of learning materials, challenges, and activities to cater to different interests and skill levels.
Continuously update and expand the content to keep users engaged and provide ongoing learning opportunities.
Foster a supportive community where users can connect, collaborate, and share their experiences, creating a sense of belonging and encouraging sustained participation.
Sustain Interest in Learning Cybersecurity
Recognize that users have different learning preferences and time commitments and empower self-paced learning.
Inspire users to explore further, pursue cybersecurity as a career, and become lifelong learners in the field.
Sitemap
After the ideation phase, we created a sitemap for the picoCTF platform. The sitemap outlines the hierarchy and relationships between different pages and sections. It lets us see the overall navigation flow and content organization, ensuring a logical and intuitive user experience.
A/B Testing
We conducted a simulated A/B test to evaluate the effectiveness of the table design compared to the existing modular design of the challenge list. We wanted to assess which design variation improves user satisfaction and task completion time. Two interactive prototypes were created to reflect the functionalities and interactions of the challenge list.
Variation 2: table design of the challenge list
We recorded quantitative data of task completion time and subjective ratings. The table design demonstrates faster task completion times and higher user satisfaction scores compared to the modular design.
Variation 1: modular design of the challenge list
We also gathered qualitative feedback through direct observation, think-aloud protocols, and post-task surveys. Participants expressed that the table design provided a clearer and more organized overview of the challenges, making it easier to find challenges. Participants mentioned that the modular design felt cluttered and made it difficult to identify relevant challenges quickly. Some participants also struggled to navigate through the challenges efficiently.
These insights from A/B testing suggest that the table design is more effective in improving user engagement, task completion rates, and overall user satisfaction. Therefore, it is recommended to implement the table design as the new design for the challenge list.
Wireframes
Using the sitemap as our guide, we sketched wireframe concepts for key pages and interactions to our potential users for feedback.
Badges Design
After finalizing the wireframes, we decided to create different learning paths, which we named “workout plans” to keep the fitness metaphor as “picoGym.” In our research, we recognized the importance of gamification and providing visual indicators of achievement. We aimed to create badges that are visually appealing, easily distinguishable, and meaningful to users.
Beginner CTF
Reverse Engineering
General Skills
Forensics
Ace the Interview
Web Exploitation
Cryptography
Binary Exploitation
High-Fidelity Design
Entering picoGym
The picoGym page is the central hub for cybersecurity skill-building. Upon entering the page, users are greeted with a carousel showcasing their current workout plans. The personal progress section on the right provides users valuable insights into their daily contributions. They will gain a sense of accomplishment as they accumulate points and level up their skills. The page also includes a well-organized table of challenges for users to browse and filter based on different criteria.
Solving a challenge
The challenge-solving page offers users an interactive experience as they tackle cybersecurity challenges. This page supports users in solving individual challenges while providing the necessary information and tools to succeed. The left section is dedicated to presenting users with the challenge prompt. On the right side, users will find the webshell, which allows them to interact with the environment and execute commands. Integrating the challenge prompt and the webshell on a single page offers users a seamless and efficient experience.
Selecting a workout plan
This page serves as a gateway for users to define their own learning journey by adding a structured learning path based on their individual goals and preferences. They have the option to browse and select from a variety of pre-existing workout plans that are available on the platform. They can read the plan details, including the introduction and learning objectives to make an informed decision on which plan to start.
Beginning a workout plan
Users can start their chosen plan and embark on a structured learning experience tailored to their skill level and goal. They can start by reading the introduction, which provides an overview of the plan’s objective. Then they can dive into the learning material with a curated selection of resources, tutorials, and educational content that will enhance their knowledge. This page also presents a list of relevant challenges for users to tackle and sharpen their skills.
Earning a badge
This page is a testament to users’ dedication, skills, and progress throughout their journey. On this page, users will find a collection of badges they have earned or are striving to achieve. Each badge has a unique design, reflecting the specific area of cybersecurity it represents. As users earn badges, they will be displayed proudly on the profile, showcasing accomplishments to the picoCTF community.
UI-Kit
I created the design system with components, auto-layouts, and styles for colors and fonts. This made the design flexible, which will make it easier to work with in the future.
Nav bar
Cards
Dark
#1A1832
Button
Icons
Color palette
Primary
#3D4DD0
Moderate
#DEE2E6
Light
#FFFFFF
Secondary
#F8F8FA
Font
Evaluation
User Testing Process
With our goals for the design solution in mind, I conducted user feedback sessions with 20 participants who had different levels of familiarity with picoCTF and varied experiences with cybersecurity. Here's an overview of the process I followed:
Introduce the project goal, and establish a comfortable environment for users.
Randomly present participants with either the current or new design, and encourage users to explore the app while thinking aloud.
Ask follow-up questions and collect valuable feedback.
Let users fill out a questionnaire. Express gratitude for their participation.
Measuring Success
To quantitatively assess the success of the design, I established specific and measurable objectives based on the design goals. I then designed a questionnaire to gather data and insights that would enable me to evaluate the achievement of those objectives.
Goals
User Engagement: Measure the level of user engagement and satisfaction with the platform.
Learning Effectiveness: Assess the effectiveness of the learning materials and challenges in improving users' cybersecurity knowledge and skills.
User Satisfaction: Evaluate user satisfaction with the platform's features and functionalities.
Skill Development: Measure the growth and progress of users' cybersecurity skills and knowledge.
Recommendation and Future Usage: Determine users' likelihood of recommending picoCTF and their willingness to continue using it in the future.
Evaluation Method
Q: On a scale of 1 to 10, how enjoyable was your overall experience with picoCTF?
Q: On a scale of 1 to 10, how confident are you in your ability to solve cybersecurity challenges after using picoCTF?
Q: On a scale of 1 to 10, how satisfied are you with the user interface and navigation of the picoCTF platform?
Q: On a scale of 1 to 10, how would you rate your progress in understanding new cybersecurity concepts?
Q: On a scale of 1 to 10, how likely would you recommend picoCTF to others who are interested in learning about cybersecurity?
Results
The analysis will focus on quantitatively comparing the participants' ratings and preferences for the current and new designs of picoCTF. By examining the data, we can identify any significant differences in usability, satisfaction, confidence, and overall experience between the two designs.
On a scale of 1 to 10, how enjoyable was your overall experience with picoCTF?
• New design average rating: 9.5
• Current design average rating: 7.3
• Insights: Participants found the new design of picoCTF more enjoyable compared to the current design. The improved user interface and engaging features in the new design contributed to a more positive user experience.
On a scale of 1 to 10, how confident are you in your ability to solve cybersecurity challenges after using picoCTF?
• New design average rating: 7.9
• Current design average rating: 6.2
• Insights: Participants reported a higher level of confidence in their ability to solve cybersecurity challenges after using the new design of picoCTF. The redesigned learning materials and interactive elements helped users develop their skills and knowledge more effectively.
On a scale of 1 to 10, how satisfied are you with the user interface and navigation of the picoCTF platform?
• New design average rating: 9.0
• Current design average rating: 5.4
• Insights: The new design received significantly higher satisfaction ratings for user interface and navigation compared to the current design. The improved layout, intuitive controls, and clear information architecture contributed to a better user experience.
On a scale of 1 to 10, how would you rate your progress in understanding new cybersecurity concepts?
• New design average rating: 7.6
• Current design average rating: 6.0
• Insights: Participants reported a better understanding of new cybersecurity concepts with the new design of picoCTF. The revised learning track structure and enhanced educational resources were effective in facilitating knowledge acquisition and comprehension.
On a scale of 1 to 10, how likely would you recommend picoCTF to others who are interested in learning about cybersecurity?
• New design average rating: 9.2
• Current design average rating: 6.8
• Insights: Participants were more likely to recommend picoCTF to others with the new design. The improved overall experience and enhanced features in the new design garnered a higher level of satisfaction and advocacy.
Implementation
I embarked on the implementation phase of the solution by developing self-contained, production-ready React components specifically for the Workout Plans feature. This involved creating workout cards that effectively display relevant information about each workout, as well as designing the workout description page. These components were designed to be responsive across various screen sizes and seamlessly integrated into picoCTF's existing codebase.
To streamline the development process and ensure quality control, I utilized Storybook, a powerful framework for visualizing and testing UI components in isolation. For each component I developed, I created a corresponding ".stories" file within Storybook, allowing me to explore and test different variations and scenarios for each component in a user-friendly interface.
Furthermore, I established preliminary-type interfaces for the workout plans, laying the foundation for further adaptation and integration with picoCTF's current codebase. These interfaces can be refined and tailored to seamlessly fit within the existing architecture, ensuring a cohesive and efficient implementation.
By leveraging technologies such as React, Typescript, and HTML, as well as employing robust development practices like Storybook, I aimed to deliver high-quality and reusable components that align with picoCTF's technical requirements. This partial implementation serves as a solid starting point, demonstrating the potential for the Workout Plans feature and laying the groundwork for future development and refinement.
Example Workout Cards & Workout Plan
Although “workout plan” follows the fitness metaphor of picoGym, after communicating with our key stakeholders, we renamed it “playlist.” The term "playlist" can evoke a sense of excitement and anticipation, as users often associate playlists with personalized and enjoyable experiences. The playlist feature has now been implemented in the MVP (Minimum Viable Product) of the platform. It follows the old framework and structure, but leverages the components that we have developed to enhance the user experience.
Announcing new feature on LinkedIn
Storybook Components
Playlist feature is now launched