Disclaimer:The thoughts expressed in this case study are my own and do not necessarily reflect those of LaunchDarkly and their business. The designs, content and any other assets belong to the company in mention and should not be copied or re-used in any way. Some information and screenshots have been redacted for privacy and security purposes.

  • RoleStaff Product Design & Research

  • DurationQ3 22 - Q1 23

  • TeamLaunchDarkly

Reduce complexity around flag creation to enable users to get up and running with less effort, regardless of their experience.

Feature flags are at the heart of every feature in the LaunchDarkly platform.

In the fast-paced realm of software development, the need to swiftly and confidently deploy new features drives companies to rely on feature flags to help them innovate and release software safely.

LaunchDarkly’s value proposition centers around feature flags, which contribute to over 85% of total revenue and are the core of the platform. Flags enable companies to navigate uncertainty, make informed decisions, and mitigate risks effectively. With feature flags, software companies can iterate rapidly, experiment, and deliver customer-centric solutions with confidence, ensuring seamless product evolution in an ever-evolving landscape.

Experience launched in 2014
Experience launched in 2014

“Creating flags can feel very daunting to our employees and unnecessarily repetitive to intermediate and advanced users.” – customer quote

Early iterative sketches

Goals

Create a more streamlined and understandable experience that will help users quickly reach their value moment

  • Make flag creation easier, lower the barrier to entry
  • Use case and outcomes driven, over settings (focus on the why, not the how)
  • Flexibility with scale (flags are not all the same)

Who is our targeted user?

All users, there is no value in the LaunchDarkly platform without first creating a flag.

Challenge overview

Feature flags are risky

While leading our annual planning workshop for, this is the project that our team identified as the highest priority.

Workshop Kickoff Risk Analysis
Workshop Kickoff Risk Analysis

Flags serve as the mechanism developers utilize to incorporate their code. This enables them to effectively manage the rollout of new features, experiments, or system configurations, thereby reducing risks associated with deploying code to production environments.

Total Flags Created (90 days)
Total Flags Created (90 days)

Creating flags in LaunchDarkly is not only the main user journey, but also their first impression with the product. Despite the platform’ maturing in feature set and thousands of flags being created daily through both the UI and API, the system has remained unchanged since its 2014 launch. This has resulted in a relatively unintuitive flow with a high cognitive load. Users are required to grasp numerous new concepts and navigate through several unnecessary inputs, overwhelming beginners and feeling repetitive to intermediate and advanced users alike.

Key findings - conversion and time to create a flag
Key findings - conversion and time to create a flag

While the time it takes to create a flag is fairly minimal (2 minutes), the lack of guidance in the flag creation process often leads to misconfigured flags being created and ultimately archived and then recreated (7%) before ever being used. This also leads to many folks not discovering the advanced capabilities that a feature flag can provide them, because they’re overwhelmed just trying to meet the minimum requirements to create one in the first place.

Key problem with existing solution
Key problem with existing solution

Solution overview

Feature flags reduce risk

Reduce complexity while lowering the barrier to entry

By focusing in on the 87% of flags created primarily to stub out code, we created a solution focused around user intent rather than all the settings and knobs to be twisted. Leveraging data analysis, user behavior insights, dogfooding and interviews, we identified some key patterns and themes on how users are leveraging flags and releasing software. The result translated into a streamline experience and ready-to-use templates that were configured based on user intent.

80/20 focusing on the primary workflow
80/20 focusing on the primary workflow

This approach enables users to provide a name flag, select a template, and start with a much more maturely configured flag with minimal effort. This streamlined process not only enhances flexibility at scale but also sets the groundwork for future reusability through custom templates, stepping into a new era of efficiency and ease in feature flag management.

User journey mapping

Opportunities

HMW… Reduce complexity around flag creation to enable users to get up and running with less effort.

  • …make the simple things simple
  • …clearly show value to all type of users, from Day 0 onward
  • …leverage reusability during flag creation
Kickoff workshop defining goals, hmw and exploring solutions
Kickoff workshop defining goals, hmw and exploring solutions

Value prop

  • Creating flags will be the easiest thing to do.
  • Flag creation will be focused on out come or the user’s use case.
  • Templates will introduce reusability to the platform
  • Flexibility with scale (flags are not all the same)
Value prop
Value prop

API Considerations

Building products for developers means you have to go beyond the UI and provide solutions that fit within their existing workflows.

We have an API first product principle. This feature was built to have API and UI parity. Over 60% of flags are created by our API, so it’s very important that those users also get the same set of capabilities as the UI experience.

API first product principle
API first product principle

Guiding principles

Effective design principles serve as guiding lights, aiding in decision-making amidst design dilemmas. These principles transcend clichés like “prioritize accessibility” or “aim to delight users.” Instead, they delve into nuanced aspects of design, offering clear insights that facilitate practical application.

  • Unlock complexity: Elevate users from beginner to adept
  • Meet users where they are: Build tools to foster reusability
  • Require the bare minumum: Reduce short-term memory load
  • Comprehension over configuration: Avoid making users perform mental gymnastics to understand our product.
Effective design principles
Effective design principles

The role

Roles & responsibilities

I’m only as good as those around me

I believe that the strength of a team defines individual success, and in this project, I had the privilege of collaborating with a mix of seasoned colleagues whom I’ve previously worked with, as well as fresh talent.

Triad: Engineering, Product and Design collaborating from the start
Triad: Engineering, Product and Design collaborating from the start

In my role as lead staff product designer, I led a collaborative effort alongside my product manager and engineering manager, shaping a project identified during our team’s annual planning workshop. Together, we crafted a comprehensive team roadmap, delineated research objectives, and established key success metrics. My responsibilities extended beyond design execution to include orchestrating a research plan, conducting collaborative workshops with stakeholders across the organization, and iterating on data-informed concepts, culminating in the creation of mid and high-fidelity mockups.

Blind contour drawing team building exercise

Throughout the project, we actively engaged with internal and external stakeholders, assuming accountability to director-level leadership for the project’s success. This initiative was marked by its high visibility and impact, making a significant contribution to our organizational objectives. Additionally, I provided valuable guidance and mentorship to another mid-level designer, fostering a culture of growth and collaboration within our team.

My process

What I did

Learn, build, ship, learn, iterate, repeat

01 Project kickoff

Build together win together

My ideal process is to start the kickoff of a new project, with some type of design led, team workshop. I believe it helps everyone begin to start speaking the same language, answer a lot of greenfield questions and get the team excited about what it is we are about to build. I encourage other designers on my team to do the same.

Assumption mapping exercises - focused in the upper right
Assumption mapping exercises - focused in the upper right

We started the workshop with an assumption mapping exercise, identifying low-risk, high-confidence assumptions for validation as it related to creating flags within the platform. This allowed us to identify areas that we had high confidence and low risk around, which enabled engineers to start putting together tech specs and sizing. While we spent more time in discovery around the high confident high risk areas.

Key themes mapped to risk and confidence
Key themes mapped to risk and confidence

The hypothesis

A more streamlined experience will help more users reach their aha! moment and lead to:

  • An increase in confidence and trust
  • Increased utilization of advanced flag targeting features
  • A decrease in accident flag mis-configurations
  • Customer maturity
Concept sketches to evaluate and validate further.

02 User research

Data informed decisions

Throughout the research process, we collaborated closely with engineers, providing ongoing support for their development work while simultaneously testing and iterating on potential solution concepts.

Phase 1 internal study and concept iteration (1 week)

Following the workshop, we initiated the first phase of our research, which involved conducting an internal survey. The primary goal was to gain insights into user flag creation habits and frequency. This process not only validated some of our assumptions but also prompted us to refine others based on the gathered data.

Phase 1 Data Analysis
Phase 1 Data Analysis

Phase 1 themes

  • Often users did not make any changes to the default values provided.
  • Over 80% of flags created are basic boolean flags. This proves to be true for both the API & UI
  • A strong desire to be able to setup their own default values, or more importantly create templates or define naming convention (aka governance)
Day 1 - 3, customer maturity model
Day 1 - 3, customer maturity model

Phase 2 external customer validation (2 weeks)

We aimed to understand users’ flag creation habits and frequencies, uncovering various insights:

  • Non-developers or less-technical users feared making mistakes with flags.
  • Some customers lacked maturity to utilize advanced features.
  • Customers had internal documentation on flag creation.
  • Predictable patterns in flag creation emerged, suggesting template options.
  • Interest in a streamlined flag creation approach aligned with our principle of minimal requirements.
Co-authored PRD and flag template proposal
Co-authored PRD and flag template proposal

Concept validation

While conducting internal research and actively receiving results, we simultaneously iterated on concepts to be tested with external customers. The following were experimental wireframes and feature names used for early customer testing and iteration of core concepts that needed to validate.

Concept - Meet all users where they are
Concept - Meet all users where they are
Concept - Flags on demands
Concept - Flags on demands
Concept - Min & Max
Concept - Min & Max
Concept - 1 Click Flag Creation
Concept - 1 Click Flag Creation
Concept - Dynamic Flag Configuration
Concept - Dynamic Flag Configuration
Concept - Customized curation
Concept - Customized curation

Research outcomes

The research yielded concept validation that helped us focus our cycles. A co-authored PRD with T0 project decisions, a proposed list of flag templates and their configuration. Finalized the critical user journeys and jobs to be done. Probably most importantly is provided alignment on a shared vision with the project triad.

T0 project decisions

  • Comprehension over configuration
  • The experience should be the same, whether you are reading our docs, reviewing APIs or working within the UI.
  • Flag templates are reusable and easily customizable for increased flexibility.
Critical user journey

03 What shipped

Ship, learn, iterate & repeat

We knew because of the nature of this project there would be a lot of strong feelings on how this should work, both internally and externally. Which is why we had a goal to start dogfooding this in 4 weeks and into our first EAP in 7 weeks. We had already factored in time to learn and iterate. Not all of the concepts we tested made it to production, but the following is what was shipped.

Working closely with my engineering counterparts, we were able to identify similar patterns to reuse that would save on engineering effort.

New flag creation experience

Redefining how users conceptualize flags and streamlining the process. This shift allows users to create a flag with just two clicks, while also guiding non-technical users through a less intimidating workflow, minimizing perceived risk.

We really wanted to focus the experience around the intent on how a user is going to use the flag. So we framed each section as a question.

Iteration 1 - Select template - EAP
Iteration 1 - Select template - EAP

Clarity around steps and what it takes to create a flag was well received.

Iteration 1 - Configure flag - EAP
Iteration 1 - Configure flag - EAP

Not enough density – developers love their UIs to be packed with information.

Iteration 1 - Settings & deploy flag - EAP
Iteration 1 - Settings & deploy flag - EAP

Although the intent here was to make suggestions on naming their flags, users did not like naming the flag at the last step.

Out of the box flag templates

This aspect of the solution was pivotal, requiring attention and precision to ensure its success. If we got this part wrong, none of the other new concepts mattered.

The enhanced flag creation experience instills the product with a definitive perspective on flag templates and their utility. Given that flag creation serves as the primary workflow and initial interaction point for users, it’s essential that the templates provided value for handling complex use cases.

Iteration 1 Exploration - Curated and custom templates - EAP
Iteration 1 Exploration - Curated and custom templates - EAP

By analyzing data, user behavior, conducting interviews, and collaborating with docs, marketing, solution architects, and ecosystem team members, we developed a proposal outlining common templates. We presented this proposal to executive leadership to garner buy-in and alignment.

Iteration 2 - Bringing flag name into step 1 - EAP
Iteration 2 - Bringing flag name into step 1 - EAP
Iteration 2 - Bringing back the LaunchDarkly app chrome
Iteration 2 - Bringing back the LaunchDarkly app chrome

In the current form users did not find the summary of the flag useful.

Iteration 2 - Dynamic flag configuration
Iteration 2 - Dynamic flag configuration

Stripped everything down to an overly simplistic modal (did not scale, hid too much information)

Iteration 3 - Stripped down, min information required
Iteration 3 - Stripped down, min information required

Improved the modal, brought back the steps

Iteration 4 - Bringing back clear steps
Iteration 4 - Bringing back clear steps
Iteration 4 - Configure your flag
Iteration 4 - Configure your flag
Iteration 4 - Settings & deploy
Iteration 4 - Settings & deploy

Baby steps to governance

In addition to developing innovative templates, we enhanced a feature known as flag defaults to seamlessly incorporate these templates. This update empowered administrators to tailor the templates according to their organization’s specific needs, effectively resolving previous concerns regarding template configuration. By establishing a standardized baseline configuration while offering the flexibility for customization, we successfully addressed the diverse requirements voiced by our users.

GA'd: Step 1 - Choose your intent
GA'd: Step 1 - Choose your intent
GA'd: Step 2 - Tell us what this flag is for
GA'd: Step 2 - Tell us what this flag is for
GA'd: Step 3 - How should this flag be configured
GA'd: Step 3 - How should this flag be configured
Introducing governance for customer templates
Introducing governance for customer templates

Impact and lessons learned

Understanding the impact

Learn, iterate, ship, repeat

High adoption rate of using out-of-the-box templates over the proposed “custom flag” option. The hypothesis being it would increase confidence and trust, also an increase in more advanced flag targeting configuration. We knew that there would be a ramp-up period for users to gain trust and understanding with these new templates.

Tracking key metrics - How many flags are being created with a template?
Tracking key metrics - How many flags are being created with a template?

Of the ~32k flags created during EAP, 40% were created as custom flags, rather than utilizing a template. This exceeds our goal of >20%

Tracking key metrics - How many times are templates being overwritten at creation?
Tracking key metrics - How many times are templates being overwritten at creation?

Of flags that used a template, only 5% of flags had their template default overridden. Which exceeds our goal of <10%. The assumption was that most overrides would be around variations, which was validated by 48% of overrides being flag variations

Tracking key metrics - Are flags being created then archived decreasing?
Tracking key metrics - Are flags being created then archived decreasing?

Providing clear and repeatable guidance would significantly reduce the necessity to archive flags within one hour of their creation. The hypothesis being a streamlined process would minimize the likelihood of errors and ensure flags are created properly.

Lessons learned

Through this journey, I learned valuable lessons in tackling change aversion and navigating the complexities of altering long-established systems. Approaching these challenges from a product design, UX, and human behavior perspective has provided key insights:

Acknowledge Change Aversion
Recognizing and empathizing with users’ reluctance to change is crucial. Gradual implementation and clear explanations can help alleviate apprehension.

Respect Legacy Systems
Honoring the history and reliability of existing systems is essential. Integrating updates in a way that respects the legacy infrastructure fosters user acceptance.

Balance Diverse User Needs
Striking a balance between mature organizations and newer entities requires understanding and catering to diverse user preferences. Customizable features and adaptive interfaces can accommodate varying needs.

Our findings revealed that flag creation alone cannot address the complexities of advanced flag targeting configuration, which ultimately depends on the customer’s maturity level. This insight was gleaned from user behavior and the broader landscape of flag utilization, indicating that 75% of created flags rely on basic functionality. Customers are generally content with this level of functionality and have yet to reach a maturity level to fully exploit advanced features.

Other considerations

What made it to the backlog

Thinking deeper and longer term

Based on our research findings, we identified significant opportunities to expand beyond the initial project scope. While we did explore potential solutions for these opportunities, we made the decision to prioritize them in the backlog to ensure alignment with our project deadline. This approach allows us to dedicate appropriate time for necessary validation before implementation.

Putting flags into “draft” state

Proposed solution: Fix a long standing problem that would allow users the ability to change things like variation type and key before flags start receiving evaluations.

Creating flags via CLI

Proposed solution: Developers love using the CLI, allow users to install LD, sign up and create flags with command line.

LaunchDarkly CLI prototype
LaunchDarkly CLI prototype

Custom templates

Proposed solution: Extend the concept of templates to custom templates. Proper RBAC per template, introducing organizational best practices, enforcements and naming conventions.

GitOps FFaC

Proposed solution: Creating FFaC and other GitOps type workflows (forking a flag, versioning and draft states)

Other considerations - FFaC
Other considerations - FFaC