Designing Curology's subscription frequency feature

Summer 2020CurologyProduct design

Overview

Designing a new referral experience within the Wizehire ATS to increase new user growth and aid monetization.

Role

Lead designer on a cross functional team.

Timeline

Summer 2019 - 8 weeks

Responsibilities

Interaction + visual design, prototyping and product thinking.

Team

  • Chaitanya Prashant - Design
  • Ben Kolde - Product Design (manager)
  • Tiffany Wu - Copy
  • Rucha Makati - UX Research

Introduction

Was there an opportunity for Curology to improve the patients’ experience in that it’d increase their sentiment to the subscription?

Would providing more flexibility to a patient actually increase the personalization of a patient toward their subscription?

Would a cohort of currently subscribing patients want to change the frequency (cadence) of their shipment arrivals?

Hypothesis

We hypothesized that out of currently subscribing Curology patients, a sizable minority of patients in a scenario might either consume their shipment too quickly, or consume their shipment at a slower pace leading to an excess of formula bottles, potentially dissuading the patient and causing them to cancel their subscription.

Objectives

Offer patients the flexibility to customize the frequency of when their personalized subscription set arrives, building a stronger connection to the product.

Increase retention and mitigate churn rates amongst patients who don’t follow the default subscription frequency set by our providers.

Key results

Relative lift of 25% retention in cohort that changed frequency at least once.

10% relative lift in cohort that stayed on default frequency, but were presented with the new feature.

Final design

Entry point - on a patients' subscription page

Selection screen

Final prototype

Understanding the problem

tl;dr: We had launched a very naive version on desktop, however, in discussions with the PM and designer, there was a lot of low hanging fruit. What was this supposed feature solving?

Initial launch on desktop

Initially launched solution, experiences on both mobile and Android

The launched version of the experience saw a slight bump up in retention. The experience was disparate across platforms however, especially mobile. While the drop-down paradigm works well in Desktop designs, it scales down poorly. Even though the app is mobile web, using native components meant that the iOS experience was similar to a date-picker, while the Android experience consisted of a simple radio-button checklist.

This also scaled to different flavors of Android reacting to make the component look visually different. Was this lack of consistency worth it?

To add to the confusion, for example, the iOS experience has additional copy at the top of the date-picker that says “shipping cadence”, while the language throughout the product utilizes frequency. This was a low hanging fruit that we definitely wanted to address.

In an informal session with 3 users who had used the feature, there was confusion in terms of what changing the shipping cadence actually does. In a clear cut case, say if the user changes frequency from 6 weeks to 8 weeks, the subscription would be updated, however, there is no status update or notification on the product. It showcased the same billing and shipment date as the old frequency, and a user would only see the new changes once this date has passed.

For example, in the previously shipped experience, if their current cadence was every 8 wks (next date is 8th September), their next shipment is being delivered in a week (screen date is September 1st), but the user changes their cadence to be every 6 weeks. There is still no status update, and the status on the action card is the same as above, only showing the current frequency until the hard “date” is passed - This was a clear low hanging fruit to address and refine through unmoderated user testing, so that we were setting the right expectation as the user changes their shipment frequency.

Opportunities

People Problem

How might we offer patients more control over when their products arrive (other than the one time reschedule)?

As I described above, different patients have differing habits. For example,

  • A patient applying Curology formula twice a day, but moisturizer only once a day.
  • A patient that applies Curology once every two days, and feels that their subscription comes in too “fast”.

There was only one way to customize a patients’ shipment, and that was to edit the date of your next upcoming shipment. Whilst very much a solution to use in a scenario where a patient has to leave town for vacation and delay the upcoming shipment by one or two weeks, does this scale? Does it offer a clear area of control over what a patients’ “schedule” looks like?

Business Problem

How might we improve retention for patients who don't follow the default subscription cadence?

Insights and risks

We anticipated a point of contention for users, in terms of what their expectations might be when changing the frequency of their box. Did users only want changes applied to shipments after their most recent shipment was delivered? Or did users want a change to frequency to be reflected on their next box, and all subsequent boxes afterward applying the same setting? Important questions that we wanted to take a stab at, as well as potentially test.

Crafting the entry point

Currently designed, we clearly identified inconsistencies with the native components, disparate between Android and iOS. There was also clear visual dissonance, whereas we wanted to keep the two core sections of our subscription surface consistent from a “take an action” POV.

Initial entry point card, as previously seen on mobile

At first stab, we made sure that the action on the card was consistent with the rest of the surface, using the action button with descriptive copy and a chevron. We hypothesized early on, that emphasizing the billing and shipping date as metadata (housed in a pill) would help in delineating b/w a change in cadence settings versus actual date, and serve as a visual anchor of sorts.

Entry point card - iterating with action button, bringing the card's action in convention with our design system

Copy and micro-copy explorations

Above are some of the copy explorations I came up with alongside our superstar copywriter Tiff, with the goal of how best to “communicate” the action taken.

With option B, the thinking was it’d be similar to common subscription surfaces across various e-commerce apps, where a radio selection might be used, for example, as a set and select option with strict rules. Would this scale well considering we still had an open question as to setting expectations of whether the cadence affects the next box? Would it add to the confusion?

Option A was the simplest, most descriptive use of copy for an action button, which is a key tenet of our design system, to aim for simple, descriptive language over technical descriptors or affordances, which is what Option C presented, using the same language as the iOS picker.

Whilst a question I had was that since the action button was on the subscription surface, perhaps it would make sense for the descriptor to be referred as subscription. However, this was quickly answered in unmoderated user testing and comparing previous notes with our wonderful researcher Rucha, as language around the product during onboarding and messaging conditioned users to think about their Curology box as a customizable shipment. Armed with this rationale, we were confident in using option A.

Iterating on option A

In our final iteration for the entry point, it does away with the floating element as metadata.

Why? Whilst we came out of critique with a prerogative to reduce the amount of screen estate (especially as the metadata pill extended our modal’s expected size), we wanted to ensure a crisp visual anchor to the billing and shipping date whilst being scalable for any possible edge cases and reduce clutter on the surface. From a visual design viewpoint, it felt like a floating surface that didn’t really afford an action, so to that end, we stripped away at the anatomy of the card itself, fusing it onto the action card, making use of whitespace, and emphasized copy pertaining to date and / or frequency. This decision was also taken with the intent to ensure consistency across the subscription surface from a visual design viewpoint. On the other hand, it was important to tie it into the mental model of subscription, rather than a singular shipment

Entry point module

Feature entry point - module in context

Crafting the initial flow for a patient changing frequency

Initially, the question we asked with this flow was, how might we constrain it within a specific design pattern that already exists in our design system. The dialog modal seemed a good fit, because of vertical stack-ability, as well as being scalable on mobile web.

Entry point

Initial modal

Modal after selection

Entry point screen - end of flow

We iterated upon this by introducing meta-data onto the modal. This would communicate to the user, that the setting would not change their upcoming box’s shipment date. However, we would craft the intent of the copy in the action button to reflect the new frequency state, which we hypothesized would re-assure users of their action taken, while being informed that it would not immediately change their shipment date.

The first prototype here, assumes the simplest possible flow, and is structurally the same as the previously implemented feature. However, the modal doesn’t precisely communicate what the change in frequency is affecting. Since the setting simply communicated the old frequency (6 weeks in this case), users might not have been sure of the immediate effect.

Initial flow for changing frequency. We were optimizing for speed of task.

Before design critique, I cleaned up the meta-data in terms of visual design, to ensure that we were using as little screen estate as possible, while still bringing emphasis.

We also iterated on copy for the modal, to be friendly, yet descriptive, as was used throughout our granular settings’ surface. Shipment Frequency, while describing the action well, did not mesh with the rest of our copy for settings at a similar granularity. For this reason, we changed it to “How often should we ship your box?”, with the psychological affordance that Curology is asking a friendly question, as opposed to a task about changing a password.

Friendlier, more brand aligned copy for modal header

Design critique board that I prepared to walk through initial explorations with the team in an informal group setting

Exploration 1

Dialog modal with single-select cells. Contextual anchor to shipping date.

Initially, I kept the design simple, showing contextual meta-data that showcases the shipping date for the user.

The intent here, was to provide the user context as to when their shipment was arriving, but also to scale as I asked our design team whether it’d be useful for users to change their most recent, upcoming box.

Exploration 2

Adding micro-copy under each option to provide user with additional context. We were essentially providing each option with corresponding meta-data. However, a question raised, was would the amount of text cause choice paralysis?

Exploration 3

A variation on our second exploration, but one that would automatically self select, as seen in products such as Hims / Dollar Shave club.

The second and third prototypes here were meant to foster discussion on potentially using micro-copy under each selection as metadata, testing out whether the removal of a CTA reduces friction. It also took the approach that users would be able to change the frequency of their next immediate box, if logically possible. However, feedback that we received centered on trying to keep the visual design of the modal itself as simple as possible, and one that should aim to “reduce” cognitive load, versus options that might induce some kind of choice paralysis (2nd and 3rd explorations).

Assumption to de-risk

Initially, after chatting with our CTO during critique, we went for an edge-case proof approach, keeping the logic as simple as possible.

This meant that our solution set was to only change cadence for boxes after a patients’ most recent upcoming box. (Therefore, we wouldn’t change when your next box is on its way, but would adjust the frequency after that box arrived).

The main challenge here, was communicating to the user and setting the expectation that they had indeed changed the frequency setting, even though the box affected would not be the upcoming one, but all subsequent packages afterward.

Did setting the frequency of the subsequent box set the right expectations for our patients versus the most recent upcoming box?

  • Whilst we went ahead with the simple approach of the former for the first user test, we would be able to validate whether changing subscription frequency was setting the right expectations for users.
  • We would also go ahead and ask for their expectation within the user test, to glean insight as to whether changing subsequent boxes or changing their most recent box, and which one is expected behavior on their end.

Visual design evolution of our dialog modal

Dialog modal after critique

Previous, generative exploration - one of the reasons why I always keep old explorations stashed away, rather than deleting them. Design is a generative endeavor a lot of the time.

The above exploration was an old, generative iteration that I had come up with, with the intent to clearly delineate between selection, and meta-data. When discussing the feature after critique, we realized that if users’ did want to change their initial shipment whilst changing frequency, there would need to be screen real estate for scalable copy. An example would be if a user wanted to change their frequency, and in the default case, it would be able to show their shipment date.

How about a user changing their frequency, but it being too late to change the frequency of their current box? It would accompany copy such as “This change only affects boxes shipped after Aug 20, 2020.” Would this scale well right below the modal header? Or is there a way to delineate meta-data and selection without an edge-case situation feeling crammed, and with visual breathing room.

Visual design details - evolution of modal after feedback and iteration

Initial modal

Modal when user makes a selection. In this example a user changes their frequency from 6 weeks to every 8 weeks.

To that end, we iterated on the dialog modal, housing a recommended chip, as well as a contextual micro copy to remind user what their current frequency was.

The intent for the recommended chip came with an insight during critique, where one of our in-house providers mentioned that most providers go ahead with the 6 week duration as a matter of best practice, and in some cases, changes might not always be effective in terms of frequency change. Taking this feedback in, we added the recommended chip as a frictional affordance for our patient, so that they would change their shipment frequency only if they absolutely needed to / wanted to.

Once a user makes a selection, we utilize scalable micro-copy below the selection options, with copy in this situation clearly telling the user that the frequency change would affect boxes shipped after a certain date (as decided tentatively during critique). However, this would also scale well if the frequency changed would be immediate, and the copy would then be altered to “Next Billing and Shipping Date: [x date]”.

The change from check-mark, to a radio style selection was critical from a visual design perspective. Even though we had used the check-box within the product, recent changes in the design system meant that for single select actions, such as picking a single frequency in this case, it’d offer more clarity to use radio style single select throughout our experience.

Testing and moving forward

First round of testing

Round 1 user test - interactive prototype

We ran an unmoderated test on 10 users, and had some really interesting findings from it.

Learnings

A critical problem we found in user testing early on, was that patients were confused about how changing the frequency was affecting their shipment dates.

A few were essentially confused in the first user test over the settings change affecting their next box versus the subsequent box.

Whilst some patients got their next shipment date right, several users were also confused over the header on the card showcasing the new frequency setting, with the shipping date unchanged. (even though the setting had changed internally and we hypothesized that adding contextual copy would mitigate such confusion).

  • This was at the end-state, or when the patient had gone in the flow and completed the change of frequency.

The user testing session made clear that we needed to set the correct expectation for how updating the frequency affects a shipment date. We thus knew, that iterating on copy was essential.

However, testing validated that the action of finding and changing subscription frequency itself was frictionless and something we could build upon. Task completion, which was optimized for speed, was meeting its desired result in front of the users

Changes in design from 1st to 2nd user test

Copy for 1st user test, after setting change.

Copy for 2nd user test, after setting change.

Second round of testing

Round 2 user test - interactive prototype

On our second round of user tests, we were able to iterate on the end state of our experience. More specifically, the expectation we were setting on the subscription surface, after a frequency setting was changed. In this situation, we set a clear expectation with reactive copy on the modal (“This change only affects boxes shipped after [x date]”).

However, on the frequency card, the header shows the same frequency as previous, since in this scenario, we would only change frequency of the subscription once a patients’ most recent box is shipped. This header copy is contextualized with clear expectation setting, where if a patient has changed their frequency, it says “We’ll ship your box on [current shipment date for recent box], then every 10 weeks after that.”

  • Use of typography and weight, we hypothesized, would add clarity for our patients in the end state, and the copy itself would provide clear context as to when the changes in setting would propagate.
  • This was done in close collaboration with Tiffany, our copywriter.

Whilst the right expectation was set at our end state, 8 out of 10 users that we ran the test on, expected that changing their frequency would affect their most recent, upcoming shipment date. Although all our users, for this test, got their new date right upon completing the flow, it was an interesting insight that our users’ had expected a change in recent shipping date upon changing frequency, as well.

Taking our learnings to craft a final, scalable solution

Before getting down on crafting the final solution, we discussed with the engineers, as well as the PM who launched the initial experiment, that we would intend to use the exact backend logic as the previous experiment to cut down on development time

Whilst the current backend logic was designed to have a condition to make sure that frequency was only being changed after the current shipment date had passed, how it worked was that the logic was changing shipments’ frequency, including the current date, if it was possible to change it.

Relative lift of 25% retention in cohort that changed frequency at least once.

  • This version was not launched in the previous experiment, as there wasn't much design work done and the PM raised that it could be quite ambiguous.

A clear insight that we had from our two rounds of testing, was that users expected, across two tests, that they would be able to change the closest upcoming shipment date on average.

Final interactive prototype

To this end, we split our experience, with the constraint of our backend logic, into two discrete paths, one being the happy path, and one being the edge-case scenario.

Happy path design

Scenario - happy path

Scenario: screen date - Sep 1st, I got a box on this date.

  • Current subscription frequency: 6 weeks. Next billing and shipping date: Oct 13.
  • New frequency: 8 weeks. New billing and shipping date is Oct 27.

Edge case design

Scenario - edge case

Scenario: screen date - Sep 1st.

  • Current subscription frequency: 8 weeks. Next billing and shipping date: Sep 8.
  • After changing frequency to 6 weeks -> Modal to confirm final cadence change -> new billing and shipping date is still Sep 8, and frequency is updated after the shipping date.

The intent behind this decision making was two fold. Whilst we would use the same backend logic as the previously launched experiment (keeping it constrained), in fact with just a change of conditional statements, this would also match up with a patients’ mental model of editing their frequency, as validated by users during testing.

That’s why, it is always always important to talk and validate with the end user. If one goes into testing, attached to a solution at hand, it leads to blind spots being uncovered by the user themselves.

Validating with users across two rounds of testing here helped us de-risk an earlier assumption that users’ might prefer a change to their subscription only after their recent box came in, and we were thus confident in shipping out this solution to a third of our cohort.

Scaled to desktop

Desktop designs

Result and Impact

We released the initial feature to 1/3 of all Curology patients for the first three months, before releasing to the full user base. Update: Subscription cadence is now a permanent feature within the Curology subscription experience.

Relative lift of 25% retention in cohort that changed frequency at least once.

10% relative lift in cohort that stayed on default frequency.

Reflections

To work in tandem with a team that not only respects, but deeply values both user testing and copy as a way to enhance a patients’ experience was fantastic. We were able to clearly test out our riskiest assumptions, and iterate towards a better path forward for our patients’ which offered them flexibility, but not at the cost of confusion, and I’m glad that I was able to take the lead on such a project that had a direct impact on the business metrics of the company.

Looking back, however, one thing I would have loved to engage in was deeper multi-variate testing. To de-risk our assumptions and move towards data informed design, it would have been great to test out our solution within the 2nd user test to a broader set of users.

In essence, it would have provided statistical significance in terms of feature usage, as well as making certain which solution is more successful with users in terms of retention.

Even though we had validated with users during user testing, that our final solution was matching a users’ mental model, the certainty of a multi-variate, or even two-variant test with a larger sample size would have provided more certain evidence that the solution we went with, was the right one. This is especially true as it pertains to “feature usage over time”, and whether there would be a significant delta between the two variants.

Thank you so much for reading! If you have any questions or wanted to ask more about the project, don't hesitate to reach out on my email at [email protected]

Built with ❤️ and ☕

Find me online here: