Connect ® App — a disruptive debut
Connect ® is a new mobile banking app that Disruptive Bank rolled out this year. Its value proposition is simple, yet ambitious: self-service all your banking needs on the go and enjoy financial deals that’s tailored for you. Prior to its release, Team UX @Disruptive Bank iterated a # round of user research in a bias-free test environment.
The pre-launch usability tests, user interviews and surveys suggested that Connect ® is on its way to disrupt the mobile banking sector with its user-friendly interface and machine learning content optimization. In early March 2021, the app finally saw its high-profile launch. No one had a doubt that its robust usage rate was a clear leading indicator for the app’s uninterrupted adoption, retention and growth path.
On April 2 however, Connect ® saw a nosedive in its sign-up rate. Team UX @Disruptive Bank was baffled about the tumbling metrics and went out for another usability test. The test uncovered some signs of friction on its ID registration flow. With these test findings Team UX went into a tough negotiation with Compliance and IT Security executives. After repeated meetings and validation, Team UX finally scored a greenlight in reducing 21 identity verification fields to 10 in the app’s sign-up pages.
With the sign-up revamp, the completion rate for ID registration picked up by 40% on its usability test. Despite the tight iteration schedule, Team UX even pushed for A/B testing on the app’s landing page, which was another suspect for the app’s underperformance. With the new sign-up and landing page design, the app’s System Usability Scale reached 95%. The final user journey map of Connect ® was filled with an all time high satisfaction and referral score. Again, a clear win of the app on the usability scale.
Still, users did not come back. On May 28, Team UX was shocked to see another dramatic turn in the app analytics — the usage rate was tumbling again. Fool-proof test metrics, design interaction, delivery and deployment…… But the user churn accelerated. What’s wrong? Was there any error or bias with its test sampling and design, data collection or analysis that Team UX painstakingly went through for those eight weeks?
The Small Data Bias
Team UX stopped the usability test and went out for a churn analysis. Through some data wrangling, Team UX got hold of the usage data of the Bank’s website. Surprisingly, the web analytics also indicated a stampede of users in May, which was the second point of churn of the app. Team UX was intrigued, but the web analytics did not explain why the event happened. Team UX reached out to Customer Care managers and negotiated their access to the customer call database. Interestingly, the call history showed that there were two points of surge in customer inquiry through almost the same period, from March to May.
Team UX got intrigued. Why the call number spiked, and what the customers were inquiring and complaining about? They requested more call inquiry data. Through the topic analysis over 264 pages of call feedback and 32 cases of call recordings, Team UX discovered two trigger points that preceded the explosive call inquiries — an incident of a security breach at Competitor Bank in March and a website revamp at Disruptive Bank in May. This time, Team UX went through live chat data, customer surveys and even social media for more data mining.
After some content and sentiment analysis of the customer data and analytics across a number of channels, Team UX finally came to an Eureka! moment.
- The first churn : A trivial security incident that happened at Competitor Bank turned into a social media wildfire through March. In the aftermath of the incident, lots of banking customers on Twitter and Facebook said that they can’t trust online banking and would drop off online transactions immediately.
- The second churn : The website of Disruptive Bank has been the app’s primary acquisition channel. On its release, Connect ® saw some 35% of its traffic coming from the Bank’s website. In May, the website saw its navigation renewed which led to painful user experience. A majority of the website visitors bounced off its landing page and this, in turn, stifled much of its traffic to Connect ®, the Bank’s mobile banking app.
For Team UX, seeing such an immense Data Lake of customer intelligence was an eye-opening experience. Up until now, Team UX was proud of doing much data-driven design; but their fixation on small data was driving them to small-scale user insights. Team UX realized that the limited scale of user tests resulted in Small Data Bias and it made them overlook the customer context and various points of friction that were occurring across the customer journey.
As user demographics are adding more complexity and omni-channel experience is becoming a UX norm, an increasing number of UX deal-breakers are coming from outside the domain of user experience. Unfortunately, small scale usability data operate in its prescribed scope and the data turn-around time keeps challenging Team UX. Team UX realized that it needs to access more Big Data to derive more scalable, timely UX solutions.
Big Data, Analytics and UX Design
So, is there a magical Big Data analytics solution that quickly and dynamically updates customer/user data and turns them into actionable product strategy in real-time? Following the repeated churn incidents, Team UX proposed to senior management and its stakeholders to form a Big Data committee.
Team UX hopes that this corporate initiative will help them find answers to the following Big Data questions and come up with action plans :
Corporate agenda :
- Big Data does exist in the corporate data ecosystem. How can interested teams access the data repository to track and monitor evolving customer/user demand regularly?
- How can we create and curate data dashboards that will enable us to do clear and timely descriptive and predictive analysis? How can we tackle data visualization?
- Can we move to a cloud data warehouse for easier data aggregation and automated customer/ user data reports?
- How can we break down the data silos at the corporate level?
Team UX agenda :
- Can data scientists or analysts help Team UX scale and contextualize customer/user data, so we can make our user segmentation and customer/user journey map more data-driven?
- How can we transform the customer/user data into actionable UX insights for continued optimization of Connect ® in the security-challenged market?
- …And so forth
As I write this article in June 2021, the Big Data committee kicked off @Disruptive Bank. While the data discussion is underway, Team UX is working to fix the user experience of Connect ® based on its Big Data-informed decision :
- Prototype a two-factor authentication design.
- Draft a new set of security notifications.
- Add “security-conscious” content to the current style guide.
- Advise the sales team on how to fix the website navigation, which has been a source of friction for the Bank’s web-app conversion.
True, the above Big Data questions are not something that the design team can answer in a couple Sprints or Retrospectives.
In the meantime, the corporate Big Data committee has seen a promising start. Now, the stakeholders of Project Connect ® are working with a clear, data-driven consensus — never make strategic UX decisions in data silos. Driven by the force of volume, velocity and variety, they believe that Big Data will empower them and its stakeholders to keep up with the evolving customer/user needs and keep their user experience fresh.
