Orchestrating Better In-App Chat Experiences for 100K Concurrent Users!

Sadhana Published On July 21st, 2022 Engineering

As diligent enablers of proficient in-app communication experiences, delivering high performance at scale has always been our forethought. This meant that our product teams must go all out in developing bug-free, superior in-app solutions across all niches.

Discerning the criticality of these API & SDK-led implementations, MirrorFly engineered a pipeline of load testing measures to surpass user expectations beyond time.

Presenting the detailed walkthrough of our recent test endeavor, where we crossed off our top priority metrics: 100k concurrent users, performance, memory utility and more

The Case of Load Testing, an Overview:

Given the power to jumpstart in-app communication experiences, our prime objective here was to improve the load bearing capacity of MirrorFly’s microservices API, sockets and XMPP services.

By evaluating the structural complexities, use cases and other dependencies, our internal team fixated on key considerations to improve the scalability, performance and sustainability of the deployed chat API services.

Key Considerations:

The aforementioned experimentation strategy intended to substantiate these key objectives

1. Need to handle 100k concurrent users
2. Assess the endurance of the server and ensure to record the concurrent user capacity up until a completed chat transaction
3. Response time should be less than 5% and error rate less than 1%

Putting Scalability and Performance to Test

In MirrorFly, an end user’s communication journey entails messaging experiences fulfilled by API and XMPPs.

In pursuit of evaluating the competence and integrity of these services, our test process assessed the components with parallel load constraints for accurate chat and media-file observations.

We had relied on Tsung, the distributed load-testing tool to test the scalability and efficiency of server based applications accurately.

Likewise, for emulating the environment, average page requests per second( 400-500 seconds) and user time(8 minutes) were identified to be the test variables and conditions.

Now, before delving into the insights, here’s an outline on either of the test scenarios, tools and criterion involved.

Testing APIs as a Whole:

Primarily, an API-enabled messaging interface permits users to interact via media file sharing, call log data and contact directory access.

And in order to traverse this user journey, Apache Jmeter was utilized to gauge the service with below load criterion:

• Concurrent users – upto 100k users
• Ramp up rate – 71 users every second 

Testing XMPPs as a Whole:

Typically, a XMPP service warrants users the competence to get started by validation and start conversing with peers via instant messaging.

In our case, Tsung tool was leveraged to emulate this process flow and to assess the service fit with precision via following load criterion

• Concurrent users – upto 100k users
• Ramp up rate – 200 users every second

Key Findings: 

Upon meticulous stress assessments, we plotted down the key metrics. And obviously, most of these insights leave a positive impact on our offerings with room for improvement in selective areas.

Let’s explore these nuanced results in detail:

• The server can handle 100k users with a total of 19.6 CPU and 41.5 GB RAM (inclusive of MySQL, XMPP, API services), with an average usage of 30% CPU and 50% Ram can be handled.

MYSQL Memory Usage:

 MYSQL CPU Usage:

• Intended peak load is achieved with less than 50% memory utilization 

Kubernetes CPU Usage:

Kubernetes Memory Usage:

• Apdex score is fair for 100k users
• Resource utilization in the event of login and media entities seem to be higher however no errors are recorded on it

Scaling Growth and Innovation with Testing: 

‘How do we make in-app communication better across all our apps?’

Driven by this growth-oriented mindset, we at MirrorFly are constantly striving to create the best in-app communication experiences for our users via API and XMPP services.

Ergo, with this recent testing attempt & more such in the pipeline, we aim to continually improve our service offerings to foster error-free development all along.