WebSpellChecker.net Wiki

Home

Results of SpellCheckAsYouType (SCAYT) Load Testing

Test Cases


The following test cases were performed:

Test Case 1: Loading dependence on the number of concurrent users

A user opens the page with SCAYT and checks spelling in a text with 300 English words (~A4 page). The text is random and contains from 4% to 90% of misspellings.

During one user’s session 30 spelling check requests are sent to the spelling check server with 500 milliseconds delays. Each request (GET HTTP request) contains up to 10 words to check spelling. It means that the loading (CPU utilization, RAM usage) doesn’t depend on text size. The text size affects the duration of the load only.

The test case was performed for the following numbers of simultaneous users:

  • 100 users
  • 200 users
  • 500 users

The test case was performed on both test beds (Windows and Linux)

Test Case 2: Loading dependence on the number of misspellings in the text

200 concurrent users spell check the same size text (~300 words) for the following cases:

  • 4% of misspellings
  • 50% of misspellings
  • 90% of misspellings

Test Case 3: Loading dependence on the spelling check language (size of dictionary)

200 concurrent users check spelling on the same size (~300 words) and same number of misspellings (~4%) text for the following cases:

  • American English language (size of the dictionary is ~350KB, ~100,000 words)
  • Spanish language (size of the dictionary is ~700KB, ~300,000 words)

Test Bed configuration


Windows Server 2012 and Ubuntu Server 14.04 operating systems have been chosen as two main test beds.

Windows test bed Linux test bed
Amazon Instance t2.medium t2.medium
OS typeWindows Server 2012Ubuntu Server 14.04
Web ServerIIS 8.5Apache 2.4.7
CPUIntel Xeon Core v2 2.5 GHzIntel Xeon Core v2 2.5 GHz
RAM4 GB4 GB
HDD30 GB8 GB

Load Testing Tool: Apache Jmeter with PerfMon Metrics Collector and Server Agent (Standard Set of basic plugins)

Network topology: AWS EC2 Subnet

A machine with the application and a machine with JMeter tool are in one subnetwork. There are no any nodes between them.

NOTE: the results of performance testing will be different for the different server configurations. SCAYT performance depends on the following options:

  • Hardware configuration (CPU, RAM)
  • OS and Web Server types (Windows with IIS, Linux distributions with Apache)
  • Network topology (number of nodes between an application and AppServer, network structure, etc) and bandwidth
  • Spelling Check language (size of the dictionary)

Results


Test CasesTest DataResponse Time (ms, Windows)Response Time (ms, Linux)
By the number of concurrent users 100 concurrent users 305 34
200 concurrent users 1,071 125
500 concurrent users 3,580 907
By misspellings in the text 4% misspellings 1,071
50% misspellings 4,100
90% misspellings 6,982
By size of dictionaryAmerican English language 1,071
Spanish language 3,440

Windows Server 2012 + IIS 8.5


100 concurrent users
Time response: 305 ms

200 concurrent users
Time response: 1,071 ms

500 concurrent users
Time response: 3,580 ms

Conclusions: The test showed that CPU utilization increases from 80% (for 100 concurrent users ) to 100% (for 200 concurrent users) . When CPU utilization is limited to 100%, the average time response increases with the number of users growths. With the number of users changing from 100 to 500, the time response growed in about 10 times. It can be concluded that the main metric of SCAYT loading is the response time of the requests.

Ubuntu Server 14.04 + Apache 2.4


100 concurrent users
Time response: 34 ms

200 concurrent users
Time response: 125 ms

500 concurrent users
Time response: 907 ms

Conclusion: This test confirmed the previous one and showed that the response time is shorter in 3.5 times (in average) for Ubuntu + Apache configuration as for Windows + IIS with the same hardware configuration.

Number of Misspellings

300 words
200 concurrent users
4% misspellings
Time response: 1,071 ms

300 words
200 concurrent users
50% misspellings
Time response: 4,100 ms

300 words
200 concurrent users
90% misspellings
Time response: 6,982 ms

Conclusions: The test showed that average time response rises with number of incorrect words increasing. The time response growed about 7 times when the number of misspellings was changed from 4% to 90% as AppServer needs more time to process misspellings in the text.

Language

300 words with 4% misspellings
200 concurrent users
American English (size ~ 350KB, words ~100,000)
Time response: 1,071 ms

300 words with 4% misspellings
200 concurrent users
Spanish (size ~700KB, words ~300,000)
Time response: 3,440 ms

Conclusions: Spanish dictionary size twice more than American English one. The test showed that time response for Spanish language four times longer than for American English language as the time response depends on a dictionary size.

General Conclusions

  1. Time response is the main metric of SCAYT load testing. After CPU utilization reaches 100%, the system works but the time response increases.
  2. The graphics showed that the memory usage (RAM) hasn’t changed during all tests significantly.
  3. Performance of Ubuntu + Apache configuration is several times higher as of Windows + IIS.
  4. Average time response increases according to:
    • number of concurrent users (100 - 500)
    • number of misspelled words in the text (4% - 90%)
    • size of the dictionary (300KB - 750KB, 100,000 - 300,000 words)

Back to Top