Scaling Node on Multi-Cores System
Updated: May 13, 2020
The most interesting thing about Node.JS is its single-threaded nature, that is a deliberate design decision by the Node.JS creator to avoid semaphore and/or mutex locking mechanisms that ultimately introduce complexity and hard to debug bugs into the code.
This also means that it only works and utilizes a single core of a system and all remaining cores remain idle. However, there are two most used ways which you can use to scale node.js on multi-core systems and i will be discussing them one by one.
Node Scaling on Multi-cores
Including ‘child_process’ module
The first way is to let resource allocation happen at the system level where incoming requests are distributed to multiple single-threaded Node.js processes each running in a virtual machine assigned a single core from the multi-core processor. All the instances live behind a proxy which serves to balance incoming requests to available Node processes.
Three ways of Creating Child Process
1 exec() Runs a command and buffer the output. It returns the buffer with the max size and waits for the process to end and tries to recall it.
2 spawn() Launches a new process with given command. It should be used when the process returns large amount of data. Spawn() starts receiving the response as soon as the process starts executing. It’s more faster than the exec() call.
3 fork() It is the special case of spawn(), that returns object with a built-in communication channel in addition to having all the methods in a normal child process instance.
Including Node ‘cluster’ module
The Node community has come forward with an even better second solution which breaks in multi-core awareness and capability natively into the Node.js platform.
Clustering Node.js runs as many dedicated single-threaded processes under a master Node.js process without the need for elaborate virtual machine infrastructure, using fork() of cluster module. This results in significant performance gains without the costs associated with virtualization.
Performance Test using Single and Multi-cores
Lets look at a performance test by our developer, in which he start the main process called the master process. Master process starts and manages other processes called worker processes which do the actual work of handling incoming requests. TCP connections are shared by all worker processes and a default load balancer is included transparently in the master Node.js process which serves to distribute the load.
This test was performed on a dual core system with hyperthreading to take advantage of all the four processing threads and made a simple app using Express generator and modified it by creating a processor intensive root route to test performance under heavy load conditions.
Then ran the following test using siege to measure the performance with and without clustering.
siege -c100 -t1M
Here siege is the http benchmark by Node.JS, the c option instructs siege to throw 100 concurrent connections on the server whereas the t option instructs it to make upto 1 million total requests until the test is finished. The web server was running locally on port 3000 so the address of the server is localhost:3000.
Using Single Core
First review on the using single core Transactions: 1237 hits Availability: 100.00 % Elapsed time: 59.70 secs Data transferred: 0.03 MB Response time: 4.14 secs Transaction rate: 20.72 trans/sec Throughput: 0.00 MB/sec Concurrency: 85.68 Successful transactions: 1237 Failed transactions: 0 Longest transaction: 4.58 Shortest transaction: 0.11
A noticeable performance is shown when we test it on multi-core Transactions: 2523 hits Availability: 100.00 % Elapsed time: 59.92 secs Data transferred: 0.06 MB Response time: 1.82 secs Transaction rate: 42.11 trans/sec Throughput: 0.00 MB/sec Concurrency: 76.67 Successful transactions: 2523 Failed transactions: 0 Longest transaction: 2.86 Shortest transaction: 0.12
Now, we can say that the best practice of scaling Node is by using cluster module, we noticed that the tangible benefits are achieved with scaling on multi-core using cluster module on a single-threaded Node.JS application. You can get the example project on the link attach below