Simple Idea For Passing Data To Huge Number of Users

[ This Blog Was Moved To : ]

Three years ago, I had a situation when I was building a web application for a stocks consultant. It should provide users with updates , in a page that gets automatically refreshed every 1 minute.

My problem was that when the number of logged in users exceed 100 users, the page turns to be extreemly slowly rendered.

The problem was that I made that page retrieve the data from database every time the page refresh. Ofcourse it should.

I thought of a simple solution, as there was one stocks consultant ( for me data editor ), and it was one page that is needed to get rendered. I decided not to make it dynamically retrieved, but dynamically created.

Which means, I create the HTML page directly after the stocks consultant finishs editing his data that will be shown to his clients ( At that time I wasn’t using AJAX , and the output was fairly simple ). And, With each refresh I retrieve the HTML static page, that was previously created.

After a while, I decided to make it a little more efficient, by checking the file’s SHA1 and compare it with the latest SHA1 of the same file, if they are the samee ( File didn’t change from last retrieval ), I return back a copy that was previously stored in Memory, else I retrieve the generated file, update the memory cache, then send the memory cache back to users.

Ofcourse, If I am creating this application right now, I would be using the same idea, except for saving data in JSON not HTML and send it back to users ( ofoucrse using AJAX ) and render it ( at the client side ).

I dunno actually if compressing JSON data using GZIP is applicable or what, I hope someone tell me that !! ,, ( ofcourse if I’m creating a similar application right now , I will be asking Uncle Google’s Help for GZIP with JSON )

Hope I could be of some help.