Wednesday, January 23, 2013

Looking for JavaScript Decompressor for Data Compressed by C#


It happened that I needed to make an HTML page.  Sometimes such things happen, you know.  That page was required to be completely offline, just a snapshot of some piece of data being downloaded by users for further analysis.  At the same time that offline page was required to be as interactive as the online page with the same data.

The online page for that data contained grid with "master" rows and upon clicking on the row it presented a dialog box with "details" loaded from the server by AJAX request.  The offline page could not use AJAX so it should contain all needed data, both "master" part and all the "details", inside it.  The average amount of data used on these pages was not very small - about 3 megabytes in JSON format, so it was decided to compress that data on the server side and decompress needed pieces on the fly when they should come into the dialog.

The code for the server side was not very complex.  As the web application uses ASP.NET MVC and is written on C# I could use standard .NET classes for compression like either System.IO.Compression.DeflateStream or System.IO.Compression.GZipStream:


var details = new Dictionary<stringstring>();
foreach (DataRow row in detailsData.Rows)
{
    var text = Convert.ToString(row["details_content"]);
    var bytes = Encoding.UTF8.GetBytes(text);
    using (var ms = new MemoryStream())
    {
        using (var zipStream = new DeflateStream(ms, CompressionMode.Compress))
        {
            zipStream.Write(bytes, 0, bytes.Length);
            zipStream.Flush();
        }
        details[Convert.ToString(row["master_id"])] = Convert.ToBase64String(ms.ToArray());
    }
}
viewModel.ZippedResultsJson = EncoderHelper.SerializeToJsonString(details);

But what to do on the page?  Are there any JavaScript implementations of, say, inflaters?  Initially I thought that these days there are lots of them as probably almost all things are already implemented in JS.  But when I tried to look for them I've found that things are not that good.  Actually there are just a few implementations of deflate/inflate algorithms and not all of them look working.  Thanks StackOverflow, they know answers to almost all our questions and I've found lots of answers how to get some of JS decompressors to work with Java or Python server-side compression.

But just a pair of questions about C#-JS were answered like "just use the browser gzip support", i.e. encode the whole response.  This is not my case as I don't have response so I tried to use some of implementations to see which of them is more usable.  Things were worse than I expected, no one of relatively small inflate implementations worked with C#'s DeflateStream-compressed data (although they worked quite fine with data compressed by themselves).

So I decided to try gzip instead of the "raw" deflate.  And I've found the only JS implementation of gzip decompressor on http://jsxgraph.uni-bayreuth.de/wp/2009/09/29/jsxcompressor-zlib-compressed-javascript-code/.  I tried it and it worked!  (Of course, I've changed the DeflateStream to the GZipStream on the server side).  Lots of thanks to those guys who created that really working tool!  With it all I needed on the page was just:

$('#details-div').html(JXG.decompress(detailsData[masterId]));

And that's it!  Simple, usable, great!  And it does Base-64 decoding inside so there is no need to use separate decoding tools or libraries.  Great job guys!  Now I know the right tool for this task.

X-Post from: http://dzorin68.blogspot.com/2013/01/looking-for-javascript-decompressor-for.html

4 comments:

  1. How to implement compression strength equal to 9 in C#? I have same issues which was listed here:-
    http://stackoverflow.com/questions/14993029/jsxcompressor-usage

    My code is C#:-

    var output = "";
    var bytes = Encoding.UTF8.GetBytes(data);
    using (var ms = new MemoryStream())
    {
    using (var zipStream = new DeflateStream(ms, CompressionMode.Compress))
    { zipStream.Write(bytes, 0, bytes.Length); zipStream.Flush();
    }
    output = Convert.ToBase64String(ms.ToArray());
    }

    And JavaScript Code :-
    alert(JXG.decompress(compressData));

    ReplyDelete
  2. We had the discussion with author of that StackOverflow post in my original posting. As far as I know .NET does not allow to set the compression strength. And it looks like compression streams in different .NET versions have different strength set internally.

    So... that guy uses .NET 3.5 and he finally has found out a workaround by using ionic's zlib implementation on the server side (see his latest comment on the StackOverflow).

    I use .NET 4.0 and the standard GZipStream works for me (not DeflateStream!!!).

    So if you use .NET 4 you probably get it working by replacing DeflateStream to GZipStream on the 5th line of your code. Or in case of .NET 3.5 you probably want to use the third-party library like ZLib.

    ReplyDelete
  3. Thankx Dmitry Zorin, I am using .NET 4 and changed my code to from DeflateStream to GZipStream. It is work well in ASCII Character. But as we have noted i am using Encoding.UTF8.GetBytes(data); means i need to send utf-8 code such as "भारत".

    I am able to compress in C#.. Now my javascript code gets :-

    Compress data is :- H4sIAAAAAAAEAOy9B2AcSZYlJi9tynt/SvVK1+B0oQiAYBMk2JBAEOzBiM3mkuwdaUcjKasqgcplVmVdZhZAzO2dvPfee++999577733ujudTif33/8/XGZkAWz2zkrayZ4hgKrIHz9+fB8/Iv6jv+yv/4/+sn/gP/rL/qb/6C/7y/6fAAAA//++c5p2DAAAAA==

    After JXG.decompress(compressData);

    I get "भारत" as output.

    Thankx again for ur reply...

    ReplyDelete
    Replies
    1. Hm... That's a good point! Actually I have not tested this thing with Unicode characters but as far as I remember from the source code it has UTF-8 processing inside it. Of course it is just an emulation because JavaScript has only Unicode characters and does not have bytes at all so issues are possible there. I will try to research this point as soon as I have a bit of time for that. Thanks!

      Delete