I am writing this blog out of my own experience in using "Velocity" till CTP3 which I thought to be promising distributed in-memory caching technology:
Microsoft came up with their new project code named "Velocity" for distributed in-memory caching. As a learner i started learning this technology and found that there are already lot of players in this field of distributed caching. As a wild guess i thougt it might have come up with some new and faster approach to read bulk amount of data with their concept of Region,Tag enumerator and then LINQ and WCF.
When I test my application in compared to unmanaged memory mapped file(over network) against velocity caching, i find memory mapping is far far better. though in this case memory mapped file is a static one i cant update it, whereas velocity will allow me do that.
Let me clarify a bit about my sample application :
I have a static binary file, stores spatially indexed(quad tree) and formatted data ,which stores millions record having line geometry. I can traverse them throgh my spatial iterator very fast and can filter my spatial data very fast. But here only one problem i have i can't update it, i need to rebuild the spatial index again. During live operation, I synchronize my data with oracle back-end if any other user edited/added any record and then serve it to my client application. I was just wanted all these facility from "Velocity".
So, i indexed my spatial data key in Regions with specific tags(quad node name where it fits,identifier to locate geometry object in the named cache, any other attribute which can be indexed) and stored actual data in NamedCache. As soon as a new record comes my SpatialIndexing service finds which quad node its going to fit then it posts it to Region and NamedCache respectively.
Now i tested my application using memory mapping and velocity in .Net 3.5 environment. Yes through a PIA i've i've accessed the binary file through memory mapping. In Velocity case i used following settings-
1 - Local Cache on/off + no secondary
2- With secondaries + local cache on/off
3- Simple/Routed
Used SQL server cluster configuration. 2 nodes on the cache tier and all three nodes are linked to a common network(LAN).
1 -Quorum node - VISTA 64bit,4GB RAM,Single core
2- Node2 - XP32bit,4GB RAM,dual core
Though there were interop calls between managed and unmanaged environment, memory mapping worked ~15 times faster compared to Velocity if there is no local cache. For local cache case both are slight compairable(memory maping still leads!!), but i can't enable local cache as it will eat up my application address space. Fortunately in CTP3 we can limit numberof objects that can be stored locally and they will be evicted as a regular eviction mechanism.
I'm going to make my application run on memory mapping only. Will see if memory mapping of .Net 4.0 can help me further to gain performance.
