Jay Kimble

Shawn (of TUX -- BTW, we have a TUX meeting coming up tomorrow night) pointed me to a pretty cool article yesterday. It’s the End Bracket in this months MSDN Magazine. You can read it online here.

Perry (my boss and also of TUX) and I have been talking about some things related to our ASP.NET app’s reporting module (among other things) that additionally got me thinking.

This is pure theory (and potentially a really strange idea)... I admit this up front, but I’m going to throw it out for all to see (and potentially make fun of or to improve on).

The idea (in case you are skipping the MSDN article) is this: if we could predict what the user will do next we could fire up a thread and attempt to pre-load the result into a cache before it is even requested, so that when it is requested we could immediately respond to the user with a result. If the user doesn’t request the item, we can simply throw away the result.

What this does for the UX is that our UIs seems very fast because they are predicting exactly what the user will do. BTW, that Windows OS that no one likes (and is lampooned by the Mac commercials) does this; from what I understand it does things to try to predict what you will do next (and acts accordingly). For Windows/Desktop apps this stuff is easily accessible and used... for web apps not so much (read on).

The Caveats of Background Threads with Web Server processes
[WARNING: Before I continue, by no means am I threading guru. I understand enough to actually talk about the topic. Beyond playing with PLINQ at a very minimal level, I have never written an app that triggered background threads, so understand that like many things I am NOT an expert... just a mad scientist playing with forces he doesn’t fully understand <wink />]

Here’s the problem (as I understand it... you are welcome to correct me and I will be more than happy to correct this section). If you attempt (and I’m not sure that you can without much effort) to spin off a thread from an ASP.NET server process that thread will die the moment that the web request is complete, so you either have to wait for your thread to complete before ending the web request (not really a viable option when the idea is to pre-load something and get it later) or you have to figure out how to spawn that thread into its own process which will live until it completes (my brain starts swirling at this point... something about App Domains and other stuff... bottom line is that it doesn’t work directly from an ASP.NET process).

This is a problem I have found interesting, but have never really devoted a lot of brain cycles to because I haven’t had a use/need for it.

Jay’s Crazy Background Loader Service Idea (very early in my thought processes)
What we need is a Windows Service whose entire job is to execute background tasks for the ASP.NET. It would share our data model or any other library that our web server would use except that it would only be used for loading data into the cache and would be thrown away after a specified period of time (or if the user chose a different path your web app could clean up the cache). What would really help this thing out would be to use MS Velocity or memcached so that the ASP.NET app and the Windows Service shared the same cache area (and distribute it over several machines.. if you have them) [BTW, if you haven’t researched distributed memory cache mechanism you would be wise to do so sooner than later... I’m really impressed with the idea, and the effect they have on apps --at least what I’ve read about them-- is truly impressive].

Anyway, you have the diagram now of what’s in my head

Alternatives -- Ajax mechanisms
Since Ajax does async processing we could also do something like this with pure JS code. Simply by calling services we could predict the user’s actions and load the data in memory and simply use it when the user requested it. The only problem with this that I see is that while modern day JS still sometimes seems fraught with memory leaks and memory heavy clientscript apps sound a little scary to me. I think it’s doable, but you are definitely going to want to minimize the amount of data you would want to load. This really brings us to a bigger question...

How liberal do you get with "pre-loading"?
The more I think about all this the more I start to ponder how often would one do this. Should it be done only with the long running processes (those database queries that take more than a few seconds)? Is it OK to use it with smaller datasets (faster running database selects). Obviously we don’t pre-run database updates as that would be really bad IMO, but long running updates could be tossed into the same threading mechanism (and return quickly as long as the app didn’t need an ID back from the process. Should you load up every GB of memory your machine has? What if your web site gets popular and you are encountering hundreds of hits a minute? It’s a good problem to have, but maybe it creates bigger problems. Maybe creating a farm of machines whose purpose is to run background threads and have a central controller to direct the web app to the next machine to run a process on. It’s something to ponder. Not sure if its quite viable for ASP.NET (but I sure would like to try it... at least in a simple scenario where pre-loading the data will always net a result that is needed later).