So, the other day I was humming along for one of our larger clients – putting the finishing touches on one of their applications. The app in question handles matching of vacancies against applications. The administrator needs to quickly have the ability to browse what vacancies have what status.
One vacancy can have one of three different statuses:
Don't worry too much about the business logic, I only provide a quick background to give you some idea of what I was trying to achieve.
Oh, I nearly forgot! Of course this is an IBM Domino solution as we are a "True Blue" IBM Premium Business partner. Unfortunately this client was stuck with a fairly old IBM Domino server: 8.5.2, so I wasn't too keen on using an XPages solution. Why? Because in my humble opinion XPages just hadn't matured enough yet. (Missing HTML5 features, no SSJS debugger, old DOJO release, loooong "first hit" boot times and so on – Yes, all of this can be mended, worked around, patched etc. But sometimes you can just say enough is enough and go with old school Domino development.)
Anywho, my first inclination was to create a WebQuerySave agent that did the matching ServerSide before presenting the result to the user, a very common approach. But the performance was really bad – The WQS agent took around 4000ms although the workload was light and number of documents fairly low (in the hundreds anyway). The page itself took around a second to load with an empty cache, so we we're looking at a total page load time here in the neighborhood of five seconds – not acceptable.
So I used an approach that's known as perceived performance. Let's take a clock as example – If you use a clock that doesn't show seconds, time seems to move slower. If there's a lot of stuff going on, the general experience is that things are happening quicker. So I extracted my LotusScript code and placed it in an agent that get's called thru AJAX. This way the page loads quickly, the user can start to interact with it immediately and the state of each vacancy trickles in as the server sees fit.
Below is a screenshot of the application, waiting for the AJAX request to comeback with the data. (The only vacancies that know its state are the ones that are booked, they're marked with "Visa" in the column to the right, the rest are pending).
Finally, here we have the first icarnation of the code:
Option Public Option Declare Use "helpers" Sub Initialize Dim s As New NotesSession Dim db As NotesDatabase Dim vRequest As NotesView Dim vMatchVacancy As NotesView Dim vcRequest As NotesViewEntryCollection Dim vcMatchVacancy As NotesViewEntryCollection Dim veRequest As NotesViewEntry Dim veMatchVacancy As NotesViewEntry Dim rowIds As String Dim key As String Dim count As integer Print "content-type: text;charset=utf-8;" On Error GoTo handler Set db = s.Currentdatabase Set vRequest = db.getView("vRequest-list") Set vMatchVacancy = db.getView("vMatchVacancy") Call vRequest.refresh() Call vMatchVacancy.refresh() Set vcMatchVacancy = vMatchVacancy.Allentries Set veMatchVacancy = vcMatchVacancy.Getfirstentry() While Not veMatchVacancy Is Nothing key = veMatchVacancy.Columnvalues(0) Set veRequest = vRequest.Getentrybykey(key, true) If(veRequest Is Nothing) Then rowIds = rowIds & veMatchVacancy.Columnvalues(1) & "," End If Set veMatchVacancy = vcMatchVacancy.Getnextentry(veMatchVacancy) Wend Print rowIds exitSub: Exit Sub handler: MsgBox db.filepath & "agentMatchVacancys - " & Error & Chr(13) + "Module: " & CStr( GetThreadInfo(1) ) & ", Line: " & CStr( Erl ) Print Error & Chr(13) + "Module: " & CStr( GetThreadInfo(1) ) & ", Line: " & CStr( Erl ) Resume exitSub End Sub
Nothing too odd about the above, I would even dare to say a fairly common approach. I do what I can to speed up the process by using NotesViewEntryCollections and the ColumnValue-property. Using one view as the source, that has a column that combines the particular criteria for that vacancy. The key can look something like this:
2015-05-27DSurgeon (Date + Slot + Role)
We use this key to find any matching requests, if none is found it's added to the string that's returned to the AJAX request. We only need the ones that doesn't have a match (Missing) as the first state (View) is stored with the vacancy and the second state (List) are all that are not missing. Ideally "Missing" should always be very few documents so the data transfer over the wire should also be low, increasing performance further.
So, everything was "hunky-dory" then. The application loaded quickly and felt responsive to the user, but… The bad performance of the LotusScript code was nawing at me…
After a murky night of coding I came up with the following:
Option Public Option Declare Sub Initialize Dim s As New NotesSession Dim db As NotesDatabase Dim vRequest As NotesView Dim vMatchVacancy As NotesView Dim veRequest As NotesViewEntry Dim veMatchVacancy As NotesViewEntry Dim rowIds As String Dim key As String Dim i As Integer Dim count As Long Dim hasEmptySlots As Boolean Dim navMatch As NotesViewNavigator Dim navRequest As NotesViewNavigator Dim ve As NotesViewEntry On Error GoTo handler Set db = s.Currentdatabase Set vMatchVacancy = db.getView("vMatchVacancy") Set vRequest = db.getView("vRequest-list-match") Call vRequest.refresh() Call vMatchVacancy.refresh() Set navMatch = vMatchVacancy.createViewNav() Set navRequest = vRequest.createViewNav() count = vMatchVacancy.Allentries.Count i = 0 ' do not do AutoUpdates vMatchVacancy.AutoUpdate = False vRequest.AutoUpdate = False ' enable cache for max buffering navMatch.BufferMaxEntries = 100 ' if we are not interested in the number of children, we can go a little faster navMatch.EntryOptions = Vn_entryopt_nocountdata Set veMatchVacancy = navMatch.GetFirst Print "content-type: text;charset=utf-8;" While ( i < count ) hasEmptySlots = False key = veMatchVacancy.Columnvalues(0) Set veRequest = vRequest.Getentrybykey(key, True) If(veRequest Is Nothing) Then Print rowIds & veMatchVacancy.Columnvalues(1) & "," End if Set veMatchVacancy=navMatch.getNext(veMatchVacancy) i = i + 1 Wend exitSub: Exit Sub handler: MsgBox db.filepath & "agentMatchVacancys - " & Error & Chr(13) + "Module: " & CStr( GetThreadInfo(1) ) & ", Line: " & CStr( Erl ) Print Error & Chr(13) + "Module: " & CStr( GetThreadInfo(1) ) & ", Line: " & CStr( Erl ) Resume exitSub End Sub
The big difference here is the use of the NotesViewNavigator. When using the NotesViewNavigator object you have the opportunity to use it's cache – "BufferMaxEntries". I set it to 100 in my case as the view will show no more then 100 rows at a time. I also set the EntryOptions to "VN_ENTRYOPT_NOCOUNTDATA" as I have no parent/child relationship in the view.
All in all the the running time of the agent went from 4000ms to 25ms! Pretty darn impressive if anyone were to ask me!
This technique is nothing new, but the performance gains are so huge I thought it's well wort repeating.
I was heavily influenced by this article: "Fast Retrieval of View Data Using the ViewNavigator Cache – V8.52" and I highly recommend you check that out for more details.
After re-running the performance tests in my local test environment I've "only" managed a 200% increase in performance. Beware, YMMV….