5 minutes of effort = 1600% increase in performance

Posted by:

So, the other day I was humming along for one of our larger clients – putting the finishing touches on one of their applications. The app in question handles matching of vacancies against applications. The administrator needs to quickly have the ability to browse what vacancies have what status. 

One vacancy can have one of three different statuses:

  1. View – Show the selected application for this vacancy.
  2. List – Lists all matching applications for this particular vacancy.
  3. Missing – No current applications exists that match the criteria for this vacancy.

Don't worry too much about the business logic, I only provide a quick background to give you some idea of what I was trying to achieve.

 

Oh, I nearly forgot! Of course this is an IBM Domino solution as we are a "True Blue" IBM Premium Business partner. Unfortunately this client was stuck with a fairly old IBM Domino server: 8.5.2, so I wasn't too keen on using an XPages solution. Why? Because in my humble opinion XPages just hadn't matured enough yet. (Missing HTML5 features, no SSJS debugger, old DOJO release, loooong "first hit" boot times and so on – Yes, all of this can be mended, worked around, patched etc. But sometimes you can just say enough is enough and go with old school Domino development.)

Anywho, my first inclination was to create a WebQuerySave agent that did the matching ServerSide before presenting the result to the user, a very common approach. But the performance was really bad – The WQS agent took around 4000ms although the workload was light and number of documents fairly low (in the hundreds anyway). The page itself took around a second to load with an empty cache, so we we're looking at a total page load time here in the neighborhood of five seconds – not acceptable.

So I used an approach that's known as perceived performance. Let's take a clock as example – If you use a clock that doesn't show seconds, time seems to move slower. If there's a lot of stuff going on, the general experience is that things are happening quicker. So I extracted my LotusScript code and placed it in an agent that get's called thru AJAX. This way the page loads quickly, the user can start to interact with it immediately and the state of each vacancy trickles in as the server sees fit.

Below is a screenshot of the application, waiting for the AJAX request to comeback with the data. (The only vacancies that know its state are the ones that are booked, they're marked with "Visa" in the column to the right, the rest are pending).

 

Finally, here we have the first icarnation of the code:

Nothing too odd about the above, I would even dare to say a fairly common approach. I do what I can to speed up the process by using NotesViewEntryCollections and the ColumnValue-property. Using one view as the source, that has a column that combines the particular criteria for that vacancy. The key can look something like this:

2015-05-27DSurgeon (Date + Slot + Role)

We use this key to find any matching requests, if none is found it's added to the string that's returned to the AJAX request. We only need the ones that doesn't have a match (Missing) as the first state (View) is stored with the vacancy and the second state (List) are all that are not missing. Ideally "Missing" should always be very few documents so the data transfer over the wire should also be low, increasing performance further.

 

So, everything was "hunky-dory" then. The application loaded quickly and felt responsive to the user, but… The bad performance of the LotusScript code was nawing at me… 

After a murky night of coding I came up with the following:

Option Public
Option Declare
Sub Initialize
	Dim s As New NotesSession 
	Dim db As NotesDatabase 
	Dim vRequest As NotesView
	Dim vMatchVacancy As NotesView
	Dim veRequest As NotesViewEntry 
	Dim veMatchVacancy As NotesViewEntry 
	Dim rowIds As String
	Dim key As String
	Dim i As Integer
	Dim count As Long
	Dim hasEmptySlots As Boolean
	
	Dim navMatch As NotesViewNavigator 
	Dim navRequest As NotesViewNavigator 
	Dim ve As NotesViewEntry
	
	On Error GoTo handler
	
	Set db = s.Currentdatabase

	Set vMatchVacancy = db.getView("vMatchVacancy")
	Set vRequest = db.getView("vRequest-list-match")
	
	Call vRequest.refresh()
	Call vMatchVacancy.refresh()
	
	Set navMatch = vMatchVacancy.createViewNav()
	Set navRequest = vRequest.createViewNav()
	
	count = vMatchVacancy.Allentries.Count
	i = 0
	
	' do not do AutoUpdates
	vMatchVacancy.AutoUpdate = False
	vRequest.AutoUpdate = False
	
	' enable cache for max buffering
	navMatch.BufferMaxEntries = 100
	' if we are not interested in the number of children, we can go a little faster
	navMatch.EntryOptions = Vn_entryopt_nocountdata
	

	Set veMatchVacancy = navMatch.GetFirst
	
	Print "content-type: text;charset=utf-8;"
		
	While ( i <  count )  
		hasEmptySlots = False
		key = veMatchVacancy.Columnvalues(0)
		
		Set veRequest = vRequest.Getentrybykey(key, True)
		
		If(veRequest Is Nothing) Then
			Print  rowIds & veMatchVacancy.Columnvalues(1) & ","
		End if
		
		Set veMatchVacancy=navMatch.getNext(veMatchVacancy)
		i = i + 1
	Wend

	
exitSub:
	Exit Sub 
	
handler: 
	MsgBox db.filepath & "agentMatchVacancys - " & Error & Chr(13) + "Module: " & CStr( GetThreadInfo(1) ) & ", Line: " & CStr( Erl ) 
	Print Error & Chr(13) + "Module: " & CStr( GetThreadInfo(1) ) & ", Line: " & CStr( Erl ) 
	
	Resume exitSub 
	
End Sub

The big difference here is the use of the NotesViewNavigator. When using the NotesViewNavigator object you have the opportunity to use it's cache – "BufferMaxEntries". I set it to 100 in my case as the view will show no more then 100 rows at a time. I also set the EntryOptions to "VN_ENTRYOPT_NOCOUNTDATA" as I have no parent/child relationship in the view.

All in all the the running time of the agent went from 4000ms to 25ms! Pretty darn impressive if anyone were to ask me!

 

This technique is nothing new, but the performance gains are so huge I thought it's well wort repeating.

I was heavily influenced by this article: "Fast Retrieval of View Data Using the ViewNavigator Cache – V8.52" and I highly recommend you check that out for more details.

 

[EDIT]

After re-running the performance tests in my local test environment I've "only" managed a 200% increase in performance. Beware, YMMV….

5

How to make Lotus Notes clients shine!

Posted by:

Previously we’ve talked about how to set up Parallels 8 to work properly on a Macbook Pro Retina. (You can read all about it over here), this final instalment isn’t targeted at Retina capable devices and thus should benefit everyone.

Now it’s time for some tweaking of the Lotus Notes clients themselves (Designer and Notes both) making them shine! Maybe you need to squint a bit, but the look will at least improve. Mission impossible? I think not! Before this article is over we’ll have a look at how to install an Eclipse plugin to manage themes. This will allow us to manage, share and edit the look of our working environment with ease. We’ll make some additional tweaks to the look and another couple of tips to improve and speed-up Domino Designer. But what are we waiting for? Let’s…

Get Started!

If you’re in a hurry – you can just download this setup file which I’ve handily prepared for your coding pleasure: Lotus Notes – Performance by Infoware

You will manually have to install my favorite font: Inconsolata
(Otherwise stuff will look way weirder then usual.)

You won’t get the theme plugin installed either, but you will get the color theme. Choosing this path I’ll assume you know what you’re doing, so I won’t go into any more details (like how to install the setup file). For the rest of you – Go get a cup of joe, because we’ll be here for a while…

Before we begin, last time around I mentioned a potential issue for Chrome users. It looked like graphical glitches when GPU accelerated CSS3 was involved. It worked fine in Parallels 7 as well as in IE10 and Firefox 15 under Parallels 8. Pretty weird stuff, if anyone were to ask me… But, fortunately, it worked itself out with an upgrade of Parallels. You can read all about the details (and workaround if it ever were to resurface) over here. Ok, on to the fun stuff: You’ve probably heard that Lotus Notes is now partially based on Eclipse? Well, if you haven’t – don’t sweat it. Just take my word for it (or have a look at this snasy Wikipedia article). As of 8.0 Lotus Notes client moved to Eclipse and from 8.5 and onwards we now get to enjoy(?) Eclipse in Domino Designer as well. True to their usual shenanigans, IBM won’t let us play with the all new and shiny: For Domino Designer 8.5.3 we’re stuck with the 3.4.2 version of Eclipse, also known as Eclipse Europe. The latest version, at the time of this writing, is Eclipse 4.2 (Juno). This has implications when using plugins, as we’re about to do.

The things I've seen

When things don’t go as expected

The plugins, that you’re installing in Domino Designer, needs to be compatible with Eclipse3.4.2 or horrible things may happen. You have been warned… If you’re running in a VM then just take a snapshot before trying out any new plugins.

Installing The Plugin

But, first things first! We need to enable the ability in Domino Designer to install plugins. This is the same procedure as you need to perform when you enable Source Control in Domino Designer. If you haven’t used SourceControl yet I highly recommend it, it will literally be a life changer! Your hair will grow back, your wife will look thinner and your kids will actually do what you tell them to! (Don’t know but the wife part, but otherwise it’s all good!) If you didn’t get to go to LotusPhere 2012 then here’s another couple of articles to get you started:

But, back to the matters at hand. To enable installation of plugins in Domino Designer: File / Preferences / Domino Designer – “Enable Eclipse plug-in install”

Enable Eclipse plug-in install

When “Enable Eclipse plug-in install” is enabled you’ll get additional menu options in Domino Designer: File / Application

Now you can manage your plugins here

  1. Click on File / Application / Install.
  2. In the following dialog, select: “Search for new features to install”.
  3. Click “Add Remote Location”
    Name: <Anything You Like> (I really went wild and chose: “Eclipse Color Theme”)
    URL: http://eclipse-color-theme.github.com/update
  4. Click “Finish”
  5. Make sure “Eclipse Color Theme” is checked.
  6. Accept the terms.
  7. Click “Next”.
  8. Click “Finish”.
  9. Select “Install this plug-in”
  10. Sacrifice a goat to the hacker gods and restart the client (The sacrifice is optional, but it can’t hurt)
  11. Done!

If all went as planned you should now be able to find the Eclipse Color Theme plugin in Domino Designer at: File / Preferences / General / Appearance / Color Theme

Eclipse Color Theme plugin in Domino Designer! Woot!

These are just the default themes to choose from. There are, at the time of this writing, 10240 more themes to choose from!
You can find the rest of the themes at http://eclipsecolorthemes.org as well as the plugin we just installed.

Someone might have noticed that this plugin doesn’t support Eclipse 3.4.2, but in my experience everything seem to work fine anyway.

My current favorite is “Zenburn” (bottom of the list). But do have a good look around for yourself, feel free to report back which one(s) you prefer or if you write your own.

Now, one might think that all is good and well. But, unfortunately, there’s still some kinks to work out on our quest to stardom. If you use LotusScript you won’t appreciate this new color scheme below:

Looks rather depressing, doesn’t it?

I don’t want an angry mob of LotusScript coders kicking my door down, screaming for blood! So, let’s fix this – quickly!

Fixing LotusScript

In Domino Designer – File / Preferences / Domino Designer / LotusScript Editor / Fonts and Colors

In the right pane, change:

  • “Normal Text” – Color: rgb(246, 243, 232)
  • “Identifiers”- Color: rgb(246, 243, 232)
  • “Keywords” – Color: rgb(223, 190, 149)
  • “Comments” – Color: rgb(128, 128, 128)
  • “Multi-line Comments” – Color: rgb(128, 128, 128)
  • “Constants” – Color: rgb(165, 194, 77)
  • “Directive” – Bold, Italic, Color: rgb(165, 194, 77)

Changing the font

While we’re at it, take the opportunity to change the hideous Arial font to something nicer. It’s easy! Close your eyes and take a pick! Or use the one I prefer: Inconsolato.

Now isn’t that better?!

But why stop there, we’re on a roll here! Let’s change to the same font for everything in the Lotus Notes clients:

In Domino Designer: File / Preferences / General / Appearance / Colors and Fonts.

– In the filter box, search for “fonts” (without the quotes) and replace everything with your new shiny font. I also took the opportunity to remove anything that’s set to bold, just because…

In Notes Client: File / Preferences / Basic Notes Client Configuration / Default Fonts

– Just set everything to the new font of your choosing.

Again, in Notes Client: File / Preferences / Fonts and Colors / Mail view font

While you’re in the preferences box, you might want to take a peek under: Windows and Themes / Theme and set to “Operating System Theme”.

I counted to no less then 9(!) different fonts for the different parts of the interface, now it’s much more consistent and hopefully a bit easier on the eyes.

If I haven’t bored you to tears just yet, I’ll finish you off with a couple of more tweaks:

Have a look at this excellent article by Nathan T. Freeman
Making Domino Designer work like you want

Also I recommend changing the memory config of Domino Designer with this little utility:
Designer Mem Config

Let’s do a summary, shall we?

We’ve enabled the ability to install Eclipse plugin in Domino Designer and touched briefly on the dangers of using plugins for newer versions of Eclipse than 3.4.2. We installed the Eclipse Color Theme plugin, configured that and made some much-needed adjustments to the LotusScript editor theme. I gave you a couple of links with tips of more tweaks to top it all of.

But that does seem like an awful lot of steps doesn’t it?! You feel like you’ve read all of the above and deserve some kind of reward, right? Sure you do! As I stated in the beginning I’ve saved a file that does all of the above (including Nathans tips and adjusting the memory configuration of Domino Designer) + some small tweaks of my own – thrown in their for good measure. (AutoSave every 15 minutes, “Right double-click closes window”, disable “Check Subscriptions” and disable “Enable Java Applets”

All you need to do is to install the Inconsolata font (or follow the instructions to change the font to something else) and import my settings!

If you followed Nathans advice then, in Domino Designer, you should have a tab that says “Package Explorer”. If you haven’t, then here’s where to find it:

Window / Show Eclipse Views / Package Explorer

  1. Under Package Explorer, where its white space – right-click.
  2. Select “Export”, from the popup menu. (We’ll get to the import, just make a backup copy of your preferences first).
  3. General / Preferences
  4. Export All
  5. Name it and save some place… Safe.

Then, Import / General / Preferences, Import All and select the file you downloaded from: Lotus Notes – Performance by Infoware.

Restart the client and…. Stick a fork in me, I’m Done!

If you believe I’ve missed something essential, please share in the comments below.

As always: follow @jBoive for the latest of the greatest!

4