Random observations of a very experienced software artist.

    LotusScript needs destructors too

    John McCann  August 6 2011 07:21:34 AM
    I was upgrading a major application to a new release.  When we deployed on the QA system and started running against the full data load, the overnight processing agent that goes in and touches a good percentage of the 100K records in the application started crashing the server.   We got the dreaded "LSXBE: ****** Out of Backend Memory *******"" error.    We reported the problem to IBM who asked for all the logs, copies of the database, etc.   After having a good laugh that they would not be getting a copy of databases, they eventually pointed us to a posting that effectively said, "The cause of the problem is not recycling Domino Java API objects correctly."   Only problem is that the agent was written in LotusScript, not Java.

    We had a number of false starts and red herrings (session.getDatabase will not return a database object if consistency check going on - duh).   The failing call stack, as shown below, had us focusing on DocumentCollection processing.  We were expecting a failure on getting or traversing a collection.  We never tracked it down exactly, but are pretty sure the actual failure was on a NotesDocumentCollection.getNextDocument statement.  The failing stack was:

    ### FATAL THREAD 1/3 [   nAMgr:  0f34:  0324]
    Exception code: c0000005 (ACCESS_VIOLATION)
    @[ 1] 0x60001706 nnotes.OSLockWriteSem@4+22 (19a)
    @[ 2] 0x61cf4a78 nlsxbe.ANNote::ANNAddToCollList+56 (0)
    @[ 3] 0x61d26127 nlsxbe.ANDocColl::ANDCNavigate+535 (0)


    In one of the iterations of trying to isolate exactly where the problem was occurring, IBM support (thank you Charles) pointed out that we just may be running into resource limitations on the machine since it looked like we were running into memory problems.  It seems we had all these objects sitting around in memory.   We started looking at the code and said, no, we clean up.  See, for each record in the primary database, we create an object, process it, then set the object to Nothing at the end of handling the record.  The object we create has other objects it creates, but LotusScript should be cleaning up all those when the primary object gets destroyed, right?   Seemed logical, only in Java do you have to do your own recycling.    Hmm, I mused.   Are there any known problems with NotesDateTime variables and memory leaks?  The big change in this upgrade was to add using NotesDateTime variables instead of LotusScript date/time variants since we started worrying about time zones and localizing date and times shown.   Charles looked it up and indicated nothing known there.

    So, just for chagrins, when adding the latest tracing to produce a bunch more messages, I went through and added destructors to all my classes and changed the set to nothing to delete statements.  (I also found out that unless a class has a Delete method, don't invoke delete on the class).    Lo and behold, the problem goes away.

    I have neither the tools nor the time to examine internal memory usage at points in time.  I will leave that to the Lotus developers.  However, what I think I have discovered empirically is one of two situations.  And, I don't know which one it is, so I left the code in for both thinking (perhaps wrongly) that it does no harm.

    1) LotusScript will not clean up a list of objects contained within another object
    2) A NotesDateTime contained within an object is not properly cleaned up when the object is destroyed.

    Simplified code before (real routine is over 8K lines of LotusScript)

    Option Explicit
    Public Class cFoo
    Public ndtEvent  as New NotesDateTime("")
    End Class

    Public Class cBar
    Public lstFoo List as cFoo
    Sub New
      Dim i as Long
      Dim oFoo as cFoo
      For i = 1 to 10
         Set oFoo = New cFoo
         Set lstFoo(i) = oFoo
      Next I
    End Sub
    End Class

    Sub Initialize
    Dim session as new NotesSession
    Dim dbCur as NotesDatabase
    Dim vwCur as NotesView
    Dim docCur as NotesDocument
    Dim oBar as cBar

    Set dbCur = session.CurrentDatabase()
    Set vwCur = dbCur.GetView("SomeView")
    Set docCur = vwCur.GetFirstDocument()
    While not docCur is Nothing
         Set oBar = new cBar
         ' **** do something with the document
        Set oBar = Nothing
        Set docCur = vwCur.GetNextDocument(docCur)
    Loop
    End Sub

    I have not compiled the above routine nor tested it, so it may have typos.  But, it should provide the basics of what was going on.   I assumed (bad choice by me) that the set of oBar to Nothing would clean up the memory of the cBar object as well as the 10 cFoo objects it created.   When I changed the code to have my own destructor routines and invoke Delete as appropriate, the memory problems went away.    As indicated above, I wasn't able to determine the root cause of the memory problem, so I shotgunned and cleaned up things that had changed (Datetime) and things I knew about.

    Option Explicit
    Public Class cFoo
    Public ndtEvent  as New NotesDateTime("")
    Sub Delete
      Set ndtEvent = Nothing
    End Sub
    End Class

    Public Class cBar
    Public lstFoo List as cFoo
    Sub New
      Dim i as Long
      Dim oFoo as cFoo
      For i = 1 to 10
         Set oFoo = New cFoo
         Set lstFoo(i) = oFoo
      Next I
    End Sub
    Sub Delete
      Forall Foo in lstFoo
          Delete Foo
      End Forall
    End Sub
    End Class

    Sub Initialize
    Dim session as new NotesSession
    Dim dbCur as NotesDatabase
    Dim vwCur as NotesView
    Dim docCur as NotesDocument
    Dim oBar as cBar

    Set dbCur = session.CurrentDatabase()
    Set vwCur = dbCur.GetView("SomeView")
    Set docCur = vwCur.GetFirstDocument()
    While not docCur is Nothing
         Set oBar = new cBar
         ' **** do something with the document
        Delete oBar
        Set docCur = vwCur.GetNextDocument(docCur)
    Loop
    End Sub

    Let me know if this solves a memory problem for you and I will pass the feedback to Lotus support that there really is a problem in this area.

    2011-08-18 Update: It has been suggested that delete method for cBar Erase the lstFoo element instead.   Not having the time to test all the options, I've updated my production code to do both - issue the delete then Erase lstFoo.
    Comments
    No Comments Found