« 8.5.3 installation - What to do if you encounter "Unexpected results have occurred during the provisioning operation" | Main| Can the 10-year old that wrote Microsoft MSI please step forward »

Notes client Bookmark Export and Import database

Tags: DXL Lotus Notes LotusScript
0
From time to time I like to completely reinstall Notes, because Notes, like Windows, accumulate issues  (oh, "features" we call them, he he ...) over time. So just like reinstalling Windows once a year, I try to reinstall Notes too. One of the drawbacks with complete reinstallation, is that you loose settings, amongst them  Workplace and Bookmarks. This means that I have to manually recreate my bookmark folder structure, opening up a bunch of databases and remember to add them to bookmarks. Boring, tedious and error-prone.

I have therefore created the Bookmark Export and Import database, which let you export your bookmarks to a backup document, and then later import it again. Below you see a screenshot of my latest backup document;

A picture named M2

Yup, nothing fancy in regard of design here The backup document contains two DXL files (which the Export agents creates for you) along with a bunch of other fields to keep track of all relations in the original bookmark.nsf file. Below you see the content of the About-document
Export and Import Bookmarks

Version: 1.0

Created by Bob Voith, Proud Programmer (http://www.proudprogrammer.no)

Why I created this

One of the challenges with installing Notes from scratch, is that you loose your previously settings, such as Workspace, Bookmarks and other settings. Now, you can  copy your old bookmark.nsf, desktopX.ndk (replace X with 6 or 8) and make the Notes client use those files instead. This probably works fine for most people, most of the time!

However, I have always felt somewhat uneasy about this approach, in the way that I would like to have real fresh, new versions of those important files, all on the latest ODS and with all bells and whistles onboard. Below you see the fresh new bookmarks just after a new install;

A picture named M2
So for me, after a fresh reinstall, I have always used some time to recreate manually my bookmarks. Time consuming and somewhat boring.

For some reason Lotus has never really revealed very much about how Workplace and Bookmarks are stored in their bookmark.nsf and desktopX.ndk, nor do we have much APIs to access or modify that information.

This little database contains a script library and an Export and Import agent, which let you save your bookmarks (yup, bookmarks and -not- Workplace) to a backup document. Everything is contained within a single backup document, and you can restore/import any backup level whenever you want. It will recreate your bookmark folders, and all content within each folder.

When I have imported my previous bookmarks, and restarted the client, it looks like this;

A picture named M3
Jiiha, no need to manually recreate the folder structure!


How to use
  1. Before anything - sign this database, and ensure you are the manager of this database - or at least designer - as you need to create folders!
  • Before you upgrade your Notes client, run the Export agent  within this database. It will save a backup document with all information. Save this database somewhere safe (he he, no point in having a backup of your bookmarks in this database, if the first thing you do next is to delete your Notes data directory!)
  • Uninstall your existing Notes client, and insure that the data directory is gone (or saved somewhere, whatever)
  • Install the new Notes client
  • Copy this database from your safe-house, and run the Import agent.
  • Restart the client as soon as the agent above is finished. Why? Because Notes loads and maintains the bookmarks constantly, and in order to have your Notes client pick up the changes, it needs to restart.
    How does it work

    I was peeking around in the bookmark.nsf with Notespeek, and found that the bookmarks are actually real folders. Below you see such a folder note;

    A picture named M4
    A folder note contains several fields, such as $TITLE, $Formula, $Collation, $Collection and $Flags. One of the important fields are the $Collection field, which contains the View-object. If you try to just copy folder notes from one database to another, the $Collection-field won't be copied. This actually has sense, since a folder points to documents within this database, ie the bookmark.nsf. I actually tried to grab the $Collection field and explicitly copying it to another document, and I got a not-seen-before error, telling me that collections cannot be copied. The $Collection-field points to a bunch of URL-documents, which I also need to copy over.

    Another important field is the $Flags-field - seen in the screenshot above containing the 3PFY code. I use that to extract exactly the bookmarks folders and nothing else!

    In essence this means that I need to copy the folder notes from the old bookmark.nsf to the new one, and since $Collection-fields aren't copied over, I must repopulate the folders in the new bookmark.nsf. Now the code boils down to keeping track of what URLs are in each folder.

    Another challenge, is to recreate folder notes exactly as they should be with all their fields and content. The same goes for all the URL documents.

    I use DXL to to this - I just love DXL!. I simply export the folder-notes as a NotesNoteCollection to a file and attach it to the backup document. The same goes for URL documents, but now I export NotesDocumentCollection. Below you see a code-snippet to first create the NotesNoteCollection with folder-notes;

    ' Create a Note collection containing the folders
    notecollFolders.SelectionFormula = |$Flags = 3PFY"|
    notecollFolders.SelectFolders = True
    Call notecollFolders.BuildCollection

    ...and then the code to export them as DXL;

    ' Export the Folders-design element collection to the specified file
    Dim exporterFolders As NotesDXLExporter
    Dim streamFolders As NotesStream
                   
    Set streamFolders = m_session.CreateStream()
                   
    Call streamFolders.Truncate
                   
    Set exporterFolders = m_session.CreateDXLExporter(notecollFolders , streamFolders)
    exporterFolders.ForceNoteFormat = True
    exporterFolders.OutputDOCTYPE = False
                   
    Call exporterFolders.Process
                   
    Call streamFolders.Close()

    Inside the Export-method I build a bunch of dynamic fields (meaning that I haven't specified them in the form) which simply holds the name of the folder, and each URL inside the folder. The FolderCount  field contains the total number of exported folders.

    The Folder0  contains the name of the first folder up til FolderN. For each folder you will find FolderNURLX, fields, where Folder0URL0  contains the first UNID of the URL within Folder 0, and Folder0URL8 would contain the 7'th UNID (remember, 0 based index).

    This makes it pretty easy during Import  to loop through all the fields and connect documents to folders. Below you see a code snippet from the Import-method;

    iMaxFolderCount = GetFieldContentAsInteger(docBackup, "FolderCount")
    For iCurFolder = 0 To iMaxFolderCount -1
            strFolderName = GetFieldContentAsString(docBackup, "Folder" & CStr(iCurFolder))
                           
            iMaxURLCount = GetFieldContentAsInteger(docBackup, "Folder" & CStr(iCurFolder) & "URLCount")
            For iCurURL = 0 To iMaxURLCount - 1
                    strURLUNID = GetFieldContentAsString(docBackup, "Folder" & CStr(iCurFolder) & "URL" & CStr(iCurURL))
                    If IsElement(listURLSByUNID(strURLUNID)) Then
                            Print "Connecting URL # " & CStr(iCurURL + 1) & | to folder "| & strFolderName & |" ...|
                            Set docCur = listURLSByUNID(strURLUNID)
                            iURLTotal = iURLTotal + 1
                            Call docCur.PutInFolder(strFolderName)
                    End If
            Next
    Next

    The real core, besides keeping track of UNIDs, is the single PutInFolder call which actually put the URL inside the folder.

    Enjoy!

    And by the way, you download the database here. (and now the link works, thanks Jola)

    Update 2: My goodness, great balls of fire! Now the database isn't encrypted anymore! Thanks for pointing out! Obviously still in a frenzy aftermath here... sorry!
  • Comments

    Gravatar Image1 - hi,

    Error 404

    HTTP Web Server: Couldn't find design note - download/BEI

    ???

    Gravatar Image2 - Sorry - try { Link }

    I fix tomorrow!

    Gravatar Image3 - Is there any chance that this could be extended to include the workspace tabs and icons? Our users still rely on the good'ol workspace, but it's always a pain in the a** to solve Notes Client issues that require to delete and recreate bookmark.nsf & desktopX.ndk, for example.Emoticon

    Gravatar Image4 - Sorry Robert,


    your database is encrypted !!!

    Gravatar Image5 - Same here - the database cannot be opened because the local encryption is enabled Emoticon

    Gravatar Image6 - Sorry all, for encrypting the database! I should now be fixed!

    Gravatar Image7 - Patrick; Sorry, the Workplace is a completely other ballgame, and much more hidden compared to bookmarks. As it turned out, bookmarks were simply folders, but with a different UI to it, and thus I was able to understand how it works. Workplace on the other hand, has always been hidden for programmers. Not even the C API contains anything about icon manipulation. My best guess is to study how far Policies goes. In the later versions of Notes/Domino, earlier issues has been ironed out, and far as I understand it, you now have pretty good control over what icons to place where. If you don't reach the summit completely with standard Domino policies, then take a look at products like Pangenda MarvelClient { Link } or Cooperteam Desktop Manager { Link }

    Gravatar Image8 - Hi, Just ran this and am getting - "Export failed with error code 1". Any ideas?. Thanks

    Gravatar Image9 - @8: Export error code 1 means that I didn't find any bookmarks with the $Flags containing "3PFY". This leaves me with this; Either you haven't got any bookmarks (note, not Workplace icons, but bookmarks), OR - my code perhaps have an issue with older bookmarks flags or something. If you send me your bookmark.nsf file, I can analyze it for you.

    Gravatar Image10 - Hello Robert,

    we have the same problen (Export failed with error code 1).
    Bookmark ist there!


    Thanks

    Gravatar Image11 - Anybody experiencing problems with Export, you are all welcome to send me your bookmark.nsf file, and I will analyze it for you. Probably some old flag values.

    Gravatar Image12 - The database is encrypted??? Emoticon

    Gravatar Image13 - @12 - John, I had this issue with the first download, so if you haven't downloaded it again, head over to { Link } to download it again. The zip file contains a NTF file (template) which you can create your own database from. Hope this helps!

    Gravatar Image14 - Hi Rob
    Had to make a few minor changes to get it to run on v7 client, and also found an issue where the designer bookmarks clashed as they duplicated others - due to the add funtion failing. ( remove extra bookmarks to fix ).
    Importing appears to run ok but I can't see any changes ( after restarting client ).
    Chris

    Gravatar Image15 - Hi Bob,

    I have 1 problem, and it might sound funny, but how to run the export or the import agent within your DB?

    Gravatar Image16 - Hi Rob,

    this might sound funny, but can you or somebody pls tell me how to run the Export (or inport) from within the DB?

    Thanks :)

    Gravatar Image17 - oh, the actions menu....
    I'm so blind Emoticon Emoticon Emoticon

    Gravatar Image18 - Hi Robert. Can this be used to export from an old laptop to a new laptop? I tried it and got "Import failed with error code: 4." I think maybe the application doesn't know where to look for the dxl files. Maybe I need to stick the dxl files somewhere special?

    Gravatar Image19 - Hi Bob,
    I downloaded the database today, got the Export Error code 1. When debugging the LotusScript, it appears you are looking for bookmarks in "C:\Backup\old 852 Bookmark.nsf"

    I was able to perform an export after copying my C:\Notes\Data\Bookmarks.nsf to that location/filename

    Post A Comment

    :-D:-o:-p:-x:-(:-):-\:angry::cool::cry::emb::grin::huh::laugh::lips::rolleyes:;-)