« What's up with my Vista "Save as" dialog box? lmpgspl.ax it was!! | Main| Part 3: Using DXL to create imagery in documents »

Part 2; Programming details on Icon and Images - Enumerating files

Tags: Lotus Notes Software DXL LotusScript
In the first article I introduced the Icon and Images database, in which I store more than 1.8 million icon files in a single database. This article dive into the programming details, showing how I enumerate the files to import.
The Icon and Images database primarily consist of two distinct passes, where the first is to find out which files you want to import. I call this the enumeration  pass. The second pass is to create the icon imagery (the tables with the icons, visible to the user) and attach the files themselves.

Pass 1 - Enumerating the files to import

In order to import icons and images, I need to know where they are stored. I also need to know a lot about the directory structure the icons have. As illustrated in the previous article, the VirtualLNK icon collection has 8 different directory structure types. I need to know the structure in order to grab icon attributes like state,  shadow,  and size  for each icon. Lets take a look at an example. The Network_V2  zip file consists of yet a bunch of zip files, like this;

A picture named M2

When you dive into a sub-zip file you find that this library has a structure like this;

G:\Temp\Network_V2\Network_V2\PNG\PNG\Regular\No Shadow\48x48

Note that this directory structure not necessarily is valid for other zip files!! In order to control this, I have identified the different directory structures available, and rather created a Database Configuration document, which let me specify parts of the directory  mapped to the appurtenant directory structure, like this;

A picture named M3

In the above screenshot I can tell that any icon file which contain the Network_V2  anywhere in its path (like G:\Temp\Network_V2\Network_V2\PNG\PNG\Regular\No Shadow\48x48\Antenna.png,.,) will have the directory structure equal to 2. The available directory structures I have are;
Type Path
<Base dir>\<Library>\<Library Version>\<Shadow>\<Shadow>\<Image Format>\<State>\<Size>
<Base dir>\<Library>\<Library Version>\<Image Format>\<Image Format>\<State>\<Size>\<Modificator> (ICO same level)
<Base dir>\<Library>\<Library Version>\<Image Format>\<Image Format>\<State>\<Shadow>\<Size> (ICO same level)
<Base dir>\<Library>\<Library Version>\<Image Format>\<Image Format>\<State>\<Size> (ICO same level)
<Base dir>\<Library>\<Library Version>\<Image Format>\<Image Format>\<State>\<Size>\<Modificator> (ICO one less level)
<Base dir>\<Library>\<Library Version>\<Image Format>\<Image Format>\<State>\<Size>\<Shadow>
<Base dir>\<Library>\<Library Version>\<Image Format>\<Image Format>\<State>\<Size> (ICO one less level)
<Base dir>\<Library>\<Library Version>\<Image Format>\<Size> (ICO one less level)

To best understand the table above, start reading it from right to left. When I state that the ICO level is either at the same  level or one less  level, it indicates that the ICON files doesn't have all the levels of paths as the other files. This makes sense since the icon files often have all available sizes baked into a single ico or icns file. Thus they don't need the <size> part.

The different path-part names in the table above are;
Path Part Description
<Size> The size of the images, typically "256x256", "128x128", "16x16". Below you see a sample of the same icon ranging from "128x128" and down to "16x16";
A picture named M4 A picture named M5 A picture named M6 A picture named M7 A picture named M8 A picture named M9

<State> The state of the image, typically "Hot", "Disabled" or "Normal".Very often icons used for toolbars in applications has such states. Below you see the same icon at size "48x48" in its three different states, Normal, Disabled  and Hot;
A picture named M10A picture named M11A picture named M12
<Modificator> Several VitualLNK icon collections has a special organization where they have a so-called base image. This is the pure and clean base image. Then they have modificators "on top" of the base image, such as for "Add", "Delete", "Check" etc. Below you see a small sample of this;
The base image; A picture named M13
The base image with the "add" modificator; A picture named M14
The base image with the "chat" modificator; A picture named M15

The purpose of the modificators is to make it easy to have variations of the same icon.
<Shadow> The shadow of the icon indicates whether the icon has an shadow or not, like this;
No shadow; A picture named M16
Dark shadow; A picture named M17
Light shadow; A picture named M18
<Image Format> The image format is first and foremost it's filetype. Be aware though, that an image type like BMP may occur in two different modes, such as "BMP" and "BMP_Mask". The same goes for "ICO" files which often comes as "ICO_8Bit" and "ICO_24bit". The way I unpack the directores, I often end up with two consequent image formats path-parts, where the right-most always are the image type (BMP, PNG, ICO, GIF etc) and the next right-most are the image format as described.
<Library Version> The library version is typically the version path-path like "Network_V2" or "Business V3"
<Library> The library is the main library name, like "Network" or "Business"
<Base dir> The base directory is the left-most base directory, such as "G:\Temp"

Also please note that there are several ways of unpacking such zip files. I have chosen to unpack into separate folders - always. This is extremely convenient with WinRAR, which I use as my primary ZIP tool.

The pass 1 dialog box

A picture named M19

The above dialog box let the user specify the vendor, which directory structure type he or she is about to import, and the base import folder. Note that the dialog box above has selected AUTOMATIC for the directory structure type . This means that we use the directory-to-type map described earlier in this article. It makes it much much easier to import huge numbers of icons.

If I dive into the LotusScript code behind the pass 1 agent, I the main logic is pretty simple. First of all I use a recursive function  to walk the import folder. This function will basically enumerate all the files within the current level of directory structure. It will also remember all the directories within the current directory structure. For each sub-directory it will call itself (this is where the recursive  part comes in...) and do the same enumeration again for the next level in the directory structure. This way it walks its way through all directories beneath the initial base directory.

The code looks like this;

Function EnumerateImportFiles(session As NotesSession, _
db As NotesDatabase, _
view As NotesView, _
pstrPath As String, _
iRecursive As Integer) As Long
EnumerateImportFiles = 0 ' Default; No files found
filename = Dir$(pstrPath & WILDCARD_ALL_FILES, 16)
Do While (Not filename = "")
  If filename <> "." And filename <> ".." Then
     strFullFile = CreateFullPath(pstrPath, filename)
     lAttributes = Getfileattr(strFullFile)
     ' Do we have a directory ? If so, remember that 'till later!
     If lAttributes And 16 Then
        iNbrDirectories = iNbrDirectories +1
     listDirectories( iNbrDirectories) = strFullFile
      ' If iBasePathType is -1 it means that we have
     ' automatic base path type matching with the map
     ' specified in the Database Configuration document
     If g_iBasePathTypeFromUI = -1 Then
       iBasePathType = GetBasePathTypeFromMap(session, _
       db, strFullFile)
       iBasePathType = g_iBasePathTypeFromUI
     End If
     ' The next function split the file path into their
     ' respective parts. Note how the first parameter
     ' define how to analyze the path according to directory
     ' layout in VirtualLNK
     Call SplitPathIntoPathParts(iBasePathType, strFullFile, _
     strFPath, strFName, strFExt, strImageType, strState, _
     strShadow, strSize, strLibraryVersion, strLibrary)
     If  (iBasePathType <> -1 And (strFExt = "JPG" Or strFExt = "GIF"_
     Or strFExt = "PNG" Or strFExt = "ICO" Or strFExt = "ICNS" _
     Or strFExt = "BMP"))  Then
      lFileCount = lFileCount + 1
      Print "PASS 1: Enumerating file # " & Cstr(lFileCount + _
      g_lNbrFilesToImport) & " in library " & strLibrary


     End If ' end If iPathNbrOfParts > 5
      End If
     End If
  filename = Dir$
   ' If directories was found -and- we want to search sub dirs, then do that now
   If  iRecursive = 1 And  iNbrDirectories > 0 Then
   Forall strDirectory In listDirectories
   lAttributes = Getfileattr(strDirectory)
   If lAttributes And 16 Then
      If Right(strDirectory,1) <> "\" Then strDirectory = strDirectory _
      & "\"
      EnumerateImportFiles = EnumerateImportFiles + _
      EnumerateImportFiles(session, db, view, strDirectory, iRecursive)
      End If
   End Forall
   End If      
End Function

Some notes about the code above.

As you see, I use the LotusScript Dir$  function to list the files and directories. To make it easier for myself, I store all directories  in a temporary list, and all files are processed right away, To determine whether a file is a directory or a file, I use the GetFileAttr-function.

In order to understand what VirtualLNK directory structure type we are working with, I have a function named GetBasePathTypeFromMap, which return the directory structure type number for the file. This is used as input parameter to the function SplitPathIntoPathParts, which breaks apart the path for each file, and return the path parts. This is where it so important to really have control of the directory structure! If I recognize the file as a image file I want to process, then the code ends up at The REAL PEPPER GOES ON HERE. In the real code I basically check whether I have imported the file before or not. If it is a new file, I will create a new formIcon  document in the database.

How do I understand the uniqueness of an icon? By combining the Library Version  with the file name part  of the icon, I have the key  of the icon. This means that all variants of for example the Accountant icon will be collected within the very same Notes document. By also including the library version-part I can have multiple icons with the same name across multiple libraries.

When pass 1 is finished, I have a bunch of unique icon documents, and each document contains a Files -field containing the full path to every icon file. The number of files are also seen in the main view;

A picture named M20

The next article

The next article will dive into how I create the icon imagery in the documents. That will be a heavy-duty DXL article.


Gravatar Image1 - Love to read the next "heavy-duty DXL" article. Since we use the icons ourselves, any change you want to share the design ?

Post A Comment