With v0.8, I added the feature “don’t create the same attachment, in the same attachment folder, if it already exists.”

And I added a bug.  The bug would crash LNME when this happened:

1. Find duplicate attachment file name
2. Extract the new attachment to the %temp% directory.

It would crash if the “new” attachment was corrupted.  And here’s the irony, the old section of code had a trap for exactly this sort of behaviour.  When I coded the new section, I forgot to put in the error trapping.

Doh!

Like I said for v0.8, the next version (v0.91 maybe) will use a list table, which is a better way of detecting duplicate files/attachments across directories.

Other changes for the future:

• Export task entries
• Export to Outlook
  (unlikely to happen until Microsoft publishes the PST file format)
• Rich Text Export(?)
  An example of an Rich Text object is a picture which has been pasted into the message body.
• Appointments & Meetings are missing the body of the meeting/attachment.
• Tasks

You may download LNME v0.9 here.

Yesterday, I talked about file hashing in Visual Basic.  There was a reason, and that was that I updated my Lotus Notes Mail Exporter program with some limited duplicate attachment checking.

Limited because it’s simple.  Simple as in:

• check to see if a file attachment with the same name already exists.
• if it doesn’t, save it to disk.
• if it does, check to see if it is a duplicate.  If it is, don’t save the “new” attachment.

Can you see the flaw in my thinking?

No???

Well the BIG FLAW is that if the file attachment we’re trying to save has a unique name, we don’t check it to see if there any duplicates of it already.

The next version of LNME will fix that by using a list table (see yesterday’s post – it’s the second detection method).

The other change in LNME is that the saved message file will have it’s message creation/modified/last access date set to the actual email send date.

You can download LNME here.

Duplicate file detection is fairly easy to do, as it turns out, and it goes like this.

1. Read first file.
2. Calculate a unique fingerprint, and store the fingerprint.
3. Read second file.
4. Calculate a unique fingerprint, and compare the fingerprint to the first files’ fingerprint.
5. If they equal, the two files are duplicate files.

One way of doing this fingerprinting is to use a hashing algorithm, such as MD5 or SHA-1.  A hashing algorithm should give you a unique fingerprint for each file.  The snippet of code to do that, which I borrowed from the Visual Basic Knowledgebase is below the fold.

Now there are applications, such as DiskState, which scan your hard disk for duplicates by using MD5 hashing.  But they do this:

1. Read first file.
2. Do While we have a file to read
      Calculate a MD5 Hash, and store that value in a list, alone with the filename and location
3. If we have another file, go back to 2.
4. At this point, we’ve checked all the files on hard disk. 
      Now search though the list, and do we have any hashes that are the same?
5. If we do, we have duplicate files.

Why do I need to know how to detect duplicate files?

Well I was updating the Lotus Notes Mail Exporter program the other day, and decided to implement some duplicate attachment file checking.  I’m not sure if I’m that happy with how I’ve implemented it, time will tell.  It does give me an idea for some other programs …

(more…)

Threading, simply put, is a way for your program to do multiple things at once.

Sort of like driving and talking on the cell/mobile phone at the same time.

Yes you can do both, but they can cause bad things to happens.  Like driving through red lights, or having your computer freeze on you.

Coincidently, I use threading in my Lotus Notes Mail Exporter (LNME) program.

The part which does the actual mail message exporting, runs in it’s own thread.

I needed to use threading as I wanted the program screen to update in real time.  Before I implemented a BackGroundWorker thread, the program wasn’t able to update the Status section (highlighted above).  Using threading, it was simple.

But it was difficult to implement, because most programming examples for threading don’t explain the WHY of using a particular thread techique.
So I made a guess to use a BackGroundWorker Thread.

Which turned out ok.

Though, had I seen Joseph Albahari’s website, and the section on Threading in C#, I might have done it differently.

Why?

Well Joseph has written a guide to Threading in C# which forms THE reference to threading.  It’s packed full of information, and I wish I had it when I was starting to write LNME.

Go there, and have a read.

Which is why I purchased the Outlook 2007 Programming book.

To import messages into Outlook, what I need to do is set the Outlook Sender field to something other than “me”.

Why?  Well I’m trying to import messages from Lotus Notes, and I want to copy the Lotus Notes “From” field into the Outlook “From” field (aka Sender).

Unfortunately, the Outlook Object Model only has this Sender field as ReadOnly.

Searching around the web hasn’t helped, but looking at the Amazon “Look Inside” view of this book, it looks like there is a couple of other things I can try.

My fallback position is that I could write all the messages out into a Eudora format (mbox), and have the user import the messages into Outlook that way.

but they did.

Had a report from a user of LNME, that LNME was producing an error message, then crashing.

The cause that an message attachment had a TAB character in it’s filename, and Windows wouldn’t accept that, so it would error.
(how the TAB got into an attachment name is another question… , for another day)

Now, if you believe Windows, there is only a small number of characters it doesn’t like in file names:

… which proves to be utter tosh.  And Microsoft provides you with the proof with the Path.GetInvalidFileNameChars method.

If you use the Path.GetInvalidFileNameChars method, it will tell you there are actually 41, including both TABs.

So now, before LNME writes out the message and attachment files, LNME will now check them for more invalid characters.

What this means is …
You shouldn’t notice MUCH difference in speed, as originally I was blindly checking every filename by doing something like this:

If FilenameField <> “” Then
‘Trim Leading and Trailing space from FilenameField Field
FilenameField = FilenameField.Trim

FilenameField = Replace(FilenameField, “/”, “[SLASH]“)
FilenameField = Replace(FilenameField, “\”, “[BACKSLASH]“)
…
End If

Now I check first to see if the FilenameField has invalid characters, and if so, then do the replacement of “invalid” characters.

‘ get the list of invalid characters for this operating system
cInvalidFileNameChars = System.IO.Path.GetInvalidFileNameChars

‘ check if any of the invalid characters are in our FilenameField
iReturnValue = FilenameField.IndexOfAny(cInvalidFileNameChars)

‘ >0 means that we found some invalid characters.
If iReturnValue >= 0 Then
If FilenameField <> “” Then
‘Trim Leading and Trailing space from FilenameField Field
FilenameField = FilenameField.Trim

FilenameField = Replace(FilenameField, “/”, “[SLASH]“)
FilenameField = Replace(FilenameField, “\”, “[BACKSLASH]“)
…
End If
End If

The performance improvement appears because the majority of Subject and Attachments are correctly named, so the previous blind check will be skipped.

You can download the fixed LNME, here.

The major change is that LNME now supports exporting messages from a particular date.

Date to export from
Why Sunday, 1 January 1989?
Lotus Notes 1.0 was released in 1989, and I’m making the assumption that you won’t have messages older that.

Delayed Write Errors
If you get “Delayed Write Errors” from Windows, it most likely means that you’re running LNME over a network, and the network is having issues.

For the technically minded, this seems to occur when you are extracting attachments.  The culprit is the Lotus Notes ExtractFile function.  It loses the network, or causes the network to drop out.  I could write a bit of code to detect if the network is still there before calling ExtractFile, but it will be a massive performance hit.

The workaround? Ensure that the “Output Folder” is written to the local PC.

What’s next?
I’d really like to get “Export to Outlook”  & “Export Task entries” implemented.  It’s just a matter of finding the time.

Where do you get it?
By clicking here, and then clicking the download link.

is a small program I’ve written to export attachments and emails from the Lotus Notes email client.

Now there are commercial programs which can do the same thing, but since I had a little time on my hands, I wrote LNME.  Learnt a bit about Lotus Notes programming at the same time.

It’s difficult to code for Lotus Notes, which is why you don’t see many code examples on the internet.

And I found that Visual Studio 2008 is more forgiving of syntax errors than VS 2005.  Things which would just work in VS 2008, would throw an exception in VS 2005.

When I get some more time, I will be making further improvements to LNME.

Anyhow, you can read further about LNME, and download it, here.

Much of what is out there is plain wrong, which is not surprising.  Of the stuff which does work, it’s poorly documented.  That does not surprise either.

Here are a couple of pointers for novice Lotus Notes/Visual Basic programmers.

Lotus Notes and Domino 6 Programming Bible
Get a copy of the Lotus Notes and Domino 6 Programming Bible.  It is considered a good source for Lotus Notes programming.  And if you want to do Lotus Script or JavaScript programming, it’s really helpful.  After having done some Notes related programming, I’m going to re-read it to pick up what I missed the first time around.

NotesPeek
I spent plenty of time peering at NSF files using NotesPeek.  NotesPeek let you look into the structure of Notes’ database files.  A much simpler way of working things out, then the “French Cafe” technique.
I’ve also used NotesPeek allowed me to identify an issue with a popular Notes <-> Windows Mobile synchronisation product.
I’m very grateful that Ned Batchelder wrote it.  You can download it here.

Some websites/articles
Lotus Notes Ninjas
Notes411
Lotus Notes Mail Exporter – my first Visual Basic/Lotus Notes program
OpenNTF – Detach files using COM
Language differences between LotusScript and Visual Basic
IBM – General Instructions for Using Component Object Model (COM)
Common ground- COM access to Domino objects
Calling Notes CAPI from C#/Visual Studio