Friday, June 17, 2005

Mime Decoder Pipeline Component in Biztalk 2004 (the solution)

I promised to publish a solution to read the .eml files. This solution works for both Biztalk 2004 and Biztalk 2004 SP1 installations.



Step 1: Create a custom decoder component

You can download a wizard that will create any pipeline component here. Use this to create your decoder component. Make sure you have the following design time properties defined:


  • FromRegex: this will contain the regular expression to find the sender

  • FromMatch: this will contain the index of the Group and Capture

  • ToRegex: this will contain the regular expression to find the receiver

  • ToMatch: this will contain the index of the Group and Capture

  • SubjectRegex: this will contain the regular expression to find the subject

  • SubjectMatch: this will contain the index of the Group and Capture

  • PropertyNamespace: this will contain the namespace of the properties we're promoting (don't forget to create your property schema with MessageContextPropertyBase properties)


If you're not familiar with regular expressions, you can use the following expression to match the sender:
(?i)from:\s?([^\n\r]*)
If you apply this expression to the incoming message, the sender is found in Group 1, Capture 0. You can use this tool to test your regular expressions.



Step 2: Fill the gaps
Create a private member that will contain Biztalk's default decoder:

        private MIME_SMIME_Decoder containedDecoder = new MIME_SMIME_Decoder();


Some additional members well need:

        private RegExpMatcher fromMatcher = null;

        private RegExpMatcher toMatcher = null;

        private RegExpMatcher subjectMatcher = null;


We'll see the implementation of the RegExpMatcher later on.
Initialize the decoder:

        public void InitNew()

        {

            containedDecoder.InitNew();

        }


Make sure the component is able to load its configuration:

        public virtual void Load(IPropertyBag pb, int errlog)

        {

            string regExp = ReadPropertyBag(pb, "FromRegex") as string;

            string regExpMatch = ReadPropertyBag(pb, "FromMatch") as string;

            fromMatcher = GetRegExpMatcher(regExp, regExpMatch);

            ...

        }


        private RegExpMatcher GetRegExpMatcher(string regExp, string regExpMatch)

        {

            if (StringUtils.IsNotEmpty(regExp) && StringUtils.IsNotEmpty(regExpMatch))

            {

                return new RegExpMatcher(regExp, regExpMatch);

            }

            return null;

        }


Make sure the component is able to save its configuration:

        public virtual void Save(IPropertyBag pb, bool fClearDirty, bool fSaveAllProperties)

        {

            WritePropertyBag(pb, "FromRegex", fromMatcher.RegExp);

            WritePropertyBag(pb, "FromMatch", fromMatcher.RegExpMatch);

            ...


Additional properties:

        public string FromRegex

        {

            get

            {

                if (fromMatcher != null)

                {

                    return fromMatcher.RegExp;   

                }

                return null;

            }

            set

            {

                GetRegExpMatcher(ref fromMatcher).RegExp = value;

            }

        }

 

        public string FromMatch

        {

            get

            {

                if (fromMatcher != null)

                {

                    return fromMatcher.RegExpMatch;

                }

                return null;

            }

            set

            {

                GetRegExpMatcher(ref fromMatcher).RegExpMatch = value;

            }

        }


        private RegExpMatcher GetRegExpMatcher(ref RegExpMatcher aRegExpMatcher)

        {

            if (aRegExpMatcher == null)

            {

                aRegExpMatcher = new RegExpMatcher();

            }

            return aRegExpMatcher;

        }



Step 3: Create the Execute method

        public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)

        {

            /*

            * Get the original stream for the incoming message's

            * body part.

            */

            Stream stream = inmsg.BodyPart.GetOriginalDataStream();

 

            /*

            * Now, read the stream in blocks of 1024 bytes, until we

            * find the mime header. If the header is found at position

            * X, reset the stream to position X. The contained mime

            * decoder will start reading from this point forward.

            */

            int mimeHeaderIdx = -1;

            int num = 1;

            byte[] buffer = null;

            StringBuilder text = new StringBuilder();

            while (mimeHeaderIdx < 0 && num > 0)

            {

                buffer = new byte[0x400];

                num = stream.Read(buffer, 0, 0x400);

                ASCIIEncoding encoding = new ASCIIEncoding();

                text.Append(encoding.GetString(buffer, 0, num).ToUpper(CultureInfo.InvariantCulture));

                mimeHeaderIdx = text.ToString().IndexOf(MIME_HEADER);

                if (mimeHeaderIdx >= 0)

                {

                    stream.Seek(mimeHeaderIdx, SeekOrigin.Begin);

                }

            }

            IBaseMessage outmsg = containedDecoder.Execute(pc, inmsg);

            try

            {

                outmsg.Context.Promote("From", propertyNamespace, fromMatcher.Match(text.ToString()));

                outmsg.Context.Promote("To", propertyNamespace, toMatcher.Match(text.ToString()));

                outmsg.Context.Promote("Subject", propertyNamespace, subjectMatcher.Match(text.ToString()));

            }

            catch (Exception e)

            {

                throw e;

            }

            return outmsg;

        }



Step 4: Create the RegExpMatcher component
This is the class that will ease our use of regular expressions:

    public class RegExpMatcher

    {

        private int group = -1;

        private int capture = -1;

        private string regExp = null;

        private string regExpMatch = null;

        private DelimitedString delimitedString = null;

 

        public RegExpMatcher()

        {

        }

 

        public RegExpMatcher(string aRegExp, string aRegExpMatch)

        {

            if (StringUtils.IsEmpty(aRegExp))

            {

                throw new ApplicationException("Regular expression was empty.");               

            }

            if (StringUtils.IsEmpty(aRegExpMatch))

            {

                throw new ApplicationException("Regular expression match was empty.");               

            }

            regExp = aRegExp;

            regExpMatch = aRegExpMatch;

            CreateGroupAndCapture(regExpMatch);

        }

 

        private void CreateGroupAndCapture(string aRegExpMatch)

        {

            delimitedString = new DelimitedString(aRegExpMatch);

            if(delimitedString.Parts.Length < 2)

            {

                throw new ApplicationException(string.Format(@"{0} should be a delimited string having 2 parts.", regExpMatch));

            }

            group = IntUtils.ParseInt(delimitedString.Parts[0]);

            capture = IntUtils.ParseInt(delimitedString.Parts[1]);       

        }

 

        public string RegExp

        {

            get

            {

                return regExp;

            }

            set

            {

                regExp = value;

            }

        }

 

        public string RegExpMatch

        {

            get

            {

                return regExpMatch;

            }

            set

            {

                regExpMatch = value;

                CreateGroupAndCapture(regExpMatch);

            }

        }

 

        public int Group

        {

            get

            {

                if(group < 0)

                {

                    throw new ApplicationException(string.Format("Group should be an integer, found {0}.", delimitedString.Parts[0]));

                }

                return group;

            }

        }

 

        public int Capture

        {

            get

            {

                if(capture < 0)

                {

                    throw new ApplicationException(string.Format("Capture should be an integer, found {0}.", delimitedString.Parts[1]));

                }

                return capture;

            }

        }

 

        public string Match(string part)

        {

            Match m = Regex.Match(part, regExp, RegexOptions.Singleline);

            if (m.Success)

            {

                object found = m.Groups[this.Group].Captures[this.Capture];

                if (found != null)

                {

                    return found.ToString();

                }

            }

            return "";

        }

    }



Step 5: Build and deploy your component
Now you're ready to build and deploy the component. Just copy the resulting .dll to C:\Program Files\Microsoft BizTalk Server 2004\Pipeline Components and you're set.

Step 6: Use your component
After you successfully deployed your component, you're ready to use it:

  1. Add the component to your toolbox in Visual Studio

  2. Create a new Biztalk project

  3. Add a pipeline

  4. Drag your component on the decode stage of your pipeline

  5. Configure your component

  6. Test

2 comments:

Mark Brimble said...

Kenny,
I found this blog article very useful when I was trying to create an alternative to the POP3 adapter. Thanks. Can you post the following comment that might be useful to other readers of your blog.

To get the custom pipeline to work add the following to the project;

using System.Text.RegularExpressions;
using System.Globalization;

private const String MIME_HEADER = "MIME-VERSION:";

public class DelimitedString
{
private char[] delimiters = { ' ', ',', '\n', '\r' };
public String[] Parts = null;


public DelimitedString(string delimitedString)
{
Parts = delimitedString.Split(delimiters);
}

}

Add reference to "C:\Program Files\Microsoft BizTalk Server 2006\Pipeline Components\Microsoft.BizTalk.Pipeline.Components.dll".

Replace StringUtils.IsNotEmpty with another check fro an empty string.

Finally replace IntUtils.ParseInt with another way to parse a string as an int.

Mark Brimble said...

For a better way of doing this using a unknown property with just the MIME decoder see http://connectedthoughts.wordpress.com/2009/08/12/receiving-mime-encoded-email-files-and-a-hidden-pop3-context-property/