MSMQ between two computers

According to Wikipedia, Microsoft Message Queuing or MSMQ is a message queue implementation developed by Microsoft and deployed in its Windows Server operating systems since Windows NT 4 and Windows 95. It's a super-easy method of Inter-Process Communication in Windows.

Recently I wanted to use Hangfire to coordinate jobs across internal and external web servers, but Hangfire has a 15s delay thanks to it's use of polling SQL Server. You can use Redis or MSMQ to reduce that delay to "instantaneous", but these are two separate computers, so how do they communicate using MSMQ? There are a few gotchas, as you'll see below!

This is a proof of concept, and has not been subjected to security or reliability concerns. This method will just enable you to get going, after which you'll need to lock down firewalls, access, etc.

Concept

The concept is that we have two processes running on separate devices, and two queues for coordinating work. The "internal" process can write to the "external" queue, and the "external" process can write to the "internal" queue. Each process reads off their own queue.

Think of this like an external (public facing) web server, and an internal (back office) web server. The external web server may need to queue up jobs such as password recovery emails, and the internal web server may need to send messages to the external web server to invalidate a cache after an administrative change.

The queues and SQL server exist only on the internal device.

Firewall

Your Windows firewall needs to be opened up on both computers:

1. allow incoming ports 135, 1801, 2101, 2103, 2105 and 2107
2. allow all ICMP traffic
3. allow the incoming program msdtc.exe (usually in %SystemRoot%\System32\)
4. allowing incoming access to SQL Server on the internal device

Note that these ports can be abused, so you should secure them only between the communicating devices.

MSMQ

MSMQ is setup on the internal device:

1. You may need to add MSMQ in the control panel "Programs and Features" > "Turn Windows features on or off":
   
2. Open computer management and navigate to Services and Applications > Message Queuing > Private Queues
3. Right click on Private Queues and choose New > Private Queue
4. Give it a name, such as "hangfire-internal" and select "Transactional"
5. Click OK, then find the queue in Private Queues, right click and choose Properties
6. give everyone full control (for a production application, you would restrict service accounts to certain access)
7. Follow steps 3. to 6. and create another queue called "hangfire-external"

On the external device, follow step 1.

SQL Server

Create a database and make sure SQL Server is accessible to the external computer. More information can be found here:

https://msdn.microsoft.com/en-us/library/ms345343.aspx

Distributed Transaction Coordinator (DTC)

1. Enable Transactional Remote Receive:
1. Open Component Services and navigate to Component Services > Computers > My Computer > Distributed Transaction Coordinator > Local DTC
2. right click Local DTC and select Properties
3. In the Security tab, enable Network DTC Access, Allow Remote clients, Allow Inbound, Allow Outbound, and Mutual Authentication Required:
   
2. Start the Distributed Transaction Coordinator Service

Follow these steps on both the internal and external device.

Testing

Here is a console app that tests Hangfire and MSMQ on the two configured devices. It's basically a text chat app.

1. Create a Visual Studio console application for C#
2. add the packages Hangfire.Core, Hangfire.SqlServer and Hangfire.SqlServer.MSMQ using nuget
3. copy and paste the code below into Program.cs
4. edit the app.Config file and make sure you have a valid connection string, such as
   <add name="hangfire" connectionString="data source=(local);initial catalog=HANG_DEV;integrated security=True;MultipleActiveResultSets=True;" />
1. note that the name hangfire must match the ConfigurationManager.ConnectionStrings key
2. if have a SQL Server named instance you might need to use .\SQLEXPRESS or similar instead of (local)
5. build the application, and copy the bin/Debug directory to the internal computer
6. edit the app.Config again and change the connection string to match the way you would connect from the external computer using SSMS. E.g. change (local) to HOSTNAME.
7. Edit the source code on the line starting with .UseMsmqQueues and change @".\Private$\hangfire-{0}" to: @"FormatName:Direct=OS:HOSTNAME\private$\hangfire-{0}".
8. build the application, and copy the bin/Debug directory to the external computer
9. on the internal computer, run the application as
   HangfireConcept.exe internal
10. on the external computer, run the application as
    HangfireConcept.exe external
11. type text in either console. After pressing enter the text should appear in the other console on the other computer

using System;
using System.Configuration;
using Hangfire;
using Hangfire.SqlServer;
using Hangfire.SqlServer.Msmq;

namespace HangFire
{
    class Program
    {
        public const string InternalQueue = "internal";
        public const string ExternalQueue = "external";
        public static bool InternalVersion = false;

        static void Main(string[] args)
        {
            GlobalConfiguration.Configuration
                .UseSqlServerStorage(ConfigurationManager.ConnectionStrings["hangfire"].ConnectionString,
                    new SqlServerStorageOptions {PrepareSchemaIfNecessary = true, SchemaName = "HangFire"})
                .UseMsmqQueues(MsmqTransactionType.Dtc, @".\Private$\hangfire-{0}", ExternalQueue, InternalQueue);
            //LogProvider.SetCurrentLogProvider(new ColouredConsoleLogProvider());

            if (args[0].ToLower() == "internal")
            {
                InternalVersion = true;
            }

            var options = new BackgroundJobServerOptions
            {
                Queues = new[] { InternalVersion ? InternalQueue : ExternalQueue }
            };

            using (var server = new BackgroundJobServer(options))
            {
                Console.WriteLine("Hangfire {0} Server started.", InternalVersion ? "internal" : "external");
                // Console.ReadKey();
                while (true)
                {
                    var c = Console.ReadLine();
                    if (InternalVersion)
                    {
                        BackgroundJob.Enqueue(() => WriteStuffExternal(c));
                    }
                    else
                    {
                        BackgroundJob.Enqueue(() => WriteStuffInternal(c));
                    }
                }
            }
        }

        [Queue(InternalQueue)]
        public static void WriteStuffInternal(string message)
        {
            Console.WriteLine("ext: {0}", message);
        }

        [Queue(ExternalQueue)]
        public static void WriteStuffExternal(string message)
        {
            Console.WriteLine("int: {0}", message);
        }
    }
}

Conclusion

This is a very convoluted method of chatting between two computers! But of course the power of Hangfire is in queing up services for running later or elsewhere, and MSMQ gives instant access to those queues.

For further reading:

• How MSMQ security blocks RPC traffic
  https://blogs.msdn.microsoft.com/johnbreakwell/2010/03/24/understanding-how-msmq-security-blocks-rpc-traffic/
• Transactional remote receive:
  https://msdn.microsoft.com/en-us/library/ms700128(v=vs.85).aspx
• Read from remote queues:
  https://msdn.microsoft.com/en-us/library/ms699854(v=vs.85).aspx
• Hangire using SQL server:
  http://docs.hangfire.io/en/latest/configuration/using-sql-server.html
• Hangfire Using SQL Server with MSMQ:
  http://docs.hangfire.io/en/latest/configuration/using-sql-server-with-msmq.html