Re: C# DQPublisher Manager


Frank Quinn <fquinn.ni@...>
 

Hi Mathias,

I was only testing with producer / consumer which is only one symbol so perhaps that's part of the issue here. I'll test with more symbols too when I get back to my machine. What sort of data rates were you running at?

Cheers,
Frank

On Mon, Jun 22, 2015 at 4:25 PM, Mathias Kim <Mathias.Kim@...> wrote:

Hi Frank,

 

thanks for the answer.

 

I tried using Proton 0.8 but that changed nothing.

Also, I checked the queue depth. Publisher and Subscriber app had a depth of 0.

In addition, I checked the broker with capturereplayc and mamalistenc. More or less the same results: with one symbol there was no visible latency. After subscribing to 5 symbols I saw the same behavior as with our C# Publisher/Subscriber. So I think the issue lies with the Windows Broker. Also, I was running it on a Server within my network and the delay got even larger.

So I think the delay is caused by the (Windows) Qpid broker.

 

Since I have no clue how to modify the Windows Qpid broker, I’m going to test it with a Fedora Virtual Machine.

 

Regards,

 

Mathias

 

 

 

From: Frank Quinn [mailto:fquinn.ni@...]
Sent: Samstag, 20. Juni 2015 13:09
To: Mathias Kim
Cc: openmama-dev@...
Subject: Re: [Openmama-dev] C# DQPublisher Manager

 

Hi Mathias,

I just tried an equivalent test with Qpid Broker 0.32 and proton 0.9.1 (with a few small changes to let it compile as proton changed their interface again) and the producer keeps crashing... so we'll get back to what's going on there later.

So I tried with Proton 0.7 on linux (Fedora 22) and I can see:

1. Logging doesn't need to be disabled - seems to work regardless.

2. Performance is pretty much the same with the broker as it is point to point

3. CPU is spread across several cores

4. Latency is around 4ms at data rates of around 10,000 msg/s

If you're getting latencies in the 2 second region, my first guess is that you're queuing in your client, have you tried printing out the queue size in your application? This is all finger in the air stuff though because this is all running on my local laptop.

When you're operating point to point, the producer is essentially throttled by the speed of the consumer. When a broker is involved, this relationship is effectively decoupled, so you may find that the producer is actually publishing faster than you expect now (enqueuing for sending is not the same as actually sending). That or else your applications haven't been compiled with multithreading support and they're all just fighting with each other for scheduler time. I'll try a windows test when I get a chance to see if I can replicate what you're seeing.

What sort of data rates and message sizes were you putting through?

 

Cheers,

Frank

 

On Tue, Jun 16, 2015 at 4:31 PM, Mathias Kim <Mathias.Kim@...> wrote:

Hi Frank,

 

Testing the broker feature took us a while since building Qpid isn’t as straightforward on Windows as it seems on Linux.

As you suggested when using the broker the slowdown after client disconnect disappears. However, the broker seems to be very slow.

 

We were testing it with the same C# classes that showed acceptable performance without the broker. Using the broker, every received message adds a delay of roughly two seconds at the client app. The publisher app is not affected and works as fast as before. This only occurs if we simulate a very busy symbol. Symbols with fewer updates are not affected.

Also, Qpid only uses one core whereas the publisher/client setup without broker uses all 4 available cores evenly.

 

How is the performance on Linux?

 

We’re using

-          Windows 8

-          Broker, publisher and client on the same machine

-          Qpid 0.32 & Proton 0.7

 

Also, we noticed a very strange thing: this whole thing only works if we enable all logging(-t). Without, no communication happens.

 

Regards

 

Mathias

 


Join Openmama-dev@lists.openmama.org to automatically receive all group messages.