High load design question

Jun 21, 2013 at 7:11 PM
Hello !

I am collecting windows performance counter values via WMI in my app.
I want to publish about 1.400 [yes, thousend] events in 2.5 seconds to up
to 3-5 subscribers [which are viewing live charts]. The incoming data will probably - over the time - build peaks with about 1.000 events per second.

One of the subscribers is a webserver [from where I'll publish the data via SignalR to the clients]. The webserver is also something like a "consolidation-point" [??!], which can server more then one clients by itself, so that the
source of the data must not serve all subscribers.

My app is collecting the wmi-data and then will publish them. So I have to build a method which does the final POST. Should I re-use the port or
is it possible to keep the instance and reuse it?? When I use post, I assume, this will use CCR threads, is this right?? Should I configure CCR to use
say 25, threads ??

I have done this already with MSMQ and this is doable. In my experiment,
the receiver app was working - and is thought of - a big cache, where
I can keep a short history live. The usual way in face of performance problems is, that ther is no fine-grained data resolution [and other aspects] with the
usual tools.

Each other thought is welcome too!

Thanks anyway and
best regards,
++mabra
Coordinator
Jun 24, 2013 at 7:15 AM
Hi mabra,

That's a high load you want to process. The problem is that the number of messages multiplies with the subscribers, which could lead to 5000 messages per second needing to be sent when there are 5 subscribers (because everything is sent by TCP, and there is no low-level broadcast). It should be possible, but that heavily depends on the speed of your network connection and the size of your messages. In general the appspace has good performance, but I cannot guarantee that it is sufficient in your specific case, so you will just need to try it out. What I would additionally think about is if this high frequency of events is really necessary - I have also been working on gathering some performance counter data and sending it over appspace a short while ago, but many values don't change that fast, so you could throttle them and greatly reduce the overall message count.

I am not sure if I understand your question concerning ports: What do you mean if you ask whether you should reuse the port or keep the instance?
Yes, the post will use a ccr thread, but an internal one: For that purpose the appspace has an internal dispatcher that is just used for sending messages. Its thread count is fixed to the CCR default, being equal to the number of cores, which should be sufficient also in a high performance scenario.

Best regards
Thomas