Dispatch Router load balancing config questions

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Dispatch Router load balancing config questions

Dan Langford
Last week I had a thread with lots of little questions around Dispatch
Routers. Ted Ross has been awesome to answer most of those. As a result I
feel like I have my QDR config shaping up a bit better. HOWEVER with some
more very focused questions I thought it would be best to start a new
thread. The problem I am seeing is that the routers are not distributing
the message load across other brokers on the network. Here is a little
diagram:


RouterA-03 ----> ArtemisBrokerA
 |
 |
 |
\/
RouterB-05 ----> ArtemisBrokerB


*NOTE: i am currently using Qpid Dispatch Router from a RHEL repo. v 0.7.0*


Connections from clients come in through an F5 VIP which forwards those
connections to either host L-03-A or L-05-B. Each of those hosts have a
Qpid Dispatch Router installed in front of an Artemis broker. dispatch
router on L-05-B is listening on an additional port that dispatch router
L-03-A connects to for inter-router communication.

If I go around my F5 VIP so i know I am connecting straight to L-05-B and I
send any number of messages into the router there all of those messages end
up in ArtemisBrokerB. I was hoping that some would go to BrokerB and some
would go via RouterA over to BrokerA. Now when BrokerB is taken down
CURRENTLY the messages sent to Router L-05-B ARE routed through Router
L-03-A and then out to ArtemisBrokerB. (Currently receiving messages pull
them in from both hosts so my questions only lies in message production at
the moment.)

Do you agree that with this configuration you would expect messages to be
load balanced between the two routes? I would like to paste in some of our
config and results from running qdstat. If you observe something that is
misconfigured and are able to highlight it i would be very appreciative.
First the config files for QDR instances. I tried my best to scrub them of
specific IPs and hostnames.

*L-03-A qrouterd.conf*


*router {*

*    mode: interior*

*    id: Router.A*

*}*

*log {*

*    module: DEFAULT*

*    enable: debug+*

*    timestamp: yes*

*}*

*sslProfile {*

*    name: my-ssl*

*    certFile: /opt/org/my-ssl-info.pem*

*    keyFile: /opt/org/my-ssl-info.pem*

*    password: hellokitty42*

*}*

*listener {*

*    role: normal*

*    host: 0.0.0.0*

*    port: 5671*

*    authenticatePeer: no*

*    saslMechanisms: ANONYMOUS*

*    sslProfile: my-ssl*

*}*

*connector {*

*    name: local-artemis*

*    role: route-container*

*    host: L-03-A*

*    port: 61616*

*    saslMechanisms: ANONYMOUS*

*}*

*connector {*

*    name: routerb*

*    role: inter-router*

*    host: L-05-B*

*    port: 6671*

*    saslMechanisms: ANONYMOUS*

*}*

*address {*

*    prefix: foo.#*

*    waypoint: yes*

*    distribution: balanced*

*}*

*autoLink {*

*    addr: foo.bar*

*    dir: in*

*    connection: local-artemis*

*}*

*autoLink {*

*    addr: foo.bar*

*    dir: out*

*    connection: local-artemis*

*}*


*L-05-B qrouterd.conf*

*router {*

*    mode: interior*

*    id: Router.B*

*}*

*log {*

*    module: DEFAULT*

*    enable: debug+*

*    timestamp: yes*

*}*

*sslProfile {*

*    name: my-ssl*

*    certFile: /opt/org/my-ssl-info.pem*

*    keyFile: /opt/org/my-ssl-info.pem*

*    password: hellokitty42*

*}*

*listener {*

*    role: normal*

*    host: 0.0.0.0*

*    port: 5671*

*    authenticatePeer: no*

*    saslMechanisms: ANONYMOUS*

*    sslProfile: my-ssl*

*}*

*listener {*

*    role: inter-router*

*    host: 0.0.0.0*

*    port: 6671*

*    authenticatePeer: no*

*    saslMechanisms: ANONYMOUS*

*    sslProfile: my-ssl*

*}*

*connector {*

*    name: local-artemis*

*    role: route-container*

*    host: L-05-B*

*    port: 61616*

*    saslMechanisms: ANONYMOUS*

*}*

*address {*

*    prefix: foo.#*

*    waypoint: yes*

*    distribution: balanced*

*}*

*autoLink {*

*    addr: foo.bar*

*    dir: in*

*    connection: local-artemis*

*}*

*autoLink {*

*    addr: foo.bar*

*    dir: out*

*    connection: local-artemis*

*}*




*L-03-A ~]$ qdstat -b:5671 -g*

*Router Statistics*

*  attr       value*

*  =====================*

*  Mode       interior*

*  Area       0*

*  Router Id  Router.A*


*L-05-B ~]$ qdstat -b:5671 -g*

*Router Statistics*

*  attr       value*

*  =====================*

*  Mode       interior*

*  Area       0*

*  Router Id  Router.B*




*L-03-A ~]$ qdstat -b:5671 -c*

*Connections*

*  Id      host             container                             role
        dir  security     authentication*

*
==================================================================================================================*

*  14      L-03-A:61616     L-03-A
route-container  out  no-security  anonymous-user*

*  107132  L-05-B:6671      Router.B
inter-router     out  no-security  anonymous-user*

*  107267  127.0.0.1:37916 <http://127.0.0.1:37916>
180851ba-7ce4-44d0-a937-97de16e781c4  normal           in   no-security
anonymous-user*


*L-05-B ~]$ qdstat -b:5671 -c*

*Connections*

*  Id   host                container                             role
        dir  security     authentication*

*
==================================================================================================================*

*  1    L-05-B:61616        L-05-B
route-container  out  no-security  anonymous-user*

*  2    10.<ip>.32:57182  Router.A
inter-router     in   no-security  anonymous-user*

*  143  127.0.0.1:48610 <http://127.0.0.1:48610>
93e6b3e3-443e-43d9-9c2c-0d648266ec19  normal           in   no-security
anonymous-user*




*L-03-A ~]$ qdstat -b:5671 -n*

*Routers in the Network*

*  router-id  next-hop  link*

*  ===========================*

*  Router.A   (self)    -*

*  Router.B   -         0*


*L-05-B ~]$ qdstat -b:5671 -n*

*Routers in the Network*

*  router-id  next-hop  link*

*  ===========================*

*  Router.A   -         0*

*  Router.B   (self)    -*




*L-03-A ~]$ qdstat -b:5671 --linkroutes*

*Link Routes*


*L-05-B ~]$ qdstat -b:5671 --linkroutes*

*Link Routes*




*L-03-A ~]$ qdstat -b:5671 --autolinks*

*AutoLinks*

*  addr     dir  phs  link  status  lastErr*

*  ==========================================*

*  foo.bar  in   1    8     active*

*  foo.bar  out  0    9     active*


*L-05-B ~]$ qdstat -b:5671 --autolinks*

*AutoLinks*

*  addr     dir  phs  link  status  lastErr*

*  ==========================================*

*  foo.bar  in   1    4     active*

*  foo.bar  out  0    5     active*




*L-03-A ~]$ qdstat -b:5671 -l*

*Router Links*

*  type            dir  conn id  id  peer  class   addr
phs  cap  undel  unsett  del  presett  acc  rej  rel  mod  admin    oper*

*
================================================================================================================================================*

*  endpoint        in   14       8         mobile  foo.bar               1
  250  0      0       80   0        80   0    0    0    enabled  up*

*  endpoint        out  14       9         mobile  foo.bar               0
  250  0      0       40   0        40   0    0    0    enabled  up*

*  router-control  in   107132   15
  250  0      0       524  524      0    0    0    0    enabled  up*

*  router-control  out  107132   16        local   qdhello
  250  0      0       527  527      0    0    0    0    enabled  up*

*  inter-router    in   107132   17
  250  0      0       0    0        0    0    0    0    enabled  up*

*  inter-router    out  107132   18
  250  0      0       0    0        0    0    0    0    enabled  up*

*  endpoint        in   107336   33        mobile  $management           0
  250  0      0       1    0        1    0    0    0    enabled  up*

*  endpoint        out  107336   34        local   temp.liHSN39azHuUPkg
  250  0      0       0    0        0    0    0    0    enabled  up*


*L-05-B ~]$ qdstat -b:5671 -l*

*Router Links*

*  type            dir  conn id  id  peer  class   addr
phs  cap  undel  unsett  del  presett  acc  rej  rel  mod  admin    oper*

*
================================================================================================================================================*

*  endpoint        in   1        4         mobile  foo.bar               1
  250  40     0       0    0        0    0    0    0    enabled  up*

*  endpoint        out  1        5         mobile  foo.bar               0
  250  0      0       0    0        0    0    0    0    enabled  up*

*  router-control  out  2        6         local   qdhello
  250  0      0       524  524      0    0    0    0    enabled  up*

*  router-control  in   2        7
  250  0      0       527  527      0    0    0    0    enabled  up*

*  inter-router    out  2        8
  250  0      0       0    0        0    0    0    0    enabled  up*

*  inter-router    in   2        9
  250  0      0       0    0        0    0    0    0    enabled  up*

*  endpoint        in   212      24        mobile  $management           0
  250  0      0       1    0        1    0    0    0    enabled  up*

*  endpoint        out  212      25        local   temp.BJPESwMGFo+7qiy
  250  0      0       0    0        0    0    0    0    enabled  up*





*L-03-A ~]$ qdstat -b:5671 -a*

*Router Addresses*

*  class   addr                   phs  distrib    in-proc  local  remote
cntnr  in  out  thru   to-proc  from-proc*

*
==================================================================================================================*

*  local   $_management_internal       closest    1        0      0
0      0   0    0      6        6*

*  local   $displayname                closest    1        0      0
0      0   0    0      0        0*

*  mobile  $management            0    closest    1        0      0
0      9   0    0      9        0*

*  local   $management                 closest    1        0      0
0      0   0    0      0        0*

*  router  Router.B                    closest    0        0      1
0      0   0    19     0        19*

*  mobile  foo.bar                1    balanced   0        0      0
0      80  520  0      0        0*

*  mobile  foo.bar                0    balanced   0        1      1
0      0   40   0      0        0*

*  local   qdhello                     flood      1        1      0
0      0   0    0      269,611  269,622*

*  local   qdrouter                    flood      1        0      0
0      0   0    0      12       0*

*  topo    qdrouter                    flood      1        0      1
0      0   0    8,992  8,991    8,996*

*  local   qdrouter.ma <http://qdrouter.ma>                 multicast  1
    0      0       0      0   0    0      4        0*

*  topo    qdrouter.ma <http://qdrouter.ma>                 multicast  1
    0      1       0      0   0    3      0        3*

*  local   temp.u6NMF1AFV0JpdBi        closest    0        1      0
0      0   0    0      0        0*


*L-05-B ~]$ qdstat -b:5671 -a*

*Router Addresses*

*  class   addr                   phs  distrib    in-proc  local  remote
cntnr  in  out  thru  to-proc  from-proc*

*
=================================================================================================================*

*  local   $_management_internal       closest    1        0      0
0      0   0    0     6        6*

*  local   $displayname                closest    1        0      0
0      0   0    0     0        0*

*  mobile  $management            0    closest    1        0      0
0      9   0    0     9        0*

*  local   $management                 closest    1        0      0
0      0   0    0     0        0*

*  router  Router.A                    closest    0        0      1
0      0   0    6     0        6*

*  mobile  foo.bar                1    balanced   0        0      0
0      0   0    0     0        0*

*  mobile  foo.bar                0    balanced   0        1      1
0      0   0    0     0        0*

*  local   qdhello                     flood      1        1      0
0      0   0    0     570      571*

*  local   qdrouter                    flood      1        0      0
0      0   0    0     4        0*

*  topo    qdrouter                    flood      1        0      1
0      0   0    20    21       21*

*  local   qdrouter.ma <http://qdrouter.ma>                 multicast  1
    0      0       0      0   0    0     4        0*

*  topo    qdrouter.ma <http://qdrouter.ma>                 multicast  1
    0      1       0      0   0    0     0        1*

*  local   temp.xwHdS0zzku0s3lR        closest    0        1      0
0      0   0    0     0        0*



is there something off in my configuration? or in my understand of QDR
netwokin just not up to snuff? should i care a little bit less about load
balancing because the VIP will round robin new connections out to different
codes?

Thank you so much for your time and insight
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dispatch Router load balancing config questions

Ted Ross
Dan,

There's one issue with your configuration which doesn't affect the load
balancing but will cause problems with receiving messages from the
brokers.  In the address.prefix, you use "foo.#".  This is a pure prefix
and it should simply be "foo".  The wildcards are coming in the next
release but are not implemented in the code you are using.

Regarding your actual question.  I assume that you are testing this
configuration under light load (i.e. sending one message at a time).

The way that the balancing works is that it will route to the consumer
(broker) with the fewest outstanding deliveries + inter-router cost.  This
means that it will favor the local broker over the remote one if there are
no in-flight deliveries.  The default (and minimum) cost for an
inter-router connection is 1.  You can set it to a higher value in the
listener or connector.

If you are sending one-at-a-time synchronous sends, they will always go to
the local broker because the broker's zero outstanding deliveries will
always be less than the inter-router cost of 1.  If you send multiple
deliveries asynchronously, you will see them being distributed to both
brokers in the network.  You can make the local-affinity stronger by
increasing the inter-router cost.

-Ted

On Tue, Aug 1, 2017 at 3:47 AM, Dan Langford <[hidden email]> wrote:

> Last week I had a thread with lots of little questions around Dispatch
> Routers. Ted Ross has been awesome to answer most of those. As a result I
> feel like I have my QDR config shaping up a bit better. HOWEVER with some
> more very focused questions I thought it would be best to start a new
> thread. The problem I am seeing is that the routers are not distributing
> the message load across other brokers on the network. Here is a little
> diagram:
>
>
> RouterA-03 ----> ArtemisBrokerA
>  |
>  |
>  |
> \/
> RouterB-05 ----> ArtemisBrokerB
>
>
> *NOTE: i am currently using Qpid Dispatch Router from a RHEL repo. v 0.7.0*
>
>
> Connections from clients come in through an F5 VIP which forwards those
> connections to either host L-03-A or L-05-B. Each of those hosts have a
> Qpid Dispatch Router installed in front of an Artemis broker. dispatch
> router on L-05-B is listening on an additional port that dispatch router
> L-03-A connects to for inter-router communication.
>
> If I go around my F5 VIP so i know I am connecting straight to L-05-B and I
> send any number of messages into the router there all of those messages end
> up in ArtemisBrokerB. I was hoping that some would go to BrokerB and some
> would go via RouterA over to BrokerA. Now when BrokerB is taken down
> CURRENTLY the messages sent to Router L-05-B ARE routed through Router
> L-03-A and then out to ArtemisBrokerB. (Currently receiving messages pull
> them in from both hosts so my questions only lies in message production at
> the moment.)
>
> Do you agree that with this configuration you would expect messages to be
> load balanced between the two routes? I would like to paste in some of our
> config and results from running qdstat. If you observe something that is
> misconfigured and are able to highlight it i would be very appreciative.
> First the config files for QDR instances. I tried my best to scrub them of
> specific IPs and hostnames.
>
> *L-03-A qrouterd.conf*
>
>
> *router {*
>
> *    mode: interior*
>
> *    id: Router.A*
>
> *}*
>
> *log {*
>
> *    module: DEFAULT*
>
> *    enable: debug+*
>
> *    timestamp: yes*
>
> *}*
>
> *sslProfile {*
>
> *    name: my-ssl*
>
> *    certFile: /opt/org/my-ssl-info.pem*
>
> *    keyFile: /opt/org/my-ssl-info.pem*
>
> *    password: hellokitty42*
>
> *}*
>
> *listener {*
>
> *    role: normal*
>
> *    host: 0.0.0.0*
>
> *    port: 5671*
>
> *    authenticatePeer: no*
>
> *    saslMechanisms: ANONYMOUS*
>
> *    sslProfile: my-ssl*
>
> *}*
>
> *connector {*
>
> *    name: local-artemis*
>
> *    role: route-container*
>
> *    host: L-03-A*
>
> *    port: 61616*
>
> *    saslMechanisms: ANONYMOUS*
>
> *}*
>
> *connector {*
>
> *    name: routerb*
>
> *    role: inter-router*
>
> *    host: L-05-B*
>
> *    port: 6671*
>
> *    saslMechanisms: ANONYMOUS*
>
> *}*
>
> *address {*
>
> *    prefix: foo.#*
>
> *    waypoint: yes*
>
> *    distribution: balanced*
>
> *}*
>
> *autoLink {*
>
> *    addr: foo.bar*
>
> *    dir: in*
>
> *    connection: local-artemis*
>
> *}*
>
> *autoLink {*
>
> *    addr: foo.bar*
>
> *    dir: out*
>
> *    connection: local-artemis*
>
> *}*
>
>
> *L-05-B qrouterd.conf*
>
> *router {*
>
> *    mode: interior*
>
> *    id: Router.B*
>
> *}*
>
> *log {*
>
> *    module: DEFAULT*
>
> *    enable: debug+*
>
> *    timestamp: yes*
>
> *}*
>
> *sslProfile {*
>
> *    name: my-ssl*
>
> *    certFile: /opt/org/my-ssl-info.pem*
>
> *    keyFile: /opt/org/my-ssl-info.pem*
>
> *    password: hellokitty42*
>
> *}*
>
> *listener {*
>
> *    role: normal*
>
> *    host: 0.0.0.0*
>
> *    port: 5671*
>
> *    authenticatePeer: no*
>
> *    saslMechanisms: ANONYMOUS*
>
> *    sslProfile: my-ssl*
>
> *}*
>
> *listener {*
>
> *    role: inter-router*
>
> *    host: 0.0.0.0*
>
> *    port: 6671*
>
> *    authenticatePeer: no*
>
> *    saslMechanisms: ANONYMOUS*
>
> *    sslProfile: my-ssl*
>
> *}*
>
> *connector {*
>
> *    name: local-artemis*
>
> *    role: route-container*
>
> *    host: L-05-B*
>
> *    port: 61616*
>
> *    saslMechanisms: ANONYMOUS*
>
> *}*
>
> *address {*
>
> *    prefix: foo.#*
>
> *    waypoint: yes*
>
> *    distribution: balanced*
>
> *}*
>
> *autoLink {*
>
> *    addr: foo.bar*
>
> *    dir: in*
>
> *    connection: local-artemis*
>
> *}*
>
> *autoLink {*
>
> *    addr: foo.bar*
>
> *    dir: out*
>
> *    connection: local-artemis*
>
> *}*
>
>
>
>
> *L-03-A ~]$ qdstat -b:5671 -g*
>
> *Router Statistics*
>
> *  attr       value*
>
> *  =====================*
>
> *  Mode       interior*
>
> *  Area       0*
>
> *  Router Id  Router.A*
>
>
> *L-05-B ~]$ qdstat -b:5671 -g*
>
> *Router Statistics*
>
> *  attr       value*
>
> *  =====================*
>
> *  Mode       interior*
>
> *  Area       0*
>
> *  Router Id  Router.B*
>
>
>
>
> *L-03-A ~]$ qdstat -b:5671 -c*
>
> *Connections*
>
> *  Id      host             container                             role
>         dir  security     authentication*
>
> *
> ============================================================
> ======================================================*
>
> *  14      L-03-A:61616     L-03-A
> route-container  out  no-security  anonymous-user*
>
> *  107132  L-05-B:6671      Router.B
> inter-router     out  no-security  anonymous-user*
>
> *  107267  127.0.0.1:37916 <http://127.0.0.1:37916>
> 180851ba-7ce4-44d0-a937-97de16e781c4  normal           in   no-security
> anonymous-user*
>
>
> *L-05-B ~]$ qdstat -b:5671 -c*
>
> *Connections*
>
> *  Id   host                container                             role
>         dir  security     authentication*
>
> *
> ============================================================
> ======================================================*
>
> *  1    L-05-B:61616        L-05-B
> route-container  out  no-security  anonymous-user*
>
> *  2    10.<ip>.32:57182  Router.A
> inter-router     in   no-security  anonymous-user*
>
> *  143  127.0.0.1:48610 <http://127.0.0.1:48610>
> 93e6b3e3-443e-43d9-9c2c-0d648266ec19  normal           in   no-security
> anonymous-user*
>
>
>
>
> *L-03-A ~]$ qdstat -b:5671 -n*
>
> *Routers in the Network*
>
> *  router-id  next-hop  link*
>
> *  ===========================*
>
> *  Router.A   (self)    -*
>
> *  Router.B   -         0*
>
>
> *L-05-B ~]$ qdstat -b:5671 -n*
>
> *Routers in the Network*
>
> *  router-id  next-hop  link*
>
> *  ===========================*
>
> *  Router.A   -         0*
>
> *  Router.B   (self)    -*
>
>
>
>
> *L-03-A ~]$ qdstat -b:5671 --linkroutes*
>
> *Link Routes*
>
>
> *L-05-B ~]$ qdstat -b:5671 --linkroutes*
>
> *Link Routes*
>
>
>
>
> *L-03-A ~]$ qdstat -b:5671 --autolinks*
>
> *AutoLinks*
>
> *  addr     dir  phs  link  status  lastErr*
>
> *  ==========================================*
>
> *  foo.bar  in   1    8     active*
>
> *  foo.bar  out  0    9     active*
>
>
> *L-05-B ~]$ qdstat -b:5671 --autolinks*
>
> *AutoLinks*
>
> *  addr     dir  phs  link  status  lastErr*
>
> *  ==========================================*
>
> *  foo.bar  in   1    4     active*
>
> *  foo.bar  out  0    5     active*
>
>
>
>
> *L-03-A ~]$ qdstat -b:5671 -l*
>
> *Router Links*
>
> *  type            dir  conn id  id  peer  class   addr
> phs  cap  undel  unsett  del  presett  acc  rej  rel  mod  admin    oper*
>
> *
> ============================================================
> ============================================================
> ========================*
>
> *  endpoint        in   14       8         mobile  foo.bar               1
>   250  0      0       80   0        80   0    0    0    enabled  up*
>
> *  endpoint        out  14       9         mobile  foo.bar               0
>   250  0      0       40   0        40   0    0    0    enabled  up*
>
> *  router-control  in   107132   15
>   250  0      0       524  524      0    0    0    0    enabled  up*
>
> *  router-control  out  107132   16        local   qdhello
>   250  0      0       527  527      0    0    0    0    enabled  up*
>
> *  inter-router    in   107132   17
>   250  0      0       0    0        0    0    0    0    enabled  up*
>
> *  inter-router    out  107132   18
>   250  0      0       0    0        0    0    0    0    enabled  up*
>
> *  endpoint        in   107336   33        mobile  $management           0
>   250  0      0       1    0        1    0    0    0    enabled  up*
>
> *  endpoint        out  107336   34        local   temp.liHSN39azHuUPkg
>   250  0      0       0    0        0    0    0    0    enabled  up*
>
>
> *L-05-B ~]$ qdstat -b:5671 -l*
>
> *Router Links*
>
> *  type            dir  conn id  id  peer  class   addr
> phs  cap  undel  unsett  del  presett  acc  rej  rel  mod  admin    oper*
>
> *
> ============================================================
> ============================================================
> ========================*
>
> *  endpoint        in   1        4         mobile  foo.bar               1
>   250  40     0       0    0        0    0    0    0    enabled  up*
>
> *  endpoint        out  1        5         mobile  foo.bar               0
>   250  0      0       0    0        0    0    0    0    enabled  up*
>
> *  router-control  out  2        6         local   qdhello
>   250  0      0       524  524      0    0    0    0    enabled  up*
>
> *  router-control  in   2        7
>   250  0      0       527  527      0    0    0    0    enabled  up*
>
> *  inter-router    out  2        8
>   250  0      0       0    0        0    0    0    0    enabled  up*
>
> *  inter-router    in   2        9
>   250  0      0       0    0        0    0    0    0    enabled  up*
>
> *  endpoint        in   212      24        mobile  $management           0
>   250  0      0       1    0        1    0    0    0    enabled  up*
>
> *  endpoint        out  212      25        local   temp.BJPESwMGFo+7qiy
>   250  0      0       0    0        0    0    0    0    enabled  up*
>
>
>
>
>
> *L-03-A ~]$ qdstat -b:5671 -a*
>
> *Router Addresses*
>
> *  class   addr                   phs  distrib    in-proc  local  remote
> cntnr  in  out  thru   to-proc  from-proc*
>
> *
> ============================================================
> ======================================================*
>
> *  local   $_management_internal       closest    1        0      0
> 0      0   0    0      6        6*
>
> *  local   $displayname                closest    1        0      0
> 0      0   0    0      0        0*
>
> *  mobile  $management            0    closest    1        0      0
> 0      9   0    0      9        0*
>
> *  local   $management                 closest    1        0      0
> 0      0   0    0      0        0*
>
> *  router  Router.B                    closest    0        0      1
> 0      0   0    19     0        19*
>
> *  mobile  foo.bar                1    balanced   0        0      0
> 0      80  520  0      0        0*
>
> *  mobile  foo.bar                0    balanced   0        1      1
> 0      0   40   0      0        0*
>
> *  local   qdhello                     flood      1        1      0
> 0      0   0    0      269,611  269,622*
>
> *  local   qdrouter                    flood      1        0      0
> 0      0   0    0      12       0*
>
> *  topo    qdrouter                    flood      1        0      1
> 0      0   0    8,992  8,991    8,996*
>
> *  local   qdrouter.ma <http://qdrouter.ma>                 multicast  1
>     0      0       0      0   0    0      4        0*
>
> *  topo    qdrouter.ma <http://qdrouter.ma>                 multicast  1
>     0      1       0      0   0    3      0        3*
>
> *  local   temp.u6NMF1AFV0JpdBi        closest    0        1      0
> 0      0   0    0      0        0*
>
>
> *L-05-B ~]$ qdstat -b:5671 -a*
>
> *Router Addresses*
>
> *  class   addr                   phs  distrib    in-proc  local  remote
> cntnr  in  out  thru  to-proc  from-proc*
>
> *
> ============================================================
> =====================================================*
>
> *  local   $_management_internal       closest    1        0      0
> 0      0   0    0     6        6*
>
> *  local   $displayname                closest    1        0      0
> 0      0   0    0     0        0*
>
> *  mobile  $management            0    closest    1        0      0
> 0      9   0    0     9        0*
>
> *  local   $management                 closest    1        0      0
> 0      0   0    0     0        0*
>
> *  router  Router.A                    closest    0        0      1
> 0      0   0    6     0        6*
>
> *  mobile  foo.bar                1    balanced   0        0      0
> 0      0   0    0     0        0*
>
> *  mobile  foo.bar                0    balanced   0        1      1
> 0      0   0    0     0        0*
>
> *  local   qdhello                     flood      1        1      0
> 0      0   0    0     570      571*
>
> *  local   qdrouter                    flood      1        0      0
> 0      0   0    0     4        0*
>
> *  topo    qdrouter                    flood      1        0      1
> 0      0   0    20    21       21*
>
> *  local   qdrouter.ma <http://qdrouter.ma>                 multicast  1
>     0      0       0      0   0    0     4        0*
>
> *  topo    qdrouter.ma <http://qdrouter.ma>                 multicast  1
>     0      1       0      0   0    0     0        1*
>
> *  local   temp.xwHdS0zzku0s3lR        closest    0        1      0
> 0      0   0    0     0        0*
>
>
>
> is there something off in my configuration? or in my understand of QDR
> netwokin just not up to snuff? should i care a little bit less about load
> balancing because the VIP will round robin new connections out to different
> codes?
>
> Thank you so much for your time and insight
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dispatch Router load balancing config questions

Dan Langford
( github gist with all the config and data from the Original Post
https://gist.github.com/danlangford/4944dcc6c0d2703ffb8555603ed27340 )

YES i was under pretty light load. a couple hundred or thousand messages at
a time were all getting funneled into the local broker. You are right once
i got 7 or 8 simultaneous connections all pushing in a couple million
message i started to see the load overflow to the other router and broker.
about 10% of messages in my setup were overflowing. thank you for your
patience. I think this config is going to work great. i do think i just
need a small adjustment to how i think about load balancing. when the cost
was the same to each broker then i would get approx 50/50 split (that was
when each router connected to each broker). with this setup where the local
cost is less its more like "balancing out once under load" and not the
traditional "balance evenly to avoid load". but now that i understand this
its fine and i know what behavior to expect.

so I do have some producers and consumers that need to use session
transactions. we have seen those work fine over amqp1.0 when connected to a
QPID Broker or Artemis broker. but with the config you see here connecting
to a qpid dispatch router (directly or through our VIP) I cannot create a
session with "transacted=true". I get a NullPointerException

javax.jms.JMSException: java.lang.NullPointerException
at
org.apache.qpid.jms.exceptions.JmsExceptionSupport.create(JmsExceptionSupport.java:86)
at
org.apache.qpid.jms.exceptions.JmsExceptionSupport.create(JmsExceptionSupport.java:108)
at org.apache.qpid.jms.JmsConnection.createResource(JmsConnection.java:609)
at
org.apache.qpid.jms.JmsLocalTransactionContext.begin(JmsLocalTransactionContext.java:125)
at org.apache.qpid.jms.JmsSession.<init>(JmsSession.java:143)
at org.apache.qpid.jms.JmsConnection.createSession(JmsConnection.java:299)
at org.myorg.mymessaging.PostOffice.buildSession(PostOffice.java:149)
...
Caused by: java.io.IOException: java.lang.NullPointerException
at
org.apache.qpid.jms.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
at
org.apache.qpid.jms.provider.amqp.AmqpTransactionCoordinator.processDeliveryUpdates(AmqpTransactionCoordinator.java:117)
at
org.apache.qpid.jms.provider.amqp.AmqpProvider.processUpdates(AmqpProvider.java:928)
at
org.apache.qpid.jms.provider.amqp.AmqpProvider.access$1800(AmqpProvider.java:93)
at
org.apache.qpid.jms.provider.amqp.AmqpProvider$18.run(AmqpProvider.java:790)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at
org.apache.qpid.jms.provider.amqp.AmqpTransactionCoordinator.processDeliveryUpdates(AmqpTransactionCoordinator.java:93)
... 10 more

are transactions expected to work with qpid dispatch router? on that note
do you think i could also get XA transactions working? would i need to use
a link route? i dont know for sure but it sounds like a link route is a
little more low level that an autolink.

thank you

On Tue, Aug 1, 2017 at 6:47 AM Ted Ross <[hidden email]> wrote:

> Dan,
>
> There's one issue with your configuration which doesn't affect the load
> balancing but will cause problems with receiving messages from the
> brokers.  In the address.prefix, you use "foo.#".  This is a pure prefix
> and it should simply be "foo".  The wildcards are coming in the next
> release but are not implemented in the code you are using.
>
> Regarding your actual question.  I assume that you are testing this
> configuration under light load (i.e. sending one message at a time).
>
> The way that the balancing works is that it will route to the consumer
> (broker) with the fewest outstanding deliveries + inter-router cost.  This
> means that it will favor the local broker over the remote one if there are
> no in-flight deliveries.  The default (and minimum) cost for an
> inter-router connection is 1.  You can set it to a higher value in the
> listener or connector.
>
> If you are sending one-at-a-time synchronous sends, they will always go to
> the local broker because the broker's zero outstanding deliveries will
> always be less than the inter-router cost of 1.  If you send multiple
> deliveries asynchronously, you will see them being distributed to both
> brokers in the network.  You can make the local-affinity stronger by
> increasing the inter-router cost.
>
> -Ted
>
> On Tue, Aug 1, 2017 at 3:47 AM, Dan Langford <[hidden email]>
> wrote:
>
> > Last week I had a thread with lots of little questions around Dispatch
> > Routers. Ted Ross has been awesome to answer most of those. As a result I
> > feel like I have my QDR config shaping up a bit better. HOWEVER with some
> > more very focused questions I thought it would be best to start a new
> > thread. The problem I am seeing is that the routers are not distributing
> > the message load across other brokers on the network. Here is a little
> > diagram:
> >
> >
> > RouterA-03 ----> ArtemisBrokerA
> >  |
> >  |
> >  |
> > \/
> > RouterB-05 ----> ArtemisBrokerB
> >
> >
> > *NOTE: i am currently using Qpid Dispatch Router from a RHEL repo. v
> 0.7.0*
> >
> >
> > Connections from clients come in through an F5 VIP which forwards those
> > connections to either host L-03-A or L-05-B. Each of those hosts have a
> > Qpid Dispatch Router installed in front of an Artemis broker. dispatch
> > router on L-05-B is listening on an additional port that dispatch router
> > L-03-A connects to for inter-router communication.
> >
> > If I go around my F5 VIP so i know I am connecting straight to L-05-B
> and I
> > send any number of messages into the router there all of those messages
> end
> > up in ArtemisBrokerB. I was hoping that some would go to BrokerB and some
> > would go via RouterA over to BrokerA. Now when BrokerB is taken down
> > CURRENTLY the messages sent to Router L-05-B ARE routed through Router
> > L-03-A and then out to ArtemisBrokerB. (Currently receiving messages pull
> > them in from both hosts so my questions only lies in message production
> at
> > the moment.)
> >
> > Do you agree that with this configuration you would expect messages to be
> > load balanced between the two routes? I would like to paste in some of
> our
> > config and results from running qdstat. If you observe something that is
> > misconfigured and are able to highlight it i would be very appreciative.
> > First the config files for QDR instances. I tried my best to scrub them
> of
> > specific IPs and hostnames.
> >
> > *L-03-A qrouterd.conf*
> >
> >
> > *router {*
> >
> > *    mode: interior*
> >
> > *    id: Router.A*
> >
> > *}*
> >
> > *log {*
> >
> > *    module: DEFAULT*
> >
> > *    enable: debug+*
> >
> > *    timestamp: yes*
> >
> > *}*
> >
> > *sslProfile {*
> >
> > *    name: my-ssl*
> >
> > *    certFile: /opt/org/my-ssl-info.pem*
> >
> > *    keyFile: /opt/org/my-ssl-info.pem*
> >
> > *    password: hellokitty42*
> >
> > *}*
> >
> > *listener {*
> >
> > *    role: normal*
> >
> > *    host: 0.0.0.0*
> >
> > *    port: 5671*
> >
> > *    authenticatePeer: no*
> >
> > *    saslMechanisms: ANONYMOUS*
> >
> > *    sslProfile: my-ssl*
> >
> > *}*
> >
> > *connector {*
> >
> > *    name: local-artemis*
> >
> > *    role: route-container*
> >
> > *    host: L-03-A*
> >
> > *    port: 61616*
> >
> > *    saslMechanisms: ANONYMOUS*
> >
> > *}*
> >
> > *connector {*
> >
> > *    name: routerb*
> >
> > *    role: inter-router*
> >
> > *    host: L-05-B*
> >
> > *    port: 6671*
> >
> > *    saslMechanisms: ANONYMOUS*
> >
> > *}*
> >
> > *address {*
> >
> > *    prefix: foo.#*
> >
> > *    waypoint: yes*
> >
> > *    distribution: balanced*
> >
> > *}*
> >
> > *autoLink {*
> >
> > *    addr: foo.bar*
> >
> > *    dir: in*
> >
> > *    connection: local-artemis*
> >
> > *}*
> >
> > *autoLink {*
> >
> > *    addr: foo.bar*
> >
> > *    dir: out*
> >
> > *    connection: local-artemis*
> >
> > *}*
> >
> >
> > *L-05-B qrouterd.conf*
> >
> > *router {*
> >
> > *    mode: interior*
> >
> > *    id: Router.B*
> >
> > *}*
> >
> > *log {*
> >
> > *    module: DEFAULT*
> >
> > *    enable: debug+*
> >
> > *    timestamp: yes*
> >
> > *}*
> >
> > *sslProfile {*
> >
> > *    name: my-ssl*
> >
> > *    certFile: /opt/org/my-ssl-info.pem*
> >
> > *    keyFile: /opt/org/my-ssl-info.pem*
> >
> > *    password: hellokitty42*
> >
> > *}*
> >
> > *listener {*
> >
> > *    role: normal*
> >
> > *    host: 0.0.0.0*
> >
> > *    port: 5671*
> >
> > *    authenticatePeer: no*
> >
> > *    saslMechanisms: ANONYMOUS*
> >
> > *    sslProfile: my-ssl*
> >
> > *}*
> >
> > *listener {*
> >
> > *    role: inter-router*
> >
> > *    host: 0.0.0.0*
> >
> > *    port: 6671*
> >
> > *    authenticatePeer: no*
> >
> > *    saslMechanisms: ANONYMOUS*
> >
> > *    sslProfile: my-ssl*
> >
> > *}*
> >
> > *connector {*
> >
> > *    name: local-artemis*
> >
> > *    role: route-container*
> >
> > *    host: L-05-B*
> >
> > *    port: 61616*
> >
> > *    saslMechanisms: ANONYMOUS*
> >
> > *}*
> >
> > *address {*
> >
> > *    prefix: foo.#*
> >
> > *    waypoint: yes*
> >
> > *    distribution: balanced*
> >
> > *}*
> >
> > *autoLink {*
> >
> > *    addr: foo.bar*
> >
> > *    dir: in*
> >
> > *    connection: local-artemis*
> >
> > *}*
> >
> > *autoLink {*
> >
> > *    addr: foo.bar*
> >
> > *    dir: out*
> >
> > *    connection: local-artemis*
> >
> > *}*
> >
> >
> >
> >
> > *L-03-A ~]$ qdstat -b:5671 -g*
> >
> > *Router Statistics*
> >
> > *  attr       value*
> >
> > *  =====================*
> >
> > *  Mode       interior*
> >
> > *  Area       0*
> >
> > *  Router Id  Router.A*
> >
> >
> > *L-05-B ~]$ qdstat -b:5671 -g*
> >
> > *Router Statistics*
> >
> > *  attr       value*
> >
> > *  =====================*
> >
> > *  Mode       interior*
> >
> > *  Area       0*
> >
> > *  Router Id  Router.B*
> >
> >
> >
> >
> > *L-03-A ~]$ qdstat -b:5671 -c*
> >
> > *Connections*
> >
> > *  Id      host             container                             role
> >         dir  security     authentication*
> >
> > *
> > ============================================================
> > ======================================================*
> >
> > *  14      L-03-A:61616     L-03-A
> > route-container  out  no-security  anonymous-user*
> >
> > *  107132  L-05-B:6671      Router.B
> > inter-router     out  no-security  anonymous-user*
> >
> > *  107267  127.0.0.1:37916 <http://127.0.0.1:37916>
> > 180851ba-7ce4-44d0-a937-97de16e781c4  normal           in   no-security
> > anonymous-user*
> >
> >
> > *L-05-B ~]$ qdstat -b:5671 -c*
> >
> > *Connections*
> >
> > *  Id   host                container                             role
> >         dir  security     authentication*
> >
> > *
> > ============================================================
> > ======================================================*
> >
> > *  1    L-05-B:61616        L-05-B
> > route-container  out  no-security  anonymous-user*
> >
> > *  2    10.<ip>.32:57182  Router.A
> > inter-router     in   no-security  anonymous-user*
> >
> > *  143  127.0.0.1:48610 <http://127.0.0.1:48610>
> > 93e6b3e3-443e-43d9-9c2c-0d648266ec19  normal           in   no-security
> > anonymous-user*
> >
> >
> >
> >
> > *L-03-A ~]$ qdstat -b:5671 -n*
> >
> > *Routers in the Network*
> >
> > *  router-id  next-hop  link*
> >
> > *  ===========================*
> >
> > *  Router.A   (self)    -*
> >
> > *  Router.B   -         0*
> >
> >
> > *L-05-B ~]$ qdstat -b:5671 -n*
> >
> > *Routers in the Network*
> >
> > *  router-id  next-hop  link*
> >
> > *  ===========================*
> >
> > *  Router.A   -         0*
> >
> > *  Router.B   (self)    -*
> >
> >
> >
> >
> > *L-03-A ~]$ qdstat -b:5671 --linkroutes*
> >
> > *Link Routes*
> >
> >
> > *L-05-B ~]$ qdstat -b:5671 --linkroutes*
> >
> > *Link Routes*
> >
> >
> >
> >
> > *L-03-A ~]$ qdstat -b:5671 --autolinks*
> >
> > *AutoLinks*
> >
> > *  addr     dir  phs  link  status  lastErr*
> >
> > *  ==========================================*
> >
> > *  foo.bar  in   1    8     active*
> >
> > *  foo.bar  out  0    9     active*
> >
> >
> > *L-05-B ~]$ qdstat -b:5671 --autolinks*
> >
> > *AutoLinks*
> >
> > *  addr     dir  phs  link  status  lastErr*
> >
> > *  ==========================================*
> >
> > *  foo.bar  in   1    4     active*
> >
> > *  foo.bar  out  0    5     active*
> >
> >
> >
> >
> > *L-03-A ~]$ qdstat -b:5671 -l*
> >
> > *Router Links*
> >
> > *  type            dir  conn id  id  peer  class   addr
> > phs  cap  undel  unsett  del  presett  acc  rej  rel  mod  admin    oper*
> >
> > *
> > ============================================================
> > ============================================================
> > ========================*
> >
> > *  endpoint        in   14       8         mobile  foo.bar
>  1
> >   250  0      0       80   0        80   0    0    0    enabled  up*
> >
> > *  endpoint        out  14       9         mobile  foo.bar
>  0
> >   250  0      0       40   0        40   0    0    0    enabled  up*
> >
> > *  router-control  in   107132   15
> >   250  0      0       524  524      0    0    0    0    enabled  up*
> >
> > *  router-control  out  107132   16        local   qdhello
> >   250  0      0       527  527      0    0    0    0    enabled  up*
> >
> > *  inter-router    in   107132   17
> >   250  0      0       0    0        0    0    0    0    enabled  up*
> >
> > *  inter-router    out  107132   18
> >   250  0      0       0    0        0    0    0    0    enabled  up*
> >
> > *  endpoint        in   107336   33        mobile  $management
>  0
> >   250  0      0       1    0        1    0    0    0    enabled  up*
> >
> > *  endpoint        out  107336   34        local   temp.liHSN39azHuUPkg
> >   250  0      0       0    0        0    0    0    0    enabled  up*
> >
> >
> > *L-05-B ~]$ qdstat -b:5671 -l*
> >
> > *Router Links*
> >
> > *  type            dir  conn id  id  peer  class   addr
> > phs  cap  undel  unsett  del  presett  acc  rej  rel  mod  admin    oper*
> >
> > *
> > ============================================================
> > ============================================================
> > ========================*
> >
> > *  endpoint        in   1        4         mobile  foo.bar
>  1
> >   250  40     0       0    0        0    0    0    0    enabled  up*
> >
> > *  endpoint        out  1        5         mobile  foo.bar
>  0
> >   250  0      0       0    0        0    0    0    0    enabled  up*
> >
> > *  router-control  out  2        6         local   qdhello
> >   250  0      0       524  524      0    0    0    0    enabled  up*
> >
> > *  router-control  in   2        7
> >   250  0      0       527  527      0    0    0    0    enabled  up*
> >
> > *  inter-router    out  2        8
> >   250  0      0       0    0        0    0    0    0    enabled  up*
> >
> > *  inter-router    in   2        9
> >   250  0      0       0    0        0    0    0    0    enabled  up*
> >
> > *  endpoint        in   212      24        mobile  $management
>  0
> >   250  0      0       1    0        1    0    0    0    enabled  up*
> >
> > *  endpoint        out  212      25        local   temp.BJPESwMGFo+7qiy
> >   250  0      0       0    0        0    0    0    0    enabled  up*
> >
> >
> >
> >
> >
> > *L-03-A ~]$ qdstat -b:5671 -a*
> >
> > *Router Addresses*
> >
> > *  class   addr                   phs  distrib    in-proc  local  remote
> > cntnr  in  out  thru   to-proc  from-proc*
> >
> > *
> > ============================================================
> > ======================================================*
> >
> > *  local   $_management_internal       closest    1        0      0
> > 0      0   0    0      6        6*
> >
> > *  local   $displayname                closest    1        0      0
> > 0      0   0    0      0        0*
> >
> > *  mobile  $management            0    closest    1        0      0
> > 0      9   0    0      9        0*
> >
> > *  local   $management                 closest    1        0      0
> > 0      0   0    0      0        0*
> >
> > *  router  Router.B                    closest    0        0      1
> > 0      0   0    19     0        19*
> >
> > *  mobile  foo.bar                1    balanced   0        0      0
> > 0      80  520  0      0        0*
> >
> > *  mobile  foo.bar                0    balanced   0        1      1
> > 0      0   40   0      0        0*
> >
> > *  local   qdhello                     flood      1        1      0
> > 0      0   0    0      269,611  269,622*
> >
> > *  local   qdrouter                    flood      1        0      0
> > 0      0   0    0      12       0*
> >
> > *  topo    qdrouter                    flood      1        0      1
> > 0      0   0    8,992  8,991    8,996*
> >
> > *  local   qdrouter.ma <http://qdrouter.ma>                 multicast  1
> >     0      0       0      0   0    0      4        0*
> >
> > *  topo    qdrouter.ma <http://qdrouter.ma>                 multicast  1
> >     0      1       0      0   0    3      0        3*
> >
> > *  local   temp.u6NMF1AFV0JpdBi        closest    0        1      0
> > 0      0   0    0      0        0*
> >
> >
> > *L-05-B ~]$ qdstat -b:5671 -a*
> >
> > *Router Addresses*
> >
> > *  class   addr                   phs  distrib    in-proc  local  remote
> > cntnr  in  out  thru  to-proc  from-proc*
> >
> > *
> > ============================================================
> > =====================================================*
> >
> > *  local   $_management_internal       closest    1        0      0
> > 0      0   0    0     6        6*
> >
> > *  local   $displayname                closest    1        0      0
> > 0      0   0    0     0        0*
> >
> > *  mobile  $management            0    closest    1        0      0
> > 0      9   0    0     9        0*
> >
> > *  local   $management                 closest    1        0      0
> > 0      0   0    0     0        0*
> >
> > *  router  Router.A                    closest    0        0      1
> > 0      0   0    6     0        6*
> >
> > *  mobile  foo.bar                1    balanced   0        0      0
> > 0      0   0    0     0        0*
> >
> > *  mobile  foo.bar                0    balanced   0        1      1
> > 0      0   0    0     0        0*
> >
> > *  local   qdhello                     flood      1        1      0
> > 0      0   0    0     570      571*
> >
> > *  local   qdrouter                    flood      1        0      0
> > 0      0   0    0     4        0*
> >
> > *  topo    qdrouter                    flood      1        0      1
> > 0      0   0    20    21       21*
> >
> > *  local   qdrouter.ma <http://qdrouter.ma>                 multicast  1
> >     0      0       0      0   0    0     4        0*
> >
> > *  topo    qdrouter.ma <http://qdrouter.ma>                 multicast  1
> >     0      1       0      0   0    0     0        1*
> >
> > *  local   temp.xwHdS0zzku0s3lR        closest    0        1      0
> > 0      0   0    0     0        0*
> >
> >
> >
> > is there something off in my configuration? or in my understand of QDR
> > netwokin just not up to snuff? should i care a little bit less about load
> > balancing because the VIP will round robin new connections out to
> different
> > codes?
> >
> > Thank you so much for your time and insight
> >
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dispatch Router load balancing config questions

Gordon Sim
On 01/08/17 22:20, Dan Langford wrote:
> are transactions expected to work with qpid dispatch router?

The router itself does not support transactions (it does not do any
storage of messages). From 0.8 you should be able to link route (local)
transaction requests to a transaction coordinator such as a broker
(using the pseudo address $coordinator) e.g. by adding something like:

linkRoute {
     prefix: $coordinator
     dir: in
     connection: txbroker
}

However this only really works if you also ensure that all transactional
operations go to the same broker as any control/coordination messages.

> on that note
> do you think i could also get XA transactions working?

No, at present there is no support for XA over AMQP 1.0 in any broker.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dispatch Router load balancing config questions

Robbie Gemmell
Administrator
In reply to this post by Dan Langford
On 1 August 2017 at 22:20, Dan Langford <[hidden email]> wrote:

> ( github gist with all the config and data from the Original Post
> https://gist.github.com/danlangford/4944dcc6c0d2703ffb8555603ed27340 )
>
> YES i was under pretty light load. a couple hundred or thousand messages at
> a time were all getting funneled into the local broker. You are right once
> i got 7 or 8 simultaneous connections all pushing in a couple million
> message i started to see the load overflow to the other router and broker.
> about 10% of messages in my setup were overflowing. thank you for your
> patience. I think this config is going to work great. i do think i just
> need a small adjustment to how i think about load balancing. when the cost
> was the same to each broker then i would get approx 50/50 split (that was
> when each router connected to each broker). with this setup where the local
> cost is less its more like "balancing out once under load" and not the
> traditional "balance evenly to avoid load". but now that i understand this
> its fine and i know what behavior to expect.
>
> so I do have some producers and consumers that need to use session
> transactions. we have seen those work fine over amqp1.0 when connected to a
> QPID Broker or Artemis broker. but with the config you see here connecting
> to a qpid dispatch router (directly or through our VIP) I cannot create a
> session with "transacted=true". I get a NullPointerException
>
> javax.jms.JMSException: java.lang.NullPointerException
> at
> org.apache.qpid.jms.exceptions.JmsExceptionSupport.create(JmsExceptionSupport.java:86)
> at
> org.apache.qpid.jms.exceptions.JmsExceptionSupport.create(JmsExceptionSupport.java:108)
> at org.apache.qpid.jms.JmsConnection.createResource(JmsConnection.java:609)
> at
> org.apache.qpid.jms.JmsLocalTransactionContext.begin(JmsLocalTransactionContext.java:125)
> at org.apache.qpid.jms.JmsSession.<init>(JmsSession.java:143)
> at org.apache.qpid.jms.JmsConnection.createSession(JmsConnection.java:299)
> at org.myorg.mymessaging.PostOffice.buildSession(PostOffice.java:149)
> ...
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.qpid.jms.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
> at
> org.apache.qpid.jms.provider.amqp.AmqpTransactionCoordinator.processDeliveryUpdates(AmqpTransactionCoordinator.java:117)
> at
> org.apache.qpid.jms.provider.amqp.AmqpProvider.processUpdates(AmqpProvider.java:928)
> at
> org.apache.qpid.jms.provider.amqp.AmqpProvider.access$1800(AmqpProvider.java:93)
> at
> org.apache.qpid.jms.provider.amqp.AmqpProvider$18.run(AmqpProvider.java:790)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NullPointerException
> at
> org.apache.qpid.jms.provider.amqp.AmqpTransactionCoordinator.processDeliveryUpdates(AmqpTransactionCoordinator.java:93)
> ... 10 more
>

I've raised https://issues.apache.org/jira/browse/QPIDJMS-307 to
investigate this and improve the handling. Though it did fail to
create the session, as it should have given Gordon's earlier
explanation, it didn't do it very well so there is definitely a client
bug there. Possibly also a router one if my hunch of what happened is
accurate.

> are transactions expected to work with qpid dispatch router? on that note
> do you think i could also get XA transactions working? would i need to use
> a link route? i dont know for sure but it sounds like a link route is a
> little more low level that an autolink.
>
> thank you
>
> On Tue, Aug 1, 2017 at 6:47 AM Ted Ross <[hidden email]> wrote:
>
>> Dan,
>>
>> There's one issue with your configuration which doesn't affect the load
>> balancing but will cause problems with receiving messages from the
>> brokers.  In the address.prefix, you use "foo.#".  This is a pure prefix
>> and it should simply be "foo".  The wildcards are coming in the next
>> release but are not implemented in the code you are using.
>>
>> Regarding your actual question.  I assume that you are testing this
>> configuration under light load (i.e. sending one message at a time).
>>
>> The way that the balancing works is that it will route to the consumer
>> (broker) with the fewest outstanding deliveries + inter-router cost.  This
>> means that it will favor the local broker over the remote one if there are
>> no in-flight deliveries.  The default (and minimum) cost for an
>> inter-router connection is 1.  You can set it to a higher value in the
>> listener or connector.
>>
>> If you are sending one-at-a-time synchronous sends, they will always go to
>> the local broker because the broker's zero outstanding deliveries will
>> always be less than the inter-router cost of 1.  If you send multiple
>> deliveries asynchronously, you will see them being distributed to both
>> brokers in the network.  You can make the local-affinity stronger by
>> increasing the inter-router cost.
>>
>> -Ted
>>
>> On Tue, Aug 1, 2017 at 3:47 AM, Dan Langford <[hidden email]>
>> wrote:
>>
>> > Last week I had a thread with lots of little questions around Dispatch
>> > Routers. Ted Ross has been awesome to answer most of those. As a result I
>> > feel like I have my QDR config shaping up a bit better. HOWEVER with some
>> > more very focused questions I thought it would be best to start a new
>> > thread. The problem I am seeing is that the routers are not distributing
>> > the message load across other brokers on the network. Here is a little
>> > diagram:
>> >
>> >
>> > RouterA-03 ----> ArtemisBrokerA
>> >  |
>> >  |
>> >  |
>> > \/
>> > RouterB-05 ----> ArtemisBrokerB
>> >
>> >
>> > *NOTE: i am currently using Qpid Dispatch Router from a RHEL repo. v
>> 0.7.0*
>> >
>> >
>> > Connections from clients come in through an F5 VIP which forwards those
>> > connections to either host L-03-A or L-05-B. Each of those hosts have a
>> > Qpid Dispatch Router installed in front of an Artemis broker. dispatch
>> > router on L-05-B is listening on an additional port that dispatch router
>> > L-03-A connects to for inter-router communication.
>> >
>> > If I go around my F5 VIP so i know I am connecting straight to L-05-B
>> and I
>> > send any number of messages into the router there all of those messages
>> end
>> > up in ArtemisBrokerB. I was hoping that some would go to BrokerB and some
>> > would go via RouterA over to BrokerA. Now when BrokerB is taken down
>> > CURRENTLY the messages sent to Router L-05-B ARE routed through Router
>> > L-03-A and then out to ArtemisBrokerB. (Currently receiving messages pull
>> > them in from both hosts so my questions only lies in message production
>> at
>> > the moment.)
>> >
>> > Do you agree that with this configuration you would expect messages to be
>> > load balanced between the two routes? I would like to paste in some of
>> our
>> > config and results from running qdstat. If you observe something that is
>> > misconfigured and are able to highlight it i would be very appreciative.
>> > First the config files for QDR instances. I tried my best to scrub them
>> of
>> > specific IPs and hostnames.
>> >
>> > *L-03-A qrouterd.conf*
>> >
>> >
>> > *router {*
>> >
>> > *    mode: interior*
>> >
>> > *    id: Router.A*
>> >
>> > *}*
>> >
>> > *log {*
>> >
>> > *    module: DEFAULT*
>> >
>> > *    enable: debug+*
>> >
>> > *    timestamp: yes*
>> >
>> > *}*
>> >
>> > *sslProfile {*
>> >
>> > *    name: my-ssl*
>> >
>> > *    certFile: /opt/org/my-ssl-info.pem*
>> >
>> > *    keyFile: /opt/org/my-ssl-info.pem*
>> >
>> > *    password: hellokitty42*
>> >
>> > *}*
>> >
>> > *listener {*
>> >
>> > *    role: normal*
>> >
>> > *    host: 0.0.0.0*
>> >
>> > *    port: 5671*
>> >
>> > *    authenticatePeer: no*
>> >
>> > *    saslMechanisms: ANONYMOUS*
>> >
>> > *    sslProfile: my-ssl*
>> >
>> > *}*
>> >
>> > *connector {*
>> >
>> > *    name: local-artemis*
>> >
>> > *    role: route-container*
>> >
>> > *    host: L-03-A*
>> >
>> > *    port: 61616*
>> >
>> > *    saslMechanisms: ANONYMOUS*
>> >
>> > *}*
>> >
>> > *connector {*
>> >
>> > *    name: routerb*
>> >
>> > *    role: inter-router*
>> >
>> > *    host: L-05-B*
>> >
>> > *    port: 6671*
>> >
>> > *    saslMechanisms: ANONYMOUS*
>> >
>> > *}*
>> >
>> > *address {*
>> >
>> > *    prefix: foo.#*
>> >
>> > *    waypoint: yes*
>> >
>> > *    distribution: balanced*
>> >
>> > *}*
>> >
>> > *autoLink {*
>> >
>> > *    addr: foo.bar*
>> >
>> > *    dir: in*
>> >
>> > *    connection: local-artemis*
>> >
>> > *}*
>> >
>> > *autoLink {*
>> >
>> > *    addr: foo.bar*
>> >
>> > *    dir: out*
>> >
>> > *    connection: local-artemis*
>> >
>> > *}*
>> >
>> >
>> > *L-05-B qrouterd.conf*
>> >
>> > *router {*
>> >
>> > *    mode: interior*
>> >
>> > *    id: Router.B*
>> >
>> > *}*
>> >
>> > *log {*
>> >
>> > *    module: DEFAULT*
>> >
>> > *    enable: debug+*
>> >
>> > *    timestamp: yes*
>> >
>> > *}*
>> >
>> > *sslProfile {*
>> >
>> > *    name: my-ssl*
>> >
>> > *    certFile: /opt/org/my-ssl-info.pem*
>> >
>> > *    keyFile: /opt/org/my-ssl-info.pem*
>> >
>> > *    password: hellokitty42*
>> >
>> > *}*
>> >
>> > *listener {*
>> >
>> > *    role: normal*
>> >
>> > *    host: 0.0.0.0*
>> >
>> > *    port: 5671*
>> >
>> > *    authenticatePeer: no*
>> >
>> > *    saslMechanisms: ANONYMOUS*
>> >
>> > *    sslProfile: my-ssl*
>> >
>> > *}*
>> >
>> > *listener {*
>> >
>> > *    role: inter-router*
>> >
>> > *    host: 0.0.0.0*
>> >
>> > *    port: 6671*
>> >
>> > *    authenticatePeer: no*
>> >
>> > *    saslMechanisms: ANONYMOUS*
>> >
>> > *    sslProfile: my-ssl*
>> >
>> > *}*
>> >
>> > *connector {*
>> >
>> > *    name: local-artemis*
>> >
>> > *    role: route-container*
>> >
>> > *    host: L-05-B*
>> >
>> > *    port: 61616*
>> >
>> > *    saslMechanisms: ANONYMOUS*
>> >
>> > *}*
>> >
>> > *address {*
>> >
>> > *    prefix: foo.#*
>> >
>> > *    waypoint: yes*
>> >
>> > *    distribution: balanced*
>> >
>> > *}*
>> >
>> > *autoLink {*
>> >
>> > *    addr: foo.bar*
>> >
>> > *    dir: in*
>> >
>> > *    connection: local-artemis*
>> >
>> > *}*
>> >
>> > *autoLink {*
>> >
>> > *    addr: foo.bar*
>> >
>> > *    dir: out*
>> >
>> > *    connection: local-artemis*
>> >
>> > *}*
>> >
>> >
>> >
>> >
>> > *L-03-A ~]$ qdstat -b:5671 -g*
>> >
>> > *Router Statistics*
>> >
>> > *  attr       value*
>> >
>> > *  =====================*
>> >
>> > *  Mode       interior*
>> >
>> > *  Area       0*
>> >
>> > *  Router Id  Router.A*
>> >
>> >
>> > *L-05-B ~]$ qdstat -b:5671 -g*
>> >
>> > *Router Statistics*
>> >
>> > *  attr       value*
>> >
>> > *  =====================*
>> >
>> > *  Mode       interior*
>> >
>> > *  Area       0*
>> >
>> > *  Router Id  Router.B*
>> >
>> >
>> >
>> >
>> > *L-03-A ~]$ qdstat -b:5671 -c*
>> >
>> > *Connections*
>> >
>> > *  Id      host             container                             role
>> >         dir  security     authentication*
>> >
>> > *
>> > ============================================================
>> > ======================================================*
>> >
>> > *  14      L-03-A:61616     L-03-A
>> > route-container  out  no-security  anonymous-user*
>> >
>> > *  107132  L-05-B:6671      Router.B
>> > inter-router     out  no-security  anonymous-user*
>> >
>> > *  107267  127.0.0.1:37916 <http://127.0.0.1:37916>
>> > 180851ba-7ce4-44d0-a937-97de16e781c4  normal           in   no-security
>> > anonymous-user*
>> >
>> >
>> > *L-05-B ~]$ qdstat -b:5671 -c*
>> >
>> > *Connections*
>> >
>> > *  Id   host                container                             role
>> >         dir  security     authentication*
>> >
>> > *
>> > ============================================================
>> > ======================================================*
>> >
>> > *  1    L-05-B:61616        L-05-B
>> > route-container  out  no-security  anonymous-user*
>> >
>> > *  2    10.<ip>.32:57182  Router.A
>> > inter-router     in   no-security  anonymous-user*
>> >
>> > *  143  127.0.0.1:48610 <http://127.0.0.1:48610>
>> > 93e6b3e3-443e-43d9-9c2c-0d648266ec19  normal           in   no-security
>> > anonymous-user*
>> >
>> >
>> >
>> >
>> > *L-03-A ~]$ qdstat -b:5671 -n*
>> >
>> > *Routers in the Network*
>> >
>> > *  router-id  next-hop  link*
>> >
>> > *  ===========================*
>> >
>> > *  Router.A   (self)    -*
>> >
>> > *  Router.B   -         0*
>> >
>> >
>> > *L-05-B ~]$ qdstat -b:5671 -n*
>> >
>> > *Routers in the Network*
>> >
>> > *  router-id  next-hop  link*
>> >
>> > *  ===========================*
>> >
>> > *  Router.A   -         0*
>> >
>> > *  Router.B   (self)    -*
>> >
>> >
>> >
>> >
>> > *L-03-A ~]$ qdstat -b:5671 --linkroutes*
>> >
>> > *Link Routes*
>> >
>> >
>> > *L-05-B ~]$ qdstat -b:5671 --linkroutes*
>> >
>> > *Link Routes*
>> >
>> >
>> >
>> >
>> > *L-03-A ~]$ qdstat -b:5671 --autolinks*
>> >
>> > *AutoLinks*
>> >
>> > *  addr     dir  phs  link  status  lastErr*
>> >
>> > *  ==========================================*
>> >
>> > *  foo.bar  in   1    8     active*
>> >
>> > *  foo.bar  out  0    9     active*
>> >
>> >
>> > *L-05-B ~]$ qdstat -b:5671 --autolinks*
>> >
>> > *AutoLinks*
>> >
>> > *  addr     dir  phs  link  status  lastErr*
>> >
>> > *  ==========================================*
>> >
>> > *  foo.bar  in   1    4     active*
>> >
>> > *  foo.bar  out  0    5     active*
>> >
>> >
>> >
>> >
>> > *L-03-A ~]$ qdstat -b:5671 -l*
>> >
>> > *Router Links*
>> >
>> > *  type            dir  conn id  id  peer  class   addr
>> > phs  cap  undel  unsett  del  presett  acc  rej  rel  mod  admin    oper*
>> >
>> > *
>> > ============================================================
>> > ============================================================
>> > ========================*
>> >
>> > *  endpoint        in   14       8         mobile  foo.bar
>>  1
>> >   250  0      0       80   0        80   0    0    0    enabled  up*
>> >
>> > *  endpoint        out  14       9         mobile  foo.bar
>>  0
>> >   250  0      0       40   0        40   0    0    0    enabled  up*
>> >
>> > *  router-control  in   107132   15
>> >   250  0      0       524  524      0    0    0    0    enabled  up*
>> >
>> > *  router-control  out  107132   16        local   qdhello
>> >   250  0      0       527  527      0    0    0    0    enabled  up*
>> >
>> > *  inter-router    in   107132   17
>> >   250  0      0       0    0        0    0    0    0    enabled  up*
>> >
>> > *  inter-router    out  107132   18
>> >   250  0      0       0    0        0    0    0    0    enabled  up*
>> >
>> > *  endpoint        in   107336   33        mobile  $management
>>  0
>> >   250  0      0       1    0        1    0    0    0    enabled  up*
>> >
>> > *  endpoint        out  107336   34        local   temp.liHSN39azHuUPkg
>> >   250  0      0       0    0        0    0    0    0    enabled  up*
>> >
>> >
>> > *L-05-B ~]$ qdstat -b:5671 -l*
>> >
>> > *Router Links*
>> >
>> > *  type            dir  conn id  id  peer  class   addr
>> > phs  cap  undel  unsett  del  presett  acc  rej  rel  mod  admin    oper*
>> >
>> > *
>> > ============================================================
>> > ============================================================
>> > ========================*
>> >
>> > *  endpoint        in   1        4         mobile  foo.bar
>>  1
>> >   250  40     0       0    0        0    0    0    0    enabled  up*
>> >
>> > *  endpoint        out  1        5         mobile  foo.bar
>>  0
>> >   250  0      0       0    0        0    0    0    0    enabled  up*
>> >
>> > *  router-control  out  2        6         local   qdhello
>> >   250  0      0       524  524      0    0    0    0    enabled  up*
>> >
>> > *  router-control  in   2        7
>> >   250  0      0       527  527      0    0    0    0    enabled  up*
>> >
>> > *  inter-router    out  2        8
>> >   250  0      0       0    0        0    0    0    0    enabled  up*
>> >
>> > *  inter-router    in   2        9
>> >   250  0      0       0    0        0    0    0    0    enabled  up*
>> >
>> > *  endpoint        in   212      24        mobile  $management
>>  0
>> >   250  0      0       1    0        1    0    0    0    enabled  up*
>> >
>> > *  endpoint        out  212      25        local   temp.BJPESwMGFo+7qiy
>> >   250  0      0       0    0        0    0    0    0    enabled  up*
>> >
>> >
>> >
>> >
>> >
>> > *L-03-A ~]$ qdstat -b:5671 -a*
>> >
>> > *Router Addresses*
>> >
>> > *  class   addr                   phs  distrib    in-proc  local  remote
>> > cntnr  in  out  thru   to-proc  from-proc*
>> >
>> > *
>> > ============================================================
>> > ======================================================*
>> >
>> > *  local   $_management_internal       closest    1        0      0
>> > 0      0   0    0      6        6*
>> >
>> > *  local   $displayname                closest    1        0      0
>> > 0      0   0    0      0        0*
>> >
>> > *  mobile  $management            0    closest    1        0      0
>> > 0      9   0    0      9        0*
>> >
>> > *  local   $management                 closest    1        0      0
>> > 0      0   0    0      0        0*
>> >
>> > *  router  Router.B                    closest    0        0      1
>> > 0      0   0    19     0        19*
>> >
>> > *  mobile  foo.bar                1    balanced   0        0      0
>> > 0      80  520  0      0        0*
>> >
>> > *  mobile  foo.bar                0    balanced   0        1      1
>> > 0      0   40   0      0        0*
>> >
>> > *  local   qdhello                     flood      1        1      0
>> > 0      0   0    0      269,611  269,622*
>> >
>> > *  local   qdrouter                    flood      1        0      0
>> > 0      0   0    0      12       0*
>> >
>> > *  topo    qdrouter                    flood      1        0      1
>> > 0      0   0    8,992  8,991    8,996*
>> >
>> > *  local   qdrouter.ma <http://qdrouter.ma>                 multicast  1
>> >     0      0       0      0   0    0      4        0*
>> >
>> > *  topo    qdrouter.ma <http://qdrouter.ma>                 multicast  1
>> >     0      1       0      0   0    3      0        3*
>> >
>> > *  local   temp.u6NMF1AFV0JpdBi        closest    0        1      0
>> > 0      0   0    0      0        0*
>> >
>> >
>> > *L-05-B ~]$ qdstat -b:5671 -a*
>> >
>> > *Router Addresses*
>> >
>> > *  class   addr                   phs  distrib    in-proc  local  remote
>> > cntnr  in  out  thru  to-proc  from-proc*
>> >
>> > *
>> > ============================================================
>> > =====================================================*
>> >
>> > *  local   $_management_internal       closest    1        0      0
>> > 0      0   0    0     6        6*
>> >
>> > *  local   $displayname                closest    1        0      0
>> > 0      0   0    0     0        0*
>> >
>> > *  mobile  $management            0    closest    1        0      0
>> > 0      9   0    0     9        0*
>> >
>> > *  local   $management                 closest    1        0      0
>> > 0      0   0    0     0        0*
>> >
>> > *  router  Router.A                    closest    0        0      1
>> > 0      0   0    6     0        6*
>> >
>> > *  mobile  foo.bar                1    balanced   0        0      0
>> > 0      0   0    0     0        0*
>> >
>> > *  mobile  foo.bar                0    balanced   0        1      1
>> > 0      0   0    0     0        0*
>> >
>> > *  local   qdhello                     flood      1        1      0
>> > 0      0   0    0     570      571*
>> >
>> > *  local   qdrouter                    flood      1        0      0
>> > 0      0   0    0     4        0*
>> >
>> > *  topo    qdrouter                    flood      1        0      1
>> > 0      0   0    20    21       21*
>> >
>> > *  local   qdrouter.ma <http://qdrouter.ma>                 multicast  1
>> >     0      0       0      0   0    0     4        0*
>> >
>> > *  topo    qdrouter.ma <http://qdrouter.ma>                 multicast  1
>> >     0      1       0      0   0    0     0        1*
>> >
>> > *  local   temp.xwHdS0zzku0s3lR        closest    0        1      0
>> > 0      0   0    0     0        0*
>> >
>> >
>> >
>> > is there something off in my configuration? or in my understand of QDR
>> > netwokin just not up to snuff? should i care a little bit less about load
>> > balancing because the VIP will round robin new connections out to
>> different
>> > codes?
>> >
>> > Thank you so much for your time and insight
>> >
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dispatch Router load balancing config questions

Dan Langford
Thank you all very much. i will upgrade to dispatch router 0.8.0 and forget
about XA transactions. i was thinking that all of these brokers out there
support XA transactions and i am now realizing that they support XA on
protocols that are not AMQP 1.0. in the past few days i have also studied
more about XA and realize that for all of my clients they do not need
transactions or that the local transactions will be just great.

On Wed, Aug 2, 2017 at 4:43 AM Robbie Gemmell <[hidden email]>
wrote:

> On 1 August 2017 at 22:20, Dan Langford <[hidden email]> wrote:
> > ( github gist with all the config and data from the Original Post
> > https://gist.github.com/danlangford/4944dcc6c0d2703ffb8555603ed27340 )
> >
> > YES i was under pretty light load. a couple hundred or thousand messages
> at
> > a time were all getting funneled into the local broker. You are right
> once
> > i got 7 or 8 simultaneous connections all pushing in a couple million
> > message i started to see the load overflow to the other router and
> broker.
> > about 10% of messages in my setup were overflowing. thank you for your
> > patience. I think this config is going to work great. i do think i just
> > need a small adjustment to how i think about load balancing. when the
> cost
> > was the same to each broker then i would get approx 50/50 split (that was
> > when each router connected to each broker). with this setup where the
> local
> > cost is less its more like "balancing out once under load" and not the
> > traditional "balance evenly to avoid load". but now that i understand
> this
> > its fine and i know what behavior to expect.
> >
> > so I do have some producers and consumers that need to use session
> > transactions. we have seen those work fine over amqp1.0 when connected
> to a
> > QPID Broker or Artemis broker. but with the config you see here
> connecting
> > to a qpid dispatch router (directly or through our VIP) I cannot create a
> > session with "transacted=true". I get a NullPointerException
> >
> > javax.jms.JMSException: java.lang.NullPointerException
> > at
> >
> org.apache.qpid.jms.exceptions.JmsExceptionSupport.create(JmsExceptionSupport.java:86)
> > at
> >
> org.apache.qpid.jms.exceptions.JmsExceptionSupport.create(JmsExceptionSupport.java:108)
> > at
> org.apache.qpid.jms.JmsConnection.createResource(JmsConnection.java:609)
> > at
> >
> org.apache.qpid.jms.JmsLocalTransactionContext.begin(JmsLocalTransactionContext.java:125)
> > at org.apache.qpid.jms.JmsSession.<init>(JmsSession.java:143)
> > at
> org.apache.qpid.jms.JmsConnection.createSession(JmsConnection.java:299)
> > at org.myorg.mymessaging.PostOffice.buildSession(PostOffice.java:149)
> > ...
> > Caused by: java.io.IOException: java.lang.NullPointerException
> > at
> >
> org.apache.qpid.jms.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
> > at
> >
> org.apache.qpid.jms.provider.amqp.AmqpTransactionCoordinator.processDeliveryUpdates(AmqpTransactionCoordinator.java:117)
> > at
> >
> org.apache.qpid.jms.provider.amqp.AmqpProvider.processUpdates(AmqpProvider.java:928)
> > at
> >
> org.apache.qpid.jms.provider.amqp.AmqpProvider.access$1800(AmqpProvider.java:93)
> > at
> >
> org.apache.qpid.jms.provider.amqp.AmqpProvider$18.run(AmqpProvider.java:790)
> > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> > at
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:748)
> > Caused by: java.lang.NullPointerException
> > at
> >
> org.apache.qpid.jms.provider.amqp.AmqpTransactionCoordinator.processDeliveryUpdates(AmqpTransactionCoordinator.java:93)
> > ... 10 more
> >
>
> I've raised https://issues.apache.org/jira/browse/QPIDJMS-307 to
> investigate this and improve the handling. Though it did fail to
> create the session, as it should have given Gordon's earlier
> explanation, it didn't do it very well so there is definitely a client
> bug there. Possibly also a router one if my hunch of what happened is
> accurate.
>
> > are transactions expected to work with qpid dispatch router? on that note
> > do you think i could also get XA transactions working? would i need to
> use
> > a link route? i dont know for sure but it sounds like a link route is a
> > little more low level that an autolink.
> >
> > thank you
> >
> > On Tue, Aug 1, 2017 at 6:47 AM Ted Ross <[hidden email]> wrote:
> >
> >> Dan,
> >>
> >> There's one issue with your configuration which doesn't affect the load
> >> balancing but will cause problems with receiving messages from the
> >> brokers.  In the address.prefix, you use "foo.#".  This is a pure prefix
> >> and it should simply be "foo".  The wildcards are coming in the next
> >> release but are not implemented in the code you are using.
> >>
> >> Regarding your actual question.  I assume that you are testing this
> >> configuration under light load (i.e. sending one message at a time).
> >>
> >> The way that the balancing works is that it will route to the consumer
> >> (broker) with the fewest outstanding deliveries + inter-router cost.
> This
> >> means that it will favor the local broker over the remote one if there
> are
> >> no in-flight deliveries.  The default (and minimum) cost for an
> >> inter-router connection is 1.  You can set it to a higher value in the
> >> listener or connector.
> >>
> >> If you are sending one-at-a-time synchronous sends, they will always go
> to
> >> the local broker because the broker's zero outstanding deliveries will
> >> always be less than the inter-router cost of 1.  If you send multiple
> >> deliveries asynchronously, you will see them being distributed to both
> >> brokers in the network.  You can make the local-affinity stronger by
> >> increasing the inter-router cost.
> >>
> >> -Ted
> >>
> >> On Tue, Aug 1, 2017 at 3:47 AM, Dan Langford <[hidden email]>
> >> wrote:
> >>
> >> > Last week I had a thread with lots of little questions around Dispatch
> >> > Routers. Ted Ross has been awesome to answer most of those. As a
> result I
> >> > feel like I have my QDR config shaping up a bit better. HOWEVER with
> some
> >> > more very focused questions I thought it would be best to start a new
> >> > thread. The problem I am seeing is that the routers are not
> distributing
> >> > the message load across other brokers on the network. Here is a little
> >> > diagram:
> >> >
> >> >
> >> > RouterA-03 ----> ArtemisBrokerA
> >> >  |
> >> >  |
> >> >  |
> >> > \/
> >> > RouterB-05 ----> ArtemisBrokerB
> >> >
> >> >
> >> > *NOTE: i am currently using Qpid Dispatch Router from a RHEL repo. v
> >> 0.7.0*
> >> >
> >> >
> >> > Connections from clients come in through an F5 VIP which forwards
> those
> >> > connections to either host L-03-A or L-05-B. Each of those hosts have
> a
> >> > Qpid Dispatch Router installed in front of an Artemis broker. dispatch
> >> > router on L-05-B is listening on an additional port that dispatch
> router
> >> > L-03-A connects to for inter-router communication.
> >> >
> >> > If I go around my F5 VIP so i know I am connecting straight to L-05-B
> >> and I
> >> > send any number of messages into the router there all of those
> messages
> >> end
> >> > up in ArtemisBrokerB. I was hoping that some would go to BrokerB and
> some
> >> > would go via RouterA over to BrokerA. Now when BrokerB is taken down
> >> > CURRENTLY the messages sent to Router L-05-B ARE routed through Router
> >> > L-03-A and then out to ArtemisBrokerB. (Currently receiving messages
> pull
> >> > them in from both hosts so my questions only lies in message
> production
> >> at
> >> > the moment.)
> >> >
> >> > Do you agree that with this configuration you would expect messages
> to be
> >> > load balanced between the two routes? I would like to paste in some of
> >> our
> >> > config and results from running qdstat. If you observe something that
> is
> >> > misconfigured and are able to highlight it i would be very
> appreciative.
> >> > First the config files for QDR instances. I tried my best to scrub
> them
> >> of
> >> > specific IPs and hostnames.
> >> >
> >> > *L-03-A qrouterd.conf*
> >> >
> >> >
> >> > *router {*
> >> >
> >> > *    mode: interior*
> >> >
> >> > *    id: Router.A*
> >> >
> >> > *}*
> >> >
> >> > *log {*
> >> >
> >> > *    module: DEFAULT*
> >> >
> >> > *    enable: debug+*
> >> >
> >> > *    timestamp: yes*
> >> >
> >> > *}*
> >> >
> >> > *sslProfile {*
> >> >
> >> > *    name: my-ssl*
> >> >
> >> > *    certFile: /opt/org/my-ssl-info.pem*
> >> >
> >> > *    keyFile: /opt/org/my-ssl-info.pem*
> >> >
> >> > *    password: hellokitty42*
> >> >
> >> > *}*
> >> >
> >> > *listener {*
> >> >
> >> > *    role: normal*
> >> >
> >> > *    host: 0.0.0.0*
> >> >
> >> > *    port: 5671*
> >> >
> >> > *    authenticatePeer: no*
> >> >
> >> > *    saslMechanisms: ANONYMOUS*
> >> >
> >> > *    sslProfile: my-ssl*
> >> >
> >> > *}*
> >> >
> >> > *connector {*
> >> >
> >> > *    name: local-artemis*
> >> >
> >> > *    role: route-container*
> >> >
> >> > *    host: L-03-A*
> >> >
> >> > *    port: 61616*
> >> >
> >> > *    saslMechanisms: ANONYMOUS*
> >> >
> >> > *}*
> >> >
> >> > *connector {*
> >> >
> >> > *    name: routerb*
> >> >
> >> > *    role: inter-router*
> >> >
> >> > *    host: L-05-B*
> >> >
> >> > *    port: 6671*
> >> >
> >> > *    saslMechanisms: ANONYMOUS*
> >> >
> >> > *}*
> >> >
> >> > *address {*
> >> >
> >> > *    prefix: foo.#*
> >> >
> >> > *    waypoint: yes*
> >> >
> >> > *    distribution: balanced*
> >> >
> >> > *}*
> >> >
> >> > *autoLink {*
> >> >
> >> > *    addr: foo.bar*
> >> >
> >> > *    dir: in*
> >> >
> >> > *    connection: local-artemis*
> >> >
> >> > *}*
> >> >
> >> > *autoLink {*
> >> >
> >> > *    addr: foo.bar*
> >> >
> >> > *    dir: out*
> >> >
> >> > *    connection: local-artemis*
> >> >
> >> > *}*
> >> >
> >> >
> >> > *L-05-B qrouterd.conf*
> >> >
> >> > *router {*
> >> >
> >> > *    mode: interior*
> >> >
> >> > *    id: Router.B*
> >> >
> >> > *}*
> >> >
> >> > *log {*
> >> >
> >> > *    module: DEFAULT*
> >> >
> >> > *    enable: debug+*
> >> >
> >> > *    timestamp: yes*
> >> >
> >> > *}*
> >> >
> >> > *sslProfile {*
> >> >
> >> > *    name: my-ssl*
> >> >
> >> > *    certFile: /opt/org/my-ssl-info.pem*
> >> >
> >> > *    keyFile: /opt/org/my-ssl-info.pem*
> >> >
> >> > *    password: hellokitty42*
> >> >
> >> > *}*
> >> >
> >> > *listener {*
> >> >
> >> > *    role: normal*
> >> >
> >> > *    host: 0.0.0.0*
> >> >
> >> > *    port: 5671*
> >> >
> >> > *    authenticatePeer: no*
> >> >
> >> > *    saslMechanisms: ANONYMOUS*
> >> >
> >> > *    sslProfile: my-ssl*
> >> >
> >> > *}*
> >> >
> >> > *listener {*
> >> >
> >> > *    role: inter-router*
> >> >
> >> > *    host: 0.0.0.0*
> >> >
> >> > *    port: 6671*
> >> >
> >> > *    authenticatePeer: no*
> >> >
> >> > *    saslMechanisms: ANONYMOUS*
> >> >
> >> > *    sslProfile: my-ssl*
> >> >
> >> > *}*
> >> >
> >> > *connector {*
> >> >
> >> > *    name: local-artemis*
> >> >
> >> > *    role: route-container*
> >> >
> >> > *    host: L-05-B*
> >> >
> >> > *    port: 61616*
> >> >
> >> > *    saslMechanisms: ANONYMOUS*
> >> >
> >> > *}*
> >> >
> >> > *address {*
> >> >
> >> > *    prefix: foo.#*
> >> >
> >> > *    waypoint: yes*
> >> >
> >> > *    distribution: balanced*
> >> >
> >> > *}*
> >> >
> >> > *autoLink {*
> >> >
> >> > *    addr: foo.bar*
> >> >
> >> > *    dir: in*
> >> >
> >> > *    connection: local-artemis*
> >> >
> >> > *}*
> >> >
> >> > *autoLink {*
> >> >
> >> > *    addr: foo.bar*
> >> >
> >> > *    dir: out*
> >> >
> >> > *    connection: local-artemis*
> >> >
> >> > *}*
> >> >
> >> >
> >> >
> >> >
> >> > *L-03-A ~]$ qdstat -b:5671 -g*
> >> >
> >> > *Router Statistics*
> >> >
> >> > *  attr       value*
> >> >
> >> > *  =====================*
> >> >
> >> > *  Mode       interior*
> >> >
> >> > *  Area       0*
> >> >
> >> > *  Router Id  Router.A*
> >> >
> >> >
> >> > *L-05-B ~]$ qdstat -b:5671 -g*
> >> >
> >> > *Router Statistics*
> >> >
> >> > *  attr       value*
> >> >
> >> > *  =====================*
> >> >
> >> > *  Mode       interior*
> >> >
> >> > *  Area       0*
> >> >
> >> > *  Router Id  Router.B*
> >> >
> >> >
> >> >
> >> >
> >> > *L-03-A ~]$ qdstat -b:5671 -c*
> >> >
> >> > *Connections*
> >> >
> >> > *  Id      host             container                             role
> >> >         dir  security     authentication*
> >> >
> >> > *
> >> > ============================================================
> >> > ======================================================*
> >> >
> >> > *  14      L-03-A:61616     L-03-A
> >> > route-container  out  no-security  anonymous-user*
> >> >
> >> > *  107132  L-05-B:6671      Router.B
> >> > inter-router     out  no-security  anonymous-user*
> >> >
> >> > *  107267  127.0.0.1:37916 <http://127.0.0.1:37916>
> >> > 180851ba-7ce4-44d0-a937-97de16e781c4  normal           in
>  no-security
> >> > anonymous-user*
> >> >
> >> >
> >> > *L-05-B ~]$ qdstat -b:5671 -c*
> >> >
> >> > *Connections*
> >> >
> >> > *  Id   host                container                             role
> >> >         dir  security     authentication*
> >> >
> >> > *
> >> > ============================================================
> >> > ======================================================*
> >> >
> >> > *  1    L-05-B:61616        L-05-B
> >> > route-container  out  no-security  anonymous-user*
> >> >
> >> > *  2    10.<ip>.32:57182  Router.A
> >> > inter-router     in   no-security  anonymous-user*
> >> >
> >> > *  143  127.0.0.1:48610 <http://127.0.0.1:48610>
> >> > 93e6b3e3-443e-43d9-9c2c-0d648266ec19  normal           in
>  no-security
> >> > anonymous-user*
> >> >
> >> >
> >> >
> >> >
> >> > *L-03-A ~]$ qdstat -b:5671 -n*
> >> >
> >> > *Routers in the Network*
> >> >
> >> > *  router-id  next-hop  link*
> >> >
> >> > *  ===========================*
> >> >
> >> > *  Router.A   (self)    -*
> >> >
> >> > *  Router.B   -         0*
> >> >
> >> >
> >> > *L-05-B ~]$ qdstat -b:5671 -n*
> >> >
> >> > *Routers in the Network*
> >> >
> >> > *  router-id  next-hop  link*
> >> >
> >> > *  ===========================*
> >> >
> >> > *  Router.A   -         0*
> >> >
> >> > *  Router.B   (self)    -*
> >> >
> >> >
> >> >
> >> >
> >> > *L-03-A ~]$ qdstat -b:5671 --linkroutes*
> >> >
> >> > *Link Routes*
> >> >
> >> >
> >> > *L-05-B ~]$ qdstat -b:5671 --linkroutes*
> >> >
> >> > *Link Routes*
> >> >
> >> >
> >> >
> >> >
> >> > *L-03-A ~]$ qdstat -b:5671 --autolinks*
> >> >
> >> > *AutoLinks*
> >> >
> >> > *  addr     dir  phs  link  status  lastErr*
> >> >
> >> > *  ==========================================*
> >> >
> >> > *  foo.bar  in   1    8     active*
> >> >
> >> > *  foo.bar  out  0    9     active*
> >> >
> >> >
> >> > *L-05-B ~]$ qdstat -b:5671 --autolinks*
> >> >
> >> > *AutoLinks*
> >> >
> >> > *  addr     dir  phs  link  status  lastErr*
> >> >
> >> > *  ==========================================*
> >> >
> >> > *  foo.bar  in   1    4     active*
> >> >
> >> > *  foo.bar  out  0    5     active*
> >> >
> >> >
> >> >
> >> >
> >> > *L-03-A ~]$ qdstat -b:5671 -l*
> >> >
> >> > *Router Links*
> >> >
> >> > *  type            dir  conn id  id  peer  class   addr
> >> > phs  cap  undel  unsett  del  presett  acc  rej  rel  mod  admin
> oper*
> >> >
> >> > *
> >> > ============================================================
> >> > ============================================================
> >> > ========================*
> >> >
> >> > *  endpoint        in   14       8         mobile  foo.bar
> >>  1
> >> >   250  0      0       80   0        80   0    0    0    enabled  up*
> >> >
> >> > *  endpoint        out  14       9         mobile  foo.bar
> >>  0
> >> >   250  0      0       40   0        40   0    0    0    enabled  up*
> >> >
> >> > *  router-control  in   107132   15
> >> >   250  0      0       524  524      0    0    0    0    enabled  up*
> >> >
> >> > *  router-control  out  107132   16        local   qdhello
> >> >   250  0      0       527  527      0    0    0    0    enabled  up*
> >> >
> >> > *  inter-router    in   107132   17
> >> >   250  0      0       0    0        0    0    0    0    enabled  up*
> >> >
> >> > *  inter-router    out  107132   18
> >> >   250  0      0       0    0        0    0    0    0    enabled  up*
> >> >
> >> > *  endpoint        in   107336   33        mobile  $management
> >>  0
> >> >   250  0      0       1    0        1    0    0    0    enabled  up*
> >> >
> >> > *  endpoint        out  107336   34        local
>  temp.liHSN39azHuUPkg
> >> >   250  0      0       0    0        0    0    0    0    enabled  up*
> >> >
> >> >
> >> > *L-05-B ~]$ qdstat -b:5671 -l*
> >> >
> >> > *Router Links*
> >> >
> >> > *  type            dir  conn id  id  peer  class   addr
> >> > phs  cap  undel  unsett  del  presett  acc  rej  rel  mod  admin
> oper*
> >> >
> >> > *
> >> > ============================================================
> >> > ============================================================
> >> > ========================*
> >> >
> >> > *  endpoint        in   1        4         mobile  foo.bar
> >>  1
> >> >   250  40     0       0    0        0    0    0    0    enabled  up*
> >> >
> >> > *  endpoint        out  1        5         mobile  foo.bar
> >>  0
> >> >   250  0      0       0    0        0    0    0    0    enabled  up*
> >> >
> >> > *  router-control  out  2        6         local   qdhello
> >> >   250  0      0       524  524      0    0    0    0    enabled  up*
> >> >
> >> > *  router-control  in   2        7
> >> >   250  0      0       527  527      0    0    0    0    enabled  up*
> >> >
> >> > *  inter-router    out  2        8
> >> >   250  0      0       0    0        0    0    0    0    enabled  up*
> >> >
> >> > *  inter-router    in   2        9
> >> >   250  0      0       0    0        0    0    0    0    enabled  up*
> >> >
> >> > *  endpoint        in   212      24        mobile  $management
> >>  0
> >> >   250  0      0       1    0        1    0    0    0    enabled  up*
> >> >
> >> > *  endpoint        out  212      25        local
>  temp.BJPESwMGFo+7qiy
> >> >   250  0      0       0    0        0    0    0    0    enabled  up*
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > *L-03-A ~]$ qdstat -b:5671 -a*
> >> >
> >> > *Router Addresses*
> >> >
> >> > *  class   addr                   phs  distrib    in-proc  local
> remote
> >> > cntnr  in  out  thru   to-proc  from-proc*
> >> >
> >> > *
> >> > ============================================================
> >> > ======================================================*
> >> >
> >> > *  local   $_management_internal       closest    1        0      0
> >> > 0      0   0    0      6        6*
> >> >
> >> > *  local   $displayname                closest    1        0      0
> >> > 0      0   0    0      0        0*
> >> >
> >> > *  mobile  $management            0    closest    1        0      0
> >> > 0      9   0    0      9        0*
> >> >
> >> > *  local   $management                 closest    1        0      0
> >> > 0      0   0    0      0        0*
> >> >
> >> > *  router  Router.B                    closest    0        0      1
> >> > 0      0   0    19     0        19*
> >> >
> >> > *  mobile  foo.bar                1    balanced   0        0      0
> >> > 0      80  520  0      0        0*
> >> >
> >> > *  mobile  foo.bar                0    balanced   0        1      1
> >> > 0      0   40   0      0        0*
> >> >
> >> > *  local   qdhello                     flood      1        1      0
> >> > 0      0   0    0      269,611  269,622*
> >> >
> >> > *  local   qdrouter                    flood      1        0      0
> >> > 0      0   0    0      12       0*
> >> >
> >> > *  topo    qdrouter                    flood      1        0      1
> >> > 0      0   0    8,992  8,991    8,996*
> >> >
> >> > *  local   qdrouter.ma <http://qdrouter.ma>
>  multicast  1
> >> >     0      0       0      0   0    0      4        0*
> >> >
> >> > *  topo    qdrouter.ma <http://qdrouter.ma>
>  multicast  1
> >> >     0      1       0      0   0    3      0        3*
> >> >
> >> > *  local   temp.u6NMF1AFV0JpdBi        closest    0        1      0
> >> > 0      0   0    0      0        0*
> >> >
> >> >
> >> > *L-05-B ~]$ qdstat -b:5671 -a*
> >> >
> >> > *Router Addresses*
> >> >
> >> > *  class   addr                   phs  distrib    in-proc  local
> remote
> >> > cntnr  in  out  thru  to-proc  from-proc*
> >> >
> >> > *
> >> > ============================================================
> >> > =====================================================*
> >> >
> >> > *  local   $_management_internal       closest    1        0      0
> >> > 0      0   0    0     6        6*
> >> >
> >> > *  local   $displayname                closest    1        0      0
> >> > 0      0   0    0     0        0*
> >> >
> >> > *  mobile  $management            0    closest    1        0      0
> >> > 0      9   0    0     9        0*
> >> >
> >> > *  local   $management                 closest    1        0      0
> >> > 0      0   0    0     0        0*
> >> >
> >> > *  router  Router.A                    closest    0        0      1
> >> > 0      0   0    6     0        6*
> >> >
> >> > *  mobile  foo.bar                1    balanced   0        0      0
> >> > 0      0   0    0     0        0*
> >> >
> >> > *  mobile  foo.bar                0    balanced   0        1      1
> >> > 0      0   0    0     0        0*
> >> >
> >> > *  local   qdhello                     flood      1        1      0
> >> > 0      0   0    0     570      571*
> >> >
> >> > *  local   qdrouter                    flood      1        0      0
> >> > 0      0   0    0     4        0*
> >> >
> >> > *  topo    qdrouter                    flood      1        0      1
> >> > 0      0   0    20    21       21*
> >> >
> >> > *  local   qdrouter.ma <http://qdrouter.ma>
>  multicast  1
> >> >     0      0       0      0   0    0     4        0*
> >> >
> >> > *  topo    qdrouter.ma <http://qdrouter.ma>
>  multicast  1
> >> >     0      1       0      0   0    0     0        1*
> >> >
> >> > *  local   temp.xwHdS0zzku0s3lR        closest    0        1      0
> >> > 0      0   0    0     0        0*
> >> >
> >> >
> >> >
> >> > is there something off in my configuration? or in my understand of QDR
> >> > netwokin just not up to snuff? should i care a little bit less about
> load
> >> > balancing because the VIP will round robin new connections out to
> >> different
> >> > codes?
> >> >
> >> > Thank you so much for your time and insight
> >> >
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dispatch Router load balancing config questions

Dan Langford
back from vacation now. i upgraded to 0.8.0 and local/session transactions
look like they are working as expected.

*Question about JMS filters / message selectors*: when i connect to my
artemis broker with the artemis client JMS filters work. when i connect to
my artemis broker with qpid-jms-client then jms filters work. HOWEVER when
i connect to THROUGH the dispatch routers with the qpid-jms-client the jms
filters (configured in the client code) seem to be ignored or lost. i
receive messages as if no filter was supplied. does QPID dispatch router
support filters configured programmatically via the client? with my
aforementioned configuration
https://gist.github.com/danlangford/4944dcc6c0d2703ffb8555603ed27340 would
you expect jms filters to work?

On Wed, Aug 2, 2017 at 9:46 AM Dan Langford <[hidden email]> wrote:

> Thank you all very much. i will upgrade to dispatch router 0.8.0 and
> forget about XA transactions. i was thinking that all of these brokers out
> there support XA transactions and i am now realizing that they support XA
> on protocols that are not AMQP 1.0. in the past few days i have also
> studied more about XA and realize that for all of my clients they do not
> need transactions or that the local transactions will be just great.
>
> On Wed, Aug 2, 2017 at 4:43 AM Robbie Gemmell <[hidden email]>
> wrote:
>
>> On 1 August 2017 at 22:20, Dan Langford <[hidden email]> wrote:
>> > ( github gist with all the config and data from the Original Post
>> > https://gist.github.com/danlangford/4944dcc6c0d2703ffb8555603ed27340 )
>> >
>> > YES i was under pretty light load. a couple hundred or thousand
>> messages at
>> > a time were all getting funneled into the local broker. You are right
>> once
>> > i got 7 or 8 simultaneous connections all pushing in a couple million
>> > message i started to see the load overflow to the other router and
>> broker.
>> > about 10% of messages in my setup were overflowing. thank you for your
>> > patience. I think this config is going to work great. i do think i just
>> > need a small adjustment to how i think about load balancing. when the
>> cost
>> > was the same to each broker then i would get approx 50/50 split (that
>> was
>> > when each router connected to each broker). with this setup where the
>> local
>> > cost is less its more like "balancing out once under load" and not the
>> > traditional "balance evenly to avoid load". but now that i understand
>> this
>> > its fine and i know what behavior to expect.
>> >
>> > so I do have some producers and consumers that need to use session
>> > transactions. we have seen those work fine over amqp1.0 when connected
>> to a
>> > QPID Broker or Artemis broker. but with the config you see here
>> connecting
>> > to a qpid dispatch router (directly or through our VIP) I cannot create
>> a
>> > session with "transacted=true". I get a NullPointerException
>> >
>> > javax.jms.JMSException: java.lang.NullPointerException
>> > at
>> >
>> org.apache.qpid.jms.exceptions.JmsExceptionSupport.create(JmsExceptionSupport.java:86)
>> > at
>> >
>> org.apache.qpid.jms.exceptions.JmsExceptionSupport.create(JmsExceptionSupport.java:108)
>> > at
>> org.apache.qpid.jms.JmsConnection.createResource(JmsConnection.java:609)
>> > at
>> >
>> org.apache.qpid.jms.JmsLocalTransactionContext.begin(JmsLocalTransactionContext.java:125)
>> > at org.apache.qpid.jms.JmsSession.<init>(JmsSession.java:143)
>> > at
>> org.apache.qpid.jms.JmsConnection.createSession(JmsConnection.java:299)
>> > at org.myorg.mymessaging.PostOffice.buildSession(PostOffice.java:149)
>> > ...
>> > Caused by: java.io.IOException: java.lang.NullPointerException
>> > at
>> >
>> org.apache.qpid.jms.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
>> > at
>> >
>> org.apache.qpid.jms.provider.amqp.AmqpTransactionCoordinator.processDeliveryUpdates(AmqpTransactionCoordinator.java:117)
>> > at
>> >
>> org.apache.qpid.jms.provider.amqp.AmqpProvider.processUpdates(AmqpProvider.java:928)
>> > at
>> >
>> org.apache.qpid.jms.provider.amqp.AmqpProvider.access$1800(AmqpProvider.java:93)
>> > at
>> >
>> org.apache.qpid.jms.provider.amqp.AmqpProvider$18.run(AmqpProvider.java:790)
>> > at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
>> >
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>> > at
>> >
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > at java.lang.Thread.run(Thread.java:748)
>> > Caused by: java.lang.NullPointerException
>> > at
>> >
>> org.apache.qpid.jms.provider.amqp.AmqpTransactionCoordinator.processDeliveryUpdates(AmqpTransactionCoordinator.java:93)
>> > ... 10 more
>> >
>>
>> I've raised https://issues.apache.org/jira/browse/QPIDJMS-307 to
>> investigate this and improve the handling. Though it did fail to
>> create the session, as it should have given Gordon's earlier
>> explanation, it didn't do it very well so there is definitely a client
>> bug there. Possibly also a router one if my hunch of what happened is
>> accurate.
>>
>> > are transactions expected to work with qpid dispatch router? on that
>> note
>> > do you think i could also get XA transactions working? would i need to
>> use
>> > a link route? i dont know for sure but it sounds like a link route is a
>> > little more low level that an autolink.
>> >
>> > thank you
>> >
>> > On Tue, Aug 1, 2017 at 6:47 AM Ted Ross <[hidden email]> wrote:
>> >
>> >> Dan,
>> >>
>> >> There's one issue with your configuration which doesn't affect the load
>> >> balancing but will cause problems with receiving messages from the
>> >> brokers.  In the address.prefix, you use "foo.#".  This is a pure
>> prefix
>> >> and it should simply be "foo".  The wildcards are coming in the next
>> >> release but are not implemented in the code you are using.
>> >>
>> >> Regarding your actual question.  I assume that you are testing this
>> >> configuration under light load (i.e. sending one message at a time).
>> >>
>> >> The way that the balancing works is that it will route to the consumer
>> >> (broker) with the fewest outstanding deliveries + inter-router cost.
>> This
>> >> means that it will favor the local broker over the remote one if there
>> are
>> >> no in-flight deliveries.  The default (and minimum) cost for an
>> >> inter-router connection is 1.  You can set it to a higher value in the
>> >> listener or connector.
>> >>
>> >> If you are sending one-at-a-time synchronous sends, they will always
>> go to
>> >> the local broker because the broker's zero outstanding deliveries will
>> >> always be less than the inter-router cost of 1.  If you send multiple
>> >> deliveries asynchronously, you will see them being distributed to both
>> >> brokers in the network.  You can make the local-affinity stronger by
>> >> increasing the inter-router cost.
>> >>
>> >> -Ted
>> >>
>> >> On Tue, Aug 1, 2017 at 3:47 AM, Dan Langford <[hidden email]>
>> >> wrote:
>> >>
>> >> > Last week I had a thread with lots of little questions around
>> Dispatch
>> >> > Routers. Ted Ross has been awesome to answer most of those. As a
>> result I
>> >> > feel like I have my QDR config shaping up a bit better. HOWEVER with
>> some
>> >> > more very focused questions I thought it would be best to start a new
>> >> > thread. The problem I am seeing is that the routers are not
>> distributing
>> >> > the message load across other brokers on the network. Here is a
>> little
>> >> > diagram:
>> >> >
>> >> >
>> >> > RouterA-03 ----> ArtemisBrokerA
>> >> >  |
>> >> >  |
>> >> >  |
>> >> > \/
>> >> > RouterB-05 ----> ArtemisBrokerB
>> >> >
>> >> >
>> >> > *NOTE: i am currently using Qpid Dispatch Router from a RHEL repo. v
>> >> 0.7.0*
>> >> >
>> >> >
>> >> > Connections from clients come in through an F5 VIP which forwards
>> those
>> >> > connections to either host L-03-A or L-05-B. Each of those hosts
>> have a
>> >> > Qpid Dispatch Router installed in front of an Artemis broker.
>> dispatch
>> >> > router on L-05-B is listening on an additional port that dispatch
>> router
>> >> > L-03-A connects to for inter-router communication.
>> >> >
>> >> > If I go around my F5 VIP so i know I am connecting straight to L-05-B
>> >> and I
>> >> > send any number of messages into the router there all of those
>> messages
>> >> end
>> >> > up in ArtemisBrokerB. I was hoping that some would go to BrokerB and
>> some
>> >> > would go via RouterA over to BrokerA. Now when BrokerB is taken down
>> >> > CURRENTLY the messages sent to Router L-05-B ARE routed through
>> Router
>> >> > L-03-A and then out to ArtemisBrokerB. (Currently receiving messages
>> pull
>> >> > them in from both hosts so my questions only lies in message
>> production
>> >> at
>> >> > the moment.)
>> >> >
>> >> > Do you agree that with this configuration you would expect messages
>> to be
>> >> > load balanced between the two routes? I would like to paste in some
>> of
>> >> our
>> >> > config and results from running qdstat. If you observe something
>> that is
>> >> > misconfigured and are able to highlight it i would be very
>> appreciative.
>> >> > First the config files for QDR instances. I tried my best to scrub
>> them
>> >> of
>> >> > specific IPs and hostnames.
>> >> >
>> >> > *L-03-A qrouterd.conf*
>> >> >
>> >> >
>> >> > *router {*
>> >> >
>> >> > *    mode: interior*
>> >> >
>> >> > *    id: Router.A*
>> >> >
>> >> > *}*
>> >> >
>> >> > *log {*
>> >> >
>> >> > *    module: DEFAULT*
>> >> >
>> >> > *    enable: debug+*
>> >> >
>> >> > *    timestamp: yes*
>> >> >
>> >> > *}*
>> >> >
>> >> > *sslProfile {*
>> >> >
>> >> > *    name: my-ssl*
>> >> >
>> >> > *    certFile: /opt/org/my-ssl-info.pem*
>> >> >
>> >> > *    keyFile: /opt/org/my-ssl-info.pem*
>> >> >
>> >> > *    password: hellokitty42*
>> >> >
>> >> > *}*
>> >> >
>> >> > *listener {*
>> >> >
>> >> > *    role: normal*
>> >> >
>> >> > *    host: 0.0.0.0*
>> >> >
>> >> > *    port: 5671*
>> >> >
>> >> > *    authenticatePeer: no*
>> >> >
>> >> > *    saslMechanisms: ANONYMOUS*
>> >> >
>> >> > *    sslProfile: my-ssl*
>> >> >
>> >> > *}*
>> >> >
>> >> > *connector {*
>> >> >
>> >> > *    name: local-artemis*
>> >> >
>> >> > *    role: route-container*
>> >> >
>> >> > *    host: L-03-A*
>> >> >
>> >> > *    port: 61616*
>> >> >
>> >> > *    saslMechanisms: ANONYMOUS*
>> >> >
>> >> > *}*
>> >> >
>> >> > *connector {*
>> >> >
>> >> > *    name: routerb*
>> >> >
>> >> > *    role: inter-router*
>> >> >
>> >> > *    host: L-05-B*
>> >> >
>> >> > *    port: 6671*
>> >> >
>> >> > *    saslMechanisms: ANONYMOUS*
>> >> >
>> >> > *}*
>> >> >
>> >> > *address {*
>> >> >
>> >> > *    prefix: foo.#*
>> >> >
>> >> > *    waypoint: yes*
>> >> >
>> >> > *    distribution: balanced*
>> >> >
>> >> > *}*
>> >> >
>> >> > *autoLink {*
>> >> >
>> >> > *    addr: foo.bar*
>> >> >
>> >> > *    dir: in*
>> >> >
>> >> > *    connection: local-artemis*
>> >> >
>> >> > *}*
>> >> >
>> >> > *autoLink {*
>> >> >
>> >> > *    addr: foo.bar*
>> >> >
>> >> > *    dir: out*
>> >> >
>> >> > *    connection: local-artemis*
>> >> >
>> >> > *}*
>> >> >
>> >> >
>> >> > *L-05-B qrouterd.conf*
>> >> >
>> >> > *router {*
>> >> >
>> >> > *    mode: interior*
>> >> >
>> >> > *    id: Router.B*
>> >> >
>> >> > *}*
>> >> >
>> >> > *log {*
>> >> >
>> >> > *    module: DEFAULT*
>> >> >
>> >> > *    enable: debug+*
>> >> >
>> >> > *    timestamp: yes*
>> >> >
>> >> > *}*
>> >> >
>> >> > *sslProfile {*
>> >> >
>> >> > *    name: my-ssl*
>> >> >
>> >> > *    certFile: /opt/org/my-ssl-info.pem*
>> >> >
>> >> > *    keyFile: /opt/org/my-ssl-info.pem*
>> >> >
>> >> > *    password: hellokitty42*
>> >> >
>> >> > *}*
>> >> >
>> >> > *listener {*
>> >> >
>> >> > *    role: normal*
>> >> >
>> >> > *    host: 0.0.0.0*
>> >> >
>> >> > *    port: 5671*
>> >> >
>> >> > *    authenticatePeer: no*
>> >> >
>> >> > *    saslMechanisms: ANONYMOUS*
>> >> >
>> >> > *    sslProfile: my-ssl*
>> >> >
>> >> > *}*
>> >> >
>> >> > *listener {*
>> >> >
>> >> > *    role: inter-router*
>> >> >
>> >> > *    host: 0.0.0.0*
>> >> >
>> >> > *    port: 6671*
>> >> >
>> >> > *    authenticatePeer: no*
>> >> >
>> >> > *    saslMechanisms: ANONYMOUS*
>> >> >
>> >> > *    sslProfile: my-ssl*
>> >> >
>> >> > *}*
>> >> >
>> >> > *connector {*
>> >> >
>> >> > *    name: local-artemis*
>> >> >
>> >> > *    role: route-container*
>> >> >
>> >> > *    host: L-05-B*
>> >> >
>> >> > *    port: 61616*
>> >> >
>> >> > *    saslMechanisms: ANONYMOUS*
>> >> >
>> >> > *}*
>> >> >
>> >> > *address {*
>> >> >
>> >> > *    prefix: foo.#*
>> >> >
>> >> > *    waypoint: yes*
>> >> >
>> >> > *    distribution: balanced*
>> >> >
>> >> > *}*
>> >> >
>> >> > *autoLink {*
>> >> >
>> >> > *    addr: foo.bar*
>> >> >
>> >> > *    dir: in*
>> >> >
>> >> > *    connection: local-artemis*
>> >> >
>> >> > *}*
>> >> >
>> >> > *autoLink {*
>> >> >
>> >> > *    addr: foo.bar*
>> >> >
>> >> > *    dir: out*
>> >> >
>> >> > *    connection: local-artemis*
>> >> >
>> >> > *}*
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > *L-03-A ~]$ qdstat -b:5671 -g*
>> >> >
>> >> > *Router Statistics*
>> >> >
>> >> > *  attr       value*
>> >> >
>> >> > *  =====================*
>> >> >
>> >> > *  Mode       interior*
>> >> >
>> >> > *  Area       0*
>> >> >
>> >> > *  Router Id  Router.A*
>> >> >
>> >> >
>> >> > *L-05-B ~]$ qdstat -b:5671 -g*
>> >> >
>> >> > *Router Statistics*
>> >> >
>> >> > *  attr       value*
>> >> >
>> >> > *  =====================*
>> >> >
>> >> > *  Mode       interior*
>> >> >
>> >> > *  Area       0*
>> >> >
>> >> > *  Router Id  Router.B*
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > *L-03-A ~]$ qdstat -b:5671 -c*
>> >> >
>> >> > *Connections*
>> >> >
>> >> > *  Id      host             container
>>  role
>> >> >         dir  security     authentication*
>> >> >
>> >> > *
>> >> > ============================================================
>> >> > ======================================================*
>> >> >
>> >> > *  14      L-03-A:61616     L-03-A
>> >> > route-container  out  no-security  anonymous-user*
>> >> >
>> >> > *  107132  L-05-B:6671      Router.B
>> >> > inter-router     out  no-security  anonymous-user*
>> >> >
>> >> > *  107267  127.0.0.1:37916 <http://127.0.0.1:37916>
>> >> > 180851ba-7ce4-44d0-a937-97de16e781c4  normal           in
>>  no-security
>> >> > anonymous-user*
>> >> >
>> >> >
>> >> > *L-05-B ~]$ qdstat -b:5671 -c*
>> >> >
>> >> > *Connections*
>> >> >
>> >> > *  Id   host                container
>>  role
>> >> >         dir  security     authentication*
>> >> >
>> >> > *
>> >> > ============================================================
>> >> > ======================================================*
>> >> >
>> >> > *  1    L-05-B:61616        L-05-B
>> >> > route-container  out  no-security  anonymous-user*
>> >> >
>> >> > *  2    10.<ip>.32:57182  Router.A
>> >> > inter-router     in   no-security  anonymous-user*
>> >> >
>> >> > *  143  127.0.0.1:48610 <http://127.0.0.1:48610>
>> >> > 93e6b3e3-443e-43d9-9c2c-0d648266ec19  normal           in
>>  no-security
>> >> > anonymous-user*
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > *L-03-A ~]$ qdstat -b:5671 -n*
>> >> >
>> >> > *Routers in the Network*
>> >> >
>> >> > *  router-id  next-hop  link*
>> >> >
>> >> > *  ===========================*
>> >> >
>> >> > *  Router.A   (self)    -*
>> >> >
>> >> > *  Router.B   -         0*
>> >> >
>> >> >
>> >> > *L-05-B ~]$ qdstat -b:5671 -n*
>> >> >
>> >> > *Routers in the Network*
>> >> >
>> >> > *  router-id  next-hop  link*
>> >> >
>> >> > *  ===========================*
>> >> >
>> >> > *  Router.A   -         0*
>> >> >
>> >> > *  Router.B   (self)    -*
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > *L-03-A ~]$ qdstat -b:5671 --linkroutes*
>> >> >
>> >> > *Link Routes*
>> >> >
>> >> >
>> >> > *L-05-B ~]$ qdstat -b:5671 --linkroutes*
>> >> >
>> >> > *Link Routes*
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > *L-03-A ~]$ qdstat -b:5671 --autolinks*
>> >> >
>> >> > *AutoLinks*
>> >> >
>> >> > *  addr     dir  phs  link  status  lastErr*
>> >> >
>> >> > *  ==========================================*
>> >> >
>> >> > *  foo.bar  in   1    8     active*
>> >> >
>> >> > *  foo.bar  out  0    9     active*
>> >> >
>> >> >
>> >> > *L-05-B ~]$ qdstat -b:5671 --autolinks*
>> >> >
>> >> > *AutoLinks*
>> >> >
>> >> > *  addr     dir  phs  link  status  lastErr*
>> >> >
>> >> > *  ==========================================*
>> >> >
>> >> > *  foo.bar  in   1    4     active*
>> >> >
>> >> > *  foo.bar  out  0    5     active*
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > *L-03-A ~]$ qdstat -b:5671 -l*
>> >> >
>> >> > *Router Links*
>> >> >
>> >> > *  type            dir  conn id  id  peer  class   addr
>> >> > phs  cap  undel  unsett  del  presett  acc  rej  rel  mod  admin
>> oper*
>> >> >
>> >> > *
>> >> > ============================================================
>> >> > ============================================================
>> >> > ========================*
>> >> >
>> >> > *  endpoint        in   14       8         mobile  foo.bar
>> >>  1
>> >> >   250  0      0       80   0        80   0    0    0    enabled  up*
>> >> >
>> >> > *  endpoint        out  14       9         mobile  foo.bar
>> >>  0
>> >> >   250  0      0       40   0        40   0    0    0    enabled  up*
>> >> >
>> >> > *  router-control  in   107132   15
>> >> >   250  0      0       524  524      0    0    0    0    enabled  up*
>> >> >
>> >> > *  router-control  out  107132   16        local   qdhello
>> >> >   250  0      0       527  527      0    0    0    0    enabled  up*
>> >> >
>> >> > *  inter-router    in   107132   17
>> >> >   250  0      0       0    0        0    0    0    0    enabled  up*
>> >> >
>> >> > *  inter-router    out  107132   18
>> >> >   250  0      0       0    0        0    0    0    0    enabled  up*
>> >> >
>> >> > *  endpoint        in   107336   33        mobile  $management
>> >>  0
>> >> >   250  0      0       1    0        1    0    0    0    enabled  up*
>> >> >
>> >> > *  endpoint        out  107336   34        local
>>  temp.liHSN39azHuUPkg
>> >> >   250  0      0       0    0        0    0    0    0    enabled  up*
>> >> >
>> >> >
>> >> > *L-05-B ~]$ qdstat -b:5671 -l*
>> >> >
>> >> > *Router Links*
>> >> >
>> >> > *  type            dir  conn id  id  peer  class   addr
>> >> > phs  cap  undel  unsett  del  presett  acc  rej  rel  mod  admin
>> oper*
>> >> >
>> >> > *
>> >> > ============================================================
>> >> > ============================================================
>> >> > ========================*
>> >> >
>> >> > *  endpoint        in   1        4         mobile  foo.bar
>> >>  1
>> >> >   250  40     0       0    0        0    0    0    0    enabled  up*
>> >> >
>> >> > *  endpoint        out  1        5         mobile  foo.bar
>> >>  0
>> >> >   250  0      0       0    0        0    0    0    0    enabled  up*
>> >> >
>> >> > *  router-control  out  2        6         local   qdhello
>> >> >   250  0      0       524  524      0    0    0    0    enabled  up*
>> >> >
>> >> > *  router-control  in   2        7
>> >> >   250  0      0       527  527      0    0    0    0    enabled  up*
>> >> >
>> >> > *  inter-router    out  2        8
>> >> >   250  0      0       0    0        0    0    0    0    enabled  up*
>> >> >
>> >> > *  inter-router    in   2        9
>> >> >   250  0      0       0    0        0    0    0    0    enabled  up*
>> >> >
>> >> > *  endpoint        in   212      24        mobile  $management
>> >>  0
>> >> >   250  0      0       1    0        1    0    0    0    enabled  up*
>> >> >
>> >> > *  endpoint        out  212      25        local
>>  temp.BJPESwMGFo+7qiy
>> >> >   250  0      0       0    0        0    0    0    0    enabled  up*
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > *L-03-A ~]$ qdstat -b:5671 -a*
>> >> >
>> >> > *Router Addresses*
>> >> >
>> >> > *  class   addr                   phs  distrib    in-proc  local
>> remote
>> >> > cntnr  in  out  thru   to-proc  from-proc*
>> >> >
>> >> > *
>> >> > ============================================================
>> >> > ======================================================*
>> >> >
>> >> > *  local   $_management_internal       closest    1        0      0
>> >> > 0      0   0    0      6        6*
>> >> >
>> >> > *  local   $displayname                closest    1        0      0
>> >> > 0      0   0    0      0        0*
>> >> >
>> >> > *  mobile  $management            0    closest    1        0      0
>> >> > 0      9   0    0      9        0*
>> >> >
>> >> > *  local   $management                 closest    1        0      0
>> >> > 0      0   0    0      0        0*
>> >> >
>> >> > *  router  Router.B                    closest    0        0      1
>> >> > 0      0   0    19     0        19*
>> >> >
>> >> > *  mobile  foo.bar                1    balanced   0        0      0
>> >> > 0      80  520  0      0        0*
>> >> >
>> >> > *  mobile  foo.bar                0    balanced   0        1      1
>> >> > 0      0   40   0      0        0*
>> >> >
>> >> > *  local   qdhello                     flood      1        1      0
>> >> > 0      0   0    0      269,611  269,622*
>> >> >
>> >> > *  local   qdrouter                    flood      1        0      0
>> >> > 0      0   0    0      12       0*
>> >> >
>> >> > *  topo    qdrouter                    flood      1        0      1
>> >> > 0      0   0    8,992  8,991    8,996*
>> >> >
>> >> > *  local   qdrouter.ma <http://qdrouter.ma>
>>  multicast  1
>> >> >     0      0       0      0   0    0      4        0*
>> >> >
>> >> > *  topo    qdrouter.ma <http://qdrouter.ma>
>>  multicast  1
>> >> >     0      1       0      0   0    3      0        3*
>> >> >
>> >> > *  local   temp.u6NMF1AFV0JpdBi        closest    0        1      0
>> >> > 0      0   0    0      0        0*
>> >> >
>> >> >
>> >> > *L-05-B ~]$ qdstat -b:5671 -a*
>> >> >
>> >> > *Router Addresses*
>> >> >
>> >> > *  class   addr                   phs  distrib    in-proc  local
>> remote
>> >> > cntnr  in  out  thru  to-proc  from-proc*
>> >> >
>> >> > *
>> >> > ============================================================
>> >> > =====================================================*
>> >> >
>> >> > *  local   $_management_internal       closest    1        0      0
>> >> > 0      0   0    0     6        6*
>> >> >
>> >> > *  local   $displayname                closest    1        0      0
>> >> > 0      0   0    0     0        0*
>> >> >
>> >> > *  mobile  $management            0    closest    1        0      0
>> >> > 0      9   0    0     9        0*
>> >> >
>> >> > *  local   $management                 closest    1        0      0
>> >> > 0      0   0    0     0        0*
>> >> >
>> >> > *  router  Router.A                    closest    0        0      1
>> >> > 0      0   0    6     0        6*
>> >> >
>> >> > *  mobile  foo.bar                1    balanced   0        0      0
>> >> > 0      0   0    0     0        0*
>> >> >
>> >> > *  mobile  foo.bar                0    balanced   0        1      1
>> >> > 0      0   0    0     0        0*
>> >> >
>> >> > *  local   qdhello                     flood      1        1      0
>> >> > 0      0   0    0     570      571*
>> >> >
>> >> > *  local   qdrouter                    flood      1        0      0
>> >> > 0      0   0    0     4        0*
>> >> >
>> >> > *  topo    qdrouter                    flood      1        0      1
>> >> > 0      0   0    20    21       21*
>> >> >
>> >> > *  local   qdrouter.ma <http://qdrouter.ma>
>>  multicast  1
>> >> >     0      0       0      0   0    0     4        0*
>> >> >
>> >> > *  topo    qdrouter.ma <http://qdrouter.ma>
>>  multicast  1
>> >> >     0      1       0      0   0    0     0        1*
>> >> >
>> >> > *  local   temp.xwHdS0zzku0s3lR        closest    0        1      0
>> >> > 0      0   0    0     0        0*
>> >> >
>> >> >
>> >> >
>> >> > is there something off in my configuration? or in my understand of
>> QDR
>> >> > netwokin just not up to snuff? should i care a little bit less about
>> load
>> >> > balancing because the VIP will round robin new connections out to
>> >> different
>> >> > codes?
>> >> >
>> >> > Thank you so much for your time and insight
>> >> >
>> >>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dispatch Router load balancing config questions

Gordon Sim
On 09/08/17 16:59, Dan Langford wrote:

> back from vacation now. i upgraded to 0.8.0 and local/session transactions
> look like they are working as expected.
>
> *Question about JMS filters / message selectors*: when i connect to my
> artemis broker with the artemis client JMS filters work. when i connect to
> my artemis broker with qpid-jms-client then jms filters work. HOWEVER when
> i connect to THROUGH the dispatch routers with the qpid-jms-client the jms
> filters (configured in the client code) seem to be ignored or lost. i
> receive messages as if no filter was supplied. does QPID dispatch router
> support filters configured programmatically via the client? with my
> aforementioned configuration
> https://gist.github.com/danlangford/4944dcc6c0d2703ffb8555603ed27340 would
> you expect jms filters to work?

Filters/selectors will not work in conjunction with message routing,
because the broker is unaware of the actual consumers. To allow
filters/selectors (or other link specific features), you would need to
use link routing. However it is the message routing behaviour that
provides the simple load balancing for queues.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dispatch Router load balancing config questions

Dan Langford
thank you. as i configure this Messaging service i am now for fully
understanding why Azure Service Bus and RabbitMQ do not support
Transactions and Message Selectors :-)

Question about prefetch/acquired messages: i am finding that QDR will leave
some (up to 250 quite frequently) messages as acquired on the broker. or in
artemis terms "delivering". how would I go about configuring QDRs Prefetch
to keep it low? (considering the above configuration which is a simple
message routing config for load balancing / sharding behavior)

On Wed, Aug 9, 2017 at 10:06 AM Gordon Sim <[hidden email]> wrote:

> On 09/08/17 16:59, Dan Langford wrote:
> > back from vacation now. i upgraded to 0.8.0 and local/session
> transactions
> > look like they are working as expected.
> >
> > *Question about JMS filters / message selectors*: when i connect to my
> > artemis broker with the artemis client JMS filters work. when i connect
> to
> > my artemis broker with qpid-jms-client then jms filters work. HOWEVER
> when
> > i connect to THROUGH the dispatch routers with the qpid-jms-client the
> jms
> > filters (configured in the client code) seem to be ignored or lost. i
> > receive messages as if no filter was supplied. does QPID dispatch router
> > support filters configured programmatically via the client? with my
> > aforementioned configuration
> > https://gist.github.com/danlangford/4944dcc6c0d2703ffb8555603ed27340
> would
> > you expect jms filters to work?
>
> Filters/selectors will not work in conjunction with message routing,
> because the broker is unaware of the actual consumers. To allow
> filters/selectors (or other link specific features), you would need to
> use link routing. However it is the message routing behaviour that
> provides the simple load balancing for queues.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dispatch Router load balancing config questions

Ganesh Murthy
Try setting the linkCapacity attribute of the listener. It is by default
set to 250.

The linkCapacity is the number of messages that can be in-flight
concurrently for each link.

Thanks.

On Thu, Aug 10, 2017 at 5:59 PM, Dan Langford <[hidden email]> wrote:

> thank you. as i configure this Messaging service i am now for fully
> understanding why Azure Service Bus and RabbitMQ do not support
> Transactions and Message Selectors :-)
>
> Question about prefetch/acquired messages: i am finding that QDR will leave
> some (up to 250 quite frequently) messages as acquired on the broker. or in
> artemis terms "delivering". how would I go about configuring QDRs Prefetch
> to keep it low? (considering the above configuration which is a simple
> message routing config for load balancing / sharding behavior)
>
> On Wed, Aug 9, 2017 at 10:06 AM Gordon Sim <[hidden email]> wrote:
>
> > On 09/08/17 16:59, Dan Langford wrote:
> > > back from vacation now. i upgraded to 0.8.0 and local/session
> > transactions
> > > look like they are working as expected.
> > >
> > > *Question about JMS filters / message selectors*: when i connect to my
> > > artemis broker with the artemis client JMS filters work. when i connect
> > to
> > > my artemis broker with qpid-jms-client then jms filters work. HOWEVER
> > when
> > > i connect to THROUGH the dispatch routers with the qpid-jms-client the
> > jms
> > > filters (configured in the client code) seem to be ignored or lost. i
> > > receive messages as if no filter was supplied. does QPID dispatch
> router
> > > support filters configured programmatically via the client? with my
> > > aforementioned configuration
> > > https://gist.github.com/danlangford/4944dcc6c0d2703ffb8555603ed27340
> > would
> > > you expect jms filters to work?
> >
> > Filters/selectors will not work in conjunction with message routing,
> > because the broker is unaware of the actual consumers. To allow
> > filters/selectors (or other link specific features), you would need to
> > use link routing. However it is the message routing behaviour that
> > provides the simple load balancing for queues.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
> >
>
Loading...