SSL between c++ brokers?

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

SSL between c++ brokers?

ParkiratBagga
I have 2 c++ brokers in 2 machines.

I have setup CA and made the certificates for both the c++ brokers. Also, I have added the parameters like transport, ssl-cert-db, ssl-port, and ssl-password-file to the config file.

While setting up the queue route between 2 servers:

1. If, I give the destination:<ssl port> source:<ssl port>, I am not able to set the route due to timeout.
2. Therefore, I am routing messages using queue route with "destination:<qpid port> source:<qpid port> and transport option as ssl".

Is this correct?

Also, when message get routed, I don't see, SSL is being used anywhere in between.

1. Have I configured it correctly?
2. Did I missed something?
3. Can you help me with pointers, how to setup ssl between brokers?

Regards,
Parkirat Singh Bagga.
Reply | Threaded
Open this post in threaded view
|

Re: SSL between c++ brokers?

Gordon Sim
On 09/10/2012 08:22 PM, ParkiratBagga wrote:

> I have 2 c++ brokers in 2 machines.
>
> I have setup CA and made the certificates for both the c++ brokers. Also, I
> have added the parameters like transport, ssl-cert-db, ssl-port, and
> ssl-password-file to the config file.
>
> While setting up the queue route between 2 servers:
>
> 1. If, I give the destination:<ssl port> source:<ssl port>, I am not able to
> set the route due to timeout.
> 2. Therefore, I am routing messages using queue route with
> "destination:<qpid port> source:<qpid port> and transport option as *ssl*".
>
> Is this correct?

In 2., is the port the ssl port? If so that is right. You need to
specify the ssl port _and_ specify ssl as the transport.

> Also, when message get routed, I don't see, SSL is being used anywhere in
> between.

So messages are being routed correctly? But you are not seeing any SSL
traffic (I assume through some network monitoring tool)?

Do you already have a non-ssl route in place. If so you may need to
remove that to ensure that messages are routed over SSL.

> 1. Have I configured it correctly?
> 2. Did I missed something?
> 3. Can you help me with pointers, how to setup ssl between brokers?

First step I would recommend is to ensure a regular client can connect
to the 'remote' broker using the 'local' brokers certificate database.
That lets you test the configuration in a slightly simpler fashion.

The other suggestion is to look at the logs and see if there are any errors.

There is a test script that is run as part of make check, that sets up
SSL based federation and it may be useful as an example (though its not
written as an example specifically):

http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/src/tests/sasl_fed_ex?view=markup


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SSL between c++ brokers?

ParkiratBagga
Hi,

I have succeeded till client to broker ssl.

I have 2 c++ brokers and my client is able to send messages to both the servers using ssl. This means both the servers are configured correctly.

Now I am trying to add a queue route between the 2 servers, using the command:
        qpid-route queue add <destination>:<ssl-port> <source>:<ssl-port> exchange queue

I am getting timeout while doing this. I am trying to make this route from the source side.

Error at source broker side:
qpid-route queue add <destination>:<ssl-port> <source>:<ssl-port> exchange queue
Failed: Timeout - Waiting for connection to be established with broker

Error in logs at destination broker side:
2012-09-12 08:00:58 error Error reading socket: Encountered end of file [-5938]
2012-09-12 08:00:58 debug DISCONNECTED [<destination>:5674-<source>:58871]

Please help in this as well.

Regards,
Parkirat Singh Bagga.

 
Reply | Threaded
Open this post in threaded view
|

Re: SSL between c++ brokers?

ParkiratBagga
Also I am using qpid (qpidc) 0.14 version of Qpid Server.
Reply | Threaded
Open this post in threaded view
|

Re: SSL between c++ brokers?

Gordon Sim
In reply to this post by ParkiratBagga
On 09/12/2012 01:38 PM, ParkiratBagga wrote:

> Hi,
>
> I have succeeded till client to broker ssl.
>
> I have 2 c++ brokers and my client is able to send messages to both the
> servers using ssl. This means both the servers are configured correctly.
>
> Now I am trying to add a queue route between the 2 servers, using the
> command:
>          qpid-route queue add <destination>:<ssl-port> <source>:<ssl-port>
> exchange queue
>
> I am getting timeout while doing this. I am trying to make this route from
> the source side.
>
> Error at source broker side:
> qpid-route queue add <destination>:<ssl-port> <source>:<ssl-port> exchange
> queue
> Failed: Timeout - Waiting for connection to be established with broker
>

I think that is because qpid-route is itself not using SSL. Try
connecting over plain TCP to the 'destination' broker, e.g.:

qpid-route queue add <destination>:<non-ssl-port> <source>:<ssl-port>
exchange queue

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SSL between c++ brokers?

ParkiratBagga
Thanks Gordon.

Plain TCP works fine. My msgs are being routed using tcp.
But my requirement is to do SSL between the 2 C++ brokers, either using qpid-route or something else.
In that case qpid route should use ssl.

Is it possible to do ssl over qpid-route or with some external means?

Regards,
Parkirat Singh Bagga.

Reply | Threaded
Open this post in threaded view
|

Re: SSL between c++ brokers?

Gordon Sim
On 09/12/2012 02:49 PM, ParkiratBagga wrote:
> Thanks Gordon.
>
> Plain TCP works fine. My msgs are being routed using tcp.
> But my requirement is to do SSL between the 2 C++ brokers, either using
> qpid-route or something else.
> In that case qpid route should use ssl.

Just to clarify, the qpid-route tool connects to the destination broker
and asks it to connect to the source broker.

The fact that the connection between qpid-tool and the destination
broker is not using SSL does not mean that the connection between the
destination broker and the source broker cannot be SSL.

My suggested invocation of qpid-route modified your original only to
allow qpid-tool to connect to the destination broker over plain TCP. The
connection between the two brokers would still use SSL.

Now, you can of course SSL enable the connection from qpid-route to the
destination broker if you want, but that has no impact on the connection
over which the messages are transferred (which is why I suggested the
simpler approach, at least to verify the route being established using
SSL). To use SSL even between qpid-route and the destination broker
(purely for the purposes of telling that broker to connect to the other
one) try 'amqps:' in front of your original destination broker address.



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SSL between c++ brokers?

ParkiratBagga
Thank You Gordon for the explanation.

I tried the following command:
qpid-route queue add <destination>:<non-ssl-port> <source>:<ssl-port> exchange queue

It did not gave the Timeout this time. Also, it tried to create the route but the route which is being created is always in Connecting State and it is not coming to Operational State.

Also, if route get created properly, then a consumer connection is established on the source queue, which is not coming.

Below is the link status:

qpid-route link list
Host                Port    Transport Durable  State             Last Error
=========================================================
<source host> 5674         ssl          Y     Connecting


Now what should I do further to resolve this?

Regards,
Parkirat Singh Bagga.
Reply | Threaded
Open this post in threaded view
|

Re: SSL between c++ brokers?

Gordon Sim
On 09/12/2012 09:19 PM, ParkiratBagga wrote:

> Thank You Gordon for the explanation.
>
> I tried the following command:
> qpid-route queue add <destination>:<non-ssl-port> <source>:<ssl-port>
> exchange queue
>
> It did not gave the Timeout this time. Also, it tried to create the route
> but the route which is being created is always in *Connecting State* and it
> is *not coming to Operational State*.
>
> Also, if route get created properly, then a consumer connection is
> established on the source queue, which is not coming.
>
> Below is the link status:
>
> *qpid-route link list
> Host                Port    Transport Durable  State             Last Error
> =========================================================
> <source host> 5674         ssl          Y     Connecting*
>
> Now what should I do further to resolve this?

Are there any errors in either broker's logs?

How have you setup the certificates for the two brokers? Do they each
have their own cert? Is there a common CA or how is trust established?


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SSL between c++ brokers?

ParkiratBagga
Hi Gordon,

I saw that there are no errors in the logs.

Also both brokers have there own certificates and there is one CA, which I made on one broker and copied to the other broker as well. Moreover, both broker certs are signed by that CA only.

Mainly, I followed the below document for doing the SSL setup:
http://rajith.2rlabs.com/2010/03/01/apache-qpid-securing-connections-with-ssl/

Also, I am attaching the log which got created while creating the route and the steps, that I used for creating the certs and CA.

mylog.log

SSLsetup.pdf

Regards,
Parkirat Singh Bagga.
Reply | Threaded
Open this post in threaded view
|

Re: SSL between c++ brokers?

Gordon Sim
On 09/13/2012 12:40 PM, ParkiratBagga wrote:

> Hi Gordon,
>
> I saw that there are no errors in the logs.
>
> Also both brokers have there own certificates and there is one CA, which I
> made on one broker and copied to the other broker as well. Moreover, both
> broker certs are signed by that CA only.
>
> Mainly, I followed the below document for doing the SSL setup:
> http://rajith.2rlabs.com/2010/03/01/apache-qpid-securing-connections-with-ssl/
>
> Also, I am attaching the log which got created while creating the route and
> the steps, that I used for creating the certs and CA.
>
> http://qpid.2158936.n2.nabble.com/file/n7582028/mylog.log mylog.log
>
> http://qpid.2158936.n2.nabble.com/file/n7582028/SSLsetup.pdf SSLsetup.pdf

Do you give qpid-route the fully qualified name for the source broker,
as used when creating the certificate?

The log isn't very helpful. Could you restart both the servers and
specify on the destination broker:

   --log-enable info+ --log-enable notice+:management --log-enable
debug+:link

and on the source broker:

   --log-enable info+ --log-enable notice+:management --log-enable
debug+:link --log-enable trace+:amqp_0_10

Then try qpid-route again (if your queue and exchange are not durable
you'll need to recreate those after restarting).

If you can then send me those logs I'll see if I can spot anything...

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SSL between c++ brokers?

ParkiratBagga
Thanks Gordon.

I am using fully qualified names which I used while creating the certs.

Also find the destination broker log as per your log settings:

destination_ssl.log

and the source broker logs:

source_ssl.log

Regards,
Parkirat Singh Bagga
Reply | Threaded
Open this post in threaded view
|

Re: SSL between c++ brokers?

Gordon Sim
On 09/13/2012 02:28 PM, ParkiratBagga wrote:

> Thanks Gordon.
>
> I am using fully qualified names which I used while creating the certs.
>
> Also find the destination broker log as per your log settings:
>
> http://qpid.2158936.n2.nabble.com/file/n7582034/destination_ssl.log
> destination_ssl.log
>
> and the source broker logs:
>
> http://qpid.2158936.n2.nabble.com/file/n7582034/source_ssl.log
> source_ssl.log

Ok, I have a theory. I think the problem is that you have a link between
the two brokers using plain tcp, and that is preventing the second link
from being correctly established.

Can you delete the first link, verify it has gone, then try the ssl link
again?

(And this time can I correct my mistake regarding the logging settings,
it should be --log-enable debug+:Link (not link)).

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SSL between c++ brokers?

ParkiratBagga
I am not using durable route now. So when I stop the server, it deletes the routes.

Though, I have repeated the same experiment with the new log settings, and also checked any existing routes.

Below are the logs when I tried to create the route:

================================================
Destination Side :

2012-09-13 11:23:11 info Queue "reply-ip-10-142-88-23.17751.1": Policy created: type=reject; maxCount=0; maxSize=104857600
2012-09-13 11:23:11 info Queue "reply-ip-10-142-88-23.17751.1": Flow limit created: flowStopCount=0, flowResumeCount=0, flowStopSize=83886080, flowResumeSize=73400320
2012-09-13 11:23:11 info Queue "topic-ip-10-142-88-23.17751.1": Policy created: type=ring; maxCount=0; maxSize=104857600
2012-09-13 11:23:11 info Queue "qmfc-v2-ip-10-142-88-23.17751.1": Policy created: type=reject; maxCount=0; maxSize=104857600
2012-09-13 11:23:11 info Queue "qmfc-v2-ip-10-142-88-23.17751.1": Flow limit created: flowStopCount=0, flowResumeCount=0, flowStopSize=83886080, flowResumeSize=73400320
2012-09-13 11:23:11 info Queue "qmfc-v2-ui-ip-10-142-88-23.17751.1": Policy created: type=ring; maxCount=0; maxSize=104857600
2012-09-13 11:23:11 info Queue "qmfc-v2-hb-ip-10-142-88-23.17751.1": Policy created: type=ring; maxCount=0; maxSize=104857600
2012-09-13 11:23:13 debug Link::bridge() request received
2012-09-13 11:23:13 debug Bridge declared ip-10-142-88-23: 5674 from myexchange-queue to myexchange ()
2012-09-13 11:23:13 debug Bridge created from myexchange-queue to myexchange
2012-09-13 11:23:14 debug Inter-broker link connecting to ip-10-142-88-23:5674


Source Side :

2012-09-13 11:23:14 trace SENT [10.142.88.23:5674-10.142.65.11:42053]: INIT(0-10)
2012-09-13 11:23:14 trace SENT [10.142.88.23:5674-10.142.65.11:42053]: Frame[BEbe; channel=0; {ConnectionStartBody: server-properties={qpid.federation_tag:V2:36:str16(eb83a02f-07a8-4a5b-8883-b8ddd3ed259b)}; mechanisms=str16{V2:9:str16(ANONYMOUS), V2:5:str16(PLAIN)}; locales=str16{V2:5:str16(en_US)}; }]
2012-09-13 11:23:14 trace RECV [10.142.88.23:5674-10.142.65.11:42053]: Frame[BEbe; channel=0; {ConnectionStartOkBody: client-properties={qpid.fed_link:F4:int32(1),qpid.federation_tag:V2:36:str16(8da2afe9-af66-4e24-af64-199aeb4be3b0)}; mechanism=ANONYMOUS; response=xxxxxx; locale=en_US; }]
2012-09-13 11:23:14 info Connection is a federation link
2012-09-13 11:23:14 trace SENT [10.142.88.23:5674-10.142.65.11:42053]: Frame[BEbe; channel=0; {ConnectionTuneBody: channel-max=32767; max-frame-size=65535; heartbeat-min=0; heartbeat-max=120; }]
2012-09-13 11:23:14 trace RECV [10.142.88.23:5674-10.142.65.11:42053]: Frame[BEbe; channel=0; {ConnectionTuneOkBody: channel-max=32767; max-frame-size=65535; heartbeat=120; }]
2012-09-13 11:23:14 trace RECV [10.142.88.23:5674-10.142.65.11:42053]: Frame[BEbe; channel=0; {ConnectionOpenBody: virtual-host=/; capabilities=void{}; insist=1; }]
2012-09-13 11:23:14 trace SENT [10.142.88.23:5674-10.142.65.11:42053]: Frame[BEbe; channel=0; {ConnectionOpenOkBody: known-hosts=str16{V2:26:str16(amqp:tcp:10.142.88.23:5672)}; }]
2012-09-13 11:25:14 trace SENT [10.142.88.23:5674-10.142.65.11:42053]: Frame[BEbe; channel=0; {ConnectionHeartbeatBody: }]
2012-09-13 11:25:14 trace RECV [10.142.88.23:5674-10.142.65.11:42053]: Frame[BEbe; channel=0; {ConnectionHeartbeatBody: }]
===============================================

Regards,
Parkirat Singh Bagga.
Reply | Threaded
Open this post in threaded view
|

Re: SSL between c++ brokers?

ParkiratBagga
Hi Gordon,

I followed this script, and I was able to make the qpid route connection to operational in intra qpid host.
http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/src/tests/sasl_fed_ex?view=markup

But, I noticed one thing, the TestHost which is the name of the certificate in this script is 127.0.0.1, which means intra-broker this scripts works fine.

As soon as you make that TestHost to your "machine hostname", the problem which I was concerned comes again, that the connection status does not become operational from connecting, which means the qpid-route ssl does not work properly inter-broker.

I have tweaked the above script, for your ready use, where problem is coming.

Place the below script in place of the original script in your broker machine and first run the sasl_test_setup.sh script and then sasl_fed_ex script.

sasl_fed_ex

It seems there is problem while doing ssl with hostname.

Regards,
Parkirat Singh Bagga.
Reply | Threaded
Open this post in threaded view
|

Re: SSL between c++ brokers?

Gordon Sim
On 09/14/2012 09:57 PM, ParkiratBagga wrote:

> Hi Gordon,
>
> I followed this script, and I was able to make the qpid route connection to
> operational in intra qpid host.
> http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/src/tests/sasl_fed_ex?view=markup
>
> But, I noticed one thing, the TestHost which is the name of the certificate
> in this script is 127.0.0.1, which means intra-broker this scripts works
> fine.
>
> As soon as you make that TestHost to your "machine hostname", the problem
> which I was concerned comes again, that the connection status does *not
> become operational* from *connecting*, which means the *qpid-route ssl does
> not work properly inter-broker*.
>
> I have tweaked the above script, for your ready use, where problem is
> coming.
>
> Place the below script in place of the original script in your broker
> machine and first run the sasl_test_setup.sh script and then sasl_fed_ex
> script.
>
> http://qpid.2158936.n2.nabble.com/file/n7582189/sasl_fed_ex sasl_fed_ex
>
> It seems there is problem while doing ssl with hostname.
Yes, you are right! That appears to be a regression that wasn't picked
up in testing. Not only does the test use an IP address, but it doesn't
actually verify the link becomes operation (or message flow as expected).

 From a quick scan it looks like
http://svn.apache.org/viewvc?view=revision&revision=1128067 might be the
point at which it broke.

I've raised a JIRA (https://issues.apache.org/jira/browse/QPID-4315) and
tried out a very simple patch (see attached). Are you able to apply that
and verify it works in your case? Thanks for your patience in tracking
down this bug!





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

SslSocket.patch (495 bytes) Download Attachment