Vortex Library Manual (C API)


On this manual you will find the following sections:

Section 1: An introduction to BEEP and Vortex Library

Section 2: Sending and receiving data with Vortex Library

Section 3: Doing Vortex Library to work like you expect: profiles, internal configuration and other useful information to adapt Vortex Library to your needs.

Section 4: Advanced topics

Section 5: Securing and authenticating your BEEP sessions: TLS and SASL profiles

1.1 Some concepts before starting to use Vortex

Before beginning, we have to review some definitions about the protocol that Vortex Library implements. This will help you to understand why Vortex Library uses some names to refer things such as: frames, channels, profiles, etc.

Vortex Library is an implementation of the RFC 3080/RFC 3081 protocol, also known as BEEP: Block Extensible Exchange Protocol. In the past it was called BXXP because the Xs of e(X)tensible and e(X)change. Due to some marketing naming decision finally it was called BEEP because the Es of the (E)xtensible and (E)xchange.

    BEEP:   the protocol
    Vortex: an implementation

From a simple point of view, the BEEP protocol defines how data must be exchanged between applications, also called BEEP peers, using several abstractions, that allows programmers to write network applications with stronger features such as asynchronous communication, several concurrent messages being exchanged over the same connection and so on, without worrying too much about details.

Previous abstractions defined by the BEEP RFC are really important. To understand them is a key to understand, not only BEEP as a protocol but Vortex Library as an implementation. This will also help you to get better results while using BEEP to implement your network protocol.

       BEEP abstraction layer
       |       Message       |
       |        Frame        |
       |                     |
       |      (payload)      |
       |  Channel / Profiles |
       |       Session       |

Previous table is not the BEEP API stack or the Vortex Library API stack. It represents the concepts you must use to be able to send and receive data while using a BEEP implementation like Vortex Library.

A message is actually your application data to be sent to a remote peer. It has no special meaning for a BEEP implementation. Applications using the particular BEEP implementation are the ones who finally pay attention to message format, correction and content meaning.

When you send a message (or a reply) these messages could be splitted into frames.

These frames have an special header, called the BEEP frame header, which includes information about payload sequence, message type, channels used and many things more. This data, included inside the BEEP frame headers allows BEEP peers to track communication status, making it possible to detect errors, sync lost, etc.

Payload is the way network protocol designers usually call to the application level data, that is, your data application. However, this payload could represent only a piece of your information. This is not important for you, at this moment, because Vortex Library manage frame fragmentation (internal/external) in a transparent way.

When you send a message, you select a channel to do this communication. We have seen that a message is actually splitted but from the application view, in most cases, this is not important: applications send messages over channels.

Inside BEEP protocol, every channel created must be running under the semantic of a profile definition. In fact, the part of the application that takes care about message format, correction, and content meaning is the BEEP profile.

This simple concept, which usually confuse programmers new to BEEP, is not anything estrange or special. A profile only defines what type of messages will be exchanged inside a channel. It is just an agreement between BEEP peers about the messages to exchange inside a channel and what they mean.

Actually, to support a profile means to register the string which identifies the profile and to implement it on top of Vortex Library so that profile can send and receive message according to the format the profile defines. A profile inside Vortex Library is not:

In order to use the Vortex Library, and any BEEP implementation, you must define your own profile. Later, you can read: defining a profile inside Vortex Library.

The last one concept to understand is the BEEP session. According to BEEP RFC, a session is an abstraction which allows to hold all channels created to a remote BEEP peer (no matter what profiles where used). Because Vortex Library is a BEEP implementation mapped into TCP/IP, a session is actually a TCP connection (with some additional data).

Now we know most of the concepts involving BEEP, here goes how this concepts get mapped into a concrete example using Vortex. Keep in mind this is a simplified version on how Vortex Library could be.

In order to send a message to a remote peer you'll have to create a VortexConnection, using vortex_connection_new as follows:

        char  * host = "myhost.at.frobnicate.com";
        char  * port = "55000";
        // Creates a Vortex Connection without providing a
        // OnConnected handler or user data. This will block us
        // until the connection is created or fails.
        VortexConnection * connection = vortex_connection_new (host, port, NULL, NULL);

Once finished, you have actually created a BEEP session. Then you have to create a VortexChannel, using vortex_channel_new function, providing a profile defined by you or an existing one. On that process it is needed to select the channel number, let's say we want to create the channel 2.

        // Create a Vortex Channel over the given connection and using as channel number: 2.
        VortexChannel * new_channel = NULL;
        new_channel = vortex_channel_new (connection, 2,
                                          // do not provide on
                                          // channel close handlers.
                                          NULL, NULL, 
                                          // provide frame receive
                                          // handler (second level one)
                                          // Now, on_receiving_data
                                          // will be executed for every
                                          // frame received on this
                                          // channel unless wait reply
                                          // method is used.
                                          on_receiving_data, NULL,
                                          // do not provide a
                                          // OnChannelCreated
                                          // handler. This will block
                                          // us until the channel is
                                          // created.
                                          NULL, NULL);

Once the channel is created (keep in mind that the remote peer can actually deny the channel creation) you could send messages to remote peer using this channel as follows:

       // send a reply import message to remote peer.
       if (vortex_channel_send_msg (new_channel,
                                   "this a message to be sent",
                                   NULL)) {
           printf ("Okey, my message was sent");

And finally, once the channel is no longer needed, you can close it as follows:

      // close the channel. This will block us until the channel is closed because
      // to close a channel can actually take longer time because the remote peer
      // may not accept close request until he is done.
      if (vortex_channel_close (new_channel, NULL)) {
           printf ("Okey, my channel have been closed");

      // finally, finalize vortex running
      vortex_exit ();

That's all. You have created a simple Vortex Library client that have connected, created a channel, send a message, close the channel and terminated Vortex Library function.

1.2 How a Vortex Listener works (or how to create one)

To create a vortex listener, which waits for incoming beep connection on a given port the following must be done:

   #include <vortex.h>

   void on_ready (char  * host, int  port, VortexStatus status,
                  char  * message, axlPointer user_data) {
        if (status != VortexOk) {
              printf ("Unable to initialize vortex listener: %s\n", message);
              // do not exit from here using: exit or vortex_exit. This is actually
              // done by the main thread
        printf ("My vortex server is up and ready..\n");
        // do some stuff..

   int  main (int  argc, char  ** argv) {
       // init vortex library
       vortex_init ();
       // register a profile
       vortex_profiles_register (SOME_PROFILE_URI,      
                                 // provide a first level start
                                 // handler (start handler can only be
                                 // provided at first level)
                                 start_handler, start_data, 
                                 // provide a first level close handler
                                 close_handler, close_data,
                                 // provide a first level frame receive handler
                                 frame_received_handler, frame_received_data);
       // create a vortex server using any name the running host may have.
       // the on_ready handler will be executed on vortex listener creation finish.
       vortex_listener_new ("", "44000", on_ready);
       // wait for listeners
       vortex_listener_wait ();

       // finalize vortex running
       vortex_exit ();

        return 0;

These four steps does the follow:

On the 2) step, which register a profile called SOME_PROFILE_URI to be used on channel creation, it also set handlers to be called on events happening for this listener.

These handlers are: start handler, close handler, and frame_received handler.

The start handler is executed to notify that a new petition to create a new channel over some session have arrived. But this handler is also executed to know if Vortex Listener agree to create the channel. If start handler returns axl_true the channel will be created, otherwise not.

If you don't define a start handler a default one will be used which always returns axl_true. This means: all channel creation petition will be accepted.

The close handler works the same as start handler but this time to notify if a channel can be closed. Again, if close handler returns axl_true, the channel will be closed, otherwise not.

If you don't provide a close handler, a default one will be used, which always returns axl_true, that is, all channel close petition will be accepted.

The frame received handler is executed to notify a new frame have arrived over a particular channel. The frame before been delivered, have been verified to be properly defined. But, payload content must be actually checked by the profile implementation. Vortex Library doesn't pay attention to the payload actually being transported by frames.

All notification are run on newly created threads, that are already created threads inside a thread pool.

As a test, you can run the server defined above, and use the vortex-client tool to check it.

1.3 How a vortex client works (or how to create a connection)

A vortex client peer works in a different way than listener does. In order to connect to a vortex listener server (or a BEEP enabled peer) a vortex client peer have to:

   #include <vortex.h>
   int  main (int  argc, char  ** argv) {
     // a connection reference 
     VortexConnection * connection;

     // init vortex library
     vortex_init ();
     // connect to remote vortex listener
     connection = vortex_connection_new (host, port, 
                                         // do not provide an on_connected_handler 
                                         NULL, NULL);
     // check if everything is ok
     if (!vortex_connection_is_ok (connection, axl_false)) {
            printf ("Connection have failed: %s\n", 
                    vortex_connection_get_message (connection));
            vortex_connection_close (connection);
     // connection ok, do some stuff

     // and finally call to terminate vortex
     vortex_exit ();

Previous steps stands for:

Once a vortex connection is successfully completed, it is registered on Vortex Reader thread. This allows Vortex Reader thread to process and dispatch all incoming frame to its default channel handler.

We have talked about channel handlers: the start, close and frame received handler. Due to client peer nature, it will be common to not pay attention to start and close events. If no handler is defined, default ones will be used. Of course, if it is needed to have more control over this process, event handlers should be defined.

The frame received handler must be defined for each channel created. If no frame received handler is defined for a channel used by a vortex client, virtually you won't receive any frame.

This is because a vortex client is not required to register a profile before creating Vortex Connections. Of course, if you register a profile with the handlers before creating the connection those ones will be used if not handlers are provided on channel creation. See this section to understand how the frame dispatch schema works.

2.1 How an application must use Vortex Library to send and receive data

As defined on RFC 3080, any BEEP enabled application should define a profile to be used for its message exchange. That profile will make a decision about which message-exchange style defined will use. There are 3 message exchange style.

While using Vortex Library you can send data to remote peer using the following functions defined at the vortex channel API.

As you may observe to generate the different types of message to be sent a function is provided:

The first one allows you to send a new message. Once the message is queued to be sent the function returns you the message number used for this sending attempt. This function never block and actually do not send the message directly. It just signal the Vortex Sequencer to do the frame sequencing which finally will make Vortex Writer to send the frames generated.

The second function allows to positive reply to a specific message received. In order to be able to perform a positive reply using vortex_channel_send_rpy or a negative reply using vortex_channel_send_err you have to provide the message number to reply to.

The third function allows to reply an error to a specify message received. As the previous function it is needed the message number to reply to.

The fourth function allows to reply an ANS message to a received MSG. Several calls to that send ANS replies must be always ended with vortex_channel_finalize_ans_rpy which actually sends an NUL frame.

Things that cannot be done by Vortex applications, and any other BEEP framework, is to send MSG frames to each other without using reply message (RPY/ERR/ANS/NUL).

Actually you can use only MSG type message to send data to other Vortex (or BEEP) enabled application but this is not the point. Application have to think about MSG type as a request message and RPY as a request reply message. The point is: do not use MSG to reply message received, use RPY/ERR/ANS/NUL types.

2.2 The Vortex Library Frame receiving dispatch schema (or how incoming frames are read)

Once a frame is received and validated, the Vortex Reader tries to deliver it following next rules:

The second and first level handler dispatching method are called asynchronous message receiving because allows user code to keep on doing other things and be notified only when frames are received.

The wait reply method is also called synchronous dispatch method because it blocks the caller thread until the specific frame reply is received. The wait reply method disables the second and first level handler execution.

Keep in mind that if no second level handler, first level handler or wait reply method is defined, the Vortex Reader will drop the frame.

Because wait reply method doesn't support receiving all frames in a channel, to perform blocking code, you may also be interested in a mechanism that is implemented on top of the second (or first) level handlers, that allows to get the same functionality that the wait reply method, but including all frames received. Check the following function to know more about this method.

As you note, the Vortex Library support both method while receiving data: asynchronous method and synchronous method. This is also applied to sending user data.

2.3 Printf like interface while sending messages and replies

Additionally, there are also function versions for the previous ones which accepts a variable argument list so you can send message in a printf like fashion.

   if (!vortex_channel_send_msgv (a_channel, NULL, 
                                  "Send this message with content: %s and size: %d", 
                                  content, content_size)) {
         printf ("Unable to send my message\n");

They are the same function names but appending a "v" at the end.

2.4 Sending data and wait for a specific reply (or how get blocked until a reply arrives)

We have seen in previous section we can use several function to send message in a non-blocking fashion no matter how big the message is: calling to those function will never block. But, what if it is needed to get blocked until a reply is received.

Vortex Library defines a wait reply method allowing to bypass the second and first level handlers defined inside the frame received dispatch schema.

Here is an example of code on how to use Wait Reply method:

    VortexFrame   * frame;
    int             msg_no;
    WaitReplyData * wait_reply;
    // create a wait reply 
    wait_reply = vortex_channel_create_wait_reply ();
    // now send the message using msg_and_wait/v
    if (!vortex_channel_send_msg_and_wait (channel, "my message", 
                                           strlen ("my message"), 
                                           &msg_no, wait_reply)) {
        printf ("Unable to send my message\n");
        vortex_channel_free_wait_reply (wait_reply);

    // get blocked until the reply arrives, the wait_reply object
    // must not be freed after this function because it already free it.
    frame = vortex_channel_wait_reply (channel, msg_no, wait_reply);
    if (frame == NULL) {
         printf ("there was an error while receiving the reply or a timeout have occur\n");
    printf ("my reply have arrived: (size: %d):\n%s", 
             vortex_frame_get_payload_size (frame), 
             vortex_frame_get_payload (frame));

    // that's all!

2.5 Invocation level for frames receive handler

Application designer has to keep in mind the following invocation order for frame received handler:

As a consequence:

3.1 Defining a profile inside Vortex (or How profiles concept confuse people)

Now we have to consider to spend some time learning more about profiles. The profile concept inside the BEEP Core definition is the most simple but at the same time seems to be the most confusing.

From a simple point of view, a BEEP profile is what you add to the BEEP protocol to make it useful for you. BEEP provides you building blocks that you have to organize to create a useful protocol. This "protocol configuration" usually involves creating a BEEP profile (or reuse an existing one) extending the BEEP protocol beyond its initial definition to reach your needs.

From the source code point of view, creating a profile only means:

In other words, because the profile is only a definition on how you should send messages, how to reply to them, and what types of messages you will have to recognize, its content and format, or what will happen on some particular circumstances, it is only needed to register the profile name and to implement that behavior on top of the Vortex Library to fulfill the profile especification.

Maybe the main problem a new BEEP programmer have to face is the fact that a BEEP implementation doesn't allows him to start sending and receiving messages out of the box.

This is because the BEEP definition and Vortex Library implementation is more a toolkit to build application protocols than a ready to use application protocol itself. It is a framework to allow BEEP programmers to define its network protocols in a easy, consistent and maintainable way.

Now see the tutorial about creating a simple profile involving a simple server creation with a simple client.

4.1 Using piggyback to save one round trip at channel startup

Once defined the application protocol on top of Vortex Library or any other BEEP implementation we could find that creating a channel, involving a request-reply exchange, to later starting to perform real work, represents an initial imposed latency.

This could be easily solved by using piggybacking for the initial messages exchanged. Let's see how channel creation works, without using piggybacking, to later starting to exchange data:

    (1) I: A BEEP peer send an <start> channel item --->

                    <--- (2) L: Remote BEEP peer accept the <start> channel and reply

    (3) I: The BEEP peer send the initial <message> --->

                    <--- (4) L: Remote BEEP peer receives <message> and reply to it.

Previous example shows that underlying BEEP negotiation forces us to waste time, (1) and (2), by creating the channel, and later perform real work for our protocol: (3) and (4).

To solve this, what we have to do is to piggyback the message (3) into the initial (1) start message and to piggyback the reply (4) into the initial reply (2).

This allows to the peer receiving the initial start channel message to process the channel request and later to process the initial message receiving inside it as it where the initial frame received.

Additionally, the BEEP peer that have received the initial message not only reply to the channel start but also uses this first reply done to also reply the initial piggyback received.

As a result, previous example is now like the following:

    (1) I: A BEEP peer send an <start> channel item
    (3) I: [The BEEP peer send the initial <message>] --->

                         (2) L: Remote BEEP peer accept the <start> channel and reply
                    <--- (4) L: [Remote BEEP peer receives <message> and reply to it.]

Conclusion: we have saved one round trip, the channel creation initial exchange.

Ok, but how this is actually implemented inside Vortex Library? Well, piggybacking is mostly automatic while using Vortex Library. Let's see how it works for each BEEP peer side.

For the client side, to request creating a new channel and using this initial exchange to send the initial request you could use :

Previous functions will produce a start channel request defining the initial piggyback by using the profile_content parameter.

At the server side, supposing that is a Vortex Library one, it will receive the channel start request and the initial piggyback content, if the OnStartChannelExtended handler is defined to process incoming start request.

On that handler, the initial piggyback received will be the content of the profile_content parameter and the profile_content_reply parameter provides the way to reply to the initial piggyback received.

Here is an example for the OnStartChannelExtended handler:

  axl_bool      extended_start_channel (char              * profile,
                                        int                 channel_num,
                                        VortexConnection  * connection,
                                        char              * serverName,
                                        char              * profile_content,
                                   char             ** profile_content_reply,
                                   VortexEncoding      encoding,
                                   axlPointer          user_data)
      printf ("Received a channel start request!\n");
      if (profile_content != NULL) {
          // we have received an initial piggyback, reply to it
          // by filling up profile_content_reply
          (* profile_content_reply) = ...; // dynamically allocated message

      // accept the channel to be created.
      return axl_true;

Piggyback reply processing for client side is more simple. We have two cases:

If you think this is too complicated, that's ok. It means you can survive without using piggyback feature for your protocol. However, many standard BEEP profiles makes use of this feature to be more efficient, like TLS and SASL profiles.

4.2 Using MIME configuration for data exchanged under Vortex Library

We have to consider several issues while talking about MIME and MIME inside BEEP.

Initially, MIME was designed to allow transport application protocols, especially SMTP (but later extended to many protocols like HTTP), as a mechanism to notify the application level the content type of the object being received. The same happens with BEEP.

But, what happens when application protocol designers just ignore MIME information, no matter which transport protocol they are using? Again, this only depends on the requirements of the application protocol, mainly because MIME is just an indicator.

In any case, any message exchanged by a BEEP peer must be a conforming MIME message. This implies that at least the empty MIME meader must be appended to each message sent.

For example, you if you send the following message: "test", you or your BEEP toolkit must take care of adding the CR+LF prefix:

 MSG 3 1 . 11 6

So, the remote BEEP peer will receive a message with empty MIME headers: CR+LF + "test".

4.2.1 When should I consider using MIME for a BEEP profile?

First of all, no matter how you design your profile, MIME will stay at the core of BEEP, and therefore inside your profile. You can ignore its function and nothing will happen (beyond its basic implications that you must consider).

Now, if you pretend to develop a profile that is able to transport everything without previous knowledge (both peers can't make assumptions about the content to be received), it is likely MIME is required. Think about using the more appropiate helper program to open the content received: PNG files, PDF or a C# assembly.

Because MIME is implemented inside Vortex in a way you access to the content (MIME body) and MIME headers (vortex_frame_get_mime_header if defined) in a separated way, it becomes an interesting mechanism to allow extending your profile without modifying its content.

As a conclusion: if your system will be the message producer and the message consumer at the same time, you can safely ignore MIME (but Vortex will produce MIME compliat messages for you, see vortex_channel_set_automatic_mime), because you can make assumptions about the kind of messages to be exchanged. However, if a third party software is required to be supported, that is initially unknown, or it is required to have some flexible mechanism to notify "additional" information along with your profile messages, you'll need MIME.

4.2.2 How is MIME implemented inside Vortex Library

The BEEP protocol definition states that all messages exchanged are MIME objects, that is, arbitrary user application data, that have a MIME header which configures/specify the content. MIME support implemented inside Vortex Library is only structural, that is, it implements MIME structure requirements defined at RFC 2045.

Initially, if you send a message, without using MIME, because you didn't configure anything, then frames generated won't include any MIME header. However even in this case, the MIME body start indicator (CR+LF) will be added, to allow remote BEEP peer to detect the MIME header (nothing configured) and the MIME body (your message).

For example, if you send message "test" (4 bytes) and no MIME header is configured at any level, it is required to send the following:


MIME struct overview and how it applies to a message without MIME headers

That is, even if you do not pay attention to MIME, your messages will still include an inicial CR+LF appended to your message to indicate the remote side no MIME header is defined and to make your message MIME parseable.

4.2.3 Implicit MIME headers to all messages without MIME information

BEEP assume a MIME implicit configuration, which have the following values for those messages that do not configured "content-type" and "content-transfer-encoding":

        Content-Type: application/octet-stream
        Content-Transfer-Encoding: binary

4.2.4 How can I access MIME information on a frame received?

First of all, in order to complete MIME support, you must have automatic frame joining activated (vortex_channel_set_complete_flag). This is by default activated. In the case you are taking full control on frames received you must take care of MIME structure parsing by other means. You still can use vortex_frame_mime_process function, but the function will require a frame that contains all the MIME message to work.

Assuming you did receive a complete frame with a MIME message, you can access to the MIME body by calling to: vortex_frame_get_payload.

To access all the message received, including MIME headers and MIME body separator, use vortex_frame_get_content.

You can use vortex_frame_get_mime_header to access all MIME headers found on the message received (stored on the frame).

Because MIME could allow to have several instances for the same MIME header, vortex_frame_get_mime_header returns a structure (VortexMimeHeader) that allows to get the content of the MIME header (vortex_frame_mime_header_content) but it also allows to get the next instance found for the same MIME header by using vortex_frame_mime_header_next.

4.2.5 Automatic MIME configuration for outgoing messages

The following set of function allows to configure (and check) the values to be configured, on each message sent, for the MIME headers: "Content-Type" and "Content-Transfer-Encoding", in an automatic manner:

However, this mechanism doesn't fit properly if it is required to send arbitrary MIME objects (with diffent MIME headers) under the same profile, because previous configuration will append the same MIME information to every message being sent via:

In the case no MIME configuration is found for the profile, previous functions will prepend the MIME header separator "CR+LF" on each message sent. This allows to produce MIME compliant messages that have an empty MIME header configuration.

4.2.5 Disabling automatic MIME configuration for outgoing messages

Under some situations it is required to send already configured MIME objects through the set of functions previously described. Because those functions will automatically add an empty MIME header (CR+LF) to each message sent, is required to disable this behavior to avoid breaking message MIME configuration.

This is done using the following set of functions. They work at library, profile and channel level, having preference the channel level. In order of preference:

For example, disabling automatic MIME handling at profile level while cause Vortex Engine to not append any MIME header (including the body separator) to messages sent over a channel running a particular profile:

 // disable MIME automatic headers for the following profile
 vortex_profiles_set_automatic_mime ("urn:beep:some-profile", 2);

Special attention is required to the following code because it doesn't disable MIME handling but deffer the decision to the global library configuration, which is by default activated:

 // signal to use library current configuration
 vortex_profiles_set_automatic_mime ("urn:beep:some-profile", 0);

In any case, if the Vortex Engine finds the MIME automatic headers disabled, it will take/send messages received "as is", being the application level the responsible of producting MIME compliant messages.

NOTE: If the channel MIME handling (vortex_channel_set_automatic_mime) isn't configured (vortex_channel_get_automatic_mime returns 0) but the profile level (vortex_profiles_get_automatic_mime) or library level (vortex_conf_get VORTEX_AUTOMATIC_MIME_HANDLING) are configured, then the configuration is copied into the channel. This is done to improve library performance.

4.2.6 Default configuration for automatic MIME header handling.

By default only library level comes activates (vortex_conf_set VORTEX_AUTOMATIC_MIME_HANDLING). This means that, without any configuration, all channels created will automatically add a MIME header for each message sent.

NOTE: In the case no configuration is found on every level (0 is returned at vortex_channel_get_automatic_mime, vortex_profiles_get_automatic_mime, vortex_conf_get VORTEX_AUTOMATIC_MIME_HANDLING), then it is assumed automatic MIME handling is activated.

4.2.7 What happens if a wrong MIME formated message is received.

From a frame perspective (BEEP framing mechanism), the vortex engine considers as valid frames all of them as long as BEEP framing rules are observed.

From a MIME perspective, which is considered on top of the BEEP framing mechanism, if the message inside a VortexFrame is not MIME ready, the content returned by the function vortex_frame_get_content and vortex_frame_get_payload will be same, that is, the message received is returned untouched.

Obviously, under this situation, all API that belongs to the MIME support will provide no function:

So, if a message not conforming MIME rules is received, Vortex won't discard it, and it will be delivered "as is" to frame delivery handlers defined. Under this situation, a log will be reported to signal that a MIME parse error was found:

 (proc 25045): (warning) vortex-reader: failed to update MIME status for the frame, continue delivery

4.2.8 Could new MIME support break compatibility with previous Vortex Library?

It is possible under some situations. Before Vortex Library 1.0.15 and its corresponding 1.1.0 release, a bug was fixed in the way messages was produced. In the case no MIME header was configured, the message produced wasn't prefixed by a CR+LF pair. This is wrong since the remote BEEP peer expects to receive a MIME compliant message, with at least the CR+LF to signal that no MIME header was configured.

In any case, if problems are found, these are the solutions:

1) The obvious solution is to upgrade both (client and server) peers to support same MIME implementation.

2) In the case you want to only update your BEEP client but still connect to Vortex peers running 1.0.14 or previous, you have to disable automatic MIME header handling. See previous sections. A direct hack to disable it globally could be:

 // library level desactivation for automatic MIME header 

3) In the case you want to only upgrade your BEEP listener, but you still want to receive connections from old and new clients, nothing special is required. This is automatically supported by new vortex engine.

No other compatibility issue is reported.

3.2 Implementing the request-response pattern

When it comes to implement the request/response interaction pattern, BEEP is a great choice. On this section, it is provided some tips to enable people to properly implement this pattern type with Vortex Library.

While implementing the request/response pattern the very first thing to control is how a reply is processed. If pattern is implemented using a synchronous invocation, there is not too much problems. The fun comes while implementing it in an asynchronous manner.

Asynchronous, request/response, pattern needs to solve how to associate and process replies received with the proper handler provided at the time the request was performed. Inside Vortex Library, asynchronous request replies are processed by providing a frame received handler (provided at vortex_channel_new or vortex_channel_new_full).

However, this handler is provided at the channel creation time, making it available to all requests performed under that channel.

The very first thought to solve previous problem is to provide a different frame receive with some application data for every request performed so each reply is processed by its corresponding handler. This is actually done by using vortex_channel_set_received_handler.

Well, this will cause a race condition making responses to be processed by request handlers that are not the associated. This is because the frame receive handler for a channel could be only one at the same time, which is applied to all messages replies received on the given channel. Several call to this function will make that the frame received handler set, will be the value set on the last call to this function.

This means that, if several request are performed, followed by its corresponding call to this function, and knowing that several request on the same channel are replied in the same other they were issued, the frame received handler that will process the first message will be the last one set not the one set for that first message.

Here is an example of code on how to produce a race condition:

 // set the frame received for the request A
 vortex_channel_set_received_handler (channel, process_requestA, data_A);

 // perform request A
 vortex_channel_send_msg (channel, "request A", 9, NULL);

 // set the frame received for the request B
 vortex_channel_set_received_handler (channel, process_requestB, data_B);

 // perform request B
 vortex_channel_send_msg (channel, "request B", 9, NULL);

Obviously, this seems to be pretty clear if you place the problem at a very few lines. However, previous interactions are usually produced by a function called, in your code that is likely to be named as make_invocation, which calls to vortex_channel_set_received_handler and then to vortex_channel_send_msg, which starts to be not so obvious.

The key concept here is to ensure that every message reply to be received must be processed by the right frame receive handler. This could be accomplish using several techniques:

At any case it is recommended, while using the request/response pattern, to use the Channel Pool feature which will allow you to avoid race conditions while getting a channel that is ready but being asked by several threads at the same time. It will also negotiate for you a new channel to be created if the channel pool doesn't have any channel in the ready state.

It is really easy to change current code implemented to use vortex channel pool. Here is an example:

 VortexChannelPool * pool;
 void init_channel_pool (VortexConnection * connection) {

    // create the channel pool (this should be done for each 
    // connection only one time avoiding several threads to call this
    // function

     pool = vortex_channel_pool_new (connection,
                                     1, // how many channels to be created at first time
                                     // close handler stuff
                                     NULL, NULL, 
                                     // received handler stuff (set it to null)
                                     NULL, NULL,
                                     // async notification for pool creation stuff
                                     NULL, NULL);
 VortexChannel * get_channel_available (VortexOnFrameReceived received,
                                        axlPointer            user_data) {

      // get the next channel available (this function could be
      // called by several threads at the same time

      result = vortex_channel_pool_get_next_ready (pool, axl_true);
      // now, Vortex API is ensuring us we are the only one owner for the channel
      // result, let's change the frame receive handler
      vortex_channel_set_received_handler (result, received, user_data);
      // return the channel
      return result;

 void release_channel (VortexChannel * channel) {

      // once finished release the channel, to return it to the
      // pool. The following must be do only, and only once the
      // channel have received its reply.

      vortex_channel_pool_release_channel (pool, channel);

Previous source code have tree functions. Here is the explanation:

3.3 Configuring Vortex Library IO layer

Default Vortex Library implementation doesn't require you to pay attention to such details like:

However, it turns out that these are topics that are likely to be asked. For your information, Vortex Library internal function have a default configuration that makes uses of the BSD send and recv function to perform IO read and write operations. This default configuration could be changed by using the following function which are specific for each connection:

As an example, Vortex TLS implementation uses previous handlers to configure how Vortex Library reads and sends its data once the TLS negotiation have finished. This allows to keep on writing higher level code that expect to have a function that is able to send and receive data from its underlying transport socket, no matter if it is TLS-ficated (vortex_connection_is_tlsficated).

User space only needs to implement a small piece of code inside the handler required and Vortex Library will call it at the right time. Previous function requires to define the following handlers:

You'll find that previous definition doesn't allows to pass in a pointer to the function so you could implement a kind of special operation based on the data received. To accomplish this task, you should use the following set of function which will allow you to store arbitrary data inside a connection, key-index hash like, even allowing the destroy function to be used once the connection is closed or the value is replaced:

For the case of the IO blocking mechanism, Vortex now checks for the presense of select(2), poll(2) and epoll(2) system call, selecting the best mechanism to be used by default. However, you can change to the desired mechanism at run time (even during transmission!) using:

Previous functions, provides a built-in mechanism to select already implemented mechanism. In the case you want to provide your own user space implementation to handling I/O waiting, you can use the following handlers to define the functions to be executed by the Vortex I/O engine at the appropiate time:

Previous handlers must be defined as a whole, it is not possible to only define a certain piece reusing the rest of the handlers. If the handlers are properly implemented, they will allow Vortex Library to perform IO operation with the API you have provided.

As an example, inside the IO module (vortex_io.c) can be found current implementation for all I/O mechanism supported by the library.

4.3 General considerations about transfering files

There are several methods that can be used to transfer a file. They differ in the way they consume memory and how difficult they are to properly implement them. These methods are:

The first method (sending one big single MSG) it's the most simple. It is not required to split the file and assamble it in the remote side.

However, this method consumes a lot of memory because in general terms you must load all the file into memory, them pass it to the Vortex API, which also does its own copy, now you have twice memory loaded and them the memory is retained until the message is completely sent in smallers MSG frames.

In the same direction, people using this method usually do not configure vortex_channel_set_complete_flag which causes all frames conforming the big message to be hold into memory at the remote side until the last frame is received. In this case, once the last message is received, Vortex will allocate enough memory to consolidate all frames into one single content. Again, more memory consumed.

So, FIRST METHOD is easy, but really poor in performance terms.

NOTE: the fact that vortex has the hability to split your messages into the allowed remote channel window size is at the same time a valuable feature and a source of problems. This automatic splitting makes more easy to do not care about content size sent. So, as a general rule, try to control the size of the content sent.

SECOND METHOD involves the developer in the process of preparing the content to be sent, and to take advantage of local store. In this method, the sender open the file and reads chunks of 2k/4k/8k/12k/16k, and send them by using MSG frames.

This makes memory consumption to be lower than previous case because the entire file isn't loaded and, as the transfer progress, the remote side can consolidate all chunks received directly into a file rather holding it into memory.

This method also requires that frame received activation to be serialized because you have to place all pieces received in other. This is archived by using vortex_channel_set_serialize.

This method is more difficult but results are better. The same happens to the following method.

THIRD METHOD involves doing pretty much the same like SECOND METHOD, but using ANS/NUL (one-to-many) exchange style.

Because BEEP requires all issued MSG to be replied, in the second method each MSG sent requires the receiving side to reply with a RPY (usually empty). This is not required with ANS/NUL exchange style.

In this case, we ask the receiving side to issue a MSG request (asking for download the file). Then, the sending side opens the file to be transferred and send its pieces by using ANS messages.

The receiving side consolidates into file all pieces received without requiring to reply to each ANS message received.

THIRD METHOD is the best in terms of memory consuption and network efficiency.

We have also to consider other key factors for an effective and fast transfer. In general they are two: window size and frame fragmentation.

The first concept (window size) is part of the BEEP way to do channel flow control (see http://www.beepcore.org/seq_frames.html). The initial window size for all channels is 4k. This value must be elevated to something bigger that fits your environment. This is controlled via vortex_channel_set_window_size.

The second concept (frame fragmentation) talks about how your messages are splitted in the case they don't fit into the remote window size. Take a look on the following handler: VortexChannelFrameSize

Now take a look into the following examples included in the test directory:

5.1 Securing a Vortex Connection (or How to use the TLS profile)

As we have said, the main advantage the BEEP protocol have is that is solves many common problems that the network protocol designer will have to face over again. Securing a connection to avoid other parties to access data exchanged by BEEP peers is one of them.

The idea behind the TLS profile is to enable user level applications to activate the TLS profile for a given session and then create channels, exchange data, etc. Inside Vortex Library, the is no difference about using a connection that is secured from one that is not.

Common scenario for a Vortex Library client application is:

For the listener side of the TLS profile, we have two possibilities:

  1. Use predefined handlers provided by the Vortex to activate listener TLS profile support.

    This is a cheap-effort option because the library comes with a test certificate and a test private key that are used in the case that locator handlers for such files are not provided.

    This enables to start developing the whole system and later, in the production environment, create a private key and a certificate and define needed handlers to locate them.

    On this case, we could activate listener TLS profile support as follows:

     // activate TLS profile support using defaults
     vortex_tls_accept_negotiation (NULL,  // accept all TLS request received
                                    NULL,  // use default certificate file
                                    NULL); // use default private key file

    NOTE: This option is highly not recommended on production environment because the private key and certificate file are public!

  2. Define all handlers required, especially those providing the certificate file and the private key file:

    In this case:

    • A VortexTlsAcceptQuery handler should be defined to control how are accepted incoming TLS requests. Here is an example:
       // return axl_true if we agree to accept the TLS negotiation
       axl_bool      check_and_accept_tls_request (VortexConnection *connection, 
                                                   char  *serverName)
           // perform some especial operations against the serverName
           // value or the received connection, return axl_false to deny the 
           // TLS request, or axl_true to allow it.
           return axl_true;  

    • A VortexTlsCertificateFileLocator handler should be defined to control which certificate file is to be used for each connection. Here is an example:
       char * certificate_file_location (VortexConnection * connection, 
                                         char             * serverName)
           // perform some especial operation to choose which 
           // certificate file to be used, then return it:
           return vortex_support_find_data_file ("myCertificate.cert"); 

      Please note the use of vortex_support_find_data_file function which allows to write portable source code avoiding native-full-paths. Check out this document to know more about this.

    • A VortexTlsPrivateKeyFileLocator handler should be defined to control which private key file is to be used for each connection. Here is an example:
       char * private_key_file_location (VortexConnection * connection, 
                                         char             * serverName)
           // perform some especial operation to choose which 
           // private key file to be used, then return it:
           return vortex_support_find_data_file ("myPrivateKey.pem"); 

    • Now use previous handlers to configure how TLS profile is support for the current Vortex Library instance as follows:

       // activate TLS profile support using defaults
       vortex_tls_accept_negotiation (check_and_accept_tls_request,

      NOTE: If some handler is not provided the default one will be used. Not providing one of the file locators handler (either certificate locator and private key locator) will cause to not work TLS profile.

There is also an alternative approach, which provides more control to configure the TLS process. See vortex_tls_accept_negotiation for more information, especially vortex_tls_set_ctx_creation and vortex_tls_set_default_ctx_creation.

Now your listener is prepared to receive incoming connections and TLS-fixate them. The only step required to finish the TLS issue is to produce a certificate and a private key to avoid using the default provided by Vortex Library. See next section.

5.2 How to create a certificate and a private key to be used by the TLS profile

Now we have successfully configured the TLS profile for listener side we need to create a certificate/private key pair. Currently Vortex Library TLS support is built using OpenSSL (http://www.openssl.org). This SSL toolkit comes with some tools to create such files. Here is an example:

  1. Use CA.pl utility to create a certificate with a private key as follows:

      $ /usr/lib/ssl/misc/CA.pl -newcert
      Generating a 1024 bit RSA private key
      writing new private key to 'newreq.pem'
      Enter PEM pass phrase:
      Verifying - Enter PEM pass phrase:
      You are about to be asked to enter information that will be incorporated
      into your certificate request.
      What you are about to enter is what is called a Distinguished Name or a DN.
      There are quite a few fields but you can leave some blank
      For some fields there will be a default value,
      If you enter '.', the field will be left blank.
      Country Name (2 letter code) [AU]:ES
      State or Province Name (full name) [Some-State]:Spain
      Locality Name (eg, city) []:Coslada
      Organization Name (eg, company) [Internet Widgits Pty Ltd]:Advanced Software Production Line, S.L.
      Organizational Unit Name (eg, section) []:Software Development
      Common Name (eg, YOUR name) []:Francis Brosnan Blázquez
      Email Address []:francis@aspl.es
      Certificate (and private key) is in newreq.pem

    Now you have a certificate file at newreq.pem having both items: the certificate and the private key inside it. The content of the file should look like this:

     -----BEGIN RSA PRIVATE KEY-----
      Proc-Type: 4,ENCRYPTED
      DEK-Info: DES-EDE3-CBC,91635CC2D1C00C1F
      -----END RSA PRIVATE KEY-----
      -----BEGIN CERTIFICATE-----
      -----END CERTIFICATE-----

    You can split the content of the file generated into two files: the private key and the public certificate part. The private key file is the piece which is enclosed by BEGIN/END-"RSA PRIVATE KEY". The public certificate is the piece enclosed by BEGIN/END-"CERTIFICATE".

    Splitting the certificate produced into two files is not required beucase openssl lookup for the appropriate part while providing the same file for the certificate or the private key.

  2. Now it is required to remove the pass phrase because Vortex Library already doesn't support providing a callback to configure it. This is done by doing the following:
     openssl rsa -in newreq.pem -out private-key-file.pem
     Enter pass phrase for newreq.pem:
     writing RSA key

    As a result for the previous operation, the resulting file should look like:

     $ less private-key-file.pem 
     -----BEGIN RSA PRIVATE KEY-----
     -----END RSA PRIVATE KEY-----

  3. Now, you can remove the private key part from the initial certificate generated and put it into a file to store the private key with pass phrase. However, this is not required because the openssl provides a way to configure the pass phrase to private key without it.

5.3 Authenticating BEEP peers (or How to use SASL profiles family)

While using or designing network protocols, a common issue to solve and support is to identify and authenticate users on top of it. This security issue is supported inside BEEP through SASL profiles.

SASL (RFC2222) is a security framework which defines how security protocols are implemented so the same program structure could take advantage no matter which security protocol is being used.

SASL (Simple Authentication and Security Layer) provides not only a way to identify users but also to secure connections. At this moment, SASL implementation inside Vortex Library only provides authentication (that is the authentication part of SASL).

This is not a big problem knowing that TLS profile could be enabled, making the connection secure (providing the security layer) and then negotiate user identification (and its authorization) using SASL.

Again, the idea behind this is to design your application protocol without taking into account details such as: how to ensure that the protocol session is secure and who is the user at the other side of the peer using the protocol.

Currently these are the SASL profiles implemented and tested inside Vortex Library:

5.4 How to use SASL at the client side

Now you have an overview for SASL profiles supported here is how to use them.

Vortex Library SASL implementation uses for the client side (the BEEP peer that wants to be authenticated) vortex_sasl_set_propertie and then vortex_sasl_start_auth to begin SASL negotiation.

With vortex_sasl_set_propertie you set the values required by the process such as: the user and its password. Once all values required are set a call to vortex_sasl_start_auth is done to activate SASL layer.

Every SASL profile requires different properties to be set. Some of them are optional and some of them are common to all profiles.

Let's start with an example on how to CRAM-MD5 profile to authenticate a client peer:

 VortexStatus    status;
 char          * status_message;
 // STEP 1: check if SASL is activated for the given Vortex Library
 if (! vortex_sasl_is_enabled ()) {
      // unable to activate SASL profile. This only happens when
      // Vortex Library wasn't built with SASL support
      printf ("Unable to initialize SASL profiles.\n");
 // STEP 2: set required properties according to SASL profile selected
 // set user to authenticate
 vortex_sasl_set_propertie (connection, VORTEX_SASL_AUTH_ID, "bob", NULL);

 // set the password
 vortex_sasl_set_propertie (connection, VORTEX_SASL_PASSWORD, "secret", NULL);
 // STEP 3: begin SASL negotiation
 vortex_sasl_start_auth_sync (// the connection where SASL will take place
                              // SASL profile selected
                              // SASL status variables
                              &status, &status_message);

 // STEP 4: once finished, check of authentication
 if (vortex_sasl_is_authenticated (connection)) {
      printf ("SASL negotiation OK, user %s is authenticated\n",
               vortex_sasl_get_propertie (connection, VORTEX_SASL_AUTH_ID));
 }else {
      printf ("SASL negotiation have failed: status=%d, message=%s\n",
               status, message);
 // roughly, that's all for the client side

Previous code will look the same for all SASL profile selected. The only part that will change will be properties provided (that are required by the SASL profile STEP 2).

On step 3 vortex_sasl_start_auth_sync is used to perform SASL negotiation. This function is the blocking version for vortex_sasl_start_auth. Synchronous version is only recommended for bath process because the caller get blocked until SASL profile finish. However it is easy to explain SASL function inside Vortex using synchronous version.

Asynchronous version (through vortex_sasl_start_auth) is preferred because allows to perform other tasks (like updating GUI interfaces) while SASL negotiation is running.

Here is a table with properties that are required (or are optional) for each SASL profile used.


5.5 How to use SASL at the server side

Well, as we have seeing in the previous section, SASL at the client side is entirely driven by properties (through vortex_sasl_set_propertie). However at the server is SASL is entirely driven by call backs.

There is one callback for each SASL profile supported inside Vortex Library. They allow to your server application to connect SASL authentication process with legacy user/password databases.

Vortex Library supports, through SASL, transporting and authenticating users but, at the end, the programmer must provide the final decision to allow or to deny SASL authentication request.

Here is an example on how to activate PLAIN support and validate request received for this profile:

 [...] at some part of your program (likely to be at the main)
 // check for SASL support
 if (!vortex_sasl_is_enabled ()) {
    // drop a log about Vortex Library not supporting SASL
    return -1;
 // set default plain validation handler
 vortex_sasl_set_plain_validation (sasl_plain_validation);
 // accept SASL PLAIN incoming requests
 if (! vortex_sasl_accept_negotiation (VORTEX_SASL_PLAIN)) {
        printf ("Unable  accept SASL PLAIN profile");
        return -1;
 // validation handler for SASL PLAIN profile
 axl_bool      sasl_plain_validation  (VortexConnection * connection,
                                       const char       * auth_id,
                                       const char       * authorization_id,
                                       const char       * password)

      // At this point your server application should connect
      // to its internal user/password database to validate
      // incoming request. 
      // In this case we perform a validation based on receiving
      // a pair based on bob/secret allowing it, and denying
      // the rest of user/password pairs.

        if (axl_cmp (auth_id, "bob") && 
            axl_cmp (password, "secret")) {
              // notify Vortex that the given SASL request
              // have been accepted by the application level.
                return axl_true;
        // deny SASL request to authenticate remote peer
        return axl_false;

Previous example show the way to activate the SASL authentication support. First it is configured a handler to manage the authentication request, according to the SASL method desired. In this case the following function is used:

Keep in mind that previous function doesn't allows to configure an user data pointer to be passed to the handler. If you require this you must use the extended API, which is called the same but appending "_full". In this case, the function to be used is:

For every SASL mechanism supported by Vortex Library there are two functions to configure the validation handler and the extended validation handler.

Then, a call to register and activate the profile VORTEX_SASL_PLAIN is done. This step will notify vortex profiles module that the selected SASL profile must be anonnounced as supported, at the connetion greetings, configuring all internal SASL handlers. In this case, the example is not providing an user data to the vortex_sasl_accept_negotiation function. In the case that a user data is required, the following function must be used:

Now the SASL PLAIN profile is fully activated and waiting for requests. Validating the rest of SASL profiles works the same way. Some of them requires to return axl_true or axl_false to allow or to deny received request, and other profiles requires to return the password for a given user or NULL to deny it.

Here is a table for each profile listing the profile activation function and its handler.

Validation handler settingValidation handler

Once the validation is done. You can use the following two functions to check authentication status for the connection in the future, commonly, at the frame receive handler. This allows you to authenticate, and then, at the frame receive handler check the authentication status (or at any other place of course).

The first function is really important, and should be used before any further check. This ensures you that you are managing an authenticated connection, and then you can call to the next function, vortex_sasl_auth_method_used, to get the auth method that was used, and finally call to the following function to get the appropiate auth data:

Here is an example about checking the auth status, and getting auth properties, at a frame receive handler:

 // drop a log about the sasl properties 
 if (vortex_sasl_is_authenticated (connection)) {
     // check the connection to be authenticated before assuming anything 
     printf ("The connection is authenticated, using the method: %s\n", 
              vortex_sasl_auth_method_used (connection));

     printf ("Auth data provided: \n  authid=%s\n  authorization id=%s\n  password=%s\n  realm=%s\n  anonymous token=%s\n",
              vortex_sasl_get_propertie (connection, VORTEX_SASL_AUTH_ID),
              vortex_sasl_get_propertie (connection, VORTEX_SASL_AUTHORIZATION_ID),
              vortex_sasl_get_propertie (connection, VORTEX_SASL_PASSWORD),
              vortex_sasl_get_propertie (connection, VORTEX_SASL_REALM),
              vortex_sasl_get_propertie (connection, VORTEX_SASL_ANONYMOUS_TOKEN));
 } else {
     printf ("Connection not authenticated..\n");
     // at this point DON'T RELAY ON data returned by
     // - vortex_sasl_auth_method_used
     // - vortex_sasl_get_propertie