All Data Structures Functions Variables Typedefs Enumerations Enumerator Groups Pages
Turbulence Administrator manual

Section 1: Installation notes

Section 2: Turbulence configuration

Section 3: BEEP profile management

Section 4: Turbulence module management

1.1 Installing Turbulence

Turbulence dependencies

Turbulence have the following required dependencies:

  1. Vortex Library 1.1: which provides the BEEP engine (located at: http://www.aspl.es/vortex).

  2. Axl Library: which provides all XML services (located at: http://www.aspl.es/xml).

The following optional dependencies are not required to built Turbulence, but provides really useful features.

  1. Libpcre3: a library that provides perl expressions support. It is used by all expressions used by the turbulence configuration file. If not provided all expressions provided will be matched as is.

    This regular expression library was written by Philip Hazel, and copyrighted by the University of Cambridge, England.

    The software is available at: ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre

  2. OpenSSL: a library that provides TLS support. If not provided mod-tls won't be available.

    The software is available at: http://www.openssl.org

  3. GNU SASL: a library that provides SASL support. If not provided won't be available.

    The software is available at: ftp://alpha.gnu.org/pub/gnu/gsasl

In many cases, GNU SASL, Libpcre3 and OpenSSL are already available on your platform. Check your distribution documentation.

Building Turbulence from source on POSIX / Unix environments

It is assumed you have a platform that have autoconf, libtool, and the rest of GNU tools to build software.

Get the source code at the download page: http://www.aspl.es/turbulence/downloads.html

Turbulence is compiled following the standard autoconf-compatible procedure:

>> tar xzvf turbulence-X.X.X-bXXXX.gXXXX.tar.gz
>> cd turbulence-X.X.X-bXXXX.gXXXX
>> ./configure
>> make
>> make install

Once finished, next step

Now you must configure your Turbulence installation. Check the following section.

2.1 Turbulence configuration

2.2 How turbulence is configured (configuration file location)

Turbulence is configured through XML 1.0 files. The intention is provide an easy and extensible way to configure turbulence, allowing third parties to build tools to update/reconfigure it. It is also provided a DTD to ensure that configuration file created is properly constructed at a minimum syntax level.

According to your installation, the main turbulence configuration file should be found at the location provided by the following command:

>> turbulence --conf-location
VERSION: 0.3.2.b4053.g4055
VORTEX_VERSION: 1.1.2.b4069.g4070
AXL_VERSION: 0.6.0.b4064.g4081
SYSCONFDIR: /etc
TBC_DATADIR: /usr/share
Default configuration file: /etc/turbulence/turbulence.conf

Alternatively you can provide your own configuration file by using the –config option:

>> turbulence --conf my-turbulence.conf

Turbulence main configuration file includes several global sections:

  1. <global-settings>: This main section includes several run time configuration for the BEEP server: TCP ports, listener address, log files, crash handling, etc.

  2. <modules>: Under this section is mainly configured directories holding modules.

  3. <profile-path-configuration>: Under this section is provided the profile path configuration, an administrative configuration which allows to mix, sequence, and select profiles to be provided to client peers according to several run time values such remote address, profiles already initiated, etc..

2.3 Turbulence addresses and ports used

Ports and addresses used by Turbulence to listen are configured at the <global-settings> section. Here is an example:

<global-settings>
   <!--  ... more settings ...  -->
   <!--  port to listen to  -->
   <ports>
      <port>3206</port>
      <port>44010</port>
   </ports>
   <!--  listener configuration (address to listen)  -->
   <listener>
      <name>0.0.0.0</name>
   </listener>
</global-settings>

Previous example will make Turbulence to listen on ports 3206 and 44010 for all addresses that are known for the server hosting turbulence (0.0.0.0). Turbulence will understand this section listening on all addresses provided, for all ports.

Then you can use profile path to enforce which profiles will be served on each port. For example, you can provide TUNNEL support only on port 604 (though not recommended: see Building wrong port oriented network applications).

Alternatively, during development or when it is found a turbulence bug, it is handy to configure the default action to take on server crash (bad signal received). This is done by configuring the following:

<global-settings>
   <!--   ....more settings ....   -->
   <!--   crash settings: allowed values for action attribute.
       [*] hold:   lock the current instance so a developer can attach to the
                   process  to debug what's happening.

       [*] ignore: just ignore the signal, and try to keep running.

       [*] quit,exit: terminates turbulence execution.

       [*] backtrace: creates a backtrace report of the current faulted process. 
                      this backtrace report is saved into a file for later use and
                      then the process is finished.

       Use mail-to attribute to configure the default smtp notification to
       be used when a bad signal is received. mail-to attribute is optional.
      -->
   <on-bad-signal mail-to="default" action="hold" />
</global-settings>

The "hold" option is really useful for real time debugging because it hangs right after the signal is received. This allows to launch the debugger and attach to the process to see what's happening.

Another useful option is "backtrace" which produces a backtrace report (of the current process) saving it into a file. This allows to save some error evidences and then let the process to finish. This option combined with mail-to attribute is a powerful debug option.

Inside production environment it is recommended "ignore".

2.4 Receiving SMTP notification on failures and error conditions

Turbulence includes a small SMTP client that allows to report critical or interesting conditions. For example, this is used to report backtraces on critical signal received.

This configuration is found and declarted at the <global-settings> section. Here is an example:

<notify-failures>
   <smtp-server is-default="yes" 
                mail-to="test@example.com" 
                mail-from="turbulence@aspl.es" 
                port="25" 
                server="localhost" 
                id="default" />
</notify-failures>

It is possible to have more than one <smtp-server> declared. They are later used/referenced either through id attribute or because the declaration is flagged with an is-default=yes.

2.5 Configuring turbulence log files

Turbulence logs is sent to a set of files that are configured at the <global-settings> section:

<log-reporting enabled="yes" >
   <general-log file="/var/log/turbulence/main.log" />
   <error-log file="/var/log/turbulence/error.log" />
   <access-log file="/var/log/turbulence/access.log" />
   <vortex-log file="/var/log/turbulence/vortex.log" />
</log-reporting>

These files hold logs for general information (<general-log>), error information (<error-log>), client peer access information (<access-log>) and vortex engine log produced by its execution (<vortex-log>).

Apart from the vortex log (<vortex-log>) the rest of files contains the information that is produced by all calls done to the following library functions: msg, msg2, wrn and error.

By default, Turbulence server is started with no console output. All log is sent to previous log files. The second destination available is the console output.

Four command line options controls logs produced to the console by Turbulence and tools associated:

  1. –debug: activates the console log, showing main messages. Turbulence tools have this option implicitly activated.

  2. –debug2: activates implicitly the –debug option and shows previous messages plus new messages that are considered to have more details.

  3. –debug3: makes log output activated to include more details at the place it was launched (file and line), process pid, etc.

  4. –color-debug: whenever previous options are activated, if this one is used, the console log is colored according to the kind of message reported: green (messages), red (error messages), yellow (warning messages).

If previous options are used Turbulence will register a log into the appropriate file but also will send the same log to the console.

In the case you want to handle all logs through syslog, just declare as follows. Note that once syslog is enabled, general-log, error-log, access-log and vortex-log declarations will be ignored, making all information to be sent to syslog.

<log-reporting use-syslog="yes" enabled="yes" >
   <general-log file="/var/log/turbulence/main.log" />
   <error-log file="/var/log/turbulence/error.log" />
   <access-log file="/var/log/turbulence/access.log" />
   <vortex-log file="/var/log/turbulence/vortex.log" />
</log-reporting>

2.7 Alter default turbulence base system paths

By default Turbulence has 3 built-in system paths used to locate configuration files (sysconfdir), find data files (datadir) and directories used at run time (runtime datadir) to implement internal functions.

To show default values configured on your turbulence use:

>> turbulence --conf-location

However, these three system paths can also be overrided by a configuration placed at the global section. Here is an example:

<global-settings>
   <!--  more settings  -->
   <system-paths>
      <!--  override runtime-datadir configuration  -->
      <path value="path/to/runtime-datadir" name="runtime_datadir" />
      <path value="path/to/sysconfdir" name="sysconfdir" />
      <path value="path/to/datadir" name="datadir" />
      <!--  additional domain search paths  -->
      <search path="__some_directory__" domain="websocket" />
      <search path="__some_directory__" domain="sasl" />
   </system-paths>
</global-settings>

Currently, accepted system paths are:

Additionally many modules inside Turbulence and Vortex Library find configuration and data files by call to vortex_support_domain_find_data_file. That function works by finding the provided file under a particular search domain. Each module has it own search domain. To add new search elements into a particular domain to make your files available to each particular module then use the syntax found in the example above.

2.8 Splitting turbulence configuration

As we saw, turbulence has a main configuration file which is turbulence.conf. However, it is possible to split this file into several files or even directories with additional files.

In the case you want to move some configuration into a separate file, just place the following:

<include src="file-with-config.conf" />

Then the <include /> node will be replaced with the content found inside file-with-config.conf.

NOTE: even having this feature, the resuling turbulence.conf file after replacing all includes must be a properly formated turbulence.conf file.

It is also possible to load a set of configuration files from a directory. This is useful for profile paths where all of them are stored separated into /etc/turbulence/profile.d directory. That's why the following declaration inside <profile-path-configuration>:

<include dir="/etc/turbulence/profile.d" />

Previous declaration import all content from files found in /etc/turbulence/profile.d replacing the include node.

3.1 Profile path configuration

Profile Path is a feature that allows to configure which profiles can be used by remote peers, according to several run time configurations. It is designed to make it easy to develop BEEP profiles, that are later mixed with other profiles in many ways making them more useful through the combinations created at run-time.

Let's see some examples to initially clarify the profile path concept. A usual configuration around BEEP is to provide SASL authentication and then allow using a profile which do useful work. The intention is to enforce a successful SASL negotiation to then provide a protected resource.

With profile path this can be done as follows:

<path-def path-name="not local-parts" 
          src="not 192.168.0.*" 
          server-name=".*" >
   <if-success connmark="sasl:is:authenticated" profile="http://iana.org/beep/SASL/.*" >
      <allow profile="http://iana.org/beep/TUNNEL" />
   </if-success>
</path-def>

Previous example instruct Turbulence to apply a profile path called "not local-parts" if the source of the connection comes "not from 192.168.0.X". It also teaches Turbulence to only provide SASL profiles available and only once initiated properly (and authenticated), the remote peer can use the TUNNEL profile.

Profile path is applied as a chain, composed by a set of <path-def> declarations. Once a <path-def> match the target, the profile path configuration found inside it is applied to the peer, discarding the rest of <path-def> nodes defined.

profile-path-chain.png
Profile path chain

As the previous image shows, the turbulence profile path configuration is composed by several profile path definitions:

<path-def path-name="not local-parts" 
          src="not 192.168.0.*" 
          server-name=".*" >
   <!--  profile path configuration  -->
</path-def>

Once a connection is received, a path-def is selected and the configuration inside it is applied. If no path-def is selected, the connection is rejected.

Every <path-def> supports the following matching configurations:

  1. server-name: a perl expression defining the serverName to match for the connection. This can be used to only provide services based on a virtual hosting.

  2. src: a perl expression defining the source of the connection. This can be used to only provide critical services to local administrators.

  3. dst: a perl expression defining the destination of the connection. This can be used to provide services on a particular local IP.

  4. path-name: an administrative flag that will help to recognize the connection in future process (log reporting).

  5. work-dir: defines a working directory for the profile path. This value is used by several modules to load user site especific files (database configuration, etc).

  6. chroot: Defines a file system path used to make the current process to chroot to that directory. Note in most cases this should be used in conjunction with separate flag because once the current process chroots, will not be able to do it again for new connections.

  7. separate: [yes|no] Default no. Allows to configure Turbulence to create a child process to handle the connection that matches current profile path. A new child process will be created for each connection received.

  8. reuse: [yes|no] Default no. Requires separate="yes". Once a child process is created for the first connection associated to a profile path, next connections are sent to that child rather creating a new child process.

  9. run-as-user: [user name| user id]. Makes current process to change its executing user to the provided value. Requires Turbulence startup user to have permissions to run this system operation. Note this configuration may require to use separate (and/or reuse) flag.

  10. run-as-group: [group name| group id]. Makes current process to change its executing group to the provided value. Requires Turbulence startup user to have permissions to run this system operation. Note this configuration may require to use separate (and/or reuse) flag.

  11. child-limit: [child number] In the case this profile path has a declaration of separate="yes" this flag limits the number of child process that can be created due to this profile path. Rembember to set at least child-limit="1" when reuse="yes", though it is recommended to avoid using this flag when reuse="yes".

Once a path is matched, the following are discarded and the configuration inside the profile path is applied to the connection as long as the connection is running.

Now, it is required to configure which profiles will be allowed. This is done by using two types of nodes:

  1. <if-success>: a configuration that allows to use the profile referenced by it, and additionally, allows to use profiles provided by its child nodes once a channel with the profile provided is created.

  2. <allow>: a configuration that allows to use a profile at a particular level.

Let's see an example to show how it works:

<if-success connmark="sasl:is:authenticated" profile="http://iana.org/beep/SASL/.*" >
   <!--  enforce all BEEP peers to do a successful SASL negotiation  -->
   <allow preconnmark="sasl:is:authenticated" profile="http://turbulence.ws/profiles/test1" />
   <if-success profile="http://iana.org/beep/TLS" >
      <allow profile="http://iana.org/beep/xmlrpc" />
      <allow profile="http://fact.aspl.es/profiles/coyote_profile" />
   </if-success>
</if-success>

Previous example have configured a particular profile path as follows:

[PROFILE PATH EXAMPLE]

The example is mostly self-explanatory, but one detail remains. The caonfiguration uses two attributes: connmark and preconnmark. They are used as flags that must be detected in the connection in other to allow the connection to accept a profile or the content of the following profiles.

3.2 Profile path configuration: flags supported by <allow> and <if-success>

The following are the flags supported by <allow> and <if-success>:

  1. preconnmark: if used, it means connections must have the "mark" provided before the profile can be accepted/used.
    SUPPORTED: <allow>, <if-success>

  2. connmark: can only be used from <if-success> nodes. Allows to restrict profiles allowed inside <if-success> only if the provided mark is defined.

    This is particular useful for the SASL case because having a SASL channel created isn't a warranty of a connection authenticated. Thus, an additional mark is required to properly ensure that the connection was authenticated. These "marks" are profile/module especific.

  3. max-per-con: allows to configure the maximum amount of channels running the profile provided in the particular connection instance.
    SUPPORTED: <allow>, <if-success>

So, a good question at this point is "how are those marks created?". These marks are profile dependant and are created using the interface provided by the vortex connection module to store data. You must check the profile documentation to know which marks are supported as they are particular to each BEEP profile.

In this case, Vortex Library SASL implementation flags the connection as authenticated using the provided flag: "sasl:is:authenticated".

3.3 Profile path configuration: expression examples

For the case an expression is required to match source or destination the following expressions area available:

3.4 Turbulence execution model (process security)

It is posible to configure Turbulence, through profile path configuration, to handle connections in the same master process or using child processes. Here is a detailed list:

By default, when Turbulence is stopped, all created childs are killed. This is configured with <kill-childs-on-exit value="yes" /> inside <global-settings> node.

3.5 Making turbulence to start without profiles defined

By default Turbulence checks after module start up (init method) if there are at least one profile to serve. It is found that no profile is still defined, Turbulence will refuse to continue. This is done to prevent starting Turbulence with a wrong configuration.

To control or change this behaviour check <allow-start-without-profiles value="yes or no" /> inside <global-settings> node.

3.6 Declaring additional search paths inside profile paths

Some modules, like mod-sasl, mod-python, etc, may use a search path to find configuration files. This search path uses default values (profile path working directory or global system configuration directory) but it is possible to declare alternative paths for those files by using <search> node declarations inside profile path.

Here is an example:

<path-def separate="yes" 
          path-name="test-13.server services" 
          src="127.*" 
          server-name="test-13.another-server" >
   <!--  additional search path to be added when profile path is activated  -->
   <search path="test_12_module" domain="sasl" />
   <search path="test_13_module" domain="python" />
   <!--  request sasl auth first  -->
   <if-success connmark="sasl:is:authenticated" profile="http://iana.org/beep/SASL/.*" >
      <allow profile="urn:aspl.es:beep:profiles:reg-test:profile-11" />
      <allow profile="urn:aspl.es:beep:profiles:python-test-2" />
   </if-success>
</path-def>

The detail about what domain and what files are found with those declarations is module especific. Check the particular module documentation to find out how to use this.

4.1 Turbulence modules configuration

Modules loaded by turbulence are found at the directories configured in the <modules> section. Here is an example:

<modules>
   <!--  directory where to find modules to load  -->
   <directory src="/etc/turbulence/mods-enabled" />
</modules>

Every directory configured contains turbulence xml module pointers having the following content:

<mod-turbulence location="/usr/lib/turbulence/modules/mod-sasl.so" />

Each module have its own configuration file, which should use XML as default configuration format.

4.2 Turbulence module filtering

It is possible to configure Turbulence to skip some module so it is not loaded. This is done by adding a <no-load /> declaration with the set of modules to be skipped. This is done inside <modules> section:

<modules>
   <directory src="/etc/turbulence/mods-enabled" />
   <no-load>
      <!--  do not load a module called mod-skipped  -->
      <module name="mod-skipped" />
   </no-load>
</modules>

4.3 Enable a turbulence module

To enable a turbulence module, just make the module pointer file to be available in one of the directories listed inside <modules>. This is usually done as follows: