Af-Gen tutorial ML4

Index

Introduction

This tutorial will show you:

The RPC live cicle model

You probably have heard talk about RPC systems which aims to provide developers the hability to execute remote procedures as local functions. This sound really good because you write applications that communicates with other ones in a client-server manner without having to face with complicated code such as socket, threads, mutex, and so on.

But, as you know, every software technology has disadvantages and RPC is not the exception.

When you program in a RPC enviroment such as SUN RPC, CORBA, Java RMI and Af-Arch you have to develop some extra software components that enables you to make remote procedure calling.

Those components typically are the STUB connector (o library) and the skeleton (or server).

The application code is actually your application that makes use of the services provided by the server component.

Let's see the next diagram:

rpc_classic_system_diagram.png

RPC classic diagram

If you were developing an application that calls to local functions you probably won't need the stub and server component. As you can see, the STUB component hide to app the complex details on how to call to functions inside server in a way that app thinks it is calling to local function.

So, you have jumped from the local-function-call model in which you only need to program 2 components, that is, the app and the functions that makes the work, to a remote-function-call model in which you have to program more components and more complex. Yes, this is true but always there is a shortcut.

Such programming enviroment comes with some special tools that helps you to create quickly the server component (known as skeleton because you have to fill up the functions inside it) and the stub.

At this point, you only need to define which functions or services are going to be exported by your component throught an idl definition and the idl compiler will do the rest for you. In other words, generate the STUB connector library and the skeleton.

Remember, you always need those components while programming on a RPC environment. This components can be created by youself manually or by an automatic source code tool.

How things are (without af-gen)

Af-Arch is one of those RPC environment with some special qualities. It has been written thinking about building enterprise applications and moreover, to be productive.

But as RPC environment, it wasn't enough complete because the lack of an idl compiler.

In this context, we have been programming all the extra software components we have seen in previous section: the stub and the server for all components we have made. This is a really a hard work because is a huge amount of source code and, as you know, the more source code the more bugs we have.

So af-gen solves this problems and takes some steps ahead.

Let's see the Af-Arch library layer to ilustrate what pieces of code af-gen will help us to generate. The following diagram shows how is layered the client side:

af-arch-layer-diagram.png

Af-Arch API stack

As you can see, a client application which uses the server application services has to program against LibAfDal and some libafdal-server-specific libraries. One for each server or component connected to.

The LibAfDal library contains lot of useful support code and architectural functions that enables client applications comunicate with other Af-Arch enabled components.

If you program your own Af-Arch component you will need your a libafdal-yourcomponent on top of LibAfDal. This library, libafdal-yourcomponent, enables client applications comunicate with your server in an abstract manner.

At the same time, on server-side, if you want to build your own Af-Arch component you will need to program the Server application layer.

Now you know what elements are necesary to build an Af-Arch enabled applications, the good new is that af-gen generate sucessfully those software components automatically and a bit more.

Why af-gen is different

Af-arch is not a general purpose RPC architecture like Sun RPC, CORBA and Java RMI. Af-Arch only has some of the features provided by previous RPC environments making it simple, easy to use and self-contained for enterprise applications developing.

Typically, an idl compiler doesn't generate the source code inside skeleton servers (that's why they are called skeleton).

This is because the idl compiler doesn't know what code is going to be executed on server side, so it leaves service function empty. Developers can fill it up later (and they will do so or the component will do nothing).

But, in many cases, af-gen knows what source code is going to be executed on server side and thus it doesn't leave service function empty.

Moreover, af-gen can go one step futher by generating the sql schema needed by your server component.

This differences are caused by the CFA concept and the Af-Arch enterprise application development purpose.

There are some services executed by Af-Arch components that are common across all of them. Those reserved services such as: new and remove some some object are so formal that af-gen can generate not only the skeleton but also the source code inside it.

This source code generated by af-gen inside your server services is called CFA. CFA concept, that is, to generate the service source code, is the main difference between the Af-Arch environment and the others RPC environment.

This means Af-Arch can be used only for certain problems: n-tier database oriented application development.

Let's see a real example

Since theory is boring we are going to put in action the previous concepts by developing a really simple Af-Arch component that manages products and providers.

So, our application have the following requierements:

Defining your application data

A good starting point to create our example application is to design how is modeled the application data. We can do this easily by using entity-relation model diagrams. The entity-relation diagram which represent our application data is:

data-definition.png

Model definition

As we can see a product can be sold by 0..N providers and a provider can sell 0..N products.

Letting af-gen do the work

Next step is to implement the server componet which export services to manage the previous data model and then implement the libafdal-server library who enables client applications to communicate with the server in an abstract manner.

Here is the idl code which represent our application data:

server interface af-test {
        relation sells;
        module product (string name,
                        string description,
                        string ref,
                        decimal(10,2) price) implicit definition {

                product sells provider with 0..n;
        }
        
        module provider (string name,
                         string address) implicit definition {
                provider sells product with 0..n;
        }
}

As we can see, we are defining the service interface for af-test server. The ``af'' prefix is needed due to the naming convention used.

Next we have a relationship declaration. Relationships have to be declared before using them.

Next lines declares a module called product. All source code inside a Af-Arch server component is organized by modules and inside modules are the services.

You can, of course, organize all your services inside one module, but this is not recomended.

While defining a module, you need to specify module types. In this case we have the following type definition for module product: (string name, string, description, string ref, decimal (10,2) price).

af-gen (ML4) has support for the following types. Check af-gen supported types to know more about.

Following the module type declaration is found the implicit definition modifier. Modifiers are keywords to instruct af-gen to change its default behaviour. In this case, we are telling af-gen to figure out what services we are going to need.

This is a really useful modifier because save you lot of work thinking about services name and its functionality.

Of course, if you don't use the implicit definition modifier, you'll have to define by yourself what services are needed and and implement them.

Inside braces are found module service definitions and relationship declarations. In this case, we have only a relationship declaration between modules product and provider throught the ``sells'' relationship.

Using af-gen to compile our example

Now, we need to compile our example, this is done like:

bash:~$ af-gen --base-dir ./ af-test.idl

With the previous instruction we are telling af-gen to check and compile our idl file: af-test.idl and generate all source code under base-dir argument.

When af-gen is done, you'll find two new directories: libafdal-af-test and af-test. This two directories are your server component: af-test and your stub library: libafdal-af-test.

You can now enter inside this two directories and compile its content by using:

bash:~$ cd af-test
bash:~$ ./autogen.sh 
bash:~$ make 
And for libafdal-af-test the same:
bash:~$ cd libafdal-af-test
bash:~$ ./autogen.sh 
bash:~$ make 

Let's take a look inside af-test content

Now, I want to show you what have happen while generating this source code, and more important, what is the mission of each element.

If you enter inside af-test automatic created directory you'll find the following:

bash:~$ cd af-test
bash:~$ ls 
data  src  configure.ac  autogen.sh  Makefile.am

There are two directories: data and src, and three files: configure.ac, autogen.sh and Makefile.am. This three files are really familiar for unix-like developer because are ``the-facto'' automatic source code configuration engine.

This configuration files are really important and a nice feature of af-gen because saves you lot of work on autoconfiscation those pieces of code.

Inside src directory is located the af-test server source code. If you look inside src directory you'll find source files and automake configuration files ready to be configured and compiled.

There are two files for each module definition inside the idl file (af-test.idl): the header(.h) and the implementation(.c), in this case:

$ ls src/
af_test_product.c   af_test.c           af_test_product.h
af_test_provider.c  af_test_provider.h  Makefile.am

You have to know that af-gen is prepared to support changes inside this files directly and support for merge differences between af-gen source code and your modified source code version.

So, af-gen differs from the others idl compilers because it lets you work not only with idl files but also with the generated source code.

Now, I want you to give a look inside the data directory:

$ ls data/
install.sql    uninstall.pl  af-test.cfg.example
uninstall.sql  install.pl    Makefile.am

The install.sql file has all necesary sql code to be installed. This sql code has all necesary table declarations, permissions stuff, relationships and so on, needed by the server: af-test.

af-gen not only have genereated the stub and the server (and its content) but also have defined the sql needed by the server.

The uninstall.sql file has all necesary sql code to uninstall the database. Both files have been proveen with Postgresql 7.2 or higher. Finally, the last interesting files is: af-test.cfg.example. This contains a configuration example for the af-test. You will have to modify it to conform your needs.

Let's take a look inside libafdal-af-test

As we have said in the previous section, the libafdal-yourcomponent is an essential piece of code that enables you to talk with your component or server in an abstract manner.

This is usually called: the STUB library. In this case, our stub library is libafdal-af-test.

Now, if you get inside the libafdal-af-test you will find the following:

$ ls libafdal-af-test/
autogen.sh  configure.ac         src  
test        afdal_af_test.pc.in  Makefile.am

You will observe autoconf files are also generated for the stub library so, more productivity. The rest of elements are two directories: src and test.

The src directory contains the library code. You will have to use the functions defined there to contact with the server af-test. If you look inside the src directory:

$ ls -1 libafdal-af-test/src/
afdal_af_test_product.c
afdal_af_test_provider.c
afdal_af_test_product.h
afdal_af_test_provider.h
afdal_af_test.h
Makefile.am

You can find two different files. The first type are header definition files (files ended by .h) and the second one are source code files (files ended by .c)

On most situation you will use the header files which contains all service interface function. Take a look inside this header files to figure out what contains.

This is another important key concept: all types defined inside the stub libraries are GObject which allows among other things, to inspect type attributes. This is used, for example, to obtain the columns names at run time to graphical user interfaces.

But now

At this point you have developed the simple component that manages products and providers. Now it is ready to be used in your application but, this is not useful enough.

Keep on reading the nexts sections.

Automatic test

af-gen has an special feature: to generate source code to test your components.

This automatic test code checks the STUB library and the server. If you look inside the test directory you'll find the test units, support code and a perl script that run all test.

$ ls -1 libafdal-af-test/test/
test_unit_afdal_af_test_product.c
test_unit_afdal_af_test_provider.c
test_unit_support.c
Makefile.am
run-all-test.pl
test_unit_support.h

This feature is really productive not only for you but also for us because allows developers to check, every time they need, the correctness of the server and the stub library.

This feature have helped us to improve much faster the behaivor and correctness of af-gen and Af-Arch. And now, you can use it for your components.

So, we are going to run the test generated.

Installing Af-test

We asume you have already installed Af-Arch. If not, check this seciton to know more about installing Af-Arch.

Get inside the af-test/data directory and execute:

$ ./install.pl
-- Database installer for af_test --

Where is the server running the database? [localhost]
Postgres user ? [acinom]
Running postgres port ? [5432]
Creating database for af_test ...
Password:
CREATE DATABASE
Installing sql schema for af_test...
Password:
BEGIN
CREATE USER
CREATE GROUP
ALTER GROUP
CREATE
CREATE
CREATE
CREATE
ALTER
ALTER
ALTER
ALTER
ALTER
REVOKE
GRANT
GRANT
COMMIT

This script will create the necesary sql stuff needed by af-test. You can take a look inside this script to know what's really happening.

Now you have to configure the server inside /etc/af-arch directory. The file config is: af-test.cfg. You will find an example in the same directory named as: af-test.cfg.example. You can rename it and use the following configuration:

# Config file
#
# Lines starting with # are comments.

listening port = 55200
listening hostname = localhost

kernel server = localhost
kernel port = 55000

database server = localhost
database port =
database name = af_test
database user = af_test
database password = changeme


log file = /tmp/af-test.log
log active = yes

afkey expiration = 3600

# The file must end with a new line

I've only modified the listening port parameter. Keep in mind that the parameter listening hostname is used to locate the component across the network. You will have to set another value if you want your component to be used from outside your box. Anyway, if you dubt, use any.

Now you need to register your new component. This is done by using:

$ af-test --update-services

This command will install inside the af-kernel all the services provided by af-test. So, af-kernel will be able to manage permissions for this new server.

The Af-Arch root user

Af-Arch root users are those who have access to all af-kernel services without restrictions, this means: manage users and group, manage installed servers and customize permissions. You can have several Af-Arch root users.

In case you want to create an Af-Arch root user you can use aspl-clm or create a super user by using af-kernel as follow:

$ sudo af-kernel --add-superuser aspl
Enter password for the new superuser:
Retype the password, please:
User aspl created successfully. Exiting..

Now you need to give permissions to the user you have created to access all af-test services by using aspl-clm as follows:

$ aspl-clm
ASPL Clm: Command Line Manager for ASPL Fact system
Copyright (C) 2002, 2003 Advanced Software Production...

Type 'help' or '?' for help

[not connected] aspl-clm > open localhost
Enter login name: aspl
Enter password:
Connecting to localhost:55000...Login ok

[localhost] aspl-clm > set perm user
Enter user id to set permissions (use tab):
  Number of users 6.
   ID  USER       DESCRIPTION
   1    aspl         Superuser
Enter user id to set permissions (use tab): 1
setting permissions:..Operation completed

[localhost] aspl-clm > quit
operation ok, exiting..

Now you are prepare to run the automatic test. You need you have a running af-kernel and a running af-test. Simply execute them: first af-kernel and the af-test.

Get inside the libafdal-af-test/test directory and type:

$ ./run-all-test.pl

 [ OK ] [afdal_af_test_product]:
        Testing: test new/remove functions

 [ OK ] [afdal_af_test_product]:
        Testing: test new/remove/list functions

 [ OK ] [afdal_af_test_product]:
        Testing: test new/remove/list/edit functions

 [ OK ] [afdal_af_test_product]:
        Testing: test add_element/
                      remove_element/
                      list_element functions

 [ OK ] [afdal_af_test_product]:
        Testing: test update_all function

 [ OK ] [afdal_af_test_product]:
        Testing: test set_element functions

 [ OK ] : all test passed for module product

 [ OK ] [afdal_af_test_provider]:
        Testing: test new/remove functions

 [ OK ] [afdal_af_test_provider]:
        Testing: test new/remove/list functions

 [ OK ] [afdal_af_test_provider]:
        Testing: test new/remove/list/edit functions

 [ OK ] [afdal_af_test_provider]:
        Testing: test add_element/
                      remove_element/
                      list_element functions

 [ OK ] [afdal_af_test_provider]:
        Testing: test update_all function

 [ OK ] [afdal_af_test_provider]:
        Testing: test set_element functions

 [ OK ] : all test passed for module provider

 [ OK ] : ALL TESTS PASSED!!!

This test are run for each module genareted and test the functionality for each services in a strong way. You can also activate the console debug by prefixing the command with:

$ AF_DEBUG=1 ./run-all-test.pl

Try this option to figure out how many debug information you can get.

But now, what's next?

Ok, this is a good question because you need now to starting to develop your application using all this new stuff and you need some insights.

First of all, you are going to work with a really tested environment, that this, the Af-Arch and the autogenerated components: the stub library and the server. So, in this context you only will have to worry about how to program your user interface.

You can check this section to connect Af-Arch components with graphical user interfaces.

You can also take a look inside the source code of the test files (inside the libafdal-af-test/test directory) to figure out how to take the initial steps: afdal initialitation, afdal petitions and how to handle afdal requests.

Conclusion

We have seen how to use af-gen tool to develop the components needed by all Af-Arch applications and how af-gen can save us lot of time developing and debuging not only for Af-Arch related stuff but also lot of work around the data stuff.

Think that with the af-gen concept, you will only need to worry about the user interface and let the data manipulation to be managed by Af-Arch and af-gen.

Some advantages of this model are: