OMF & OML Tutorials for w-iLab.2

Deprecated

OMF and OML are deprecated and unsupported. You should not use these for new experiments.

The tutorials below are derived from the OMF6 home page. This documentation is focussed on the installation and usage of OMF6 in w-iLab.2 .

For documentation on older versions, see the OMF home page.

As of 2017 OMF has become unsupported. Some tutorials are still available via the github.

Install OMF6 Resource controller

IF YOU ARE USING THE DEFAULT UBUNTU14.04 IMAGE ON w-iLab.2 THEN THE OMF RC IS PRE-INSTALLED !! ** JUST UPDATE IT AS SHOWN BELOW: **

gem install omf_rc -v 6.2.3 --no-ri --no-rdoc

The instructions below only apply if you are not using the default image on w-iLab.2

Check the original install instructions here.

This tutorial was tested on a UBUNTU14.04 system.

Follow these steps to install an OMF6 RC (as root):

sudo apt-get update
sudo apt-get install software-properties-common
sudo apt-add-repository ppa:brightbox/ruby-ng
sudo apt-get update
sudo apt-get install ruby2.3 ruby2.3-dev build-essential libssl-dev
#both of the following commands should result in versions 2.3
#if not, check here how to solve it: https://www.brightbox.com/docs/ruby/ubuntu/
ruby -v ; gem -v
gem install omf_rc -v 6.2.3 --no-ri --no-rdoc
install_omf_rc -i -c
nano /etc/omf_rc/config.yml
Change the amqp server to amqp://labwiki.test.atlantis.ugent.be
start omf_rc
Check if the RC is running: ps -eaf | grep omf

Install OMF6 Experiment controller

Check the original install instructions here.

This tutorial was tested on a UBUNTU14.04 system.

Because of high CPU usage, it is recommended to use one of the servers to run the EC. (server1-server8)

Follow these steps to install an OMF6 EC. The -v 6.2.1 option install a specific version to be compatible with the RC pre-installed on the testbed nodes. If you will also install the RC’s, then you can leave out this option and just install the latest version.

sudo apt-get update
sudo apt-get install software-properties-common
sudo apt-get install python-software-properties # only on UBUNTU 12 (SERVER NODES)
sudo apt-add-repository ppa:brightbox/ruby-ng
sudo apt-get update
sudo apt-get install ruby1.9.1 ruby1.9.1-dev build-essential libssl-dev
# run the update-alternatives commands first to make sure you are not using ruby2.X (this will result in very high CPU load):
sudo update-alternatives --config ruby #and then select option 0 (or the option that corresponds to a ruby2.X version)
sudo update-alternatives --config gem #and then select option 0 (or the option that corresponds to a gem2.X version)
ruby -v ; gem -v
gem install omf_ec -v 6.2.3 --no-ri --no-rdoc

Run your experiment by executing this command:

omf_ec -u amqp://labwiki.test.atlantis.ugent.be <PATH_TO_YOUR_OEDL_FILE>

Install AMQP server

We recommend to use the AMQP server at amqp://labwiki.test.atlantis.ugent.be

If you need to install one yourself, check out this page: http://mytestbed.net/doc/omf/file.set_up_communication_server.html.

Brief install instructions for RabbitMQ:

  • Uninstall any other AMQP servers first
  • Make sure port TCP 5672 is open in your firewall (if needed)
  • Install RabbitMQ on Ubuntu / Debian:
sudo apt-get install rabbitmq-server

If you encounter any issues, see http://mytestbed.net/doc/omf/file.set_up_communication_server.html.

Install OML Server

Check if you have the right sources to download OML

ls -la /etc/apt/sources.list.d/oml2.list

If this file doesn’t exist, create it and add the following content

deb http://download.opensuse.org/repositories/devel:/tools:/mytestbed:/stable/xUbuntu_14.04/ /

Then run the following commands to install the OML-server

wget http://download.opensuse.org/repositories/devel:/tools:/mytestbed:/stable/xUbuntu_14.04/Release.key
sudo apt-key add - < Release.key
sudo apt-get update
sudo apt-get install oml2-server sqlite3

Check that the OML server is running correctly with

ps aux|grep oml2-server
netstat -an|grep 3003

If not, start it with

/etc/init.d/oml2-server start

And check the log file

tail -f /var/log/oml2-server.log

By default, the OML server will log its databases in this directory

/var/lib/oml2

If you want to have this data available after your experiment has ended, copy it to your local machine or to your NFS mounted directory

/groups/wall2-ilabt-iminds-be/<PROJECT_NAME>

If you want to save your current image so you don’t have to reinstall it every time, please run the following command just before saving the image

sudo su
/usr/testbed/lib/prepare -M

Running an OMF-OML Experiment

If you are using the w-iLab.2 testbed, then you don’t have to install the OMF Resource Controllers as they are pre-installed for you.

However, you do need to install the OMF Experiment Controller and the OML Server on 1 node in your experiment (the type of the node doesn’t matter). Follow the steps in the topics above to install these components.

On your EC node, download the application definition (iperf.rb) and the experiment definition (iperfTutorial.rb):

cp /share/upload/omf6/iperf.rb .
cp /share/upload/iperf/iperfTutorial.rb

In case you cannot access the /share directory:

wget http://ec.wilab2.ilabt.iminds.be/upload/iperf/iperf.rb
wget http://ec.wilab2.ilabt.iminds.be/upload/iperf/iperfTutorial.rb

Modify the following lines in iperfTutorial.rb:

defProperty('theSender','client.testsetup.testbed.wilab2.ilabt.iminds.be',"ID of sender node")
defProperty('theReceiver','ap.testsetup.testbed.wilab2.ilabt.iminds.be',"ID of receiver node")

Fetch the DNS name of the sender and the receiver by running hostname in an SSH terminal on the client.

Run the experiment with

omf_ec -u amqp://labwiki.test.atlantis.ugent.be exec --oml_uri tcp:`hostname`:3003 iperfTutorial.rb

In this example the OML server stores its data in a local SQLite backend. Every run will create a database in /var/lib/oml2/ on the EC node. You can access your data as follows:

sqlite3 /var/lib/oml2/<DB NAME>.sq3
.tables
select * from _senders ;
select * from iperf_transfer ;
.schema iperf_transfer
.quit

Using the OML Proxy Server

In mobile and/or wireless experiments, we cannot assume that there is a permanent connection between the client that makes the measurements and the OML server. Furthermore, in some other experiments this measurement channel could also be the experiment channel and therefore the experimenter would like to postpone the collection of measurement until the channel is available again. In order to solve these two problems, OML provides a proxy-server that acts as a buffer that can be paused and resumed. See the OML website for more info on the proxy server.

The proxy-server can be in either of the following three states:

  • Paused: in this state the proxy stores both in a file and in memory the measurements form the different clients connected to it.
  • Resumed: in this state the proxy transfers the measurements to the server.
  • Stopped: in this state the proxy finishes transferring the measurements to the serve and closed all its incoming and outdoing connections.

When you start the proxy, it is in the paused state. In order to change between the states three commands are available:

  • OMLPROXY-PAUSE: change from Resumed to Paused
  • OMLPROXY-RESUME: change from Paused to Resumed
  • OMLPROXY-STOP: set to Stopped definitely

Install the oml2-proxy-server (not needed on w-iLab.2 nodes with the default image):

sudo sh -c "echo 'deb http://download.opensuse.org/repositories/devel:/tools:/mytestbed:/stable/xUbuntu_14.04/ /' >> /etc/apt/sources.list.d/oml2.list"
sudo apt-get update
sudo apt-get install oml2-proxy-server

Add the following application definition to your OEDL:

defApplication("omlproxy") do |app|
        app.binary_path = "/usr/bin/oml2-proxy-server"
        app.description = "Proxy server to buffer measurements in disconnected experiments"

        app.defProperty('listen', 'Service to listen for TCP based clients', '-l', {:dynamic => false, :type => :integer})
        app.defProperty('control', 'Service to listen for commands', '-c', {:dynamic => false, :type => :integer})
        app.defProperty('debug-level', 'Debug level error:1 .. debug:4', '-d', {:dynamic => false, :type => :integer})
        app.defProperty('resultfile', 'File name for storing received data', '-r', {:dynamic => false, :type => :string})
        app.defProperty('size', 'Page size for buffering measurements', '-s', {:dynamic => false, :type => :integer})
        app.defProperty('dstport', 'Downstream OML server port', '-p', {:dynamic => false, :type => :integer})
        app.defProperty('dstaddress', 'Downstream OML server address', '-a', {:dynamic => false, :type => :string})
end

The control port of the proxy server listens for commands to change its state. To be able to send these commands from your OEDL to the proxy server, a helper script named write_to_socket.rb is provided.

/share/upload/omf6/omlproxy/write_to_socket.rb

Also add the application definition for the helper script to your OEDL:

defApplication("writesocket") do |app|
        app.binary_path = "/share/upload/omf6/omlproxy/write_to_socket.rb"
        app.description = "Helper program to send STDIN input to socket"

        app.defProperty('host', 'Host to send the data to', '-h', {:type => :string, :dynamic => false})
        app.defProperty('port', 'Port to send the data to', '-p', {:dynamic => false, :type => :integer})
        app.defProperty('data', 'STDIN data to forward to socket', '-d', {:dynamic => true, :type => :string})
end

When defining your groups, add the above applications to all groups.

defProperty('proxystate','empty',"OMLPROXY-PAUSE (default),OMLPROXY-RESUME, OMLPROXY-STOP")

defGroup("AccessPoint","ap.testslice.wall2-ilabt-iminds-be.wilab2.ilabt.iminds.be") do |node|
        node.addApplication("omlproxy") do |app|
                app.setProperty('listen',3003)
                app.setProperty('control',3333)
                app.setProperty('dstaddress','am.wilab2.ilabt.iminds.be')
                app.setProperty('dstport',3004)
        end
        node.addApplication("writesocket") do |app|
                app.setProperty('host','localhost')
                app.setProperty('port',3333) #should be the same as the control port of the omlproxy
                app.setProperty('data',property.proxystate)
        end
end

At start of your experiment, the measurements will be logged to a file. If you want to change the state of the proxy server, just change the proxystate property:

onEvent(:ALL_UP_AND_INSTALLED) do |event|
        after 5 do
                allGroups.startApplications
                info "All my Applications are started now..."
        end

        after 10 do
                info "Resuming OML PROXY..."
                property.proxystate = 'OMLPROXY-RESUME'
        end

        after 20 do
                Experiment.leave_memberships
                info "Free up the resources"
        end

        after 25 do
                info "Experiment is finished"
                Experiment.done
        end
end

Start your experiment with the following options:

omf_ec -u amqp://labwiki.test.atlantis.ugent.be --oml_uri tcp:localhost:3003 <OEDL_FILE>

Note that the oml_uri is now specified as localhost:3003. This should reflect the configuration of the oml proxy server. Because the OMF Experiment Controller will also try to send its log messages to this OML server, we also have to start an OML proxy server on the EC:

/usr/bin/oml2-proxy-server -l 3003 -c 3333 -a am.wilab2.ilabt.iminds.be -p 3004

Don’t forget to type ‘OMLPROXY-RESUME’ to the STDIN of the above process, so that the logs are dumped instantly to the actual OML server.

Make a reservation (users of OLD emulab interface)
Hardware
Ghent University imec
© Copyright imec and Ghent University