 |
Aquila
2.0 prealpha
Cognitive Robotics Architecture
|
|
Aquila
2.0 prealpha
Cognitive Robotics Architecture
|
|
|
|
The modelling of the integration of various cognitive skills and modalities requires complex and computationally intensive algorithms running in parallel while controlling high-performance systems. The distribution of processing across many computers has certainly advanced our software ecosystem and opened up research to new possibilities. While this was an essential move, we would like to augment the possibilities in cognitive robotics research by providing an easy-to-use, hi-performance, modular and scalable software architecture. This paper presents Aquila 2.0, an open-source cross-platform software architecture that makes use of independent heterogeneous CPU-GPU modules that can run anywhere across the network in any number of instances and using any number of available GPU devices. These modules are loosely coupled with their graphical user interfaces dynamically generated by Aquila on demand. In the following sections we provide a full description of the software architecture together with the instructions on how to use it and develop new modules with integrated tools that automate the process thus saving a considerable amount of time.
Cognitive robotics is highly multidisciplinary filed concerned with endowing robots with high-level cognitive capabilities to enable the achievement of complex goals in complex environments. The cognitive capabilities include perception processing, attention allocation, planning, anticipation and reasoning about their own mental states and about other agents.
Creating cognitive robots is challenging task demanding collaboration between researchers from different backgrounds (e.g. computer scientists, roboticists, neuroscientists, philosophers, linguists, etc.) and industries that are able to produce and deliver specific hardware and related software. Such collaborations are flourishing in environments that encourage producing hardware and software that everyone can use, reuse or extend. One of the most outstanding projects that embarked on such mission is RobotCub, which involved consortium of several European universities extensively collaborating with industries worldwide. RobotCub project strictly followed an open-source philosophy, which led to the design of one of the most advanced humanoid robots in the world named iCub (Cognitive Universal Body) (www.icub.org). iCub is a small humanoid robot approximately 105cm high, weights around 20.3kg and its design was inspired by the embodied cognition hypothesis. In addition, Tikhanoff et al. have developed an open-source simulated model of the iCub, which enabled researchers and students worldwide to get involed and start contributing without access to a physical robot. The simulator has been also widely adopted as a functional tool within the cognitive robotics community, as it allows researchers to develop, test and evaluate their models and theories before using them on the actual robot.
Worldwide collaboration of researchers and industries is not only essential to develop robust hardware, sound scientific methods but also vital for the development of software that can facilitate bootstrapping of cognitive skills through constant interaction with the environment while dealing with influx of stimuli from multiple different sensory modalities. The iCub robot is a true achievement when it comes to the number of joints that can be controlled (53 deg. of freedom) and the variety of different sensors that can accessed (vision, sound, skin, touch sensors, force-torque sensors, position sensors, gyroscopes). Dealing with so much data while running software designed to make sense of it has been a challenge for years. Fitzpartick et al. have developed YARP (Yet Another Robotic Platform), which addressed this issue, standardised the software development and encouraged platform-independence, modularity, scalability and reuse via location-independent processes that can independently communicate via hardware-non-specific protocols. Good documentation and tutorials provided by YARP and iCub software developers led to many new modules being contributed to the iCub software repository, which on its part helped the development of higher-level modules inheriting functionalities of the existing modules.
A good YARP module is typically independent from graphical user interface (GUI), easy to compile, runs on any platform and can be communicated with via network ports. YARP manager can be used in case of the requirement to connect multiple different modules together and/or run them on different computers across network. It is also possible to connect the output of some of the modules to the input of YARP viewers that can then display the incoming images, which helps provides a user with good feedback. Some modules such as robotMotorGui or iCubGui provide nice graphical user interfaces while others do not need or have one. If they do have a GUI then they are typically tied to run on a machine with running x-server and cannot be run elsewhere, which is perfectly fine for certain type of modules such as the already mentioned robotMotorGui or iCubGui. However, in cognitive robotics, we often use complex biologically-inspired models where a good graphical user interfaces and visualisation tools are essential to be able to understand and easily manage system behaviour. The training of large-scale artificial neural networks, self-organising maps, running image-processing or any other task should be able to run on any computer and make the most of its resources without compromising or completely removing their GUIs.
We have been working with students and cognitive robotics researchers for several years now and our experience is that not all are skilled programmers. In fact, some are just looking to use certain features of the iCub (e.g. record vision, sensorimotor movements, control objects in the simulator) and others would like to develop their own modules with a nice graphical user interfaces and the ability ro run heterogeneous CPU-GPU modules in any number of instances anywhere across the network while centralising their management and GUI parts on the machine they are using for visualisation. The technical challenges involved in this, however, discourage many from embarking on such development.
We present Aquila Cognitive Robotics Architecture that addresses these issues and was developed by researchers for researchers, students and enthusiasts who do not necessarily want to spend days writing new modules. Aquila provides a clearly structured architecture that separates modules from their GUI counterparts easily manageable via simple, yet powerful graphical interface that comes with Aquila. Developing a new module together with a graphical interface takes little time thanks to one of Aquila's tools called Module Generator, which creates an empty module and GUI with user-specified names. Module Generator updates all the necessary project files and connects the module with its GUI counterpart. After a module is generated, a user can add the required functionalities to the already existing structure and then launch the module anywhere across the network, in any number of instances using any available processors, which includes both CPUs and GPUs.
The next section talks about the main motivation behind Aquila development, which is followed by the description of its architecture. The Developers section provides useful guidelines and tips for those who would like to develop their own modules. The Installation section explains three necessary steps to get Aquila up and running, which is followed by the last part with our conclusions.
The modelling of the integration of various cognitive skills and modalities requires complex and computationally intensive algorithms running in parallel while controlling high-performance systems. The processing requirements are increasing with every added feature and it is not uncommon that at the end of the software development stage a particular system is unable to cope with fast-response robot-control tasks. YARP provides means for distributing any number of processes across any number of machines using any of the available underlying communication protocols. YARP has become a standard for developing and interconnecting modules.
Distributing processing across many computers has certainly advanced our software ecosystem and opened up research to new possibilities. While this was an essential move, we would like to augment the possibilities in cognitive robotics research by allowing computationally demanding processes to utilise the immerse power of massively parallel GPU processors.
Around the year 2003, to overcome the energy consumption and heat-dissipation problems of standard PC processors, manufacturers started to produce computers with multiple cores. This had a strong impact on the software developer community. In the meanwhile, manufacturers have been looking into new technologies that would increase the number of transistors per wafer. However, reducing these dimensions comes at a price since the current leakage becomes a problem.
Since 2003, the production of semiconductors has been divided into multicore and manycore design trajectories. Manycore design aims to increase the processing power by increasing the number of cores in a processor. This number was doubling with each semiconductor process generation starting with dual-core chips and reaching hyper-threaded hexa-core systems. A manycore system is fundamentally different with regards to its design philosophy. While CPUs (Computer Processing Units) are optimised for the processing of sequential code and feature sophisticated control logic and large cache memories, the GPU (Graphic Processing Units) design philosophy emerged from the fast growing video industry where massive numbers of floating point operations are required to render every single frame. As a result, a GPU chip has most of its area dedicated to processing of the floating point operations and features only tiny cache memories.
In 2006, NVidia released GeForce 8800 GPU, which was capable of mapping separate programmable graphics processes to an array of GPUs, which paved the way to first general purpose computing using parallel GPU processors. GPGPU was an intermediate step where graphics card programmers had to use the OpenGL or DirectX API to implement their programs. Using the GPGPU technique many different applications have achieved dramatic speed improvements. For example, Kruger and Westermann developed a framework for solving linear algebra, Harris and colleagues designed a cloud dynamics simulation based on partial differential equations, Rodrigues and colleagues implemented molecular dynamics simulation and Nyland and colleagues N-body simulation.
More recent GPU developments, based on the NVidia Tesla GPU architecture, provide clusters of GPUs used as individual programmable processors and allow more efficient parallel processing tools. The CUDA (Compute Unified Device Architecture) programming tool has been designed on purpose to support mutual CPU/GPU application execution.
Parallel computing using CUDA and GPU cards is being increasingly taken up by industry and academies. Many commercial and research applications have migrated from using solely standard CPU processors to a heterogeneous CPU-GPU programs where each architecture does what is best at. Most of these applications achieve tremendous speed-ups in performance. Since quantum computing is still in its infancy and CPUs are approaching the processing limits constrained by the physical laws, it seems that heterogeneous CPU-GPU parallel computing is the next paradigm, which we would like to support and fully utilise in Aquila 2.0.
Aquila was written entirely in C++ object-oriented language, which was motivated by the fact that C++ is well supported and widely used by many developers worldwide. NVIDIA CUDA was chosen for the development of GPU-accelerated functions. CUDA is well-written and documented, widely adopted and supported by an install base of over 300 million CUDA-enabled GPUs in notebooks, workstations, compute clusters and supercomputers. Aquila is based on Qt, a leading development framework designed for creating applications and user interfaces for desktop, embedded and mobile platforms. YARP enables Aquila modules to communicate with their graphical user interfaces as well as other modules and devices. CMake is used to control software compilation process using simple platform and compiler independent configuration files. All of the above frameworks, libraries and tools are cross-platform and free to use.
Once of the most important aspects and contributions of Aquila lies in its fundamental architectural design, which makes a clear division between modules that do the actual work and their graphical user interfaces that provide easy-to-use controls over modules and the possibility to visualise their behaviour. The module GUIs are compiled as part of Aqula GUI, which is able to dynamically add module GUIs under new tabs. A user can then use and manage these modules, see their visualisations, duplicate or close them just as one would in a typical web browser. Each module and its GUI can be uniquely identified on the network and therefore be executed in any number of instances without creating port conflicts. In this case Aquila GUI simply generates a new module GUI with a unique index. The modules can be distributed in any number of instances on any computer on the network and therefore they need to be able to communicate with Aquila GUI regardless where they run. This is communication is facilitated via YARP network ports, which connect modules' ports with the ports assigned to their GUIs. The inherent modularity of this cross-platform architecture coupled with its ability to achieve hi-performance via GPU processors are directly enhancing its scalability, usability and reuse.
The following sub-sections describe the architecture in detail. Each sub-section focuses on different aspect of the architecture such as the structural organisation of project files, grouping of functionalities into libraries, the description of design and implementation of graphical user interfaces, modules and the communication links between them.
Aquila modules are configurable programs that can run in multiple instances across network, be used with or without GUIs and that have the option to execute their code on GPU processors. All Aquila modules have interfaces providing access to their functionalities and facilitating communication between them and other entities. Each module's implementation of this interface can be found in their Interface class (interface.h and interface.cpp) inheriting from QThread to enable communication via Qt signals/slots as well as listening on the input ports. In addition to the interface, each module would typically have another class ModuleName (moduleName.h and moduleName.cpp) inheriting from QThread to enable execution of module's function in a separate thread and remaining consistent with Qt signal/slot communication system. This class provides the actual functionalities via heterogeneous CPU-GPU code. In the case that these functionalities are already present in one of the Aquila's libraries and no additional functions are required, then Aquila module can leave out the ModuleName class altogether and directly call the library from the Interface.
Modules are configured from their config.ini file located in /conf sub-directory. These files would typically contain default values for various parameters (e.g. robot iCub/iCubSim), which can be overridden from terminal or later via network ports. Each module has at least one input and one output port through which it communicates with its graphical user interface and/or other modules. Regardless of specific functionalities, each module needs to be able to return a list of GPU devices found on the system where it is running and change its execution mode (between CPU, GPU and multi-GPU) if applicable.
Aquila is an application that provides management tools, communication and graphical user interfaces for Aquila modules. Aquila comes with a graphical user interface, which is inherently dynamic and was designed to be easy to use, intuitive and clean with all the complexities hidden unless required. Aquila provides a default GUI regardless of whether any Aquila modules are available or not. This default GUI allows users to start different tools, probe local and remote servers, see their parameters and detect any Aquila modules that could be launched on them. Available modules can be easily added using shortcuts (CTRL+T or CMD+T on OSX), from the main menu (/File/Add tab)}or the context menu by right-clicking on the tab bar and selecting "Add tab" options. Any of these action will result in opening of a dialog where users can select which module to start and where. Once users select a module from a list and a server where the module executes, Aquila will try to establish a connection with it via YARP ports. If Aquila successfully connects to the module, a new tab with a module GUI is added. From this point on, all of the actions triggered in the module GUI are directly linked with the module and vice versa.
As already mentioned, any number of modules can be launched in any number of instances and on any available servers. For example, the image below shows multiple modules running in multiple instances across three different servers. The unique identification numbers are shown in brackets next to the module tab name. The first number refers to module instance while the second to server identification. If module tabs do not contain any numbers, this means that the modules are running their first instances and on a local server. All the tabs can be moved, duplicated or removed via context menu similarly as one woud in a web browser.
(multiple modules image)
This section covers two main systems that Aquila uses for communication. The first one is based on YARP and it is used for communication between processes distributed across any number of machines. The second one is based on Qt and it is used for communication between objects.
Aquila needs to be able to communicate with its modules and vice versa. Aquila modules and their GUIs have dedicated interfaces providing communication between them via YARP ports. Every interface listens for incoming messages and sends out data when necessary, which is achieved via named entities, called "ports". YARP ensures that if one knows a specific port name, it is all that is necessary to communicate with it. Just as Internet DNS name service converts domain names to IP addresses, YARP name server (YNS) allows communication between ports providing that users know their names. A port can send data to any number of other ports. A port can also receive data from any number of other ports. The communication between ports can use different transports and protocols (TCP, UDP, multicast, shared memory or text mode) and can be freely added or removed. YARP communication is based on the observer software design pattern in which an object maintains a list of observers and notifies them automatically of any state changes usually by calling their methods. This design pattern is used by many different libraries, framworks and almost in all GUI toolkits (see the next section). Following the observer design YARP port objects are capable of delivering messages to any number of other port objects (observers) in any number of processes distributed across any number of machines using any of the available underlying communication protocols.
Aquila implements interfaces for modules and module GUIs where each interfaces has at least one input and one output port. Module GUIs send data to modules via output ports and receive data from modules via input ports. Similarly, module interfaces receive data from module GUIs via input ports and send out data via output ports. Aquila ensures that the interface ports have always unique names. Module GUI interface names its ports following this pattern /aquila/hostName/moduleName/instanceID which is then suffixed with :i for input ports or :o for output ports. On the other hand, module interfaces follow this pattern: /moduleName/instanceID suffixed with :i for input ports or :o for output ports. This unique naming ensures that modules can be running in any number of instances without causing port name conflicts.
In GUI programming, when one widget changes, other widgets often need to be notified. More generally, it is necessary that all types of objects are able to communicate with one another. Previously, this type of communication was achieved using callbacks, which are pointers to functions. Callbacks are normally passed to a processing function, which is then able to call the callback when required. Callbacks are strongly coupled with processing functions as they need to know which callbacks to call. In addition, callbacks are not type-safe as it is not possible to be certain that the processing functions will call the callbacks with the right parameters.
In order to address these limitations, Qt developed an alternative solution to callbacks based on signals and slots. A signal is emitted when a particular event occurs, while a slot is a function that is called in response to a particular signal. An object that emits a signal does not know if anything receives it just as a slot does not know if it has any signals connected to it. Signals and slots mechanism is not only loosely coupled but also completely type-safe in contrast to callbacks. This is because the signature of a signal must match the signature of a slot allowing compiler to detect possible type mismatches.
Signals and slot mechanism has been adopted by Aquila to communicate between different types of objects. Connecting all GUI widgets via signals and slots is the classical way of using this mechanism. In addition to this, Aquila and its modules use this system extensively to communicate with their Interface objects that are sending and receiving messages over YARP network (see next section). When an interface receives a new message, a specific signal is emitted (see xxx) and required slot called. Different signals are aimed for different objects. For example, messages related to module settings would typically emit a signal directly passing received values to a slot defined in the settings object's class.
Aquila currently works on Linux, OSX and Windows and can be downloaded as a source package or directly from our SVN repository, which will give you the most recent code.
Download Aquila source package
The latest version is available from here: http://sourceforge.net/projects/aquila/files/latest/download
Download Aquila from SVN (Linux, OSX or Cygwin)
Use the command below to download the latest 'work in progress' code to 'Aquila' directory:
Download Aquila via SVN client (Windows, Linux, OSX)
This option will get you the latest 'work in progress' code. Before you start, you will need an SVN client. If you are on Windows we highly recommend using TortoiseSVN. If you are on Linux or OSX there you might like SmartSVN. You will need the link below when checking out from repository:
This section describes a simple tree-step process to get Aquila up and running:
1. Dependency satisfaction
If you are on Windows you might like to use YARP installer instead of compiling everything from the source code.
Windows YARP Installer: http://wiki.icub.org/wiki/Downloads
If you are on Linux or OSX then your gcc should be 4.6 so that NVIDIA NVCC compiler works. If you are on OSX then you might need to have gcc 4.6 also because of the OpenMP problems reported in previous versions. OSX instructions: http://staticimport.blogspot.co.uk/2012/02/building-gcc-462-on-os-x-lion.html
2. Environment setup
The only requirement setting up environment is that AQUILA_ROOT environmental variable points to the root folder of Aquila.
On Windows this can be done by simply adding a new user or system variable under Environmental Variables tab in:
On Linux this can be accomplised by updating the {/env/environment} file with for example:
Alternatively on systems that use bash shell, it is also possible to update ~/.bashrc file with for example:
Adding AQUILA_ROOT to /etc/environment file makes the variable system-wide while adding it to the ~/.bashrc file makes it user-wide.
OSX system-wide variables can be set in /etc/profile by adding for example:
User-wide variables can be set in ~/.profile by adding for example:
3. Compilation and installation
On Linux and OSX Aquila can be configured, compiled and installed using the following command, which assumes working from /build sub-directory inside the AQUILA_ROOT directory:
On Windows, use CMake to generate project files for Visual Studio 2010 and then run the Visual Studio with administrator privileges, right-click on the INSTALL category in solution explorer and select build. This will build the whole project and place the binaries into the installation folder.
We prefer using Qt creator as programming IDE as it also integrates tools for designing graphical user interfaces, which are essential part of Aquila. Qt creator is also able to generate its own project files from the main CMakeLists.txt configuration file, which makes it even more convenient.
iCub software is essential for our work with the iCub humanoid robot or the simulator. Many of Aquila modules are completely independent from the iCub software, however, some modules may need to connect to specific ports created by other modules from the iCub software. Therefore, installing iCub sofware is not required unless you want to use or develop a module that communicates with iCub software.
Installation instructions: http://wiki.icub.org/wiki/ICub_Software_Installation
Aquila communicated is baed on YARP and therefore it needs a yarpserver running before it can be used. The yarpserver can be started with this command:
The yarpserver does not have to be running on the machine running Aquila. If that is the case, the machine running Aquila needs to be configured so that YARP knows where to look for the yarpserver. For example, if the yarpserver is running on port 10000 and machine with IP 10.0.0.1 then the following command would be used to properly configure it:
Aquila can distribute its modules anywhere across the network providing that Aquila was installed on the target machines and that each on has Aquila (yarprun) server running, which can be done using the following command:
The server identification number is given by ID, which can currently be anywhere between 1 and 10. This limit can be changed in source code by modifying MAX_SERVERS definition in mainWindow.h.
Once these requirements are met, a user can start Aquila using in default module simply by running its executable. The default mode is that Aquila probes remote servers but does not add any module. A user can then add whichever module is required, which can be done using shortcuts (CTRL+T or CMD+T on OSX), from the main menu (/File/Add tab) or the context menu by right-clicking on the tab bar and selecting "Add tab" options. Any of these action will result in opening of a dialog where users can select which module to start and where. Once users select a module from a list and a server where the module executes, Aquila will try to establish a connection with it. If Aquila successfully connects to the module, a new tab with a module GUI is added.
The default behaviour can be overridden by passing different arguments to Aquila:
For example, a user can start Aquila with the following command, which will run Aquila in debug mode and start multiple modules in multiple instances (e.g. mtrnn) on a local machine:
Running modules on remote servers can be done by for example running the following command, which will run three modules on /aquila/server/1 and four on /aquila/server/2:
These steps described how to get Aquila up and running on local as well as remote servers.
Aquila was built to be easy to use and easy to develop for. The following are some of the main benefits of developing in Aquila software architecture:
Developers are now able to create new Aquila modules together with their graphical user interface components within few seconds using Module Generator, which is a tool integrated in Aquila. Module Generator uses a set of templates for creating new modules, which are then automatically added to correct directories. This process is followed by updating CMake configuration files and Aquila source code. Next time the project is rebuilt, the new changes will be reflected and new modules installed. Once a new module is created and the project rebuilt, it is ready to run anywhere across the network in any number of instances. However, at this stage the module does not do much and a developer needs to update its module functions, GUI design and GUI functions. The last step is to make the module communicate with GUI and vice versa, which is done by modifying the already mentioned interface files. The Module Generator can be started using SHIFT+M shortcut or from the Tools menu options. Module Generator needs to know the desired module name and the developer's name, which is only used to generate credits section. When using the Module Generator simply follow the instructions, which provide the guidelines and feedback during the process.
Aquila project has a website, which provides detailed documentation, instructions and links to many relevant resources:
Our team can provide new developers with their own development branches in Aquila SVN software repository as well as technical support. In case of interest, please contact us via our SourceForge website:
New feature requests, suggestions, bugs, etc. can be submitted using the following link:
Below are links to different resources realted to Aquila:
|
|
|
We would like to thank to everyone who supported Aquila project. In particular we would like to thank to:
Aquila is licensed under FreeBSD.
1.8.3
