Intelligent Distributed Systems
Kieran Greer
OCT 24, 2012 08:00 AM
A+ A A-

The lightweight Internet-based communication for autonomic services (Licas) system started as a research platform for testing dynamic linking possibilities, to optimize a network for query processes. It was innovative in that respect, but the architecture design is now quite standard. As such, it can also be used to describe these standard features that a distributed and intelligent system would typically require.

Note that the system is still mainly a framework that provides the basic components on which to add more intelligence. The linking mechanism has been written about in detail previously. Basically, it allows for permanent links between nodes that can define a permanent network structure. It then also allows for dynamic links that can change and would reflect more the current system use. It is also relatively easy to extend the base classes to add new services with specific functionality. While resources are limited, the system is relatively stable and could be used in real world scenarios. The system has been written to be J2ME compatible and so porting to a mobile platform looks possible. It does rely quite a lot on Java Reflection however, so some problems still exist there.

Architecture

The licas system is peer-to-peer, but uses a hybrid approach, with the server receiving the requests before automatically transferring them to the appropriate service that is running on it. This means that each service or node can both ask for and receive requests for work. It also means that each node is responsible for the consequences of its own actions, where only security measures might be provided by the supporting sever platform. Other architectures include client-server, where the client and the server have different and specific roles, or 3-tier and n-tier applications, where middleware applications perform additional processing on behalf of the client. This allows the client to be more lightweight (a browser, for example) and is what Web applications are typically built as.

Communication

A distributed system is composed of a number of individual components that interact with each other, sometimes over a network. The key to the distributed nature is the fact that processing is not centralized in one place (no central control), but is distributed to the different components in the system. Because of this, a global picture can only be obtained if these components can then communicate with each other. A communication language is therefore required, where licas uses an XML-RPC style of language. That is: code any communication into an XML-based script and send this to the requesting component. XML provides a standard format and also a tagging system, so that specific parts of the messages can be labeled and understood. This is also what Web Services use, but in a slightly different format. Agent-based systems, on the other hand, use a more sophisticated format that includes specifically defined interaction protocols, so that "conversations" can take place.

Distributed Autonomic Processing

As there is no central control, another requirement is that each individual component can perform its operations largely by itself and when it is required to do so. This is described by the idea of an autonomous system. Autonomous can mean possibly one of two things. The first meaning would be simply that a component can perform its own operations independently. It has its own internal control system and understanding of what it is required to do. The second meaning is based more on a reactive system with sets of self-features, based loosely on the human nervous system. These features operate automatically and in the background, without prompting and monitor the system, to ensure that it is able to deal with potential faults or threats.

There are great technical problems with building a system that would be 100 percent autonomous as this would also require the system to be very intelligent. This can be helped by allowing a system administrator to define rules for the system through admin scripts, or by simply limiting the level of intelligence that the system is allowed to use.

The licas system is both distributed and provides a framework for adding autonomic monitoring components. The communication language allows individual nodes to talk to each other, allowing them to perform evaluations based on local information, typically between two services only. The other classical setup is to bring all of the information together into a centralized place. In that case, a more global and complete picture can be obtained, but the processing requirements are much more heavyweight.

Licas allows for a centralized problem solver to be run. This can request or receive information from the services running on its network, process them, including AI tree searches for solutions and then send the results of this back to the individual nodes. So both the lightweight localized processing and the more heavyweight centralized processing can be accommodated for.

The individual servers can also link up, making scalability to larger systems possible. One intention is for the system to still be useful as a research tool and so the problem solving aspect is particularly important there. Figure 1 is a schematic of the general framework and architecture of a service provided by licas.

The base class is a Service class. For autonomic components, the extended Auto class is required. You then extend this yourself with your own service behavior. The Auto class includes an Autonomic Manager framework, with the four modules — Monitor, Analyze, Plan, and Execute. There is also a Message Interface. On executing a behavior, the Auto class will automatically send the result to the Autonomic Manager using the message interface. If autonomic modules have been implemented, they can monitor and evaluate the messages. If there is a fault, both the service itself and the base server can be informed. There is default metadata relating to the service and also a Contract Manager, to store and reason over proposed contracts for the service work. Nested or utility services can be added to any other service, but should be used more as an aid, rather than fully independent and autonomic services themselves.

Figure 1: Autonomic Manager

Figure 1. Schematic of the licas service architecture.

Search and Organization

For large-scale distributed systems, there is a need to organize them efficiently, to optimize for time and resource usage. The individual components also need to be found and ideally, be able to link or combine with each other, to allow for more complex tasks and interactions. This is known as service Deployment, Discovery and Composition. Using licas, individual services can be loaded at run time and coded to perform any sort of action. They can be configured through scripts and can communicate and link with any other service. There is also a query language and service-specific metadata, to allow for search and find operations.

Security

Key to these distributed systems becoming autonomous is when they can be sufficiently trusted. Human error is always possible, but the user must have confidence that the system will operate as it is supposed to. When processing information, individual components must be able to understand the information that they receive from other nodes. At the very least, they must not allow an operation to take place based on false or incorrect information (maybe allowing scope for what a human would do).

When considering the complexity of the human language and text-based information, this is a huge challenge. The idea of pushing a button and the system will run itself looks very unlikely and so we can only try to automate more aspects of the system instead. This is important, because the size and scale of these systems will increase and so it will be important that they can monitor and manage themselves, at least to some degree.

A recent research project has looked at the idea of concept trees. These are semantically-based linked lists of textual words or terms. The terms are related through their use in the system and can be used to define a certain bounds or range of values that the text might take. A basic linking method can be used to build the trees and also allows for a certain degree of normalization.

If new information is input that does not match the known tree structures, it can be considered to be suspicious or flagged, for security reasons. Text processing can be looked at as a natural language problem where the meaning of the text is the only concern. Alternatively, it can be looked at more as a mathematical problem, where the text and text sequences are simply a different form of numbers and equations. When looked at this way, it is possible to process text through general mathematical methods and derive a low level of understanding. This generic process is what is required for the concept trees, which is why they are worth investigating further. This is not useful for specific meaning, but for flagging potential faults and so could be important for security/autonomic processing. The normalization concentrates links towards certain terms or concepts, where information would tend to flow through the most commonly used ones.

Providing valid or expected boundaries for information, either numerical or textual, is one area where an autonomous system would be practically useful. If through human error, values were clearly incorrect, an autonomous system could flag this for further checking. A non-autonomous system would maybe allow the transaction to take place as instructed. An autonomous one, with only a small amount of intelligence, could block or query the transaction and allow it only after further confirmation. Creating the valid boundaries, however, is still a problem to be solved and would either be very general or application specific.

Discussion

The author is not in favor of putting everything in the Cloud, but also recognizes that it is the easiest way to make a piece of work accessible, with the minimum amount of effort. He has his own software products, for example, on other vendors' servers. A lightweight framework, such as that provided by licas however, would offer a nice solution to running your own server. Your mobile phone is usually on and connected to a network. You would not typically host static web pages on your phone, but process location-based or information-based data, in a much more dynamic way.

The system therefore also needs to be helpful, when autonomic processing can accommodate that. The licas system can therefore be used to build more localized networks between small numbers of people that do not flow through the major vendors' systems. This might even help with network congestion problems.

The open networks allow for the ease of access to information, but also for a more permissive use of it. More localized networks offer a certain degree of privacy in that respect. As the applications become more dependent on interaction over a network and as they become more intelligent, they are more vulnerable to cyber attacks.

Autonomous methods could therefore be used to some extent, or an app running on a private network could still be written with autonomous security features. The Wikipedia description of peer-to-peer notes the word "responsible" for a service's actions, highlighting a problem to autonomous behavior. Counter to that is the problem of human error and so any technology that is used needs to be safe and properly implemented. Licas does not provide a full solution, but it does demonstrate the potential pros and cons of this difficult problem.

Kieran Greer has over 10 years of experience working on research projects at the two Universities in Northern Ireland. He specializes in R&D, in the areas of Artificial Intelligence and Distributed Information Systems. Kieran runs his own software company <a data-cke-saved-href="http://distributedcomputingsystems.co.uk/" href="http://distributedcomputingsystems.co.uk/" target="_blank" "="">DistributedComputingSystems.co.uk</a>.</em><br></p>

FIRST
PREV
NEXT
LAST
Page(s):
[%= name %]
[%= createDate %]
[%= comment %]
Share this:
Please login to enter a comment:
 

Computing Now Blogs
Business Intelligence
by Keith Peterson
Cloud Computing
A Cloud Blog: by Irena Bojanova
The Clear Cloud: by STC Cloud Computing
Careers
Computing Careers: by Lori Cameron
Display Technologies
Enterprise Solutions
Enterprise Thinking: by Josh Greenbaum
Healthcare Technologies
The Doctor Is In: Dr. Keith W. Vrbicky
Heterogeneous Systems
Hot Topics
NealNotes: by Neal Leavitt
Industry Trends
Internet Of Things
Sensing IoT: by Irena Bojanova

 

RESET