XdbE - System Features

When we first attempted to formulate what our system should look like and how it should operate we realised how vast the area in question is and how many decisions we have to make to come to a number of working principles that will define the role and the function of the system and hence it's functionality and applicability. Starting from just a general belief that there should be no artificial barriers between various online communities we came a long way refining and improving every detail of it turning it into a solid concept guided by a number of principles. The principles listed below will help to explain why the system is designed in this particular way and not in any other. It will also help to demonstrate what this system is capable of when deployed at full scale.

XdbE System Diagram

Open Data Access

This is the most fundamental principle of XdbE. Let Public Data remain Public could be the motto of the project itself. Let me explain drawing on the example of Web 1.0. Internet was designed as an open environment where the majority of the content was deliberately placed in Public Domain unless otherwise requested. It was based on the principle of universal accessibility. This principle was easy to implement until the sheer volume of data available on the net made it unfeasible to sift through all of the data in order to find out the piece of information you require right now. That is how search engines came about. They appeared in order to answer the need to make sense out of this massive amount of textual information. The demand was there but it could not be met with built-in open mechanisms and specifications so it was met by proprietory systems of which Google the perhaps the most famous one. And here is the fundamental contradiction of the Web 1.0, the contradiction between publicly generated data and privately controlled access to this data. Now nobody "owns" the Internet but in effect Google "owns" structured access to the Internet.
The basics of the Web 2.0 are different. We are dealing not with texts and their interrelationships but with Users and relationships between them and this information is in Public Domain too. But we are about to repeat the same mistake once again. Unless there will be a a way not only to provide the results free at the point of request, but also to the data itself we are facing an inevitable emergence of a new Web 2.0 monopoly.
XdbE aims to provide not only access to public user data, but the data itself openly without any restrictions so that any developer or end user will be able to build applications, analyse or use this data in any other way they may see fit.

Decentralisation and Data Redundancy

To explain why Decentralisation and Data Redundancy are at the core of XdbE system let us consider what drawbacks we would have had if it was designed otherwise. Suppose we had an element in the system that has dedicated servers for sending data to clients and recieivng it from sources. Firstly it would inevitably create potentially dangerous situations caused by traffic and data overload. As the rest of the system depends on these dedicated servers their malfunctioning would effectively render the whole network dead. Secondly, such dependancy presents an organisational danger as well. Those who own and operate such key servers would in effect "own" the system as a whole and even if always acting in good faith may endanger the existence of the system by merely not being able to continue supporting these servers. Now suppose the system is designed in a way that it keeps different data segments at different servers and no single server has complete dataset. This would entail not only a rather complex algorythm calculating current location of the requested data and the address to send the new data to, but also would clog the system whith the amount of service traffic comparable in volume with the actual data traffic. Besides, it is rather risky to have only one copy of data, so some data multiplication has to be introduced even in this case anyway, which will further complicate not only the system architecture but internal routing as well.
XdbE architecture avoids these drawbacks by introducing horisontal hierarchy in which each Data Sserver is hierarchically equal to any other. New data sent to XdbE can be received and processed by any Data Server and equally any Data Server can answer requests from Data Clients. All XdbE servers have complete copies of the whole dataset at any given time. This solution might look a bit demanding in terms of disk space needed to hold this amount of data at each server but on the upside it ensures unprecedented stability and speed of XdbE network as a whole and enables each pareticipating server to "own" a complete dataset. In case of falure of any Data Server requests can be immediately redirected to any other XdbE server. This allows clients to choose the nearest and the fastest Data Server to communicate with thus ensuring load balancing throughout the whole XdbE network. It also allows adding and removing Data Servers whenever nessesary without reconfiguring the system architecture. Needless to point out that it provides additional freedom for application developers eliminating the need to draw data from remote servers. Having all nesessary data locally allows building speedy and robust applications and tools.

Real-Time Data Delivery

The most significant drawback of systems and standards proposed so far in this field is a significant delay between an event happening at originating server and registering this event within the system. Depending on system architecture proposed such a delay may range from hours to days and weeks which highly complicates building . XdbE eliminates this drawback by introducing achitecture capable of accepting event data from originating services and propagating this event throughout the while network within miliseconds. It is hard to expect from a system based on the "best effort" network transport which Internet undoubtedly is any mission-critical real-time data exchange speed but XdbE is the solution capable of delivering data in real-time in terms of the Internet. Thanks to the decentralised architecture described above network delays are minimised as each originating server is capable to adapting itself to network conditions by choosing the nearest and the fastest XdbE Data Server for each individual tranaction. This allows building user-centered applications based on data provided by XdbE that will always contain up-to-date information thus ensuring unprecedente degree of user satisfaction.

Data Security

Sending important data over the Internet is always a risky business. There are two potential risks in sending unprotected data: firstly the data can be "sniffed" and consequently end up in wrong hands and secondly data may become either unintentionally damaged or intentionally "poisoned" while travelling over the Internet which will result in corrupt data injected into the system. To counter malisious attempts to access or alter data passed within the system XdbE has built-in encrypting procedures allowing if needed to transfer not only public data but also private and sensitive information as well. To counter possible data loss or data inconsistensy XdbE includes data verification and data rollback procedures ensuring data consistency across the entire network.

User Privacy and AGL

Although XdbE provides all nesessary protection mechanisms to deal with sensitive personal information the core of the system is designed not to include any user information that may raise concerns about privacy issues. As the issue of privacy in relation to Social Networks and Online Communities remains open and the very idea of building a database containing personal data would scare off many potential participants XdbE deliberately limits the amount and quantity of information circulated within the system to an absolute minimum. The minimal nessesary set of user data required for a system of this sort to function is best described by the AGL (Age, Gender, Location) concept. This is the minimal set of data that combined with user name represents a set of data that on one hand does not infringe on user privacy and is not personally identifiable information but on the other hand sufficient for the system to have meaningful functionality and provide complete usability. Once again it is important to point out that XdbE is designed to represent Social Graph or User Circle and not to contain all possible user information. All information beyound the scope of AGL is published at corresponding Online Community Services or Social Networks and should remain there and be accessed from their interfaces. XdbE only connects those different communities and networks to each other.