XdbE - History

XdbE - Project History

For some the very idea of XdbE as a Global Social Graph aggregator seems to be not immediately apparent due to its complexity and innovative nature. And indeed it was not a momentary sparkle of inspiration as the idea itself had to pass multiple formation stages accompanied by intensive discussions and brain-storming sessions before it emerged in its current shape and form. Therefore it makes perfect sense to give some background on the project explaining how we came up with the idea and what stages it had to pass before turning into a concept and before the actual programming begun.

The Formation or How it all Started

As early as winter 2007 the future founders of the Project in their discussions of the possible directions Web 2.0 is taking managed to localise a problem that essentially blocks the development and creates an obstacle for both end users and Online Community Hosts. The problem being later on commonly nicknamed "walled garden" phenomenon, is a single major stumbling block preventing the Web 2.0 from thriving full scale. Briefly, if Web 1.0 was about texts and their classification, Web 2.0 in turn is mainly about people and their online presence and activities. In Web 1.0 a passive surfer is able to read online documents all over the Internet and those documents are essentially free and accessible for all surfers regardless of who they are and where they come from. Web 2.0 in contract is characterised by active engagement of web users in content creation, communication and so on. The difference well described by LZZR in his article on Internet States of Presence is that Web 2.0 surfer is not only active but also when acting online can only act as a USER. So the reach of results of user-generated activities is in many aspects limited by the framework of that given community. Without established mechanisms of sharing data between communities they were becoming more and more isolated making their users struggle with maintaining their presence on many unrelated networks. Going this way Web 2.0 quite quickly became a patchwork of isolated Online Communities some of which like Facebook even decided to effectively exclude themselves from the rest of the Internet making their pages accessible only after authorisation. It became apparent that in the long run this is not the way forward for Web 2.0.

In an ideal world the solution might look like something that would allow user to roam freely from one online community to another both remaining recognisable as him or her self and being able to aggregated relate his or her own stuff and friends across communities. One possible solution could have been some form of a universally accepted online identity, but issues of privacy and the sheer scale of this undertaking make it unworkable in the near future.
A different approach would be to relate various online communities to each other Pretty much in the way various Online Profile URL aggregators were already doing. Instead of aggregating every bit of content, which is not a problem as such as there are plenty of Feed Mashing services out there, one might rather aggregate user profiles from different communities providing only links to externally hosted contend. There are only two things missing here: firstly even if we have all profiles of a given user in one place - what about his or her friends and secondly, if we are to deal with friends data - what about data updates? How do we get this data in real-time?

The solution dawned in a form of just one word - PING. Ping, already widely used in blogsphere was not something new and is commonly used by online communities as well as free-standing blogs to inform in real-time about a new blog post being published. Nothing new but what if to use the same technology to update not only about things being published but about users registering or unregistering and making friends at various online community services? Imagine, User A registers at Community One and this community immediately sends a ping with this information to a Community Two and vise versa. Add special pings for establishing and deleting friendship relationships between users and special pings to say that User A of Community One is the same as User C of Community Two. Wouldn't it be a near perfect solution to bridge two communities and integrate their users? At this phase we just need to set up a protocol which both of our servers understand and we are done - two communities are integrated at a user level.
Not so fast! But what if we decide to integrate not a single pair of communities but tree or four of them? Not only it complicates the whole system making each community remember all those partners but also it increases the load as each ping should be sent separately to each participating community. So, scaleability is an issue. Certainly it would be more convenient to have some centralised server like for instance Technorati for blogsphere where each participating community sends their ping only once. However, our comparison ends here as blogsphere pings end at ping services. Blogsphere ping services are the end pints - they represent information passed to them by pings but do not pass it as pings any further. What we need is some form of relay ping service that would have been able to accept pings and immediately pass them to other services. And now, if there is no way to solve it with protocol only without introducing a specialised server into the system - why not to make this server not only to relay those pings but also to collect and store ping data? And if we have the data in place, why not to allow others to access this data? So, that's how the skeleton of the idea was formed. Basically we have all elements present here. We have online communities able to send their data to each other through the proposed system that acts as an intermediary and stores data at the same time. The system is also capable of providing the data it stores upon request. Thoughts begun to crystalise and the project had acquired a working name Ping 2.0.

Social Graph or Project Halted

Towards the end of summer 2007 when the outline for the project was almost complete an unexpected event disrupted all our plans. It seemed that ideas were all in the air when Brad Fitzpatrick published his article titled Thoughts on the Social Graph. There is no need to repeat the idea of Brad's article here, suffice to mention that despite of obvious differences in terminology in quite many aspects it discussed things covered by our Ping 2.0 Project. At the time the article looked like a declaration of intent raising rather high expectations. It seemed that Brad was not only able to pose those problems but already had pre-manufactured solutions up his sleeve. What followed immediately after seemed to confirm this assumption. First Google came up with their Open Social initiative and shortly after the Data Portability project was formed. With two such powerful players already on the pitch it looked like all problems may be solved in a matter of months if not weeks. In this situation our small venture looked redundant and with heavy heart we decided to put our Ping 2.0 Project on hold.

Some months had passed after the initial stir and when the dust begun to settle it became apparent that although quite many people were thinking along similar lines and realised the same problems nobody was actually ready to come up with a solution. Brad Fitzpatrick himself rather than developing ideas he proposed in his article in a straightforward way became largely involved in Open Social initiative. Open Social in turn rather than concentrating on Social Graph (AKA User Circle) data itself became an application oriented service while Data Portability happened to be more like a sort of a social club. There was still some room to fit our project in. However all these events provided a substantial food for thought that lead us to further refinements of our own project.

The Second Breath - Project Refined

After the initial scare caused by a sir that followed Brad's article it turned out that our project has quite shiny perspectives as nobody else was willing or ready to step forward and offer working solutions to problems described in the article. Yet the article itself was a great source of inspiration as it highlighted the issue of Openness. Somehow before that it was not apparent that the service in question should be Open allowing not only for anyone to access the data externally, but also for anyone who would wish so to run their own data server containing their own copy of data. The article helped to realise also the strength of the project which came from the fact that initially it was thought to be a tool for rapid data sharing rather than storage thus providing real-time data delivery, a feature missing in Brad's vision. Data Portability Project in turn helped to see the institutional for which our project might take as a result. In addition Facebook Beacon cock-up raised once again the issue of privacy and ownership of personal data which helped us to refine the concept of AGL (Age, Gender, Location) the triad of data necessary and sufficient for the system to function and at the same time not infringing user privacy.

Ready, Steady, Go

In early 2008 the concept of the project became solid enough to start thinking about actual production stage. At this time we also decided to rebrand the whole venture and after long and painful deliberations the name XdbE was chosen for being both catchy and short. Our blog was set up and since this date you can follow the development from there...