Don't want to let this blog end (or just hibernate, time will tell) on that all's doom and gloom note of the previous post, so i'll add this: there still is a slim chance for the internet to go back to its roots, i.e. with everybody having a direct IP connection, and this slim chance comes from IPv6. However, the reason i call IPv6 just a slim chance is that although IPv6 can bring back "the internet of peers" to the world, the ISPs (and, to a lesser extent, the various router vendors) can still f* this up e.g. by deciding to install default firewalls inside their residential gateway devices, in which case your average Joe will still not be able to set up his router to accept incoming connections. And no incoming connections means no P2P. And no P2P means no internet.

As i said in the previous post, at this point in time i'm incapable of assessing whether the internet is dying, or it's headed towards a rebirth (via IPv6), or whatever else, so for the time being i'll just wait. The only thing that i can think of possibly doing at this point is to somehow start an awareness campaign for including "you need to have an unrestricted public IP" in the very definition of "internet connection"; any other type of connection should simply not be called "internet connection". Maybe i'll make a clip on youtube about this if/when i'll find the time and energy to do it...

Later edit
Apparently i'm not alone in my paranoia: www.isoc.org/tools/blogs/scenarios

There is some good news, some bad news, and a killer (or maybe just artificially-induced coma - this remains to be seen) conclusion.

The good news:
There seems to be a way of shouldering one-billion users on a P2P network with $1,000/mo central server traffic: it's called Kademlia (search, seminal article). Kademlia essentially creates an "overlay network" of micro-servers which, in turn, sustain the rest of the network users, while a central server is only used to log on to the network. But there's a catch: in order to distribute micro-server services over a P2P network there have to be at least ~1% of the participating peers that can act as micro-servers, i.e. they must:

1. be directly connected to the internet (i.e. they are not behind a NAT or firewall)
2. have relatively stable connections, i.e. they should stay connected tens of minutes after they completed a network operation (e.g. a chat session, a file transfer, etc)

Both of the above conditions are easily met in today's internet topology: (1) is achieved by any peer that has maximum one router and the router is UPnP-enabled, and (2) is achievable by having the P2P application remain running in the background for a certain amount of time after a specific p2p session ends. BTW, both of the above are used by the Skype client, while BitTorrent clients automatically deal with (2) because during a file transfer session a peer participates as a micro-server during the entire time of a session, and sessions are  long enough for a large part of the network to learn about, and utilize, the peer's micro-server services.

The bad news:
In brief, the bad news is that the good news don't do me much good. Because:

• a) i no longer trust that (1) will be maintained in the future. People increasingly install cascaded routers in their homes, and UPnP does not give any signs whatsoever that it is willing to address this issue (i.e. make a computer directly accessible on the internet when connected through cascaded routers). Furthermore, ISPs can deploy new methods of preventing p2p applications from running at any time (and i don't mean filters, but rather generic methods such as NATs and firewalls, and i really don't think this is very far fetched)
• b) the new mobile communications paradigm will become an increasingly significant burden that a depleting pool of directly-connected peers will have to deal with (because mobile data plans are always offered through operator NATs - but, even if they weren't, a mobile peer can't be used a a server because of mobile traffic costs). And BTW, i have this feeling that one of the main reasons for which skype was sold was the realization that the pool of "supernodes" (skype terminology for shamelessly using people's bandwidth and making $8 billions out of this scheme) is depleting and some big-pockets company will eventually need to step in and shoulder the network with dedicated servers of their own (sure, that's just a hunch, but it will be interesting to see if/when that "use UPnP" checkbox will go away from skype's connection settings, cause that'd pretty much say that skype fully transitioned to in-house supernodes)

My killer conclusion:
At this point in time i'm incapable of evaluating just how serious the above challenges (1) & (2) are, let alone other unforeseen issues that might creep in; and since implementing Kademlia would require anywhere from 6 moths to one year of hard work, i'm just not ready to plunge into such an effort alone and empty-handed.

So i stop. Very likely for good.

Mea culpa for not updating this blog in quite a while, but it's not because i slowed down work on it, but rather, much worse, i've hit a roadblock which is pretty darn serious. In brief, when trying to write the algorithm that would enable a p2pOS client to act as a relay for its plugins (i.e. to "connect the crossed red lines" that i talked about in a previous post), i also had to try to define the low-level API that the plugins will be using to connect to one another (by means of their associated p2pOS clients that will be acting as the relays, one relay at each end of a P2P connection). And as one thing led to another, i eventually reached the conclusion that what i need to do first is to establish exactly how a peer joins the network and connects to another peer. And this is where things got really, really ugly.

Without entering into too many technical details, the important point here is that in order to have two peers connect to each other, they have to go through an initial "handshaking phase" during which the two peers learn some essential things about one another (e.g. their IP addresses, what kind of router(s)/firewall(s) they are behind, etc), and this handshaking phase has to be negotiated through a dedicated handshaking server. Well,  i can try to hide behind all sorts of technical arguments, but that fact of the matter is that ever since i started this project i never tried to calculate exactly how much traffic such a central handshaking server would require for a large P2P network (i'm talking about 100,000,000...1,000,000,000 users being online), only to find out now that the numbers are astronomical: namely, we're talking about thousands, or even tens of thousands, of terrabytes/month, which translates into a handshaking server operating cost somewhere in the hundreds of thousand, maybe millions, of dollars a month. This in turn means that hosting such a server is not something that just about any punk can do in his basement, which in turn means a large company would be required to finance the network operation. Or, in layman's terms: the network can never be truly open, no matter what license will be covering this project, no matter what verbal commitments a company would make, etc. And since the (maybe only) non-negotiable objective of this project is to create a truly open P2P platform, well... you guessed it: i'm stuck.

But, as i said at the beginning of this post, all this mess doesn't mean i gave up on the project; in fact, because it seems increasingly likely that some sort of distributed handshaking algorithm will be necessary, i made quite a few tweaks in the program in order to reduce the traffic between connected peers (e.g. i managed to reduce the P2P keep-alive traffic by about an order of magnitude by detecting the peers' routers' port timeouts and only send keep-alive messages at the required rate), i refined the router classes such that over 90% of the routers models can now act as relays, and i introduced an algorithm that detects if a peer is directly connected to the internet (or via UPnP) such that it can serve as handshaking server in the network. This is how the new "Settings" panel looks like now:


So what i'm doing right now is study what other smart-@$$ P2P projects have done (e.g. Gnutella, Freenet, etc), i'm trying to learn about various DHT approaches (there's a very nice tutorial talking about the basics here), etc, and i'll see if i'll be able to come up with a solution. Keep ya fingers crossed for me, it's in the world's best interest and sh*t :)

PS
Here's how a p2pOS-based P2P session is established through NAT routers with the help of a handshaking server: the blue messages are P2P messages, while the green messages are relayed via the handshaking server (once the handshaking phase is completed, all messages from one peer go directly to the other, i.e. they are P2P messages):

The German wing of The Pirate Party just got a few comfy seats in the Berlin state parliament. While the news will sound exciting in and of itself for grass-root movements supporters and ideologists, maybe even more significant would be to mention here that such a development could only happen in (some sort of) a proportional representation democracy, i.e. forget about first taking Manhattan and then taking Berlin, that will never work, but the other way around...

Interesting times ahead or just another protest vote? Time will tell.

Took a lil' bit longer than i expected (those "little things", you know), but the main p2pOS application now reads the plugin list from a 'plugins' sub-directory and can start plugins.

Essentially, a p2pOS plugin is an application that runs on a client computer (i.e. alongside the main p2pOS application) and which implements a localhost sockets-based communication protocol which allows it to connect to the p2pOS main application (said communication protocol specification is part of the p2pOS API). Thus, because the only requirement for a p2pOS plugin is to implement a sockets-based communication protocol, a plugin can be written in any programming language which provides sockets access, including interpreted languages e.g. python, or tcl/tk, etc.

As the title of this post says, it's only the first stage of the plugin framework that's been implemented so far (specifically having p2pOS detect and run plugins), or, in other words, this is how things look like now:

The next step is to implement the crossed thicK red line(s) in the image below:

So basically, once the crossed-red-lines above will be implemented, a plugin will be able to communicate with another [remote] plugin by tunneling its messages through a p2pOS-managed p2p connection, which in terms of nuts and bolts means something like this:

Finally, because i love screenshots, here's the p2pOS main application starting a plugin...


...and then stopping it:

The P2P network manager now fully supports multiple connections. The image below shows three P2PCommunicator instances running, with user 'a' logging on in the bottom-right window, and users 'b' and 'c' logging on in the other two P2PCommunicator windows; in this setup, the following actions occur (in the listed order):

1. user 'a' connects to user 'b'
2. user 'a' connects to user 'c': user 'a' is now connected to both 'b' and 'c'
3. user 'a' sends the message "message from a to b" to user 'b' (i.e. this message is typed in user 'a's window and it appears in user 'b's window)
4. user 'a' sends the message "message from a to c" to user 'c' (i.e. this message is typed in user 'a's window and it appears in user 'c's window)
5. user 'b' sends the reply "message from b to a" to user 'a' (i.e. this reply message is typed in user 'b's window and it appears in user 'a's window) 
6. user 'c' sends the reply "message from c to a" to user 'a' (i.e. this reply message is typed in user 'c's window and it appears in user 'a's window)

So the image above shows one user ('a') connected with two other users ('b' and 'c'); now here's how things look like with three peers ('a', 'b', and 'c') all connected to each other:


Together with implementing multiple-connection support, the old disconnect algorithm (which was designed for a single-connection environment) had to be re-written, and it now fully supports multiple connections (btw, properly managing disconnects is no simple feat, took me about a week just to figure out how to do it).

1. implement the plugins architecture (i.e. allow third-party applications running on separate computers to connect to each other using the P2P network as managed by P2P OS); this is no simple thing as it requires defining a P2P direct-link frame structure which should be tunneled by P2P OS, and then implementing a 'DirectP2P' API that will support said frame structure (so this will most likely be a multi-stage task that will keep me busy for several weeks)
2. allow relay-based connections between two Non-BEHAVE peers (currently a Non-BEHAVE client can only connect to a BEHAVE-compliant client); i expect relay-based connections to be the last major client-side algorithm update, but you never know...

PS

This is an idea sitting dormant at the back of my mind ever since i started this P2P OS project (well, should i ever be able to make any real money out of it :p): http://www.washingtonpost.com/blogs/blogpost/post/peter-thiel-founder-of-paypal-invests-124-million-to-create-floating-micro-countries/2011/08/17/gIQA88AhLJ_blog.html

PS
Just to make things clear from my personal opinions' point of view: when i say "this is an idea sitting dormant at the back of my mind" i mean the independent [micro-]states idea, but i don't have any specific form of government/society in mind just yet. My reasoning behind the micro-states idea is to allow people to build and trade things inside a small-business-friendly legislation, but i can hardly see any appeal for someone in her right minds to actually be living on a boat somewhere in the middle of the ocean.

Just stumbled upon an article discussing skype's alleged security and how it was built into skype from day one (as opposed to being an afterthought), and felt like throwing my 2 cents on this.

For some bizarre reason, it is commonly thought that a skype p2p link offers anything and everything that can translate to "ultimate privacy" and some more, but this is... anything but. Here's the drill:

1. privacy has two major components, the first of which is protecting your communication from potential eavesdroppers. On this front, skype praises itself with unbeatable encryption, which is technically true, but what skype omits to say is that a good encryption only servers to protect a link that already exists; in other words, if you talk to someone and wish to know that your communication is shielded from eavesdropping, then skype is the real thing. However, what skype does not, and cannot, guarantee, is that you are indeed talking to the person you think you are: specifically, when you set up your communication link with your peer, the link setup stage is vulnerable to what is called a "man in the middle attack" which essentially means that someone has just placed a listening device between you and your peer, and everything that you'll be talking with your peer will in effect go through said man-in-the-middle eavesdropper. In the security world, this is called "the authentication problem" (i.e. making sure that what you're so carefully whispering to your peer indeed goes to your peer and not to someone else), and this is the hardest (and philosophically unsolvable) part of the secure communications problem.
• note: without entering into too many technical details, it is technically true that skype itself cannot eavesdrop on your communication by having one of its engineers flip some switches at their headquarters, but what skype does not explicitly say is that what it can do if it so wishes (or is legally forced) is to provide third parties with a technical equipment (i.e. a piece of skype-certified software) that can eavesdrop on any communication whatsoever if some conditions are met (e.g. if this eavesdropping software is placed on your ISP's routers then there is absolutely no problem whatsoever to wiretap your communications)
2. the second big issue related to communication privacy is whether or not a third-party can know who you're talking to, even if it won't be able to actually listen into your conversation. On this front, skype not only does not offer any protection, but its algorithms actually don't even make any effort whatsoever in this direction; in other words, there are absolutely no provisions in skype to even attempt to address the issue of protecting the identities of the two ends of a skype link.

To rise, none of the two fundamental attributes that could together label a communication link as "private" are present in skype, and while the "skype is encrypted" mantra is technically true, this mantra is blown out of proportions in terms of communication privacy.

P2P OS is different. I will not say much about the cornerstones (or implementation details) of P2P OS' security framework at this stage, but i will just say that it addresses both of the privacy issues described above, and it will [attempt to] provide the highest level of communication privacy that is philosophically achievable within the technical limitations of a given digital communications equipment.


Later edit
As i said in the post above, i don't want to get [too] technical here, but in light of some new developments i'll just mention that the cornerstone of skype's security model (and just about any security model out there for that matter) are the security certificates, and if a bunch of geeks can do things like this just imagine what a more potent attacker can do. And don't take my word on this, just look at what they say.
Needless to say (since i'm hereby flaming this model), P2P OS will not use this model.

The current client algorithm can log on to a server when it runs behind just about any type of home modem/router, with the notable exception of some 3G connections which purposefully use port-hopping algorithms in an attempt to make persistent P2P connections impossible (a partial solution to this problem exists, but implementing it is very low on my priority list).

In terms of connectivity, a client can currently connect to another client directly (i.e. P2P, without any relays) if at least one of the client's connections is BEHAVE-compliant; specifically, if both connections are BEHAVE-compliant, then the connection is quasi-instantaneous, while if one of the clients uses a Non-BEHAVE connection then the BEHAVE client will perform a port-scan on the Non-BEHAVE client and will find the P2P connection port.

The image below shows a simulation of a BEHAVE-to-Non-BEHAVE P2P connection.

• note: in order to simulate the BEHAVE-to-Non-BEHAVE connection, the automatically-detected connection settings on the clients had to be manually overridden, and this is the reason for the warnings in the clients' main windows.

Apparently if you don't tweet you don't exist, so here we go: http://twitter.com/p2pOS
And one day there might be some vids, here: http://youtube.com/p2postv

The mess inside...


And yeah, it actually works...

Google might have its issues and there are definitely quite a few haters out there, but it's still one of the very few large-scale corporations with a fair share of good deeds, the latest of which being the open-source release of the VP8 video codec as part of the WebM project. What this means for the garage-based developer (like me) is that anyone can now build top-notch multi-media applications "that just work", without needing to dive into that funky world of cosine transforms and the like. As far as the P2P OS project is concerned, this is heavenly-sent.

Making progress (you can get an idea of the size of the project by looking at the scroll bar handle's size and position), but it's painstakingly slow...

Just found out that WebKit is an integral part of Qt, so any geek can now build (or integrate in their own apps) a fully-functional web browser. Kewl.